Multi-Representation Storage in ecpfs: What Do You Actually Gain from Keeping More Than One Version of Your Vectors?

ecpfs stores one representation of each vector per index. This project adds the ability to store more than one, full vectors, PCA-reduced vectors, or both, and lets you choose which representation gets used at which point in the search: for routing, for final ranking, or both. Once that capability exists, the real question is what combinations are actually worth it: when does keeping a second representation around pay for itself, and when does it just add cost without helping?

• Get ecpfs running: build an index, run a search, check the results against a brute-force baseline.
• Extend the index to store a second representation (PCA-reduced vectors) alongside the original, and control which one gets used during search.
• Try different combinations, PCA everywhere, full vectors only for routing, both stored at the leaf level, and compare recall, speed, and disk usage across them.
• Check whether wrong answers come from searching the wrong part of the index, or the right part with a bad ranking.
• Anything else you want to try is fair game, including representations beyond PCA if you’ve got an idea worth testing.