Why Your Vector Index Returns Five Copies of the Same Doc

The content describes a common failure mode in RAG systems where the vector index returns multiple copies of the same document chunk, filling the LLM's context window with redundancy. This prevents the LLM from accessing diverse information and providing nuanced answers; the solution involves hash-deduplication before ranking and MMR.