Use Late Interaction for Fine-Grained Neural Retrieval
Use Late Interaction for Fine-Grained Neural Retrieval A single text embedding is convenient: encode a query into one vector, encode each document into one vector, then rank documents by vector similarity. That design scales well, but compression happens early. A paragraph containing several distinct ideas must squeeze all of them into one fixed-size representation before the query arrives. Late interaction keeps more of that detail. Instead of representing each text with only one vector, it retains multiple contextual token vectors and compares them at retrieval time. The document can still be encoded ahead of time, but the final relevance score is computed from fine-grained query-to-document matches.