Representation Geometry

Measure Embedding Anisotropy Before Vector Retrieval

Cosine similarity is often treated as a local comparison between one query embedding and one candidate. That interpretation becomes less informative when most vectors occupy a narrow set of directions. Unrelated items can then share a substantial common component, compressing the range of angles that retrieval uses to separate candidates. This directional concentration is commonly described as embedding anisotropy. It is a property of a vector distribution, not a defect implied by any single similarity score. For developers, the practical issue is that a fixed cosine value has no universal meaning. Its usefulness depends partly on the geometry of the embedding population in which it was produced.