Two embedding vectors can have a high cosine similarity even when the items they represent are not close in the task-specific sense a retrieval system needs. One source of this mismatch is embedding anisotropy: vectors are distributed unevenly across representation space, often with substantial mass concentrated around shared directions.
Cosine similarity removes vector magnitude from the comparison, but it does not remove a common directional component. If many vectors point partly in the same direction, unrelated pairs can start from an elevated cosine baseline. The useful distinction between relevant and irrelevant items then has to appear within a narrower score range.
That geometry matters whenever software treats cosine scores as if they had a stable semantic scale.
Isotropy is a property of the vector distribution
An isotropic distribution has no strongly preferred direction. In an anisotropic embedding space, some directions account for much more of the vector distribution than others.
Consider unit-normalized embeddings written schematically as:
x = shared_component + item_specific_componentIf the shared component is large and points in roughly the same direction for many items, pairwise dot products inherit part of that common alignment. Since cosine similarity between unit vectors is their dot product, the shared component affects the score even when it carries little information for the retrieval task.
This is different from a simple norm problem. Normalizing every vector to length one controls magnitude, yet the vectors can still occupy a narrow cone. Unit normalization changes radius; it does not force directions to spread uniformly over the sphere.
The practical symptom is often score concentration. Many candidate pairs receive similarities in a relatively tight interval, while the ranking signal is encoded in smaller differences inside that interval.
A high cosine baseline changes threshold semantics
Suppose one embedding collection produces these illustrative scores for a query:
relevant item 0.91
borderline item 0.86
irrelevant item 0.82A second representation of the same items might spread them differently:
relevant item 0.72
borderline item 0.48
irrelevant item 0.14The numbers alone do not establish that the second representation is superior. They show that an absolute threshold such as 0.80 has no representation-independent meaning. A threshold is tied to the embedding model, preprocessing path, similarity function, corpus, and query distribution that produced its score distribution.
Anisotropy makes this coupling easier to miss because high similarities can look intuitively strong. A score near one may reflect both task-specific alignment and a broad common direction in the embedding space.
For retrieval systems, ranking and acceptance are separate decisions. A nearest-neighbor index can still place a useful item first when all candidates have high scores. A downstream rule that accepts every result above a fixed cosine threshold can behave poorly if that threshold was chosen under a different score distribution.
Mean centering removes one kind of common component
A direct geometric adjustment is to estimate the mean embedding over a reference collection and subtract it:
mu = mean(x_1, x_2, ..., x_n)
x_centered = x - muThe operation moves the reference mean to the origin. If a large part of the common direction is represented by that mean, centering can reduce its contribution to pairwise comparisons.
Centering is not a universal correction for anisotropy. A distribution can remain strongly directional after its mean is removed. Multiple dominant directions can remain, and the geometry can differ across domains or subsets of the corpus.
Cosine similarity also requires another decision after centering. If the system expects unit vectors, centered vectors need normalization before using a dot product as cosine similarity:
z = (x - mu) / norm(x - mu)The same transformation must be applied consistently to indexed documents and incoming queries. Applying a corpus-derived transform only to one side changes the coordinate relationship used by retrieval.
The reference set used to estimate mu matters as well. A mean computed from one corpus describes that corpus. Moving to a substantially different data distribution can change the common components, so the transform itself becomes part of the deployed retrieval state.
Removing dominant directions is a stronger intervention
Mean centering addresses translation of the vector cloud. Another approach projects vectors away from selected dominant directions, often estimated from principal components of a reference embedding set.
For a unit direction u, removing its component from vector x can be written as:
x_projected = x - (x dot u) * uSeveral directions can be removed in sequence or through a projection matrix. This can spread cosine scores when those directions mainly encode broad shared structure rather than distinctions useful to the target task.
The risk is equally concrete: a dominant direction can contain task-relevant information. Projection removes that information from every transformed vector. Variance alone does not identify a component as nuisance signal.
This makes component removal an evaluation question rather than a cosmetic normalization step. The relevant test is not whether the resulting vector cloud looks more uniform. It is whether retrieval behavior improves on representative queries and relevance judgments without damaging cases that depended on the removed structure.
Index geometry and application policy need separate evaluation
Approximate nearest-neighbor indexes operate on the geometry supplied to them. They do not know whether a common embedding direction is semantically useful, incidental, or harmful to a downstream acceptance rule.
A useful evaluation therefore separates at least two effects. Ranking metrics test whether relevant items appear ahead of irrelevant ones. Score-distribution analysis tests whether application thresholds, margins, or confidence rules remain meaningful.
For example, two embedding variants can produce similar top-k ordering while producing very different gaps between the first and later candidates. That difference can affect a system that rejects weak matches, routes ambiguous queries elsewhere, or combines vector similarity with another score.
Thresholds should consequently be calibrated on the final representation pipeline. Changing the embedding model, centering transform, projection, normalization rule, or corpus can alter the score distribution even when the vector database and similarity operator stay unchanged.
Anisotropy can vary across slices of the same corpus
A single global score histogram can hide local geometry. Code, prose, short labels, repeated templates, and multilingual text can occupy different regions of an embedding space. A common component estimated over the full collection can represent those mixtures rather than a single uniform bias.
This matters for both diagnosis and correction. If one content slice has much tighter cosine concentration than another, a global threshold can create different acceptance behavior across slices. A transform estimated from the entire corpus can also help one slice while distorting another.
Slice-level checks do not require a complicated model of the space. Pairwise cosine distributions, nearest-neighbor score gaps, and retrieval metrics grouped by relevant content type can reveal whether the geometry is stable enough for a shared policy.
The same principle applies to query embeddings. Document geometry alone is incomplete if queries come from a different distribution. Retrieval depends on the relation between query and document representations, so evaluation needs both sides of that relation.
Representation geometry is part of the retrieval contract
Cosine similarity is a precise geometric operation, but its numeric output does not carry a universal semantic calibration. Embedding anisotropy is one reason: shared directions can raise the background similarity between vectors and compress the range in which useful distinctions appear.
Normalization, centering, and projection each modify a different aspect of the geometry. None can be assumed to improve retrieval merely because it makes vectors appear more evenly distributed. Any transformation must preserve the information the application needs and must be applied consistently at indexing and query time.
For developers, the durable boundary is simple: treat embedding geometry, similarity scoring, and application thresholds as one versioned pipeline. A change to any part can alter the meaning of the scores consumed by the rest of the system.