Cosine similarity assumes that vector direction carries useful discrimination. That assumption becomes less informative when many embeddings occupy a narrow region of the space. In that case, unrelated items can share a substantial directional component, cosine scores can cluster into a compressed range, and small residual differences can decide the ranking.
This geometric pattern is often described as embedding anisotropy. It exists before a vector index chooses candidates, so index tuning alone cannot establish whether the representation has enough angular separation for the retrieval task.
Isotropy is about directional spread
An isotropic distribution has no privileged direction in the relevant representation space. Real embedding distributions rarely satisfy that ideal exactly. The practical question is how strongly the vectors concentrate around a small set of directions and whether that concentration interferes with the similarity function used for retrieval.
A simple diagnostic starts with the corpus mean. For embeddings x_1 ... x_n, compute:
mu = (1 / n) * sum(x_i)If mu has substantial magnitude relative to typical vector norms, many vectors share a common component. That observation is not sufficient to declare the representation defective. A shared direction can encode useful structure, and its effect depends on normalization, the query distribution, and the retrieval objective.
The covariance spectrum provides a second view. After subtracting the mean, a few large eigenvalues indicate that variance is concentrated in a limited set of directions. A flatter spectrum indicates broader directional spread. Neither shape supplies a universal pass threshold; the useful comparison is between geometric measurements and retrieval behavior on the same data.
Cosine normalization does not remove a shared direction
L2 normalization removes vector magnitude from cosine ranking, but it does not subtract a common component. For nonzero vectors a and b, cosine similarity is:
cos(a, b) = (a dot b) / (||a|| * ||b||)After normalization, this becomes a dot product between unit vectors. If both unit vectors still point partly along the same dominant direction, that contribution remains in the score.
This distinction matters when diagnosing a narrow score distribution. Normalization can make cosine computation convenient and can align cosine ranking with Euclidean ranking on unit vectors, but it does not make the embedding distribution isotropic.
A useful measurement is therefore the distribution of cosine similarities for pairs that are not expected to match. If those scores occupy a high, narrow band, inspect whether a shared direction contributes materially before treating the score values as calibrated evidence of semantic closeness.
Centering changes every vector relative to the corpus
Mean centering replaces each vector with:
x_centered = x - muThis removes the empirical mean of the fitted corpus. It can increase angular separation when the mean direction is a nuisance component, but it is not a neutral cleanup operation. The transform changes query-candidate geometry and therefore changes rankings.
The fitted mean also becomes part of the representation contract. A query must be transformed with the same mean used for the indexed corpus. Recomputing mu for each query batch or silently replacing it after the corpus changes creates a mismatch between stored and query vectors.
If centered vectors are L2-normalized afterward, the order of operations matters: subtracting the mean and then normalizing is not equivalent to normalizing first and then subtracting a mean computed in another space. The index and query pipeline need one explicit sequence.
Removing dominant components is a stronger intervention
A more aggressive transform can project vectors away from one or more dominant directions. If u is a unit direction, removing its component from x gives:
x_projected = x - (x dot u) * uApplying this to principal directions can reduce variance associated with those axes. It can also remove task-relevant information. A dominant component is a statistical property, not proof of noise.
That makes the number of removed directions a model-selection choice rather than a formatting detail. The transform should be fitted only on the intended reference data, stored with the embedding version, and evaluated against relevance judgments. A spectrum that looks more uniform after projection does not establish that retrieval became more useful.
Whitening goes further by rescaling centered principal components according to their variance. This changes both direction and relative scale. It can produce a more even covariance structure on the fitted data, but it also amplifies low-variance directions, including noise when such directions carry little stable signal. The resulting vectors define a new retrieval space and should be treated as such.
Geometry diagnostics and retrieval metrics answer different questions
Anisotropy measurements describe the shape of an embedding distribution. Retrieval metrics describe whether ranked candidates satisfy an application criterion. Improving one does not guarantee improvement in the other.
A defensible evaluation keeps those roles separate. Measure the original representation, apply a candidate transform, rebuild or re-encode the affected index state, transform queries through the identical pipeline, and compare retrieval quality on fixed relevance judgments. Also inspect score distributions because a transform can change any thresholds that depend on raw similarity values.
Approximate nearest-neighbor recall is another separate variable. If exact search and approximate search disagree, the index contributes error beyond the representation geometry. If both produce similar rankings with weak semantic separation, changing search depth or probe settings cannot create information that the vectors do not express.
The fitted transform belongs to the embedding version
Any data-derived centering, projection, or whitening parameters are part of the vector representation. Mixing vectors produced under different fitted transforms can make similarity scores hard to interpret even when all vectors have the same dimensionality.
That boundary is especially relevant during corpus refreshes. A newly fitted mean or projection basis can move old and new items differently. Re-encoding the corpus under one fixed transform avoids comparing coordinates from incompatible spaces.
Embedding anisotropy is therefore most useful as a diagnostic, not as a verdict. Directional concentration can expose a representation constraint that raw cosine scores conceal, but geometric correction only earns a place in the retrieval pipeline when the transformed space improves the task-specific ranking criteria that matter.