Normalize Embeddings Before Dot-Product Similarity
Embedding systems often compare vectors with cosine similarity or a dot product. The formulas look similar enough that it is easy to treat the two metrics as interchangeable. They are not interchangeable for arbitrary vectors. A dot product depends on both the angle between two vectors and their magnitudes. Cosine similarity removes magnitude and compares direction only. That difference can change nearest-neighbor rankings, retrieval results, and similarity thresholds. This article builds a practical mental model for deciding whether to normalize embeddings. You will see why L2 normalization makes dot product equivalent to cosine similarity, how inconsistent normalization breaks comparisons, and when preserving vector magnitude may be intentional.