Version Embeddings for Safe Semantic Search Migrations
Semantic search systems often look simple from the outside: encode a document, store its vector, encode a query, and compare the vectors. The operational difficulty appears later, when the embedding model changes.
Two models can produce vectors with the same dimension and still define completely different coordinate spaces. Mixing vectors from model A with query vectors from model B can silently destroy ranking quality without producing an obvious error.
The safe approach is to treat an embedding model as a versioned data dependency, not a drop-in function.
What must be versioned
An embedding deployment has at least four coupled pieces:
- the model identifier and revision;
- preprocessing rules such as normalization or chunking;
- the stored vectors;
- the query encoder configuration.
A useful record therefore stores more than a vector:
document_id: docs-1842
embedding_model: text-embedding-model-v3
embedding_revision: 2026-08
chunking_policy: paragraphs-v2
vector: [...]The exact field names are less important than making compatibility explicit.
Why vector dimensions are not enough
A dimension check catches only one class of mistakes. Two 768-dimensional models can encode unrelated geometries. Even a new revision of the same model family can move neighborhoods enough to change nearest-neighbor results.
Treat the model name, revision, and preprocessing policy as a compound schema version. A query should only search an index built with a compatible schema.
Keep indexes immutable during migration
Avoid rewriting vectors in place while production queries are still reading the index. Build a new index beside the old one:
products-v7 -> model A, chunking v2
products-v8 -> model B, chunking v2Populate products-v8 from source documents, validate it, then move query traffic gradually. This blue-green style keeps rollback cheap and prevents old and new vectors from sharing the same search space.
Design a migration path
Re-embed from canonical source data
Do not use old vectors as the source for new vectors. Re-run the new encoder against the original text or another canonical representation.
Changing model, chunking, and cleaning rules in one migration makes evaluation harder. When possible, change one major variable at a time.
Measure ranking quality offline
Use a stable evaluation set containing representative queries and relevant document judgments. Compare metrics such as recall at K, mean reciprocal rank, or task-specific success rates.
Inspect important slices too. A new model may improve long natural-language queries while hurting identifiers, code snippets, or another language.
Shadow production queries
Before returning new results to users, issue the same production query to both index generations and record differences. Shadow traffic exposes real query distributions without changing behavior.
Compare latency, empty-result rate, overlap among top results, and downstream quality signals where available.
Shift traffic gradually
Route a small percentage of requests to the new index, then increase it while watching reliability and relevance metrics. Keep the old index available until the rollback window closes.
Handle writes during a long migration
Large corpora may take hours or days to re-embed. New or edited documents must not disappear from the replacement index.
Common approaches include:
- dual-write new content to both embedding pipelines;
- record changes in an append-only queue and replay them after the bulk backfill;
- snapshot source data, backfill from the snapshot, then process every change after that boundary.
Whichever approach you choose, define a clear cutover point so every document is accounted for in the new generation.
Put the version in cache keys
An embedding cache keyed only by normalized text can return a vector produced by the wrong model. Include the embedding schema in the key:
sha256(model_revision + chunking_policy + normalized_text)Apply the same rule to batch outputs, precomputed query embeddings, and feature stores.
Common pitfalls
Updating the query model first
If the query encoder changes while stored document vectors do not, rankings can fail immediately. Move compatible query and index versions together.
Deleting the old index too early
A successful backfill does not prove equivalent relevance. Preserve rollback until production metrics remain healthy for an agreed period.
Comparing raw similarity scores across versions
Similarity-score distributions are not necessarily comparable between models. Compare ranking outcomes and task metrics instead.
Ignoring preprocessing changes
Whitespace cleanup, truncation, chunk boundaries, and metadata concatenation all affect embeddings. Version those rules when they can change representation.
Model one deployable generation
At request time, select a complete embedding generation:
generation = {
model_revision,
preprocessing_revision,
index_name
}Do not assemble those fields independently from unrelated settings. One generation should describe one tested, rollback-safe unit.
Conclusion
Embedding migrations are data migrations. Version the representation, build replacement indexes side by side, evaluate with fixed examples, shadow real traffic, and cut over gradually. The extra structure prevents one of the most damaging semantic-search failure modes: a system that is technically healthy but ranks almost everything incorrectly.
