Embedding Model Migration: Keep Vector Indexes Compatible

An embedding model migration changes the meaning of your stored vectors, even when the new model returns the same number of values. Treat it as a search-data migration: build a separate index, regenerate document embeddings, route matching query embeddings to it, and retain the old index until the new path is proven.

A dimension mismatch usually fails loudly. Mixing two incompatible vector spaces can fail quietly, returning plausible but irrelevant documents. The goal is to prevent both outcomes without turning a model upgrade into a long search outage.

Write down the embedding contract

Create a small manifest describing the model revision, tokenizer, input preparation, output dimensions, normalization and similarity metric. Include the query and document instructions if the model uses different input formats for each. A marketing model name alone is not enough to reproduce yesterday's vectors.

{"index":"help-v2","model_revision":"pinned-revision",
 "dimensions":768,"metric":"cosine","normalized":true,
 "document_recipe":"support-doc-v3","query_recipe":"search-v2"}

The numbers are an illustrative configuration, not a universal model requirement. Read the actual model documentation. Google's embedding inference guide demonstrates why output shape and input preparation deserve explicit attention. Inspect your own manifest with the JSON Viewer, then compare proposed changes using JSON Diff.

Why equal dimensions do not guarantee compatibility

Imagine two maps with the same width and height. One describes Tokyo; the other describes Berlin. Coordinates from one do not become useful on the other simply because the dimensions match. Embedding coordinates similarly depend on the model that produced them.

Do not send new query vectors against old document vectors unless that compatibility is expressly supported and tested. Also avoid filling a single index with old and new document vectors during a rolling migration. Records need an embedding version, and requests need an unambiguous matching version.

Shortening vectors is another contract change. Only use a model's documented dimension-reduction method; arbitrary slicing can discard information in ways the model was never trained to tolerate. Recheck ranking and normalization after any supported reduction.

Build the replacement beside the original

Start from a stable document snapshot. Give every source document a durable identifier and retain its content version. Generate new vectors into a separate collection or index namespace. Track successful records, rejected records and retryable failures independently.

For a million documents, a successful job completion message is weaker evidence than reconciled counts. Compare the source inventory with the set of indexed document IDs. Include deletions and changed documents that arrived during backfill, or arrange a brief final synchronization before switching traffic.

Keep checkpoints at the document level. If the process restarts, a completed embedding should not be generated again solely because a worker lost its memory. An input digest can identify unchanged content, but combine it with the embedding contract version; identical text still needs a new embedding when the model changes.

Compare search behavior without changing users' results

Run representative queries through both paths. Initially, return the old result to the user and record a redacted comparison from the new path. This is a shadow query, not a reason to duplicate confidential text into unrestricted logs.

Look for missing known documents, changed language coverage, filters that stop working and unexpectedly slow queries. Use task judgments rather than demanding identical ranking. A replacement may legitimately order good documents differently. Conversely, an impressive average can conceal a serious regression for one customer group.

Review the query filters too. A migration must preserve tenant boundaries, visibility rules and deletion semantics. Better semantic ranking is irrelevant if a private document appears in another account's results. Make unauthorized-document checks part of the switch criteria.

Cut over with a reversible routing decision

Route a small share of traffic to the new query encoder and new index together. Keep the pair behind one configuration setting so a deployment cannot accidentally change only half. Expand traffic when error rates, ranking checks and latency meet your agreed targets.

Record which contract served each request. That allows an incident responder to distinguish an application bug from a model/index mismatch without inspecting the entire vector payload. If rollback is necessary, switch both query encoding and index routing back to the original pair.

Do not delete the old index immediately. Define a retention period based on the team's ability to diagnose regressions and the cost of storage. Keep document updates flowing to both indexes during that period, or explicitly accept the stale-data limitations of rolling back.

Common migration mistakes

• Updating a model alias without pinning the underlying revision.
• Reusing an index because its dimension setting still accepts inserts.
• Backfilling active documents while forgetting archived or deleted ones.
• Comparing only English queries for a multilingual product.
• Switching the query encoder separately from the document index.

Treat the migration report as a release artifact. It should state the source snapshot, counts, contract versions, failures and the exact routing change. A teammate should be able to understand it without having attended the deployment.

Finish with a compatible pair

A safe embedding upgrade preserves the relationship between document vectors, query vectors and index settings. Version that relationship, build separately, reconcile coverage and switch reversibly. For ranking checks during the transition, continue with Cosine Similarity vs Dot Product.