Start typing to search this publication.
techcirkle
Open menu

Subscribe to techcirkle

Get new posts delivered straight to your inbox.

Your Embedding Model Will Change. Design For the Reindex Now.

Changing an embedding model invalidates every vector you have stored. Versioned collections and a switchable query layer turn an outage into a routine task.

techcirkle
4 min read
Cover image for Your Embedding Model Will Change. Design For the Reindex Now.
Database and infrastructure systems
Database and infrastructure systems

Here is a prediction that costs nothing to make and is essentially always correct: the embedding model you choose today is not the one you will be running in two years.

A better model ships. Pricing changes. A provider deprecates an endpoint with a migration window measured in weeks. Retrieval quality plateaus and someone finally tests an alternative properly.

And when it happens, every vector you have stored becomes worthless — not degraded, worthless. Embeddings from different models occupy different vector spaces. Distances between them are meaningless. There is no conversion, no adapter, no partial migration. The entire index has to be rebuilt.

Most teams discover this constraint at the point of needing to act on it. It is straightforward to design for in advance and genuinely painful to retrofit.

The failure sequence

The version that plays out repeatedly:

A team ships retrieval. One table, one embedding column, one index. It works.

Eighteen months later the corpus is several million chunks and a materially better model is available. Someone estimates the migration.

The problem is that the schema has one embedding column. Rebuilding in place means either taking retrieval offline for the duration of the reindex, or accepting a period where some rows hold old-model vectors and others hold new-model vectors — which does not produce degraded results, it produces incoherent ones, because distances across the two spaces are not comparable.

So the migration becomes a maintenance window. Someone estimates reindexing time, adds contingency, and books a weekend.

What to build instead

Four decisions, none of them expensive at the start.

1. Version the collection, never mutate one index

Make the model part of the collection identity.

documents_v1_ada002
documents_v2_embed3large

Or, within a single table, a discriminator column with the embedding dimension and model recorded per row, partitioned by version. Either way, two model generations must be able to coexist.

2. Store the model identifier with every vector

CREATE TABLE doc_chunks (
  id            bigserial PRIMARY KEY,
  document_id   bigint NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
  chunk_index   int NOT NULL,
  content       text NOT NULL,
  embedding     vector(3072),
  model_id      text NOT NULL,
  model_version text NOT NULL,
  embedded_at   timestamptz NOT NULL DEFAULT now()
);

This is cheap and it answers questions you will otherwise be unable to answer: which chunks are stale, how far through a backfill you are, and — during an incident — whether a quality complaint correlates with a particular model generation.

3. Make the query layer switchable

The retrieval path should resolve which collection to query from configuration, not from a hardcoded name. A flag, an environment value, a feature toggle per tenant if you want to roll out gradually.

This turns the cutover from a deployment into a configuration change, which means the rollback is also a configuration change. That difference matters a great deal at 11pm.

4. Separate chunking from embedding

Store the chunk text independently of its vector. If chunks are only reachable through the embedding pipeline, reindexing means re-running document parsing and chunking as well — which is slower, more expensive, and introduces the risk that your chunking logic has changed since the original ingest and produces different boundaries.

With chunks stored separately, reindexing reads existing text and calls the new embedding endpoint. That is a bulk job you can throttle, resume, and monitor.

Developer working on retrieval systems
Developer working on retrieval systems

The migration, once you have this

The whole thing becomes routine:

  1. Create the new versioned collection alongside the existing one.

  2. Backfill embeddings from stored chunk text in batches, with resumable checkpoints and rate limiting against the embedding provider.

  3. Dual-write new content into both collections during the backfill so the new index does not fall behind.

  4. Evaluate the new collection against a held-out query set, comparing retrieval quality rather than assuming the newer model is better — it usually is, occasionally it is not for your specific corpus and that is worth knowing before cutover.

  5. Switch the query layer behind the flag, initially for a subset of traffic.

  6. Retain the old collection for a rollback window, then drop it.

No downtime. No weekend. And step 4 is the one people skip, which is how teams migrate to a newer model and quietly lose quality on domain-specific queries.

A note on evaluation

Keep a fixed set of representative queries with known-good expected results, built before you need it. Migration is not the only time you need it — it is also how you evaluate chunking changes, retrieval parameters, and hybrid weighting.

Without it, comparing two embedding models is guesswork dressed up as a decision.

Frequently Asked Questions

Why can't embeddings from different models be mixed?

They occupy different vector spaces, so distances between them are meaningless. A table containing both produces incoherent rankings rather than merely degraded ones, and there is no conversion between spaces.

Do I need a separate table per model version?

Not necessarily — a discriminator column recording the model and version, with partitioning, works too. The requirement is that two generations can coexist while one serves traffic and the other backfills.

Why store chunk text separately from vectors?

So reindexing reads existing text rather than re-running parsing and chunking. It is faster, cheaper, and avoids the risk that chunking logic has changed and now produces different boundaries.

How do I know the new model is actually better?

Evaluate against a fixed, representative query set with known-good results before cutover. Newer models are usually better but not always for a specific domain corpus, and finding out afterwards is expensive.

What makes rollback safe?

Resolving the target collection from configuration rather than code, and retaining the previous collection for a rollback window. Cutover and rollback both become configuration changes rather than deployments.


The full database selection guide — categories, Postgres defaults, filtered vector search, managed versus self-hosted and cost structure: Best Database Software in 2026: A CTO Selection Guide.

More on our approach to LLM integration and machine learning development.

Subscribe to techcirkle