Re-embedding a Knowledge Base

Re-embedding a Knowledge Base (Switching the Embedding Model)

Audience: Unique Solution Engineering (running on customer-managed tenants) and platform administrators with chat.admin.all. This is an advanced maintenance operation that re-processes a tenant's entire vector store and can run for hours on large tenants. Coordinate with Unique before running on a production tenant.

Overview

Re-embedding regenerates the vector embeddings for all of a company's ingested content using a different embedding model, then atomically switches the company's live vector collection to the new model. Use it when you need to:

• Upgrade to a higher-quality / higher-dimensional embedding model (e.g. text-embedding-ada-002 → text-embedding-3-large).
• Change the embedding dimension of a tenant's vector store.
• Re-baseline embeddings after a model or provider change.

What it does under the hood (operation type SWITCH_EMBEDDING_MODEL):

1. preparing — creates a new, temporary vector collection sized for the new model's dimension.
2. reembedding — streams every chunk, re-embeds it with the new model, and writes the vectors into the new collection (in batches).
3. swap — once the new collection is complete, atomically points the company's live collection at it and updates the relational store.
4. cleanup — removes the old collection.

The job is checkpointed (a worker restart resumes from the last position) and non-destructive by default: the old collection and live search keep working throughout the run, and the swap only happens at the very end.

Who can run it

• Permission:chat.admin.all (Zitadel) andCONTENT:MANAGE.
• Scope: the companyId is taken from the authenticated admin user — you run it as an admin of the company being re-embedded.

Before you start (prerequisites)

The operation is resilient to concurrent ingestion and to transient vector-DB / rate-limit errors (they are retried automatically). For the cleanest, fastest run you may still choose to pause heavy ingestion for the company during the switch.

Step 1 — Dry run (preview, makes no changes)

mutation {
  createMaintenanceJob(input: {
    operationType: "SWITCH_EMBEDDING_MODEL"
    newEmbeddingModel: "text-embedding-3-large"
    newEmbeddingDimension: 3072
    dryRun: true
  }) { id status }
}

A dry run validates the request, estimates the number of chunks to process, and completes without changing any data. Read the job back (Step 3) — its result contains the plan, including estimatedChunks, the current collection, and the target model/dimension. Use the estimate to size the run (see Sizing).

Step 2 — Run the switch

mutation {
  createMaintenanceJob(input: {
    operationType: "SWITCH_EMBEDDING_MODEL"
    newEmbeddingModel: "text-embedding-3-large"
    newEmbeddingDimension: 3072
    destructive: false
    dryRun: false
  }) { id status createdAt }
}

Input fields

Invoking it (cURL) — authenticate with a chat.admin.all bearer token; companyId is derived from the token:

curl -X POST "$INGESTION_GRAPHQL_URL" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"mutation { createMaintenanceJob(input:{operationType:\"SWITCH_EMBEDDING_MODEL\",newEmbeddingModel:\"text-embedding-3-large\",newEmbeddingDimension:3072,destructive:false,dryRun:false}){ id status } }"}'

$INGESTION_GRAPHQL_URL is the tenant's ingestion GraphQL endpoint and $ADMIN_TOKEN is a Zitadel token for a chat.admin.all user — ask Unique for the endpoint if you don't have it. The mutation returns the new job's id.

Step 3 — Monitor

A background worker claims the job within ~60 seconds of creation. Track it with:

query {
  maintenanceJob(jobId: "mjob_xxx") {
    id status phase processed total failed errorMessage startedAt completedAt result
  }
}

List all jobs for the company with maintenanceJobs { id status phase processed total operationType createdAt }.

Cancel a running job with cancelMaintenanceJob(jobId: "mjob_xxx").

Sizing & duration

Throughput is bounded by the embedding model's tokens-per-minute (TPM) quota:

estimated minutes  ≈  (total_chunks × avg_tokens_per_chunk) ÷ TPM_quota
chunks per minute  ≈  TPM_quota ÷ avg_tokens_per_chunk

Worked example: 750,000 TPM quota, ~400 tokens/chunk → ~1,800 chunks/min → a 1.9M-chunk tenant ≈ **17 hours**. A few-thousand-TPM quota would make the same tenant take many days — raise the quota for large tenants.

Re-embedding is processed sequentially (one batch at a time). The single biggest lever on duration is the embedding deployment's TPM quota.

Safety, atomicity & rollback

• Non-destructive by default: users keep searching on the existing model for the entire run; the switch is atomic at the very end. If the job fails before the swap, nothing changes for users.
• Checkpoint / resume: progress is recorded continuously; a worker restart resumes from the last checkpoint rather than starting over.
• destructive: true deletes the old collection up front — only use it when you cannot keep both collections and you accept that there is no rollback.
• Rollback after a completed switch: run the switch again, targeting the previous model and dimension (this is a full re-embed back to the old model).

Troubleshooting

Configuration reference

Note on multi-endpoint embedding distribution: in environments where embedding requests are distributed across multiple endpoints, confirm that per-company model overrides are honored for the target company before running — otherwise the re-embed may use the default model. Check with Unique if your tenant uses multi-endpoint distribution.

Known limitations

• One maintenance job per company at a time (enforced by a per-company lock).
• AFAILED job is not auto-resumed — re-trigger it to restart (it re-embeds into a fresh collection; the partial collection from the failed run is garbage-collected).
• Runtime scales with corpus size and embedding quota — large tenants take hours; plan accordingly and monitor.