Production Deployment

Notes on running mormot2-pgvector outside a developer's laptop.

Recommended transport stack

For any deployment that issues hundreds of embed calls (i.e. any real ingest), wrap the live transport in two decorators:

inner    := TMormotHttpTransport.Create(
              {receive_timeout_ms=}120000,
              {keep_alive_ms=}30000);
retrying := TRetryingHttpTransport.Create(inner,
              {max_attempts=}3, {initial_delay_ms=}250);
limited  := TConcurrencyLimitedHttpTransport.Create(retrying,
              {limit=}4);
client   := TOllamaEmbeddingClient.Create(limited,
              base_url, model, dim);

The composition gives you:

• One persistent TCP socket reused across requests (KeepAliveMS on the request header).
• A ReceiveTimeoutMS long enough to tolerate the slowest single embed call you expect.
• Automatic retries on HTTP 5xx / 429 / network errors with binary exponential backoff (250 / 500 / 1000 ms with defaults).
• A hard cap on in-flight calls so the server (typically a single-instance Ollama with one model loaded) is not asked to queue more requests than it can process.

Concurrency tuning

The right Limit depends on what is on the other end:

• Local Ollama with a single model loaded: 1 (or a small number). Ollama serializes requests for a model internally; sending more in parallel just queues them server-side and inflates per-request latency.
• A managed cloud embedder (OpenAI, Voyage, …): whatever the provider's documented per-account concurrency cap is.
• A self-hosted embedder behind a load balancer: tune up to the point where p99 latency stops decreasing.

The accompanying IngestParallel(SrcRoot, Factory, Workers) exposes the worker count separately. A typical pattern for Ollama is:

• Workers = 4 for the parallel ingest.
• The factory produces a fresh TMormotHttpTransport per worker (each worker owns its own socket).
• The transport stack does NOT wrap with TConcurrencyLimitedHttpTransport because the mod-4 limit is already enforced by Workers = 4.

Choosing an embedding dimension

embeddinggemma (the default) supports Matryoshka representation learning, so you can index at 128 / 256 / 512 / 768 dimensions and trade accuracy for storage and HNSW build / query time:

Construct the embedder with the chosen dim in strict mode:

emb := TOllamaEmbeddingClient.Create(transport, base_url, model,
  {ADim=}256);   // strict mode + MRL fallback

The schema must match: EmbeddingDim := 256 in the TPgVectorSchemaConfig you pass to SetupSchema.

Library location

libpq.dll (Windows) and libpq.so.5 (Linux) are NOT statically linked into the binaries. They are resolved at runtime through one of two mechanisms:

1. The library lives in a directory on the system search path (PATH on Windows, LD_LIBRARY_PATH on Linux). Postgres' install does this by default.
2. The application sets SynDBPostgresLibrary (mORMot 2 global) to an explicit absolute path before the first connection. The CLI programs accept this path via --libpq.

For production deployments that need single-binary distribution, the recommended pattern is to ship libpq.dll (and its OpenSSL dependencies on Windows) alongside the executable, set the env var, and let the OS dynamic loader find it.

Schema migrations

The canonical schema is created via PgVectorSetupSchema (or the multi-statement string returned by PgVectorSchemaSql). Both use CREATE TABLE IF NOT EXISTS and CREATE INDEX IF NOT EXISTS, so re-running the helper is idempotent.

To migrate to a different dimension after rows are already populated:

ALTER TABLE mormot_chunks DROP COLUMN embedding;
ALTER TABLE mormot_chunks ADD COLUMN embedding VECTOR(256);
DROP INDEX mormot_chunks_embedding_hnsw_idx;
CREATE INDEX ON mormot_chunks USING hnsw (embedding vector_cosine_ops)
  WITH (m=16, ef_construction=64);
-- then re-ingest to populate the new column

The unit_name, family, kind, file_path, and other metadata columns are dimension-independent; only the embedding column needs to be rebuilt.

Resource discipline

Every long-lived object in the production stack has clear ownership:

• IEmbeddingClient and IHttpTransport are reference-counted interfaces; they release the underlying transport when the last reference goes out of scope.
• TSqlDBPostgresConnectionProperties is a regular TObject; the caller must Free it after the last orchestrator using it has been freed.
• TRagOrchestrator does not own the properties or the embedder; you pass them in and they continue to exist after orch.Free.
• ISqlDBStatement is reference-counted; prepared statements are released automatically when the local variable falls out of scope, even on exception paths.

Observability

The CLI programs print configuration messages to stderr and the final summary / answer to stdout. The chat CLI also prints provenance lines (one per retrieved chunk with its unit:line_start-line_end and the three score components) to stderr, so you can pipe stdout while still seeing where each fact came from.

For deeper diagnostics, mORMot 2's TSynLog infrastructure is available and the test runner uses it:

TSynLogTestLog := TSqlLog;   // enable verbose logging in tests

In production code, instantiate a TSynLog family and add the relevant log levels you care about (sllInfo, sllError, sllException).

Backup

A pg_dump of the mormot_chunks table includes the embeddings as binary BLOBs. Restore is symmetric. There is no special handling required: pgvector indexes are rebuilt from the embedding column on restore.

Re-embedding

There is currently no incremental git-diff-driven re-embedding path shipped in v1. To refresh the index after the source has changed:

1. Truncate the mormot_chunks table (or use --reset on 01_ingest_mormot2).
2. Re-run 01_ingest_mormot2 --src <new_source_root>.