Troubleshooting

Recipes for the issues most likely to bite a first-time user.

Build issues

Cannot open include file "mormot.defines.inc"

Cause: the FPC compiler did not find the mORMot 2 source on its include path.

Fix: verify .claude/mormot2.config.json points at the actual mORMot 2 root, then either rebuild with lazbuild --build-all (which reads the config) or invoke fpc directly with -Fi<MORMOT2>/src.

Cause: the FPC install on this machine is missing its Windows import libraries (the lib/x86_64-win64/ directory).

Fix: build via lazbuild instead of bare fpc — Lazarus knows where to find the import libs. If you must use bare fpc, point -Fl at the directory containing libkernel32.a.

F2613 Unit 'SysUtils' not found (Delphi)

Cause: dcc64 was invoked without the modern Delphi namespace flag. The unit you are referencing as SysUtils actually lives in System.SysUtils.dcu.

Fix: pass -NSSystem;Winapi;System.Win (and any other namespaces your code uses, like Vcl for VCL). See the Cross-platform page for the full invocation.

Runtime issues

Ollama returned HTTP 408

Cause: the embed request timed out. The default TSimpleHttpClient receive timeout is 30 s, which is short for big batches against a CPU-only Ollama with a 300 M-parameter model.

Fix: use the persistent TMormotHttpTransport with its 120 s default (this is the production stack), or shrink the per-call batch size inside TRagOrchestrator (see DEFAULT_EMBED_BATCH in mormot.ai.rag.pas).

Ollama returned HTTP 404 model 'gemma3:1b' not found

Cause: you have not pulled the model.

Fix:

ollama pull gemma3:1b

The exact tag matters: gemma3:1b is different from gemma3:latest, which may not exist in your Ollama install.

Ollama: embeddings[0][0] is not a number

Cause (rare): the response JSON contains a bare-number token that mORMot's TDocVariantData parser stored as a varOleStr instead of a varDouble. Verified on Delphi but possible on FPC under unusual locales.

Fix: this is handled automatically in TOllamaEmbeddingClient.ParseEmbeddings via AnyVariantToDouble plus a try-except direct-cast fallback. If you see it, check whether your inner transport is mangling the JSON body (some proxies add BOMs or re-encode).

null value in column "unit_name" violates not-null constraint

Cause (historic): a chunker bug emitted standalone-comment chunks with NULL metadata. Fixed in commit 959f320 by replacing a with outChunks[H] block with explicit field assignments, because the with block shadowed the procedure parameter FilePath and the local unitName variable.

Fix if you see this again: make sure you are running a build at or after commit 959f320. Verify by checking the symbol table of build/test_runner.exe includes a method TPascalChunk.Family that is non-empty after ChunkerDecompose.

embedder returned X vectors for Y chunks

Cause: the embedder is returning a different count than what the orchestrator sent. Either the model truncated mid-batch, or the Workers configuration on IngestParallel is sending mismatched batches.

Fix: shrink the batch (DEFAULT_EMBED_BATCH = 16 is the canonical value), and verify the embed model is loaded correctly server-side.

dimension N does not match expected M (server-side error from PG)

Cause: the bound vector's dimension does not match the VECTOR(M) column's declared size. The embedder returned a non-default dimension, or the schema was created with a different dim than the one in your config.

Fix: align the schema dim and the embedder's ADim. The canonical pattern is the probe-and-recreate dance the chat CLI does:

emb := TOllamaEmbeddingClient.Create(http, url, model);  // ADim=0 -> flexible
dim := emb.Embed('probe').Dim;
emb := TOllamaEmbeddingClient.Create(http, url, model, dim);  // ADim=dim -> strict
cfg.EmbeddingDim := dim;
PgVectorSetupSchema(conn, cfg);

Some tests FAILED even though every Total failed = 0/N

Cause: a published test method completed without invoking any Check. mORMot's TSynTests.Run flags this as a soft failure because a method that passes the test framework's exception filter without registering a single assertion usually indicates an early exit on a code path that should have asserted something.

Fix: add at least one Check(true, '...skipped because reason') to every gated path that may short-circuit on a missing env var. The live tests in this project already do this consistently; it is only worth checking when you add a new test method.

Configuration issues

Live tests skip silently

Cause: one of the required env vars is missing. The skip is by design; the suite does not have a way to express “skipped” that is distinct from “passed”, so each gated test logs a Check(true, '...') and exits early.

Fix: set the missing env vars (the Configuration page lists them all) and rerun. If a specific test class skips even with the env vars set, check that the binaries exist where the test expects them (the MORMOT2_PGVECTOR_CLI_BIN_DIR smoke for the spawn-based suite).

Smoke probes fail after a small ingest

Cause: the smoke probes assume a representative ingest (orm + db + net subdirectories of mORMot 2). A small ingest of a single file will not contain the canonical content the probes look for, and they will fail.

Fix: run 01_ingest_mormot2 --src <MORMOT2>/src to populate enough context, then rerun the probes.

libpq not found on the runtime

Cause: the system loader cannot find libpq.dll (or libpq.so.5).

Fix on Windows: either add <PG_INSTALL>\bin to PATH, or set the MORMOT2_PGVECTOR_LIBPQ env var to the absolute path, or pass --libpq to the CLI program. Setting the env var sets SynDBPostgresLibrary before the first connection so the rest of the process picks it up automatically.

Performance gotchas

Ingest is slower than expected

Most likely the embedder is the bottleneck. Tools to investigate:

  1. Check whether Ollama is GPU-accelerated. The tag listing (ollama list) will tell you the model size; CPU-only inference on a 300 M-parameter model is roughly 1-2 s per single embed call, much faster in batches.
  2. Increase the per-call batch (the default 16 chunks fits comfortably in the 120 s receive timeout). Larger batches amortize HTTP overhead.
  3. Increase Workers in IngestParallel, but only if the embedder itself can serve concurrent requests. Local Ollama with a single model loaded does not benefit; cloud embedders or multi-replica self-hosted setups do.

Retrieval is slower than expected

Two main suspects:

  1. The HNSW index has not been built yet, or was dropped. Verify with \d mormot_chunks in psql; the output should list a hnsw index on the embedding column.
  2. The embedding column dimension is too large for the table size. For tens of thousands of rows, 256 dims is usually a better trade-off than 768.

tsvector rank is always 0

Cause: PostgreSQL's to_tsvector('simple', content) does not tokenize Pascal source code well — it splits on whitespace + a few punctuation characters, but does not understand CamelCase identifiers like TRestOrmClient.

Fix: either accept the limitation and rely on vector similarity (higher α), or switch the ts column to a custom configuration that splits on case boundaries. Out-of-the-box this requires writing a custom dictionary in PostgreSQL; that work is not in v1.