mormot.ai.sdk

A unified multi-provider LLM client for Free Pascal and Delphi, built on mORMot 2. One API for OpenAI, Anthropic and Gemini — the provider-specific request/response transformations happen inside the SDK.

LLM := TLiteLLM.Create;
try
  SetLength(Messages, 1);
  Messages[0].Role := 'user';
  Messages[0].Content := 'Say hello in French.';
  Response := LLM.Chat('openai/gpt-4o', Messages);
  WriteLn(Response.Content);
finally
  LLM.Free;
end;

What it does

• Three providers — OpenAI, Anthropic, Gemini, selected by a provider/model string. Register your own with RegisterProvider.
• Streaming — SSE with an OnDelta callback; the accumulated TLLMResponse is returned at the end.
• Tool calling — define tools once in OpenAI format; they work on every provider, including the agentic call/result loop.
• Router — multiple deployments per logical model, five load-balancing strategies, per-deployment cooldown, RPM limits and fallback chains.
• Cost tracking — dollar cost computed from token usage after each call.
• Proxy — an OpenAI-compatible HTTP server so OpenAI-only applications can reach any provider.

Units are named mormot.ai.litellm.*, reflecting the origin of the port (the LiteLLM Python SDK). Only the SDK layer is ported — not the LiteLLM gateway.

Pages