Getting Started

Prerequisites

  • Delphi 12+ or Free Pascal trunk (FPC 3.3.1+)
  • mORMot v2 — clone from github.com/synopse/mORMot2
  • An API key for at least one provider: OpenAI, Anthropic, or Google Gemini

Installation

  1. Clone this repository alongside your mORMot v2 installation:
your-workspace/
  mORMot2/          <-- mORMot v2 source
  mormot-ai-sdk/    <-- this SDK
    src/             <-- SDK units
    ex/              <-- examples
    docs/            <-- documentation
  1. Add the src/ directory to your project's unit search path, along with the standard mORMot v2 paths:
mORMot2/src
mORMot2/src/core
mORMot2/src/net
mORMot2/src/lib
mORMot2/src/crypt
mORMot2/src/app
mormot-ai-sdk/src
  1. Add the SDK units to your uses clause:
uses
  mormot.ai.litellm.types,
  mormot.ai.litellm.client;

That's all. No packages to install, no components to register.

API Key Configuration

The SDK resolves API keys in this order:

  1. Explicit parameter — pass ApiKey directly to Chat()
  2. .env file — a KEY=value file next to the executable
  3. OS environment variable — OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.

.env File

Create a .env file in the same directory as your compiled executable:

# OpenAI
OPENAI_API_KEY=sk-proj-...

# Anthropic
ANTHROPIC_API_KEY=sk-ant-api03-...

# Google Gemini (either key name works)
GEMINI_API_KEY=AIzaSy...
# or: GOOGLE_API_KEY=AIzaSy...

The SDK reads this file using FindIniNameValue — standard KEY=value format, one per line, # for comments.

Environment Variable Names per Provider

Provider Environment Variable Fallback
openai OPENAI_API_KEY —
anthropic ANTHROPIC_API_KEY —
gemini GEMINI_API_KEY GOOGLE_API_KEY

First Chat Call

program HelloLLM;

{$I mormot.defines.inc}
{$APPTYPE CONSOLE}

uses
  sysutils,
  mormot.core.base,
  mormot.core.variants,
  mormot.ai.litellm.types,
  mormot.ai.litellm.client;

var
  LLM: TLiteLLM;
  Messages: TLLMMessageDynArray;
  Response: TLLMResponse;
begin
  LLM := TLiteLLM.Create;
  try
    SetLength(Messages, 1);
    Messages[0].Role := 'user';
    Messages[0].Content := 'Say hello in French, Japanese, and Arabic.';

    Response := LLM.Chat('openai/gpt-4o', Messages);

    WriteLn('Model: ', Response.Model);
    WriteLn('Content: ', Response.Content);
    WriteLn('Finish: ', Response.FinishReason);
    WriteLn('Usage: ', Response.Usage.ToString);
    WriteLn('Cost: $', FormatFloat('0.000000', Response.Cost));
  finally
    LLM.Free;
  end;
end.

What Happens Under the Hood

  1. TLiteLLM.Create reads .env, creates the HTTP handler, registers OpenAI/Anthropic/Gemini providers, initializes the router and cost tracker.
  2. ParseProviderModel('openai/gpt-4o') splits into Provider='openai', Model='gpt-4o'.
  3. ResolveApiKey('openai', '') reads OPENAI_API_KEY from .env or environment.
  4. ResolveApiBase('openai', '') returns 'https://api.openai.com/v1'.
  5. ChatWithRetry calls Chat on the handler with up to 2 retries on retryable errors.
  6. TOpenAIProviderConfig.TransformRequest builds the OpenAI JSON body.
  7. A THttpClientSocket sends the POST request to api.openai.com:443/v1/chat/completions.
  8. TOpenAIProviderConfig.TransformResponse parses the JSON response into TLLMResponse.
  9. TLiteLLMCostTracker.CalculateCost computes the dollar cost from token usage.

Switching Providers

Change one string to switch providers — no other code changes needed:

// OpenAI
Response := LLM.Chat('openai/gpt-4o', Messages);

// Anthropic Claude
Response := LLM.Chat('anthropic/claude-sonnet-4-20250514', Messages);

// Google Gemini
Response := LLM.Chat('gemini/gemini-2.5-flash', Messages);

Each provider handles its own format differences internally:

  • OpenAI: Quasi pass-through (OpenAI format is the SDK's canonical format)
  • Anthropic: System messages extracted to top-level system field, content blocks, input_schema for tools, x-api-key auth header, anthropic-version header
  • Gemini: Messages converted to contents[].parts[], system prompt to systemInstruction, tools to functionDeclarations, API key via URL query parameter

Multi-Turn Conversation

Build conversation history by appending messages:

var
  LLM: TLiteLLM;
  Messages: TLLMMessageDynArray;
  Response: TLLMResponse;

  procedure AppendMsg(const Role, Content: RawUtf8);
  var n: Integer;
  begin
    n := Length(Messages);
    SetLength(Messages, n + 1);
    Messages[n].Role := Role;
    Messages[n].Content := Content;
  end;

begin
  LLM := TLiteLLM.Create;
  try
    // System prompt
    AppendMsg('system', 'You are a helpful Pascal programming assistant.');

    // First user message
    AppendMsg('user', 'How do I create a dynamic array in Free Pascal?');
    Response := LLM.Chat('openai/gpt-4o', Messages);
    AppendMsg('assistant', Response.Content);

    // Follow-up
    AppendMsg('user', 'Show me how to sort it.');
    Response := LLM.Chat('openai/gpt-4o', Messages);
    WriteLn(Response.Content);
  finally
    LLM.Free;
  end;
end;

Parameters

The simple Chat overload accepts MaxTokens and Temperature:

function Chat(const Model: RawUtf8;
  const Messages: TLLMMessageDynArray;
  const ApiKey: RawUtf8 = '';
  const ApiBase: RawUtf8 = '';
  MaxTokens: Integer = 4096;
  Temperature: Double = 1.0): TLLMResponse;

For advanced parameters (top_p, stop, response_format, seed, etc.), use ChatRaw with a params variant:

var
  LLM: TLiteLLM;
  Messages: TLLMMessageDynArray;
  Tools: TLLMToolDynArray;
  Response: TLLMResponse;
  Params: variant;
begin
  LLM := TLiteLLM.Create;
  try
    // ... build Messages and Tools ...
    Params := _Obj([
      'max_tokens', 2048,
      'temperature', 0.7,
      'top_p', 0.9,
      'stop', _Arr(['END', '---']),
      'response_format', _Obj(['type', 'json_object'])]);

    Response := LLM.ChatRaw('openai/gpt-4o', Messages, Tools, Params);
  finally
    LLM.Free;
  end;
end;

ChatRaw normalizes empty params to _Obj([]) and injects the stream flag when Stream=True, then passes the params variant to the provider's TransformRequest. Each provider maps the OpenAI-style params to its own format automatically.

Next Steps

  • Providers — Provider-specific details and transformations
  • Streaming — Real-time SSE streaming with callbacks
  • Tools — Tool definitions and agentic tool call loops
  • Router — Multi-deployment routing and load balancing
  • Proxy — OpenAI-compatible proxy server
  • API Reference — Complete public API