Tool Calling

Tool calling (function calling) lets the LLM request execution of functions you define. The SDK provides a unified tool format that works across all three providers.

Defining Tools

Tools use the OpenAI format with TLLMTool and TLLMToolFunction records:

var
  Tools: TLLMToolDynArray;
begin
  SetLength(Tools, 2);

  // Tool 1: get_weather
  Tools[0].Type_ := 'function';
  Tools[0].Function_.Name := 'get_weather';
  Tools[0].Function_.Description := 'Get current weather for a location';
  Tools[0].Function_.Parameters := _Obj([
    'type', 'object',
    'properties', _Obj([
      'location', _Obj([
        'type', 'string',
        'description', 'City name, e.g. "Paris"']),
      'unit', _Obj([
        'type', 'string',
        'enum', _Arr(['celsius', 'fahrenheit'])])]),
    'required', _Arr(['location'])]);

  // Tool 2: search_web
  Tools[1].Type_ := 'function';
  Tools[1].Function_.Name := 'search_web';
  Tools[1].Function_.Description := 'Search the web for information';
  Tools[1].Function_.Parameters := _Obj([
    'type', 'object',
    'properties', _Obj([
      'query', _Obj([
        'type', 'string',
        'description', 'Search query'])]),
    'required', _Arr(['query'])]);
end;

Parameters is a JSON Schema as a TDocVariant. Use _Obj/_Arr from mormot.core.variants to build it.

Making a Tool Call Request

var
  Response: TLLMResponse;
begin
  Response := LLM.Chat('openai/gpt-4o', Messages, Tools);

  if Response.HasToolCalls then
  begin
    WriteLn('LLM wants to call ', Length(Response.ToolCalls), ' tool(s):');
    for i := 0 to High(Response.ToolCalls) do
    begin
      WriteLn('  ', Response.ToolCalls[i].Name);
      WriteLn('  Args: ', VariantSaveJson(Response.ToolCalls[i].Arguments));
    end;
  end
  else
    WriteLn(Response.Content);
end;

TLLMToolCall Record

TLLMToolCall = packed record
  Id: RawUtf8;           // unique call ID (provider-assigned or synthetic)
  Type_: RawUtf8;        // always 'function'
  Name: RawUtf8;         // function name matching your tool definition
  Arguments: variant;    // parsed JSON arguments as TDocVariant
  RawData: variant;      // provider-specific raw data (e.g. Gemini thoughtSignature)
end;

• Id: Used to match tool results back to calls. OpenAI/Anthropic provide real IDs; Gemini gets synthetic call_0, call_1, etc.
• Arguments: Already parsed from JSON into a TDocVariant object. Access fields with _Safe(Arguments)^.U['location'].
• RawData: Contains provider-specific data needed for round-trip fidelity. For Gemini, this includes the original functionCall part and thoughtSignature if present.

Agentic Tool Call Loop

A typical agent loop executes tools and feeds results back until the LLM stops requesting tools:

var
  Messages: TLLMMessageDynArray;
  Response: TLLMResponse;
  ToolCall: TLLMToolCall;
  ToolResult: RawUtf8;
  i, n: Integer;

  procedure AppendMessage(const Role, Content: RawUtf8;
    const Name: RawUtf8 = ''; const ToolCallId: RawUtf8 = '';
    const ToolCalls: variant = Unassigned);
  begin
    n := Length(Messages);
    SetLength(Messages, n + 1);
    Messages[n].Role := Role;
    Messages[n].Content := Content;
    Messages[n].Name := Name;
    Messages[n].ToolCallId := ToolCallId;
    Messages[n].ToolCalls := ToolCalls;
  end;

begin
  // Initial system + user messages
  AppendMessage('system', 'You are a helpful assistant with tool access.');
  AppendMessage('user', 'What is the weather in Paris and Tokyo?');

  repeat
    Response := LLM.Chat('openai/gpt-4o', Messages, Tools);

    if not Response.HasToolCalls then
    begin
      // Final answer — done
      AppendMessage('assistant', Response.Content);
      WriteLn(Response.Content);
      Break;
    end;

    // Append assistant message with tool calls (for conversation history)
    // Build tool_calls variant from Response.ToolCalls for round-trip
    AppendMessage('assistant', Response.Content, '', '',
      BuildToolCallsVariant(Response.ToolCalls));

    // Execute each tool call and append results
    for i := 0 to High(Response.ToolCalls) do
    begin
      ToolCall := Response.ToolCalls[i];

      // Execute the tool (your application logic)
      if ToolCall.Name = 'get_weather' then
        ToolResult := FormatUtf8('{"temp": 22, "unit": "celsius", "city": "%"}',
          [_Safe(ToolCall.Arguments)^.U['location']])
      else
        ToolResult := '{"error": "unknown tool"}';

      // Append tool result
      AppendMessage('tool', ToolResult, ToolCall.Name, ToolCall.Id);
    end;
  until False;
end;

Building the Assistant Message with Tool Calls

For OpenAI, the assistant message must include the tool_calls array in the conversation history. The simplest approach is to build it from Response.ToolCalls:

function BuildToolCallsVariant(const ToolCalls: TLLMToolCallDynArray): variant;
var
  i: Integer;
begin
  Result := _Arr([]);
  for i := 0 to High(ToolCalls) do
    _Safe(Result)^.AddItem(_Obj([
      'id', ToolCalls[i].Id,
      'type', 'function',
      'function', _Obj([
        'name', ToolCalls[i].Name,
        'arguments', VariantSaveJson(ToolCalls[i].Arguments)])]));
end;

// Usage:
AppendMessage('assistant', Response.Content, '', '',
  BuildToolCallsVariant(Response.ToolCalls));

Gemini thoughtSignature

For Gemini models with thinking enabled, function call parts may include a thoughtSignature field. This must be preserved and sent back in the next request for the model to maintain its reasoning chain.

The SDK stores the raw data but replay requires manual wiring: TLLMToolCall.RawData stores the complete Gemini Part object (including thoughtSignature). When building the assistant message's tool_calls variant, you must copy RawData into a _raw_fc field — TGeminiProviderConfig.BuildParts checks for this field and replays the original Part verbatim.

When building assistant messages for Gemini, store the raw tool call data:

// Store raw tool_calls in the assistant message for Gemini round-trip
for i := 0 to High(Response.ToolCalls) do
begin
  TC := _Obj([
    'id', Response.ToolCalls[i].Id,
    'type', 'function',
    'function', _Obj([
      'name', Response.ToolCalls[i].Name,
      'arguments', VariantSaveJson(Response.ToolCalls[i].Arguments)])]);
  // Preserve raw Gemini data for thoughtSignature round-trip
  if not VarIsVoid(Response.ToolCalls[i].RawData) then
    _Safe(TC)^.AddValue('_raw_fc', Response.ToolCalls[i].RawData);
  _Safe(ToolCallsArr)^.AddItem(TC);
end;

Tool Choice

Control whether the LLM must use tools:

// Using ChatRaw with params variant
Params := _Obj([
  'max_tokens', 4096,
  'tool_choice', 'auto']);     // default: let LLM decide

// Force tool use
Params := _Obj(['tool_choice', 'required']);

// Force specific tool
Params := _Obj(['tool_choice', _Obj([
  'type', 'function',
  'function', _Obj(['name', 'get_weather'])])]);

Response := LLM.ChatRaw('openai/gpt-4o', Messages, Tools, Params);

Tool choice values are automatically mapped per provider:

Provider Differences

All these differences are handled by the SDK — you define tools once in OpenAI format and they work everywhere.