LLM calls

A call

- id: summarize
  type: llm_call
  depends_on: [fetch]
  system_prompt: "You write short, factual summaries."
  prompt: "Summarize in three sentences: {state.report.body}"
  model: gpt-4o
  temperature: 0.2
  max_tokens: 400
  output: summary

The step also takes the fields every step has: output, timeout, retry and depends_on.

What the step writes

• output receives the reply’s text, or the parsed value with a response_schema. With tools, it is the text of the model’s last reply.
• steps.<id>.output always holds the raw reply text: with a response_schema, that is the reply as the model wrote it, any words around the JSON included.
• The step’s result carries the tokens, reasoning tokens included, the cost, the model and provider that answered, and the provider’s own finish_reason, such as stop, end_turn or STOP. agentloom run --json prints it.

Tokens and cost cover every call the step made: each turn of a tool loop, each re-ask for valid JSON and each conversation summary.

When a call fails

Reliability covers retries, timeouts and what a failed step does to the rest of the run.

Attachments

- id: describe
  type: llm_call
  model: gpt-4o
  prompt: "What is in this picture, and is the receipt total legible?"
  attachments:
    - source: "{state.receipt_url}"          # downloaded by AgentLoom
    - type: pdf
      source: invoices/2026-05.pdf           # a local file
  output: description

How the source is read:

• A file, a download, a data: URL or a base64 string may be up to 20 MB.
• URLs at private, loopback or link-local addresses are refused, with either fetch.
• The sandbox applies. sandbox.allow_network: false and sandbox.allowed_domains restrict downloads even when the sandbox is disabled. With the sandbox enabled, a local file must sit under readable_paths or allowed_paths.

An attachment the provider can’t take, such as a PDF for OpenAI, fails before any request is sent. The call moves on to the next provider for the model, and the step fails without a retry when none can take it.

In a conversation, attachments go with this step’s user turn only and aren’t stored in the conversation. A conversation step without a prompt has no user turn, and sends no attachments.

Reasoning

- id: plan
  type: llm_call
  model: claude-sonnet-4-5
  max_tokens: 8000                 # above the budget
  prompt: "Plan the migration of {state.service} to the new queue."
  thinking:
    enabled: true
    budget_tokens: 4000
  output: plan

• Reasoning tokens count in the step’s tokens and are billed at the output price. OpenAI and Google report them apart; Anthropic and Ollama count them in the reply’s tokens.
• With capture_reasoning on, the reasoning text reaches the step’s result as reasoning_content, where agentloom run --json prints it, and is redacted when the step’s output is. It doesn’t reach the state or a recording. OpenAI doesn’t return it.
• OpenAI’s reasoning models get max_tokens as max_completion_tokens, and no temperature. gpt-5-chat isn’t one of them: it gets both as given.
• Agents don’t take a thinking block.

Tool calling

The model decides which tools to call, and with what arguments. AgentLoom runs them and sends back their results until the model answers.

- id: ask
  type: llm_call
  prompt: "{state.question}"
  tools:
    - name: http_request
      description: "Make an HTTP request to a given URL and return the response body."
      parameters:
        type: object
        properties:
          url: { type: string, description: "Full URL to GET." }
          method: { type: string, enum: ["GET", "POST"] }
        required: [url]
  tool_choice: auto
  max_tool_iterations: 3
  output: answer

The name must match a registered tool. agentloom run registers the four built-in tools: http_request, shell_command, file_read and file_write. Tools of your own are registered from Python. A registered tool that the step doesn’t declare stays out of the model’s reach.

The loop

1. AgentLoom calls the model with the tools.
2. A reply without tool calls is the answer: the step succeeds with it.
3. Otherwise, AgentLoom runs every call in the reply at once, sends the results back in the order of the calls, and calls the model again.
4. At max_tool_iterations, the step still succeeds, with the last reply’s text and finish_reason: max_tool_iterations. The tools that last reply asked for don’t run, as no call would read their results: with max_tool_iterations: 1, no tool ever runs.

• A failing tool doesn’t fail the step. An unknown tool, arguments that aren’t valid JSON, an exception or a sandbox refusal go back to the model as the tool’s result, and the loop goes on. Arguments aren’t checked against parameters.
• required and a named tool apply to the first call only. The later calls of the loop use auto, so the model can read the results and answer. A model that keeps calling tools up to max_tool_iterations and leaves an empty reply ends with finish_reason: max_tool_iterations_no_answer.
• A retry runs the whole loop again, the tools included. A tool with side effects, such as a write, a message or a payment, should be safe to repeat, or the step should set retry: {max_retries: 0}.
• Tools and a response_schema can’t share a step. An agent with an output_schema combines them.
• A step with tools doesn’t stream: its answer reaches the stream in one piece at the end.

OpenAI, Anthropic, Google, Ollama and generic_openai all take tools; Ollama ignores tool_choice, and whether a local model calls tools depends on the model. A model calling a tool is the dynamic counterpart of a tool step, where the workflow picks the tool.

Structured output

response_schema holds the reply to a JSON shape, and the step writes the parsed value to the state.

- id: classify
  type: llm_call
  prompt: "Classify the intent of: {state.message}"
  response_schema:
    type: pydantic
    model: examples.structured_output_schemas.Classification
  output: classification          # a Classification instance

- id: extract
  type: llm_call
  prompt: "Extract the name and date from: {state.text}"
  response_schema:
    type: json_schema
    schema:
      type: object
      properties:
        name: { type: string }
        date: { type: string, format: date }
      required: [name, date]
  output: extracted               # a dictionary

- id: ideas
  type: llm_call
  prompt: "Suggest five names for {state.product}, as a JSON object."
  response_schema:
    type: json_object
  output: ideas

Validation. AgentLoom reads the JSON from the reply, from a fenced JSON block in it, or from its first {…}, and validates it. An invalid reply goes back to the model with the validation error, and the model tries again, up to retry.max_retries more calls within the same attempt. If none is valid, the step fails without a retry. A streamed reply is validated once it ends, and a failure there fails the step without a retry.

Storage. output receives a Pydantic instance or a dictionary, which templates read like any other value: {state.classification.label}. Checkpoints and --json write it as a plain dictionary, so a resumed run reads a dictionary. Redaction patterns reach the fields of a Pydantic instance, as those of a dictionary.

Conversations

A conversation is a list of messages kept in the state, which several steps continue. Each step adds its user turn and the model’s reply.

state:
  chat:
    messages:
      - { role: system, content: "You are a concise, cheerful assistant." }
    token_budget: 800
    trim_policy: drop_pairs

steps:
  - id: turn_1
    type: llm_call
    conversation: state.chat
    prompt: "Recommend one Python library for CLI apps."
    output: reply_1

  - id: turn_2
    type: llm_call
    depends_on: [turn_1]
    conversation: state.chat
    prompt: "Any second choice for larger apps?"
    output: reply_2

conversation names a state key, as state.chat or chat. A key that doesn’t exist yet starts an empty conversation, and a plain list of messages is accepted as one. The key holds:

A turn

1. A system_prompt is stored as the first message if the conversation has none. If it already has one, a later step’s different system_prompt is ignored.
2. The prompt is added as a user turn. Without a prompt, the last message must already be the user’s or a tool’s.
3. The conversation is trimmed to its budget.
4. The model answers from the whole conversation, and its reply is added as an assistant turn.

• Only the final text is stored. The calls and results of a tool loop, and the re-asks for valid JSON, aren’t. A structured reply is stored as the model wrote it.
• The conversation and output are written only when the step succeeds.
• Two steps of one layer on one conversation are refused when the file loads: each would write its own copy, and one of the turns would be lost. Chain them with depends_on. Branches of one router are exempt, because only one of them runs.

Trimming

The budget is counted roughly: about one token per four characters, plus a little for each message’s role and name. Leading system messages always stay.

• drop_oldest removes the oldest messages until the rest fits. If the system messages and the summary alone reach the budget, nothing is removed, and the request goes over.
• drop_pairs removes the oldest user and assistant pairs. If the system messages and the summary alone reach the budget, nothing is removed, and the request goes over.
• summarize_oldest folds the oldest messages into summary, with one extra call to the step’s model, and sends the summary as a system message. If that call fails, the messages are joined as plain text instead. The summary counts in the budget and isn’t trimmed: once it and the system messages reach the budget, nothing more is folded, and the request goes over.

Trimmed messages leave the stored conversation too. A budget too small to hold the new user turn fails the step at once.

Several speakers

speaker names the assistant turns of a step, so several voices can share one conversation. It takes letters, digits, _ and -, up to 64 characters:

- id: planner
  type: llm_call
  conversation: state.debate
  speaker: alice
  system_prompt: "You propose plans. Be concrete."
  prompt: "Propose a rollout plan for {state.feature}."
  output: plan

- id: critic
  type: llm_call
  depends_on: [planner]
  conversation: state.debate
  speaker: bob
  system_prompt: "You find the weakest point of a plan."
  output: critique

With a speaker, the step’s system_prompt applies to its own turn only and isn’t stored, so each speaker keeps its own instructions. OpenAI receives the speaker’s name as the message’s name; the other providers see it as a [alice] prefix on the turn. Agents are speakers by their name.

Streaming

stream: true on a step, or config.stream: true for every step, asks the provider for the reply as it is written. agentloom run --stream sets config.stream for one run; a step’s own stream: false still wins.

• agentloom run --stream prints the text as it arrives, without step names, so parallel steps interleave. With --json, nothing is printed until the end. Without --stream, steps that stream print nothing as they go.
• A streaming step records its time to first token in its result and metrics.
• Steps with tools, agent turns and conversation summaries don’t stream: their text arrives in one piece. A structured reply streams as raw JSON and is validated at the end.
• A streamed call falls back to another provider only before its first chunk. After that, a provider error fails the attempt, and the step’s retry decides what comes next. A 429 doesn’t count against the provider’s circuit breaker.

What’s next

• Agents

  A model, its tools and its rules, defined once and run as a step.

• Providers

  Each provider’s models, capabilities and pricing.

• Tools & sandbox

  The built-in tools, tools of your own, and what the sandbox allows.