Talking to a model

Every applied-AI system in this primer, from a one-shot classifier to a multi-agent research team, talks to the model the same way: it sends a list of messages and gets one message back. This lesson shows that exchange exactly, and then the one feature that turns a chatbot into an agent: tool calling.

Think of the model as a brilliant consultant who works by post. You mail a letter with your whole conversation so far (the consultant keeps no notes between letters) and get one letter back. The reply is either an answer, or a filled-in request form: "please look up the weather in Paris and send me the result." The consultant never picks up the phone themselves. You decide whether to run the request, run it, and mail back the result, together with the whole conversation again.

Two facts fall straight out of this picture, and they explain most of the behaviour of real systems:

1. The model executes nothing. A tool call is a request. Your code is the only thing that acts, so your code is where safety lives.
2. Every call resends everything. The model has no memory between calls, so each request carries the full history. Long conversations cost more on every single turn.

A user asks "What's the weather in Paris?" and the application offers one tool, get_weather. Four messages later, the user has an answer:

Message 3 has role user even though no human typed it: tool results always travel back in the user's turn. The id in message 2 and the tool_use_id in message 3 match, which is how the model knows which request a result answers when it asked for several at once. trace_tool_call_round_trip() produces exactly this transcript.

sequenceDiagram
  participant U as User
  participant A as Your code
  participant M as Model
  participant T as get_weather tool
  U->>A: What's the weather in Paris?
  A->>M: messages [1] + tool definitions
  M-->>A: tool_use get_weather {city: Paris}  (stop_reason: tool_use)
  A->>A: validate the arguments, check permissions
  A->>T: get_weather("Paris")
  T-->>A: 18°C, sunny
  A->>M: messages [1, 2, 3 = tool_result]
  M-->>A: It's 18°C and sunny in Paris.  (stop_reason: end_turn)
  A-->>U: It's 18°C and sunny in Paris.

In code: ToolCall holds the request in message 2, and tool_result_block builds the reply in message 3, carrying the matching id.

Messages use the Anthropic Messages API shape directly, so what you learn here maps one-to-one onto production code:

{"role": "user", "content": "What's our PTO policy?"}
{"role": "assistant", "content": [
    {"type": "text", "text": "Let me look that up."},
    {"type": "tool_use", "id": "toolu_1", "name": "search_kb", "input": {"query": "PTO"}},
]}
{"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "toolu_1", "content": "hr-001: ..."},
]}

A tool definition is a name, a description and a JSON Schema for the input. The model chooses tools by reading those descriptions, so writing them well is prompt engineering (see primer.agents.tools).

In code: LLMResponse is one reply, normalized: its text, its ToolCall list, its stop reason, its Usage and the exact content blocks to append as the assistant turn. content_blocks, last_user_text, tool_results and tool_calls_so_far read a conversation in this format.

Every agent lesson is written against one small interface, LLM, with one method, LLM.complete(system=..., messages=..., tools=...). Two classes implement it:

flowchart LR
  L["LLM interface<br/>complete(system, messages, tools)"]
  L --> S["ScriptedLLM<br/>a Python function decides the reply<br/>offline, instant, deterministic"]
  L --> C["ClaudeLLM<br/>the real model via the anthropic SDK<br/>needs an API key"]
  S --> D["demos and tests<br/>reproduce any failure on purpose"]
  C --> P["production<br/>same agent code, real behaviour"]

In code: ClaudeLLM.complete builds its request with claude_request and turns the API's reply into an LLMResponse. OllamaLLM is a third implementation for a local open model (text only), using ollama_request and parse_ollama_reply.

Because the model is stateless, input tokens are billed per call on the entire history. In an agent loop with calls, where each call adds about new tokens to a history that started at tokens, the total input billed is:

n, h_0, t = 10, 2000, 500
# call i re-sends h_0 and i - 1 steps
calls = [h_0 + (i - 1) * t for i in range(1, n + 1)]
calls[0], calls[-1]  # → (2000, 6500)
# Σ over every call
sum(calls)  # → 42500
# the shortcut on the right agrees
n * h_0 + t * n * (n - 1) // 2  # → 42500

In code: loop_input_tokens lists the tokens billed on each call of such a loop. estimate_tokens is the four-characters-per-token rule of thumb, and conversation_chars measures everything a call resends, which is how ScriptedLLM gives its fake replies realistic Usage.

Answer each one out loud or on paper before you open it. If you can explain it, you know it.