Multi-step planning

What follows measures the arithmetic, then builds the three remedies one mechanism at a time.

An agent that does one thing is easy. An agent that does twenty things in a row mostly fails, for a reason that is pure arithmetic. This lesson measures that arithmetic, then builds the three remedies: decompose the task into steps you can check, verify each step with something outside the model, and checkpoint so a failure costs one step instead of the whole run.

Everyday picture A relay race with ten runners. Each runner drops the baton only 1 time in 20 during their leg. That sounds safe, but the team needs all ten legs to go cleanly, and it loses about 4 races in 10.

Tiny worked example Each step of an agent succeeds 95% of the time.

On the example: and . A 95%-reliable step is a 60%-reliable 10-step task.

In Python:

p = 0.95
# ten steps that must all succeed
round(p ** 10, 2)  # → 0.6
round(p ** 20, 2)  # → 0.36

In code: end_to_end_success multiplies the per-step rate by itself once per step, which is the whole formula above.

Why it matters The remedies all attack or : fewer steps (higher-level tools, see primer.agents.tools), checks that catch failures, retries of just the failed step, and human review at critical points.

Everyday picture Cooking from a recipe. You read the whole recipe first (the plan), then do the steps. Halfway through you discover you're out of eggs. You don't start over, and you don't pretend you have eggs. You revise the rest of the recipe with what you've got, keeping everything you've already chopped.

Tiny worked example Goal "Reconcile Q3 invoices". The planner's first plan uses an old API. Here's the actual run:

planner  -> {"steps": ["fetch_payments", "fetch_invoices_v1", "match", "draft_summary"]}
execute  fetch_payments      ok
execute  fetch_invoices_v1   failed: invoices API v1 was retired on 2026-07-01; use fetch_invoices_v2
planner  <- "Completed so far: fetch_payments. Step fetch_invoices_v1 failed: ...retired..."
planner  -> {"steps": ["fetch_invoices_v2", "match", "draft_summary"]}
execute  fetch_invoices_v2   ok
execute  match               ok
execute  draft_summary       ok     (fetch_payments was NOT run again)

flowchart TD
  G[Goal] --> P[Planner writes a plan]
  P --> X[Execute next step]
  X --> C{Step result<br/>meets expectations?}
  C -->|yes| M{More steps?}
  M -->|yes| X
  M -->|no| D[Done]
  C -->|no| R{Replans left?}
  R -->|yes| F[Tell the planner:<br/>what's done + why it failed]
  F --> P
  R -->|no| S[Stop: report the failure]

The code plan_and_execute(planner, goal) asks for a JSON plan, runs each action, records outputs in state, and on StepFailed asks for a revised plan. Steps already in state are skipped.

In code: PlanRun is what plan_and_execute returns: every plan the planner wrote, each executed step with "ok" or its failure, and the replan count.

Why it matters Deciding one step at a time (a pure ReAct loop, see primer.agents.agent_loop) drifts on long tasks. An upfront plan keeps the agent on track and lets a person see its intent before it acts. The risk is following a plan that reality has invalidated, which is why replanning exists.

Everyday picture Moving house. "Move house" isn't something you can check off, but "every box labelled", "van booked for Saturday" and "keys handed back" are. Each has a clear definition of done.

Tiny worked example "Reconcile Q3 invoices" becomes five subtasks, each with a check:

When the payments API times out once, only fetch_payments runs again: attempts are {fetch_invoices: 1, fetch_payments: 2, match: 1, ...}.

flowchart LR
  A[fetch_invoices] --> C[match]
  B[fetch_payments] --> C
  C --> D[list_mismatches]
  D --> E[draft_summary]
  B -. timeout .-> B

In code: Subtask pairs a step's work with its check (the definition of done); run_subtasks runs them in order, retries only the step whose check failed, and returns a RunReport of outputs and attempts. reconcile_q3 builds the five subtasks in the table.

Checkpoints: how much work a failure costs

Without checkpoints, a failure anywhere means starting again from step one. With them, you retry just the failed step. The expected number of step executions to finish an -step task:

On the example: , : with checkpoints step runs; restarting from scratch . At it's about 52.6 vs. 240.

In Python:

def with_checkpoints(n, p):
    # each step needs 1/p attempts on average
    return n / p
def restart_from_scratch(n, p):
    return (1 - p ** n) / ((1 - p) * p ** n)
round(with_checkpoints(10, 0.95), 1), round(restart_from_scratch(10, 0.95), 1)  # → (10.5, 13.4)
round(with_checkpoints(50, 0.95), 1), round(restart_from_scratch(50, 0.95))  # → (52.6, 240)

In code: expected_step_runs evaluates both formulas: with checkpoints, the restart formula without.

Everyday picture Proofreading your own essay versus having someone run the numbers in it. You read what you meant to write, so your own blind spots stay blind. A calculator doesn't share them.

Tiny worked example The model writes a leap-year function:

def is_leap(year):
    return year % 4 == 0        # forgets that 1900 was not a leap year

Reflection (asking a model to review it) replies "Looks correct: years divisible by 4 are leap years." External verification (running it against known answers) replies is_leap(1900) returned True, expected False. Fed that failure, the model's second draft passes every case.

flowchart LR
  G[Model writes code] --> R1[Reflection:<br/>model reads its own code]
  R1 -->|same blind spot| A1[Approved, still wrong]
  G --> V[Verification:<br/>run tests / check schema / query the DB]
  V -->|1900 fails| F[Failure fed back]
  F --> G2[Second draft]
  G2 --> V2[Checks pass]

In code: self_review is the top path (a model reads the code and approves or not); run_checks is the bottom path (it runs the code against known answers); generate_until_checks_pass loops the bottom path, feeding each failure back to the model until the checks pass.

With verification and retries, the per-step success rate rises:

On the example: , perfect check , one retry: per step, so ten steps succeed of the time instead of 60%. With a check that catches only half the failures (): .

In Python:

def p_step(p, d, r):
    # p Σ_{i=0}^{r} ((1 - p) d)^i
    return p * sum(((1 - p) * d) ** i for i in range(r + 1))
# a perfect check, one retry
round(p_step(0.95, 1, 1), 4)  # → 0.9975
# ten such steps
round(p_step(0.95, 1, 1) ** 10, 3)  # → 0.975
# a check that catches half the failures
round(p_step(0.95, 0.5, 1), 5)  # → 0.97375
round(p_step(0.95, 0.5, 1) ** 10, 2)  # → 0.77

In code: step_success computes from , and ; feed its result into end_to_end_success to get the ten-step curves.

Answer each one out loud or on paper before you open it. If you can explain it, you know it.