11 min read

Agent Loops

A practical guide to the control loop behind AI agents: model turns, tool calls, state, stopping conditions, safety boundaries, and evaluation.

TL;DR

An agent loop repeatedly asks a model what to do next, executes any requested tools, returns the observations, and stops only when a clear terminal condition is reached.
The loop—not the prompt—is the product boundary: it controls what the model can see, what it can change, how long it can run, and when a human must intervene.
Start with one agent and a small toolset. Add planning, handoffs, or multiple agents only when traces show that the simpler loop cannot solve the task reliably.
Define hard budgets for turns, time, tokens, and money, plus semantic stop conditions for success, refusal, blocked work, and required approval.
Treat tool calls as untrusted requests. Validate arguments, authorize every side effect, make writes idempotent, and return compact, actionable errors to the model.
Persist checkpoints around expensive work and external side effects so a retry can resume safely instead of repeating completed actions.
Evaluate full trajectories as well as final answers because an agent can reach a plausible result through unsafe, wasteful, or irreproducible steps.

Quick view

Best for

Open-ended tasks with verifiable progress

Core primitive

Model → action → observation → repeat

Do first

Build one bounded, observable loop

Primary risk

Unbounded or unsafe side effects

What & why

An agent loop is the runtime cycle that lets a language model do more than produce one response. The model receives a goal and the current state, chooses either a tool call or a final answer, observes the result of that choice, and repeats until the work is complete or the runtime stops it.

This feedback cycle is what makes the system agentic. A chatbot usually responds once; an agent can plan, act, inspect what happened, revise its approach, and continue. Anthropic describes the practical pattern as plan, act, observe, adjust, and repeat, while the OpenAI Agents SDK formalizes the same idea as repeated model turns, tool execution, handoffs, and final output [1] [2].

The important engineering insight is that the model is only one part of the agent. The loop owns execution: it assembles context, exposes tools, validates requests, records state, enforces permissions and budgets, and decides whether another turn is allowed.

Use an agent loop when the path cannot be specified reliably in advance but progress can still be observed and checked. If the steps are known, stable, and deterministic, an ordinary workflow is usually cheaper, faster, and easier to test [3].

Mental model

Think of the model as a policy inside a state machine. It proposes the next transition; your application decides whether that transition is valid and performs it. The model never directly sends an email, updates a record, or runs a command. It emits a structured request that the runtime may validate, reject, pause, or execute.

A minimal loop has four phases: orient from the goal and state, choose the next action, observe the action result, and decide whether to continue. Planning can happen inside any model turn; it does not need to be a separate subsystem.

Each iteration should create evidence of progress. A useful tool result changes what the agent knows or what exists in the environment. If repeated turns only restate the plan, retry the same failing call, or expand context without reducing uncertainty, the loop is stuck.

The loop, one turn at a time

01
Assemble the next model input
Provide the goal, relevant instructions, available tool schemas, and the smallest useful view of current state. Include recent observations and durable facts, but avoid replaying every historical detail forever.
Keep application-only data—credentials, database clients, permission objects, and secrets—outside model-visible context.
02
Ask the model for the next action
The response should resolve to a small set of machine-readable outcomes: one or more tool calls, a final result, a request for human input, a refusal, or a declared blocker.
Structured output reduces ambiguity at the boundary, but it does not make the proposed action trustworthy.
03
Validate and authorize
Parse arguments against a strict schema, check resource ownership and current permissions, apply rate and size limits, and determine whether the action requires approval.
Perform authorization at execution time. Instructions such as 'only modify my files' are useful guidance, not a security control.
04
Execute tools and capture observations
Run safe independent reads in parallel when it reduces latency. Serialize dependent operations and consequential writes so their ordering stays explicit.
Return a compact result with stable identifiers, changed state, and recoverable error details. Tool outputs should help the next turn decide, not dump an entire backend response into context [4].
05
Checkpoint and evaluate the stop conditions
Record the new state, tool results, usage, and trace before the next expensive or consequential step. Durable checkpoints make interruption and recovery possible [5].
End on validated success, a clear blocker, a refusal, a required human decision, a canceled run, or an exhausted budget. Otherwise, start the next turn.

A minimal loop in pseudocode

Initialize state from the user goal and assign budgets for turns, time, tokens, and cost.
While the run is active, call the model with instructions, state, recent observations, and tool definitions.
If the model returns a final result, validate the result and finish.
If it requests human input or approval, persist state and pause.
If it requests tools, validate and authorize each call before execution.
Append normalized tool results to state, checkpoint the turn, update budgets, and continue.
If any hard limit or cancellation signal is reached, stop with a typed reason and a resumable state.

Build a production-ready loop

01
Define the state contract
Separate conversation items, task state, runtime metadata, and application context. Give each tool call and external effect a stable ID.
Store facts and artifacts explicitly instead of expecting the transcript to remain a perfect database.
02
Design a small, legible tool surface
Prefer a few tools with distinct jobs, precise names, strict schemas, and concise descriptions. Overlapping tools force the model to guess which contract applies.
Return meaningful context and next-step hints while keeping responses token-efficient; official tool-design guidance emphasizes clear boundaries, composability, and useful results [4].
03
Make failure part of the protocol
Classify failures as invalid input, unauthorized action, transient dependency failure, permanent failure, conflict, or timeout. Tell the model what can change on retry.
Retry transient infrastructure failures in code with backoff. Do not spend model turns rediscovering that a service is temporarily unavailable.
04
Bound autonomy
Set independent ceilings for model turns, wall-clock time, token use, tool calls, and spend. A turn limit alone does not constrain a tool that runs for an hour or returns a million tokens.
OpenAI's runner uses an explicit maximum-turn boundary; production systems should add domain-specific limits and a controlled result for exhausted budgets [2].
05
Gate consequential actions
Require approval for actions with financial, legal, privacy, publication, deletion, or broad communication impact. Show the reviewer the exact tool, arguments, expected effect, and relevant diff.
Pause and resume from serialized state rather than holding a process open. Current agent runtimes expose this interruption pattern directly [5] [6].
06
Make side effects idempotent
Use idempotency keys, conditional writes, upserts, or read-before-write checks. A timeout after a successful remote write must not cause the next retry to duplicate the action.
Keep irreversible effects after validation and approval, and record enough evidence to reconcile ambiguous outcomes.
07
Trace every transition
Capture model calls, tool requests, tool results, state changes, approvals, errors, timing, and usage as one end-to-end run. Redact or exclude sensitive payloads deliberately.
Traces make it possible to distinguish a model mistake from a bad tool contract, stale context, an authorization rejection, or a runtime bug [7].
08
Compact context without erasing state
As runs grow, summarize resolved history, retain recent raw observations, and keep durable facts or artifacts outside the transcript. Measure whether compaction removes information needed later.
Long-running work benefits from tractable chunks and structured handoff artifacts rather than one indefinitely expanding conversation [8].

Choose the right control structure

01
Use a workflow when the path is known
If the steps, branches, and success criteria can be written in code, encode them directly. Models can still fill individual steps without controlling the entire route.
02
Use one agent loop when the path is open-ended
A single model with a focused toolset is the best default for research, troubleshooting, code changes, and other tasks where the next useful action depends on the latest observation.
03
Add an evaluator loop for checkable quality
Generate, evaluate against explicit criteria, feed back actionable defects, and revise until the output passes or the budget ends. Prevent endless polishing by limiting revisions and defining a minimum acceptable score.
04
Add handoffs only for real specialization
Use another agent when it needs different tools, permissions, context, or expertise. A handoff should narrow responsibility, not merely rename another call to the same model.
05
Parallelize independent work, then synthesize
Fan out only when branches do not depend on one another and the benefit exceeds the added tokens and coordination cost. Give the synthesizer provenance and conflict information, not just polished summaries.

Stopping conditions that belong in code

Success: the final output matches its schema and task-specific validation passes.
Blocked: a required input, permission, dependency, or resource is unavailable and the agent cannot make meaningful progress.
Human decision: the next action crosses an approval boundary or requires a judgment the user has not delegated.
Refusal: policy or product rules prohibit the requested action.
No progress: recent turns repeat substantially equivalent actions or fail to change relevant state.
Budget exhausted: the run reaches its maximum turns, time, tokens, tool calls, or cost.
Canceled or superseded: the user or system withdraws the task or replaces it with a newer request.
Runtime failure: state cannot be recovered safely or an external effect has an ambiguous outcome that requires reconciliation.

Evaluate the loop, not just the answer

01
Score the final outcome
Measure whether the requested state actually exists: the test passes, the correct record changed, the cited claim is supported, or the user received the required artifact.
02
Inspect the trajectory
Check tool selection, argument quality, redundant calls, recovery behavior, approval compliance, and whether the agent used trustworthy evidence. A lucky final answer can hide a broken process.
03
Track operational quality
Record completion rate, human takeover rate, turns, latency, token and tool cost, transient failures, repeated actions, and unsafe-action attempts.
04
Build evals from real failure modes
Start with a small set of representative tasks and deterministic graders where possible. Add cases from production traces, especially regressions caused by prompt, model, tool, or context changes.
Agent evaluation should reflect the work itself: Anthropic recommends combining outcome, process, and domain-appropriate quality dimensions rather than relying on one generic score [9].

Common failure modes

Letting the model decide whether it has permission to perform an action.
Using one vague tool that accepts free-form instructions for every backend operation.
Feeding raw, oversized tool responses back into every later turn.
Retrying non-idempotent writes after timeouts without checking whether they already succeeded.
Stopping because the model says 'done' without validating the requested outcome.
Allowing unlimited turns or relying on a single global timeout as the only budget.
Adding planners, critics, routers, and specialist agents before a traced single-agent loop has been tested.
Evaluating only the final prose while ignoring unsafe calls, fabricated observations, and wasted steps.
Persisting complete traces with secrets or personal data by default.

Final checklist

The task genuinely benefits from model-directed next steps instead of a fixed workflow.

Every model response resolves to a typed action, final output, pause, refusal, or blocker.

Tool arguments are schema-validated and authorized outside the model.

Consequential actions show an exact preview and require approval where appropriate.

External writes are idempotent or have an explicit reconciliation strategy.

The run has limits for turns, time, tokens, tool calls, and cost.

Success and no-progress conditions are checked by code, not model confidence alone.

State can be checkpointed, paused, resumed, canceled, and recovered safely.

Tool results are concise, actionable, and safe to place in model context.

Traces cover model calls, tools, state changes, errors, approvals, and usage.

Sensitive data is redacted or excluded from context and observability systems.

Evaluation measures both task outcome and trajectory quality.

Beyond the loop: learning between tasks

Everything above happens inside one task. When the run ends, the loop terminates and its context disappears. A second, slower cycle operates between tasks: after a task completes, the agent distills what worked into a stored, reusable procedure and retrieves it the next time a similar task appears.

That inter-task cycle is a different engineering problem—write paths, retrieval, versioning, and a new security surface—so it gets its own recipe. The agent loop handles a task; the learning loop is what makes the agent better at the next one.

References

[1] Trustworthy agents in practice — Anthropic's current definition of agents as self-directed systems that plan, act, observe, adjust, and repeat.
[2] OpenAI Agents SDK: Running agents — The runner lifecycle, tool-call loop, final outputs, handoffs, turn limits, and recovery behavior.
[3] Building effective agents — A practical distinction between fixed workflows and model-directed agents, with guidance to start simply.
[4] Writing effective tools for agents — Guidance on tool boundaries, descriptions, composability, context, and token-efficient results.
[5] LangGraph interrupts — Checkpointed pause-and-resume patterns for human review and durable agent state.
[6] OpenAI Agents SDK: Human-in-the-loop — Approval rules, interruptions, serialized run state, and resuming sensitive tool calls.
[7] OpenAI Agents SDK: Tracing — End-to-end traces for model generations, tools, handoffs, guardrails, and custom events.
[8] Harness design for long-running application development — Lessons on decomposing long-running work and carrying context through structured artifacts.
[9] Demystifying evals for AI agents — Current guidance for evaluating multi-turn agents, trajectories, and domain-specific outcomes.

More recipes

Subscribe to all Software Cookbook updates

If this recipe was useful, subscribe for the next ones. New pages arrive with the same concise, technical format.