A state budget for production AI agents

The easiest way to make an AI agent look impressive in a demo is to give it a long context window, a few tools, and no memory policy. The easiest way to make that same agent painful in production is to let every turn, tool result, approval, and summary accumulate without a budget. State is not just context. It is product behavior, operational cost, privacy surface, and debugging evidence.

The checklist

I like to force an agent design through five questions before I worry about more models or more tools.

State type	Keep it when	Drop it when
Conversation history	The user expects continuity	It is only scaffolding for one task
Tool results	They explain a decision or artifact	They are reproducible from source systems
Checkpoints	A run may pause, fail, or need review	The workflow is a single short call
Long-term memory	It captures stable user or project preference	It is a one-off instruction
Compacted summaries	The raw trace is too expensive to carry	The exact wording or evidence matters

That table is deliberately boring. Most agent incidents I have seen are not caused by a missing orchestration framework. They come from storing the wrong thing, replaying stale state, or compacting away the line that would have explained why the agent acted.

Separate session memory from workflow checkpoints

OpenAI's Agents SDK sessions are useful because they remove the manual stitching between turns. A session can fetch previous conversation items, add the new input, and persist the run output for the next call. That is the right abstraction for conversational continuity.

A checkpoint is different. LangGraph's persistence docs frame checkpoints as graph state snapshots that support human review, failure recovery, time travel, and thread-scoped memory. That makes checkpoints operational state, not just chat history.

The practical rule: session memory answers, "what should the agent know next turn?" Checkpoints answer, "where can this workflow resume or be inspected?" Mixing those two usually creates a blob nobody wants to own. A support bot can keep session memory. A refund agent that pauses for approval needs checkpoints. A coding agent doing both needs both, with different retention and audit rules.

Put a hard ceiling on the loop

Loop control is a state budget too. The Vercel AI SDK documents stopWhen and prepareStep as controls for when an agent loop stops and how each step can change its model, tools, or messages. OpenAI's run configuration has a similar operational concern with turn limits, guardrails, tool execution settings, and session behavior.

I treat these as production limits, not developer convenience flags.

A small policy is enough for most systems:

1. Cap total turns per run.
2. Cap tool concurrency separately from model-side parallel tool calls.
3. Remove risky tools after the planning step.
4. Stop when the latest tool result already contains the requested artifact.
5. Escalate to a human when the run wants more budget.

That last rule matters. If an agent burns through ten steps trying to classify a ticket, the next step should not be step eleven. It should be a different workflow.

Compact only after deciding what must survive

Compaction is tempting because it makes state cheaper. The OpenAI Agents SDK sessions docs describe automatic Responses history compaction, including a warning that compaction can delay streaming completion if it runs at the wrong time. Claude Code's memory docs make a related point from the coding-agent side: persistent instruction files and auto memory are loaded differently, and only concise startup memory is available by default.

The engineering lesson is simple: compaction is a lossy product decision unless you define the invariant first.

For a customer-support agent, the invariant might be: preserve the user's confirmed order number, the final answer, unresolved promises, and every external write. For a code agent, preserve the requested scope, files changed, commands run, failing test output, and explicit user constraints. Everything else can be summarized or discarded.

I would rather ship a crude compaction policy with clear invariants than a clever summarizer that tries to preserve "important context" in the abstract. Important to whom? For what future action? With what retention limit?

Store tool results like evidence, not chat filler

Tool results are the part of agent state most teams under-specify. They can be huge, sensitive, and stale within minutes. They are also the best evidence when something goes wrong.

My default is:

1. Store the tool name, arguments, result digest, external record id, timestamp, and approval id.
2. Store full output only when replay would be impossible or legally important.
3. Prefer a pointer to the source system when the data can be fetched again.
4. Redact secrets before the model sees the result and before it lands in storage.
5. Attach tool state to the workflow checkpoint, not only to the conversation transcript.

This pairs well with an agent telemetry contract. Telemetry tells you what happened. State tells you what can safely happen next.

Decide what a human is approving

Human review is weaker when the state is unclear. If the approval screen shows only the model's proposed action, the reviewer is guessing. The approval should include the relevant checkpoint, the exact tool arguments, the source facts the agent used, and what will be persisted after approval.

That is the state-budget version of human handoff gates for long-running AI agents. A human does not need every token. They need the minimum state required to make the decision and reconstruct it later.

The shape I would ship first

For a production agent, I would start with a plain state contract:

1. session_id for user-facing continuity.
2. run_id for one workflow attempt.
3. checkpoint_id for resumable steps.
4. approval_id for human decisions.
5. tool_call_id for external side effects.
6. memory_scope for user, workspace, account, or global state.
7. retention_until for data that should not live forever.

No custom framework is required to name these fields. The value is the discipline. Once every stored item has a scope, owner, retention rule, and replay story, the agent becomes easier to operate.

The next wave of agent work will not be won by agents that remember everything. It will be won by systems that remember the right small set of facts, can resume from the right point, and can explain why stale state was ignored.