Tool calls are where an AI system stops being a chat interface and starts changing state. A model can ask to create a ticket, charge a card, send an email, update a CRM field, or kick off a deploy. The boring question I now ask before wiring any of those actions is simple: what happens if this exact call runs twice?
The contract I want before the first mutating tool ships
My minimum contract is not a framework. It is one required field and one small table of behavior.
Every mutating tool should accept an idempotency key generated by the client, runner, or orchestration layer. The key should identify the intent, not the transport attempt. If the network drops after the handler starts, the retry uses the same key. If the user changes the instruction, the system creates a new key.
Stripe's API docs are still the cleanest reference pattern here. They use an idempotency key so a retried create or update request does not accidentally perform the same operation twice, and they compare later parameters against the original request to catch misuse. That pattern matters more for agents than it does for a normal form submit because agents can retry, branch, resume, and call several tools in one turn.
For AI tool calls, I want the contract to say four things:
| Case | Handler behavior |
|---|---|
| First valid key | Execute once and persist the result |
| Same key, same parameters | Return the original result |
| Same key, different parameters | Reject as a caller bug |
| Validation failed before execution | Do not save a successful result |
That table is enough for most product tools.
Why tool schemas are not enough
OpenAI's Agents SDK and Anthropic's tool-use docs both make the same important split: the model can produce structured tool calls, but your runtime still executes many of them. Anthropic calls out client tools as code that runs in your application, while server tools run on the provider side. OpenAI separates hosted tools, local runtime tools, function tools, agents as tools, MCP, and sandbox capabilities.
Structured inputs reduce ambiguity. They do not make side effects safe.
A schema can say invoice_id is required. It cannot prove that the handler will not send two reminder emails if the model repeats itself. It cannot tell whether a resumed agent run is continuing the same user intent or starting a changed one.
That is why I treat idempotency as part of the tool's public API, not as retry glue hidden in the HTTP client. The runtime should issue the key, log it, and pass it through every mutating boundary.
Where I draw the boundary
I use three buckets.
Read-only tools do not need idempotency keys. Search, retrieval, metadata lookups, and dry-run validators can stay simple. They still need timeouts and result validation, but replay is not dangerous in the same way.
Externally visible mutations need keys. Email sends, billing actions, ticket creation, comment posting, database writes, deployment triggers, calendar edits, and permission changes all qualify. If a user would notice two copies, the tool needs a key.
Long-running operations need keys plus state. A tool that starts a background job should store the key beside the job record. A retry should return the existing job id rather than starting another worker. If the job has already finished, the retry should return the final status.
The runner has to respect concurrency
The tricky failure mode is not only network retry. It is parallelism.
OpenAI's Agents SDK exposes two separate controls worth keeping straight: whether the model may emit parallel tool calls, and how many local function tools the SDK executes concurrently. Anthropic's docs show a similar need to control parallel tool use when the caller wants one tool call per turn.
If two emitted calls carry the same idempotency key, the storage layer has to serialize them. I do not rely on process memory for that. The lazy, reliable version is a unique database constraint on the key scoped to the tool name or tenant, plus a stored parameter hash. The first handler creates the row. The loser reads the row and returns the saved result, waits, or fails with a retryable conflict.
The storage model I usually want is boring:
create unique index tool_runs_key_idx
on tool_runs (tenant_id, tool_name, idempotency_key);
Then the handler stores parameter_hash, status, result_json, error_json, and timestamps.
MCP tools need the same product rule
The Model Context Protocol tool spec already tells servers to validate inputs, enforce access control, rate limit invocations, and sanitize outputs. It also tells clients to log tool usage and distinguishes protocol errors from tool execution errors. That is the right baseline, but I add one product rule for any MCP server that exposes mutations: define replay behavior in the tool description and schema.
For example, a create_invoice MCP tool should not only describe invoice fields. It should require an idempotency key or a stable client operation id. The server should reject reused keys with different input. The client should display sensitive calls for approval, but approval alone is not dedupe.
My review checklist
When I review a mutating AI tool, I ask these questions before prompt wording:
- Does the tool mutate external state?
- Who generates the idempotency key?
- Is the key scoped by tenant, user, and tool name?
- Are parameters hashed and compared on replay?
- What result is returned when the first execution is still running?
- Are validation failures kept separate from started executions?
- Does telemetry include the key without leaking personal data?
- Can support staff search for the key during an incident?
That last point connects directly to the telemetry contract I use for production agents. If a tool call creates a customer-visible side effect, the key belongs in trace attributes, logs, and admin history. It is also a good companion to the sandbox manifest contract for coding agents, because both force agent side effects into reviewable artifacts.
The takeaway for the next agent feature
I do not want a large agent platform before the first useful workflow ships. I do want every mutating tool to answer replay, retry, and resume questions before production traffic touches it. Idempotency keys are an old API trick, which is exactly why they fit here. The agent stack is new enough. The side-effect contract should be boring on purpose.