How to verify what an AI agent actually did

Why “the agent said it worked” is not enough

When an AI agent calls a tool, three different things can be true at once: the agent believes it succeeded, the API returned a success code, and the business system is in some state. Only the third one matters, and it is the one most agent stacks never check.

Real systems disagree with their own responses more often than people expect. A write can be acknowledged and then dropped by a background job. A totals calculation can round differently than the caller assumed. A response can be lost after the change was already applied. An agent can also misread a response and report success on a partial result.

Independent verification — Confirming the outcome of an action by reading the target system through a separate request after the action, rather than relying on the action’s own response or the agent’s report.

Step 1: Write down the postconditions before the action runs

A postcondition is a statement that must be true about the system after a correct action. Define them per tool, not per prompt, so every call is checked the same way.

• Create a deal → a deal with the returned id exists, and its amount and name equal the requested values.
• Create an invoice → the invoice exists, the sum of line amounts equals the total within a tolerance, and its status is draft.
• Post an invoice → its status moved from draft to posted (capture the “before” value first).
• Delete a record → reading it now returns not found.

Postconditions should be deterministic: given the same system state they always give the same answer. That is what makes them evidence.

Step 2: Capture preconditions where the change matters

Some checks only make sense relative to the state before the action. A status transition check needs the old status; an idempotency check needs the old count. Read and store those values immediately before execution, then compare after.

Step 3: Read the system again — through a separate request

After the action, issue fresh read requests for each postcondition. Do not reuse the write response body as the evidence; use it only to learn identifiers such as the new record id. If the identifier itself is wrong, the read-back will fail, which is exactly what you want.

{
  "type": "arithmetic_invariant",
  "read": { "path": "/invoices/{{result.id}}" },
  "terms": [{ "path": "lines[].amount" }],
  "equals": { "path": "total" },
  "tolerance": 0.005
}

Step 4: Report honest outcomes

Every check should end in one of four states, and the overall outcome should be derived conservatively:

• passed — the read succeeded and the condition holds.
• failed — the read succeeded and the condition does not hold (including “record not found”).
• inconclusive — the read failed for transport reasons or the data was not interpretable. This is not a pass.
• not run — the check needs a human, or no checker was available.

A sensible rule: any required failure means the action failed; any required inconclusive or not-run check means a human must review it; the action is “verified” only if every required check passed and at least one of them was deterministic or human-attested.

Step 5: Keep the evidence

Store what was read, what was expected, the result and a digest of the evidence alongside the action. Hash-chain these records so later edits are detectable. This turns “I think it worked” into something an auditor, a customer or a future you can check.

Common mistakes

• Treating HTTP 200 as success without reading back.
• Letting an LLM judge success from the response text.
• Converting timeouts or 5xx errors into “probably fine”.
• Verifying with the same credentials and code path that may have caused the problem without a separate read.
• Only verifying in tests, not on every production action.