Updated 2026-10-13
from the news desk
How can you verify what an AI agent did? New paper says name the records first
A 5 October 2026 preprint by Yue Zhao and co-authors says claims about AI agents are only checkable if they specify the policy, scope, records, record-writers and decision rule in advance. An 8 October 2026 ICO notice points in a related…
Yue Zhao and co-authors argue that you can verify what an AI agent did only if the claim names the policy, scope, records, record-writers and decision rule in advance. Their 5 October 2026 preprint says independent records matter because signed logs can still miss omitted actions, and an 8 October 2026 ICO notice says safeguards become more critical as AI systems operate with greater autonomy.
What the new paper argues
In a preprint submitted on 5 October 2026, Yue Zhao, Jiate Li, Li Li, Yi Nian, Jinbo Liu, Xiaolin Zhou and Xiyang Hu argue that operators often make broad statements about agent systems without saying how an outsider could test them later. The paper opens with the kinds of assurances organizations already make: “Organizations make claims about their AI agents: a person approves every external email, every action is logged, an evaluation shows the agent is safe to deploy.”
The authors' answer is procedural. Yue Zhao and co-authors say a claim becomes auditable only when its test is specified before anyone reads the evidence. Their definition states: “A claim about an AI agent is auditable when four things are specified: policy and version, actions and period covered, required records obtainable from named writers, and the decision rule.” In plain English, that means a verifier needs to know what rule applies, what actions and time period are covered, which records would settle the issue, who produces those records, and how the verdict will be reached.
How the authors say you can verify an agent's actions
The paper says verification starts by narrowing the claim. In its worked example, Yue Zhao and co-authors take a broad statement about human approval and rewrite it into a specific March 2026 claim about external emails with attachments, a named policy version, and three records written by three different parties. The paper says that structure matters because one authenticated in-scope violation can contradict the claim, while missing evidence can block any firm answer.
The paper's verdict ladder has three outcomes: supported, contradicted, and undecidable. Yue Zhao and co-authors note that even a well-framed claim can still fail to yield a yes-or-no result: “Even a claim that meets it can be UNDECIDABLE when a required record is lost.” In the paper's teaching example, an email logged by the mail gateway but unsupported by any matching authorization-server grant contradicted the claim, while a later lost server log made another email undecidable. The paper also states that its records are constructed for teaching and that no agent was run.
Why the paper says signed logs are not enough
Yue Zhao and co-authors say agents add three extra conditions to ordinary audit practice: independent coverage of actions, authorization tied to each action's exact arguments, and completeness beyond integrity. The paper summarizes the point this way: “Agents add three conditions: coverage by an independent record, authorization bound to each action's arguments, and completeness beyond integrity.” The distinction is important because a record can be authentic without being complete.
On the authors' model, a signed or hash-chained log can authenticate the entries supplied, but it cannot by itself prove that nothing was omitted. Yue Zhao and co-authors therefore argue that support for a strong claim needs a second record the agent does not write, such as a gateway or authorization server. The paper adds that its claim-check table is anchored in current NIST, IETF and OWASP drafts, while noting that the documents the authors reviewed did not specify, for a particular claim, which records would settle it.
Why this matters beyond one audit method
One reading is that the paper offers a concrete answer to a recurring governance problem: when an operator says an agent was supervised or safe, what would let a buyer, auditor or regulator check that later? Yue Zhao and co-authors say their verdicts do not establish human intent or legal responsibility. That leaves separate questions discussed in this site's briefings on whether AI can be a legal agent and whether AI can be morally responsible. The narrower claim here is that verification should turn on named evidence rather than broad assurances.
A stronger reading would be that organizations should already be preparing to defend agent claims this way. An ICO notice dated 8 October 2026 does not adopt Yue Zhao and co-authors' framework, but it points to a similar accountability concern as UK scrutiny extends to agentic AI. Richard Nevinson, the ICO's Director of Technology Regulation, said: “But as AI systems operate with greater autonomy, robust data protection safeguards become even more critical.” He also said: “Our message is clear: the fact AI agents act with autonomy is not an excuse for poor compliance.”
A skeptical reading is available too. Yue Zhao and co-authors say the chapter proposes no standard, did not test the method, and relies in part on draft anchors that are still moving. On that reading, the paper is best understood as a framework for making claims checkable, not as proof that current deployments already generate all the records needed to verify them. Readers comparing public documentation such as system cards can treat the paper as a checklist for what those documents would need before a claim becomes independently testable.
Frequently asked questions
What does the paper say you need before a claim about an AI agent is auditable?
Yue Zhao and co-authors say four things must be specified first: the policy and version, the actions and period covered, the required records obtainable from named writers, and the decision rule. The paper says those elements must be fixed before any verdict.
Can a signed log prove everything an AI agent did?
Yue Zhao and co-authors say no. Their paper argues that signatures and hash chains can help show that supplied entries were not altered, but they do not by themselves show that no action was omitted. For support, the paper says a verifier needs coverage from an independent record the agent does not write.
Does a human confirmation in the interface count as approval?
Not necessarily. In the paper's worked case, Yue Zhao and co-authors say a local interface confirmation did not settle the question because the relevant policy required a prior user-approved grant from an authorization server covering the action's arguments. The paper says the confirmation recorded an interface event, not by itself a verifiable authorization.
Does the paper create a standard or a legal rule?
No. Yue Zhao and co-authors state that the chapter proposes no standard, and they distinguish their audit verdicts from separate judgments about legal compliance and responsibility. The ICO notice is likewise a regulatory update and call for evidence, not an adoption of the paper's framework.
Caveats: Most of this briefing rests on a single 5 October 2026 arXiv preprint by Yue Zhao and co-authors. The paper says its worked case uses constructed teaching records, proposes no standard, and did not test the method; the ICO notice is a separate UK regulatory development, not an endorsement of the paper.