Introduction

Artificial intelligence systems often output confident results without leaving a trail of proof. A new auditing framework called ClaimAudit changes that by demanding verifiable evidence at every step, transforming opaque outputs into checkable records.

What Happened

The framework processes any question through five sequential stages. A planner decomposes the query into atomic claim types, labeling each by shape such as balance, transaction, or TVL. A retriever then fetches the primary on-chain data, like an eth_call balanceOf or an eth_getTransactionReceipt, recording the endpoint and response hash. A verifier compares the returned data against the original claim, checking pins and cryptographic hashes. An adjudicator assigns one of four possible verdicts, and the writer outputs only the details that survived scrutiny. A live audit of a BNB Chain treasury buyback showed the system in action: most figures aligned, but two small residual outflows triggered a BLOCKED verdict, prompting the operator to address the discrepancy.

Why This Matters

Without a pinned record and cryptographic anchoring, a balance claim becomes impossible to defend once the underlying data shifts. The framework makes a critical distinction: a search engine returns the closest number it can find, while an auditor only validates what it can defensibly defend. When no primary source exists, the system returns BLOCKED, refusing to manufacture confidence. This matters for any AI operating in finance, governance, or supply chains where accountability cannot be optional.

Key Takeaways

  • The audit pipeline consists of five stages: Planner, Retriever, Verifier, Adjudicator, and Writer recorded in an append-only JSONL ledger called H-MEM.
  • Every claim must root itself in a primary source: a transaction receipt, an RPC call, or an official API response.
  • Evidence must be pinned, hashed with SHA-256, and measured against a strict rule; vague judgments such as it looks consistent are automatically rejected.
  • The BLOCKED verdict appears when no verifiable primary source exists, preventing false confidence from slipping through.
  • The open-source repository provides deterministic fixtures, a three-pane user interface, and a self-test suite ensuring reproducibility.

Conclusion

Verifiable AI does not claim to establish absolute truth. It produces records that a stranger can replay, pin, and hash. The next generation of AI infrastructure will be judged by whether others can independently check the work, not merely by how often the system happens to be right. Build for the second test.