Introduction
Most conversations about trustworthy AI focus on the model itself—attested hardware, reproducible weights, and proofs of inference. But the AI output people actually act on usually arrives as a claim inside a document, a README, or a configuration file. That distinction matters because a configuration error can cause far more damage than a misleading sentence.
I learned this the hard way after publishing a detailed guide to Claude Code and a companion repository of the configurations I use daily. I assumed the guide was accurate. A rigorous re-examination proved otherwise.
What Happened
I reviewed every claim in the guide twice. The first pass checked it against Claude Code version 2.1.234 in August. The second pass, completed this month, tested it against version 2.1.311 using four Claude Code agents working in parallel. I expected to find outdated facts. Instead, I uncovered ten statements that were incorrect even on the day I originally wrote them.
Several errors stood out. I described a "Delegate Mode" triggered by Shift+Tab, but that feature never exists; the key combination cycles permission modes. I claimed imported files defer loading, when they are actually expanded into context at launch. I listed an eight-layer memory hierarchy, missing the managed-policy CLAUDE.md layer entirely, bringing the count to ten. I also advised keeping CLAUDE.md under 500 lines, though published guidance recommends staying under 200.
Other false claims involved version numbers, percentages, and limits that sounded precise but had no source behind them. For example, I stated that auto-compaction triggers at roughly 95 percent of context. There is no fixed percentage; the system compacts at the model's context limit. I also wrote that a skill costs about one line of context at startup, when the real cost includes the skill's description and usage text, capped at 1,536 characters per skill.
Broken hooks illustrated the same problem. A Prettier hook piped file paths through xargs, which splits on spaces, so it failed in a directory named "Ben Ryan". A destructive-command guard blocked git push --force-with-lease but missed rm -fr / and rm -r -f /. The fixed version now accepts test JSON through stdin: echo '{"tool_input":{"command":"rm -rf ~"}}' | .claude/hooks/block-destructive.sh
The audit also exposed a failure in my own verification pipeline. An automated fact-checker had "confirmed" two statistics by surfacing my own website, then quoting that same page back as evidence. Circular verification does not count as verification.
Why This Matters
These errors were fluent, specific, and plausible—the exact kind of output language models generate easily and agents repeat confidently. A reader cannot check them without an independent source. This reveals a broader problem: verifiable AI cannot remain a property of the model alone. It must be a property of the output, and that output must be open enough for someone else to audit.
Version specificity creates additional risk. Official documentation for the API and official documentation for Claude Code can disagree about the same model. One surface may describe default effort as high while another describes it as medium. Both can be correct in their own context, but an agent reading only one source will turn a true claim into a false one.
Environment also matters. I left out a detail about Agent Teams because the experimental flag had lived in my own settings file for so long that I stopped noticing it. Every pattern worked on my machine and failed everywhere else. Documentation cannot catch missing environment context; only re-running the instructions in a clean environment can.
Key Takeaways
- Every numeric claim needs a traceable source, or it should be rewritten as a range or removed entirely.
- Verification must be independent. A checker cannot cite the article it is auditing.
- "Unknown" is a legitimate verdict. "I could not find it" is not the same as "it is correct."
- Source provenance must specify which system and version a document describes.
- Reproducibility requires running instructions in an environment that is not your own machine.
- Public correction logs provide the cheapest and most transparent audit trail for readers.
Conclusion
The agents performed most of the checking in the second pass, but "an AI checked it" carries no weight by itself. What made the results trustworthy was a trail any reader could follow: a source for every claim, a version for every source, checks that re-run rather than re-read, and evidence that stands apart from the claim itself.
The guide has been updated for Claude Code version 2.1.311, and the companion repository contains the corrected configurations, fixed hooks, and a changelog documenting every correction from both review passes. If you spot something I missed, the repository is open for issues. Verifiable AI is a practice, not a model feature.

Discussion
Join the conversation
Thoughtful reactions, questions, and follow-up ideas help shape the next story.