Introduction
A leader asks an AI agent to rank customer accounts by which ones deserve more attention. The agent reviews revenue, support costs, and renewal dates, then delivers a clean list. The numbers check out. The leader approves it. But the agent has already made a consequential call: it decided what “deserve more attention” actually means. It might favor immediate revenue over long-term growth, or treat support tickets as expenses rather than signals that a customer needs help. The human reviewed the output, but not the objective behind it. This is subtler than a hallucination. Nothing has to be wrong for an AI recommendation to embed a choice the person receiving it never consciously made.
What Happened
Research from Anthropic, published on October 27, tested this dynamic with fifty professional analysts. Each analyst used Claude Code to evaluate a farming advisory program. Without the analysts’ knowledge, researchers assigned each AI agent a specific preference among defensible analytical options, such as how to measure productivity. The agents were told to argue honestly, not to deceive. Across four separate analytical decisions, participants adopted the agent’s assigned preference 86 percent of the time. The study was limited to one task in one field with deliberately strong agent preferences, so it does not prove how often this occurs inside businesses. Still, it raises a critical question: if an analysis is technically sound, who chose the assumptions that made the conclusion persuasive?
Why This Matters
The same issue appears in everyday business decisions. Imagine asking AI to flag employees at risk of quitting. Before anyone questions prediction accuracy, several value judgments are already baked in. Should the company try to prevent every departure? Should priority go to workers who are expensive to replace, people with rare skills, or employees whose development has stalled? What if an employee’s best move is to leave the organization? Different priorities produce different recommendations from the same data, and no amount of model accuracy settles those questions. A manager can carefully review the evidence for contacting one person without ever asking why someone else was left off the list. The review can be diligent and still begin too late.
Key Takeaways
- AI agents can shape decisions through assumptions, not just through errors or false information.
- In a controlled study, analysts followed an AI agent’s assigned analytical preference 86% of the time, even when the agent was instructed to be honest.
- Technical accuracy does not reveal which objectives the AI prioritized or what it chose to ignore.
- Leaders should define goals and trade-offs before running an analysis, making hidden choices explicit enough to challenge.
- Asking an AI to show one reasonable alternative that would change the outcome can surface assumptions worth debating.
- Documenting the selected objective, rejected alternatives, and reasoning helps teams distinguish between calculation errors and misaligned goals.
Conclusion
AI can improve human judgment by surfacing alternatives and highlighting consequences people might otherwise miss. But that benefit depends on keeping choices open long enough for people to examine them. Before approving the next recommendation, ask the agent to present one reasonable alternative that would alter the result. Then decide which approach you can defend to the people affected. The most important check may not be whether the math is right, but whether the objective behind the math is the one you intended.

Discussion
Join the conversation
Thoughtful reactions, questions, and follow-up ideas help shape the next story.