Judgment and AI
AI vs Human Decision Making: Who Should Make Which Call.
AI already beats people at some decisions, and putting a person in the loop does not always help. The real work is deciding who decides what.
I have watched an executive stare at a dashboard full of real-time predictions and hesitate. Not because the data was unclear, but because the decision carried trade-offs and consequences no model could fully price.
In that moment, no system could decide for them. That is the honest starting point: the machine is often better at the prediction, and it still cannot own the call.
The usual answer, "use both", is also less safe than it sounds. Done carelessly, adding a person to an AI decision makes it worse. What follows is what each side does better, what the research actually shows, and a simple test for deciding who should make which call.
How AI and people actually make decisions.
An AI model finds patterns in past data and returns a prediction or a recommendation: this customer will probably leave, this invoice looks like fraud, this candidate resembles your best hires.
It does this fast, at scale and the same way every time. It has no stake in the result, and it cannot tell when the world has changed in a way its data never saw. As I put it in Rewired Leadership, AI delivers the certainty of the past.
People decide differently. We combine a prediction with values, context, politics and the knowledge that we will have to explain ourselves afterward. That is why human judgment in AI-assisted work cannot be removed from some calls, and also why we are slow, inconsistent and prone to bias.
So can AI make decisions? Yes, within limits a person sets, and in many companies it already does: approving routine refunds, routing support tickets, repricing stock. What it cannot do is own a decision.
The useful distinction is between prediction and judgment. AI makes prediction cheap. Deciding what a prediction is worth, which trade-off to accept and who answers for the result is still judgment.
It also matters which kind of AI you mean. Predictive models, such as credit scores, demand forecasts and risk scores, fail by being confidently wrong at scale. Generative AI, which drafts options, documents and summaries, fails by sounding right when it is not. Both need a person who knows the difference.
As work becomes more automated, leadership becomes more human.
AI vs human decision making, side by side.
Set side by side, AI vs human judgment differs on eight factors that matter for real decisions. The evidence behind each row is in the next section.
| Factor | AI | People |
|---|---|---|
| Speed and scale | Thousands of decisions a second at almost no extra cost | One at a time, and slower when tired |
| Consistency | The same input gets the same answer | The same case gets different answers depending on which expert gets it |
| Accuracy on repeated predictions | Usually higher where there is plenty of data and fast feedback | Usually lower, even for experienced experts |
| New or changed situations | Weak, because it can only learn from what already happened | Strong, because people can spot a break from the past and adapt |
| Trade-offs between values | Optimizes the goal it is given | Can weigh goals against each other and question the goal itself |
| Bias | Learns whatever bias sits in historical data, then repeats it at scale | Personal, situational and hard to audit |
| Explaining the decision | Often hard to explain, especially for complex models | Can give reasons, though not always the real ones |
| Accountability | Cannot be held responsible | A named person can answer for the outcome |
What the research says.
Where AI wins: repeated predictions with plenty of data
Where a decision repeats and leaves a record, the newest evidence points one way. In a study published in Science in April 2026, researchers at Harvard and Beth Israel Deaconess tested an OpenAI reasoning model on the records of 76 real emergency-room patients.
Given the same records as two experienced physicians, the model's diagnosis at triage was exact or very close in 67% of cases. The doctors managed 55% and 50%.
It holds even for strategy. In a 2026 working paper, researchers at NYU Stern, Michigan and Indiana ran a live tournament on Kickstarter ventures launched after the AI models' training data ended, so nothing could be looked up.
The best AI model picked the stronger of two ventures nearly four times in five. The combined ranking of 346 experienced managers was statistically no better than chance (Csaszar, Peterson and Wilde, 2026).
People are less consistent than they think
Poland's judicial exam ran the test for real: 232 judicial candidates, most at the end of three years' training, each sentenced the same burglar from the same 104-page court file, with six and a half hours to get it right.
They could not agree on the type of punishment, let alone the amount (Uhl and Pickett, Criminology, December 2025).
That variation is not bias, which pushes every decision the same way. It is noise, which scatters them, and it makes a decision unfair even when it is right on average. Two defendants with the same file get very different sentences depending on who picks it up. A model does not have a bad Monday.
Sentence chosen by 232 judicial candidates for the same burglar
Same file, same law, same offender: one candidate in eleven sent him to prison, and one in twenty fined him.
Source: Uhl and Pickett, Criminology, December 2025
Where people win: change, novelty and values
Models are weakest when the situation stops matching what they learned from. Researchers at Esade, the University of Sydney and NYU Stern put seven core strategy questions to leading language models (Harvard Business Review, March 2026).
The models recommended the same fashionable answers even when the context changed. The authors call it trendslop, and it is the risk in any decision where something has shifted that the model cannot see.
And a model can repeat a bias at a scale no single recruiter could. A Stanford-led team followed 4 million job applications that one vendor's hiring AI scored for about 150 employers (Bommasani and colleagues, May 2026).
Checked job by job, 26% of Black applicants had applied to positions where the tool recommended their group at under four-fifths the rate of the most favored group. Pooled across every job, the same data showed no adverse impact at all.
A person in the loop does not guarantee a better call
This is the finding most writing on the subject leaves out. A team at the Max Planck Institute for Human Development reanalyzed 74 experiments that compared people, AI and the two working together (Berger and colleagues, PNAS, September 2026).
On average, the pair did worse than whichever was better on its own. Only 10 of the 74 studies ever told people whether their calls had been right, and where people got that feedback along with the AI's explanations, the pair tended to beat either one alone.
Medicine shows how wide the gap can get. In a randomized trial with 1,298 members of the public, language models tested alone identified the right condition in 94.9% of cases. People using the same models found the relevant conditions in fewer than 34.5% (Bean and colleagues, Nature Medicine, February 2026).
People who used any source they liked did better. Benchmark scores had not predicted the failure. Only testing with real users exposed it.
The NYU Stern team found the same thing with ventures: once the AI beat the human baseline, blending human and AI predictions stopped helping.
The likely reason is the same everywhere. People rarely learn when to trust the machine and when to trust themselves, because they rarely see how their calls turned out. So they override it when it is right and follow it when it is wrong.
Two ways teams get it wrong.
The first is algorithm aversion: the model makes one visible mistake and the team quietly stops using it, even though its record still beats the people it replaced.
The fix is a bounded override, a set range within which people may adjust the output, rather than a choice between obeying and ignoring. Then let the override log, not the last mistake, settle which of the two to trust.
The second is the opposite: rubber stamping. People accept outputs beyond what the model's track record justifies, and "a human reviews it" turns into a signature. It is most likely when people are busy, the output sounds confident and nobody has ever been asked to explain an override.
Experts are not immune. In a 2026 experiment in PNAS Nexus, more than 1,300 teachers in Greece reviewed a deliberately harsh grade. When it was labeled as AI-generated, their final marks ended up 22% further from the correct grade than when a colleague had given it.
So how should human judgment be applied to AI outputs? Not as a final check on every answer, but to the logic behind it. Did this decision stay inside the boundaries we set? What would have to be true for it to be wrong?
In the book I describe this as a shift from asking "Was this the right result?" to "Was the decision made within the right boundaries?" The same discipline cuts the other way: stop overriding decisions simply because they feel unfamiliar.
Both failures are leadership habits, not technology faults. Which one a team falls into depends mostly on how its leaders react the first time the model gets something wrong.
How to decide who decides: a five-question test.
Decision making with AI works when every recurring decision has an owner and a mode agreed in advance, rather than argued case by case. Five questions settle most of them.
- Has this decision been made many times, with outcomes you can check? If so, it is a prediction problem, and AI should lead. Pricing, fraud flags, demand planning and routing usually qualify. One-off strategic bets do not.
- Is the AI measurably better than your people at it today? Test it on your own past cases before you assume either way. If it is better, keep the human role to exceptions, because a reviewer second-guessing every answer tends to lower accuracy. If it is not, the AI advises and a person decides.
- Has something changed that the data cannot see? A new market, a shock, a first deal of its kind, a regulation that did not exist last year. The model is extrapolating from a world that is gone, so a person decides.
- Does it trade off values, or people's jobs and livelihoods? Layoffs, promotions, prices that hit vulnerable customers. A person decides. AI can supply the options and the assumptions behind each, but the choice and its consequences stay human.
- Who answers for it if it goes wrong, and can they explain it? If nobody can, the decision is not ready to hand over, whatever the accuracy figures say.
The answers sort each decision into one of three modes.
| Mode | Who has the final say | Example |
|---|---|---|
| AI executes within guardrails | The system, inside limits a named person sets and reviews | Repricing stock within an agreed band |
| AI recommends, a person decides | The person, who records any override and the reason for it | Shortlisting candidates for interview |
| A person decides, AI informs and challenges | The person, after AI has tested the assumptions | Restructuring a division or entering a new market |
AI vs human decision making: five questions, three modes
Has this decision been made many times, with outcomes you can check?
If noA person decides, AI informs and challenges
Is the AI measurably better than your people at it today?
If noAI recommends, a person decides
Has something changed that the data cannot see?
If yesA person decides, AI informs and challenges
Does it trade off values, jobs or livelihoods?
If yesA person decides, AI informs and challenges
Is there a named owner who can answer for it and explain it?
If noAI recommends, a person decides
Passes all fiveAI executes within guardrails that a named person sets and reviews
Any one question can keep a decision with people. Only a decision that passes all five is handed to AI.
This is the sorting the Judgment Audit in Rewired Leadership does for a whole organization: every task is mapped to execute (automate), synthesize (augment) or judge (keep for people). Most of the how can be handed over, once the procedure that defines the execute bucket is written down.
In the book I tell leaders to aim to delegate 95 percent of it to their teams and their AI systems, and keep their own attention for the 5 percent of decisions that need ethical and strategic judgment.
The modes only hold when the boundaries are written down. Leaders who already replace approvals with guardrails for their people find this easy, because the same three questions apply. What risk is acceptable without escalating? Which decisions need consultation but not approval? Which outcomes are non-negotiable?
In organizational decision making with AI, the hard part is rarely the model. It is knowing which decisions your leaders are ready to hand to AI, and which ones they will quietly take back the first time something goes wrong.
What this changes for leaders.
In Rewired Leadership I argue that AI will not replace leaders. It will expose them. The one most exposed by its analytical power is the leader I call the Solver, whose value lies in analyzing data and crafting answers.
AI is better at that, so the Solver's job shifts to defining the right problem and building judgment in others. Close behind is the Controller, who insists on approving every decision and becomes the bottleneck AI simply routes around.
Both shifts change who should be next in line, which is why I advise planning succession for the role you will actually need, not the one on today's org chart.
The payoff for leaders who make the shift is leverage, which I describe as the time AI frees up multiplied by the quality of judgment applied to what remains ambiguous. If AI frees 40 percent of your review time and you fill it with more internal meetings, your leverage is close to zero.
Two habits help. First, use AI to widen your thinking rather than confirm it.
Ask it for alternatives your team has not considered, for the hidden assumptions behind each recommendation ("this assumes the competitor will not respond for ninety days"), and for the weak signals that cut against the story everyone already believes.
Second, when a decision proposal reaches you, do not edit the analysis. Ask a question: "What assumption in our model carries the highest risk if proven wrong?"
That is coaching judgment instead of correcting the analysis, and it is how a team learns to decide without you. It is also the difference between a coach's question and a mentor's answer, which matters more now that no mentor has run a team of people and agents.
It also means protecting your own attention for the decisions only you can make, and knowing when to switch the machine off and think without it.
How to tell if it is working.
Most companies measure AI by adoption: how many people use it, and how often. That says nothing about the quality of the decisions. These four measures do.
- Decision cycle time. The time from a signal appearing to a decision being made. The benchmark I use in the book is under 48 hours for 90 percent of delegated decisions.
- Override rate, and who was right. Log every time a person overrules the model, then check the outcome later. Within a few months this tells you, decision by decision, which of the two to trust. Almost nobody keeps this log.
- Delegated decisions that stand. The share of decisions handed down that are not quietly pulled back or overruled by a senior leader. The target in the book's scorecard is above 98 percent.
- Escalation rate. How often decisions reach leaders that did not need them.
Once AI handles the routine work, these belong in the performance review too, and the reviews themselves have to shift to measuring the quality of judgment rather than the volume of activity.
Frequently asked questions.
Yes, for repeated, well-defined decisions inside limits people set, such as routing tickets, approving routine refunds or adjusting prices within a band.
It should not make decisions that trade off values, affect people's livelihoods or depend on a situation its data has never seen. Someone still has to own every decision it makes.
At repeated predictions with plenty of data and fast feedback, usually yes: in 2026 studies, AI models out-diagnosed physicians on real emergency-room records and out-forecast experienced managers on new ventures.
In new situations, sudden change and value trade-offs, people still do better. The useful question is which is better at the specific decision in front of you.
Not automatically. AI is more consistent, so it avoids the random noise that makes human decisions unfair. But it can repeat a bias at scale, and a 2026 Stanford study of a hiring AI found bias that showed up only job by job. Audit the model and the people.
To the logic, not to every answer. Check whether the output stayed within agreed boundaries, what assumptions it rests on and what would make it wrong.
Give people a set range to adjust within rather than a yes or no, log every override, and later check who was right.
It will replace much of the analysis and coordination managers do today, and managers whose value lies there will be exposed.
It will not replace the judgment calls: ambiguous trade-offs, ethical choices and decisions someone has to answer for. Those become a larger share of the job.
The short version.
AI predicts. People decide what a prediction is worth, and answer for it. Neither is better across the board, and putting a person on top of a model is not a safeguard by default. Decide who decides before the model goes live, write down the boundaries, then keep score of who was right.