The verification tax: why accurate AI can still cost experts time
A system can be right most of the time and still slow its users down. The cost is not in the answer. It is in checking it.
A credit reviewer opens her queue on a Monday morning. The first file carries a recommendation from the new decision-support system: approve, with a limit. The system is accurate. Nobody on her team disputes that.
She still cannot sign. Her name goes on the decision, and the recommendation does not say why it was made. So she does what any careful person would do. She opens the application, rechecks the income documents, looks at the existing exposure and rebuilds the reasoning for herself. At the end she agrees with the system. The recommendation was right. It saved her almost nothing.
Accuracy is not time saved
We call this the verification tax: the time and attention an expert spends to make an answer safe enough to sign.
Accuracy is a property of the model. Time saved is a property of the work. Between the two sits verification, and accuracy alone does not shrink it. The reviewer does not know which answers are the wrong ones. Even if wrong answers are rare, they look exactly like the right ones. So she must either check them all or accept a risk she cannot see.
Accuracy tells you how often the answer is right. It does not tell you which answers to check.
Three ways people respond
Faced with answers that arrive without reasons, people settle into one of three habits.
- They check everything. The tax is paid in full. The system adds a step instead of removing one.
- They trust everything. The tax disappears and the risk grows quietly. Researchers call this automation bias. It is hard to see from the outside, because the decisions look fine until they do not.
- They stop using it. Often without saying so. The system stays in the contract and leaves the workflow.
None of these is a failure of character. Each is a reasonable response to an answer with no reasons attached.
Reasons change the work
Take a second scenario. A ward administrator receives a proposed bed plan for the afternoon: which patients move, which beds are cleaned and reassigned, and in what order.
The plan is probably good. But she knows things the system cannot. A family arrives at two. A side room is held for a patient who may need it. With the plan alone she has two choices: accept it, or redo it. With the reasons she has a third. The plan shows which constraints it applied, what it assumed and what it could not check. She reads the steps and finds the one assumption that is wrong: the side room is marked as free. She corrects it, the plan changes, and she signs.
She checked what mattered and nothing else. That is what lowers the tax. Not a cleverer answer, but an answer shaped for checking.
What “shaped for checking” asks of a system
- Reasons in the expert’s own terms. The steps use the vocabulary and the order a reviewer already thinks in, not the internals of the model.
- Visible foundations. The answer shows the evidence and the rules it rests on, so a faulty premise can be found in seconds instead of rebuilt from scratch.
- Honest confidence. The system says where it is sure and where it is not, so attention goes to the weak cases. This works only if the confidence is calibrated. A system that is always sure teaches people to ignore it.
- A decision that stays human. The system proposes. The accountable person decides, and her correction is information the system can learn from.
Those four describe one direction, from system to person. The tax has a mirror image. The reviewer carries intent the system cannot see: this client is on a watch list, this quarter’s policy has an exception, the case is urgent. If that intent never reaches the system, it answers a slightly different question and she repairs the answer by hand. That repair is a tax too. So the link must run both ways: reasons travel to the person, and intent travels back, as goals, constraints and priorities the system can hold.
Measure the team, not the model
If the tax is real, the usual scorecard misses it. A model can lead an accuracy benchmark and still leave its users slower, or worse, over-trusting. So judge the system by the decisions of the team. How often is the decision right? Does reliance match reliability: trust when the system is right, questions when it is wrong? How long does the decision take?
A system that is accurate but expensive to verify has not yet done its job.
The verification tax is not a law of nature. It is a design problem. At NeuroLink AI it is the problem we work on: AI that gives its reasons in a form an expert can check, and takes in what she intends, so the time it saves is time she actually gets back.