Evaluate overconfidence, underconfidence and actual reliance as separate parts of a professional workflow.
More trust is not the objective
A successful AI trial should not simply make people more confident. It should help them place confidence in the right contributions, for the right reasons. An adviser can have high confidence in a reconciled total while remaining uncertain about the forecast assumption behind it. Treating those as one confidence score hides the important distinction.
Separate confidence from reliance
Confidence concerns what someone believes about a claim or judgement. Reliance concerns what they actually use. A reviewer may express doubt and still accept a recommendation, or express confidence while repeating all the preparation. Evaluation needs both: stated confidence tied to specific questions, and observable decisions about accepting, rejecting, checking or seeking further evidence.
Examine both directions
Overconfidence may turn a tentative assumption into an apparently settled conclusion. Underconfidence may leave sound analysis unused, prompt avoidable escalation or consume capacity through repeated checks. These possibilities should inform the test, but they are not labels to apply merely because someone agrees or disagrees with AI. The reference is the evidence and the task, not the model’s preferred answer.
Define what a good judgement requires
Some claims have a straightforward reference, such as whether the calculation reconciles. Others involve interpretation and reasonable professional disagreement. Work with the firm’s subject experts to define criteria, appropriate actions and the reasoning expected. Record disagreement and the limits of the evidence. Do not manufacture a single correct answer for a question that genuinely permits more than one defensible judgement.
Design a credible comparison
Include reliable and unreliable AI contributions, supported and unresolved assumptions, and cases where the expert’s initial view deserves revision. Use matched cases or suitable groups to limit learning and case recall. Keep model and source versions recorded and reserve held-out cases. If asking for a confidence score changes how people work, that is part of the evaluation design to consider. Choose the sample size around the decision and the uncertainty the firm can accept.
Keep the conclusion proportionate
A useful readout distinguishes better decisions, justified confidence and appropriate use of AI. It reports total review effort and important failures, not only an average score. Model self-reported certainty is not automatically a calibrated probability and needs its own validation where it is used. Our research page explains the underlying distinctions; the finance example shows how they apply to a practical task.
Bring one workflow into focus.
Explore the examples, then tell us where your team wants AI and professional judgement to work better together.
Explore the worked examples ↗