The research
Good ideas need
evidence in context.
Hybrid intelligence means experts and AI working on the same decision, each doing what it does best, by design. Research helps identify promising designs. Testing establishes what they contribute to your work.
Putting people and AI together is a design choice.
The timing of a suggestion, the evidence shown and the judgement required of the expert all change the work. A useful AI output is one part of that arrangement.
Axrea examines the whole decision: what the expert contributes, what the AI contributes and what happens when one relies on the other.
We distinguish a finding from a research study, a design worth trying and a result observed in your own test.
Three questions guide the test.
Does the combination improve the work?
Compare experts alone, AI alone and the combined workflow against criteria agreed for the task.
What does the expert contribute?
Examine reliance on the AI, source checking, reasons for disagreement and the judgement recorded.
What effort does it require?
Count expert time across the task, including checking and rework. Time released is capacity.
Research you can inspect.
- When combinations of humans and AI are usefulVaccaro, Almaatouq and Malone · Nature Human Behaviour · 2024
A systematic review and meta-analysis examining when combining people and AI improves performance. It supports examining the combination rather than assuming it adds value.
- To trust or to thinkBuçinca, Malaya and Gajos · Proceedings of the ACM on Human-Computer Interaction · 2021
Research into cognitive forcing functions and overreliance in AI-assisted decisions. A basis for testing the order in which experts see evidence and suggestions.
- Does the whole exceed its parts?Bansal and colleagues · CHI · 2021
Research on AI explanations and complementary team performance. A reason to examine what an explanation enables the expert to do.
A comparison you can question.
Before a test, we agree the task, case selection, scoring rules and treatment of professional disagreement. We plan who sees each case, in which condition, and how to reduce effects from repeat exposure.
The report distinguishes cases from readings, records model versions and identifies uncertainty. The redesign is tested on held-out cases with the same scoring rules.
Test, redesign, test again. A result can support retaining a design, changing it or testing further. It does not need to support the original idea.