Should You Actually Add AI To That? A Four-Question Audit For Product Teams

本文由「Should You Actually Add AI To That? A Four-Question Audit For Product Teams」的公开内容整理，完整内容与后续更新请点击文末链接阅读原文。

17 min readAI, UX, DesignShare on Twitter, LinkedInAbout The AuthorVictor Yocco, PhD, is a UX Researcher with a focus on AI and the author of Designing Agentic AI Experiences (CRC Press, 2026), Design for the Mind (Manning, … More about Victor ↬Email NewsletterWeekly tips on front-end & UX.Trusted by 182,000+ folks. Meet AI 2027 Conference, December 1, 2026. Design Patterns For AI Interfaces with Vitaly Friedman Celebrating 10 million developers Design Patterns For AI Interfaces, 30 lessons + UX training How To Measure UX and Design Impact, 8h video + UX trainingAn AI feature can save users time on a task, only for them to spend it checking the results. This practical audit helps product teams decide where AI earns its place and what users need to verify its output.A product team at a large health insurer wants to add AI that writes the full written determination for benefits appeals specialists. The technology works as specified. However, the specialists stop using it, and the ones who keep using it take longer on each case than the ones who ignore it.A short evaluation with the specialists surfaces the problem fast: reading an AI-written determination carefully enough to sign it takes about as long as writing one, and the specialist also has to check for mistakes they would never have made themselves. The feature adds work to a task these people had already mastered.Rescoping the release fixes it. A second version surfaces the policy provisions and prior decisions relevant to each appeal and leaves the writing to the specialist. The search step disappears, review time drops, and the part of the job the specialist is good at stays where it was.The technology is identical in both versions, and the difference is which part of the task the AI handles.Finding The Break-even LineEvery interface change costs the user something before it pays anything back. They have to rebuild their picture of how the tool works, learn where things are now, and decide how far to trust what the AI hands them. Nelson Cowan’s review of working memory research (2001) put the limit at about four items held in mind at once. All three of those jobs draw on the same small supply.An AI feature in an enterprise software setting requires more from the user than a redesigned menu, because it changes what they do all day. Hand the synthesis to an AI and the user becomes a checker. Give it the recommendation too, and the job shrinks to signing off. Either change costs user attention, and for people whose training is their standing at work, either one can feel like a demotion. For example, in one study, Sophia Ackerhans (2025) and colleagues found that physicians asked to sign AI-assisted decisions reported a more serious threat to their professional identity, and explanations of the AI’s reasoning raised both trust and that threat together.An AI feature earns its place when what it returns beats what it costs the user to adjust, for one user doing one task.We can call that the break-even line. Returns to users come in three forms:Effort the feature removes,Errors it catches that the person would have missed, andConsistency across people doing the same work.For example, an AI-powered fraud screening tool shows the errors-caught form at work, since the analyst reviews flags they would never have raised themselves, which costs attention, and the catch rate pays for it. That puts the feature above the break-even line.Figure 1: Net value of an AI addition. The return has to overtake what the user spends adjusting before the feature pays anything back. Anything below the break-even line leaves the user working harder than before. (Large preview)Four questions can help your team locate the break-even line before design work starts. Each one produces a design condition the brief can act on.These questions do not replace the build decision. Funding, strategy, and existing commitments all weigh on that, and four questions cannot see any of it.Run through all four question