Designing AI that knows when to hand over to a human

There is a moment in every AI product where the system should stop talking and get a person involved. In a support assistant it is a refund dispute. In a triage tool it is a symptom that does not fit the rubric. In a wellness companion it is risk language. The technical detection of that moment is usually the easy part. The design of what happens next is where products quietly fail.

The three bad handovers

We see the same failure patterns across sectors, and they are all failures of design rather than detection.

• The dead end. The system announces it cannot help and stops. The user has now spent four minutes explaining a problem to something that was never going to solve it, and they have to start again.
• The silent transfer. The conversation is passed to a human with no context, so the first thing the person says is 'can you tell me what this is about?' — which reads as a punishment for having used the assistant.
• The reluctant handover. The system keeps trying, several turns past the point where it should have stopped, because a completion metric rewards resolution over appropriateness.

What a good handover actually does

It carries the context. A structured summary — what the user said, what was classified, what was already attempted, what the confidence was — arrives with the conversation, so the human starts informed. It sets an expectation with a real number: who will respond, through which channel, within what time. And it leaves the user something to do in the meantime, even if that is only a reference number or a link to local resources.

It also does not apologise for existing. A handover framed as a failure teaches users that asking the AI first was a mistake. A handover framed as escalation — this needs a person, here is that person — teaches them that the system knows its own limits, which is the single most trust-building thing an AI product can demonstrate.

Instrument the exit door

The metric that matters is not how often the system hands over. It is how often it hands over at the right moment. We track two error rates separately: late handovers, where a human review says the system should have stopped earlier, and unnecessary handovers, where it stopped on something it could have handled. Optimising only the second number produces a system that will not let go of a conversation it is losing — and in a health or wellbeing context, that is not a metrics problem.

Build the handover in the first sprint, not the last. It is not the edge case around your product. In every deployment we have shipped, it is the part users remember.