A health chatbot can have the information needed to answer a medical question and still fail to help someone use it. A controlled study of UK adults illustrates that gap: performance when a model received a complete scenario directly did not carry through to people discussing the scenario with it. The result concerns the interaction as well as the model.
The distinction matters for healthcare teams assessing patient-facing AI. A high score on a medical question set answers one question about a system. Whether a person leaves a conversation with an appropriate understanding and next step is another.
When the user becomes part of the test
The Nature Medicine study involved 1,298 UK adults and ten simulated medical scenarios. Participants used GPT-4o, Llama 3 or Command R+, or resources of their choice in a control group. The tasks asked for relevant conditions and a course of action, such as whether to seek care.
When the researchers gave complete scenarios directly to the models, at least one relevant condition appeared in 90.8% to 99.2% of responses, depending on the model. This was a match against the researchers’ list, rather than a complete diagnosis or a patient outcome. Participants using the chatbots did substantially less well. Chatbot assistance did not show an improvement in the choice of course of action over the control group; condition identification was worse.
These were controlled tasks using model versions tested in 2024, not observations of treatment outcomes or a ranking of today’s services. The study cannot tell us how every purpose-built clinical tool performs.
Its publisher correction fixes a Figure 2a axis label. It does not report a change to the study’s numerical findings.
Knowledge and communication need separate evidence
The practical evaluation question is where useful information gets lost. A complete written scenario gives the model a defined starting point. An open conversation also depends on what the person describes, what the system asks, and what the person understands from its reply.
Those are different components of the task. A system could improve its medical knowledge without improving all of them. Equally, evidence from a generic chatbot should not be treated as a verdict on a service with a different interface, a narrower purpose and its own user testing.
An evaluation can therefore separate three questions: can the model handle the information, can the interface elicit and communicate it, and what does the user decide afterwards? That is an analytical framework for reading the evidence, rather than a claim that this study validated a particular design solution.
Evaluate the service people actually encounter
WHO’s guidance announcement on generative AI in health calls for defined tasks, early involvement of patients and other affected groups, and assessment after deployment. It also identifies the risk of people overlooking errors because they defer to automated output.
For a healthcare team, the useful evidence is consequently more specific than a vendor’s strongest benchmark. Who used the tested service? What task did they attempt? What comparator was used? Did assessment stop at the generated answer, or include the user’s understanding and action?
These questions do not supply a universal pass mark. They make clear which part of the service a result describes. A model answering correctly and a person receiving useful help are related achievements, but neither should be assumed from the other.