Doctors use OpenEvidence to ask about treatment and management as well as diagnosis. A US study published in JAMA on 15 October examines about 271 million queries, but measures information-seeking rather than whether the platform gives good advice or improves care. Original study
That distinction makes the findings useful to people evaluating medical AI. A question can reveal a gap in information without showing how an answer was used. It is the difference between observing what someone looks up and observing what happens to their patient.
What the study counted
The cross-sectional analysis covered January 2024 through June 2026. It included 952,008 US clinicians with National Provider Identifier–verified accounts, including 556,200 physicians. A query meant an individual question; a follow-up counted again. The analysis concerned the main interface, excluding institutional deployments, scribing and other integrations. Population and methods
The researchers used an AI classifier on a stratified subset of about 39 million queries. Among physician questions, the adjusted estimates put treatment and management at 33.1% and diagnosis at 13.9%. These are category estimates, not manually reviewed labels for all 271 million queries. Nor are these two categories an exhaustive breakdown.
For a service evaluating a similar tool, the practical implication is to test a range of information tasks. A diagnostic benchmark alone would leave important uses unexamined. This is an editorial inference about evaluation, rather than an outcome established by the study.
A reference tool has to preserve a route back to evidence
Harvard Medical School’s account frames the work around clinicians seeking medical information across a broad range of questions. It describes this same study, rather than providing another test of the platform. Institutional account
The accompanying JAMA editorial identifies a consequential distinction: a generated answer can contain a genuine source link and still interpret that evidence incorrectly or apply it to the wrong patient. A citation gives the reader somewhere to check; it does not complete that check. Editorial
Consider a question about treatment options. Finding the cited trial is one task. Establishing that its participants, comparator and follow-up match the clinical question is another. This example explains the verification problem; it is not a report of an error found in the study.
Use cannot stand in for benefit
The logs did not establish answer accuracy, clinicians’ subsequent decisions or patient outcomes. They also could not distinguish questions asked during encounters from preparation or education. Company employees were among the authors, and the company supplied the data. Study limitations and disclosures
The editorial also discloses that JAMA Network licenses content to OpenEvidence. Its call for independent evaluation should be read alongside that relationship. The next evidential step is to connect information retrieval with the quality of decisions and care, while examining what clinicians learn or cease to check. Those questions remain open even when a platform attracts sustained use. Editorial discussion and disclosure