The Machine Learned to Take a History. Medicine Stopped Taking One.

Two papers this week, both with Eric Horvitz as senior author, one with Eric Topol beside him. The first, in Science, reports that an AI system diagnosed New England Journal of Medicine case records correctly 83 percent of the time against 36 percent for practicing generalist physicians, and emergency admissions 80 percent against 47, at lower testing cost. The second, in BMJ Digital Health, lays out six dimensions of medical intelligence and says that superintelligence means surpassing the strongest human-led health systems across all of them, integrated, over the full arc of care.

The headline will be the first number. The interesting number is 25.

What the machine actually did

The Science paper is better than its headline, and the reason is a design choice. Every previous benchmark handed the model the complete case, chief complaint through test results, in one prompt. The authors refused to. Their benchmark starts each case with a short vignette and makes the diagnostician, human or machine, ask for the rest: questions about history, examination findings, tests, each test billed at chargemaster prices, until the diagnostician commits. A gatekeeper answers from the record. A judge grades the diagnosis. Cost and accuracy are reported together.

Inside that benchmark, the AI system, an orchestrator that simulates a panel of clinicians arguing, maintains an explicit differential, challenges its own leading hypothesis and prices every test before ordering it, asked an average of 25.5 questions before it ordered its first test. The physicians asked 5.6. The system's advantage over a bare model prompt was concentrated, in the authors' words, in diagnoses "that depend on better history-taking." When physicians erred, their wrong answers were further from the truth than the machine's. And the orchestration advantage grew, not shrank, when the model underneath was upgraded.

Read that as a physician and the result is not that the machine out-knew the doctors. It is that the machine took a history. It did the thing every medical student is taught in the first month and most practicing physicians have not had time to do properly in years. It won on the oldest skill in the building, because it had an unlimited budget for the one resource medicine never has.

I have been here before

In February I wrote that consumer health AI failed at triage because it had been trained on the documented end of the clinical story and deployed at the undocumented beginning. In March, reading a sixty-point collapse between benchmark accuracy and real-user accuracy, I argued the gap was not a model problem but an architecture problem: nobody had built the system that asks "tell me more about the chest pain" instead of answering whatever it was handed. The same week I wrote that nobody taught medical students to wait for clean data; the physician's method under fragments is risk first, differential second, the minimum set of questions that discriminates, then every other source you can reach.

The Science paper is that architecture, built and measured. Sequential elicitation. An explicit differential. The marginal value of each question and each test weighed before it is spent. The design brief reads like the March article, and the result is the one I predicted: build the process around the model and the gap closes. The authors even confirm the half I cared most about, that it was the orchestration and not the model that did it.

So I should be pleased, and I am. The rest of this article is about why I am not satisfied, and the answer is in the paper's own limitations section.

Medicine never had time for twenty-five questions

The physicians in the study asked under six questions because physicians ask under six questions. Not from laziness. Six is what the room allows. A full history and physical, the kind that produces twenty-five discriminating questions and an hour of examination, is the thing we teach, examine, and almost never perform after residency, because the emergency department does not have an hour and the clinic has twelve minutes.

What the emergency physician does instead is the thing the benchmark cannot see. She looks at the patient from the doorway. Posture. Breathing, its rate and its effort. Skin color. Eye contact and gaze. Whether he is holding his chest. Whether he is coherent, intoxicated, in pain, able to answer easily. Whether there is a rash visible across the room. That look takes ten seconds and carries most of the information she will use in the next six questions, and the six questions are chosen because of it. Nothing about the look is a test. Nothing about it is billed. None of it is in any record the gatekeeper could answer from.

The authors name the gap honestly. The benchmark, they write, does not capture "multimodal data such as imaging, waveforms, or pathology slides that are central to real diagnostic workflows." That sentence is true and it is also the whole problem, because look at what counts as multimodal in it. The image. The waveform. The slide. Artifacts, every one, produced by a machine, about the patient, after the patient has already been seen. The one modality that comes first in every real encounter, the patient himself, seen by a person who knows what sick looks like, is not on the list. It is not on the list because it is not in the chart, and it is not in the chart because medicine stopped putting it there.

We pre-cooked ourselves

This is the argument of the first chapter of my book, The Second One in the Room, and the Science paper is its best evidence yet.

For most of medical history the physician's distinctive contribution lived in the body: the percussion that found the liver edge, the lymph node that felt wrong, the smell at the bedside that meant ketoacidosis, the look from the doorway. That knowledge was resistant to machines because it lived where no chart could reach. Then, over fifty years, reimbursement paid for what could be coded, documentation requirements expanded, visits compressed, liability rewarded data over intuition, and imaging and laboratory medicine moved the decisive information off the body and onto the screen. The physical examination you receive today is shorter than the one your parents received. In routine practice most of what decides a diagnosis is now structured data a machine can ingest natively.

The AI did not cross a frontier into the physician's territory. The frontier moved. Medicine voluntarily contracted its diagnostic surface area to fit the shape of the structured record, and then a model arrived that reads structured records very well. The benchmark in Science is honest about measuring "diagnostic reasoning under idealized information conditions rather than bedside performance." The uncomfortable addendum is that idealized information conditions are where most of medicine now happens. The physicians in the study were also reasoning over a text oracle. They were just slower at it, and tired.

The modest proposal

Follow the trend to its end and the proposal writes itself. If the diagnostic surface area has already contracted to what a machine can read, finish the job. Put a scanner at the emergency department door. Every patient gets a full-body CT, a complete panel, a continuous waveform, before any physician sees them. Hand the machine the record it was built for, let it ask its twenty-five questions of an oracle that can actually answer, and the benchmark becomes the clinic.

Every physician reading that heard the radiation dose, the incidentaloma cascade and the biopsy of a nodule that would never have mattered before the sentence ended. That is the point. It is absurd stated plainly, and it is most of what we already do by slower means. We image first and examine later, if at all. We have replaced observation of the patient with observation of artifacts about the patient, and called the artifacts the record, and trained a machine on the record, and the machine is now better at the record than we are. Whether that is medicine is a question the Science paper is not asking, and the BMJ paper is.

What the second paper admits

Read together, the two papers are more honest than either. The BMJ piece, same week, same senior author, defines medical intelligence across six dimensions: knowledge and its application, diagnostic reasoning and triage, therapeutic action, longitudinal context, clinical coordination, alignment with human values. It says progress must be measured against the strongest health systems, not individual clinicians. It warns that when a machine exceeds the human judge, "superior reasoning may initially appear to be an error," which is the supervision problem I have spent a year on, arriving from the other direction.

By that paper's own yardstick, the Science result is one dimension of six, under oracle conditions, against individual generalists, on the diagnostic phase alone. The authors say as much. What the two together establish is narrower and more useful than the headline: given a complete record and time to ask, a well-orchestrated model reasons about it better than a tired generalist. That is real. It is also a description of the one setting medicine built for the machine without meaning to.

What I am still waiting for

I have argued since my first article that the hospital, not the valley, is the north star for AI, because medicine solved the problems of acting under uncertainty with a confident instrument before anyone had a model. This week two excellent papers moved the field toward the hospital and stopped at the door.

I am waiting for the agent that can stand in the doorway. That takes in what the emergency physician takes in before she speaks: posture, color, breathing, gaze, the hand on the chest, the coherence of the first sentence. That then asks six questions, not twenty-five, because six is the budget, and chooses them because of what it saw. That system would be practicing medicine the way medicine was practiced before we taught ourselves to practice from the chart. Nobody has built it, and nobody is benchmarking for it, because the data it needs was never collected, because we stopped paying anyone to collect it.

The second one in the room reads the chart better than we do. We made sure of that. The thing it still cannot do is the thing we stopped doing first.

This article is part of an ongoing series on healthcare AI, clinical workflow, and the systems that connect them.