Which Model Gets to Read the Chart?

My feed has filled with physicians building their own AI tools. Not waiting for the EHR vendor, not waiting for the health system: a doctor with a weekend and an API key, wiring up something to summarize the labs, flag what the intake form missed, surface the pattern a tired brain might not catch at hour eleven of a shift. The excitement is real and I understand it, because the promise is not replacing anyone. It is reducing the cognitive load a physician carries into every visit. The large cloud models can clearly do this kind of work now. That stopped being the interesting question a while ago. The interesting question is the one that comes right after the excitement, which is that patient data often cannot leave the building, so a smaller model running locally enters the picture, and now someone has to decide which one deserves to be in the room with a chart.

"Trust me, it's a 70B" is not an answer a physician would accept from a colleague presenting a case. It should not be an answer anyone accepts from a model either. Parameter count is not a credential. It is a number on a spec sheet, and spec sheets do not tell you whether the thing in front of you can be trusted with a patient's record.

So I ran the evaluation I would want run before any model gets near a chart, not on charts themselves but on my local job-market research agent, the tool I have written about here before that helps me study how AI performs on real-world text as I work on this material. The task shape is close enough to clinical work that the lesson transfers directly. The model had to make an assessment and cite the exact line of source text that supported it. Not a paraphrase, not a summary of its own reasoning, a verbatim quote that actually exists in the document. In medicine this is not a nice-to-have. A resident presenting a finding on rounds has to point to the note, the lab value, the image. You do not get to say "the potassium was high" without being able to show the number. Verbatim evidence is not a formality in a chart. It is the entire culture of the chart.

I ran four local models through the same task on the same hardware, a 128GB Apple silicon workstation, and scored them on one metric I built for exactly this purpose: grounded yield, the share of outputs whose cited evidence is a real, verifiable quote from the source rather than an invented one. A dense 27B-class model came back at 90 percent grounded. A larger 70B model, more than double the parameters, came back at 65 percent. A fast mixture-of-experts model came back at 35 percent, which meant 40 percent of its outputs were permanently unusable, not because they were slow to fix but because a second guess from an ungrounded model is not meaningfully more grounded than the first. A 120B model, the largest of the four, landed around 20 percent. The biggest model in the test was the worst model in the test. The fastest was the most confidently wrong.

Translate that into a chart and the stakes stop being abstract. A model with high coverage and no grounding is the colleague whose differential sounds completely plausible and who has quietly invented the potassium value that supports it. The diagnosis might even be right. That is almost the more dangerous case, because a wrong conclusion reached through fabricated evidence looks, on the page, exactly like a right conclusion reached through real evidence, until someone goes back to the actual lab printout and finds nothing there. I saw a version of this pattern directly in the rankings. All four models roughly agreed on which items mattered most and which mattered least, independent of how well-grounded their reasoning was. In clinical terms, the assessments sounded alike across every model. The difference was never the conclusion. It was whether the data behind the conclusion was real.

The finding that stayed with me longest was about the exit nobody took. The scoring format I used allowed a model to say, plainly, score zero, no supporting evidence found. That is the correct move when the source genuinely does not support a claim, and it is the clinical equivalent of a colleague telling you honestly that the chart does not support the diagnosis they were reaching for. None of the weaker models used it. Instead of taking the honest exit the schema explicitly offered, they manufactured a quote to fill the field, because an empty answer looks like failure and a fluent, confident sentence looks like success. In medicine, the inability to say "I don't know" is one of the most dangerous traits a colleague can have. It is worse than being wrong occasionally. It means you cannot tell, from the outside, when the ground has disappeared under what you are being told.

Here is the turn that actually matters for anyone making this decision for real. Choosing grounded yield as the metric was the only part of this entire exercise that required judgment. Building the pipeline was mechanical. Running four models on one machine was mechanical. Deciding that verbatim evidence integrity, not speed and not benchmark accuracy against someone else's label, was the property worth measuring for this task, that decision was not handed to me by any leaderboard. And it will not be handed to a clinic either. A clinic evaluating local models for anything patient-facing will be handed plenty of leaderboards: tokens per second, benchmark accuracy, memory footprint. Nobody will hand them a leaderboard for whether a model fabricates its evidence, because that number does not exist yet as a standard measure. It is also the only one that maps to what physicians already require of each other every day on rounds.

None of this argues for letting a local model reach a conclusion on its own. The frame that survives this evaluation is assistance under supervision, evidence grounded, reviewed by the physician before it touches a decision, the same way a resident's differential is reviewed before it becomes an order. What the evaluation earns you is a defensible answer to the question of which assistant belongs in that room in the first place.

Trust in a clinical AI system is not a property of how many parameters it has. It is a property of the evaluation someone ran before that model was allowed anywhere near a chart, and whether that evaluation asked the one question medicine has always asked of anyone presenting a finding: can you show me where you got that.