Medicine spent twenty years building governance frameworks for finding-first medicine. Fleischner. Bethesda. Lung-RADS. Each took years, named authors, bounded scope. Consumer health AI synthesizes your entire medical record before you see your doctor. Nobody built the framework for that.
Clinical reasoning was built on a simple sequence: observe, hypothesize, test. Three disruptions have quietly inverted it. The signs still work. The sequence that gave them their meaning is disappearing.
The judgment layer just shipped. A patient arrives with a Google Health Coach synthesis of her last two years of medical records. Her physician receives a pre-processed narrative - no author, no methodology, no version history. The default answer is now Google.
AI raises your confidence whether it's right or wrong. Two preprints from MIT and Wharton show it also degrades the skill you need to catch it when it fails. Aviation solved this problem decades ago. Medicine and software haven't.
A new BMJ paper asks why humans are still in the loop now that AI outperforms physicians on reasoning tasks. The answer is a framework. The problem is that framework assumes a physician whose independent competence AI is quietly eroding.
We are running an experiment on human oversight without a control group, without outcome tracking, and without a policy framework designed for the result. The current generation of experts may be the last one capable of catching what AI gets wrong.
Most AI clinical studies measured the wrong thing. They constrained the reasoning mechanism and then evaluated what was left. The Brodeur Science paper finally tests the model the way medicine actually works. The results are hard to dismiss.
Utah just became the first US state to let an AI autonomously renew prescriptions. The legal debate is real. The design question nobody is asking is more important: who designed the rungs on the autonomy ladder before the climb began?
Amazon owns the patient at the moment of health decision intent. OpenEvidence owns the physician at the moment of prescribing intent. Both are monetized by the same pharmaceutical industry. The prescription is the handshake between them.
AI was trained on what physicians write. Not on how they reason. The note comes hours after the decision, shaped by billing codes and fatigue. What actually saved the patient was never recorded.
Microsoft just analyzed 500,000 Copilot health conversations. The paper is honest: they observed questions but not outcomes. The physician in me cannot stop thinking about it. In medicine we audit what happened to the patient, not just what we considered. Why should AI be different?
A patient record can be comprehensive and clinically wrong at the same time. The AI reasons from what it receives. The failure is upstream, in the data, not in the model. That is the hallucination type nobody is naming.