Stanford open-sourced an evaluator that checks whether AI-generated medical text is faithful to its input. Everything the evaluator knows about being wrong comes from these 840 examples. Here they all are.
The complaint about clinical AI for two years has been that nobody built the measuring stick. This week somebody did, and put it on GitHub: an evaluator trained without a single physician label, statistically non-inferior to one human expert, small enough to run on a laptop.
The interesting part is not the model. It is what the model was measured against.
The evaluator got distilled. The taxonomy didn’t.
Click Input length above and the dots resize to the average length of what the evaluator is handed. Then look at the same numbers laid out to scale.
A real emergency department encounter is not a paragraph. It is a chart: prior notes, an active problem list, a medication reconciliation, labs with trends, imaging with priors, and the three things the patient said that nobody wrote down. Every benchmark on the market is a pre-digested excerpt of that, and this one is more honest about it than most — the authors say plainly that their task suite “does not encompass the full spectrum of clinical document types and scenarios.”
Click The 90 that carry the claim. Fifteen examples per task were graded by more than one physician. That subset — 90 of 840 — is the entire basis for both the inter-physician agreement statistics and the headline that the evaluator is non-inferior to a human expert.
| Task | n | Krippendorff α | Avg. input |
|---|---|---|---|
| medication2answer | 135 | 0.904 | 10 tok |
| query2question | 120 | 0.560 | 82 tok |
| report2impression | 190 | 0.861 | 50 tok |
| impression2simplified | 190 | 0.872 | 69 tok |
| bhc2spanish | 120 | 0.943 | 543 tok |
| dialogue2note | 85 | 0.830 | 1,497 tok |
query2question the physicians reached α = 0.560 with each other. The paper reports a correlation of r = 0.67 between how well physicians agreed on a task and how well the tuned GPT-4o scored on it — which is worth sitting with. The evaluator looks most expert on the tasks where experts find it easiest to agree.Twelve physicians — four internists, three bilingual residents, four radiologists and a radiology resident — graded every output against a taxonomy they defined: four risk levels, and eleven named error types under three families.
| Family | Errors |
|---|---|
| Hallucinations | fabricated claim · misleading justification · detail misidentification · false comparison · incorrect recommendation |
| Omissions | missing claim · missing comparison · missing context |
| Certainty misalignment | overstating intensity · understating intensity |
The model is a commodity now. The judgment about what constitutes a dangerous omission in a discharge summary is not, and it doesn’t come from an engineer.
The evaluator is free and the benchmark is free, which means the defensible asset is an annotated corpus for your own population. Take 100 outputs from the tool you already run. Two clinicians. These four risk levels. Grade them independently and then look at where your two annotators disagreed — that number will tell you more about your deployment than any published F1.