A health system almost renewed a clinical decision-support contract on the strength of its validation study and FDA clearance. Then the CMO pulled the override log: physicians had reversed more than 40% of the tool's recommendations over six months. Nobody had flagged it. Here are 100 of those recommendations.
Primary source: Healthcare AI's next challenge isn't adoption, it's reliability (SmartBrief, Jul 2026) — Sina Bari, MD
Each square is one recommendation the cleared tool made at the point of care. Navy means the clinician accepted it. Red means they silently overrode it. Press play — then ask the only question that matters: of the reversals, how many did anyone review?
The debate about whether doctors will use AI is settled — daily physician use hit 63% this year. The unanswered question is whether the tools already running still work where patients depend on them, and whether anyone would know if they didn't.
The comforting read is “40% overrides means the safety net worked — the doctor caught the tool's mistakes.” Sometimes. But an unmonitored 40% doesn't tell you humans are catching errors. It tells you nobody knows what the tool is doing: whether it drifted, whether it's crying wolf, whether the 40% is smart clinicians or a broken model. A safety net you can't measure is just a story you tell the renewal committee.