Every device class the FDA heard about in 2024, dropped into a triangle by how it fails. Then the same 560 classes, scattered the way randomness would actually scatter them. The difference is the entire argument for building a detector.
“When an AI makes a mistake in coding, they’re not uniformly random,” Aron Szanto said on HIMSSCast this week. “They make telltale mistakes.” His team built a reviewer around that one observation, and the observation is doing all the work: you cannot build a detector for random. You can build one for telltale.
It is a claim about the shape of a failure distribution, and it is testable — just not on AI code, where nobody publishes. It is testable on devices, because every US adverse event report since 1991 has been tagged with one of three outcomes and dumped into a public API.
Malfunction. Injury. Death. Three outcomes, so every device class is a point in a triangle. A class whose reports are all malfunctions sits in one corner. All injuries, another corner. A perfectly indifferent class — failing every way equally — sits dead centre.
Here are 560 of them. Every dot is one device class; the bigger the dot, the more reports behind it.
In the real data, 339 of 560 device classes — 61% — are more than 90% a single outcome. Forty-four are 100%: every report they generated all year, the same result. The middle of the triangle is close to empty.
In the random version, that number is 11. The middle fills in, because that is what “no pattern” looks like when you draw it.
Continuous glucose monitors: 340,130 reports, 98.6% malfunction. Endosseous dental implants: 536,012 reports, 99.7% injury. Two of the largest objects on this chart, sitting in opposite corners, neither of them anywhere near the middle.
That is a detector’s working conditions. If you know the class, you know the failure mode before you see the report — which is precisely the property that lets Redesign Health run a specialist reviewer over AI-written code instead of a senior engineer they cannot afford.
The corner your device sits in tells you what your error handling is actually for. A malfunction-corner device fails quietly — it stops sending, the number goes stale, nothing alarms. An injury-corner device fails loudly and late, after somebody is already hurt. Those need opposite designs, and you can look up which one you have in about thirty seconds.
The death corner is a lie of scale. At true proportions every dot is welded to the bottom edge, because deaths are 0.58% of all 2024 reports. The magnifier exists so the vertical dimension is visible at all — at ×1, which is honest, this chart is essentially a line. Any time a visualisation lets you stretch an axis, ask what it looked like before.
The corners are partly a filing convention. Dental implants are 99.7% “injury” because a failed osseointegration gets coded as harm to a patient; glucose sensors are 98.6% “malfunction” because a sensor that reads wrong gets coded as a device problem. Both choices are defensible and neither is the same decision. 93.2% of these reports were filed by manufacturers, each applying its own house style. The pattern is real; some of its sharpness belongs to the filers.
The classes are not classes. generic_name is free text: 14,902 distinct strings in 2024 for roughly 6,000 real product codes, including “NO MATCH”, “UNKNOWN”, hundreds of catalogue SKUs, and both “VENTRICULAR (ASSIST) BYPASS” and “VENTRICULAR (ASSISST) BYPASS” as separate entries. Some of the 44 perfectly-pure classes are pure because a single manufacturer files one way under a name only it uses. The enumerable taxonomy is not free with the data; somebody has to build it, and for these strings nobody has.
Which is the real lesson for anyone hoping to do this for clinical AI output. The hard part of an Argus for hallucinated citations and silent unit conversions is not the detector. It is thirty years of somebody arguing about what the categories are.