Medicare grades 2,316 hospitals on heart-failure readmissions and docks the payments of the ones it calls worse than expected. Follow every hospital left to right — from the number of cases it actually had, through the grade CMS gives it, to whether that grade is distinguishable from chance.
ARPA-H just put $62.7 million into autonomous heart-failure agents, with a second AI built to supervise the first, and the six health system leaders Becker’s asked could not say who carries the liability when the supervisor misses something. That argument assumes a prior step: that when the agent changes an outcome, somebody can tell.
Here is the outcome in question, as Medicare already measures it.
The first column is not a quality signal. It is arithmetic: 849 hospitals (36.7%) had fewer than 200 heart-failure discharges in three years, the median had 274, and only 560 had 500 or more. The second column is the grade, and it divides the field almost exactly in half — 1,187 worse than expected, 1,129 better. The third column is the same hospitals tested against their own expected readmission count.
1,978 of 2,316 — 85.4% — land in the grey. Their observed readmissions are inside the range chance alone would produce at their volume. They are still graded, still ranked, and still paid accordingly.
Every hospital in the grey band has a published verdict on its heart-failure care that its own case count cannot support.
The collapse is not uniform, and the pattern is the point. Among hospitals with fewer than 200 cases, 10.4% of grades survive. Among those with 500 or more, 22.7% do. Raise the confidence bar to 99.8% — the three-sigma limit a funnel plot conventionally uses — and the survivors fall to 60 hospitals, 2.6% of the field.
Denver Health’s Daniel Kortsch, MD, told Becker’s the evidence under the AI-supervises-AI model runs to four conditions, none of them heart failure, with a headline trial of 32 patients. At the national heart-failure readmission rate, a 32-patient sample’s 95% interval spans 6.3% to 31.3% readmitted. The median hospital in this file, with 274 cases over three years, still lands in the grey. If you are promising a regulator that an agent moved this outcome, the sample that could show it is larger than most American hospitals generate.
Joy Oh of The Christ Hospital asked who is responsible when a supervisory agent fails to catch a worker agent’s bad recommendation. Before a court gets there, somebody has to establish that the outcome moved and that the agent moved it. On this measure, at these volumes, at a single site, that is not available in twenty-four months.
Which points at what to instrument instead: the override. Every time a clinician rejects the agent is a labelled event, generated at the rate of care rather than the rate of readmissions, and it is the only dataset a supervisory model can actually be evaluated against. Nobody in the ADVOCATE architecture has said who is capturing it.