At Epic's CIO Forum, MyChart's AI told a patient it was fine to play golf. The room's first thought was lawyers. Here is the question nobody asked: who is on the other end? Every US county — 3,144 dots, 260.8 million adults — on ten CDC health measures.
A patient asks the assistant whether it is okay to play golf. Based on the record, the assistant says yes. A CIO who is also a practising clinician raises his hand: you do realise that's medical advice. On This Week Health's Newsday, the word the panel reaches for is lawyers.
All of that is downstream of a question nobody asked: was the answer correct? And correctness here is not a property of the model. It is a property of the person typing. Eighteen holes is four to five miles of walking. For most adults that is exercise. For an adult with coronary heart disease, or COPD, or a recent stroke, it is a clinical question whose answer depends on things a chart may or may not contain.
An escalation threshold is not a policy. It is a number applied to a population. So look at the population.
Each dot is one county. Dot area is the adult population — from Loving County, Texas (37 adults) to Los Angeles County (7.7 million). Colour is the uncertainty index: the average width of the 95% confidence interval across all ten measures for that county. Navy dots are tightly estimated. Red dots are not.
Leave the horizontal axis on coronary heart disease. The national figure is 6.8% of adults. The county range runs from 3.4% in Madison County, Idaho to 14.9% in La Paz County, Arizona — a 4.4× spread. Four hundred and ninety-four counties, holding 9.3 million adults, sit at 10% or above.
None of that is visible to a patient-facing assistant reading one record. It is not supposed to be. But it is exactly the thing an escalation threshold gets calibrated against, and a threshold tuned on the population that produced the pilot data is a threshold tuned on somebody else's county.
Before you argue about who is liable for a patient-facing answer, write down the confidence floor below which the assistant says call us — as a number, with an owner, and a re-tuning date. If your governance document does not contain that number, you do not have governance. You have indemnity.
Drag min adults all the way right and watch the correlation. Nine hundred and seventeen counties — 29% of all counties, holding 1.8% of US adults — have fewer than 10,000 adults. In most county datasets, dropping them collapses whatever pattern the tiny denominators invented.
Here it barely budges. Coronary heart disease against fair-or-poor general health is r = 0.66 across all 3,144 counties and r = 0.59 among counties with 50,000+ adults. That stability is not evidence the finding is robust. It is evidence of what these numbers are.
PLACES county estimates are not measurements. They are model-based small-area estimates — multilevel regression on BRFSS responses, poststratified onto census demographics. In a county with a handful of respondents, the estimate is mostly the model talking. The same covariates drive every county, so some of the correlation between measures is built in. Filtering by size does not remove the model. It removes the counties where the model did the most work.
The uncertainty ceiling is the honest version of the small-n filter. Slide it down and the dots that vanish first are not the wrong counties — they are the counties where CDC is telling you, in the width of the interval, that it does not really know.