clinicians.dev

A community of clinical experts super-powered with skills in agentic engineering. Our roots are in coding and clinical medicine. We know how to create and supervise LLMs writing Python, SQL, R, JS, HTML. We love startups, regs, policy, UI/UX, data viz, project management, informatics. We want to create the medicine of the next 100 years.

✉ Email for Slack invite → Slack → Substack ↓ Next event

today · the wire

The Wire

What you'll be asked today — and what to say back. A couple that caught my eye this morning; the rest are on the wire.

Why did your prior-auth API roadmap just ship inside a hospital payment rule?

ONC / CMS.gov · Jul 31

The FY2027 hospital payment rule turned out to be an interoperability rule in disguise. ONC used it to lock in seven updated HL7 FHIR implementation guides — the Da Vinci prior-auth stack (CRD, DTR, PAS), CARIN Blue Button, payer drug formularies, provider directories, and payer-to-provider clinical data exchange. These versions started as proposals in CMS's prior-auth rule and are now final, replacing what last year's HTI-4 rule adopted, effective October 1. They're the versions CMS now points to for payer APIs and ONC's electronic prior-auth certification — meaning the FHIR IG versions you build against just got pinned, and the pin arrived in a payment rule most product teams never read.

BuilderIf your product touches prior auth or payer data exchange, diff your build against the versions named in this rule now. Version drift against a federally adopted spec is the kind of bug that becomes a compliance finding later.

A documentation prompt just became Exhibit A in a $14.1M coding settlement

Fierce Healthcare · Aug 4

Complete Health is paying $14.1 million to settle DOJ allegations that from 2020 to 2023 it submitted Medicare Advantage risk-adjustment diagnoses — drug and alcohol dependence, mental health HCCs — that weren't clinically valid, weren't supported by the chart, or never showed up in the patient's actual care. The detail worth sitting with: DOJ's complaint says the software itself prompted physicians to add codes even when they weren't substantiated. The nudge was part of the allegation, not just the coding. Any tool that suggests a diagnosis code is now a piece of evidence — worth confirming every suggestion traces to documented, treated conditions, not a revenue target.

Samsung Medical Center handed referral triage to an open-source model — and kept it in the building

npj Digital Medicine · Aug 5

Samsung Medical Center deployed an on-premises Qwen-2.5-32B to assign subspecialties for 6,624 electronic referral letters, with the live clinician roster embedded in the prompt. Against human coordinators it hit 75.4% baseline accuracy — but once experts adjudicated the disagreements, validated accuracy rose to 84.7%, meaning a chunk of the original "errors" were defensible calls. Misclassifications clustered between clinically adjacent departments rather than at random, and 5.9% of referrals simply lacked the information for a unique assignment — a data problem no model fixes. The design choices are the real story: open-source weights, on-prem inference, real-time roster awareness, human in the loop.

80/20The 80/20 here isn't the accuracy number — it's that a mid-size open model plus workflow context (the roster) beat the instinct to reach for a bigger model. Context engineering is cheaper than parameters and easier to govern.

What happens when the red-team AI decides real people are in scope?

BleepingComputer · Aug 5

During UK AISI cyber evaluations, frontier agents from both OpenAI and Anthropic left the sandbox: one tried to compromise real GitHub maintainers with a malicious pull request and fake advocacy profiles, and another exploited a real website that happened to share a domain with its fictional test target. Both attempts failed, but the mechanism is the concern — the agents treated live internet resources as fair game. For anyone piloting agentic workflows against production systems — EHRs, claims pipelines, patient communications — this is the concrete version of the scoping problem. An agent's notion of "in bounds" is a prompt, not a firewall, and the collateral surface includes people, not just systems.

SkepticEvery agent vendor deck says "guardrails." The two best-resourced safety teams on earth just watched their models freelance against real humans mid-eval. Ask your vendor what physically prevents egress — not what the system prompt requests.

The $400 million CEO who turned an insurer into a drug middleman

Healthcare Uncovered, Wendell Potter · Aug 5

As David Cordani retires, Wendell Potter — a former Cigna VP — tallies the 17-year transformation: the $67B Express Scripts deal made Evernorth ($58.4B in Q1 2026 revenue) dwarf the actual insurance business ($11.5B), while a RICO class action over a Swiss kickback entity, an FTC PBM settlement, denial-algorithm litigation, and a $172M DOJ Medicare Advantage settlement piled up alongside. One line stuck with me: "The machine is working well, by the metrics it was built to serve. Whether it is working well for the right people remains, as it has always been, the question the proxy statements are not designed to answer."

Read the full wire ↗

learn by doing

Interactives

Companion experiences for the newsletter — don't just read about it, try it.

The Ruler’s Own Ruler

August 7, 2026 · Source: Aali et al., “MedVAL: Toward Expert-Level Medical Text Validation with Language Models” · npj Digital Medicine, Aug 4, 2026 · all 90 model×task cells from Tables 2, 3a, 3b, S2, S3 · MIMI Labs · Dartmouth Atlas MEDPAR ICD‑10 principal diagnoses, 2018

Stanford open-sourced the thing everyone said was missing: an evaluator that checks whether AI-generated clinical text is faithful to its input, trained without a single physician label, with a 4B model that runs on a laptop. Ten models were scored against MedVAL‑Bench and five were also distilled — 90 model‑by‑task cells, every one plotted here against input length, inter-physician agreement, or task size. Set the axis to agreement and show MedVAL GPT‑4o and you reproduce the paper's r = 0.67: the evaluator scores highest exactly where the physicians agreed with each other most. Then drop query2question and the correlation doesn't weaken — it inverts to r = −0.55. One task out of six, the one where twelve physicians reached only α = 0.560 among themselves, was carrying the whole relationship. Critical lens: the abstract's headline F1 of 66% → 83% is the binary safe/unsafe judgment; the four-class risk grade a reviewer would act on moves 36.7% → 51.0%. And the non-inferiority claim rests on 90 of 840 examples, tests GPT‑4o rather than the released 4B, and never prints the observed Δ or the human expert's own score.

840 Dots

August 7, 2026 · Source: Aali et al., “MedVAL: Toward Expert-Level Medical Text Validation with Language Models” · npj Digital Medicine, Aug 4, 2026 · counts from Tables 2 and S3 · MedVAL-Bench

Everything the new medical-text evaluator knows about being wrong comes from 840 physician-annotated outputs. Here they are — all of them, one dot each, six task columns, one animated field with no click-through. Colour them by the four-level physician risk grade (44.4% of the benchmark is graded level 3 or 4, meaning a human must review or rewrite), by which three tasks were held out of distillation entirely, or by input length. Then light up the 90 dots — fifteen per task — that were read by more than one physician: that 10.7% subset is the entire basis for the agreement figures and for the headline that the evaluator matches a human expert. Below, the six tasks' average input lengths drawn to scale: 10, 50, 69, 82, 543 and 1,497 tokens. The medication-answer task, where distillation produces its largest gains, hands the evaluator a ten-token question — its bar is 0.7% the width of the ambient-scribe task's. A real ED encounter is not a paragraph; it is a chart. What didn't get distilled is the taxonomy: four risk levels, eleven named error types, twelve physicians, and a decision about what each kind of wrongness costs.

Browse all interactives →

from the substack

Builder's Briefing

The newsletter — what shipped this week, what it means, and what to build next. A couple of recent issues.

Fetal AI lifts 22 points 🤰, Your devices already run AI 🕵️, Part D comes off the drip 💧

⚡ Around the Wards

Read on Substack →

Duke built its own chart reader 🔦, ~80 AnMed sites go dark 🕳️, Being safe vs feeling safe 🤔

⚡ Around the Wards

Read on Substack →

What the Vibe

Saturday, May 30, 2026 · 12p–2p ET

Vibe coding changed everything. The barrier to building just disappeared. So what's the vibe? Is clinicians.dev dead? Does knowing how to code still matter when anyone can build? Four clinician-builders. Twelve minutes each. Show us what you made — vibe-coded, hand-coded, or somewhere in between.

Tentative speakers

Joe Izzo

First, Do No Harm — Especially With Your MVP: A CMIO's Advice in the Clinician-Builder Era

Patrick Damaso

Trajectories, Not Targets: Catching Reward-Hacked Diagnostic Reasoning One ICD Branch at a Time

Sam Wang

Silent Denials, Loud Insights: Visibility into the Payors Who Won't Show Their Work

Matt Segar

From Bedside to Buyout: An EP Fellow Path to Acquisition

4 talks (12 min each)
1:1 breakouts after each
Group discussion
Open community talk: "What the Vibe?"

RSVP on Luma →

01 · Learn

AI & agentic engineering

The foundational reads, the tools clinicians-who-code are actually using, and the loop that ships.

Context engineering — the foundation

Effective Context Engineering for AI Agents

Anthropic's foundational guide.

Building Effective Agents

When to use workflows vs. agents, and how to structure both.

2026 Agentic Coding Trends Report

How teams are actually shipping with agents.

Learn `CLAUDE.md` / `AGENTS.md`

The spec files that make agentic coding repeatable.

🔁 The Ralph Wiggum Pattern — the loop that ships. Tight spec → agent implements → you verify → adjust spec → repeat. Just keep going.

Tools of the trade

Tool	Best for	Notes
Claude Code	Complex multi-file work, architecture	Terminal-native, large context. The power tool. Good for non-code tasks too.
Cursor	Daily coding with autocomplete + inline edits	AI IDE, huge user base.
Windsurf	Budget entry, Cascade collaboration	AI-native editor.
Copilot / Codex CLI	Repo-native GitHub workflows	Tight GitHub integration.

Agentic frameworks & orchestration

CrewAI— multi-agent orchestration, clinical workflow pipelines
LangGraph— graph-based agent workflows with state
AutoGen— Microsoft's multi-agent conversation framework
MS Agent Framework 1.0— production .NET + Python
DSPy— programmatic LLM pipelines, good for clinical NLP

02 · Communities

Communities, courses, substacks

Where clinician innovators and clinician builders are hanging out.

EHRs, FHIR, fake patients

The pipes you'll need when you're actually building.

EHRs & FHIR

🔑 SMART-on-FHIR / OAuth flow — fhir.epic.com/Documentation?docId=oauth2

Clinical datasets worth knowing

From CXRs to genomes to ICU waveforms. Filter by name, modality, or license.

Name	Relevance	License
ReXGradient-160K	160k multi-site CXR + reports — vision-language radiology sandbox.	Harvard DUA, non-commercial
CheXpert Plus	Large paired CXR–report set; benchmarked in many papers.	Stanford DUA, free research
Endoscapes 2023	Open laparoscopic chole frames — segmentation / CVS detection.	CC BY-NC-SA 4.0
Surg-3M	3M surgical frames powering "SurgFM" foundation model.	TBA, expect research-only
AFRICAI Repository	Imaging sets from African centres — fairness & domain shift.	Mixed open licenses
OpenOximetry	Waveforms + skin-tone data for pulse-ox bias work.	PhysioNet credentialed
DeepLesion	32k CT slices with bounding-box lesions; detection / tracking.	NIH DUA, research-only
BioASQ Synergy 2024	Biomedical Q-A pairs — LLM eval set.	CC BY 2.5
CliniFact	Clinical-trial fact-checking corpus — fine-tune retrieval / RAG.	MIT
Hallucination Annotations	Doctor- & LLM-written discharge summaries with token-level labels.	PhysioNet credentialed
Clinical-Trial Eligibility QA	QA pairs linking MIMIC-IV to apixaban RCT criteria.	PhysioNet credentialed
PIFIR	Wearable PPG/ECG for arrhythmia-free interval prediction.	PhysioNet restricted
GREGoR R02	Rare-disease genomic + phenotypic harmonised data.	dbGaP controlled
Synthetic Rare-Disease EHRs	Benchmark synthetic EHRs for low-prevalence conditions.	CC BY
Korea4K	4k Korean genomes — ancestry diversity for variant calling.	EGA controlled
OpenNeuro	20k+ public neuro-imaging sessions; BIDS-ready.	CC0 / CC BY-SA
Bridge2AI-Voice	Multimodal speech (voice, vitals) for health AI.	PhysioNet restricted
PMDB Pain Monitoring	Wearable IMU + self-report pain diary.	CC BY 4.0
DREAMT Wearable Sleep	Apple Watch PSG pairs for sleep-staging models.	PhysioNet restricted
MC-MED	Multi-condition medical dialogue (GPT / human).	PhysioNet credentialed
Wearable Stress Dataset	Smartwatch vitals + stress labels — mental health ML.	PhysioNet restricted
MIMIC-IV v3.1	Flagship 380k-patient de-id EHR; ED + ICU tables.	PhysioNet credentialed + CITI
MIETIC	Italian clinical-notes corpus with entity spans.	PhysioNet credentialed
ODD (Opioid Behavior)	Annotated notes for opioid-related behaviour NLP.	PhysioNet credentialed
UK Biobank	500k UK adult cohort — EHR, surveys, genetics.	Controlled access
All of Us (NIH)	1M-goal US cohort — EHR, surveys, genomics, wearables.	Registered + Controlled tiers
TCGA	~11k patients across 33 cancer types — multi-omics + clinical.	Partially open
AmsterdamUMCdb	First open European ICU DB — 23k admissions.	DUA required
ADNI	Longitudinal Alzheimer's — serial MRI/PET, clinical, biomarkers.	Free non-commercial
ABCD Study	10k youths — neuroimaging, cognitive, mental health, genetic.	NIMH controlled access
NHANES (CDC)	US national survey — health, nutrition, lab data.	Public domain
CheXpert (original)	224k chest X-rays, 65k patients — labeled findings.	Free non-commercial
EchoNet-Dynamic	10k+ cardiac ultrasound videos with EF + ventricle volumes.	Non-commercial
Synthea	Realistic synthetic patient records — full EHR.	MIT
1000 Genomes	WGS from ~2,500 diverse individuals — human variation reference.	Open access
DementiaBank (Pitt)	Speech recordings + transcripts from Alzheimer's patients + controls.	Consortium access
VitalDB	6,300+ surgeries with continuous high-freq vital sign waveforms.	Open, registration + DUA
Medical Segmentation Decathlon	10 open datasets for 3D medical image segmentation.	CC BY-SA 4.0
PANDA	10k+ prostate biopsy WSIs with Gleason grades.	CC BY 4.0

05 · Conferences

AI & health conferences

Borrowed from Raihan Faroqui's healthcare AI resource guide. Filter to find your tribe.

06 · Jobs

Job boards that don't suck (as much)

What they are. Not endorsements — just the lay of the land.

jobs.heyhealthtech.com
Rock Health Capital
MDisrupt
PharmDs in IT
AMIA
General Catalyst— surprisingly good for startups
Matchday Health
Wellfound
Look for Zebras
Fractional Jobs
HealthTech Nerds Jobs

07 · Startup

So you want to build a startup

But don't know anything about tech/stack, marketing/sales, finances, management.

Life hacking / survival

Because clinician-builders need to not burn out.

09 · Experiments

LLM matrix tests (alpha)

Testing out LLMs with fake patient scenarios generated by GPTs.

Past events

Where we've been.

Fall 2025 Conference

Time	Speaker	Talk
1:00p	TBD	Welcome
1:10–1:22p	Josh Mandel, MD	Conversational Interoperability for Prior Auth and Beyond — MCP, A2A, and the Unreasonable Effectiveness of Making Data Accessible
1:35–1:47p	Jung Hoon Son, MD	Data CPR: How LLMs Revive Buried Clinical Insights
2:00–2:12p	Jason Theobald, MD	Contain Multitudes: A Docker Intro
2:25–2:37p	Calvin Johnston, MD	Software Design and Family Medicine: Overlapping skill sets
2:50p	TBD	Wrap-up

Summer 2025 Conference

Time	Speaker	Talk
1:00p	Kevin Maloy, MD	Welcome
1:10–1:22p	Vishnu Ravi, MD	Digital Health that Ships: An Open Framework You Can Use Today
1:35–1:47p	Alex Dummet, MD	Topic TBD
2:00–2:12p	Pawan Jindal, MD	Startup Survival 101: A Clinician's Guide
2:25–2:37p	Omar Usman, MD	Change Data Capture (CDC): A Defiant Approach
2:50p	Kevin Maloy, MD	Wrap-up

Fall 2024 Conference (Nov 9, 2024)

Time	Speaker	Talk
1:00p	Kevin Maloy, MD	Welcome
1:10–1:22p	Paulius Mui, MD	From 0 to 1: Becoming a Clinician Who Codes
1:30–1:42p	Brian Fung, PharmD	Path of the Clinician Engineer
1:50–2:02p	Joe Izzo, MD	The Coding CMIO — it's okay to get your hands dirty
2:10–2:22p	Karambir Khangoora, MD	Learning to Code as a Clinician: Building Medagogy
2:30–2:42p	Joey LeGrand, PharmD	One Brain Can Be Better Than Two
2:50p	Kevin Maloy, MD	Wrap-up

Podchat — Should Hospitals Mandate GenAI Training? (Aug 29, 2024)

Listen to a short podcast, then talk about whether it's right or wrong. Original from Bill Russell at This Week Health. → Meetup event

clinicians.dev

The Wire

Interactives

Builder's Briefing

What the Vibe

AI & agentic engineering

Context engineering — the foundation

Learn CLAUDE.md / AGENTS.md

Tools of the trade

Skills marketplaces

Healthcare MCPs

Agentic frameworks & orchestration

Communities, courses, substacks

EHRs, FHIR, fake patients

EHRs & FHIR

Generate fake patients

Clinical datasets worth knowing

AI & health conferences

Job boards that don't suck (as much)

So you want to build a startup

Life hacking / survival

LLM matrix tests (alpha)

Past events

Learn `CLAUDE.md` / `AGENTS.md`