What Does Good AI in Healthcare Look Like? CHAI CEO Brian Anderson Is Trying to Get Every Hospital in America to Agree

Key Podcast Moments

• Why an EHR rollout at Mass General was the moment that pushed Brian Anderson out of the exam room and into building technology instead
• How a strange finding during Operation Warp Speed, the same data producing wildly different AI outputs, became the founding question behind CHAI
• The gap CHAI's model cards still haven't closed: they work for hospital administrators making procurement decisions, but not yet for the physician with fifteen minutes and no data science background
• Breaking news, live on the show: PULSE, a national initiative giving public health agencies free access to Claude and ChatGPT, plus a health sector specific cybersecurity effort building on Anthropic's Project Glasswing
• Why Brian pushes back hard on "human in the loop" as a checkbox, and what he thinks physicians actually need to know before trusting a model's output

Dr. Brian Anderson didn't set out to run a coalition of 3,000 organizations. He set out to be a pediatrician. That plan held until an EHR rollout at Massachusetts General Hospital showed him a different kind of medicine than the one he'd trained for, one shaped by clunky software and constant time pressure. He tried to fix it the way a lot of physicians do, by choosing his own tools in private practice. Instead, a friend offered him a job helping build an EHR company's product, and within a few years Brian wasn't practicing full time anymore. He was building teams. First in digital health, then in AI.

That path eventually led to the Coalition for Health AI, which Brian co-founded in 2021 after a strange and unsettling discovery during his work on the White House COVID Task Force: feed the same data into different AI models, and you'd get wildly different answers. There was no shared definition of what "good, responsible AI" even meant in healthcare. CHAI started as eight organizations trying to answer that question. It's now more than 3,000, spanning health systems, developers, and patient groups, all trying to agree on standards none of them could set alone.

The clearest product of that effort is the model card, an idea Brian describes simply: if a physician is accountable for a tool that touches a patient, that physician should be able to know what's inside it. Think of it as a nutrition label for AI, information on what a model was trained on, who trained it, and where it's known to struggle. It's already useful for the hospital administrator deciding which vendor to trust with a multimillion dollar contract. What it hasn't solved yet, and what Brian is candid about, is the physician walking into an exam room with fifteen minutes and no way to translate any of that into a decision about the patient in front of them.

Coalition for Health AI (CHAI) Launches PULSE, a National Initiative to Help Public Health Agencies Responsibly Implement AI at Scale

That same instinct, that trust in AI has to be earned rather than assumed, shows up in the biggest news of the episode. Announced the same day this episode goes live, CHAI is launching PULSE, short for Public health Use case and Learning Scaling Engine. The initiative will give up to 2,000 public health practitioners access to enterprise licenses donated by OpenAI and Anthropic, organized into communities of practice around five specific use cases: biosurveillance and drug wave prediction, mapping social determinants of health, operations and community feedback analysis, multilingual public communications, and automated clinical data retrieval through FHIR. Ten jurisdictions, ranging from state health departments to tribal authorities, will be selected from a national applicant pool this fall, with Accenture managing onboarding and CHAI's 17-member leadership council overseeing selection. As Brian describes it on the show, years of underinvestment left public health agencies without the tools or trust to respond effectively, first during COVID, now with AI. PULSE is built to turn "pockets of excellence at the frontlines in public health" into playbooks every other jurisdiction can use, with a public release targeted for next year.

The episode also breaks a second piece of news. Building on Anthropic's Project Glasswing, CHAI is launching a health sector specific cybersecurity initiative aimed at hospitals, which Brian describes as the most attacked critical infrastructure sector in the country. The plan includes operational, defensive, and policy playbooks, with an international scope involving multiple governments, a direct response to years of ransomware attacks and PHI breaches that public health agencies and health systems have had no shared playbook for confronting.

Threaded through both announcements is Brian's real preoccupation: what it actually takes for a physician to trust a tool they didn't build and can't see inside of. He's blunt that "human in the loop" has become a hollow phrase, a box hospitals check without asking what it should actually require. For Brian, that means physicians learning to ask where a model's training data came from and where its blind spots are likely to be, the same instinct that lets a clinician question a colleague's read of a chest X-ray. CHAI is also launching a post-deployment monitoring network this year to start tying model performance to real clinical outcomes, something Brian admits should have happened years ago.

Thank you to our wonderful sponsors for supporting the podcast:

Evidently - Leading AI-powered clinical data intelligence https://evidently.com/

Top 4 Takeaways

Brian Anderson's story started with a single broken system he couldn't fix alone. What he's built since is an attempt to make sure no physician has to.