The Missing Data That Could Change Oncology with Dr. Caroline Chung

Beyond the Chart Podcast Guest

Dr. Caroline Chung, MD  

University of Texas MD Anderson Cancer Center, Co-Director of the Institute for Data Science in Oncology; former Vice President and Chief Data & Analytics Officer; Tenured Professor, Radiation Oncology and Diagnostic Imaging; Co-chair, the American Society of Clinical Oncology (ASCO) AI Community of Practice

Key Insights

• When clinicians filter out details that don't yet seem clinically relevant, they lose them for good.
  One patient's story about her aunt's farm might read as background color. But it becomes a research lead the day a cousin from the same farm shows up with the same diagnosis. Once a detail is left out of the note, there's no way to recover it later.
  ‍
• An AI model's accuracy threshold depends entirely on what it's replacing.
  There is no single bar that every oncology AI tool has to clear. Something that's right 90% of the time can be a genuine safety net in a community setting with no local oncologist, but a distraction inside a cancer center where physicians are actively writing the next set of NCCN guidelines.
  
  
• Most AI models are tested on data that doesn't resemble a real EHR.
  Dr. Chung draws a distinction between "real-world data"—the clean, curated datasets used to train and validate models—and what she calls "real real-world data," the actual mess of a live chart. The harder it is to get your training data clean, she argues, the harder it will be to use your model in practice.
  

• In oncology, the biggest blocker to AI adoption is rarely the technology.
  Dr. Chung's experience building MD Anderson's data governance program points to process and operational clarity as the real gating factor; specifically, whether a care team has agreed on the problem it's actually trying to solve before it starts evaluating vendors.
  
  
• Long before "context engineering" was a term, MD Anderson built  and trademarked a system by that name.
  ‍The Context Engine, MD Anderson's data management framework, predates the current AI vocabulary by roughly five years, built on the same principle: A model is only as good as the metadata describing how its data was captured.
  ‍

What's the difference between real-world data and "real real-world data" in oncology AI?

Dr. Chung's core distinction is between two kinds of real-world data. The first is the readily available, curated version. It’s clean, structured, ready for a model to train on. The second, which she calls "real real-world data," is the version that actually lives inside a hospital's EHR. It’s inconsistent, incomplete, and shaped by dozens of local conventions that never made it into a data dictionary.

As an example, she points to something as simple as counting a hospital admission. Some systems count any portion of a calendar day as a full day of admission, others use a rolling 24-hour window. Neither definition is wrong, they just answer different operational questions. But a model trained on one and deployed against the other will misfire.

Dr. Chung’s advice to anyone building oncology AI is to measure the gap between the curated training data and the live data before you measure model accuracy. If it takes months of cleanup to get your data model-ready, think of that as the same amount of friction a health system will endure to actually use your tool. That gap, not benchmark performance, is often what determines whether a model gets adopted at all.

Is there a universal accuracy threshold for AI in oncology?

Dr. Chung's answer is no, and she uses the same hospital to illustrate both extremes. At a comprehensive cancer center staffed with oncologists who are actively defining the next set of treatment guidelines, a model with 90% accuracy can be a distraction: it competes with expertise that's already ahead of the benchmark.

But in a community setting with no local specialist, the same model, performing at the same 90%, can be the difference between a patient getting some informed input and getting none at all.

"I don't think we can say, 'The threshold has to be 90% across the board' because 90% is not good enough if you're surrounded by experts literally defining the next threshold of success."

The comparison she and Matthew Ko return to throughout the conversation is autonomous vehicles. Although self-driving cars already cause fewer accidents per mile than human drivers, every incident makes headlines in a way that human-caused accidents rarely do. 

The idea is not that oncology AI should be held to a lower bar. It's that the concept of "good enough" is a moving target defined by a specific care setting, not a fixed number that applies everywhere.
‍

What details get left out of a clinical note?

Here’s where the conversation turns from data architecture to something more personal. Dr. Chung describes a patient who shared a detail about her aunt's farm. It was the kind of scene-setting that most physicians would leave out of a note as "extra social information," because it doesn't obviously bear on a diagnosis in the moment.

Years later, however, that detail matters. If the same aunt is later diagnosed with the same condition, and then a cousin from a neighboring farm develops it too, a pattern emerges that the medical record has no way of surfacing—because nobody thought to capture it the first time.

"You filter out what you think is clinically relevant based on what’s known in medicine today, rather than saying, 'What if we don't know whether that detail is relevant or not?'"

Dr. Chung is careful to note this isn't a failure of any individual physician. Even a careful, well-rested clinician filters information based on what's considered clinically relevant today. But today's definition of what’s relevant keeps changing. MD Anderson's own microbiome research program is a case in point, turning questions about diet and probiotics that used to be irrelevant small talk into active areas of study.
‍

Why do most AI adoption failures come down to process?

Dr. Chung's advice to health system leaders evaluating AI vendors starts with one question: Has the team that will actually use this tool agreed on what problem it's solving? It’s a question that most governance committees skip.

In Dr. Chung’s experience, early AI adoption in healthcare tended to bolt new tools onto existing workflows as standalone widgets. But that’s just one more system to check, one more login, one more source of alert fatigue. Every tacked-on addition made the underlying process more complex, not less.

"I always come back to Einstein saying 'If I'm going to solve a problem, I'll spend 95% of my time defining the problem and five on the solution.'” 

The alternative she's seen work at MD Anderson is smaller, faster-moving teams who define success before they start any pilot, and who are willing to redesign a workflow around a technology rather than force the technology into an unchanged one.

She points to Einstein's framing of problem-solving, in which you spend the bulk of your time defining the problem and the solution takes care of itself. For Dr. Chung, that’s the discipline most AI evaluations skip.

Matt connects this directly to what DeepScribe has learned rolling out ambient documentation: The tools that succeed are those that weave clinical intelligence into an existing workflow, not another widget competing for a clinician's attention.
‍

What should be next for ambient AI in oncology?

Dr. Chung closes the conversation with what she calls a photo metaphor, that even a well-captured clinical encounter is like a Polaroid, just a single frame in a much longer story. The opportunity ahead, she says, is connecting that frame to everything else—from family history to prior encounters to wearables—to build a fuller patient picture over time. In parallel, it’s essential to acknowledge the privacy and security questions that come with capturing more.

“The clinical interaction is like a Polaroid in the movie of life. You're getting more contextual information within that interaction, but it's an intermittent piece."

As Matt closes asking Dr. Chung what she wishes technologists and oncologists understood about each other, her answer doubles as a closing thesis for the whole conversation. For Dr. Chung, both a clinician and data leader, neither side should assume there's only one way care can be delivered. In fact, the fastest way to build something that works is to set that assumption aside entirely.

‍

‍Subscribe to Beyond the Chart to be notified of future episodes.‍

‍Learn about DeepScribe’s ambient AI built for oncology. 

You may also like:
Startup Thinking Meets Scientific Research in Oncology with Alicia Zhou, PhD

How Do You Build Technology That Oncologists Actually Trust? Two Tech Founders Talk

‍

‍