JUSTIN NORDEN: Welcome back to the Stanford Healthcare AI

podcast, and we're so excited to be joined by Dr. Amy

Abernathy who really needs no introduction,

but physician scientists, formerly

ran Verily, Flatiron, and roles at the FDA

and is now running Highlander Health, which

is promoting companies that generate

new evidence both from investment

and a philanthropy perspective.

And we have a ton of topics to run through today on all

the advancements.

So welcome, Amy.

AMY ABERNATHY: It's awesome to be here with you.

Thank you.

JUSTIN NORDEN: So the first topic we wanted to jump into,

and I'll pull up some data, is all of these new advancements.

And so, Matt, help ground us into all these new things

we're seeing here from the new AI companies and models.

MATT LUNGREN: Yeah, it's really interesting.

I know we talk a lot about the acceleration

of the capabilities of these models, but it's almost--

I don't want to, obviously we keep pretty close tabs on it.

But in general, I feel like we see

results like what you're seeing on the screen,

which again, we can argue whether this

is the way that you would test, quote unquote intelligence.

But nonetheless, we see this repeatedly

that these benchmarks that we've put out there,

whether we're trying to make them harder or more

focused on certain areas, they keep getting saturated.

But I also feel like it's not really

reaching the public consciousness, maybe as

much or at least the physician community,

folks maybe still dabbled in ChatGPT six months ago

or maybe they're using it occasionally

for a quick question answering.

But are we really taking advantage

of what I'm starting to see as is an overhang of capability

versus what they're being used for?

I guess my point is, would we even AGI

if it happened and would anyone even talk about it at this point

because it just seems like the advancements are so fast.

AMY ABERNATHY: I mean, it's interesting

and I'll be curious what the two of you think, as well.

If I start in the beginning, Matt,

and your point is that capability

sets of the models that we have in our tool suite right now

are largely way more powerful than probably how we're

using them as clinicians today or even

understand that they could be used.

And a lot of this is about adoption curve.

You're figuring out how do you put a solution into your day

to day workflow.

How do you understand a combination?

I'm going to leverage this solution

and I'm going to also be able to communicate.

And I'm going to come back to the clinical context in just

a second, communicate what I've learned in a way that now helps,

for example, the person sitting in front of me

know what to do next.

And you were saying that people are largely

just dabbling and trying things out,

but a lot of what is going to be needed for uptake

is that dabbling.

I personally-- some days I use Gemini, some days I use ChatGPT,

it changes around on the day for no good reason, to be honest,

and I know I'm not pulling the greatest capabilities out.

And part of it is because frankly, I'm

still trying to figure out which use cases are most productive.

I'm still practicing how to prompt, quite frankly.

In that particular scenario, my son is way better at teaching me

how to prompt than anything I read in the medical literature

or online, so my 24-year-old is more effective than papers about

prompting in medicine.

So I need to learn how to prompt.

I'm trying to understand also, within the portfolio

of my work, what is legitimate.

So, for example, I am trying to figure out if--

I was working on an editorial over the last three days,

is it legitimate to ask LLMs to help me edit?

Is it legitimate to ask them to help me think?

Is it that they're a brainstorming partner?

Is it all of the above, or should I just tell the LLM,

go for it, write my editorial, I'll see you in a day or two?

And so in thinking about how to legitimately embed LLMs

into my work, there isn't a training manual for that either.

And then you take something that's

way higher risk than writing my editorial, which

is the care of a patient.

Yesterday my problem was my mother

had a series of radiology tests, sent me the tests,

and then wanted me to tell her what was going to happen next.

And I was trying to decide, how legitimate

is it for me to ask ChatGPT to help better

communicate all this information to my mother,

and where's the liability?

How do I know I'm communicating the right information?

It was a radiology test that I actually

didn't know how to interpret, so I

didn't have a way of being sure the LLM was

going to be able to do his job.

So the first part of this is really my saying to you,

I think we're all really learning

how to incorporate lens into our work and our daily life.

How to master them and what's legitimate.

Then you jump to this other question

that you asked, which is, would we AGI if we saw it?

And given the fact that we're kind of bumbling at the moment,

I suspect we probably would know AGI if we saw it.

Even if we were really bullish, AGI is coming tomorrow.

But I'm going to stop there with my soliloquy

and ask the two of you, would we know AGI if we saw it?

JUSTIN NORDEN: It's hard to keep up.

And I think this is something and this

is part of the reason I think we're

doing this is we struggle to keep up

with the current advances, and it's more or less Matt

and I's full time jobs in the work that we're doing,

focusing on AI.

And we're struggling to keep up testing all these new pieces.

And I think maybe the point that you brought up

is so interesting.

And I'll pull up this one other chart

is, the consumer adoption is crazy of these tools.

Everyone is using them at home.

You're a physician, and you're now saying, wait,

hey, should I be using this to interpret radiology results

because it will help explain this to my mom.

But you're uncertain of the capabilities.

And these are just the real issues

that I don't think we are talking enough

about as a community in medicine because we

pretend it's not happening.

Many of the health systems kind of pretend,

well, we're not going to give people access to these tools.

People aren't going to experiment,

but it's happening whether we like it or not.

On just how many people are using these tools now,

and so no, I think to your question,

I don't think we'll know AGI when we see it in health.

AMY ABERNATHY: Yeah, there's this piece

that feels so 2000 to me.

Maybe it's 1995 but I'm an oncologist by background.

And there just became this period of time

when every patient that walked into the clinic

had a stack of paper they printed off, probably on a dot

matrix, but on their printer of things

that they had pulled off the internet.

And there was some combination of help

me make sense of all of this from the patient,

and shame on you for not knowing all of this from the patient

or maybe I was sort of shame on myself

because I was thinking to myself, holy cow,

I don't know what this all means.

And that as clinicians, we felt a little baffled

about what to do.

You could just say, don't listen to the internet.

Well, that seemed kind of a silly point of view.

And for us as clinicians to ignore that our patients are

incorporating LLMs into their day-to-day life,

let alone into their clinical questions seems silly.

We could go all the way on the other side

and say, use LLMs for everything,

but we know that LLMs are fallible.

And so that's probably moving too far

in a different direction.

So guiding patients thoughtfully down this path

is the new 2025 plus task.

It's really been with us for a couple of years now and probably

should look back to how did we adapt quickly in the late 1990s,

and what were some of the lessons learned.

MATT LUNGREN: Yeah, I really like that analogy

because I can't think of a clinical day I've

had in the last 10 years where I didn't use

the internet at least one time.

But in the 90s, that would have been like,

why would I use the internet but I just--

what this progression, I'm feeling

and I don't know if I've articulated this quite fully

formed thought yet, but I feel like computers--

Let's just say we're like, OK, they

offloaded our need to do computation.

And then with internet, I feel like they offloaded our need

to remember stuff at scale.

And then with the social media, it's offloaded,

our attention is now gone.

And I feel like this frontier is our cognition is now

being potentially threatened.

Do you know what I mean?

It feels like there's this slow progression.

Then of course, we can get into robotics, which

I don't know if we have enough context around, but just for me,

how much longer will it be till it's normal for just

like I would use the internet today,

would I use LLMs just for everything,

just interacting with them?

Hey, this is what I'm thinking about doing for this treatment.

This is the test I'm thinking about.

Just having these conversations or frankly, even agents

and abstracting that even further and say, hey,

can you go run and do an analysis on this chart

before I have a conversation?

It's really fascinating, but I think that the part of this

is the daily use, like you said, the

dabbling to get the familiarity and understanding

the capabilities.

And on the other side, the patients are pushing us.

I see it already too.

It used to be like your Google search isn't

a replacement for my medical degree kind of thing

because, I mean, there's a lot of sources.

You could be like, where did you go, but now it's like,

which model did you use, and how am I

supposed to have all the understanding of where

the capabilities of each individual model are?

You we've done some work for these benchmarks that we're just

trying to barely get our hands around and it's-- we know we

can't use MedQA as law in terms of what they can do, but yet,

we keep finding in these head-to-head comparisons,

physicians with AI, physicians alone, AI alone.

That AI just tend to do better than AI plus physicians, which

calls into question, even if I'm using these tools,

it might still performing as well

as I need that this is some work from Google that was published

in nature around their AI AMIE system, which admittedly,

is somewhat more geared towards medical tasks.

But look at these lines, these trend lines.

Even the clinician with that tool underperforms.

And this is not just an isolated result.

We see this time and time again.

In fact, to the point where when I used to be asked,

"Will AI replace physicians?"

I'd be like, of course not.

It's always going to be AI plus physicians.

And we just keep seeing these kinds of results.

Here's another, o1-preview by itself outperforms

physicians using these different tools.

So I don't know the answer, and part of it I think,

is do we have the best practices of how

we interact with these models?

We're kind of fumbling around in the dark right when

we're using them.

But still, I don't know what to advise folks at this point.

AMY ABERNATHY: So I'm going to go back

to something you just said about the sequential offloading.

So I'll start there first, which is

we're at this place of, as you described it,

offloading cognition to LLMs.

And what that is, at least if you

look through the progression you talked about,

it was computation at scale.

It was information finding at scale.

So now it's really integration of myriad, very diffuse

potential data points now to be able to have a moment

of inference or decision.

So that's where we are, and what we start to offload to LLMs

is the ability to reach across all

of those potential sources of information

and make sense of them in the moment.

So that's where we are.

But we're not offloading social interaction,

we talk about, for example, LLMs are kinder

and write nicer messages, but at least I sincerely believe

some of that is that as we've had

a number of new technologies come online, including,

as you mentioned, social media and offloading our attention,

our ability to deal with that overload as humans has gotten

caught up in our ability to be empathetic and emote.

And that LLMs to help offload some of that demand,

so we can get back to being more human

again seems very important.

The second thing that I would say

as it relates to that is that it is not writing nice emails

is not the same as being able to be a confident leader of people.

Off-setting vision and course in a way

that everybody wants to move towards a combined

future of being able to help a scared patient and family

figure out how to navigate a really tough decision.

That likely incorporates a series

of facts for the patient that would not necessarily

be embedded facts if you just were

to look at the usual data set.

And so I suspect that there's some offloading of some

of the demand of emoting and therefore, the opening up

of a new frontier that looks much a lot more like LLMs having

taken care of some of our need to push cognitive overload

to individual data points and pull some of that cognitive work

to the part of the work that's distinctly human.

So that's my-- and maybe that's my glass

is half full point of the world.

But that's where I hope to see us go.

And then the other thing that you brought up,

and the figures highlight is, at least right now,

specific discrimination tasks, differential diagnosis,

pulling a whole bunch of facts together

and making a medical decision about what to do next,

LLMs look better on their own than LLM plus physicians.

But just like anything else, probably

that's some version of physician still

not knowing how to use LLMs, as you said, Matt.

And I think about one of the things

that we're trained to do as doctors.

We are trained as doctors and we go through internship right,

as interns.

And I don't know what clinicians you're

trained as, so I think actually both of you are radiologists,

internal medicine and radiology, is that right?

JUSTIN NORDEN: Matt's matched radiology.

I went into the startup world.

AMY ABERNATHY: Oh, OK.

Well, there we go.

Internal medicine and oncology, and when we basically

train internal medicine residents,

part of the internship is teaching interns

to go from sequential tasks to that moment in time

of having the gut instinct of sick versus not sick.

At that point, clinicians have hit a new plateau

of how they integrate facts and apply it to individual patients.

And that instinctive plateau basically,

is embedded with many parts of our learning and thinking,

including our own biases that we have not built LLMs

into hitting that plateau.

So I would suspect that clinicians

pull LLMs down a little bit because those two things are

working at odds a little bit for some period of time

until we learn as clinicians how to do the structured tasks,

like what we teach interns, that transition to integrated

thinking and fuzzy thinking to then that transition to fuzzy

thinking plus LLMs.

And we just-- I don't know how we hope at Stanford

you're thinking about how do we train this.

It certainly hasn't been something

I've been talking about yet.

JUSTIN NORDEN: We're talking about it.

I don't know that we or anyone has all the answers here.

The recent things that we see over the past month,

you see people like Bill Gates talking about how we

won't need doctors in 10 years.

Folks at DeepMind, Eric, we see all these very, very sensational

headlines.

I agree with you.

There's so many components of the work

of the human connection that won't go away despite what

these headlines are.

But I'm curious to just touch on these few parts, and Amy,

you have such a unique viewpoint of being an oncologist,

having led at these technology companies,

having led at the FDA, and now you're

really focused on evidence generation

and what do we need to show.

And so we pull up these charts, we

pull up these different studies.

The evidence seems to be mounting.

But what evidence do we need to start actually

diffusing this into medicine.

Because as we look at it, or as I look at it,

it seems like we're just getting this divorce between where

the technology is capable of, certainly what we're training

people to do, but then what is actually happening

in our health systems?

What's actually different today?

Not that much.

And so, what is the evidence that we

need to start to change it, to start to make it look different.

And from a few of your hats, and especially,

how does the regulation tie or not tie into this adoption?

AMY ABERNATHY: Yeah, I mean, so many important questions

built into one, Justin.

And frankly, in my mind, made worse

by the current state of overwhelm of the system itself.

So the health care delivery system

itself is not set up to easily adopt and integrate

new capabilities.

Partly, it's just kind of swallowing

water of what needs to happen.

My first observation is, we're continuing

to look for details and facts around LLM performance,

but we have fewer details and facts about LLM implementation.

So the evidence side around what does implementation look like?

How does implementation happen?

Well, right.

And calling it best practices I think

is an underwhelming definition because it

should be around evidence-based optimization

and getting to a fine-tuned understanding.

I just think about how much detail

we have about the embedding of algorithms

of all different types into everything from search engines

to all these things that we use in our consumer lives.

And that comes from many, many different types

of testing causal inference, ABA, like the list goes on,

and figuring out that same mechanism to embed and adopt

really is an implementation science that we really

need to be talking about.

The other part of that implementation

is figuring out implementation plus performance

because as you implement LLMs into clinical delivery settings,

the clinical delivery setting itself starts to change.

And so there now are a series of ongoing underlying

changes including change in the conduct of care,

change in the underlying data that

is being generated that now feeds subsequent action.

And we need to be able to not only test implementation

and study best ways to implement,

we actually need to study how that actually

looks across time in the myriad different settings

where we're implementing.

I think that's a huge part of the work.

Also, embedded in your question, Justin,

was does LLMs and does AI just totally take away our need

for doctors or clinicians.

And I think certainly, we need to define clinicians

very broadly.

And it probably reduces our demand signal

for certain kinds of clinicians.

But if we go back to my point before,

we've offloaded some of the cognition

but we've now increased the space for humanity.

We're going to actually need much more of humans

in the loop to interpret and make

sense for patients and humanity and to point the way.

I still think that's true, but they need a differing load

balance of the different kinds of individuals and capabilities

across the system to get that work done.

And then the last thing you asked

is from the regulatory frame.

I think for the regulatory frame,

there's a lot of struggle that's happening to figure out,

what is the best regulatory approach.

There is those things that are clearly regulated.

So when it's software as a medical device,

and we're thinking about an AI algorithm to read mammograms

and we're--

it's very clear what we need to do

and how we need to think about regulating those devices.

If we look all the way on the other side,

and we look at LLMs to support the practice of medicine

and clinical decision support, it

becomes less clear exactly what the regulations look like.

And all of this right now presupposes a fixed world

where you have to have a predetermined change

control plan or something else just actually

to set a set of parameters upfront

that may or may not really actually make that much sense

or be workable.

So I personally believe that we're

going to see the need for more flexible governance systems,

even both from the standpoint of self-governance

from industry and also some of that

set through FDA in order to set the path of how we go forward.

I know we're at time so maybe the next time we talk,

I give you some ideas of what I've

been thinking along this way, and we can bat those around.

JUSTIN NORDEN: We would absolutely love that.

And I know we've had some discussions before on this topic

about how do we generate the evidence,

how do we think about the monitoring,

how do we start to share, how we govern these things.

I couldn't agree more.

Matt will give you the last word here before we wrap.

MATT LUNGREN: I'm extremely bullish on the idea

that we will come together and come up with a new category

entirely for both implementation,

but I think for the regulatory aspect of this

and how we use it.

So I'll increasingly keep an eye on where the capabilities are.

But my gut tells me that we're going

to continue to have this overhang

for the foreseeable future.

AMY ABERNATHY: And we're going to have to figure out

how to navigate through it.

So thank you for inviting conversations

that at least start to get to the surface

but we got to figure out.

JUSTIN NORDEN: How amazing.

Thank you so much.

AMY ABERNATHY: Thank you.

MATT LUNGREN: Thanks.