JUSTIN NORTON: Welcome to our second episode of the Stanford

Health Care AI Water Cooler discussions,

and I'm Justin Norton, a Stanford faculty and co-founder

of Qualified Health.

We have Matthew Lundgren, our co-host,

who's chief scientific officer for Microsoft Health

and also on Stanford faculty.

And we're thrilled to be joined by Dr. Graham Walker, who

is also a Stanford Medicine--

GRAHAM WALKER: Former Stanford faculty.

MATTHEW LUNDGREN: Former.

JUSTIN NORTON: Former Stanford faculty.

Perfect, who's also co-directs the advanced technology group

for Kaiser Permanente, which is all the advanced technology, AI,

and new things coming.

And he's also the founder of MDCalc,

who I'm sure many people have used,

a new company called Offcall, and many other things.

So thank you, Graham, so much for joining us.

GRAHAM WALKER: Thanks for having me, guys.

Yeah, good to be back on the virtual farm as it were.

MATTHEW LUNDGREN: This counts.

Yeah.

So for today, I mean, so from our first episode,

we had a ton of great responses, some

comments about some technical things, some things

that people wanted to see us cover.

And as you know this is really meant

to be of a conversation that reflects the ones that we have

when we're not on camera and recording.

And we have a few topics today, and then we'll

get into just some general discussion

about what we're seeing today.

So we're going to talk a little bit about GPT 4.5,

kind of one of the newer models that have hit the scene,

although we have a bunch.

And then we'll talk about the bunch as well.

We kind of have a paralysis by choice

at this point with so many different models choose from.

And then we'll talk about some new data that's

coming out around physicians and their use of both

the public models and even some of the solutions

that have been deployed into practice and get full stake.

And there's just a few odds and ends,

some headlines that have come our way that have

raised a lot of discussions.

One around some proposals around regulation

and maybe even new capabilities for AI models

prescribing capabilities, which is kind of interesting idea,

and we'll get everyone's reaction.

JUSTIN NORTON: Sounds good.

Well, should we start with some of the new capabilities?

MATTHEW LUNDGREN: Yeah, let's do it.

I think maybe it's worth it.

And Graham, feel free to chime in on this.

Yeah, I just wanted to just to level set.

Well, so for those of you who aren't

familiar with the law behind the TikZ Unicorn,

this is Sebastien Bubeck.

This is a hearkens back to a paper he wrote, Sparks of AGI

basically paper.

The original GPT 4, which surprised a lot of people

that the model was able to put together,

as you can see in the middle, something

that resembles a unicorn.

Again, this was a language model.

This was a shocking result back in 2022,

and obviously a leap from 3.5.

So to in the recent 4.5 sort of evaluations,

he posted the new kind of updated impression of a TikZ

Unicorn just to give a visual sense.

But if you actually look at some of the data around the model's

performance, again, we're not seeing this massive leap,

but remember that some of these benchmarks

are relatively close to saturation in certain areas.

I think for some of these simple ones, some important things

to take away.

We're used to these huge jumps between three to four,

and we're seeing some maybe somewhat incremental

or slowing improvement.

And this is really reflective I think

of the pre-training, the traditional approach

to pre-training, the scaling laws of exponents input

for somewhat linear improvements,

so it gets-- these models get bigger and bigger.

They become harder and harder to make, quote unquote, "smarter,"

but we have some other things to work with and test time compute.

But on the right hand side, which I'm really excited about,

which is that the hallucination rate, which does

tend to limit or really force you

to build a lot of scaffolding around applications

off the shelf is starting to look quite a bit better,

and I think a lot of people would agree

when they're using the models.

I don't know Graham or Justin, if you've

been playing with any of these newer models

and noticing that the stuff that you

used to pick up all the time, this thing was wrong

or this seemed off.

It's starting to look a little bit better.

GRAHAM WALKER: I mean, Matthew, I'm a little surprised that it

still is 37%.

I'm not super familiar with the simple QA data set,

but I mean that still seems way higher.

I think there's probably two things going on.

People are more comfortable with the LLM responses,

and so they're probably checking them

less to be completely honest.

These things have met some sort of acceptable criteria

for humans to start using and trusting.

I mean, I think Google's partly to blame for that too.

I mean, they've just been embedding them

in all your search terms.

I mean, often I'm now seeing-- like if I'm

like getting in an argument with a friend,

a friend will send me the Google answer.

But it's not the Google answer, it's the Google AI answer,

and that's not really the same thing.

It used to be, oh, yeah, here's the screenshot

of the fact from the web page, or it's from census.gov.

Nope, it's now the screenshot.

And I don't know if my friends know

that they're sending an AI version

or if they are just implicitly trusting them,

but that's the thing.

We've reached some level where people are just implicitly

trusting these things.

I'm not sure that humanity should be trusting them just

yet, but it seems like we're there.

JUSTIN NORTON: Yeah, there's almost a--

MATTHEW LUNDGREN: Yeah, there's a trust but verify,

I think, to at least for those of us

that use them all the time.

And in this metacognition thing, where you're like,

OK, I know it's going to get me like 80% of the way there,

but it's still going to save me some time.

But the worst part of this is what we would call the GPT slop.

I think which by the way, I think

was in the running for the term of the year last year

Webster's dictionary new term.

But that's where-- I mean, I've gotten to the point,

I think, there was been some data on this

where I can read some of these posts

and say that looks like a clod or that looks like a GPT 4

to me, and you can tell there's not

a lot of effort put into the editing.

GRAHAM WALKER: Well, this is the thing

I want to pull up that kind of brings both of those things

together, which is one, there are so many of these models

out now.

Or what's changed in two years?

Versus, oh, my gosh, look at these capabilities.

They've come out.

One, there are so many models out there

now and competition for that.

That Graham to your point, the companies are just

embedding the AI capabilities in their native applications,

which lots of people thought would come.

And the reason to do that is to keep people in the platform

and it's a way to compete with others.

And so there are so many of these models

out there for I guess most of us.

And people listening, it's reminiscent of the old search

pages where you'd use AltaVista and Ask Jeeves and just

a long list of companies where you'd

ask multiple times to try to get to your answer

before Google really kind of won out.

And as we're looking at these models, a question

that I get all the time is like, well,

what model do I use for what?

And the challenging part of that is it's

changing every few weeks.

It's not just GPT 4.5 that came out.

It's Cloud 3.7 just came out, and there's

new models every couple of weeks and so that's changing.

But the interesting thing to me, at least,

is it seems like this is moving to a world

where there's going to be multiple models to choose from,

both for a consumer, both for companies, both

for health organizations.

And so when I think about it, how do you set yourselves up

to use multiple different things?

And to be able to access multiple different things

is something that I think is super important, but curious

Matt, Graham, what do you guys think about that?

What are you using for yourself?

MATTHEW LUNDGREN: What you're showing on the screen

is I was just doing the back of the napkin.

That's about $300 a month of subscriptions.

GRAHAM WALKER: Tools.

Yeah.

JUSTIN NORTON: Assuming you're not paying the $200 a month

for OpenAI pro.

GRAHAM WALKER: For the pro.

Yeah, I'm mostly using ChatGPT and Claude.

ChatGPT tends to give me answers that are a bit more unique.

And then I'm using Claude really around more editing capability,

when I want feedback on my writing,

but I don't want my writing to change as much.

ChatGPT is more than happy to tell me

that oh, I love this article that you just wrote,

and then rewrite it in its own words.

And it's like both complimenting me,

but also it's like a backhanded compliment

that it's redone the whole thing because it wasn't that good.

Versus Claude typically will stick with my own words

and then try to clean them up, make them a little bit more

cohesive without changes changing as much,

so those are the two I'm using the most.

I've played around with Deep Seek a fair amount.

I found that Deep Seek when I've just been testing Deep Seek.

Deep Seek will let you get away with more stuff,

and it'll be a little bit more-- allow a little bit

more devious behavior, which is probably

a factor because open source and not as the weights are all open,

so I imagine they probably have to allow more of that.

And I think that's the testing that I've seen as well.

Like you can get Deep Seek to tell you

how to rob a bank much easier than you can Claude,

or ChatGPT, or some of the other kind of closed models.

MATTHEW LUNDGREN: Have you tried them all?

I mean, you still practice, and so do you ever throw in

and just see if it comes up with a differential that

seems reasonable?

Do you ever do that or--

GRAHAM WALKER: The difference--

MATTHEW LUNDGREN: Yeah.

GRAHAM WALKER: Yeah.

The differentials are always reasonable.

They so far they've-- and I'm excited to try out 4.5.

Matthew if you want to give me access?

The models all have some gap that still makes

me very leery to rely on them.

I think, they're content generation,

so they're great for generating a differential.

But I'm not yet convinced that they're

going to be a comprehensive differential

or that the number one thing is always

going to be the correct one at the top.

And then you add one more complicating factor

like, oh, it's a kid, or it's a pregnant patient, or something

like that, and then they start to show a little bit more

of their challenges, I think.

But, again, growth mindset, these things are--

every year, these things are getting better and better,

and it's harder and harder to find

those challenge-- find those areas where they're not as good.

MATTHEW LUNDGREN: Yeah, I remember when 3.5 came out,

you'd see these viral threads.

It's super simple medical questions.

It'd be wildly off base.

And like I said, I mean, obviously I use it all the time.

I partisan red teaming work and things

and it's increasingly hard.

In fact, it actually stretches some

of your own medical knowledge, and then

you have to go back to the source like that sounds right,

but I need to double check.

And that kind of gets me to deep research.

And I don't know if you've played with those Graham

Richardson, but I've had moments of just

like almost just like the initial GPT

moment with deep research, where I'm like,

this is a game changer.

And then there'll be times when like,

it's a pretty superficial read of some of the content,

but you know where this is headed.

I mean, to your point, this is the worst it'll ever be.

That kind of thing you hear said a lot.

But if you talk about access to all of your journals and all

the things that we subscribe to and our medical libraries,

now you're talking.

I mean, that could be pretty powerful.

And I don't know if you've used these, and kind of tested out

with topics that really well to get a sense of how close it is?

JUSTIN NORTON: Yeah.

Well, speaking of medical journals and access,

there's a recent announcement, a headline OpenEvidence just

raised a boatload of money.

And they claimed, I think, 25% of physicians

were now using the tool.

In my head I think about it as an up to date competitor

of being able to ask open ended questions,

cite this with real research, reduce that hallucination

rate, which was something that came up before.

Drop that down where initially when

people were asking questions with GPT 4,

I was like, oh, my gosh, look at all these citations.

Then you look at the citations and they were all made up.

But as the scaffolding, I think that Matt talked about before

around these models, where you constrain what they're

able to look through to only be able to produce certain things.

It's not just that the models are getting better,

basically as software developers.

And I feel like this is actually something a lot of people

miss when they're talking about, at least in health,

of people who aren't maybe familiar with the technology.

It's like, oh, did the models just get better?

It's like that is one aspect of how

these tools are getting better.

We're also getting a lot better as engineers

for how to use them and use them in a way that actually produces

meaningful results, and so that's just

an example recently of something that has really kind of gained

a lot of traction in the last few months and years.

MATTHEW LUNDGREN: Yeah I was going to say,

Graham, have you ever tried to throw in prompts to this say

cloud or something and just say create an MDCalc from scratch?

Have you ever tried that?

GRAHAM WALKER: Totally.

Yeah., and there are some calculators

that it's been able to do really well.

And there's others that it's like, oh,

this is dangerously bad because it's going to--

I mean, that's the other thing that I

think about a lot is it's that level of trust.

I mean, it's the Google embedding it in search results

thing that I do worry that even with hallucination rates

going down and scaffolding getting better,

you could feed the model back to itself

and verify that the answer is-- that the model thinks

the model is accurate.

I still do worry that somebody is

going to make a decision based on an LLM piece of information

that is maybe it's not wrong, but it's just not comprehensive.

And they do not have the training or the experience

to know that, oh, yeah, usually you can give that drug.

In this particular circumstances,

that was a really bad idea.

And either the model gave you the wrong information

or you didn't give the model enough information for the model

to give you the right answer.

I mean, pregnant patients take--

for some reason, I'm thinking about digoxin toxicity

and giving them calcium and stone heart, which

I think has mostly been debunked,

but still like you can imagine a human doctor is going

to have to try to take in all the pieces of information

and then take an action.

But if you're just giving it the two liner from your H&P,

that's often not sufficient information

to give a right answer to.

JUSTIN NORTON: And I think the usage is interesting.

But one thing actually, that has come up

before and through a ton of conversations, Graham,

to argue the other side as compared to what?

So are there going to be AI misses?

Yes, yes, and yes.

Often it'll come up and usually they're overblown,

but you always hear the media headlines of look

how many mistakes physicians make.

And now we're starting to see the articles of patients

working with AI to get the diagnosis

that the doctor missed.

How do you think?

What should we compare that AI against?

Or even for yourself.

How are you comparing it for yourself,

for when and where you should use these tools?

GRAHAM WALKER: Yeah, I think of the Tesla full driving

capability thing where I don't really know that--

it feels ethically because it's like the loss of control

that these tools need to be not just as good, not

just one point better than the average human driver.

Because you can always argue that the average human driver

may have hit the brake fast enough or something like that.

It does feel because you're losing the control factor,

it feels like they need to be at the 99.9 percentile of the best

driver in the world.

And then the other thing I always

think about is to your point, Justin,

we can't think about risk of tools in a vacuum.

We have to think of total risk.

I mean, it's like when I give somebody a blood transfusion,

I tell them your risk of walking across the street

after I get you feeling better and you get hit by a bus

is way higher than your risk of HIV or hepatitis

C, which are, of course, the two most common things

that people worry about.

But they're like one in a million, one in 10 million,

and so you have to think about all the risk.

And certainly the other risk is patients waiting six months

to see a cardiologist, because there's a backlog to get seen

and the patients are going to get sicker in that time.

So I think we do have to think about, consider all the risks

and not just think about the risks of AI

or just think about the risks of humans,

but think about the risks of being

a patient in American healthcare.

JUSTIN NORTON: Yeah.

The interesting thing that I want to make sure we

get to here on people using these tools and how and why.

You brought this up a little bit,

but this was a recent survey.

The AMA published a second generation of AI use.

And they call I use interestingly, not

artificial intelligence, they call it augmented intelligence

but across clinicians.

And this number was shocking to me.

Before and it seemed to be around a third of clinicians

were using AI in their practice.

Now that jumped to 2/3.

And the context I'm always still shocked by this is 2/3

of clinicians are using this.

Most work settings haven't given people access to these tools.

And so it's just this amazing.

It's amazing things.

And I know we've talked about this before,

but it's like people are going around and using

phones, using other things, but what do you make of this?

GRAHAM WALKER: I would love to see if this trend is

true in other countries.

I tend to assign a lot of the things that we see in the US

as due to our very dysfunctional health care system.

And so like I see the rapid adoption here

as a sign of doctors in the US are really

struggling to keep up with all of the stuff,

whether it's clinical, or administrative, or prior auth,

or anything like that.

So I would be fascinated to know if this trend is

an international trend?

Is the UK--

I mean, actually every country's physicians and nurses

and healthcare workers in the whole world are struggling.

Many are quitting.

Many med students across the world

don't want to practice at the bedside.

So it could be that this is a wide trend,

or it could be like American medicine is particularly bad

and so doctors are way more likely to use these in the US

than other places.

MATTHEW LUNDGREN: I mean, I feel like we

talked a little bit about this last time, Justin, too

which was the sort of British Medical Journal had a paper

on surveying GPs and the NHS.

And they weren't quite as granular as maybe

what I'm seeing here.

And they were actually focusing on just the public API,

basically the public models that you'd have put on your phone.

And I think the surprising result that we talked about,

which I think is echoed here, but just I can't quite

match the results there to here as well, because I'm assuming

some of these are software that was sold from vendors that

have vetted the capabilities.

But I guess almost a third of these docs

were using it for what you would consider a medical device.

Something that would be a clinical decision

support device and that.

That's surprising, but also at the same time not.

Of course they are, you knowl what I mean?

At some level, to your point, they're stressed,

they're working through things, and maybe

they're finding these tools useful.

I'm hearing stories of folks are literally just kind

of using the voice mode around in ER settings,

urgent cares, and basically kind of almost doing

like you would when you were an intern or a resident

presenting the case and then getting feedback.

And I've heard other stories of folks,

hoping to have one of them on the pod,

because they're doing a study on this.

But remember in the old when we were in med school,

at least when I was, it was kind of a novel idea

to bring an infectious disease or a farm a pharmacist

actually with you on rounds, particularly in ICU

to look at the huge list of meds we're putting patients on

and give feedback.

And now, a similar thing is happening,

but they're literally wheeling around GPT basically and having

it listening to the discussion and then make comments.

I think it's a fascinating idea.

And again, just kind of brings the point of maybe

we don't know how to practice medicine and take advantage

of these tools in the best way yet.

GRAHAM WALKER: Yeah, it's still very, very early days.

JUSTIN NORTON: Yeah, what's interesting about that though,

is this one other slide and then I

promise I'll put away some of the AMA data here.

But the definite advantage group to me

is what I'm tracking in my head.

Obviously, scribing has gained a ton of momentum

over the past couple of years as these tools have gotten better.

And there's still a mixed, people have different opinions

for how helpful or where it's helpful and things like that,

but there are certain people who swear by it now.

And there are certain people, who

started to use these other AI tools that are absolutely

seen advantage.

I've talked to former students who've

taken generative AI medicine course who are using

these things all the time.

And they think they save, just from the public tools

where they're trying not to put in any patient information,

a couple hours a day when they were an intern, because it

helped with discharge summaries, planning tough conversations

with patients, looking up current evidence, and things

like this.

And so the reason I track this kind of definite advantage group

is it's kind of a bellwether for what's to come.

There's always this kind of curve of adoption of technology.

But when you track that group and people are absolutely

seeing the benefits, to me, it just says it's a matter of time

before this gets more widespread,

and that's one of the things that Yeah, again, this

is the worst it will be.

And so we're really already seeing

benefits that if you scale it to the issues

that Graham was talking about around burnout issues,

people leaving medicine, it has started to at least give me

some hope.

GRAHAM WALKER: I think we need to have more people sharing

how they're using them.

I mean, Matthew, I'd never heard of that kind

of like a virtual AI pharmacist idea.

Love it.

I think there's so many ways that these tools can

be used that people just aren't thinking about.

I mean, I was showing my mom like,

oh, you could talk to it in Spanish.

You want to practice your Spanish?

Just ask it to do super beginner Spanish with you and it'll just

do it, and she hadn't even thought of that

as a possibility.

And I had like AI had a challenging palliative care

discussion with a patient in the ED, and I like--

the day later it was still bugging me.

I was like, God, could I have done that better?

And so I kind of described without, of course,

PHI or details I described how the patient and their family

was feeling and what I was trying to convey.

And I said, hey, can you pretend to be this family member?

Be the daughter of my patient, and then I'm

going to talk to you.

And then when I say, that's a wrap or something, let's pause

and give me feedback.

How did I do?

And so there are just--

shout out Sanford did my simulation medicine training

there, but there's so many ways we

could use these tools outside of just clinical medicine.

Think about simulation.

Think about difficult conversations with patients.

Breaking bad news.

AI pharmacist.

I heard somebody, a guy at Yale is

using a bridge with med students and having the med student write

their own note.

Having a bridge write the note, and then they

have to critique what they liked about their note

versus a bridges and why?

And why did a bridge include that here in the HPI,

but you put it in social history?

There are ways we can use these tools not to replace us,

but to teach and educate and help us, for sure.

MATTHEW LUNDGREN: Yeah, the education topic

is one that I'm surprised I don't hear more often.

I mean, you brought up the really, really important use

case, which is the empathetic coaching.

There's been a few papers on this.

I think that's a phenomenal use case.

First of all, we don't get that many turns of battle.

Thankfully in a lot of our specialties,

except for obviously palliative care or something,

where we would get seminars from folks in palliative care

who do this as experts, and they would come and give us

some tips on how we can have these hard discussions

with patients.

But hearing it a couple times, maybe reading some--

but then actually when you're standing

there having to have that discussion,

man, I would almost take any sort of life preserver

to help me coach up for those kinds of discussions.

But on the education topic, there's

been some phenomenal papers that just aren't really

getting a lot of attention.

I'm surprised by one--

I don't know if Justin, you have that one

that it was a news article that recently talked

about a school in Nigeria that used GPT 4 just as a tutor,

and two grade levels of advancement

within like a six week intervention, which

is insane to me.

And then obviously, just like you,

I mean, you talked about your mom.

My daughter takes Japanese and her--

we obviously live in Palo Alto, so everyone's

thinking about GPT anyway.

But her teacher just said to the students straight up,

you need to be practicing your Japanese conversation

with the advanced voice mode of one of these models,

and I think that in it's been just tremendous.

I wonder whether there's a medical equivalent.

You mentioned a couple of use cases.

I remember back in the day, we used

to just go through question banks for the step,

but these models is I think Nigam Shah's group showed

can actually write step questions that

are indistinguishable from real ones.

And actually, when you score the people on them it's equivalent,

so it really is a phenomenal study.

Again, I feel like there's a lot.

Maybe this is going on again under the covers

and we're just not hearing as much about it, but we're just--

it's early, I feel like is what I end up concluding.

JUSTIN NORTON: The other thing, I'll just call out,

a few from students have come up over the years

from learning just explaining concepts.

Hey, explain the Krebs cycle to me.

No, no, no.

Explain it like I'm an eighth grader.

No, no, no.

Explain it like I'm 6.

And you get these different reps and versions of these concepts

that these models might have an inherent ability

to tackle from different perspectives.

That's what an expert teacher does.

They have such a good grasp of the material.

They can get six different ways to explain it,

and that's what these models can do.

And to Graham, your point before,

people are still just learning these different ways

to interact.

And what's scary to me, kind of seeing

this is when I hear students' use of these models,

they're all across the spectrum.

Some are using these models 10 hours a day in their learning.

It's always on transcribing lectures, summarizing lectures,

asking questions, translating teaching.

Others are barely using them at all,

and the gaps of just what people can

do and understand with these models that alarms me

a little bit, especially when we see how much more some people

can get done with them.

And so just-- and that's part of the reason we're doing this

is, hey, great.

Maybe Graham's going to try advanced voice mode on rounds

with a pharmacist next time.

Or people are going to get ideas for ways to do this,

but hopefully people will do so safely.

I'll just I have that disclaimer.

MATTHEW LUNDGREN: I hope the models get smart enough

to tell the student, you don't need to know the Krebs cycle.

It's all a scam.

You'll never use it again.

GRAHAM WALKER: I mean, Matthew, that's part of that--

that's part of this.

I have this idea that these models will be--

if we train them appropriately without misinformation,

these models will be more objective arbiters of the truth,

and they may actually drive us to reform the medical education

curriculum.

I mean, I still why were we all taught the brachial plexus?

Not just that it exists, but the chords and the--

I don't even know.

I remember memorizing those, and I remember a bunch of mnemonics

to memorize them.

But why do I need to know how those lines connect and then

overlap and cross?

I don't know.

I don't think either of you could answer either,

but it's part of tradition.

I mean, have you guys seen one of the first aid for step 1 now?

It is at least twice as thick as when I bought it

in 2004 or something like that.

I mean, it's insane how much more they have to memorize.

And I mean, I thought it was bad enough

to have to answer like stuff about cyclic GMP,

but now it's so much worse that it may--

these tools may force humans to disrupt some of our tradition

and be like, oh, well, it's important

that people know that where the nerves are on the body.

But do they need to the way that the brachial plexus connects?

Maybe not.

The Krebs cycle.

It's important to know that thing exists.

I think it does like oxidative phosphorylation,

or is that the other one?

But anyway.

MATTHEW LUNDGREN: That's not bad.

Yeah.

GRAHAM WALKER: Anyway, we need to know

what that is because aspirin overdoses block that.

But besides that, do you need to have memorized the whole thing?

It's like a running joke.

MATTHEW LUNDGREN: Totally.

Well, I will say that.

Well, there's two-- I feel like I have two minds on this.

One is like, I feel like we are LLMs

in some if we want to use this analogy,

and we're doing we have to do some pre-training.

So there's some just like exercising

the muscle like that we do.

I sort of still am happy with the way

that we at least attempt to get as much information.

It forces you to become someone that

can absorb vast amounts of information,

process it, and still be able to manipulate the concepts.

There's something to that.

I think in terms of just as being

able to functionally keep up, and to your point

with the acceleration.

But there's also this idea of skill atrophy,

which is kind of real.

I mean, I haven't used a map in 20 years.

I probably could barely find my way outside of my neighborhood

anymore without a GPS, and so that's

real to how much does that impact me?

Not maybe as much.

Am I more efficient?

Sure.

So there's got to be some balance here, but where that is,

I don't know.

But I do agree though, to your point that the medical education

system, just like all the rest of I would say,

the traditional education system is kind of staring into this

and wondering, what does this really mean for them?

JUSTIN NORTON: And with that said, there have been people--

people have been asking this question about medical education

for a long time.

And I'll give a shout out to Dr. Charles Prober, who

flipped the classroom, did a lot at Stanford and really

across the nation.

And we've had the internet and ways

to get access to raw memorization.

Let's have people apply this information through cases,

learn in the classroom together to do that.

And so to me this is just a supercharging

of those concepts, which is let's

get people more into real simulated situations,

whether simulated with AI and use AI to advance it.

It just kind of really accelerates that.

Maybe the one last topic to call out before we wrap

is there was this bill proposed on AI

being allowed to prescribe.

Matt, I'll start with you if you wanted to go through this.

Walk us through what it was and what you think

is going to happen.

MATTHEW LUNDGREN: Well, I mean, I

think that-- so this isn't the first time,

but this one did catch headlines.

I think just given the kind of current conversation around AI

and health and all these things that I

think are starting to be more mainstream.

But I guess the general thought is that a model,

it could legally prescribe a medication.

And I don't know exactly how you'd even really

implement that per se, just given how things would work.

But I can imagine a scenario where

there's some confined scope of certain medications, where

almost like you're starting to see with some of these apps that

are willing and fairly, I guess, they kind of loosen

the rules in terms of how deep you need

to go with your medical history and your physician

to actually get a medication that's common.

But I think it's something to that effect.

But now imagine you could have-- your app also has this ability

to then literally prescribe a medication after interaction.

Again, I don't see this passing, but I always

look for these kinds of signals that

are like that what Ethan calls the jagged edge,

the jagged frontier of where the capabilities are starting

to meet some real world impact that

has pretty big ramifications if we let it continue.

GRAHAM WALKER: I do think the regulations that require--

I mean, just getting back to I mean,

the same traditions of medical education

are the same legacies of medical licensure.

I mean, the fact that we all need a California license.

And if we drive to Nevada, we need a Nevada license.

I think those structures are starting to break down.

I would love to have a hospital be able to prescribe.

Have a hospital owning the risk.

And yeah, if the hospital wants to use an AI to do that,

give the hospital a license to prescribe medicines

or to dispense medicines or something.

And I agree, Matthew, if you look

at many of the telehealth providers, what

are those prescribers doing?

They are as fast as possible clicking boxes that have already

been evaluated and checked by the patient, which then have

routed through an algorithm to say this person

is appropriate to receive Prilosec or something.

I mean, and then they're writing a prescription

and then they're charging their insurance a bunch of money

for it, and then they're filling the prescription.

How much labor is the human doing in that piece?

Not much.

Often those medicines are over the counter,

and so it's a little bit of a shell game anyway.

I don't support AI prescribing anytime soon,

but when you start to-- again, looking at the whole picture,

when you start to compare it to what we're allowing companies

to make billions of dollars doing for prescribers that

are sitting at home, how different is that?

I don't know.

JUSTIN NORTON: Well said.

Well, anyway, with that very optimistic note--

GRAHAM WALKER: Always.

That's all you get for me is optimism.

JUSTIN NORTON: Well, we'll wrap it there, but thank you so much

Graham for joining us.

We'll have to have you back soon as we're all

trying to figure out what's going to happen next

with health and AI.

GRAHAM WALKER: Anytime.

Thanks for having me, guys.

MATTHEW LUNDGREN: Thanks, Graham.