- Welcome to "The Minor Consult" where I speak with leaders shaping our world in diverse ways. I'm your host, Dr. Lloyd Minor, Dean of the Stanford School of Medicine and Vice President for Medical Affairs at Stanford University. Today, I'm excited to continue our special series of episodes called "The Big Idea", spotlighting some of society's most critical issues and transformative solutions that promise to lead us into the future. In this episode, we're asking the question, how can artificial intelligence and mixed reality open new frontiers for healthcare and human health? Our guest is Dr. Freddy Abnousi, Vice President of Health Technology at Meta. Welcome to "The Minor Consult", Freddy.

- Thank you so much.

- So, Freddy, tell us about your background. You have a remarkable background. You're a physician, you've studied a variety of different subjects and topics in addition to medicine. And we'd all love to know at the outset here how you got to where you are today and the path that you followed to get to where you are today.

- Oh, sure, happy to share, and thanks so much for having me on. So, let's see. I'm a interventional cardiologist, and I've been at Meta for almost a decade now. So, that's me currently. My path. I suppose I sort of dabbled in a lot of things trying to figure out how to potentially make a impact in as many people's lives as possible. And so I guess your question around what I ended up doing, I did a bachelor's degree and then a couple of graduate degrees and then medical school, and started out in surgery and then went to where all confused people go as far as I can tell, which is McKinsey, and spent some time there trying to figure out basically what to do with my life. And one of the things I realized very quickly was how much I miss taking care of people. And so I came back and completed a residency and then got really sucked into the amazing life of cardiology, interventional cardiology, and how interesting that was. And particularly at Stanford, as you know, it's the birthplace of so much innovation in interventional cardiology with Paul Yock and all the work he's done over the years. And so it was so attractive to do that work and be able to take care of patients, et cetera. But the place I've always come back to is, gosh, can't we do more? Can't we do more to prevent these people from showing up in the cath lab in the first place? And there's a number of different approaches to that, and the one I've settled on is to try and impact that life and that health through tech.

- That's great. Freddy, you recently wrote in "STAT News" about the evolving role of data in healthcare and how clinicians would be better equipped to serve their patients if they could access data sources beyond the electronic health record. How can we get to this next era of digital health, and what pitfalls do we need to avoid?

- Thanks for the question. The thesis of the article basically is that lived experience matters, and we as clinicians believe that and know that, and that's why we spend so much time on it. And when we see people in clinic, we spend, let's say you have a 30-minute appointment, five to 10 minutes is on the physical exam, 15 minutes or so is on the history and understanding the patient and their background, and a minor portion of time is on the plan. So we invest the majority of our time trying to understand people and how their symptoms started, how they evolved, what their lives are like, what they're looking for and what their goals are. And that is a experience that occurs outside of clinic that we try to sort of summarize in a short period of time. That is our investigation, that is, you know, exquisitely human. That is us going back and forth. That's not transcription, that's not us sitting in clinic and transcribing what the patient says, but it's us sitting in clinic or in the hospital and asking questions, trying to, Oh, did you say that? Did you mean this? How about this? Can I pressure test this? It's really detective work, and, you know, decades of clinical experience get you to be able to do that detective work differently and better. So it's a real skill. And I think one of the things we're trying to do is decipher real signal that's happening out there for me, in the context of, when did the symptoms start? Have they gotten worse? What does activity do? And so forth in the context of cardiovascular medicine and for other specialties differently. But understanding those signals and trying to pull them out of, you know, a very highly emotional situation where someone's having a heart attack and is having difficulty recalling facts, et cetera, or perhaps doesn't have a real grasp of what's going on, et cetera. In those situations, I wonder if we can get to the root of those objective findings through understanding lived experience in a way that's finally able to become scaled for us to get at. And so in that context, wouldn't it be wonderful to be able to direct people better by having this set of objective findings transcribed or available to the healthcare system? Now, in order to do that, you've gotta have this ambient AI in various places. It's gotta be able to make sense of an abundance of data. It's gotta be evidence-based to do that. It's gotta have privacy protections. It's also gotta land in a way that is not overwhelming to likely a clinical AI in the medical record. And so that process of bidirectional data flow between an EMR and a human being on the outside of the clinic, that process of privacy, and then, most importantly, workflow integration into clinical AI and what a clinician sees on the other end. That's a lot of what ifs to make work. But I can imagine us getting towards that if we start building the architecture to be able to understand this data, house the data, and transform it into the healthcare system rather than just limiting it to what is happening outside. Currently, most of the wearables that are out there, Apple watches and so forth, we get calls from our patients, we get people doing printouts and bringing 'em to clinic, et cetera. And those are interesting, but very unscalable. And I hope that with more ambient AI being able to give us more information of lived experience, not just signals through PPG, for example, we can get to a place where, particularly on cardiovascular disease, we can understand, assess and intervene early to prevent disease.

- That's fascinating. And, Freddy, I know that Meta is also doing some exciting work around solutions that converge AI and mixed reality to enhance clinical care. First, to start out with, can you, would you define mixed reality, how that relates to the AI you've spoken about, both the AI, ambient AI, also AI agents used to perform tasks and AI used to assimilate knowledge? And for our audience members that may not be familiar with these technologies, could you explain how they work together, how they work independently, then how they work together?

- I sure will give it a try. So let me sort of start with augmented and virtual reality. And so, if you think about reality, it's just basically what you're looking at. If you think about virtual reality, then you're putting on a headset and you only see what is within the headset. So virtual reality is completely devoid of reality. It's completely virtual, meaning I can be on the moon if I want to be, and that's the moonscape I will look at.

- Right.

- That's very different in reality. And then the question is, how is augmented reality different than those two? Augmented reality is you're basically looking onto the real world, and the glasses you're wearing in this case for us, the glasses you're wearing allows you to place things onto that real world. Meaning, if I wanted to put a screen up here in my field of view and work on it, I could. And so it's augmenting reality as opposed to changing reality and defying the laws of physics. Okay? So that's virtual reality, augmented reality, and just straight reality. So, from our perspective, there's a future in augmented reality, which becomes really interesting. And if you can think about it from an educational perspective, you can imagine if you're in the middle of a surgery and you're a trainee, having a screen that helps you visualize what you are doing and potentially what your next steps are is really interesting. And so from a educational perspective, certainly that comes to mind fairly quickly. From a virtual reality perspective, if you're, you know, sitting in your room and you want to practice hammering a bone, then you could probably do that if you're an orthopedic resident, and I think that sort of puts you in a new frame of work, but in the existing world, whether you're in the cath lab or in the hospital, augmented reality would layer things on top of what you're looking at. And, you know, you can think of it as, for example, as simple as a vein finder where you're trying to get access to a vein if you're a nurse, to, if someone, if a resident for the first time is trying to get a central line and they're looking at the neck and just help you walk through that. I remember back in the day we used to watch "New England Journal" videos, you know, in preparation for doing the first off procedure, et cetera, but now you could interact with it directly. So augmented reality, virtual reality and just straight reality. And we're headed towards a pretty advanced, interesting augmented reality from the work we're doing. And I think in, in that context, AI becomes the interaction model for those things. Right? So I'll start, like these glasses I'm wearing today, they do not have screens in them, but they have camera and audio, so mics and speakers, but they're general, they're powered by our Meta AI. And so you it has interaction, voice interaction with what I'm doing. It also has an EMG that I can detract with what is, what I'm doing. But I can now ask a questions around whatever I'm interested in, whether it's looking at the a certain thing and asking what piece of art this is, or asking you questions, really just audio and not using the camera. But you can imagine you have basically this AI-powered tool that's interacting with the real world and has screens and projections to allow you to layer things on top of it.

- That's great. You've mentioned some examples. What if you think out, say three, five years, although one of the things that I've learned, I think we've all learned is that when we're looking at the evolution of technologies related to AI, the timescale we're seeing is remarkably compressed. And then the real timescale becomes how quickly can advances be adopted safely and effectively in real world uses? And in the cases you're mentioning in the context of patient care where the bar has to be very high to make sure, that first and foremost, we're ensuring safety. But what, where are the places that you think are going to be most transformed in health and healthcare by AI and by augmented reality? You mentioned some, for example, that resident learning for the first time, how to do a central line.

- Yep.

- A surgeon wanting to be able to visualize the next steps in a complicated procedure based upon imaging that the a AI has been trained on. But what are some of the other examples of augmented reality, and in particular from the patient's point of view, where do you see the applications of augmented reality or the applications of the glasses that you're wearing today?

- Yeah. So let me give you one example that's live today.

- Okay.

- Let me give you a framing and architecture of what I suspect will occur in the next couple years. And then let me also give you a final word that would break the whole thing in pieces. So--

- Okay.

- So the first thing is, I remember... You know, I haven't been in ob GYN clinic for, since I was in me in medical school, but I do remember deeply how difficult it was for women who were at risk of gestational diabetes to track their food. And so we would ask them to log their food, bring their logs in, and we review them and compare that to the, the lab data and give them advice and so forth. Now, as a cardiologist, we review food logs all the time.

- Right.

- Because it's such an important part of what happens to your cardiometabolic labs as well as your future outcomes. In that context, the... Tracking what you intake should be easier if you've got tools with you. And so we recently released a nutrition product where you, using the glasses, as you can imagine, if you're wearing them all day, you can then get a sense of exactly what you're eating and exactly what you're consuming and drinking, which believe it or not, even for me is massively telling, because what we end up remembering of what we've done--

- Sure.

- Is not right most of the time. And the things we end up forgetting end up being fairly material. Oh yeah, I forgot about that drink of, of glass of wine that I had at 9:00 p.m. And I was snoring last night and my wife was telling me about it in the morning, and so on and so forth. But in that context, being able to identify, assuming that you're interested in it, being able to identify things as food or as drink, being able to understand what those things are and being able to categorize them so that you can interact with that log is pretty, and hopefully at some point it becomes something that happens in the background as opposed into, in the foreground where you're actively acting on it. It becomes pretty powerful if you do that day in and day out, and is a massive sort of interaction between a piece of hardware and really groundbreaking AI to get you there. I think that kind of thing where lived experience is captured and understood and implemented against is really important. That's one thing. So example today, and please go out there and use it. I'm a huge fan of it, and it's available. So that's the nutrition piece. In terms of the architecture overall, I suspect we're gonna be in a position in the next few years where there's multiple parts of this architecture. One is you've got like a data part, which is your archive. And that archive should include all of your data, meaning your medical data and then any data that you're generating such as, you know, in this case, nutrition logs. But as we grow more sophisticated in data architecture, we, that becomes much more complete. And hopefully at one day exertional symptoms and so forth are in there as well and from lived experience. Then the next piece that that data archive interacts with through the archivist is a diagnostician. And this is where I believe evidence-based medicine really matters. Regulation really matters, which is you have to be running evidence-based algorithms against this data bank to assess for the risk of disease, the likelihood of success against your goals and be able to match that into your day-to-day activity. So you've got the archivist, you've got the diagnostician in between that's evidence-based and driven by clinicians and regulators and so forth, and that's running all the time. So if you are able to lose that 20 pounds and your VO2 max really has improved, et cetera, and that's showing a meaningful change in your long-term trajectory, you should be able to understand what's happening to you and change it. Then the next piece of this, all right, you've got the archivist, you've got the diagnostician, the next piece of this is, okay, how do I get this into a planner? How do I get this into real life? And for example, if you turn 45 and you should get screened for GI cancers with a colonoscopy, and we know that's evidence-based, can that planner figure that out from your pretest probabilities and timeline, be able to schedule that for you with the best doc for you in your region with your insurance and be able to make that work for your timeline, order the meds so that you've got the go lightly in advance and so forth? That would be the planner. And you wouldn't interact with any of these, but you would interact with the last one, which is a coach, which helps you really take all of this information, diagnostics, evidence-based and bureaucracy that gets us to healthcare and manage that in a way you would interact with your buddy if he was a doctor, or your, your, your friend if she was a doctor, that gets you into the healthcare system. So I think that's the overarching architecture that needs to occur. And the part that I believe blows this up in a lot of ways has been the movement over the last year to really an agentic-driven improvement in both large language models as well as data access and so forth. Because I can imagine a place where we are, the architecture isn't just a data center archive followed by diagnostic and followed by planner and coach, but a series of agents who are working on your behalf to connect the various data points that exist in the architecture, bring them for you, help you make sense of them, and then deploy on your behalf. So everybody will have their own clinical medical team that's working for them. And I believe that the agentic path will likely be the one that lands far more, far faster than rebuilding this thing from scratch and developing a new architecture and new, new sort of, you know, fire protocols and all that stuff.

- That makes sense. What... You know, deployment of these technologies is complex. Can you go over some of the technical complexities and also some of the policy complexities associated with? And let's just start out with these example you focused on before, and that is, you know, looking at caloric intake, also integrating data from a variety of different sources and moving into scheduling things and maintaining your personal and professional schedule. What are some of the barriers that have to be overcome? And how would you compare this technological evolution and revolution to the previous digital technological revolutions? I mean, it wasn't... Some of us still remember the days where the internet was much more primitive to what it was today before we started managing all of our calendars and interactions electronically. And that evolution took a comparatively, I would say, long period of time, at least compared to the advances we're seeing in AI and the potential applications of those advances in our professional and personal lives.

- Yeah, I mean, I mean, I would go back to, like on the clinical side, you remember when everything was paper.

- Yep, I do.

- And that, that transition was quite abrupt, and it came with, it came with significant promise. And a lot of us would argue that that promise has not been met in the way we hoped it would be. And so I think you gotta take this with a grain of salt in that context. But I do think the thing that makes AI different is I believe the potential impact is step functions higher than digitization.

- Right.

- So, so if you can imagine, the way I think about it is if I can give everybody the access that I have to clinicians and so forth, would that be a positive thing? I'm guessing it'll be positive, but certainly I can also imagine a lot of ways this ends up just causing a lot of noise. From what I've seen in the software world in the impacts AI has had, I would bet big large dollars that it will be pretty impactful, but we've gotta figure out how to get it right. Now, here's some areas that we're we're gonna have to struggle with. I'm gonna start with privacy. If you start thinking about lived experience and getting that information in a way--

- Exactly.

- That connects to healthcare records, making that useful to the clinician and having that interaction go back and forth in a, in a real way, that requires real thoughtfulness to get right, and we don't have that in place currently. But, and I would say a symptom of the need to do this as well as the caution around it is this entire industry of companies that have developed to connect various data sources. And so in this context, I think if we get the architecture right, then all of the companies that are currently in the data connection business will cease to exist. And so not to suggest I want that, but what, the architecture's gonna be very difficult for us to do, but is so important to get right, and I think that's where agents perhaps come into play so you don't have to build a new architecture, et cetera. So I would say the privacy piece is, we have to be thoughtful about it as many, many ways to figure this, figure this out, but one of them is through more connectors, the other is through architecture and hopefully we land somewhere in between. The next piece is historically, we've wanted control over, as patients, we've wanted control over our story, and perhaps we, we don't want this third party, this new stakeholder in the game to be able to have all that information and to be able to communicate all that. So agency becomes another core question to deal with. The next piece of this is, all right, there's gonna be such a multitude of data from lived experience that has to translate to a clinical action, which means that it has to go to another clinical AI. There's not enough clinicians in the world to be able to interpret all of the, you know, EKG tracings and so forth that are gonna be coming in. And so do those clinical AIs, what bounds whether they're good enough? And currently benchmarking is an approach here. It's primarily been driven by, by safety, and I'm not quite sure we're quite there yet. I think we've got... You know, we always find things even in randomized control trials when the results show positive, and then in the real world, when you're practicing and you're dealing with real, real life circumstances, they don't pan out quite as well. So I think understanding, regulating and requiring our AI counterparts in the clinical world to be able to do this well, be held accountable for that and have the benchmarks and regulation in place to control that is a different sort of thing to go after as well. So I'd say we've got privacy issues to deal with, agency issues to deal with. Are the LLMs and the clinical AI evidence-based, can they be judged and held accountable and meet the demands that we have of ourselves, at least as clinicians? And all of that is yet to be worked out.

- Well, Freddy, this has been a fascinating conversation. Maybe one last question on this topic before we move to a couple concluding questions, and that is, fast forward to a decade from now, how is the healthcare experience going to change? How is our interaction with information about our health going to change? And how, for those of us who are physicians, how is our role going to change? And just the entire healthcare delivery experience for patients and providers, in your crystal ball, what do you see a decade from now?

- Good question. So let me break it down into three areas. One, like what will change for consumers or users or patients? What will change for clinical practice, and what will change for medical training? And I'm most confused on the last one, just to lay it out there.

- Yeah, yeah.

- In terms of what will change for the average person outside what my dad would use, what my friends would use, what people in Arkansas would use, I think the majority of primary care will be able to be geographically dispersed without the need for as much access. I think if people can get labs and basic drugs locally, they may be able to skip out a lot of the primary care, preventative care that occurs in PCP clinic and be able to do that locally. We kind of faced this during COVID and went to virtual. I'm also not sure we need all those virtual things if we can build LLMs good enough to do those interactions, as long as they meet certain bars, which currently we have not suggested. I will also say passing board exams is not how to judge an LLM. They're really good at that, and it means, it means, for me, it's a nice step forward, but that's not their requirement. In terms of the delivery of care... Gosh! There's an entire layer of clinical AI that needs to be built to deal with this massive data that's coming. And in order for clinicians to be able to weigh in on whether things matter, don't matter, should be triaged, should have an intervention, should have a diagnostic or an interaction with a patient, and that clinical error doesn't currently exist. You're starting to see some of it in radiology where radiologists will have a read with a AI and they'll together decide what the outcome is. And I think that's one place. But if you can imagine finally, like you've got the ability to actually, actually track cardiometabolic health outside of the clinic, and you don't want to ignore that, but gosh! How do you actually incorporate into your workflow? I think that's a problem. And so I think the workflow integration, the clinical AI piece to be able to deal with all of these incomings is gonna be really critical. And then how do you incentivize and reimburse that? I suspect, you know, as much like in the military where there's folks who sit in a control room and drive things from afar, we likely will have that kind of new specialization in the clinical world for people to be looking at it. I mean, this has happened a lot of ICUs where you have one ICU doc that's looking at a bunch of ICU screens at different locations and trying to make judgements. So that whole layer needs to get built out. And then finally, for clinical education, this is where I'm most confused. In fact, my my kids have recently been asking me, should I... You know, they're eight, 10, and 12, and we talk about what they wanna do in life, and yeah, I really want them to be doctors. I mean, like, I'm certainly biased. I think it's amazingly rewarding profession, but they were asking me what they should study and what is, what clinic is gonna look like, and I don't really know. And there's, you know, there's this concept that like you can basically look up all the things, and at some point doing all that takes away the core skill of understanding, which means like how do you then disagree with a model? Do you need a second model to disagree with? I mean, I don't know, but certainly I could tell you like the reason why clinicians do well with these model outputs currently, from my perspective, is because they can bat away the bad stuff and just use the good stuff. But if you just train on that, then what do you... At some point, your skillset diminishes materially, which means that you either need a new skillset, and I'm sure that is, but it hasn't been defined. I mean, your position in education, I wonder how you're thinking about it. You know, it's my alma mater, so I'm really excited and interested to hear your thoughts on this. But I don't know in 10 years whether, you know, learning what I learned when I was in medical school will be valuable. I also know that in emergency situations, if you don't know what I learned in medical school will be critically bad. And so how do we sort of bridge this in between? I would love your take on that actually.

- I think we have a lot of exploration to do, a lot of serious thinking to do. We've moved gradually away from a pure memorization curriculum in the preclinical years of medical school. But still, as you well know, there's a lot of factual knowledge that we expect our students to have learned at least once and therefore to be able to refer back to it. You know, physician education today is a lot like, maybe the closest analogy is to learning a foreign language. You have to know the vocabulary, you have to know the grammar, then you have to put the vocabulary and the grammar together to solve problems or to communicate. And now the vocabulary at least is being handled by large language models and agents are handling the grammar. So, you know, where does that leave the human in the process? But I think we're, we're wise to, to take our time to study things carefully and not to abandon things that are imperfect, but I think have served the profession well historically over time. How does that resonate with you?

- Yeah, it's... I mean, it sounds right. I would, I would... I would reckon that there is a subset of skills that we need to be able to make judgements on clinical data, which should become more prioritized in education as opposed to all of the clinical data itself. And I also suspect that will change, that will result in a changing of what the specializations currently are. I suspect the procedural... Until AI breaks into the robot cath lab and into the operating room, procedural outcomes will... I mean, you still need to know all the things.

- Sure.

- But judgment-based ones of should you operate, like you just discussed, or, I suspect the outcome is gonna be that we will use the agents to give us the closest we can get to perfect data, and then we should be able to make judgements on clinical care and outcomes. And I'm not quite sure how you train that, but I also can't imagine you can make that judgment without the underlying, Underlying medical education. So, you know, I'm going back and forth here, but I really love my education and I think it's been incredibly valuable, but I do, I can't imagine that it's gonna be really exciting. I still tell my kids to do it, so I guess--

- That's great.

- I believe in it.

- Well, Freddy, this has been a great conversation. I wanna end with two questions that I ask all my guests, and for this series, I'm adding a twist. First, what do you think are the most important qualities for a leader to pioneer a transformative approach to human health?

- Curiosity and empathy.

- Right.

- I think the majority of what we do in clinical medicine is to try and understand another human being. and that ultimately is empathy. And so that, that really applies across the board. If we are to move large groups of people to what we believe is a better destination and intention is to improve health outcomes or improve lives, then you have to be able to empathize where the other person is and be able to walk a mile in shoes. And the second is you have to be curious about alternative ways of approaching problems that are not the way you normally do things. And I suspect curiosity is harder than empathy 'cause we all grow up learning how to empathize, understand other human beings, but we don't spend as much time, particularly in medicine, being wrong.

- Right.

- And being curious about a separate approach. There's guidelines, there's right answers.

- And finally, what gives you hope for the future?

- Oh gosh! You know, as many doomsday scenarios as I hear about AI and how the world is ending and and so forth, and I cannot tell you the number of people I know that take their labs and take a screenshot of their medical records and throw it into ChatGPT or Gemini or Llama and ask it for, you know, 20 or 30 minutes of back and forth where they show up to me and understand their LDL, HDL, A1C much better than, honestly, an intern would.

- Sure.

- So I am so optimistic about the potential upsides of the tech development that we're doing. I love the fact that we're a leading position in this country. I think for health, this will be the first time that there's a real shot at improving the disparities we have across the country, because this has never happened before. And I think if we can build the right tools and do the right things, be thoughtful about where other people are coming from and be curious about the problems and how to solve them, we can address privacy. I think we can address agency. I think we can address the missing layers in architecture. And my hope is, and what I'm most excited about is, you know, friends, family, my dad, others being able to use this and feel better and be better, and I think we can get there.

- Wonderful. Well, Freddy, thank you so much, and thank you for listening to "The Minor Consult" with me, Stanford School of Medicine, Dean Lloyd Minor. I hope you enjoyed today's discussion with Freddy Abnousi, Vice President of Health Technology at Meta. Send your questions by email to theminorconsult@theminorconsult.com, and check out our website, theminorconsult.com for updates, episodes and more. To get the latest episodes of "The Minor Consult", subscribe on Apple Podcasts, Spotify or wherever you listen. Thank you so much for joining me today. I look forward to our next episode. Until then, stay safe, stay well and be kind.