ASGBI Quality
===

[00:00:00] Hello everyone and welcome back to Behind The Knife. We're back with another installment of our BTK and HGBI collaborative series. My name is Agnes Premar. I'm one of the Behind the Knife Education Fellows, and I'm a general surgery resident who's representing the US side, along with our UK based co-host, Jared Volga and Gita Lingham.

Hi, I am Gee Lingham. I'm a surgical resident based in the UK and the vice president of the Moynehan Academy. It's great to be back here again.

Hi, I'm Jared Volga, a surgical resident based in the UK and the president of the Moyen Academy, which is the resident group of the Association of Surgeons of Great Britain and Ireland.

Awesome. So today we're tackling a word that gets thrown around constantly in surgery quality. It's on dashboards tied to reimbursement discussed in m and m, and increasingly being posted online.

And yet when we say quality in healthcare, we're often not talking about the same thing. Is it outcomes, processes, patient experience, safety, culture, or just who has the lowest complication rate?

To help us unpack this.

We're joined by two [00:01:00] fantastic guests. Dr. Mark Cheatham is one who's joining us from the uk. He has a lot of experience in national audits and system level quality improvement. He is a colorectal surgeon and the national clinical lead for general surgery at the Getting It Right first time program in NHS England.

And Dr. Alexander Perez from the us. Who's a board certified general surgeon and minimally invasive surgeon at Baylor St. Luke's Medical Center. He has worked extensively with institutional quality programs and is the current assistant dean for patient safety simulation and process involvement at the Baylor College of Medicine.

Thanks to both of you for being here.

Pleasure to be here.

Pleasure.

All right, so let's start at square one. When we say quality and surgery, what do we actually mean?

Well, I think there's no one definition of quality, and I think really it's important to look across several domains of quality. Probably one of the simplest, if you look at what Donna Badin said in the 1960s, he talked about the structure, process, and outcome.

And if we think across those [00:02:00] three domains, I think that's a really useful construct. More recently, the IHI, the Institute for Health Improvement in the States has come up with these six domains of quality in, in healthcare, which are, is the service safe? Is it effective? Is it efficient? Is it patient centered?

Is it accessible, and does it provide equitable care? And I think looking across those six different domains gives us a richer idea of what quality is or what it could be.

It is

definitely

a multidimensional item. I think that we can see it from multiple perspectives. I think the framework has shifted from it being something perceived and analyzed from the healthcare professional's perspective now, including the patient and the patient perspective a whole lot more.

So I think it's really much more complicated than like any one particular parameter or one particular lens. But I think we're all trying to do our best to improve all these things that we can measure because [00:03:00] we can't improve that, which we cannot measure.

Thanks for the great overview.

So why don't we zoom out? How does the US approach differ from the NHS model in the United Kingdom?

Sure, absolutely. I think that overall there's definitely a culture of wanting to do what's right and best for the patient. In an attempt to do that we try to standardize how we deliver care. And by measuring different outcomes such as length of stay wound, infection rates, et cetera we can compare outcomes from one institution to another.

And with transparency then patients and insurers providers could really see where you could get one procedure or manage one condition. For a certain price or a certain level of investment and have a, a certain ability to consistently deliver that outcome. And so, one way is to have that transparency and drive improved outcomes by individuals trying to [00:04:00] compete against each other and other institutions.

But the way that the system works and seem to be most efficient is by having those regulations be linked to payment. People know that

if

there's definitely money on the line then people will tend to listen, especially administrators that tend to have influence over how processes are done at the level of the institutions.

Dr. Perez, just out of interest, is that information widely available even to non-healthcare individuals?

One way or part of the information is I think the complexity is that there's many things that are being measured by lots of different institutions and entities. But there are some that are more nationwide and are more generalizable and adaptable to multiple environments.

But most of the data is comparing like to like, apples to apples large academic medical centers compared to others. But the idea is still it [00:05:00] is not mandatory and it is expensive to participate in these initiatives. So I think that that's one thing that is definitely favorable in the uk.

It being more of a nationwide mandatory compulsory sort of component as opposed to in the US where it's more of a, institutions competing against each other to have a higher rank more notoriety, performing better so that they will attract more clients, more patients more providers more healthcare, insurance providers to work with them.

Judge Cheater, any thoughts on the uk? I remember a while back we did use to publicize things like individual surgeon outcome data.

Yeah, we did. We've kind of moved away from that. Perhaps you can touch on that team or, or individual performance later on in the podcast. But, but we, we do publish team mortality for specific procedures such as elective cancer resections still.

And the source data for that relies on a [00:06:00] number of national audits. So there's a number of national audits such as EN Boca, the National Bowel Cancer Audit, and there's similar ones in upper GI. Cancer and more recently in pancreatic cancer. So we have a set of national audits.

Participation in those is mandatory. And it's, tied into payment for that service. I think that's been really, really good. And there's been a real rich level of data from that because the data is inputted by clinicians and there's case adjustment based on tumor stage and patient's comorbidity.

But I guess we also, we are lucky that we have a national repository of all the activity data called Hess. Which is hospital e episode statistics data. This was originally designed for monitoring activity and paying organizations, but we've started to use this for benchmarking and also for tracking improvement.

So outside of the national audits, we have a couple of sources that are available to all people working within the National Health Service.

And I suppose the other thing in the UK [00:07:00] that the, most of the care is provided in government institutions and funded by the government. So actually we can control who does what procedure where. So I think some of the great strengths of that have been consolidation of major trauma to major trauma centers over the last few years, and the consolidation of high-end cancer surgery such as upper GI cancer surgery or maxillofacial cancer surgery and into centers serving a population of one to 2 million.

I think we hinted at this earlier, but we've been talking about two different groups, so excellent surgeons and then hospitals that are getting distinguished for providing excellent care. So how does that difference translate into quality metrics and when you're applying it to a hospital system versus an individual surgeon?

Well, great to sort of look at it from different perspectives. I think that each individual healthcare provider, whether they be a surgeon or any other specialist or part of that team wants to do better. But they can't do better unless they know what it is that well standard should be [00:08:00] for that particular procedure, that particular condition.

And I think that at the level of, let's say, within the specialty of surgery. There's surgery specific data like Quip as we mentioned earlier. That gives you sort of the standard of care across the country and allows you to compare, and being able to say how your department might be performing versus another department.

Now, clearly we understand that patients aren't treated by just one individual provider, so it's very hard to say. One particular surgeon is solely responsible for the outcomes of one particular patient. It's definitely much more team taking care of a patient. And at least in a recent study, it's shown that more than 20 healthcare providers are taking care of one individual surgery patient during their stay.

So you can imagine it's getting harder to. Allocate the responsibility of one particular data point to one individual. And so the value of expanding this [00:09:00] beyond the individual to a team and then to probably teams of teams within an entire system. And so for example, within the Department of Surgery of multiple institutions across the country, there are weekly conferences that are called morbidity and mortality conferences that focus on.

What's happened during that week with regard to complications, deaths, and a very close analysis of these outcomes are done and looking for patterns. And within the Department of surgery the leadership is able to steer initiatives and be able to ensure that changes are made in a very timely fashion.

Really raising the bar. So not just setting a standard for nation needs to be, but raising the bar at the individual department level. And then for institutions there are data sets like Vizient, for example, that use administrative data and cost analysis and length of stay and all these other things with regard to processes and how efficient they are.

So you could definitely then compare how does one [00:10:00] institution compare to another with regard to how long does it take to take care of a patient with one condition, how expensive it is to do that, how often they're readmitted. And so there you start then competing at the institutional level and not just one department or individual.

Yeah, we've had a similar experience, so we have previously published mortality data for an individual surgeon level for colorectal cancer resections, for instance. I think that there are some issues with that because case volume is such that even in a fairly common procedure, to get to a statistical point where it means something is it takes a number of years.

So you are kind of allowing practice to develop and I think the other issue is this was published on the internet. So, it was in the public view. And not everyone could understand mortality statistics to that level required. And then within a team, there's often one or two surgeons within a team who have a different risk appetite or have a different experience and may take on those higher risk [00:11:00] patients or those higher risk tumors and their outcomes without case adjustment are gonna look worse.

So because of those factors, we've recently moved away from publishing. Public domain, individual surgeon outcomes. But we do continue to publish team and unit outcomes. And I think they're useful because it kind of drives up behavior. As Dr. Perez said, it's difficult to disentangle the teamwork from the individual surgeon that there's an old adage, surgery's a team sport until it goes wrong and then it's the surgeon's fault.

And that kind of feels like we, where we get to sometimes and I think particularly in emergency surgery. Where you may be admitted under one surgeon, have a, an operation under another, and then someone else may deal with the complications. You're really reliant on that team.

So I think in elective practice and high volume elective practice, you can look at surgeon outcomes. But, you know, 30 day mortality, a pretty crude statistic. And patients want to know more than that. I think from us, they, particularly if we're not doing [00:12:00] high risk surgery, that people assume they're gonna survive a hernia repair, for instance.

So that's a, that's a not a useful metric, I think.

Dr. Sheen, it's an interesting point that you mentioned about the public domain and about what we put out there. With the push towards social media social media presence amongst doctors, do you feel a certain pressure to maintain a five star rating?

If we start with Dr. Perez, what's your feeling on this?

Really, I think our natural instinct is to want to perform better and we will be. Looking at these reports and how we perform and we've taken personally for sure. I think the challenge is our attention span also is very short in getting shorter all the time.

And there's so many influx of data points that it's hard to keep on track. And so I think that's what the value of the culture of safety. That instills the need to not only set that standard, but to continuously raise the bar so that we can always keep [00:13:00] adapting to the best way of taking care of our patients.

And so I think it's a process that we have to keep working at. And while we realize what went wrong we also have to realize all the things that went right. And sometimes we don't focus enough on all that. I think now the culture shift should be to we need a team to build resilience, to feel, to fill the gaps between the gaps of care between providers.

No matter how excellent they are, it will be handoffs between those providers and teams. And so we have to focus on building that team and really making it so that the patient's at the center of that team

I think certainly in private practice there's growing interest in in social media and there's sites such as We want Great, I want great care, which rate doctors.

And I think what you're really looking at a fairly crude measure of patient experience. So I have used those previously. And I think it's important that we capture patient experience, but we should probably do this in a more structured way using PREMs or patient [00:14:00] recorded experience measures on a much wider level rather than the, that consultation that went really well, asked them to rate you, which I think is what it gets down to.

So yes, I think we should. Record that, but probably better. And you can't really compare across different environments. So if I'm working in emergency surgery versus elective practice versus different subspecialties in surgery the patient experience is gonna be very difficult to compare across those domains, I think.

Yeah, I think it's very interesting to look at those reviews. 'cause you see everything from how the office staff behaved to how long they've waited, right. All reflecting under the surgeon's name and how likely they're right there.

Yeah. I guess the other thing is complaints. So, we have to have an annual appraisal where we deal with all compliments and complaints and some of the complaints are what I call the burnt toast complaint.

There's a whole load of stuff about the surgery . That's great. And then this and the toast was called that. You know, that's not really my fault.

Yeah, very true. I think we will potentially see more of [00:15:00] this in the future. I mean, even when you book your holiday now, you don't do it without checking the ratings, everything.

So, I can definitely see this being a topic that we discuss again. But who actually polices this variation in care to create the standards. So we have all these outcomes. We have this, but who's, who's in charge of it? I know Dr. Per, you mentioned Eminem, but anything else?

Sure. There's definitely multiple layers.

So, within the department there are chiefs of different sections and divisions. There are chiefs of quality vice chairs of quality and of patient safety. I think that. Within the institution. There's also the chief medical officer and other positions that are focused specifically on taking care of the patient and addressing these quality metrics.

So I think it's looked at from all angles and the level of transparency and the level of involvement of leadership in this is very palpable. I think that not a day goes by that you don't think [00:16:00] about. All of these things that we're doing, are we doing the right thing? Are we at the standard and how can we do better?

And I think it's a definitely a different world that we're living in right now. And I think that's only gonna get more and more transparent. And to your point about, these subjective feelings of how care was perceived. I think that in the US we have hcaps, which is basically a consumer assessment.

So the hospital sort of captures a survey of patients who've experienced care and after that care they ask them specific questions that are standardized throughout the country with regard to would they feel that the information they were given to them was communicated clearly. What was the condition of the environment that they were in?

So it's not so much how do they feel about their experience. It was more about how was their experience overall. And so I think that there's an attempt at really trying to become much more objective with a lot of the subjective data. [00:17:00] But it's all being policed and controlled at multiple levels.

I think this is a really big topic. So prior to taking on this national role, I was divisional medical director for surgery, anesthetics, and cancer across two hospitals in a fairly big trust. And that meant I was responsible for the quality of safety of all sorts of surgical.

Disciplines , and oncology, which I haven't practiced in. So actually as I say that I realized that's quite a big responsibility. So I was really reliant on the individual clinical directors and clinical leads in their area to, give me advice on how they're performing and that information comes through several sources.

There's some internal that comes through from reporting of incidents and complaints. There's some external benchmarking from measures that I have discussed earlier. Most of those things can be dealt with in-house if there's a, an area that feels to be going to be going wrong. And I think one of the important things for me was cross discipline.

Learning from different or from different [00:18:00] specialties, particularly if we're talking about improving theater safety, where. We need a standardized approach that doesn't matter which specialty you're on. In fact, if you are from anesthetics or surgery, we need a similar approach.

But that's really helpful in bringing some external scrutiny from peers outside the organization. If you get to the point where there's significant quality concerns and you don't know who, whose opinion to trust. And that's really helped me transform and turn some services around. And I guess over the last four years I've been working nationally and I've been privileged to visit every general surgical unit in England and Wales over the last four years.

So I've got a really good idea of the variation. And what I, what we've done in those visits is we've brought us a data pack that we've given to the teams beforehand. We ask them to present some information about their context, how their recruitment, and how they deliver their service.

And then we'll deliver some routine data. And then we have a conversation about, so what does that mean? So the data [00:19:00] typically will show how they benchmark against peer trusts.

But really the data's an excuse to have a structured conversation about what's going well and what they want to change, and try and guide them and coach them to improving their service.

Yeah. I imagine when you go to a lot of these institutes, the first thing that everybody wants to say is, well, you didn't account for the fact that we are doing really hard cases, or our patients are really old and and frail and it's not fair because we have a difficult patient cohort here.

Do you think the current metrics adequately account for things like emergency work, frailty, redo? I,

I don't think they do completely. And that's something we. We're developing, but it is quite hard to adjust the case mix. We do make, we can make some adjustments for comorbidity, but remember that the Hess data was really collected to benchmark activity.

The other thing I get is that the data, your data's wrong, but actually it's not my data.

It's the data's come from your organization. All I've done is processed in it a standardized way and compared it to data coming [00:20:00] out of other organizations. So if the source data's wrong, then you need to go back and look at your coding. And sometimes those conversations take 20 minutes, and then eventually we can reach a point where, look, the data's not perfect, but it's kind of the same elsewhere.

So we're comparing you to the data with all its biases and flaws elsewhere. So what's that showing us? And what do we do next? And it always interests me how teams get to that point quicker or slower than others.

Yeah, I suppose that's saying something in itself, isn't it? Dr.

Perez, do you get the same sort of comments about your data in the us?

I think that everyone is hesitant about negative outcomes and negative performance. I think everyone sort of would rather not have that. But at the end of the day, after. You adapt to that culture, that it's all about trying to improve.

I think at the end of the day whether you are in an environment that shames and blames. It will be very hard to make any significant improvements. People will hide their outcomes. They will hide their poor performances [00:21:00] as opposed to an environment that is transparent and supportive and wants to create that environment of teamwork and resilience.

And even if you have a perfect score sheet, for example, of how many comorbidities a patient might have, the patient may not provide you with all the information. You may not be able to extract that information. You may not incorporate or include that information into the electronic medical record.

So it's not a perfect capturing system. And and once you start having more than a few. Complications or comorbidities? They are not additive. It is much more logarithmic with regard to how things start interacting with each other. So it's definitely not a straightforward possible, we have to start somewhere.

We all agree. Someone with one comorbidities very different than one with 10. So I think that we will eventually get to a point where we can process the data a whole lot better. But I think the first step is having that culture transparency and that culture of wanting to be better.

And I think you hinted on this earlier, [00:22:00] Dr.

Perez, but I think we often automatically assume that good care leads to good outcomes. Do you both agree with that statement?

Not always. I was gonna take you back Lisa Yone, who is a surgical outcomes researcher in Harvard. She says she's got this thing called the algebra of effectiveness, and she says that the quality of care.

Plus patient factors plus random variation determines what the outcome is. Now, often we look at the outcome and infer what the quality of care is, and we miss those two steps about the patient factors and that random variation.

Yeah, and there's also a saying of that my is king, patient selection is queen and technique are the prince and princesses of jms. So I think that you could technically try to do a procedure a certain way, but if you choose the wrong patient because of the type of patient complications comorbidities, et cetera, or you choose the wrong disease.

Because the biology is completely refractory to what you're planning on doing, then there's no way of [00:23:00] impacting that. So I think there's certain things that we can impact, certain things we can't. But what we're really trying to do is to make it more predictable, more standardizable. Once you know what you're going for, it's like your favorite restaurant is your favorite restaurant because the food is consistently good.

I think it's also important to think about which population are we talking about? Are we talking about the pe, the population of people who had surgery, or the population of people who could have had surgery?

'cause they're different. And if we turn people down. That's one really good way of, of getting your results better, but it may not benefit that patient population.

And you've both alluded to it, we're not perfect. There's random variation. There's biology to take into account when complications happen. How do these systems respond?

So I guess it depends on the environment. It depends on how much time has got elapsed from the actual problem. It depends on the ability to communicate to all the stakeholders. Because again, if you're only [00:24:00] having a weekly meeting within silos department and the complication involves multiple healthcare providers, it's hard to really enact solution at a higher level and about 80% of errors and complications.

Miscommunication or failure of communication as the root of the problem. And so you can have great individual providers, but if they're not talking to each other and if they're in different siloed departments you won't really enact any change.

I think there's several layers to this. The first, if there's been a devastating incident, there needs to be an immediate response to, to ensure that other patients are safe, or if that patient is safe. And also to look after our staff. We sometimes forget that. The surgeon and I'm gonna say particularly surgeons, because having had a devastating complication for a surgeon is a real psychological insult to, to them in a way that it isn't to, other branches of medicine.

And I'm really pleased that the Royal College of [00:25:00] Surgeons have recognized this and we're enrolled in a program called support, which provides immediate psychological first aid for a surgeon involved in something like a death on the table by a surgeon because we know that surgeons are really poor at seek at seeking help from other.

People, but they will accept it from another surgeon because they kind of under, they feel that they can understand that. And then I guess the next step is if something's gone wrong, how do we make sure it doesn't happen again? So we've stabilized the immediate situation, we've looked across staff. What and this is where I think sometimes it goes wrong.

And in some organizations where we blame the person who was nearest the patient at the time and sometimes. They just happened to be there. It's just a, you know, it's just an accident of fate that the incident happened then, and that the thing that caused it may have been a systems problem months ago in a computer system, in another building.

So it's important that I think we need to think in systems and we need to be dispassionate about this and step back and really [00:26:00] investigate this thoroughly. More recently, in England, we are moving away from root cause analysis. 'cause I think that's taking you down the wrong route. And and actually historically we've spent an inordinate amount of effort investigating really rare things that, don't happen very often.

And we really, what we need to do is look at how our systems function at 98% of the time and make them more reliable. And if you just investigate the big problems, you'll miss out on some of that richer learning from the way things happen normally. So with the PS IF, the Patient Safety Invest Incident response framework, we're moving away from individual root cause analysis, but more looking at thematic reviews of what happens when things go well and what happens during normal practice.

'cause that's where I think you can make systems more reliable.

Yeah. And I think the listeners would really appreciate you telling your pigeon story, to demonstrate this.

Okay. I've gotta take it back. 20 years ago, I was a fairly new consultant.

And I'd been practicing for about six months. I felt like I'd had more [00:27:00] than my fair share of obese men with low rectal cancers. In those days. We were doing mainly open surgery and I'd had a few deep incisional wound infections and I got a message to, and I needed to go to the medical director's office and had to discuss my wound infection rate, and no one told me the.

No one gave me any statistics or any benchmarking. I just got a phone call and I thought, I've only been here a few months. This feels quite threatening. So I walked into the office and I shared an office with another consultant. I said, I've been called up to the medical director's office to discuss my wound infection rate.

It all sounds a bit scary, and he said. I learned so many things from this interaction. He said, there are patients, there are wound infections. I'm coming with you. And he picks his jacket up, put his jacket on, and walked up with me to the medical director's office where we finally got to see some data.

Agreed that wound infection rate was high of an average. And I can't remember the outcome from that, but I think it was kind of try harder. So I decided to try harder. And a few months [00:28:00] later they were doing some building works in the theater ceiling, and they re, they, they cleaned out the air handling plant in our theater and they found a dead pigeon and the air, one of the air fans was on the wrong way.

So once they cleaned that out and put the fan on the right way, funnily enough, everything got better. And what I learned about that is want to move from my patient to our patients. That's really key in a healthy team working, I think. The other is if there's a problem, give people the data so they can digest it and analyze it before having a high stakes meeting.

And the other is about supporting people and thinking about those systems factors. So, when, as, as I said earlier, when it goes well, it's a team sport. Everyone in the theater's taking all the, the, we are doing great cancer surgery. When it's, when it's a problem, it's the surgeon's fault, and we need to move away from that.

And we need to look more holistically at the s systems and processes in which a surgeon works. So I'm, [00:29:00] pleased that in NHS we've got a thing called the Just Culture tool and that takes you through the results of a patient safety investigation and tells you what you should do. So how to deal with individual people and.

The good thing is it is quite hard to sanction someone from this on one incident because usually one incident is nothing to do with their behavior or their skill level. It's usually that something's gone wrong and it's something that's quite random. And it encourages you to look at their motivation.

Did they intend to harm someone? Did they have a psychological illness? Were they using drugs or alcohol at the time? And if the answer to all of these is no, then actually you need to look at the system.

That's a that's a very unique story, and I understand the point.

Why do we end with a thought experiment? If you could redesign surgical quality metrics from scratch, what stays and what would go?

I think that both systems one in the US and one in the uk are doing things very well.

[00:30:00] And I think that actually a combination of the two might cover the gaps that are existing. I think the UK system seems to provide a very solid foundation across the entire country to make sure that no patient falls through the cracks. The system in the us sort of tries to have this competitive level to raise the bar consistently.

And so I think having a strong foundation and raising the bar will provide the therapist and most contemporary care that will provide the best care for our patients moving forward.

Yeah, I'd echo that. I'd take things I'd really like better risk adjustment. I'd like us to collect some outcome measures. So outside of mortality for cancer, we don't really collect outcome measures in a very structured way.

I like your end script. I think the clinical detail in there is much richer than the stuff that we get from administrative data, but it's expensive collect to collect. So how do we balance tho those two? But I'd like everyone to [00:31:00] take a responsibility for quality and everyone to look at a broader dimension of quality, because if you can't access the service, it's not a high quality service because we need to talk about the people who we didn't see.

That was great. I think that was a great overview and like we discussed a big topic and very broad with different details and nuances. here's to a better culture of just promoting quality care within our hospitals and countries. So thank you again, Dr.

Perez and Dr. Chitin for your thoughtful discussion. And thank you to our listeners for tuning in. And until next time, keep asking what quality really means.

​