SAGES Guideline Methodology
===

[00:00:00] Hello, BTK listeners. I'm Nicole Petka, a general surgery resident at Emory University. I'm here with Dr. Scott Steele, our listeners' favorite Dominate the Day superstar. He's the president of Cleveland Clinic Main Campus, where he practices as a colorectal surgeon, and having authored over 35 guidelines, he's a true expert within this space.

All right, all you BTK listeners, so welcome back, and we're so happy that you could join us today. We're gonna dive into something that every surgeon has heard about, referenced, and maybe occasionally even questioned, and that's clinical practice guidelines. So we're gonna break down how these guidelines are built, how they should be applied in our clinical practice, and we absolutely have some superstars here to join us, great guests to help us unpack it all.

So Nicole, why don't you do the introduction of our guests?

Of course. So our first guest is Dr. Bethany Slater. She's a pediatric surgeon and division chief at Mount Sinai and serves as the chair of the SAGES Guideline Committee. Next is Dr. Francesco Palazzo. He's an MIS surgeon and the vice chairman of the Department of [00:01:00] Surgery at Thomas Jefferson University.

He also serves as the co-chair for the SAGES Guideline Committee. And last but not least, we have Dr. Alisa Calabrese. She's a general surgery resident at UCSF East Bay and is the SAGES RACS Guideline Fellow. Bethany, Francesco, Alisa, welcome to Behind the Knife.

Happy to be here.

Thanks so much for having me.

Thanks for having me, too.

Well, I'm excited about the episode, and I really wanna start with the origin story. So there are a million clinical topics out there. How does a topic actually become worthy of a clinical practice guideline?

At SAGES, topics are proposed in a variety of methods from our membership guideline committee members, other committees, and leadership based on areas where there is variation in practice, uncertainty in the evidence, or a need for updated guidance.

The guidelines committee and the executive board then evaluate those topics and decide which ones are most relevant to practicing surgeons. Once a topic is selected, the major step is to find the key clinical [00:02:00] questions using the PICO framework. That means identifying the population, intervention, comparator, and outcomes that matter most.

Those questions ultimately drive the entire guideline process from the systematic review through the final recommendations.

Along those same lines, there are so many of us who just see the final published PDF. So can you give us a little insight to what's happening behind the scenes? Who's at the table?

What the standard practice is to create a guideline, and how long does it actually take to go from an idea to a published document?

So as Dr. Slater mentioned, it all starts with selecting the right topic. Assuming you've selected your topic, then I like to think about this in two parts, a systematic review portion and a guideline portion.

So as part of a systematic review, you define your key clinical questions using the PICO framework, which helps determine exactly what the guideline is trying to answer. To get the ball rolling, you need a content lead, the guidelines chair, a methodologist, a librarian, and then in our case, the guidelines fellow, [00:03:00] and we incorporate patient representatives as well.

So the team develops a search strategy and conducts a comprehensive systematic review of the literature across multiple databases. And then after that, we have trainee researchers that help us complete title and abstract screening, full text review, data extraction, and risk of bias assessment of all articles When appropriate, and most of the time for our guidelines, the evidence is pooled by a statistician through meta-analysis and summarized in evidence tables using GRADE or the Grading of Recommendations Assessment, Development, and Evaluation.

GRADE is just a method for determining the quality of your meta-analysis data in a systematic way. In parallel to your systematic review, a guidelines panel is assembled, which typically includes 10 to 15 content experts, depending on the topic. Ideally, it's multidisciplinary, so it may include surgeons, interventional radiologists, endoscopists, anesthesiologists, depending on the [00:04:00] topic.

Once your systematic review is complete, we move on to the guideline portion. The expert panel reviews the evidence tables, and we follow a framework called evidence to decision framework to generate the recommendations. This is a critical step because it provides a transparent method for generating recommendations from evidence.

It not only considers the evidence and the certainty in that evidence but also considers factors such as the patient values and preferences, feasibility, resource utilization, and the balance of benefits and harms. After that, we finally draft the recommendation and systematic review separately and undergo multiple rounds of review with the panelists.

After the guidelines panel has finished their review, the SAGES Guidelines Committee, SAGES leadership, and the broader membership will also review the guideline before publication. So while the final guideline may appear straightforward, it's really the culmination of a rigorous, transparent, and highly collaborative process that often takes many months and [00:05:00] sometimes more than a year to complete.

So that's pretty amazing when we think about, about kind of the process that it used to be with a few experts in the room just sitting around and just drafting what they think to just this unbelievable process that is detailed. So Francesco, I'll ask you first. During those rounds of review, despite the fact that there may be all of that rigorous process and checks and balances, do you ever come into a position where somebody maybe is an author or has a very strong opinion?

Or how do you deal with the, the people who constantly wanna nitpick at that guideline when you know this is what the evidence shows and in some cases, the evidence isn't that strong? What do you do in that situation?

Well, that's where we have to tackle the strength of the recommendation and making a decision as a panel as to what is worth providing a strong or a conditional recommendation, and obviously balancing the level of the evidence that we have.

And if you allow me, I may elaborate a little bit on that. [00:06:00] The two terms, strength of recommendation and certainty of evidence, are related, but they're not the same thing. A strong recommendation means that the panel believes that for most patients, the benefits may clearly outweigh the harms or vice versa, and most informed patients would choose that course of action.

Where in practice, that means a surgeon generally can feel confident adopting that recommendation as their standard approach Whereas the conditional recommendation, on the other hand, acknowledges that there may be more uncertainty, whether because the evidence is limited, patient preferences vary, resources differ, or the balance of benefits and harms is less clear.

It doesn't mean don't do it. It means that the best decision may depend on the specific patient and clinical context. The certainty of the evidence tells how confident we are that the estimated effects are correct. High-certainty evidence means future studies are unlikely to change our confidence or results, whereas low or very low-certainty evidence means additional research could meaningfully change our understanding.

One of the unique aspects of the GRADE process [00:07:00] is that recommendations are not based solely on the certainty of the evidence. The panels also consider patient values and preferences, feasibility, resource utilization, equity, and the balance between desirable and undesirable effects. So it is possible to have a strong recommendation based on lower certainty evidence in some circumstances, and a conditional recommendation despite moderate or high-certainty evidence when patients' preferences or implementation consideration vary.

For clinicians, the practical takeaway is that a strong recommendation generally supports a standard approach for most patients, whereas a conditional recommendation should prompt a more individualized discussion that considers the patient goals, values, and clinical situation.

I really liked what you said with high-certainty evidence, maybe future studies aren't gonna change it.

But for residents or fellows looking for research projects, those guidelines that have low certainty of evidence, that might give them some good ideas. So I think that's a great way to interpret that. But I do wanna bring us back to the real world because most people aren't pulling up these documents in the middle of the night when we're caring for a really sick patient.

So how do [00:08:00] we bridge this gap between a level one evidence deep dive and really taking care of our patients and the high-stakes decision-making in the moment? What's the best way to apply a guideline to your clinical practice?

Yeah, I'll take this one here. When we write them in the American Society of Colorectal Surgeons, we've done a few shared guidelines.

One of the things we like to think about is you have the parameter u-up front and then you have the GRADE of evidence, as Francesco said. And so we try to surmise what we can. I think one of the important points about a lot of these particular clinical practice guidelines is we try not to paint surgeons in a corner.

So you'll see words like typically or should or most of the time or normally. It's not because that we're trying to hedge our bets or anything, but we understand, as you say, that there is a real world and there's a patient in front of you, and we're trying to provide recommendations. I would just say in general, this is where technology is really advancing us.

There's websites that are out there that are powered by a lot of artificial intelligence technology that can get you that particular best recommendation, and we [00:09:00] wanna make it accessible. Ultimately, we would like this to be a part of the natural education for our trainees and for our staff surgeons.

But in the heat of the moment, I don't know how to best say it other than the fact that we want to make it accessible where you see the parameter up front and then the supporting evidence below it Combine exactly what Francesca said with what is the strength of that evidence to support that? In many cases, for example, when I wrote the polyp and idol guideline myself, it, it just wasn't good out there.

It was a lot of retrospective reviews. It was a lot of case series. There wasn't a lot of head-to-head evidence, and so you gotta make the best of what you can get.

Yeah. That's great. And you touched a little bit on the elephant in the room, so I wanna bring it up. I've heard concern that guidelines could lead to lawsuits, and there's, I think, this uncertainty about when is it acceptable to deviate from a guideline?

And as a surgeon, if I decide to use my clinical judgment to deviate from the guideline, am I putting myself at risk of a legal process because of that deviation?

I think that's probably one of the most common [00:10:00] concerns surgeons have when they hear the word guideline. The reality is that guidelines are intended to inform clinical decision-making, not replace it.

In fact, most guideline documents, including ours at SAGES, explicitly state that they're not meant to establish a legal standard of care or dictate management for every patient. They provide recommendations based on the best available evidence and expert interpretation at a particular point in time, but they also recognize that patient resources and clinical circumstances may vary.

So if a surgeon chooses not to follow a guideline recommendation, that does not mean that they are practicing outside of the standard of care. The key is whether the decision is thoughtful, patient-centered, and supported by sound clinical reasoning. There are many situations where deviating from a guideline may be entirely appropriate because of a patient's anatomy, comorbidities, preferences, operative findings, or even resource limitations.

So I actually think this, again, highlights why conditional recommendations are so important. They [00:11:00] acknowledge that there may be multiple reasonable approaches and that individualization of care is expected. Ultimately, guidelines should be viewed as tools to support clinicians, not as rigid rules. The real goal is to improve decision-making and patient outcomes while still preserving the surgeon's ability to exercise professional judgment when caring for the individual patient.

That's really reassuring to hear. Speaking about how often guidelines come up in lawsuits, do we know if this is common, or do any societies collect data on how frequently guidelines are cited in a lawsuit?

The information on this is very limited and settlements are obviously very frequently confidential, and it's probably true that the real frequency on guideline use in legal proceedings is, and likely will remain very much either unknown or grossly underestimated.

We certainly have to remember though, that guidelines could be leveraged both to accuse as well as defend the [00:12:00] actions of a physician. But as Bethany said, today's courts now place more value on whether sound clinical judgment was used for the specific patient situation.

Yeah, and I would agree with that.

That I know of, there's no formal process by which we track how many guidelines and which ones are used in lawsuits, unfortunately. There's very limited empirical evidence on this. There is one study from the '90s that used two malpractice insurance companies, OBGYN and anesthesia claims, and analyzed them.

They found less than a six percent rate of guidelines being used within lawsuits. But again, this is from 30 years ago, and it's not within the field of surgery. So take it for what you will. Unfortunately, this information isn't really at our disposal in general, again, because most lawsuits are confidential.

But I think that there's a lot of fear surrounding the topic. It's important to understand that looking at other specialties' data and what we know from that, the [00:13:00] lawsuit rate isn't as high as perhaps we fear. But of course, in theory, guidelines could be used against you or to support your decision-making.

So I think it's just important to keep it in mind, especially with a strong recommendation, since they are generally defended by more robust evidence.

Yeah, and I think that's what's really important for clinicians to understand, is that the authors of the guidelines are considering this when they make them.

But let's move on to a less stressful topic. Where do clinical practice guidelines fall in this as far as standard of care, and are they defining it or are they just part of the puzzle when defining standard of care?

That's an important distinction. Clinical practice guidelines are generally considered part of the standard of care conversation, but they are not by themselves the definition of the standard of care.

Really, the standard of care is broader and reflects what a reasonably prudent physician would do under similar circumstances, taking into account all the available evidence, the clinical judgment, patient factors, and local resources. [00:14:00] The guidelines really contribute to that discussion because they synthesize the best available evidence and expert interpretation, but they are only one piece of the puzzle.

In fact, many guidelines, including the ones that SAGES state that they are intended to guide but not dictate clinical care. I think of the guidelines as a framework rather than a rule book. They help define what the evidence suggests is the preferred approach for a typical patient, but they do not eliminate the need for individualized decision-making.

Ultimately, the standard of care is determined by the totality of the evidence, accepted clinical practice, and sound professional judgment, not any one single guideline document.

Yeah, I'll just piggyback onto what Bethany just said right there, and, and she said it best. Remember, there is a legal definition of the standard of care, and that's exactly what she said, a reasonably prudent surgeon would do under similar circumstances given that clinical situation.

As a matter of fact, if you look at many of the guidelines, including the ones from the American Society of Colorectal Surgeons, there [00:15:00] is a disclaimer. It is not meant to be a legal disclaimer, but it is in there saying that they're not proscriptive. The guidelines provide information upon which decisions can be made.

They don't dictate any specific form of treatment. They're not intended to do anything other than to simply provide information for those who desire the care that are addressed by the topics. We even further put in there the fact that the guidelines should not be inclusive of all proper methods of care, exclusive of other methods, and ultimate judgment is to the individual treating physician given what that individual patient presents with.

This has been a really great discussion about the methodology. Now I want to bring all the things we've been talking about back to a clinical scenario. So you're on call, and you get called to the emergency department about a twenty-nine-year-old woman who's thirty weeks pregnant with her first child.

She's coming in with right lower quadrant abdominal pain that started about five hours ago and has been steadily worsening. She reports a fever, nausea, emesis, and diarrhea, and her labs show leukocytosis with no other [00:16:00] significant abnormalities. The ED obtained an MRI which demonstrates acute appendicitis with an appendicolith.

So what guidelines would you use to guide us with this case?

So there are three that we're gonna mention here from SAGES. The first is the panel suggests that appendectomy rather than non-operative treatment be used for acute appendicitis during pregnancy, and this is a conditional recommendation with very low certainty of evidence.

The second is the panel suggests that laparoscopic appendectomy rather than open appendectomy be used for acute appendicitis when the fundus of the uterus is below the umbilicus, and this was based on expert opinion due to the very low quality of evidence available. The third is the panel suggests that laparoscopic or open appendectomy be used for acute appendicitis when the fundus of the uterus is above the umbilicus at the surgeon's discretion.

The panel additionally suggests the open establishment of pneumoperitoneum when the laparoscopic approach is utilized, and this was based [00:17:00] on expert opinion, again, due to the very low quality of evidence available.

Thank you for that summary. I know SAGES guidelines are our go-to a lot of the times, but there are other organizations out there and sometimes guidelines conflict with each other.

So do you have any tips to quickly determine what guideline to follow or what organization to go with when that happens?

Yeah. I think that's so important to remember that not all guidelines, recommendations, and consensus statements are created equal. There are formal appraisal tools that we can use like the AGREE Instruments, and they evaluate guideline quality, but they're not really practical for most busy clinicians on a day-to-day basis.

Instead, there are a few things I like to keep in mind. First, are the clinical questions clearly defined with an intervention and comparator? Second, was the literature search comprehensive and robust? Third, how was the expert panel selected, and does it include the appropriate stakeholders, and is it multidisciplinary?

And finally, are the methods transparent and consistent? [00:18:00] In other words, did they systematically assess the evidence and clearly explain how they arrived to their recommendations? And if you can't answer one or more of those questions, it's worth interpreting the recommendation with a little bit more caution.

It doesn't necessarily mean that the guideline or consensus is wrong, but it does increase the possibility of bias and may limit how applicable those recommendations are to different patients or clinical settings or your specific situation.

Yeah, that's great. That's a easy checklist to go down. I think that's really helpful.

Now, building on this in the context of the current guideline-based recommendations, how would we manage this patient if we were being consulted?

As Lisa mentioned, the best use of the guideline comes when the key questions, the clinical questions are clearly defined. In this case, maps directly into the guideline.

So as we heard, we have a pregnant woman with MRI-confirmed appendicitis with an appendicolith, fever, leukocytosis, progressive pain. The picture is certainly not ambiguous. And there are two [00:19:00] parts to the answer the clinician is looking for, right? Should we use antibiotics or surgery, which has been a big topic of conversation in the adult population in Europe and the US.

And if we pick surgery, what would the best approach be for a pregnant woman? So on the first question, operative versus non-operative management, the guideline makes a conditional recommendation in favor of appendectomy. The evidence certainly is very low. Pregnant women were excluded from major antibiotic trials like the CODA trial.

But what really drives the recommendation is the biology. An appendicolith substantially increases the risk of perforation and failure of non-operative management, even in non-pregnant adults. In pregnancy, the stakes are compounded for the mother and child. And additionally, for a pregnant woman with appendicitis, there is two and a half times more likelihood to experience a preterm birth and then one and a half times more likelihood to suffer an intrauterine death.

And those numbers worsen if you experience a complication like perforation. So this patient definitely goes to the operating room. [00:20:00] Regarding the approach, at 30 week, the fundus is above the umbilicus, and the guideline specifically addresses this. It leaves the choice between laparoscopic and open appendectomy to the surgeon's discretion with a recommendation to use open entry for a null peritoneum if laparoscopy is chosen.

This is expert opinion, and similar to the example that Scott made about the pilot idol guideline, this is far from strong evidence, and the guideline is transparent about that distinction. The grade framing that we're using to put these guidelines together is really important for our listeners and something that definitely any practicing physician should at least be aware of.

Both recommendations are conditional, meaning that the panel is saying we suggest, not we mandate. And several recommendations in this guideline were driven by expert opinion because the evidence base in pregnant population is so thin. So that's precisely where the physician's judgment steps in. It reflects the reality of how these recommendations are built.

You guys have mentioned the expert opinion a few times. So when [00:21:00] there's an expert opinion in guidelines, what does that really mean, and how does that change how you use it in your clinical practice compared to a conditional or strong recommendation?

I can touch on that. So most guidelines classify their statements into three broad categories.

One, strong recommendations, two, conditional, also known as weak recommendations, and three, expert opinion, which I think we've all talked about throughout this conversation. But to get into the weeds of it, a strong recommendation means the panel is confident that the benefit clearly outweighs the harm or vice versa.

And in general, you would apply it to most patients, and most clinicians should follow it in routine practice. Conditional or weak recommendations reflect more uncertainty in the data, and again, that just means that you're taking into consideration the balance of the benefits and harms, but also patient values, available resources, et cetera, when you're making your decision.

These recommendations or conditional recommendations are more dependent on the individual patient and the clinical setting [00:22:00] that you're in, so shared decision-making becomes especially important. Finally, you sometimes see expert opinion, and this is used when there isn't enough direct evidence to make a formal evidence-based recommendation or when the recommendation is based on overwhelming clinical experience rather than studies that can be systematically analyzed.

As we mentioned earlier, this is how guidelines used to be made all the time. We would just use expert opinion. The panel is essentially saying that based on our expertise and the available information, this is what we believe is the best approach. So in practice, you should have a good reason for deviating from a strong recommendation.

Conditional recommendations deserve thoughtful consideration but allow more flexibility based on patient preferences and local resources. And expert opinion can provide valuable guidance, especially for uncommon or understudied situations, but it's important to recognize where it's coming from, which is largely expertise rather than comparative data.

And I think that in surgery, we have the unique situation where our [00:23:00] evidence base is not as strong as perhaps in non-surgical or medical big pharma, so we do often lean on our expert opinions when we don't have data.

That was very well said, and this has just been a, such a great episode. So Francesca, one of the things that has come up as an opportunity to have a little bit more involvement, even at a trainee level in terms of guidelines, learning how to do guidelines.

Can you tell the listeners a little bit about that?

Yeah. Thank you, Scott. We have a two-year fellowship that's been sponsored by SAGES for now a number of years that is a great opportunity for all surgical trainees to be part of. This is something that becomes available both through SAGES and through the Royal College of Australia that is sponsoring Elisa's position, and are both becoming available on a rotation every other year.

I think that would be a great opportunity for anyone interested in learning more and publishing a lot and really making a difference during their research years.

Yeah, if I could add, I do think this is a wonderful opportunity for our [00:24:00] surgical trainees. There is information when the applications are available on the SAGES website, so look out for that, and another great opportunity to interact with a number of surgeons.

A giant thank you to our guests for joining us today and sharing their expertise and opinions. And as always, we want to thank our listeners here on BTK.

I couldn't agree more. I really appreciate you guys breaking down the methodology behind clinical practice guidelines in a way that's digestible, because it's definitely a hard topic.

But if there's a guideline that you want to have us break down next, please reach out to us at hello@behindtheknife.org. We would love to hear from you. Now, until next time.

Dominate the day.

Dominate the day.