A brief appraisal on the GRADE methodology and the development of recommendations on the management of prolonged air leaks by the PALAS Study Group

Introduction

Reduction of practice variation in prevention and management of prolonged air leak (PAL) after pulmonary resection remains one of the most critical issues to be solved, especially in the era of Enhanced Recovery After Surgery (ERAS). The lack of evidence-based guidelines on PAL burdens the routine activity of the thoracic surgeon, especially in the younger generation. Recently, the PALAS Study Group, through an international 3-round Delphi survey, reached an agreement on some relevant issues concerning the detection and management of intraoperative alveolar air leak and PAL but failed on others (1).

Therefore, it is a top priority for the PALAS Study Group to provide new generations of thoracic surgeons with clear and easily applicable recommendations on preventing, detecting, and managing PAL after pulmonary resection. Here, we describe the methodology that we will use.

What is GRADE methodology?

Grading of Recommendations, Assessment, Development and Evaluation (GRADE) is a tool that systematic reviewers and guideline developers use to evaluate the quality of evidence and determine whether to propose an intervention (1). GRADE is distinct from previous appraisal tools. It separates evidence quality and recommendation strength and assesses the quality of evidence for each outcome. Lastly, observational studies can be upgraded if they meet specified criteria. The GRADE technique consists of five distinct steps.

• ❖ Step 1: assign randomised controlled trials a high ranking and observational studies a low ranking a priori. Randomised controlled trials receive a better beginning grade than observational research because they are typically less prone to bias.

• ❖ Step 2: diminish or increase the initial rating. It is not unusual for randomised controlled trials and observational research to be downgraded due to detectable bias. Additionally, observational studies can be enhanced when high-quality research demonstrates consistent findings.

• ❖ Step 3: for each crucially significant outcome, provide a final rating of the evidence’s quality as high, moderate, poor, or very low (Table 1). Upgrades and downgrades necessitate sound judgment and consideration. As a general guideline, one category per issue should be moved up or down. For example, the quality of evidence for an outcome studied in randomised controlled trials may begin as high and decrease to moderate due to a high risk of bias in the included studies, then to low if there was significant unexplained heterogeneity between the trials, and finally to very low if there were few events leading to confidence intervals that included both significant benefits and harms (Table 2).

• ❖ Step 4: consider other variables that affect the intensity of a suggestion for a course of action. A strong recommendation does not always follow high-quality evidence. Apart from the quality of evidence, recommendations must consider other variables. The first consideration is the balance of desired and unintended consequences. Specific therapies, such as antibiotics prophylaxis to prevent infections associated with certain thoracic operations, have benefits and few adverse effects, making a solid recommendation uncontroversial. When the benefit-harm ratio is unclear, it is critical to examine patient values and preferences and expenses carefully.

• ❖ Step 5: provide a recommendation that is either strong or weak. When the treatment’s net benefit is evident, patient values and circumstances are unlikely to influence the decision to pursue treatment, and a strong recommendation is justified (3).

Table 1. Grade the overall certainty of evidence (2).

Grade	Quality level	Definition
A	High	We are confident that the actual impacts closely match the estimated effects
B	Moderate	The true impacts are likely to be similar to the estimates, but there is a chance that they are significantly different
C	Low	The actual impacts may differ significantly from the estimated effects
D	Very low	The estimates are quite uncertain and frequently will be wildly inaccurate

Table 2. Method of rating the certainty of the evidence for an outcome (2).

Step 1: starting grade according to study design

Randomised trials = high

Observational studies = low

Step 2: lower if

Risk of bias

Serious

Very serious

Inconsistency

Serious

Very serious

Indirectness

Serious

Very serious

Imprecision

Serious

Very serious

Publication bias

Serious

Very serious

Step 3: higher if

Large effect

Large

Very large

Dose response

Evidence of a gradient

All plausible confounding

Would reduce a demonstrated effect

Would suggest a spurious effect when results show no effect

Step 4: determine final grade for quality of evidence

High

Moderate

Low

Very low

The GRADE summary of findings tables provides quick access to the most current evidence. To serve patients, as mentioned previously, a thoroughly informed dialogue about the benefits, risks, and practical implications of choosing one alternative over another is necessary. Utilising the medical literature to resolve patient concerns necessitates the transformation of these interactions into structured clinical inquiries. To provide satisfactory answers to clinical concerns, strong systematic reviews and appropriate meta-analyses are required that convey the best available data in easily digestible formats, rather than relying on selective interpretation of the literature. GRADE’s solution is to summarise findings tables in systematic reviews and guidelines. The tables of summary findings include vital outcomes, the number of studies and patients, the relative and absolute impacts, the degree of certainty (quality) of the findings, and straightforward language (4). Frequently, surrogate results do not transfer into an actual benefit (5,6). Instead, GRADE users place a premium on patient-centred outcomes. As opposed to important but not critical outcomes, defining crucial outcomes for patients can be accomplished through formal processes such as systematic reviews of pertinent literature and patient focus groups. Patient-important outcomes are classified into two categories: binary and continuous. Dichotomous outcomes refer to events that can occur or do not occur. The occurrence of such events in treated and untreated individuals could be assessed in relative terms using risk ratios or odds ratios or in absolute terms using hazard ratios for time-to-event outcomes (survival analysis) (4).

Other techniques

Other techniques for developing evidence-based clinical guidelines have been created in the past. The Scottish Intercollegiate Guidelines Network (SIGN) was established primarily to address effectiveness issues. The SIGN method evaluates the quality of evidence-based on the type of study in which it was published and places a premium on trial design rather than trial quality. SIGN assigns quality to studies based on their design, with little regard for their outcomes or publishing bias. For instance, using the SIGN classification approach, even the most poorly conducted randomised controlled trials can be classed as valuable, and any recommendations based on this evidence are scored as strong. While SIGN is subjective, the categorisation system requires that all recommendations from randomised controlled trials or meta-analyses of randomised controlled trials obtain a high-quality grade (even if there is a substantial probability of bias), making the method reasonably reproducible. On the contrary, the GRADE system considers additional factors when making recommendations, but they are not included in assessing the quality of evidence.

Why GRADE methodology?

Medicine is a challenging profession to practice. All clinicians battle every day and throughout their careers with fiduciary responsibilities. They can, however, refer to recognised principles of optimal clinical decision-making, several of which are related to evidence-based practice. GRADE is a critical component of evidence-based medicine (EBM). It is widely recognised as the most acceptable approach for summarising and interpreting data and guiding decision-making (4).

In 2004, a panel of international specialists in methodology and practice guidelines produced the initial version of the GRADE technique to assess the quality of evidence supporting medical interventions and develop recommendations. Since then, the group has released a six-part series aimed at clinicians who use GRADE guidelines and a series of papers aimed at assisting writers of systematic reviews and guideline groups that use GRADE in their work. Since its inception, over 100 organisations worldwide have endorsed or implemented GRADE, including the World Health Organization, the Cochrane Collaboration, the Joanna Briggs Institute, the American College of Physicians, DynaMed Plus, and UpToDate. Any physician employing formal guidelines or recommendations in online literature such as UpToDate is now unlikely to escape encountering GRADE suggestions. The GRADE approach’s popularity stems from its numerous advantages: establishing rigorous and comprehensive criteria for assessing the quality of evidence (also known as certainty or confidence in evidence); assessing the quality of evidence using systematic summaries of the entire body of relevant studies rather than selected individual studies; assessing the quality of evidence for each relevant outcome; progressing from evidence to recommendations, evaluating the importance of outcomes from a patient perspective, including both benefit and harm (clinicians, patients and policymakers) (7).

The limitations of GRADE methodology

An often-noted shortcoming of GRADE standards is that they are narrowly focused on a subset of the broader area. GRADE always prioritises the most rigorous evidence available for each outcome. This may be different for beneficial outcomes (randomised controlled trials are more likely to explore) than for uncommon unfavourable effects (requiring observational studies). GRADE does not rely on unsupported opinions. Third, many suggestions in many guidelines are conditional, which might be a source of frustration. The issue is not with GRADE but with the low quality of evidence, which makes decisions (extra) susceptible to patient values and preferences and other contextual factors. Conditional recommendations are subject to change considering new evidence. Thus, guidelines are crucial in identifying research gaps and driving the generation of new practice-changing data that would justify a guideline update (4). In some situations, the grade for the internal validity of a study could properly lower the overall grade by three grades (e.g., from high to very low) as opposed to the two-grade decline advised by the GRADE system (8).

On the other hand, GRADE is less concerned with the type of study and evaluates numerous aspects of the available data when determining the quality. Nonetheless, the GRADE system is time-consuming since it requires users to examine various factors when assessing quality, subject to some degree of subjectivity. Thus, one legitimate critique of the GRADE system is that it looks to have been established by academics for academics. If the greater complexity of GRADE in comparison to other approaches prevents clinical guideline developers from comprehending the process, GRADE will be unable to achieve its goal of standardisation. One possible option is to develop alternate techniques for displaying data to make the recommendations more user-friendly for surgeons, who are the ultimate users. Surgeons may find the guidelines less complicated and easier to comprehend if merely the quality assessment outcome was published. This suggestion was created following a rigorous evaluation utilising the GRADE approach (9).

Conclusions

Clinicians make several daily decisions with their patients on the use of tests or therapies. The typical approach to these decisions is based on the clinician’s training and expertise. However, applying EBM concepts to comprehend and use evidence about management options and knowledge of the critical nature of patients’ values and preferences are required for effective patient care. Clinicians employing GRADE-compliant guidelines will examine the suggestions’ applicability to the patient in front of them. Clinicians working with developing evidence and/or limited solid advice will assist patients best by utilising the GRADE domains (4). GRADE is expected to play a variety of functions in the broader ecology of evidence. Innovative approaches may improve the efficiency of systematic reviews and GRADE applications. The rapid recommendations collaboration between the British Medical Journal and the Making GRADE the Irresistible Choice (MAGIC) group is an example of this initiative, which uses GRADE methodology to rapidly generate a small number of recommendations addressing a specific clinical context in response to potentially practice-changing randomised controlled trials. Additional GRADE-related activities involve patient engagement in evaluating the evidence and providing recommendations (7). For instance, the Italian Association of Medical Oncologist (AIOM) Clinical Practice Guidelines were developed in accordance to the GRADE methodology (10-12). Nonetheless, obstacles persist. To fully exploit the benefits of GRADE, it will be necessary to improve GRADE’s understanding and application in guideline or recommendations creation and at the bedside. Thoracic surgeons’ associations should endeavour to strengthen their members’ understanding and application of GRADE concepts, and those responsible for developing clinical practice guidelines should frequently incorporate the GRADE approach. Clinicians committed to providing the best possible care for their patients will incorporate GRADE principles into their daily practice.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.