Abstract
Background
The Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument has been developed to inform the methodology, reporting and appraisal of clinical practice guidelines. Evidence suggests that the quality of surgical guidelines can be improved, and the structure and content of AGREE II can be modified to help enhance the quality of guidelines of surgical interventions.
Objective
To develop an extension of AGREE II specifically designed for guidelines of surgical interventions.
Methods
In the tripartite Guideline Assessment Project (GAP) funded by United European Gastroenterology and the European Association for Endoscopic Surgery, (i) we assessed the quality of surgical guidelines and we identified factors associated with higher quality (GAP I); (ii) we applied correlation analysis, factor analysis and the item response theory to inform an adaption of AGREE II for the purposes of surgical guidelines (GAP II); and (iii) we developed an AGREE II extension for surgical interventions, informed by the results of GAP I, GAP II, and a Delphi process of stakeholders, including representation from interventional and surgical disciplines; the Guideline International Network (GIN); the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group; the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) initiative; and representation of surgical journal editors and patient/public.
Results
We developed AGREE‐S, an AGREE II extension for surgical interventions, which comprises 24 items organized in 6 domains; Scope and purpose, Stakeholders, Evidence synthesis, Development of recommendations, Editorial independence, and Implementation and update. The panel of stakeholders proposed 3 additional items: development of a guideline protocol, consideration of practice variability and surgical/interventional expertise in different settings, and specification of infrastructures required to implement the recommendations. Three of the existing items were amended, 7 items were rearranged among the domains, and one item was removed. The domain Rigour of Development was divided into domains on Evidence Synthesis and Development of Recommendations. The new domain Development of Recommendations incorporates items from the original AGREE II domain Clarity of Presentation.
Conclusion
AGREE‐S is an evidence‐based and stakeholder‐informed extension of the AGREE II instrument, that can be used as a guide for the development and adaption of guidelines on surgical interventions.
Keywords: AGREE II, AGREE‐S, evidence, guidelines, methodology, quality
INTRODUCTION
The landscape of clinical practice guidelines has changed over the past decades. The guideline ecosystem has shifted from a consensus‐based approach, to a structured, transparent and evidence‐informed development of recommendations for clinical practice and health policy. 1 , 2 , 3 , 4 , 5 , 6 Major organizations and initiatives, such as the Grading of Recommendations Assessment, Development and Evaluation (GRADE), the Guidelines International Network (GIN), the Scottish Intercollegiate Guidelines Network (SIGN), the National Institute for Health and Care Excellence (NICE), and the World Health Organization (WHO), among other, have fostered a culture of evidence‐informed methodology in guideline development. 7 , 8 , 9 , 10 , 11 , 12
Key summary.
AGREE II was developed to inform the methodology of clinical practice guidelines.
Research suggests that adaption of AGREE II to be used in guidelines of surgical interventions is needed.
The AGREE‐S methodological guide is an AGREE II extension for surgical interventions, and has been developed through a systematic, structured, evidence‐based and stakeholder‐informed consensus.
The AGREE (Appraisal of Guidelines for Research and Evaluation) initiative has created a comprehensive framework for optimal development, reporting and appraisal of clinical practice guidelines. 13 The AGREE instrument was developed in 2003 and updated as AGREE II in 2010 by a multidisciplinary team of researchers and experts in health policy and guideline development. 14 AGREE II is a generic instrument that can be applied across different disciplines in healthcare. 15 Empirical and research evidence suggests that the quality of an important proportion of surgical guidelines is poor. 16 Development of an instrument to enhance methodological rigour and improve the quality of surgical guidelines appears a reasonable scientific objective.
Surgery is a discipline with distinct characteristics in health care delivery and with unique features of its evidence ecosystem. Un‐like non‐surgical and pharmacological guidelines, surgical experience and expertise play an essential role in evaluation of the evidence and in the external validity of practice recommendations. Furthermore, organizational and health system infrastructures affect the implementation of guidelines on surgical interventions. New interventional techniques are mastered and frequently advocated by experts in the field, whose input is often indispensable in the development of relevant guidelines; however, their contribution may be considered to be biased due to competing interests. These parameters are not specifically addressed in the AGREE II instrument. In addition, some items of the AGREE II instrument might not be fully applicable to surgical guidelines in their original form. For instance, resource implications can rarely be considered within an evidence‐based context, due to the scarcity of cost‐effectiveness studies in the field of surgery. Applicability considerations need to take into account not only the healthcare setting, but also the surgeons' experience and expertise, which is underreported in surgical guidelines. Such considerations prompted an international, multidisciplinary group of stakeholders to develop an extension of AGREE to be used in the development of surgical guidelines.
This article reports on the development of the AGREE‐S methodological guide, an AGREE II extension for guidelines on surgical interventions. The AGREE‐S reporting checklist and the AGREE‐S appraisal tool will be published separately.
METHODS
The project was named ‘Guideline Assessment Project (GAP): Filling the GAP in surgical guidelines’ and consisted of three parts, GAP I, GAP II and GAP III. The steering group developed a project protocol in advance. 17
In GAP I, we performed a systematic search to identify guidelines published by surgical organizations with an international scope. We assessed the quality of 67 guidelines using the original AGREE II appraisal instrument. We performed exploratory and post hoc analyses to identify variables associated with higher quality. These variables were (i) development by an organization regularly involved in guideline production (≥1 guideline per year), (ii) using the GRADE methodology, and (iii) development by an organization with a guidelines committee. 16
In GAP II, we performed correlation and reliability analyses, and applied the item response theory using the AGREE II scores obtained from GAP I. The statistical models suggested that removing and rearranging some items across domains and reducing the number of domains to 4 or 5 may increase the reliability of AGREE II in the context of surgical guidelines. 18
In GAP III, which is reported herein, we selected a panel of stakeholders to participate in a Delphi process to inform the development of an AGREE II extension for surgical interventions. We aimed for representation of various groups of stakeholders, including guideline developers, guideline users, patient advocates, representatives of organizations advocating transparency in research (Enhancing the QUAlity and Transparency Of health Research – EQUATOR) and quality guideline development and adaption methodology (GRADE, GIN), and for diversity in geographic and ethnic backgrounds.
We asked Delphi panelists to nominate new items to be included in the AGREE II extension. We thematically summarized their responses and presented them in the first Delphi round, together with the findings of GAP I and GAP II, asking candidate items to be included in, or excluded from, the AGREE II extension. We asked panelists to rate their importance on a 5‐point Likert scale, 1 indicating disagreement and 5 indicating agreement, and suggest a modification of the proposed items if they did not fully agree with the content or the formulation.
We modified items according to panelists' responses and presented them in a second iterative round. Consensus was defined as agreement (score of 4 or 5 on the Likert scale) among at least 80% of panelists. If consensus was reached in the first or second round, the item was considered eligible for inclusion, and if no consensus was reached after 2 rounds, the item was discarded. After two rounds of Delphi, 5 new items and 1 modified AGREE II item were shortlisted for inclusion, and 4 items were shortlisted for exclusion.
The steering group discussed the findings of GAP I, GAP II and GAP III in an in‐person consensus meeting. The findings of the three parts of the project informed the development of the AGREE‐S – AGREE II extension for surgical interventions.
The development of this project was supported financially by United European Gastroenterology and the European Association for Endoscopic Surgery (EAES). The intellectual property lies with the authors of this manuscript, the GAP Consortium, and EAES.
AGREE‐S METHODOLOGICAL GUIDE
Scope and purpose
-
1.
The guideline is developed according to a protocol. – New item
A priori development of a protocol serves as a reference for the guideline development (or adaption) group and it reduces bias in the development process. The protocol does not need to be excessively detailed, but it should provide fundamental methodological principles (e.g., key questions, evidence search and synthesis approaches, funding source, and panel members, among others). Deviations from the protocol may be necessary, but these must be justified and reported in the executive summary. The topic of the guideline is usually prespecified, however the selection of specific question frameworks may be informed by the panel. 19 The steering group may prepare the protocol in collaboration with the evidence synthesis group and the guideline panel. It may also be made available through international guidelines registries, 20 the organizational website, the social media, or other means, for public input.
-
2.
The guideline has (a) specific overall objective(s).
The guideline should have a specific and clear objective that drives its development (or adaption). Guidelines on surgical interventions may aim to improve the outcomes and experience of patients with a specific surgical disease or condition, to improve the diagnostic process, to enhance the prevention of disease and/or to inform policymakers and healthcare authorities.
-
3.
There is/are specific health question(s) to be covered by the guideline [patient, interventions/procedures, outcomes]. – Modified item
Defining the clinical question(s) early in the process of guideline development is of paramount importance. The clinical question(s) should be structured within a question framework (e.g., PICO – patient, interventions/procedures, outcomes, for questions on interventions; patient, index test, reference test, outcomes for questions on diagnostic tests). Framing the question avoids making spurious assumptions and judgements when developing the recommendations. Generic questions such as “Which is the best approach to treat patients with malignant biliary obstruction?” do not define the competing interventions and will inevitably lead to confusion among the evidence research group, panel members, and other involved stakeholders. A structured question, such as “Should percutaneous biliary drainage be preferred over endoscopic stent placement for the management of patients with malignant obstruction of the extrahepatic biliary system?” will facilitate individual processes of evidence outreach, summary, appraisal and development of recommendations. Defining the outcomes is best made through panel consensus (including patient representatives or advocates), for example, by grading their importance 21 ; therefore, outcome measures may not be available at the outset.
Stakeholders
-
4.
The guideline is supported by a guideline development committee, including a guideline methodologist. – New item
Experts in the field facilitate the process of guideline development (or adaption). A guideline methodologist will define key methodological parameters, will instruct the panel how to appraise the evidence and will coordinate the process, will accommodate the discussion on how the evidence and other considerations will inform the recommendations (e.g., evidence to decision framework), and will safeguard the quality of the guideline development process. Evidence suggests that surgical guidelines produced by organizations with an ad hoc guidelines committee are of higher quality. 16
-
5.
The guideline development group includes individuals from all relevant professional groups and patients. – Modified item
The guideline panel should include all stakeholders affected by the content of the guideline. These may be other than surgical and interventional specialists, such as primary care physicians, nurses, and physiotherapists. For example, a guideline on robotic colorectal surgery might involve general and colorectal surgeons, oncologists, radiotherapists, pathologists, nurses, healthcare economists, surgical technology specialists, and patient representatives. Patient representatives or patient advocates should participate as ordinary panel members, with equal contribution and voting rights from the start of the guideline development process. The involvement of patient representatives might not always be feasible, for example, in a guideline on orthopaedic surgery in the octogenarians. On such occasions, patient advocacy groups or representatives of caregiver support groups might need to be involved. Guideline development (or adaption) groups should aim for at least two representatives from each stakeholders' group. Particular care should be given to avoid the over‐ or underrepresentation of stakeholders, depending on the topic of the guideline. Furthermore, the guideline development process should be supported from a diverse group of people, such as experts in evidence outreach (e.g., health information specialists), evidence synthesis (e.g., statisticians, methodologists), and health economists. Individual roles and tasks should be specific and defined at the outset.
-
6.
There are specific target users of the guideline. – Modified item
The guideline should be developed (or adapted) to be used by specific stakeholders. These may be surgeons, physicians, nurses, allied healthcare professionals, policymakers, or the public. Different considerations apply depending on the perspective of the guideline. For example, a guideline designed to inform policymakers has to consider cost and implementation issues. In contrast, a guideline developed to inform exclusively patient decision making might not need to focus on such issues. Other considerations refer to the appraisal of the certainty (quality) of the evidence. Guidelines developed to inform population‐based practices are best informed by intention to treat analyses (which are usually provided by randomized trials on surgical interventions). In contrast, from the patient perspective, per‐protocol analyses are more relevant. 22
Evidence synthesis
-
7.
Systematic methods are used to search for evidence.
Guidelines should be informed by all available evidence on a specific topic and avoid biased consideration of a fragment of evidence. This makes a systematic review of the literature imperative. The review is ideally performed de novo by an evidence review team. This is usually a labour‐intensive process and requires time and financial resources, and methodological expertise. Furthermore, such process requires development of an appropriate search strategy, with careful selection of thesaurus and search terms, subject headings, truncated terms, search limits, and additional topic‐specific search term combinations to capture the desired study design(s). Collaboration among the evidence search group, the evidence synthesis group, the guideline panel, and the steering group is of particular importance.
Alternatively, the guideline development group may use one or more existing systematic reviews to summarize the evidence that will inform the development of recommendations. However, these must be recent and of high quality (e.g., evaluated using the AMSTAR – A MeaSurement Tool to Assess systematic Reviews 2 tool 23 ), and they should provide evidence on the outcomes prioritized by the guideline panel. Another option is to update an existing systematic review, for example, through collaboration with the authors of the original review. This will allow updating of their work, tailoring it to the needs of the guideline project, and extracting information of interest that was not captured in the original review.
-
8.
There are specific criteria for selecting the evidence.
As in any systematic review, a guideline needs to specify criteria for selecting the evidence. These refer to the study design or type of evidence (e.g., randomized trials, observational studies, diagnostic test accuracy studies, surveys, focus groups) and the question framework (patient/population, intervention or index diagnostic test of interest, comparator or reference test). These criteria are ideally specified at the protocol stage; however, the guideline development group might need to deviate from prespecified criteria and consider alternative sources depending on the available evidence. For instance, a guideline focussing on the management of abdominal aortic aneurysms in patients over the age of 80 might need to be indirectly informed from evidence on patients over the age of 65, when appropriate evidence for the former group is not available. Such considerations should be transparently documented and be made available to guideline users.
-
9.
The strengths and limitations of the body of evidence are considered.
The guideline should consider and document the strengths and limitations of the evidence that informed the recommendation(s). Limitations may be related to the risk of bias of individual studies, indirectness of the evidence (available evidence not addressing the guideline‐specific population, intervention or diagnostic test, comparator, reference test, and/or outcome of interest), the certainty of the comparative effect estimate (imprecision), and other issues, such as heterogeneity and publication bias. Such considerations are summarized in the GRADE assessment of the certainty (quality) of the evidence 24 and can be presented in evidence tables. Guideline developers (or adaptors) should explain how these limitations affected the recommendation(s). For instance, GRADE suggests that the overall certainty of evidence on a question framework be defined by the lowest certainty of evidence on critical outcomes. A strong recommendation can rarely be provided when the overall certainty is below moderate.
Development of recommendations
-
10.
The views and preferences of the target population (patients, public, etc.) are considered.
A guideline development group is expected to involve patients and/or the public in the guideline development or adaption process. Patient representatives or patient advocates should be involved from the outset, be regular members of the guideline panel, and have equal opportunities to express their opinion and the same voting rights as the other panel members. Alternatively, patient/public views and preferences can be informed by available research, ad hoc surveys, or interviews with focus groups. Patients' and public input is particularly important for selecting patient‐centred critical and important outcomes, and for defining clinically meaningful differences. 19 , 21 , 25 Guideline developers may find the AGREE‐REX (AGREE Recommendation EXcellence) useful when formulating their recommendations under consideration of patients'/public's views. 26 .
-
11.
There are specific methods for formulating the recommendation(s).
Guideline developers (or adaptors) should predefine the methodology that will be used to formulate the recommendation(s). A specific, predefined, transparent methodology to develop (or adapt) the recommendation(s) will avoid the common pitfall of arbitrary decisions on the strength and the direction of the recommendation(s). For instance, GRADE suggests using an evidence to decision framework, which consists of the certainty of the evidence, the balance between benefits and harms, acceptability, feasibility, equity, economic/resources considerations, and patients' views and preferences. 27 Agreement among most of these parameters in favour of an intervention, in the presence of at least moderate certainty evidence on critical outcomes, suggests that a strong recommendation may be provided. Methods to reach consensus among panel members are ideally defined a priori and can involve iterative discussions, Delphi process, or voting.
-
12.
The health benefits, side effects, and risks are considered in formulating the recommendations.
In the context of evidence appraisal, harms of an intervention are outcomes in which the comparator is better than the intervention. It is of specific importance to consider both benefits and harms when developing recommendations. Failure to consider harms may result in the false confidence that an intervention is superior or inferior to the comparator. The balance between benefits and harms may need to be informed by the magnitude of the effect of the benefits and the respective magnitude of harms, along with the importance of these outcomes.
-
13.
There is an explicit link between the recommendations and the supporting evidence.
Guidelines are informed by research and empirical evidence, and through the input of stakeholders and experts in the field. Guideline users should be provided with sufficient information to review what type of evidence informed the recommendation(s) and how. Such information can be summarized in evidence summaries, systematic observation forms to retrieve expert‐based evidence, and summaries of evidence on cost, patients' values and preferences, acceptability and feasibility. For example, the link between the evidence and recommendations is summarized in evidence tables and the evidence to decision framework tables within the context of GRADE. 27
-
14.
The recommendations are specific and unambiguous.
The recommendation(s) should be unambiguous, concise, actionable and clearly formulated. The wording of the recommendation(s) should be precise and reflect its/their strength (e.g., “We recommend…” for strong recommendations and “We suggest…” for weak/conditional recommendations). It should also indicate the direction of the recommendation (e.g., “laparoscopic hysterectomy over open hysterectomy”). Lengthy recommendations with complex language may result in misconceptions about the recommended course of action and the strength of the recommendation. Such wording as “Perioperative thromboprophylaxis may be recommended…“, “…could be suggested”, or “should be considered” produce confusion among target users as to whether an intervention is strongly recommended or is suggested as the best alternative in most situations. Furthermore, recommendations may need to specify the required setting, resources, or surgical/interventional expertise for their implementation; for example, “Robotic prostatectomy is recommended over laparoscopic or open prostatectomy when a robotic platform is available, and surgical expertise is in place”.
-
15.
The different options for management of the condition or health issue are considered.
Different interventional and non‐interventional options and alternatives for treatment, management, diagnosis, prevention or screening might be appropriate for different patient populations or people at risk, and in different settings, regarding the availability of financial resources and infrastructures, surgical or interventional experience and expertise. Such options should be taken into consideration when formulating the recommendations.
-
16.
Key recommendations are easily identifiable.
Recommendations should be highlighted in the guideline publication, executive report, organizational website, etc. They must be clearly distinguished from supporting information. The authors may provide a list or table with the recommendations, and they may also be highlighted in the text, to facilitate identification. Algorithms or flow charts may be helpful in transferring a recommendation or a set of recommendations to clinical practice and decision making. Notably, the guideline manuscript should not contain in the supporting text statements that may be considered recommendations.
-
17.
The potential resource implications of applying the recommendations are considered.
Resource considerations, including the cost of interventions, instruments, hospitalization, reinterventions, infrastructures (e.g., the availability of a hybrid angiosuite for combined surgical and endovascular interventions), and specific features of organizations or health services should be considered when developing the recommendation(s). This is particularly important for guideline developers, who wish to adapt existing guidelines to be used in specific settings with defined resources. Every effort should be taken to identify relevant published evidence (e.g., economic evaluations, such as cost‐effectiveness analyses), or to perform such analyses during the guideline development or adaption process. Suppose no such information is available and there are no sufficient resources to perform de novo analyses. In that case, the guideline development group may seek input from an expert in economic analyses, or summarize the panel's collective experience regarding the use of resources. High demands on resources (e.g., high cost, limited availability of an interventional technology, or lack of wide‐scale expertise) may prompt guideline developers to provide a weak recommendation or even recommend against an intervention, even when it is more effective than the comparator.
-
18.
The guideline considers potential variability in surgical expertise of those performing the interventions/procedures. – New item
Surgical and interventional experience and expertise may affect the outcome of an intervention. Studies comprising the background evidence of a recommendation may report on the experience and expertise of those performing the interventions. The guideline should specify the experience and expertise required to perform the recommended interventions (e.g., previous courses, hands‐on training, previous experience with a specific number of procedures/interventions, training of operation room staff). If expertise in an intervention is not widely available in a given context or geographical region, the guideline panel may abstain from providing a strong recommendation. Alternatively, they may issue a strong recommendation in a given context, such as in the presence of interventional or surgical expertise, or after appropriate training. Such considerations will also inform the decision of guideline adaption groups to issue a strong or weak recommendation in a specific setting.
Editorial independence
-
19.
The views of the funding body have not influenced the content of the guideline.
The guideline should be independent of any type of financial, intellectual, or professional influence. The funding body (which includes for‐profit or non‐profit organizations) should not influence the content of the guideline, and its role should be transparent, specific and defined in advance.
-
20.
Competing interests of guideline development group members are recorded and addressed.
All individuals who contribute to or influence the content of the guideline should disclose any direct (financial) or indirect (intellectual) conflict of interest. The Guidelines International Network recommends that the guideline chair and the guideline panel be free from either direct or indirect conflicts, and that professionals with expertise but with conflict of interest related to the interventions, diagnostic tests etc. discussed in the guideline may act as consultants, but do not participate in the decision process of the direction and the strength of the recommendation(s). 28 A strong opinion favouring an intervention, course of action, diagnostic test etc., may also constitute an indirect conflict of interest. The guideline steering committee, the guideline chair(s) and/or the methodologist may need to reallocate functions and responsibilities, or exclude (from parts or the whole guideline) contributors with relevant conflicts of interest.
Conflicts of interest should be documented at least at the outset and upon completion of the guideline development process. Standardized declaration forms of peer‐reviewed journals usually do not require declaration of indirect conflicts; therefore, a detailed statement may need to be made available by different means; for example, in a supplementary file or file repository.
Implementation and update
-
21.
The guideline considers facilitators and barriers to its application.
The guideline development (or adaption) group should identify any potential barriers to the implementation of the recommendation(s), such as lack of wide‐scale experience and expertise, limited resources (e.g., financial, technologies or instruments), resistance to change, organizational culture, limited awareness of the evidence, etc. Furthermore, the guideline development (or adaption) group may identify potential facilitators, such as cost‐effectiveness, reduced requirement of resources, or ease of implementation. Both facilitators and barriers should be considered when developing or adapting the guideline. They may also inform the decision on the strength of the recommendation(s), for example, through an evidence to decision framework. 27 Such information can be collected directly from stakeholders (e.g., the guideline panel) or through pilot testing of the guideline before widespread implementation.
-
22.
There is a procedure for updating the guideline.
The guideline should specify its validity period, which may inform the timing of its updating. This may be related to expected advances in the field, identification of ongoing trials, publication of new research, or accumulation of experience with new technologies. The guideline development group may need to perform a scoping search of clinical trial registries to identify ongoing trials, or they may map the trend of new publications over a specific time period, which may predict the publication of new evidence after a specific period of time.
-
23.
The guideline provides advice and/or tools on how the recommendations can be implemented.
The guideline development group may consider providing tools and resources to facilitate application, such as guideline summary documents, patient or public versions or lay summaries, treatment/management algorithms, how‐to manuals, solutions linked to barrier analysis, tools to capitalize on guideline facilitators, the outcome of the pilot test and lessons learnt, decision aids, or smartphone applications.
-
24.
The guideline presents monitoring and/or auditing criteria.
Assessment of the applicability, uptake and impact of the guideline should be among the objectives of guideline developers. This can be achieved through a survey of target users a period of time after publication, in order to appraise applicability, and to develop and measure indicators (e.g., reduction in the incidence of surgical site infection), related to key recommendations. 29 This can be provided in a supplementary file or other resources.
A summary of the AGREE‐S methodological guide is provided in Table 1.
TABLE 1.
Summary of the AGREE‐S methodological guide
| Scope and purpose |
|
| |
| |
| Stakeholders |
|
| |
| |
| Evidence synthesis |
|
| |
| |
| Development of recommendations |
|
| |
| |
| |
| |
| |
| |
| |
| |
| Editorial independence |
|
| |
| Implementation and update |
|
| |
| |
|
WHO SHOULD USE THIS DOCUMENT
This document is intended to be used by:
-
‐
Guideline developers and adaptors, as a summary of critical methodological principles.
-
‐
Guideline committees of organizations on surgical interventions, to inform the process and methodology of guideline development.
-
‐
Educators, to teach core competencies in guideline development.
HOW YOU SHOULD USE THIS DOCUMENT
This methodological guide does not intend to replace methodologies for development of guidelines in healthcare, such as GRADE and SIGN 50, 30 , 31 or guides detailing the development process, such as the GIN‐McMaster Guideline Development Checklist. 32 It is intended to inform the de novo development of guidelines on surgical interventions about best practices, summarizing key methodological features and complementing existing resources. The present document can also support the process of adaption of guidelines on surgical interventions, after appraisal and selection of candidate guidelines using the AGREE‐S and the AGREE II appraisal instruments, and AGREE‐REX. 14 , 33 Guidelines on surgical interventions with pharmacological or non‐surgical components (e.g., on perioperative antibiotic prophylaxis) may additionally need to be informed by the original AGREE II guide. The AGREE‐S methodological guide with supporting resources can also be accessed on the AGREE‐S website at https://agree‐s.org/ and the AGREE Trust website at https://www.agreetrust.org/.
CONCLUSION
The AGREE‐S methodological guide is intended to inform the process of development and adaption of guidelines on surgical interventions.
CONFLICT OF INTEREST
The authors declare no conflicts of interest related to this work. Ivan D Florez and Melissa Brouwers are leads of the AGREE Consortium. Detailed conflicts of interest statements of the authors are provided in http://osf.io/fau4d.
ETHICS APPROVAL STATEMENT
The NHS/HSC Research Ethics Committee evaluated the project and ethics approval was waived.
PATIENT CONSENT STATEMENT
Not applicable.
PERMISSION TO REPRODUCE MATERIAL FROM OTHER SOURCES
The authors obtained permission from the AGREE Consortium to reproduce parts of the AGREE II User's Manual and 23‐item Instrument.
ACKNOWLEDGEMENTS
The Guideline Assessment Project (GAP) III received financial support from the United European Gastroenterology (UEG) and the European Association for Endoscopic Surgery and Other Interventional Techniques (EAES), both non‐profit organizations. The funders had no role in the design or development of this project.
Logullo P, Florez ID, Antoniou GA, Markar S, López‐Cano M, Silecchia G, et al. AGREE‐S: AGREE II extension for surgical interventions – United European Gastroenterology and European Association for Endoscopic Surgery methodological guide. United European Gastroenterol J. 2022;10(4):425–34. 10.1002/ueg2.12231
Contributor Information
Stavros A Antoniou, Email: stavros.antoniou@hotmail.com.
The GAP Consortium:
The GAP Consortium, Yasser Sami Abdel Dayem Amer, Luca Bertolaccini, Pablo Alonso‐Coello, Elie A Akl, Manish Chand, John J Como, Gert J de Borst, Salomone Di Saverio, Sameh Emile, Bang Wool Eom, Ramon Gorter, George Hanna, Kaisa Immonen, Quirino Lai, Nicolaas Lumen, Joseph L Mathew, Alessandro Montendori, Martin Moya, Gianluca Pellino, Alvaro Sanabria, Athanasios Saratzis, Neil Smart, Dimitrios Stefanidis, and Giovanni Zaninotto
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1. Zhang R, Yan SY, Wang YY, Huang Q, Ren XY, Tan R, et al. Analysis of the status and trends of Chinese clinical practice guideline development between 2010 and 2020: a systematic review. Front Med. 2021;8:758617. 10.3389/fmed.2021.758617 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Zhang S, Wu L, Wang Y, Zhou Q, Luo X, Mathew JL, et al. Methodological and reporting quality of pediatric clinical practice guidelines: a systematic review. Ann Transl Med. 2021;9(15):1258. 10.21037/atm-21-2686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Siedler MR, Lamadrid P, Humphries MN, Mustafa RA, Falck‐Ytter Y, Dahm P, et al. The quality of physical activity guidelines, but not the specificity of their recommendations, has improved over time: a systematic review and critical appraisal. Appl Physiol Nutr Metabol. 2021;46(1):34–45. Epub 2020 Sep 29. 10.1139/apnm-2020-0378 [DOI] [PubMed] [Google Scholar]
- 4. Barker TH, Dias M, Stern C, Porritt K, Wiechula R, Aromataris E, et al. Guidelines rarely used GRADE and applied methods inconsistently: a methodological study of Australian guidelines. J Clin Epidemiol. 2021;130:125–34. Epub 2020 Oct 29. 10.1016/j.jclinepi.2020.10.017 [DOI] [PubMed] [Google Scholar]
- 5. Oxman AD, Schünemann HJ, Fretheim A, Boyd EA, Bero LA. Tan‐torres edejer T. Improving the use of research evidence in guideline development. Knowledge centre for the health services at the Norwegian Institute of public health (NIPH), report from Norwegian knowledge centre for the health services (NOKC); 11 2007. [PubMed] [Google Scholar]
- 6. Antoniou SA, Tsokani S, Mavridis D, Agresta F, López‐Cano M, Muysoms FE, et al. EAES Consensus , Guideline Subcommittee . Insight into the methodology and uptake of EAES guidelines: a qualitative analysis and survey by the EAES Consensus & Guideline Subcommittee. Surg Endosc. 2021;35(3):1238–46. Epub 2020 Apr 2. 10.1007/s00464-020-07494-8 [DOI] [PubMed] [Google Scholar]
- 7. Djulbegovic B, Guyatt GH. Progress in evidence‐based medicine: a quarter century on. Lancet. 2017;390(10092):415–23. Epub 2017 Feb 17. 10.1016/S0140-6736(16)31592-6 [DOI] [PubMed] [Google Scholar]
- 8. Terracciano L, Brozek J, Compalati E, Schünemann H. GRADE system: new paradigm. Curr Opin Allergy Clin Immunol. 2010;10(4):377–83. PMID: 20610980. 10.1097/ACI.0b013e32833c148b [DOI] [PubMed] [Google Scholar]
- 9. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P. Board of trustees of the guidelines international Network. Guidelines international Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525–31. PMID: 22473437. 10.7326/0003-4819-156-7-201204030-00009 [DOI] [PubMed] [Google Scholar]
- 10. Lowe G, Twaddle S. The scottish intercollegiate guidelines Network (SIGN): an update. Scot Med J. 2005;50(2):51–2. 10.1177/003693300505000202 [DOI] [PubMed] [Google Scholar]
- 11. National Institute for Health and Care Exce . llence (NICE) . Developing NICE guidelines: the manual Guidance NICE. 2022. [online]. https://www.nice.org.uk/process/pmg20/chapter/introduction. Accessed 27 January 2022.
- 12. Norris SL, Ford N. Improving the quality of WHO guidelines over the last decade: progress and challenges. Lancet Global Health. 2017;5(9):e855–6. PMID: 28807174. 10.1016/S2214-109X(17)30253-X [DOI] [PubMed] [Google Scholar]
- 13. Makarski J, Brouwers MC, Agree Enterprise . The AGREE Enterprise: a decade of advancing clinical practice guidelines. Implement Sci. 2014;9:103. PMID: 25123781; PMCID: PMC4148001. 10.1186/s13012-014-0103-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE Next Steps Consortium. AGREE II: advancing guideline development, reporting and evaluation in health care. J Clin Epidemiol. 2010;63(12):1308–11. Epub 2010 Jul 24. PMID: 20656455. 10.1016/j.jclinepi.2010.07.001 [DOI] [PubMed] [Google Scholar]
- 15. AGREE Next Steps Consortium . The AGREE II instrument [electronic version]. 2009. Retrieved from http://www.agreetrust.org 20 December 2021.
- 16. Antoniou SA, Tsokani S, Mavridis D, López‐Cano M, Antoniou GA, Stefanidis D, et al. Guideline assessment project: filling the GAP in surgical guidelines: quality improvement initiative by an international working group. Ann Surg. 2019;269(4):642–51. PMID: 30188402. 10.1097/SLA.0000000000003036 [DOI] [PubMed] [Google Scholar]
- 17. Antoniou GA, Mavridis D, Tsokani S, López‐Cano M, Flórez ID, Brouwers M, et al. Protocol of an interdisciplinary consensus project aiming to develop an AGREE II extension for guidelines in surgery. BMJ Open. 2020;10(8):e037107. PMID: 32784259; PMCID: PMC7418673. 10.1136/bmjopen-2020-037107 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Tsokani S, Antoniou SA, Moustaki I, López‐Cano M, Antoniou GA, Flórez ID, et al. Guideline Assessment Project II: statistical calibration informed the development of an AGREE II extension for surgical guidelines. Surg Endosc. 2021;35(8):4061–8. Epub 2021 Jun 22. PMID: 34159464. 10.1007/s00464-021-08604-w [DOI] [PubMed] [Google Scholar]
- 19. Wiercioch W, Nieuwlaat R, Zhang Y, Alonso‐Coello P, Dahm P, Iorio A, et al. New methods facilitated the process of prioritizing questions and health outcomes in guideline development. J Clin Epidemiol. 2021;143:91–104. Epub ahead of print. PMID: 34843861. 10.1016/j.jclinepi.2021.11.031 [DOI] [PubMed] [Google Scholar]
- 20. Chen Y, Guyatt GH, Munn Z, Florez ID, Marušić A, Norris SL, et al. Clinical practice guidelines registry: toward reducing duplication, improving collaboration, and increasing transparency. Ann Intern Med. 2021;174(5):705–7. 10.7326/m20-7884 [DOI] [PubMed] [Google Scholar]
- 21. Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395–400. Epub 2010 Dec 30. PMID: 21194891. 10.1016/j.jclinepi.2010.09.012 [DOI] [PubMed] [Google Scholar]
- 22. Higgins JPT, Savović J, Page MJ, Sterne JAC. On behalf of the RoB2 Development Group. Revised Cochrane risk‐of‐bias tool for randomized trials (RoB 2). https://www.riskofbias.info/welcome/rob‐2‐0‐tool/current‐version‐of‐rob‐2. Accessed 23 January 2022.
- 23. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non‐randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. PMID: 28935701; PMCID: PMC5833365. 10.1136/bmj.j4008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Guyatt G, Gutterman D, Baumann MH, Addrizzo‐Harris D, Hylek EM, Phillips B, et al. Grading strength of recommendations and quality of evidence in clinical guidelines: report from an American College of Chest Physicians Task Force. Chest. 2006;129(1):174–81. PMID: 16424429. 10.1378/chest.129.1.174 [DOI] [PubMed] [Google Scholar]
- 25. Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13. Epub 2017 May 18. PMID: 28529184; PMCID: PMC6542664. 10.1016/j.jclinepi.2017.05.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Brouwers MC, Spithoff K, Kerkvliet K, Alonso‐Coello P, Burgers J, Cluzeau F, et al. Development and validation of a tool to assess the quality of clinical practice guideline recommendations. JAMA Netw Open. 2020;3(5):e205535. 2020.5535. PMID: 32459354; PMCID: PMC7254179. 10.1001/jamanetworkopen [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Alonso‐Coello P, Oxman AD, Moberg J, Brignardello‐Petersen R, Akl EA, Davoli M, et al. GRADE Working Group . GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines. BMJ. 2016;353:i2089. PMID: 27365494. 10.1136/bmj.i2089 [DOI] [PubMed] [Google Scholar]
- 28. Schünemann HJ, Al‐Ansary LA, Forland F, Kersten S, Komulainen J, Kopp IB, et al. Qaseem A; board of trustees of the guidelines international Network. Guidelines international Network: principles for disclosure of interests and management of conflicts in guidelines. Ann Intern Med. 2015;163(7):548–53. PMID: 26436619. 10.7326/M14-1885 [DOI] [PubMed] [Google Scholar]
- 29. Nothacker M, Stokes T, Shaw B, Lindsay P, Sipilä R, Follmann M, et al. Guidelines international Network (G‐I‐N) performance measures working group. Reporting standards for guideline‐based performance measures. Implement Sci. 2016;11:6. PMID: 26772173; PMCID: PMC4714427. 10.1186/s13012-015-0369-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Schünemann H, Brożek J, Guyatt G, Oxman A. Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach; 2013. Available in: https://gdt.gradepro.org/app/handbook/handbook.html. Accessed 23 January 2022. [Google Scholar]
- 31. SIGN 50: a guideline developer's handbook (PDF). 2019. https://www.sign.ac.uk/media/1050/sign50_2019. Accessed 20 October 2021 [Google Scholar]
- 32. GIN‐McMaster guideline development checklist. 2022. https://cebgrade.mcmaster.ca/guidecheck.html. Accessed 23 January 2022.
- 33. The AGREE‐S Consortium . AGREE‐S appraisal instrument. 2022. Accessed 31 January 2022.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.