Abstract
Introduction
Appraisal of Guidelines for Research and Evaluation (AGREE II) is an instrument that informs development, reporting and assessment of clinical practice guidelines. Previous research has demonstrated the need for improvement in methodological and reporting quality of clinical practice guidelines specifically in surgery. We aimed to develop an AGREE II extension document for application in surgical guidelines.
Methods and analysis
We have performed a structured literature review and assessment of guidelines in surgery using the AGREE II instrument. In exploratory analyses, we have identified factors associated with guideline quality. We have performed reliability and factor analyses to inform the development of an extension document. We will summarise this information and present it to a Delphi panel of stakeholders. We will perform iterative Delphi rounds and we will summarise the final results to develop the extension instrument in a dedicated consensus conference.
Ethics and dissemination
Funding bodies will not be involved in the development of the instrument. Research ethics committee and Health Research Authority approval was waived, since this is a professional staff study only and no duty of care lies with the National Health Service to any of the participants. Conflicts of interest, if any, will be addressed by reassigning functions or replacing participants with relevant conflicts. The results will be disseminated through publication in peer reviewed journals, the funders’ websites, social media and direct contact with guideline development organisations and peer-reviewed journals that publish guidelines.
Keywords: surgery, protocols & guidelines, quality in health care, health policy
Strengths and limitations of this study.
This is the first project to address guideline development and reporting in surgery.
It will combine statistical considerations, conceptual parameters to be derived from qualitative synthesis and a formal Delphi process.
It will involve a panel of stakeholders from a variety of scientific, cultural and geographical backgrounds.
The project will not address specific disciplines of surgery.
Introduction
Research evidence is the primary source to inform medical practice forming the cornerstone of evidence-based medicine.1 An average of 5639 new articles were indexed per month under the subject heading ‘Surgery’ in the National Library of Medicine over the past decade.2 Given this fact, keeping abreast of the latest evidence is a strenous task for healthcare practitioners. Clinical practice guidelines evaluate, summarise and contextualise research evidence into actionable recommendations.3 As such, guidelines have a direct impact on delivery of healthcare and surgical services. It is therefore of paramount importance to ensure the highest quality standards in developing and reporting guidelines.
A great amount of scientific endeavour in the past few years has focused on the quality of scholarly work, including clinical practice guidelines.4 Reporting standards have been developed for virtually all study designs and have been summarised by the Enhancing the Quality and Transparency of health Research (EQUATOR) Network.5 Appraisal of Guidelines for Research and Evaluation (AGREEII) constitutes a framework for developing, appraising and reporting clinical practice guidelines.6 It is endorsed by major international and national agencies, including the WHO and the National Institute for Health and Care Excellence (NICE) in the UK.7 8 AGREE II is a generic tool that applies to all disciplines of medicine, and no modification or extension of the framework has been proposed, described or developed for specific clinical branches such as surgery.
The tool is composed of 23 items organised in seven thematic domains: scope and purpose, stakeholder involvement, rigour of development, clarity of presentation, applicability and editorial independence. It concludes with an overall assessment and a statement of whether the guideline is considered of sufficient quality to be used or recommended in clinical practice (online supplementary appendix).
bmjopen-2020-037107supp001.pdf (63.4KB, pdf)
Need for an AGREE II extension
Our research group has acted as methodological and content coordinators of landmark surgical guidelines and have served as members of surgical guideline development groups.9–14 Even though members of our group, in their role as guideline developers, have made every effort to comply with the highest methodological standards, as indicated by adherence to Grading of Recommendations Assessment, Development and Evaluation (GRADE) and AGREE II methodologies,15 16 we noticed that compliance with all aspects of several parameters of the AGREE II instrument was not possible.
For example, the item ‘The potential resource implications of applying the recommendations have been considered’ may be difficult to be universally addressed. Cost-effectiveness studies are scarce in the surgical literature and relevant evidence typically varies in different settings.17 Since surgical expertise varies across countries and institutions, there is a need for the instrument to consistently apply to different healthcare settings. Surgical interventions are complex, and details on the interventions/comparators are imperative for the target users to be able to assess the external validity of the guidelines. Specialists from different specialties and allied health professionals with a wide range of expertise are involved in the treatment of surgical patients, which makes their involvement in guideline development paramount. We have hypothesised that the original AGREE II document may not be applicable to clinical practice guidelines in surgery, which often represent complex and multifaceted interventions.
Objective
There are a few guideline reporting documents in other fields of medicine18 19; however, a scoping literature review by our group has not identified any document to inform guideline development and reporting in the field of surgery. Our aim was to develop an extension of the AGREE II instrument that is specific for surgery through an evidence-informed and consensus-based approach.
Methods and analysis
We have formed an international multidisciplinary and interdisciplinary collaborative research working group that consists of surgeons, guideline developers, evidence synthesis experts, GRADE methodologists,20 biostatisticians and a lead member of the AGREE collaboration. This is a tripartite project named Guideline Assessment Project (GAP): Filling the GAP in Surgical Guidelines. A summary of the project is outlined in figure 1. The project is a result of a partnership between an international team of surgical research experts and two of the AGREE research team leads (IDF and MB). The AGREE research team is currently under a membership renovation process, and, therefore, neither of the authors can speak on behalf of the entire AGREE group. However, both AGREE research team leads state that AGREE has supported the project from its inception. Furthermore, they have agreed to support dissemination activities by making this new tool available in the AGREE website (https://agreetrust.org).
Figure 1.
Development steps of the guideline assessment project with the ultimate objective to develop an AGREE II extension document for surgical guidelines. AGREE II, Appraisal of Guidelines for Research and Evaluation.
This protocol complies with the Guidance for Developers of Health Research Reporting Guidelines.21
Patient and public involvement
No patients were involved in the development of this protocol.
GAP I: literature review and exploratory analyses
We have previously performed a structured review to identify clinical practice guidelines in the field of surgery published over a 10-year period.22 We have assessed the methodological and reporting quality of the selected guidelines using the original AGREE II criteria. Domain scores (calculated by summing up all the scores of the individual items in a domain and by scaling the total as a percentage of the maximum possible score for that domain)16 ranged between 0% and 56%, suggesting generally inadequate and highly variable guideline quality. The median overall score was 4 out of a maximum of 7, and 40% of guidelines were not considered suitable for use based on their quality as assessed using the AGREE II instrument.
In exploratory analyses, we have found guidelines produced by surgical organisations with a high (≥1 guideline per year) output (OR 3.79, 95% CI 1.01 to 12.66), and those produced by surgical organisations with a guideline committee (OR 4.15, 95% CI 1.47 to 11.77) have higher odds of reaching sufficient quality and being recommended for use.22
GAP II: statistically calibrating the AGREE II instrument
The second part of this project was focused on statistical calibration of the AGREE II instrument. We have used quality appraisal data from GAP I and employed a series of statistical methods to explore reliability, internal consistency and unidimensionality of the AGREE II instrument when it is applied in surgical guidelines. We investigated the internal consistency that refers to the extent to which all items of the instrument measure the same hypothetical construct. We explored if and how test items are intercorrelated. Large intercorrelations among test items are indicative of the items measuring the same construct. Using reliability analysis, Kendall’s tau statistics, factor analysis and the item response theory, we explored whether items of each AGREE II domain are intercorrelated and are, therefore, indicators of the same construct. Statistical modelling showed that excluding five items from the original tool (items 1, 2, 5, 7 and 8) and rearranging the remaining items into four domains instead of six would enhance the instrument. We have finally drafted a modified AGREE II document for guidelines in surgery, on the basis of the outcomes of statistical models.
GAP III: AGREE II extension for surgical guidelines
The third part of the project aims to use the information from the previous GAP projects and other published information on the topic to develop the extension document using a structured Delphi process involving relevant stakeholders.
The multidisciplinary Delphi panel will include surgical specialists, journal editors, guideline development bodies, GRADE representatives and patient representatives. Under consideration of the evidence, stakeholders will be asked to provide their input through a Delphi process, which will inform the preparation of an AGREE II extension for surgical guidelines.
Participants
The executive group consists of surgeons (ML-C, SRM, GS, GAA, NKF and SAA), members of surgical quality and research boards (ML-C, NKF and SAA), guideline developers (ML-C, IDF, MB, GS, NKF and SAA), evidence synthesis experts (IDF, GAA, DM and SAA), GRADE methodologists (ML-C and SAA),20 biostatisticians (DM and ST) and two leads of the AGREE Group (IDF and MB). It is further divided into four working groups with distinct functions and responsibilities:
The strategic steering group is responsible for overseeing the project.
The methods group coordinates the methodology of the project.
The evidence review group will review the literature for evidence on candidate new items to be included in the extension document.
The evidence synopsis group will summarise evidence for presentation to Delphi participants.
The group attended a 1-day meeting to discuss the findings of previous work, define the methodology and study design, and identify potential stakeholder groups to comprise the Delphi panel.
Delphi process
The Delphi panel will consist of key stakeholders, including representatives from different surgical disciplines (general surgery, urology, thoracic surgery, vascular surgery and paediatric surgery), guideline developers from different continents and representatives of guideline development organisations. The public will be involved by participation of patient representatives from the European Patients Forum (box 1). We will develop a web-based survey tool to facilitate Delphi exercises. Findings of previous work (GAP I and GAP II) and further evidence that will be identified through a scoping literature review will be summarised and presented to Delphi participants. Summary information will also be available on the project website (https://gap-project.org). Online links to full documents for detailed review of the evidence will be provided.
Box 1. Stakeholders to participate in a web-based Delphi process.
General surgeon
Urologist
Thoracic surgeon
Vascular surgeon
Paediatric surgeon
Journal editor
National authority representative
NICE representative
Guideline developer/representative from Europe
Guideline developer/representative from North America
Guideline developer/representative from Asia
Guideline developer/representative from middle-income country
Healthcare provider representative
Representative from GRADE
Guideline implementer
Patient representative
WHO representative
European Commission representative
GIN representative
EQUATOR representative
EQUATOR, Enhancing the Quality and Transparency of health Research; GIN: Guidelines International Network; GRADE, Grading of Recommendation Assessment, Development and Evaluation; NICE, National Institute of Health and Care Excellence.
The first round will include open-ended questions to identify candidate items for inclusion in the extension document. Responses will be grouped and summarised by the methods group before the second round is commenced.
The second round will include closed-ended questions in a 5-point Likert scale to assess participants’ opinions and level of agreement on including candidate items or excluding existing items from the extension document. As per protocol, 1/2 indicates strong/moderate disagreement; 3 indicates no opinion; and 4/5 indicates moderate/strong agreement. Candidate items will have been identified through GAP I and GAP II and the scoping literature review. We will discard low-scoring items (ie, those with a median score of 1/2 on the Likert scale) and use the shortlisted items in a third Delphi round. We will repeat the process until an agreement of 80% (4/5 on the Likert scale) is reached among Delphi participants.
The Delphi panel’s contribution will be acknowledged by group authorship in subsequent publications of the extension document, the elaboration document and supporting tools.
Qualitative research synthesis
We will perform qualitative evidence synthesis to identify factors of conceptual importance to the quality of evidence in surgery. The overarching question will be
How do clinical practice guidelines in surgery differ from non-surgical guidelines? Specific thematic questions will be addressed:
Which are the concepts that make surgical guidelines different from guidelines or summary evidence in other medical fields?
Which are potential items that may be of sufficient importance to be included in an AGREE II extension for surgical guidelines?
Which are the items of the original AGREE II that might not be relevant to surgical guidelines?
How should items of the original AGREE II instrument be modified to be more relevant to an AGREE II extension for surgical guidelines?
We will conduct a scoping search of PubMed, Embase and Google Scholar. In keeping with realist review guidelines,23 there will be no restrictions on the types of study design eligible for inclusion. We will consider editorials, letters to the editor, commentaries, opinions and any type of publication that captures the breadth discussions about development of surgical guidelines. Information will be used to identify characteristics that specifically apply to surgical guidelines.
The realist review will aim to develop an explanatory understanding of development and reporting of surgical guidelines, how surgical guidelines differ from non-surgical ones and how AGREE II can be modified to reflect the specific aspects of surgical guidelines. According to the realist synthesis methodology, studies will be assessed based on criteria of relevance (whether they contribute to the development or testing of the initial theories)24 and appropriateness for addressing the research questions.25 26
Studies will be entered into ATLAS.ti and coded to identify the specific features relevant to development and reporting of surgical guidelines. Themes will be discussed by the research team using an iterative and speculative process.26 Adjudication and triangulation will be applied to refine theories which can be used across the studies to understand findings.
Furthermore, we will invite users of social media through the project account on Twitter (@GAProject2) and through communication streams of the sponsoring bodies (Facebook, Twitter and email newsletters) to nominate parameters of importance in the development and reporting of guidelines in surgery, and will group and summarise their responses. Evidence identified from the aforementioned pathways, along with information from GAP I and GAP II, will be summarised and taken into account when developing the extension document.
Consensus meeting
Following the Delphi process, the executive group will meet to discuss the findings and compose the first draft of the extension document. We will present new items that will be identified through the Delphi exercise and the qualitative synthesis, and discuss their plausibility and possible inclusion in the instrument. Similarly, items to be excluded with the respective rationale will be discussed. The group will finalise the extension document by ordering and allocating items into domains.
The executive group will hold a further meeting with the advisory group, which is composed of journal editors and representatives of surgical associations to discuss dissemination and implementation processes of the developed extension instrument.
Pilot testing and assessment of internal validity
The extension instrument will be pilot-tested by two members of the executive group. One member will apply the instrument on the surgical guidelines, which were identified by the structured search process as described in GAP I22 and on additional guidelines that will be identified by extending the search to the present date. A second member will independently follow the same process in a randomly selected sample of 15 guidelines. The biostatistical team will assess the internal validity by applying the statistical models of GAP II. Any difficulties encountered with the use of the instrument will be documented and addressed. Results of the statistical assessment will be appraised against statistical findings of the appraisal of the original AGREE II instrument (GAP II).
AGREE II extension statement
The extension statement along with an explanation and elaboration (E&E) document will be composed by the executive group. The E&E document will detail the use of the extension instrument in developing and reporting a new surgical guideline and appraising an existing surgical guideline.
AGREE II extension checklist
A checklist including the AGREE II extension items will be developed with the aim of this checklist to be used by guideline developers (to summarise development and reporting parameters), guideline users (to appraise quality), peer reviewers and journal editors (to assess adequacy of reporting parameters).27
Feedback and criticism
We will invite constructive feedback on the instrument through the dedicated website (https://gap-project.org), and we will consider comments in letters to the editor and via the social media. An ad hoc team will collect and summarise the feedback received in 3 monthly intervals for the first year after publication, and the executive group will discuss and address this information in web-based meetings.
Monitoring, update and future steps
The executive group will monitor the use of the extension document and appraise its applicability in surgical guidelines for a reasonable period of time after dissemination and will publish their findings. Following consideration of the outcomes, feedback, criticism, suggestions and new evidence in the field, we will discuss the need for an update. The development of further extension instruments for national surgical guidelines and guidelines in distinct surgical or other interventional disciplines will be considered following discussions with key stakeholders.
Implications for practice and research
Clinical practice guidelines directly impact clinical practice and healthcare delivery and, as such, development must follow rigorous methodological and reporting standards. The AGREE II instrument has been designed as a generic tool for development and appraisal of clinical practice guidelines.6 It is not intended to substitute established detailed guidance on guideline development principles, processes and procedures, such as the GRADE approach.15 It has addressed a vital need to summarise and detail essential development steps and reporting parameters for high-quality guidelines. Furthermore, as an appraisal instrument, it may be used by healthcare practitioners, policymakers and other stakeholders to inform decisions regarding the use of an existing guideline.
In addition, AGREE II has been shown to be a valuable tool for assessment of guideline quality in several clinical disciplines and evidence fields.22 28–36 Such summaries alert the scientific community to the need for improving specific aspects of clinical practice guidelines (corresponding to the instrument domains) or the overall quality of guidelines. Our previous research has highlighted the need for improvement of the quality of surgical guidelines.22 An AGREE II extension for surgical guidelines is expected to meet this need.
The outcome of this project will be the first AGREE II extension document. Reporting Tool for Practice Guidelines in Healthcare (RIGHT) is another reporting instrument for clinical practice guidelines.37 We are aware of a planned RIGHT extension for public versions of guidelines and a RIGHT extension for adapted practice guidelines.38 39 An extension document of RIGHT for surgical guidelines would be justified as well. However, in view of the evidenced gap in methodological quality of surgical guidelines,22 we considered more appropriate to elaborate on AGREE II, as it addresses guideline development, reporting and appraisal.
Strengths and limitations
This tripartite project is the first to employ statistical models to inform the validity of an extension, modification or update document on guidelines reporting. We will correlate statistical findings with conceptual considerations of the need for adjustments/extension of the AGREE II instrument. The project methods group has adopted recommendations on developing research reporting guidelines, proposed by a collaborative team who have developed a significant number of such guidelines.21 The holistic approach to developing an extension document for clinical practice guidelines in surgery is reflected in the diverse scientific, cultural and geographical backgrounds of experts in the field involved in the project. Similarly, the Delphi panel will include stakeholders and members from a variety of backgrounds, including clinicians/surgeons, methodologists, guideline developers, policymakers and the public (patient representatives).
A face-to-face meeting of Delphi participants, instead of a full web-based Delphi process, might be more efficacious in developing the extension document allowing direct exchange of opinions, information and ideas. We will encourage a full participation and exchange of information by developing a user-friendly and interaction-allowing web-based platform. Furthermore, we will incentivise participation and engagement of potential Delphi members by proposing group authorship and participation in future associated projects.
Ethics and dissemination
Research Ethics Committee and Health Research Authority approval was waived since this is a professional staff study only and no duty of care lies with the National Health Service to any of the participants. We will request electronic informed consent from Delphi participants and the responses of the Delphi members will be anonymised.
We have obtained conflict of interest forms of all executive group members and will request electronic and/or written informed consent by Delphi participants and members of the advisory group. We will deal with potential conflicts of interest by reassigning functions or replacing participants who pose interest conflict.
We will submit the final paper with the extension document to be considered for publication in the UEG Journal and Surgical Endoscopy, as defined in the respective predevelopment agreements. We will negotiate simultaneous publications in other surgical journals for widest dissemination as recommended by the Guidance for Developers of Health Research Reporting Guidelines.21
We will make the extension document available in a dedicated website with links to the original publications. We will encourage surgical organisations with guideline development activities to use the instrument. We will further pursue dissemination through the websites of the funding bodies and channels of social media of major stakeholers, such as the Guideline International Network, GRADE and EQUATOR.
Through direct contact, we will advise international surgical and guideline development organisations and policymakers to endorse the extension instrument. Furthermore, editors of surgical journals will be advised to provide an extension instrument checklist that authors of clinical practice guidelines should submit, along with the original manuscript.
The GAP III study aims to address the need for improvement of the methodology, reporting and appraisal of surgical guidelines. An extension document specifically designed for clinical practice guidelines in surgery will further improve the value, use and applicability of the AGREE II instrument in the surgical field with the ultimate goal of enhancing patient care, experience and outcomes.
Supplementary Material
Acknowledgments
The authors wish to thank Mrs. Meropi Gioumidou for the tireless contribution and the excellent administrative support to the project.
Footnotes
Twitter: @dimi_mavridis, @STsokani, @ManuelLpezCano1, @IvanD_Florez, @melissa_SEPH_UO, @MarkarSheraz, @nader_nkfrancis, @sa_antoniou
Contributors: GAA: conception and design, interpretation of data, drafting the work, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. DM: conception and design, analysis of data, interpretation of data, drafting the work, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. ST: analysis of data, interpretation of data, revising the work critically for important intellectual data, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. ML-C: acquisition of data, analysis of data, interpretation of data, revising the work critically for important intellectual data, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. IDF: conception and design, interpretation of data, drafting the work, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. MB: conception and design, interpretation of data, revising the work critically for important intellectual data, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. SRM: interpretation of data, revising the work critically for important intellectual data, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. GS: interpretation of data, revising the work critically for important intellectual data, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. NF: interpretation of data, revising the work critically for important intellectual data, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.SA: conception and design, acquisition of data, analysis of data, interpretation of data, drafting the work, final approval for the work to be published, agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding: This project received funding from the United European Gastroenterology and the European Association for Endoscopic Surgery. The funding bodies had no influence on the development of this protocol.
Competing interests: None declared.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet 2017;390:415–23. 10.1016/S0140-6736(16)31592-6 [DOI] [PubMed] [Google Scholar]
- 2.Pubmed. Available: https://pubmed.ncbi.nlm.nih.gov/ [Accessed 2 Oct 2017].
- 3.Graham R, Mancher M, Wolman DM. Clinical practice guidelines we can trust, 2011. [PubMed] [Google Scholar]
- 4.Altman DG, Simera I, Hoey J, et al. . EQUATOR: reporting guidelines for health research. Lancet 2008;371:1149–50. 10.1016/S0140-6736(08)60505-X [DOI] [PubMed] [Google Scholar]
- 5.Equator Equator resource centre. Available: https://www.equator-network.org/
- 6.Brouwers MC, Kho ME, Browman GP, et al. . AGREE II: advancing Guideline development, reporting and evaluation in health care. J Clin Epidemiol 2010;63:1308–11. 10.1016/j.jclinepi.2010.07.001 [DOI] [PubMed] [Google Scholar]
- 7.World Health Organization WHO Handbook for Guideline development, 2012. Available: http://apps.who.int/iris/bitstream/10665/75146/1/9789241548441_eng.pdf [Accessed 2 Oct 2017].
- 8.NICE The guidelines manual, Guidance and guidelines. Available: https://www.nice.org.uk/process/pmg6/chapter/reviewing-the-evidence [Accessed 2 Oct 2017].
- 9.Antoniou SA, Agresta F, Garcia Alamino JM, et al. . European Hernia Society guidelines on prevention and treatment of parastomal hernias. Hernia 2018;22:183–98. 10.1007/s10029-017-1697-5 [DOI] [PubMed] [Google Scholar]
- 10.Muysoms FE, Antoniou SA, Bury K, et al. . European Hernia Society guidelines on the closure of abdominal wall incisions. Hernia 2015;19:1–24. 10.1007/s10029-014-1342-5 [DOI] [PubMed] [Google Scholar]
- 11.Gorter RR, Eker HH, Gorter-Stam MAW, et al. . Diagnosis and management of acute appendicitis. EAES consensus development conference 2015. Surg Endosc 2016;30:4668–90. 10.1007/s00464-016-5245-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.López-Cano M, García-Alamino JM, Antoniou SA, et al. . EHS clinical guidelines on the management of the abdominal wall in the context of the open or burst abdomen. Hernia 2018;22:921–39. 10.1007/s10029-018-1818-9 [DOI] [PubMed] [Google Scholar]
- 13.Morales-Conde S, Peeters A, Meyer YM, et al. . European association for endoscopic surgery (EAES) consensus statement on single-incision endoscopic surgery. Surg Endosc 2019;33:996–1019. 10.1007/s00464-019-06693-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Francis NK, Sylla P, Abou-Khalil M, et al. . EAES and SAGES 2018 consensus conference on acute diverticulitis management: evidence-based recommendations for clinical practice. Surg Endosc 2019;33:2726–41. 10.1007/s00464-019-06882-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Schünemann H, Brożek J, Guyatt G. GRADE Handbook for grading quality of evidence and strength of recommendations. The GRADE Working Group, 2013. [Google Scholar]
- 16.AGREE Next Steps Consortium The AGREE II Instrument [Electronic version] 2009.
- 17.Prinja S, Nandi A, Horton S, et al. . Costs, Effectiveness, and Cost-Effectiveness of Selected Surgical Procedures and Platforms In: Essential surgery: disease control priorities. Volume 1 Third Edition, 2015. [Google Scholar]
- 18.Jünger S, Payne SA, Brine J, et al. . Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med 2017;31:684–706. 10.1177/0269216317690685 [DOI] [PubMed] [Google Scholar]
- 19.Choi J, Choi T-Y, Jun JH, et al. . Preferred reporting items for the development of evidence-based clinical practice guidelines in traditional medicine (PRIDE-CPG-TM): explanation and elaboration. Eur J Integr Med 2016;8:905–15. 10.1016/j.eujim.2016.07.027 [DOI] [Google Scholar]
- 20.Norris SL, Meerpohl JJ, Akl EA, et al. . The skills and experience of GRADE methodologists can be assessed with a simple tool. J Clin Epidemiol 2016;79:150–8. 10.1016/j.jclinepi.2016.07.001 [DOI] [PubMed] [Google Scholar]
- 21.Moher D, Schulz KF, Simera I, et al. . Guidance for developers of health research reporting guidelines. PLoS Med 2010;7:e1000217. 10.1371/journal.pmed.1000217 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Antoniou SA, Tsokani S, Mavridis D, et al. . Guideline assessment project: filling the gap in surgical guidelines. Ann Surg 2019;269:642–51. [DOI] [PubMed] [Google Scholar]
- 23.Wong G, Greenhalgh T, Westhorp G, et al. . Development of methodological guidance, publication standards and training materials for realist and meta-narrative reviews: the RAMESES (realist and Meta-narrative evidence syntheses – evolving standards) project. Health Serv Deliv Res 2014;2:1–252. 10.3310/hsdr02300 [DOI] [PubMed] [Google Scholar]
- 24.Pawson R. Evidence-Based policy. London: SAGE Publications Ltd, 2006. [Google Scholar]
- 25.Pawson R, Greenhalgh T, Harvey G, et al. . Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005;10 Suppl 1:21–34. 10.1258/1355819054308530 [DOI] [PubMed] [Google Scholar]
- 26.Wong G, Greenhalgh T, Pawson R. Internet-Based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ 2010;10:12. 10.1186/1472-6920-10-12 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Brouwers MC, Kerkvliet K, Spithoff K, et al. . The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ 2016;352:i1152. 10.1136/bmj.i1152 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Chen Z, Hong Y, Liu N, et al. . Quality of critical care clinical practice guidelines: assessment with agree II instrument. J Clin Anesth 2018;51:40–7. 10.1016/j.jclinane.2018.08.011 [DOI] [PubMed] [Google Scholar]
- 29.Bazzano AN, Green E, Madison A, et al. . Assessment of the quality and content of national and international guidelines on hypertensive disorders of pregnancy using the agree II instrument. BMJ Open 2016;6:e009189. 10.1136/bmjopen-2015-009189 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Armstrong JJ, Rodrigues IB, Wasiuta T, et al. . Quality assessment of osteoporosis clinical practice guidelines for physical activity and safe movement: an AGREE II appraisal. Arch Osteoporos 2016;11:6. 10.1007/s11657-016-0260-9 [DOI] [PubMed] [Google Scholar]
- 31.Sabharwal S, Patel NK, Gauher S, et al. . High methodologic quality but poor applicability: assessment of the AAOS guidelines using the AGREE II instrument. Clin Orthop Relat Res 2014;472:1982-8. 10.1007/s11999-014-3530-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Huang T-W, Lai J-H, Wu M-Y, et al. . Systematic review of clinical practice guidelines in the diagnosis and management of thyroid nodules and cancer. BMC Med 2013;11:191. 10.1186/1741-7015-11-191 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Wong JJ, Côté P, Sutton DA, et al. . Clinical practice guidelines for the noninvasive management of low back pain: a systematic review by the Ontario protocol for traffic injury management (OPTIMa) collaboration. Eur J Pain 2017;21:201–16. 10.1002/ejp.931 [DOI] [PubMed] [Google Scholar]
- 34.Acuna SA, Huang JW, Scott AL, et al. . Cancer screening recommendations for solid organ transplant recipients: a systematic review of clinical practice guidelines. Am J Transplant 2017. [DOI] [PubMed] [Google Scholar]
- 35.van Dijk AH, de Reuver PR, Besselink MG, et al. . Assessment of available evidence in the management of gallbladder and bile duct stones: a systematic review of international guidelines. HPB 2017;19:297–309. 10.1016/j.hpb.2016.12.011 [DOI] [PubMed] [Google Scholar]
- 36.Zeng M, Yi Q, Zeng L, et al. . Quality of therapeutic drug monitoring (TDM) guidelines is suboptimal: an evaluation using the AGREE II instrument. J Clin Epidemiol 2019. [DOI] [PubMed] [Google Scholar]
- 37.Chen Y, Yang K, Marušic A, et al. . A reporting tool for practice guidelines in health care: the right statement. Ann Intern Med 2017;166:128–32. 10.7326/M16-1565 [DOI] [PubMed] [Google Scholar]
- 38.Wang X, Zhou Q, Chen Y, et al. . Protocol of reporting items for public versions of guidelines: the reporting tool for practice guidelines in health Care-public versions of guidelines. BMJ Open 2019;9:e023147. 10.1136/bmjopen-2018-023147 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Song Y, Darzi A, Ballesteros M, et al. . Extending the RIGHT statement for reporting adapted practice guidelines in healthcare: the RIGHT-Ad@pt Checklist protocol. BMJ Open 2019;9:e031767. 10.1136/bmjopen-2019-031767 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2020-037107supp001.pdf (63.4KB, pdf)