Quality assessment of clinical practice guidelines for perioperative care and use of GRADE: a systematic review protocol

Abstract

Introduction

Perioperative care is a broad field covering an array of elective and emergency procedures. Clinical practice guidelines (CPGs) for perioperative care exist with various degrees of methodological quality. We intend to critically appraise them using AGREE II instrument and investigate the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE).

Methods and analysis

We searched MEDLINE (Ovid), Epistemonikos, Cochrane Database of Systematic Reviews and PROSPERO and did not identify any similar systematic review in this area. We will search databases, repositories and websites of guideline developers and medical societies, including MEDLINE (Ovid), Embase (Ovid), DynaMed, the GIN international guideline library and registry of guidelines in development, BIGG international database of GRADE guidelines, ECRI Guideline Trust or National Institute for Clinical Evidence to identify all CPGs for perioperative care in an adult population in a general clinical setting. We will include CPGs, expert guidance, position papers, guidance documents and consensus statements published in the last 5 years by experts or international organisations that provide guidance or recommendations in the available full text with no geographical or language limitation. Excluded will be those containing only good practice statements. Two independent reviewers will perform critical appraisal using the AGREE II tool. The data presented in a narrative and tabular form will include the results of the critical appraisal for all identified CPGs for all AGREE II domains and an assessment of the use of the GRADE approach.

Ethics and dissemination

Ethics approval is not required. We will disseminate the findings through professional networks and conference presentations and will publish the results.

Strengths and limitations of this study.

• We intend to comprehensively search for the best quality clinical practice guidelines (CPGs) for perioperative care in bibliographical databases MEDLINE (Ovid) and Embase (Ovid) and specific guideline databases and repositories.

• We will identify the CPGs that cover a range of issues, such as assessing the risks of surgeries, prevention of adverse events, pain management, transfusion management, antibiotic prophylaxis and more. The review will not focus on any specific type of surgery or patient population.

• We will search for and include CPGs in all languages and geographical contexts in the last 5 years to provide an assessment of the rigour of development and methodological quality of current CPGs in perioperative care. We will also review the use of the Grading of Recommendations, Assessment, Development and Evaluations approach.

• The main aim and strength of the review will be in the comprehensive search strategy and the assessment of the methodological quality of the CPGs by professional methodologists using the AGREE II tool. We do not intend to extract the recommendations from the identified relevant CPGs. We will, however, summarise the main strengths and weaknesses of the process of CPG development in perioperative care and suggest ways of improvement.

Introduction

The worldwide increase in the prevalence of chronic diseases places an enormous clinical and financial burden on healthcare providers.¹ For these reasons, the current acutely oriented healthcare systems have been transformed into more flexible systems capable of providing effective and high-quality chronic care, and preventive measures have also been proposed for patients and healthy populations.^{2 3} Surgery is an integral part of global healthcare, with an estimated 234 million major surgical procedures performed annually.⁴ The essential surgical and anaesthetic services are increasingly acknowledged as possible vital factors in reducing death and disability for low-income and middle-income countries while remaining cost-effective.⁵

Perioperative care is a broad field covering an array of elective and emergency procedures.⁶ It includes interventional, mini-invasive, diagnostic or therapeutic care in close cooperation with nursing care. Patients must be carefully evaluated for risks, and each intervention must be weighed for benefits, patient values, complications and cost-effectiveness. Clinical practice guidelines (CPG) may streamline perioperative management of patients and improve outcomes, may even form policy and legislation. A systematic review of CPGs for perioperative care may help assess their quality and thus help choose the best available guideline for adoption or adaptation across the world.

The AGREE II instrument,⁷ published by the AGREE Collaboration in 2010, is used to appraise the quality of CPGs. It also helps guideline working groups by providing a methodological and reporting strategy for CPG development,⁸ together with the GIN-McMaster Guideline Development Checklist.⁹ It ascertains the measure of ‘confidence that the potential biases of guideline development have been addressed adequately and that the recommendations are both internally and externally valid, and are feasible for practice’.⁸ Currently, the most rigorous and transparent methodology for developing CPGs is the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach.¹⁰ However, other systems, primarily based on the assessment of the research design of primary studies, exist and are in use, as are various modifications of the GRADE approach.

In 2003, the AGREE Collaboration, a group of international guideline developers and researchers, undertook a project of developing the first AGREE Instrument to develop a tool to assess the quality of guidelines.¹¹ It was a 23-item tool organised into six quality domains. Following this first endeavour, a sub-section of the AGREE Collaboration, the AGREE Next Steps Research Consortium, was established to improve the AGREE’s reliability and validity further.^{12 13} The Consortium published the currently widely used AGREE II tool in 2010, replacing the original instrument, and developed a user’s manual to facilitate the ability of users to apply the instrument with confidence. Since then, the tool has received recognition worldwide, and various extensions have been developed and tested.¹⁴ Guideline developers are also using the tool as a reporting checklist.¹⁵

Guidelines for perioperative care exist with various degrees of methodological quality. We intend to critically appraise them using the AGREE II tool and investigate the use of the GRADE approach. We searched MEDLINE (Ovid), Epistemonikos, Cochrane Database of Systematic Reviews and PROSPERO and did not identify any similar systematic review in this area. However, such reviews exist for other topics, specific populations or more narrowly defined perioperative care aspects.

This work is a part of the Czech National project of CPG development,¹⁶ the first such national endeavour in the country. We chose perioperative care based on the growing need for evidence-based recommendations in the Czech Republic in consultation with the key stakeholders (the Ministry of Health, heads of appropriate Czech professional organisations, health insurance deputies). Based on this work, an adaptation of a high-quality CPG will occur under the Czech Ministry of Health.

Research question/objective

We framed the review question for this systematic review using the ‘Population and Clinical Areas, Interventions, Comparators, Attributes of CPGs and Recommendation characteristics’ elements¹⁷:

What is the methodological quality of CPGs containing recommendations for perioperative care in the adult population for the general surgical setting?

Methods and analysis

Eligibility criteria

Population and clinical indications

We will include CPGs for general perioperative care in the adult population (specific age defined by the CPGs, eg, typically 16 years or 18 years). We will exclude CPGs targeting particular populations or conditions (eg, only gynaecological patients, to prevent dementia, head and neck cancer, opioid-naïve patients).

Intervention and comparators

We will aim to identify CPGs in the general area of perioperative care for non-specialised surgeries (not specific for any given type of surgery). The CPG should cover a broad spectrum of questions for preoperative, intraoperative and postoperative care, such as assessing the risks of surgeries, prevention of adverse events, pain management, transfusion management, antibiotic prophylaxis, maintenance of normothermia, fluid and intake management, advice and mental preparation of patients, any supplements needed, anticoagulant therapy, control during surgery and more, and apply to any surgical setting. We will include a CPG if its scope covers at least three of these or similar areas. We will exclude CPGs targeting only one type of clinical specialist, disease, surgery or setting (eg, the role of neuroimaging, for total hip replacement only).

Attributes of eligible CPGs

We will include any self-identified CPGs, expert guidance, position papers, guidance documents and consensus statements published in the last 5 years by experts in the field or international organisations on the given topic and population, that provide guidance or recommendations in the full text with no geographical, or language limitation. We will include only the latest version of the CPG. Excluded will be CPGs with unavailable full text (we will record and report exclusions).

Recommendation characteristics

The CPG should contain recommendations and methods of their development, that is, we will exclude documents that self-proclaim as guidelines in which it is not clear which parts of the text are recommendations or how any of the provided statements were developed. Excluded will be papers containing only good practice statements.¹⁸ Good practice statements, as opposed to recommendations, do not depict certainty of evidence or strength of recommendation and do not follow the rigorous methodology typical for recommendations. However, sometimes the GDG finds a statement needs to be issued for various reasons, but to search for evidence would be a waste of the group’s time. Guideline authors are discouraged from issuing good practice statements.¹⁹

Patient and public involvement

No patients will be involved in this review.

Types of resources and search strategy

The databases and guideline repositories to be searched include MEDLINE (Ovid), Embase (Ovid), DynaMed, the GIN international guideline library and registry of guidelines in development, BIGG international database of GRADE guidelines or ECRI Guideline Trust. Sources of unpublished documents and grey literature to be searched include websites of guidelines developers as National Institute for Clinical Evidence, Scottish Intercollegiate Guidelines Network, Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften, KCE Belgian Health Care Knowledge Centre, Haute Autorité de Santé, Institute for Clinical Systems Improvement; or websites of other governmental or non-governmental organisations and medical societies/associations.

The search strategy will comprise of the following keywords and related terms: (pre-operative* OR preoperative* OR pre-surg* OR presurg* OR perioperative* OR peri-operative* OR intraoperative* OR intra-operative* OR intrasurg* OR intra-surg* OR peroperative* OR per-operative* OR postoperative* or post-operative* or post-surg* or postsurg*) AND (care* OR caring OR treat* OR nurs* OR monitor* OR recover* OR medicine). Corresponding MeSH or Emtree terms will also be used when applicable.

We will import all identified CPGs into EndNote X9.2, any duplicates removed, titles/abstracts and then full texts screened by two independent reviewers against the eligibility criteria. We will present search results in a Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart.²⁰

Data extraction

At least two independent reviewers will assess all CPGs, and extract data independently. If needed, we will contact guideline developers to clarify any uncertainty. When appraising and extracting data from CPGs written in languages other than English, we will ask experienced reviewers who are fluent in the given language from acquaintances and affiliated centres around the world or use paid translation services. The data to be extracted will include:

• Guideline field and scope

• Year of publication

• Publishing region

• The version of the guideline

• Guideline language

• Developing organisation

• System of rating evidence used in the guideline

Quality assessment

We will assess all eligible CPGs using the complete AGREE II tool to determine the quality of the guideline development process. Two reviewers will independently assess the CPGs using the AGREE II instrument and discuss any conflicts with a third senior reviewer. Then, they will have the opportunity to alter their scoring, if necessary. All review authors are trained in using AGREE II instrument. The junior methodologists will be supervised as needed.

AGREE II is a 23-item instrument divided into six domains: scope and purpose, stakeholder involvement, the rigour of development, clarity of presentation, applicability and editorial independence. We will use the instrument to appraise each identified CPG and score each item using a 7-point Likert scale, ranging from strongly disagree (1) to strongly agree (7). In addition, an overall assessment score will be given to each CPG.

Data analysis and presentation

The extracted data will be analysed and summarised in tabular and narrative form. We will provide a table with the characteristics of the identified relevant CPGs and the system used for grading the evidence and recommendations. We will narratively summarise the features of the grading systems used, the rationale behind them and how they may have influenced the overall quality of the respective CPGs.

We will report and analyse the rigour of development as depicted by the scores of the AGREE II tool, specifically, the sums of scores of each item for each of the six domains from each reviewer, and the percentage of the total scaled from the number. We will use the same approach to assess the overall score.

We will determine the guideline quality based on the scaled domain scores. We will calculate the inter-rater reliability (via the intraclass correlation coefficient) for each domain to determine the reviewer agreement.

The planned start date of the review is January 2022, and we aim to finish by the end of 2022. We will report the results of this systematic review using the PRISMA statement.²⁰ We will report and explain any changes in the methods described in this protocol. We used the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols checklist when writing our report.²¹ We fully intend to publish all results of the proposed systematic review in a scholarly open-access journal.

Ethics statements

Patient consent for publication

Not applicable.