Safety and efficacy of perioperative benzodiazepine administration: study protocol for a systematic review and meta-analysis

Abstract

Introduction

Perioperative benzodiazepines are used because of their anxiolytic, sedative and amnestic effects. Evidence has demonstrated an association of benzodiazepines with adverse neuropsychiatric effects. Nonetheless, because of their potential benefits, perioperative benzodiazepines continue to be used routinely. We seek to evaluate the body of evidence of the risks and benefits of benzodiazepine use during the perioperative period.

Methods and analysis

We will search Cochrane CENTRAL, MEDLINE, EMBASE, PsychINFO, CINAHL and Web of Science from inception to March 2019 for randomised controlled trials (RCTs) and observational studies evaluating the administration of benzodiazepine medications as compared with all other medications (or nothing) in patients undergoing cardiac and non-cardiac surgery. We will exclude studies assessing the use of benzodiazepines for procedural sedation or day surgery. We will examine the impact of giving these medications before, during and after surgery. Outcomes of interest include the incidence of delirium, duration of delirium, postprocedure cognitive change, the incidence of intraoperative awareness, patient satisfaction/quality of life/quality of recovery, length-of-stay (LOS) in the intensive care unit (ICU), hospital LOS and in-hospital mortality.

Reviewers will screen references and assess eligibility using predefined criteria independently and in duplicate. Two reviewers will independently collect data using prepiloted forms. We will present results separately for RCTs and observational studies. We will pool data using a random effect model and present results as relative risk with 95% CIs for dichotomous outcomes and mean difference with 95% CI for continuous outcomes. We will pool adjusted ORs for observational studies. We will assess risk of bias for individual studies using the Cochrane Collaboration tool for RCTs. For observational studies, we will use tools designed by the Clinical Advances through Research and Information Translation group. Quality of evidence for each outcome will be assessed using the Grading of Recommendations Assessment, Development and Evaluation approach.

Ethics and dissemination

This systematic review involves no patient contact and no interaction with healthcare providers or systems. As such, we did not seek ethics board approval. We will disseminate the findings of our systematic review through the presentation at peer-reviewed conferences and by seeking publication in a peer-reviewed journal.

PROSPERO registration number

CRD42019128144

Strengths and limitations of this study.

• Rigorous search strategy including multiple databases, grey literature and non-indexed trials.

• Comprehensive risk of bias evaluation using accepted tools.

• Quality of evidence assessment using the Grading of Recommendations Assessment, Development and Evaluation framework.

• Heterogeneity in patient populations, objectives of benzodiazepine administration and doses/drugs administered.

• Heterogeneity in the assessment of cognitive and patient satisfaction measures.

Description of the problem

The evidence summarising the risks and benefits of benzodiazepine administration before, during and after inpatient cardiac and non-cardiac surgery is limited by small sample sizes, short duration of follow-up and risk of bias. Nonetheless, these medications are commonly used for sedation and general anaesthesia because of their anxiolytic and amnestic effects, in addition to their minimal impact on haemodynamics. Systematic reviews have demonstrated their efficacy in reducing postoperative side effects,¹ postoperative nausea and vomiting (PONV)² and intraoperative awareness,³ although all of these findings were based on low or very low quality of evidence.

This evidence of benefit can be contrasted with evidence of harm. Randomised trials and observational studies in the critical care setting suggested that benzodiazepines are associated with increased risk of delirium, duration of delirium, cognitive decline, post-traumatic stress disorder and dementia.⁴ However, much of this evidence is from observational data, and suggests association but not causation. Given these concerns, guidelines issued by the Society for Critical Care Medicine (SCCM)⁵ and American Geriatric Society (AGS)⁶ have recommended that benzodiazepine administration be minimised in the elderly and critically ill populations. However, in the absence of guidelines summarising both the risks and benefits of benzodiazepine administration before, during and after inpatient cardiac and non-cardiac surgery, these medications continue to constitute a common part of perioperative practice. Thus, we will conduct a systematic review of the literature describing the risks and benefits of perioperative benzodiazepine administration.

Description of the intervention

The intervention of interest is the administration of any dose of any benzodiazepine medication before, during or after any surgical procedure.

How the intervention might work

The effects of benzodiazepine medications are attributed to their enhancement of the effects of gamma-aminobutyric acid,⁷ an inhibitory neurotransmitter in the central nervous system, as well as their opioid and sedative sparing effects.⁸ This may have both beneficial and adverse effects in the perioperative period. Potential benefits include improved patient satisfaction⁹ and haemodynamic stability¹⁰ and decreased intraoperative awareness,³ PONV² and headache.¹ Potential adverse effects include delirium,^{4 5} cognitive decline⁴ and prolonged postoperative recovery time.¹

Why is it important to do this review?

Benzodiazepines continue to be routinely used in the perioperative setting, despite recommendations from the SCCM¹ and AGS⁶ that these medications be minimised in the elderly and critically ill populations. The evidence documenting the harms of these medications has limitations, including lack of randomisation or blinding, small sample sizes and non-placebo comparisons. To counter the documented adverse effects of benzodiazepines are beliefs—supported by a low-quality body of evidence—that these medications have important benefits that are not provided by other forms of analgosedation, including amnesia, anxiolysis and minimal impact on haemodynamics. The balance of risks and harms of benzodiazepine administration during the perioperative period have not previously been described. Thus, we will summarise the body of evidence, aiming to inform clinicians and recommendations for future research.

Research question

In adult patients undergoing cardiac or non-cardiac surgery, what is the impact of benzodiazepine administration before, during or after surgery on the incidence of delirium, duration of delirium, intraoperative awareness, postprocedure cognitive function, patient satisfaction/quality of life/quality of recovery, PONV, ICU LOS, hospital LOS and postoperative mortality?

Methods

In describing this protocol, we have followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting of systematic review protocols (PRISMA-P) statement.¹¹

Eligibility criteria

Types of studies

RCTs and limited observational studies will be included in this systematic review. We will include only prospective observational studies that include a control arm and make some form of adjustment for confounders. Case-control studies that report adjusted ORs (aORs) will be included. Within patient crossover studies will be excluded. Cluster randomised and cluster crossover trials will be included. No language constraints will be placed.

Types of participants

The population of interest includes adult patients (≥18 years of age) undergoing cardiac or non-cardiac surgery. We will exclude studies that evaluate the use of benzodiazepines for procedural sedation (eg, endoscopy, bronchoscopy, cardioversion) or day surgery.

Types of interventions

Any benzodiazepine before, during or after inpatient cardiac or non-cardiac surgery. Any dose or route (ie, intravenous, oral, intranasal or intrathecal) will be eligible.

Types of outcome

The outcomes included in this scale were determined to be primary or secondary according to the responses to a survey that was circulated to all authors of this systematic review protocol and other stakeholders (patients and family members, intensivists, cardiac surgeons, anaesthetists). Respondents were asked to rank the importance of each outcome using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) scale¹² (see figure 1). Outcomes with an average rating ≥7 were included as primary and those with an average rating <7 and ≥4 were included as secondary.

Figure 1.

Grading of Recommendations Assessment, Development and Evaluation scale for ranking of outcomes (reproduced with permission).

Primary outcomes

Primary outcomes will include the incidence of delirium (binary; as measured using any validated scale), postprocedure cognitive change (as measured by the study using any validated measure of cognitive function: for example, Mini Mental Status Exam, Montreal Cognitive Assessment), the incidence of intraoperative awareness (binary; as measured by the study) and in-hospital mortality (binary).

Secondary outcomes

Secondary outcomes will include the duration of delirium (continuous; in days), patient satisfaction/quality of life/quality of recovery (as measured by the study prior to hospital discharge after the index surgical procedure), PONV (binary; as measured by the study), ICU LOS (continuous; in hours), hospital LOS (continuous; in days).

Search strategy

Databases

We will search CENTRAL, MEDLINE, CINAHL, PsychINFO and Web of Science from inception to March 5/2019. See online supplementary appendices 1 and 2 for complete MEDLINE and EMBASE search strategies.

Other sources (grey literature)

Clinicaltrials.gov, ISRCTN Register, ANZCTR and WHO ICTRP will be searched for unpublished studies. References of included studies and prior systematic reviews will be reviewed for potentially relevant studies. Conference proceedings for American Society of Anesthesiology, International Anesthesia Research Society, Canadian Anesthesia Society, European Society of Anaesthesiology, and Australian and New Zealand College of Anaesthetists annual meetings from the last 2 years will also be reviewed. We will contact experts within the field to see if they are aware of other relevant studies.

Data management

Study records and data will be managed using Covidence software.¹³

Study selection process

We will select studies to be included in the systematic review after the initial search results are obtained in a stepwise process and using specific study eligibility criteria. In the first step, two independent reviewers will review the title and abstract of each reference obtained in the initial search to assess the relevance to our systematic review. Any reference deemed relevant by either reviewer will have its full text retrieved for inclusion in the next step. The second step will involve full article review and determining whether it fulfils eligibility criteria. This will be done independently by two reviewers. Disagreements will be resolved through discussion between reviewers and consensus decision with respect to eligibility. If consensus cannot be reached, a third party will be involved to establish a final decision.

Studies excluded after full-text review will have the most pertinent justification for exclusion recorded. In the event that information regarding one eligibility criterion is not provided but all other criteria are met, study investigators will be contacted for further information. The article will be listed as ‘unclear eligibility’ until the required information becomes available.

Data collection

After identifying included studies, data extraction will be carried out independently and in duplicate using prepiloted forms. Data collected will include: study, population and procedure characteristics, specific benzodiazepine given, dose and route of administration, timing of administration, as well as previously outlined outcomes. Data disagreement will be resolved through discussion or deferral to a third party for final decision. In the event that outcome data are not available in the study report, additional information will be requested from the corresponding author. If no response is received after two contact attempts over a 2-week period, the data will be deemed unavailable.

Assessment of risk of bias

Randomised controlled trials

Two reviewers will independently use the Cochrane Collaboration tool,¹⁴ to assess the risk of bias for each included trial. Reviewers will categorise the risk of bias as ‘low’, ‘high’ or ‘unclear’ in the six domains specified by Cochrane: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, selective reporting and other sources of bias. Disagreements between reviewers will be resolved by consensus. In situations where consensus cannot be reached, a third party will be involved so that a decision can be reached.

We will consider the overall risk of bias for each paper as ‘low’ if all domains are ranked ‘low’. We will consider the risk of bias ‘unclear’ if at least one domain is ranked ‘unclear’ without any domains ranked as ‘high’. The risk of bias will be considered ‘high’ if one or more domain is ranked as ‘high’ risk of bias. The acceptable standards for each domain are presented in table 1.

Table 1.

Cochrane risk of bias tool for randomised controlled studies: acceptable standards for each domain

Domain	Criteria for determination of low risk of bias
Random sequence generation (selection bias)	We will consider sequence generation adequate if created via computer randomisation or a random values table. Coin tosses or dice rolls will also be considered adequate.
Allocation concealment	We will consider allocation concealment adequate if consistent measures have been taken to ensure that participant allocation cannot be revealed to research personnel until the participant is assigned to a trial arm.
Blinding of participants and personnel	We will consider blinding adequate if it is explicitly stated that specific measures were taken to ensure patients were unaware of their trial arm assignment and personnel who it is practical to blind have been. Risk of bias due to blinding will be assessed for each outcome in each study.
Blinding of outcome assessment	We will consider blinding of outcome assessment adequate if measures have been taken to ensure that all assessors and outcome adjudicators who it is practical to blind have been.
Incomplete outcome assessment	We will assess for attrition bias for each reported outcome whenever data regarding the number of patients assessed at different stages of the trial are available. We will consider this domain at high risk of bias if loss to follow-up is >10% at a subsequent stage.
Selective reporting	We will assess included studies associated with a published protocol or entry in a clinical trials database for selective reporting by comparing reported with prespecified outcomes. We will consider reporting adequate if all prespecified outcomes are included and reported in full.
Other sources of bias	Each independent reviewer will assess for other potential sources of bias. Examples of other sources of bias include funding provided by companies that may profit from sales of the intervention and authors’ conflict of interest.

Observational studies

Two reviewers will independently use the tools developed by the Clinical Advances Through Research and Information Translation group to assess the risk of bias in cohort studies¹⁵ and case-control studies.¹⁶ We will consider propensity score analyses as cohort studies. Reviewers will evaluate characteristics of each study for risk of bias by answering the questions in the tool as follows: ‘Definitely yes’ (low risk of bias), ‘Probably yes’, ‘Probably no’ or ‘Definitely no’ (high risk of bias). Disagreements between the two reviewers will be resolved by consensus. In situations where consensus cannot be reached, a third party will be involved so that a decision can be reached.

We will evaluate the following domains for cohort studies: selection of exposed and non-exposed cohorts drawn from the same population, assessment of exposure, confidence that the outcome of interest was not present at the start of the study, matching or adjustment for all plausible prognostic variables, confidence in the assessment of the presence or absence of prognostic factors, confidence in the assessment of outcome, adequacy of follow-up and balance of co-intervention between groups. We will evaluate the following domains for case-control studies: assessment of exposure, confidence that cases developed the outcome of interest and controls did not, case selection, control selection and case and control matching or adjustment. The acceptable standards for each domain are presented in table 2 for cohorts (including propensity score analyses) and table 3 for case-control studies.

Table 2.

CLARITY tool for risk of bias in cohort studies: acceptable standards for each domain

Domain	Criteria for determination of low risk of bias
Selection of exposed and non-exposed cohorts drawn from the same population	We will consider the exposed and non-exposed cohorts to be selected from the same population and at low risk of bias if both are drawn from the same database of patients presenting at the same points of care over the same time frame.
Assessment of exposure	We will consider secure records (eg, hospital or pharmacy records) or repeated patient interviews (or other forms of ascertainment) asking about the exposure to have a low risk of bias. Examples of ascertainment of exposure with a higher risk of bias will include structured interviews at a single point in time, written self-report and retrospective self-report (which may be subject to recall bias).
Confidence that the outcome of interest was not present at the start of the study	We will consider that there is a low risk of bias resulting from the outcome of interest being present at the start of the study if there is a complete description of all prognostically important baseline (pre-exposure) characteristics for patients included in the cohort.
Matching or adjustment for all plausible prognostic variables	We will consider studies to be at a low risk of bias if the study has used a comprehensive matching method or has statistically adjusted for al plausible prognostic variables. Studies that match or adjust for a minority of plausible prognostic variables or use not matching or adjustment at all will be considered to be at a high risk of bias. We will not consider statements of no differences between groups that are not substantiated by quantifiable data to be sufficient for establishing comparability.
Confidence in the assessment of the presence or absence of prognostic factors	We will consider studies to be at a low risk of bias if they include an interview or self-completed survey of all participants, or if there is reproducibility or documentation of the accuracy of abstraction of prognostic data. We will consider studies that extract prognostic information from a database with no documentation of quality of abstraction of prognostic variables to be at high risk of bias.
Confidence in the assessment of outcome	We will consider studies that use independent blinded assessment or record linkage to a secure record (eg, hospital or pharmacy record) to be a low risk of bias. Studies that use unblinded independent assessment or self-report will be considered to be at higher risk of bias and studies with no description of how outcome was assessed will be considered to be at high risk of bias.
Adequacy of follow-up	We will consider studies to be at a low risk of bias if there is no missing outcome data, if the reasons for missing outcome data are unlikely to be related to the true outcome, or if the missing outcome data is balanced in numbers across intervention groups, with similar reasons for missingness across groups. We will also consider studies to be at low risk of bias if missing data have been imputed using appropriate methods.
Balance of co-intervention between groups	We will consider studies to be at a low risk of bias of most or all relevant co-interventions that might influence the outcome of interest are documented to be similar in the exposed and unexposed groups.

CLARITY, Clinical Advances Through Research and Information Translation.

Table 3.

CLARITY tool for risk of bias in case-control studies: acceptable standards for each domain

Domain	Criteria for determination of low risk of bias
Assessment of exposure	We will consider studies to be at a low risk of bias if evidence of exposure comes from previously created secure records and data abstractors are unaware of the study hypothesis. We will consider studies to be at high risk of bias if evidence of exposure is acquired by patient interview, or if data collectors are not blinded to patient status or the study hypothesis.
Confidence that cases developed the outcome of interest and controls did not	Studies where cases and controls undergo valid and reliable diagnostic procedures or where surveillance for the outcome of interest is clearly unrelated to the exposure of interest will be considered to have a low risk of bias. We will consider studies where the outcome of interest is acquired by subjective methods (eg, patient interview) but reasonable steps are taken to independently validate results to have a higher risk of bias. Studies where there is no description or cases are established with diagnostic procedures associated with high rates of false positive results or controls are established with diagnostic procedures associated with high rates of false negative results to have a high risk of bias.
Case selection	We will consider studies where all eligible cases are enrolled in a defined catchment area over a defined period of time during which diagnostic procedures are unlikely to have changed to have a low risk of bias. We will consider studies to have a higher risk of bias if they include all eligible cases in a defined catchment area over a defined period of time during which diagnostic procedures are likely to have changed, and a high risk of bias if this domain is not reported.
Control selection	We will consider studies to have a low risk of bias if controls were clearly selected from the same underlying population as the cases and equally at risk of the exposure. We will consider studies to be at high risk of bias if there is a difference in the sampling frame of cases and controls that was clearly related to the exposure of interest.
Case and control matching or adjustment	We will consider studies to be at low risk of bias if it uses comprehensive matching or adjustment for all plausible prognostic variables. Studies that match or adjust for a minority of plausible prognostic variables or use no matching or adjustment will be considered to be at high risk of bias. Statements of no difference with no supporting data will not be accepted as a means of establishing comparability.

CLARITY, Clinical Advances Through Research and Information Translation.

Data analyses and assessment of heterogeneity

Data will be analysed and pooled separately for RCTs and observational studies. Cluster randomised trials will be included as RCTs if they report the following data which will allow us to adjust for the design effect¹⁷: the number of clusters randomised to each intervention arm or the average (mean) size of each cluster, the outcome data for the total number of individuals in each intervention arm (ignoring clustering), and an estimate of the intracluster correlation coefficient (ICC). If these data are not available, we will include the cluster randomised trial as an observational study. If these data are available, we will calculate the effective sample size of the cluster randomised trial using the method described by Rao and Scott¹⁸ which divides the original sample size by a quantity called the ‘design effect’ which is:

1 + (M – 1) ICC where M is the average cluster size.

A random-effects model will be used to pool the relevant studies to summarise the evidence. The results will be presented as relative risk (RR) with 95% CIs for dichotomous outcomes and as mean difference with 95% CI for continuous outcomes measured using interval or ratio scales. We will meta-analyse outcomes measured using ordinal scales using a proportional odds model is the scales have a small number of categories, the numbers within each category for each intervention arm are available, and the same ordinal scale has been used in all studies.¹⁷ If these criteria are not met then we will not pool them and, instead, describe the results qualitatively. We will pool aORs for observational studies (including propensity score analyses) and present them with 95% CI.

Heterogeneity will be assessed using the χ² test for homogeneity and the I² statistic. We will conduct subgroup analyses (see table 4) to assess clinical and methodological sources of heterogeneity in intervention effect regardless of I² value, as I² is not powered enough to detect heterogeneity in analyses with a small number of studies.¹⁷ We will look for publication bias for each outcome using funnel plots for outcomes with a minimum of ten pooled studies. These analyses will be performed using Revman V.5.3.¹⁹ If visual inspection of the funnel plot suggests potential publication bias, we will perform the Egger test for continuous outcomes or the arcsine test for dichotomous outcomes using Stata V.12.²⁰

Table 4.

A priori hypotheses to explain clinical heterogeneity

Subgroup	Hypothesis delirium	Hypothesis postoperative cognitive impairment
Comparisons to dexmedetomidine	Comparison with dexmedetomidine will show more delirium with benzodiazepines.	Comparison with dexmedetomidine will show more postoperative cognitive impairment with benzodiazepines.
Comparisons to propofol	Comparison with propofol will show more delirium with benzodiazepines.	Comparison with propofol will show more postoperative cognitive impairment with benzodiazepines.
Comparisons to opioids	Comparison with opioids will show more delirium with opioids.	Comparison with opioids will show more postoperative cognitive impairment with benzodiazepines.
Higher dose (5 mg midazolam equivalent) or infusion vs lower dose (<5 mg midazolam equivalent) or bolus dosing	Higher dose or infusion administration will be associated with an increased risk of delirium.	Higher dose or infusion administration will be associated with an increased risk of postoperative cognitive impairment.
High risk of bias vs low risk of bias	High risk of bias studies will be associated with larger risk of delirium.	High risk of bias studies will show more cognitive impairment.
Elderly (>75 years) vs younger patients	Delirium will be more common when benzodiazepines are administered to elderly patients.	Postoperative cognitive impairment will be more common when benzodiazepines are administered to elderly patients.
Preoperative and intraoperative benzodiazepine administration vs postoperative benzodiazepine administration	Delirium will be more common when benzodiazepines are administered after surgery and opposed to when benzodiazepines are administered preoperatively or intraoperatively.	Postoperative cognitive impairment will be more common when benzodiazepines are administered after surgery and opposed to when benzodiazepines are administered preoperatively or intraoperatively.

Sensitivity analyses

We will perform an analysis assuming a plausible worst-case scenario (five times the event rate in patients who were not lost to follow-up) in patients lost to follow-up.²¹ Among observational studies included in this review, we will conduct a sensitivity analysis excluding any non-prospective (ie, case-control) studies to evaluate whether this results in a difference in pooled effect sizes for our primary outcomes.

Assessment of pooled effect estimates

We will evaluate the robustness of pooled effect estimates separately for RCTs and observational studies using the GRADE approach.²² According to GRADE, data from randomised controlled trials are considered high quality evidence but can be rated down according to risk of bias, imprecision, inconsistency, indirectness or publication bias. Data from observational studies are considered low-quality of evidence and can be further rated down.

Ethics and dissemination plan

This will be a systematic review of the published literature that will not involve any direct or indirect patient contact. As such, research ethics board approval will not be sought and there are no safety considerations. The findings of this systematic review of the literature will be published in a peer-reviewed journal and will be presented at relevant academic and clinical meetings.

Discussion

Benzodiazepines are commonly used during the perioperative period. The RRs and benefits of benzodiazepines have been evaluated in the setting of endoscopy,²³ among the elderly,⁴ in the ICU,^{5 24} and for outpatient day surgery.¹ A previously published systematic review has evaluated the risks and benefits of perioperative benzodiazepines.²⁵ However, this review did not follow the PRISMA guidelines, nor did it meta-analyse the results obtained by systematic review of the literature. We believe that the lack of evidence regarding the use of benzodiazepines in perioperative medicine represents an important gap in knowledge. Thus, we will undertake this review using rigorous methods and will meta-analyse the outcomes as appropriate.

By undertaking this systematic review and meta-analysis of perioperative benzodiazepine use, we will summarise the state of knowledge and identify important gaps that need to be addressed through the conduct of randomised controlled trials. We hypothesise that evidence will be scarce with respect to the postoperative neurocognitive effects of benzodiazepines given by anaesthetists intraoperatively. We believe that this question needs to be addressed, particularly in the cardiac surgery population, where benzodiazepines are frequently given in high doses,²⁶ and delirium is common, affecting 15%–30% of patients.^{27 28}

Our review has several strengths. We will obtain a broad appreciation of the consequences of perioperative administration of benzodiazepines by including studies that examine the use of benzodiazepines before, during and after surgery, as well as by searching the grey literature and including both RCTs and observational studies. This will generate an unprecedented overview of the broad impact of these medications on adult patients undergoing surgery. We will undertake a comprehensive risk of bias assessment using accepted tools and assess the overall quality of evidence using the GRADE framework. In doing so, we will provide frontline perioperative physicians with both a pooled effect of benzodiazepines on patient-important and system-important outcomes, as well as an appraisal of the strength of this evidence that can inform whether benzodiazepines should be used in the patients they treat.