Obstetrical safety indicators for preventing hospital harms in low risk births: a scoping review protocol

Abstract

Introduction

Optimising the safety of obstetric patient care is a primary concern for many hospitals. Performance indicators measuring aspects of patient care processes can lead to improvements in health systems and the prevention of harm to the patient. We present our protocol for a scoping review to identify indicators for obstetric safety in low risk births. We aim to identify indicators addressing preventable hospital harms, to summarise the data and synthesise results.

Methods and analysis

We will use methods described by Arksey and O’Malley and further expanded by Levac et al. We will search electronic databases such as Medline, Embase, CINAHL and the Cochrane Library, and websites from professional bodies and other organisations, using an iterative search strategy.

Two reviewers will independently screen titles and abstracts of search results to determine eligibility for inclusion. If eligibility is not clear, the reviewers will screen the full text version. If reviewers’ decisions regarding eligibility differ, a third reviewer will review the record. Two reviewers will independently extract data from records that meet our inclusion criteria using a standardised data collection form. We will narratively describe quantitative data, such as the frequency with which indicators are identified, and conduct a thematic analysis of the qualitative data. We will compile a comprehensive list of patient safety indicators and organise them according to concepts that best suit the data such as the Donabedian model or the Hospital Harm Framework. We will discuss the implications for future research, clinical practice and policy-making. We will report the conduct of the review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews checklist.

Ethics and dissemination

The sources of information included in this scoping review will be available to the public. Therefore, ethics approval is not warranted. We will disseminate results in a peer-reviewed publication, conference/event presentation(s) and stakeholder communications.

Strengths and limitations of this study.

• We will follow established scoping review methods introduced by Arksey and O’Malley and further developed by Levac et al.

• We will report the conduct of the review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews.

• Our search strategy will include electronic databases and grey literature sources.

• We will limit the focus of this review to indicators from English language resources aimed at health system processes in countries classified as developed, or as having high-income or upper-middle-income economies.

• Although we will identify candidate indicators for further development, we will not appraise the included records.

Introduction

An aim of healthcare delivery is to avoid preventable harm to patients. However, human limitations and the complex nature of healthcare systems are contributing factors that make errors an expected phenomenon.¹ A systematic review by Nabhan et al found that the most frequently used definition of preventable harms is ‘presence of an identifiable, modifiable cause of harm’.² Preventable harms that occur during hospital stays can also be known as hospital harms, patient safety incidents, hospital adverse events or hospital errors. The Canadian Institute for Health Information (CIHI) defines hospital harms as the ‘rate of acute care hospitalizations with at least one occurrence of unintended harm during a hospital stay that could have been potentially prevented by implementing known evidence-informed practices’.³

In 2000, the Institute of Medicine in the USA published a ground-breaking report To Err is Human: Building a Safer Health System⁴ which raised awareness around errors in healthcare processes and their negative impact on patient morbidity and mortality. In 2004, the results of the Canadian Adverse Events Study showed that according to the sample of 3745 non-psychiatric, non-obstetric adult patient charts reviewed, the rate of preventable adverse events was 2.8% (95% CI, weighted, 2.0–3.6) and the mortality rate from preventable adverse events was 0.66% (95% CI, weighted, 0.37%–0.95%).⁵ Since then, the Canadian Patient Safety Institute (CPSI) reported that patient harm is Canada’s third highest cause of death. Preventable harms occur in 1 out of every 18 hospital visits and resulting treatments costs $C2.75 billion annually.⁶ Regarding the field of obstetrics, the most recently published hospital harm results from the CIHI and the CPSI,⁷ during the fiscal year 2018–2019, reported 5480 incidences of obstetric trauma caused by healthcare or medication-associated conditions and 4047 incidences caused by procedure-associated conditions. They also reported that during the same period, 814 incidences of obstetric haemorrhage were caused by healthcare or medication associated conditions and 997 incidences were caused by procedure-associated conditions. In addition, 920 birth trauma incidences were caused by healthcare or medication-associated conditions and 1317 incidences were caused by procedure-associated conditions.

Certain harms require exploration of each patient case to assess whether the harm is potentially preventable, and others are more obviously preventable, such as fatal medication doses⁸ or wrong-site surgeries. The Hospital Harm Framework classifies harms into four categories: healthcare/medication-associated conditions; healthcare associated infections; patient accidents and procedure-associated conditions.⁹

Several studies have shown strategies that have successfully reduced preventable patient harm. Existing research has shown that making patient safety central to the delivery of care can lead to system-level improvements. In the USA, Pettker et al implemented a comprehensive obstetric patient safety programme which led to reductions in adverse outcomes and litigation cases, and improvements in staff perceptions of safety and improvements in the safety climate.^10–12 Also in the USA, the California Maternal Quality Care Collaborative California has identified a correlation between the work of perinatal quality collaboratives and a 50% reduction in maternal mortality rates in the state of California.¹³

Audit and feedback interventions may help to prevent patient harm. The Institute for Healthcare Improvement Framework for Safe, Reliable and Effective Care identifies processes of improvement and measurement as a part of the healthcare learning system.¹⁴ In the UK, an ethnographic study of a maternity unit with an excellent record for patient safety found that staff partly attributed its success to a maternity dashboard which facilitated audit and feedback.¹⁵ The Better Outcomes Registry & Network (BORN Ontario) developed a dashboard of six key clinical performance indicators to measure data relating to the quality of maternal and newborn patient care in Ontario, Canada.^16–18 In a pilot study, a quality improvement project focusing on one of the indicators identified by BORN, led to a welcome and significant reduction in the rate of elective repeat caesarean sections in low risk women at less than 39 weeks’ gestation.¹⁹ In an interrupted time series study that evaluated postimplementation effectiveness of the maternal newborn dashboard, the researchers found that it led to improvements in four of the six key indicators.²⁰ A systematic review by Ivers et al²¹ reported that audit and feedback may lead to small improvements in the practice of care delivery by healthcare professionals. In a rapid review, Antony et al found that audit and feedback in combination with other obstetrical safety initiatives may reduce maternal and neonatal morbidity and mortality.²²

It is necessary to identify patient safety indicators to ensure the effectiveness of future audit and feedback strategies. Hearnshaw et al have highlighted the need for useful audit review criteria that are developed or selected using systematic methods.^{23 24} However, in the field of obstetrics, it is currently unclear which indicators relate to, or have the potential to improve obstetric safety. In this scoping review, we want to contribute to the improvement of obstetric patient safety by identifying indicators as the starting point of an approach to harm reduction.

The term indicator is generally understood to mean a measure that provides information on health service delivery across organisations which enables tracking and comparison of performance over time. In a paper first published in 1966, Donebedian identified three types of measures of quality which can be used to evaluate the process of care at the patient-healthcare provider level; outcome measures; process measures and structure measures.²⁵ A fourth type, balancing measures, relate to positive or negative phenomenon which happen as an unintended consequence of a change.¹⁴

Many desirable criteria for measuring the quality of care delivered by a health system have been proposed in the literature. Kessner et al²⁶ suggested using tracers which are a set of specific health conditions or activities. They identified the following criteria for tracers: having the potential to be positively impacted by healthcare activities, clearly definable and easily diagnosable, have a level of prevalence that leads to data collection adequate for statistical analysis, having a demonstrable potential to improve health conditions, relate to processes involving prevention, diagnosis, treatment or rehabilitation and contextual influences should be understood. Janakiraman and Ecker listed their ideal criteria for measures as: easily definable and observable; important to patients and healthcare professionals; identifies areas where improvements are needed and uses accessible data.²⁷

A systematic review by Saturno-Hernández et al found that despite the existence of a large number of indicators for monitoring obstetric care, they were lacking in scientific rigour and many are not suitable for implementation.²⁸ They used the following desirable criteria for indicators: presenting a full description, clearly based on explicit evidence, reliable and feasible (confirmed by pilot testing).

Although it is likely that indicators have the potential to improve patient safety there are no consistent standards for measuring obstetric patient safety. When deciding on indicators it is extremely important to understand what you want to measure and how it can be measured effectively. In order to use obstetric safety indicators as part of a harm reduction strategy they should resonate with healthcare professionals.

Quality and safety indicators are closely linked and sometimes used interchangeably. However, safety indicators are a subtype of quality indicators. For this review, we will use the definition of patient safety indicators used by Kristensen et al, that is, measures that directly or indirectly monitor preventable hospital harms.²⁹ For these indicators it is important that they are mapped to goals which are actionable. In the field of obstetric patient safety, there is a need to identify indicators which measure the aspects of care that relate to preventable harms caused by medical errors. This can be challenging because it is not always clear which indicators should be used and there are variations in practice and outcomes across hospitals, regions and so on.

It is important to note that some safety indicators may be judged as avoidable depending on the patient case. Therefore, we will focus on obstetric safety indicators that relate to the prevention of harms that are at least partly, potentially avoidable to patients.

The aim of our review will be to scope the body of relevant literature³⁰ to identify a comprehensive list of potential obstetric safety indicators which can help reduce preventable patient harms. The objectives of the scoping review are to search for and identify records discussing indicators that can optimise obstetrical safety, to summarise the data and to synthesise the results. This scoping review is being undertaken to inform the process of selecting indicators to be included in a patient safety dashboard by BORN Ontario.

Methods and analysis

We will undertake a scoping review to systematically map the available literature on performance indicators relevant to the reduction of preventable harms within the purview of obstetric safety. We will use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA ScR) checklist³¹ to guide the reporting of the scoping review. We will use the scoping study framework introduced by Arksey and O’Malley³² and the recommendations later developed by Levac et al³³ to guide our methods. We will follow all six suggested stages as described below.

Stage 1: identifying the research question

Through consultation with the author team we defined the concepts, target population and type of indicators of interest and iteratively developed the main research question for the scoping review. We ensured that it aligned with the purpose and scope of the review and the expected outcome (a comprehensive list of relevant indicators).

This question is: According to the available literature, what are the obstetrical safety indicators related to processes of care for low risk births that aim to reduce preventable hospital harms?

For the purposes of this review, we will use the Southern Ontario Obstetrical Network (SOON) definition of a low risk birth target population namely, nulliparous pregnant individuals, singleton gestation with cephalic presentation, who deliver at 37 weeks of gestation or more.³⁴ SOON has also specified exclusion criteria shown in online supplementary appendix 1. The term ‘foetal anomalies’ featured in the SOON definition is an umbrella term for multiple anomalies. Therefore, we will use the list of major sentinel congenital anomalies (see online supplementary appendix 2) that was developed in 2017 by BORN Ontario and Ontario Congenital Anomalies Committee as an additional tool during screening.

Stage 2: identifying relevant studies

The search strategy will be drafted by one of the authors who has trained as an information specialist (AC) and developed iteratively based on consultation with the author team and at least one other library and information professional. We will aim to be as comprehensive as possible while considering resource constraints (such as time) to avoid large volumes of search results. We will search electronic databases such as Medline (Ovid), Embase Classic+Embase 1947–2019 (Ovid), CINAHL (EBSCO), the Cochrane Library via www.cochranelibrary.com. We will search for grey literature resources such as those identified in the Canadian Agency for Drugs and Technologies in Health grey literature checklist Grey Matters: a practical tool for searching health-related grey literature.³⁵ We will also search relevant websites of professional bodies or organisations such as the Society of Obstetricians and Gynaecologists of Canada,³⁶ Royal College of Obstetricians and Gynaecologists,³⁷ Royal Australian and New Zealand College of Obstetricians and Gynaecologists³⁸ and American College of Obstetricians and Gynecologists³⁹ and so on. We will also search governmental bodies and non-governmental organisations that develop or use quality indicators. We will contact individuals or groups to source additional material if deemed appropriate.

We will use both thesauri terms (such as MeSH, Emtree and CINAHL headings) and free text terms as well as Boolean operators and field limiters (where appropriate), to build our search. We will report our conduct of the search including information such as: each database or source of information; dates of the searches; applied limits or filters; the number of records found (for each database or source of information) and search terms from at least one electronic database. We will undertake reference searching and citation searching for records which meet our inclusion criteria.

See online supplementary appendix 3 for our currently proposed search strategy for Medline which may be further developed before it is implemented. It will be tailored according to the functionality of each database or other resource we search.

Stage 3: selecting studies

Two reviewers will undertake a test of inter-rater reliability to increase the likelihood of reliable screening. We will select 50 records and screen them for inclusion. We will calculate Cohen’s kappa.⁴⁰ We may revise the inclusion and exclusion criteria if indicated. When we have achieved a 0.60 or higher level of agreement between raters, we will independently conduct three phases of screening. First, we will screen the titles and abstracts of records that are not randomised trials to determine eligibility for inclusion against the agreed inclusion and exclusion criteria using screening software (Covidence). Second, we will screen 30 randomised trials to see if they identify additional indicators. If this step is not useful, we will not screen the remaining randomised trials. Third, two reviewers will independently screen the full-text records for which it was not clear whether they met inclusion criteria in the previous screening phase. If there is disagreement regarding the inclusion of a record(s), another reviewer will assess the full text. We will undertake inter-rater reliability testing after every 1000 records screened.

Records will be selected using the following inclusion criteria:

• One or more indicator is proposed or in use to prevent harm and improve obstetric safety.

• Specifies or includes a low risk birth population.

• In-hospital settings.

• Care during the period from admission to discharge from hospital following birth.

• Indicators that can be improved or are controllable.

• Feature a level of measurability/auditability.

We will include any type of primary study and most other record types discussing indicators such as reports, guidance documents and discussion papers, with the exception of conference papers. We will not exclude sources based on the year they were made available or published, or on their publication status (published or unpublished). We will only include English language records due to resource restrictions for translation services. We will exclude records focusing on indicators specifically designed for use within developing countries or, countries with low-income or lower-middle income because our indicators should be applicable to a developed country (Canada). We will also exclude indicators that are no longer relevant to modern practice.

We will present an adapted PRISMA flow diagram⁴¹ to document the number of records identified, screened, assessed for eligibility and included or excluded. We will document the main reasons why records are excluded.

Stage 4: charting the data

We will identify variables that are relevant and develop a standardised data collection form.¹⁵ Two reviewers will independently carry out data extraction for each included record. Initially, we will independently extract data from five randomly selected, included records and compare the extracted data for consistency. The form may be iteratively developed throughout the charting process. Data items will include:

• Indicator source (bibliographic information).

• Study design and/or record format.

• Country of origin.

• Type of indicator (outcome/process/structure/balancing).

• Indicator name.

• Indicator definition.

• Numerator.

• Denominator.

• Measurement properties (tools, instruments, etc).

• The population that is targeted by the indicator; maternal, foetal or infant following birth (or a combination).

• The timeframe covered by the indicator; admission for birth including care during labour, during birth, during the postpartum period prior to discharge from hospital.

• Category of hospital harm being addressed.

• Whether indicator relates to a desirable or undesirable event.

• Evidence-based indicator (yes/no/unclear).

Stage 5: collating, summarising and reporting results

As recommended by Levac et al, we will divide this final stage into three parts: analysis of data; reporting of results and discussion of the implications of the results for research, practice and policy.³³

1. We will produce descriptive numerical summaries through analysis of the quantitative data, such as the frequency with which indicators are identified. The qualitative data will be uploaded to NVivo software and thematic analysis will be conducted.

2. We will report our results by compiling a comprehensive list of patient safety indicators identified during our scoping review and organising them according to concepts that best suit the data such as the Donabedian model or categories within the Hospital Harm Framework.

3. We will further explore the implications of the indicators for further research, on clinical practice within obstetric units and how policy analysts or policy-makers may find the results useful.

Stage 6: consultation

This stage is presented as optional by Arksey and O’Malley³² and Levac et al³³ argue that it should be considered a necessary element of the methodological framework. Data from our scoping review will be fed into a future consensus process. This scoping review is the first in a multistep approach to inform the development of a core set of indicators. It is unclear which measures will be considered important to stakeholders so the results of the research will be disseminated to a panel of experts in a consensus process. They will prioritise indicators and we will produce a shortlist with the potential to inform the development of a future obstetrical safety dashboard in the Ontarian health system. As an electronic audit and feedback system, the dashboard could be used to capture, monitor and report surveillance data entered by healthcare professionals and aim to contribute to improvements in obstetric safety.

Ethics and dissemination

This scoping review will not require ethical approval because included sources are or can be made available to the public.⁴²

This research will be disseminated with the intention of reaching a wide audience through the use of multiple modes of delivery such as stakeholder engagement, publication(s) in a peer-reviewed journal and conference paper(s).