Measuring quality in obstetric anaesthesia

Learning objectives.

By reading this article you should be able to:

• •Discuss common terminology used in measuring quality improvement (QI).
• •Identify useful resources for measuring quality of care in obstetric anaesthesia.
• •Explain the common methodologies used in QI data collection and analysis.

Key points.

• •Quality metrics in obstetric anaesthesia can be used to benchmark and improve individual and institutional practice.
• •Quality metrics for obstetric anaesthesia have been published by the American Society of Anesthesiologists and the Royal College of Anaesthetists.
• •Data used for QI should use routinely collected data whenever possible.
• •There is a paucity of routinely collected data for obstetric anaesthesia, which needs to be addressed.

‘We can only be sure to improve what we can actually measure’ (High quality care for all: NHS next stage review final report, June 2008.)

There is a focus in the UK and globally on the quality of maternity care. The UK Government has commissioned recent independent enquiries into the quality of maternity care, which highlighted practice that would benefit from quality improvement (QI) initiatives.^1,2 The Commonwealth Fund has reported problems with the quality of maternity care in the USA whilst the World Health Organization has identified improving maternal healthcare as a global key priority.^3,4 Maternal mortality rates in the UK are not improving and those in the USA are increasing.^5,6 Ethnic disparities in how maternity care, including obstetric anaesthetic care, is provided and delivered, have been identified.⁷ Obstetric anaesthetists have a role in improving maternity care.^8,9 Questions that the subspeciality of obstetric anaesthesia need to address include how the quality of obstetric anaesthetic care should be defined and measured, what a minimum national dataset for obstetric anaesthetic care should include and how routine data collection can be facilitated both locally and nationally to support benchmarking and QI efforts.⁸ Despite recent published work on developing quality indicators and patient-reported outcome measures (PROMs) for obstetric anaesthesia, this work is yet to embed into routine practice.10, 11, 12 The adoption of quality metrics into routine practice may be constrained by the healthcare setting and resources available.

Definitions and concepts

Quality is an attribute that describes how good or bad something is measured against a standard. In healthcare, quality is defined as ‘the degree to which health services for individuals and populations increase the likelihood of desired health outcomes. It is based on evidence-based professional knowledge and is critical for achieving universal health coverage’.¹³ A measure is a number or value at a point of time and is unit specific (e.g. number of women who underwent spinal anaesthesia for elective Caesarean section last month), whilst a metric is a relationship between measures (e.g. percentage of women who had elective Caesarean section under spinal anaesthesia) and is not necessarily unit specific. If the metric is used to evaluate care it is a clinical indicator, but if there is a standard or evidence-based value for comparison it becomes a quality indicator. If it is compared with a target, then it is a performance indicator. However, these terms are often used interchangeably.

The quality of healthcare can be evaluated through metrics of safety (e.g. incidence of high spinal block), effectiveness (e.g. incidence of intraoperative pain experienced during spinal anaesthesia) and patient-centredness (e.g. postoperative patient-reported experience or outcome).¹⁴ Equity (e.g. accessibility of antenatal information to disadvantaged patients), efficiency (e.g. theatre utilisation for elective Caesarean births) and timeliness (e.g. time taken to provide requested labour epidural analgesia) are also recommended domains for inclusion in quality evaluation.

The delivery of healthcare services can be evaluated through assessment of Donabedian's triad of domains: structure, process and outcome.¹⁵ Structure describes the context in which care is delivered, including hospital buildings, staff, financing and equipment. Process denotes the transactions between patients and providers throughout the delivery of healthcare. Finally, outcomes refer to the effects of healthcare on the health status of patients and populations. Examples of these components for evaluating the delivery of obstetric anaesthetic care are shown in Table 1.

Table 1.

Examples of healthcare evaluation for obstetric anaesthetic care.

Structure	Process	Outcomes
Number of obstetric operating theatres	Proportion of women who have an elective Caesarean birth with neuraxial anaesthesia	Number of women with neuraxial anaesthesia who experience pain during Caesarean section
Number of obstetric anaesthetists available during a shift	Mean time from epidural analgesia request to start of epidural analgesia	Adequacy of labour epidural analgesia at 45 min
Preoperative assessment service for elective Caesarean sections	Proportion of women who receive prescribed regular non-opioid analgesics after elective Caesarean birth	Patient satisfaction with pain management after elective Caesarean section on first postoperative day

A quality evaluation of a care pathway (e.g. elective Caesarean sections) or service (i.e. obstetric anaesthesia) should ideally involve quality metrics that can address all domains of quality (safety, effectiveness, patient-centredness, equity, efficiency, timeliness) across the three components of healthcare delivery (structure, process and outcomes).

What and how to measure

Not all qualities of care are so readily quantifiable that they are measurable. What is measured depends on the quality metrics selected and the reason for the quality evaluation. The impetus for the quality evaluation may arise from routine inspection (e.g. Care Quality Commission, www.cqc.org.uk), or through local concern about quality of care resulting from local incident reports, complication rates, patient complaints, or as part of regular local surveillance of service quality. It may arise from a need to benchmark against national standards or peer hospitals in response to national directives or reports (e.g. recommendations from the Confidential Enquires into Maternal Deaths reports).

Measures may require collection of new data, but this can be burdensome. Ideally the data should already be routinely collected and available.¹⁶ The key principles when collecting data are shown Table 2.

Table 2.

Key principles when collecting data for measurement in quality improvement.¹⁶

Usefulness is more important than perfection

Collect the minimum data for what is needed

Goal is improving care and not improving data quality

Reduce data burden by making data collection routine

Engage the team by informing and explaining the purpose of data collection

The challenge for obstetric anaesthetists is knowing which metrics to use to for improving the care that they provide throughout their patients' perinatal care. Research on postpartum outcomes is still new, and specific PROMs for various postpartum health-related quality of life (HRQoL) aspects including recovery, pain, anxiety, sleep and fatigue are not yet well established. National data on obstetric anaesthesia care are limited partly because of a lack of consensus on what metrics should be measured, and because the cost benefit value of such data has not yet been established.¹⁷ As there is usually no imperative for local hospitals to report data nationally, there is little local support to facilitate either routine data collection or to make the data accessible to staff.¹⁸ There have been attempts to undertake national aggregation of local data for some metrics. For many years the Obstetric Anaesthetists' Association (OAA) oversaw a national collection of local data in the UK, but data collection was suspended in 2015 because of variable participation rates by local reporters, in part as a result of the challenges of accessing local data. However, a published analysis of these aggregated data between 2009 and 2014 provides a useful resource for benchmarking.¹⁹

Published quality metrics

Because the available evidence base is limited, published quality metrics in the specialty of anaesthesia frequently emerge from consensus-driven processes or expert opinion. This is especially true for the field of obstetric anaesthesia, where various groups have attempted to identify, validate and publish quality metrics to be used to improve maternal care and reduce peripartum complications.

A Delphi survey of 133 stakeholders (including obstetric anaesthetists, other maternity care health professionals, and service users) sought to identify quality indicators considered most useful to improve quality of care in obstetric anaesthesia.¹⁰ Most quality indicators selected by service users represented the Donabedian domains of service structure and process. The core set of five quality indicators, chosen in a consensus meeting involving mostly obstetric anaesthetists, was influenced by the measurability of the quality indicator. A core outcome set for enhanced recovery in patients undergoing Caesarean delivery has also been developed using an international Delphi process with multidisciplinary stakeholders.¹² Although there may be overlap, there may be differences between those metrics considered optimal for use in research studies and those that are most useful for driving QI in routine care.

In the absence of national data or agreed minimum datasets, comparative metrics to use for quality indicators can be sought from publications by governing bodies such as the Royal College of Anaesthetists (RCoA), who provide recommendations for process and outcome quality metrics (Raising the Standards: RCoA Quality Improvement Compendium) and for structure and process quality standards (Guidelines for the Provision of Anaesthetic Services [GPAS]).^20,21 The ASA has also recently published quality metrics on obstetric anaesthesia.²² The Obstetric Practice chapter in the RCoA Raising the Standards document proposes 90 different quality metrics and suggests some comparator standards.²⁰ Without national benchmark data, much of the evidence base for comparator standards depends on a few research studies, published single centre data or on expert opinion.

Other sources of published standards include the Procedure-Specific Postoperative Pain Management (PROSPECT) guideline for elective Caesarean section, the RCoA Anaesthesia Clinical Services Accreditation scheme (using standards based on RCoA GPAS recommendations) and the Society for Obstetric Anesthesia and Perinatology (SOAP) criteria for accreditation as a SOAP Center of Excellence.23, 24, 25 An important focus for QI is the patients' perspectives of the care received, in particular the patient's view of their symptoms, ability to undertake normal daily life activities and HRQoL.²⁶ Patient perspectives can be evaluated by using PROMs, which focus on the effectiveness and safety of care, and patient-reported experience measures of care (PREM), which involve dimensions of care such as compassion, respect, timeliness and being informed.²⁷

Comparative examples of quality metrics and standards for labour epidural analgesia published by ASA and RCoA are shown in Table 3.

Table 3.

Examples of quality metrics and standards for labour epidural analgesia published by ASA and RCoA. ∗American Society of Anesthesiologists. Statement on Quality Metrics.^22†RCoA Quality Improvement Compendium: Chapter 7 Obstetric Practice.²⁰

Quality metrics (ASA)∗	Quality metrics (RCoA)†	Quality standards (RCoA)†
Time between epidural placement and adequate analgesia	Adequacy of epidural analgesia at 45 min (patient satisfaction)	Adequate epidural analgesia at 45 min after start of procedure >88%
Time from patient request for neuraxial analgesia to arrival of anaesthetist or other member of the anaesthesia care team	Percentage with response time to delivery of neuraxial analgesia after a request <30 min; 30–60 min; >60 min	Response time should not normally exceed 30 min and must be within 1 h
Percentage of eligible patients requesting neuraxial analgesia who did not receive it
Frequency of assessment for adequate analgesia during labour
Proportion of labour epidurals that requires resiting during labour	Percentage of epidurals re-sited in labour	Rate of re-sited epidurals during labour <15%
Percentage of labour epidurals that attempted conversion to surgical anaesthesia and failed
Accidental dural puncture rate of epidurals for labour or Caesarean section	Accidental dural puncture	Accidental dural puncture rate <1%
	Patient satisfaction with labour epidural analgesia	Patient satisfaction at follow-up >98%

Patient-reported outcome measures

Patient-reported outcome measures are designed to evaluate multiple health domains from the perspective of the patient including HRQoL. They can be administered to large numbers of patients using paper or electronic questionnaires. Although many PROMs have been used to evaluate different aspects of peripartum care, few current PROMs are specific for the care delivered by obstetric anaesthetists. The ObsQoR-10 (Obstetric Quality-of-Recovery-10) is the most studied PROM in this context.¹¹

The ObsQoR, initially developed to evaluate postoperative recovery from elective Caesarean section, has now been evaluated for use after emergency Caesarean and vaginal births.28, 29, 30 It is an adaptation of previous generic postoperative quality of recovery PROMs (QoR-40 and QoR-15) to better reflect those particular dimensions relevant to obstetric anaesthesia care such as the common use of neuraxial anaesthesia and the needs of the patient to provide maternal care to a newborn child.^31,32 The original ObsQoR had 11 items, but the latest version has 10, four concerning symptoms (pain; nausea/vomiting; dizziness and shivering) and six concerning HRQoL (comfort; mobilisation; ability to hold and to feed a baby without assistance; personal self-care and sense of control), each scored on a numerical scale 0–10, with 10 being the best score for each item.¹¹ The ObsQoR results are presented as an arithmetic sum of scores for all the items. There have been several validation studies of the use of ObsQoR-10 in different countries in which native languages are not English, as it is not sufficient to merely translate the words of a PROM tool. The ObsQoR-10 may be a useful tool for measuring and benchmarking QI in enhanced recovery care pathways and for use in postnatal care in general.^12,33

A practical approach to quality improvement in obstetric anaesthesia

An overview of the QI process is shown in Figure 1 and an example is illustrated in Box 1.

Fig 1.

A process for a quality improvement (QI) project.

Box 1. A case example of a QI project.

•A QI project was initiated by a patient complaint about the quality of postoperative analgesia after elective caesarean birth (ECB). •Local and national standards were that 100% of women, if no contraindications, should be prescribed regular non-opioid analgesia following ECB. •After stakeholder and clinical audit department consultations, a QI project was designed to: measure quality of analgesia prescribing and administration; do a patient experience survey on quality of post-operative analgesia provision; and undertake a ward staff survey on barriers to providing post-operative analgesia. •A 120 sample size was calculated for a simple randomised selection of electronic medical records (EMRs) prescription charts of women who had ECB in the past 12 months; a stratified random sample by ethnicity of 50 women was chosen for a patient experience survey; and a convenience sample of ward staff was surveyed about barriers to care. All data collected were anonymised. •After 4 months, results were presented at local audit meetings. A pie chart was used to show % of women prescribed regular analgesia, a bar chart was used to show frequencies of missed analgesic drug doses, and patient satisfaction scores by ethnic group. The 100% target for analgesia prescription and administration was missed, and patient dissatisfaction was recorded. •Reasons for failures were interrogated using the ‘Five Whys’ and process mapping. •A QI strategy was implemented using PDSA model. Electronic reminders for analgesia prescription and administration were embedded into the EMR and a self-administration of medicines (SAM) programme was set up. •The success of QI implementation was monitored by a series of ‘rapid-cycle samples’ audits.

Alt-text: Box 1

Any project to measure quality of care should be registered so that there is an institutional record of the project and governance associated with it. Local hospitals should have clinical audit departments with whom projects can be registered and from whom information and support can be obtained.

Different approaches to QI projects include:

• (i)Service evaluation: when a project measures the standard of a service but does not compare against predetermined standards.
• (ii)Clinical audit: when a project measures and compares the standard of service against a predetermined standard.
• (iii)Research project: if a project aims to generate new knowledge or to test or generate a hypothesis.

The category of project determines the permissions that are required to undertake it. The large majority of QI projects will be either service evaluations or audits.

Useful web resources for information on data measurement have been published by the Healthcare Quality Improvement Partnership, NHS Institute for Innovation and Improvement and by NHS Elect.^16,34,35 Key points from these resources are summarised in the following sections.

Sampling

Sample size

If the data only represent a sample of the patient population data, then there needs to be confidence that the sample is big enough and representative to draw any conclusions from the data analysis. Useful sample size calculators can be found in open access resources.^34,36,37 If simple random sampling is used, then samples sizes between 50 and 100 may be sufficiently reliable for most audit purposes.

Sample selection

Sampling should ideally be random to reduce risk of bias.³⁴ Random sampling methods include:

• (i)Simple random sampling, where each patient within the given population to be measured is allocated a sequential number then a random sample set of numbers can be generated by a computer program for the range of numbers of patients within the population. There is a random number generator function available in Microsoft Excel® (Microsoft Corp., Redmond, WA, USA). This could be used, for example, to randomise patients to receive a patient survey on the timeliness of receiving their labour epidural analgesia.
• (ii)Systematic random sampling involves selecting patients in a list by a fixed interval (i.e. every nth patient). The sampling interval used will depend on the number of patients in the population and the required sample size (i.e. population size/sample size=nth interval). This could be used, for example, to review whether an assessment of motor block and block height was documented hourly during labour epidural analgesia.
• (iii)Stratified random sampling is when the population is divided according to a categorisation (e.g. by ethnic group). With stratified sampling the sample size for each category can be varied according to the estimated percentage occurrence of the measure. This could be used, for example, to measure the quality and effectiveness of postoperative analgesia after a Caesarean birth.
• (iv)Rapid-cycle sampling may be used when it is already known that there is a problem and repeated results are required quickly to monitor care. This allows audit with relatively small sample groups, implement a QI, and re-audit with another small sample group. This could be used for PDSA (plan, do, study, act) cycles to assess effectiveness of implementing self-administrative medication to improve postnatal pain management.

There may be reasons why random sampling is not possible or is difficult. Alternative non-representative techniques can be used in QI projects. These include:

• (i)A purposive sample when the population cannot be specified or there is need to focus on a particular group (e.g. length of postnatal hospital stay for women who have experienced a post-dural puncture headache, or for women from an ethnic minority).
• (ii)A convenience sample is an opportunistic sample (e.g. a survey of a shift of ward staff about barriers to providing postnatal care, such as giving postnatal analgesia).

How to measure and interpret data

Patterns in data may suggest areas of care that could be improved. The type of data will determine the analysis. Statistical techniques can be applied to the data to determine whether the measured outcomes are more likely to have occurred as a result of chance rather than the method or quality of care that was given.

Data can be presented in tables, but graphical presentation allows patterns to be more recognisable. Examples of useful graphical presentations of QI data are shown in Figure 2.

Fig 2.

Types of graphical representations of QI data.

After measurement what next?

Once the quality of care has been measured then a judgment needs to be made whether the quality is acceptable or meets a standard. This is where benchmarks or standards are required. These standards may be local historical or those published in guidelines from professional societies (e.g. RCoA Raising the Standards; SOAP Center of Excellence criteria), peer hospital group standards or national standards.^20,25 Data must be interpreted in context after determining whether any differences demonstrated are clinically significant and whether the differences are the result of modifiable or non-modifiable factors. Examples of factors that may impact QI metrics over a data collection period include staff education (e.g. teaching quality and frequency), infrastructure (e.g. staffing ratios), equipment (e.g. availability or changes), patient (e.g. complexity of needs, preferences), anaesthesia (e.g. drug availability, seniority of anaesthetist) and surgical factors (e.g. duration of surgery, indication for surgery, seniority of surgeon).

Understanding what needs to be improved

‘ … the first difficulty is to see that the problem is difficult.’ (Bertrand Russell. An Inquiry into Meaning and Truth, 1940)

Numerous QI methodologies can be used to understand how and where care can be improved. An excellent resource that describes these methodologies and others, can be found at the Advancing Quality Alliance website: https://aqua.nhs.uk/qsir-tools/. These include:

• (i)The five whys. Write down the problem and ask the question why it is so. Then question each successive answer with another ‘why’. Exploring with successive ‘why’ questions, usually five times, may reveal the underlying primary cause. For example, ‘why does it take 30 minutes for O negative blood to arrive in an emergency?’
• (ii)Socratic method.³⁸ This is a staged critical thinking approach to a problem. An example of this method applied to a QI project would be: (i) What is the reason for why care is not optimal? (ii) How do we know that this is the reason? (iii) Is there evidence that this is the reason? What is the evidence? Is more information required? (iv) Are there alternative reasons? (v) What are the implications if the assumed reason is correct? (vi) What are the consequences if the assumed reason is wrong?
• (iii)Process mapping. This maps out a process of care to identify the number and order of stages of care involved, the tasks involved, who is involved, how care is handed over between stages, time involved, bottlenecks and duplications.
• (iv)The Fishbone diagram. This is a tool for understanding the potential causes of a problem and their effect. The diagram is made by stating the problem (i.e. ‘the fish head’), then working back along the spine of the fish listing as spine offshoots the major factors that may impact the problem. For each of these major factors, smaller spine offshoots are generated to include factors that influence each major factor. To assist in determining the problems, the ‘what’, ‘where’, ‘when’ and ‘how much’ interrogations should be used.
• (v)Pareto chart. If measurements have been charted as a bar chart, then the bar chart can be ordered so that the bars are ranked from largest to smallest. The concept is that 80% of an effect can be attributed to 20% of causes. Therefore, to improve a service most attention should be first directed to those 20% of causes that could make the biggest contribution to improving quality of care.

Implementing quality improvement

After measuring care, recognising a need to improve care and identifying how care could be improved to make a difference, the next stage is to implement a change in care that improves it. The strategies and methodologies to implement QIs are beyond the scope of this article but have been discussed and summarised in previous BJA Education reviews.^39,40 Examples of these methodologies include PDSA (e.g. to implement an enhanced recovery protocol for care after elective Caesarean sections), and Lean Six Sigma (define, measure, analyse, improve, control) (e.g. to improve the efficiency of elective lists for Caesarean sections).^41,42

Summary

Quality improvement endeavours in obstetric anaesthesia can play a key role in improving maternal healthcare through benchmarking and improving individual and institutional practice. Several professional societies have developed quality metrics by expert consensus. Measurement for QI should use routinely collected data whenever possible. Quality should be measured across all six domains of healthcare quality for the Donabedian triad of dimensions of healthcare delivery. It should include assessment of the breadth of maternal care throughout the perinatal journey to include antenatal and postnatal care, and intrapartum care. Measurement for QI involves comparison against defined standards and should be interpreted in context.

Acknowledgements

The authors thank Dr Nuala Lucas for reviewing a final draft of the article and advising on editorial improvements to the article.

Declaration of interests

JB is honorary treasurer and a trustee of the Obstetric Anaesthetists' Association, co-author RCoA GPAS for an Obstetric Population and has been a clinical assessor for MBRRACE-UK.

PS has received funding from National Heart, Lung, and Blood Institute (R01HL166253-01A1) and National Institute of Child Health and Human Development (U54HD113142-01).

MCQs

The associated MCQs (to support CME/CPD activity) will be accessible at www.bjaed.org/cme/home by subscribers to BJA Education.

Biographies

Jim Bamber MMedSci MRCGP DRCOG FRCA is a consultant at Cambridge University Hospitals. He is honorary treasurer of the Obstetric Anaesthetists' Association (OAA), coauthor of the RCoA GPAS chapter on the obstetric population and has been national co-lead anaesthetic assessor for MBRRACE-UK. He chairs the OAA quality and outcomes subcommittee.

Pervez Sultan FRCA, MD (Res) is an associate professor at Stanford University School of Medicine, USA and honorary professor at University College London. He is an NIH-funded researcher whose research interests include improving postpartum recovery and the development of the ObsQoR-10 tool. He is a board member of the Society for Obstetric Anesthesia and Perinatology (SOAP) and a member of the ASA subcommittee for obstetric anesthesia.

Matrix codes: 1I05, 2B01, 2B02, 2B03, 2B04, 3B00