Elective surgery system strengthening: development, measurement, and validation of the surgical preparedness index across 1632 hospitals in 119 countries

Abstract

Background

The 2015 Lancet Commission on global surgery identified surgery and anaesthesia as indispensable parts of holistic health-care systems. However, COVID-19 exposed the fragility of planned surgical services around the world, which have also been neglected in pandemic recovery planning. This study aimed to develop and validate a novel index to support local elective surgical system strengthening and address growing backlogs.

Methods

First, we performed an international consultation through a four-stage consensus process to develop a multidomain index for hospital-level assessment (surgical preparedness index; SPI). Second, we measured surgical preparedness across a global network of hospitals in high-income countries (HICs), middle-income countries (MICs), and low-income countries (LICs) to explore the distribution of the SPI at national, subnational, and hospital levels. Finally, using COVID-19 as an example of an external system shock, we compared hospitals' SPI to their planned surgical volume ratio (SVR; ie, operations for which the decision for surgery was made before hospital admission), calculated as the ratio of the observed surgical volume over a 1-month assessment period between June 6 and Aug 5, 2021, against the expected surgical volume based on hospital administrative data from the same period in 2019 (ie, a pre-pandemic baseline). A linear mixed-effects regression model was used to determine the effect of increasing SPI score.

Findings

In the first phase, from a longlist of 103 candidate indicators, 23 were prioritised as core indicators of elective surgical system preparedness by 69 clinicians (23 [33%] women; 46 [67%] men; 41 from HICs, 22 from MICs, and six from LICs) from 32 countries. The multidomain SPI included 11 indicators on facilities and consumables, two on staffing, two on prioritisation, and eight on systems. Hospitals were scored from 23 (least prepared) to 115 points (most prepared). In the second phase, surgical preparedness was measured in 1632 hospitals by 4714 clinicians from 119 countries. 745 (45·6%) of 1632 hospitals were in MICs or LICs. The mean SPI score was 84·5 (95% CI 84·1–84·9), which varied between HIC (88·5 [89·0–88·0]), MIC (81·8 [82·5–81·1]), and LIC (66·8 [64·9–68·7]) settings. In the third phase, 1217 (74·6%) hospitals did not maintain their expected SVR during the COVID-19 pandemic, of which 625 (51·4%) were from HIC, 538 (44·2%) from MIC, and 54 (4·4%) from LIC settings. In the mixed-effects model, a 10-point increase in SPI corresponded to a 3·6% (95% CI 3·0–4·1; p<0·0001) increase in SVR. This was consistent in HIC (4·8% [4·1–5·5]; p<0·0001), MIC (2·8 [2·0–3·7]; p<0·0001), and LIC (3·8 [1·3–6·7%]; p<0·0001) settings.

Interpretation

The SPI contains 23 indicators that are globally applicable, relevant across different system stressors, vary at a subnational level, and are collectable by front-line teams. In the case study of COVID-19, a higher SPI was associated with an increased planned surgical volume ratio independent of country income status, COVID-19 burden, and hospital type. Hospitals should perform annual self-assessment of their surgical preparedness to identify areas that can be improved, create resilience in local surgical systems, and upscale capacity to address elective surgery backlogs.

Funding

National Institute for Health Research (NIHR) Global Health Research Unit on Global Surgery, NIHR Academy, Association of Coloproctology of Great Britain and Ireland, Bowel Research UK, British Association of Surgical Oncology, British Gynaecological Cancer Society, and Medtronic.

Introduction

The COVID-19 pandemic highlighted the fragility of elective surgical services around the world, yet global surgery risks being neglected in pandemic recovery planning.1, 2, 3 At the start of 2022 an estimated 200 million patients worldwide were awaiting surgery.1, 2 For time-critical conditions, such as cancer, one in seven patients did not have their planned surgery during SARS-CoV-2 outbreaks and many more had substantial delays to their care.³ Some patients might never have accessed the surgery they required, with high associated disability and millions of years of healthy life lost.4, 5 With the existing challenges in providing accessible and safe surgical systems in low-income countries (LICs) and middle-income countries (MICs) identified by the 2015 Lancet Commission on global surgery, health systems and hospitals with less funding for infrastructure, staffing and equipment were the worst affected, with whole-societal health, economic, social, and political consequences.3, 6 The backlog of patients awaiting planned procedures is now one of the most pressing challenges to global health for the next 10 years.

Research in context.

Evidence before this study

The 2015 Lancet Commission on global surgery identified surgery and anaesthesia as an indivisible component of holistic health systems. The COVID-19 pandemic has revealed ongoing fragility in surgery and anaesthesia systems, with more than 200 million patients currently awaiting their planned procedures. We searched PubMed and Embase from the inception of each database to March 4, 2022, without date limits for indices, frameworks, or guidelines able to assess a hospital or surgical system's ability to deliver planned surgery and anaesthesia during periods of external system stress. Planned surgery included all operations done when a decision for surgery was made before hospital admission, whether this was elective or expedited. We used search terms related to “preparedness”, “resilience”, “pandemic”, or “system stress” in combination with “surgery”, “anaesthesia”, “surgical systems”, “procedures”, or “non-communicable disease”. External system stressors included airborne pandemics (eg, SARS-CoV-2), non-airborne disease (eg, Ebola virus), and other system stressors (eg, natural disasters, mass trauma, warfare, political instability, and extreme weather events). We identified 15 indices and six frameworks for assessing whole-health system preparedness, but none were specific to surgery nor were they validated against a measure of planned surgical volume. We also identified three tools to quantify essential surgical capacity: WHO Situational Assessment Tool, PIPES Tool, and Ethiopian Hospital Assessment Tool. However, these were not designed to assess preparedness and are complex to complete. Together, this shows that the features of prepared surgery and anaesthesia systems are not yet well understood or properly implemented.

Added value of this study

The Surgical Preparedness Index (SPI) is the first tool that specifically assesses elective surgery and anaesthesia system preparedness. We engaged a diverse, international, and multidisciplinary community to identify and prioritise features of prepared surgery and anaesthesia systems that were relevant across a wide variety of external system shocks. We prioritised 23 globally relevant indicators of surgical preparedness across four domains (facilities, staffing, prioritisation, and processes). The SPI was then measured across a range of hospitals and settings showing significant variability in preparedness between hospitals, regions, and countries. During the COVID-19 pandemic, a pressing and globally relevant example of a system stressor, three-quarters of hospitals reported a reduction in planned surgical volume. The SPI score was shown to be strongly associated with a hospital's ability to continue planned surgery, validating the concept of preparedness in reducing surgical cancellations, with a significant and measurable effect. This relationship was consistent across different types of hospital and health systems, suggesting that SPI measurement was generalisable across contexts.

Implications of all the available evidence

The COVID-19 pandemic highlighted the fragility of surgical services around the world, yet surgery risks being neglected in pandemic recovery planning. Without effective, integrated surgical and anaesthesia systems, non-communicable diseases cannot be effectively treated and community health declines, meaning Sustainable Development Goal 3 cannot be met. Application of the SPI can identify areas for policy change, advocacy, and investment at subnational and local levels. Hospitals should urgently implement annual SPI assessment and create local action plans to strengthen planned surgical services, thus supporting whole-health system resilience. Longitudinal assessment of surgical preparedness can now be incorporated into national surgical, obstetric, and anaesthesia planning and considered an essential indicator of surgical system strength. Future work is required to test the SPI in low-income countries (4·3% of included hospitals).

The SARS-CoV-2 pandemic presented an unprecedented stress on global health systems. Many surgery and anaesthesia services changed their processes for patient selection and reduced their volume of planned procedures, reflecting the high risk to patients planned to receive surgery of perioperative SARS-CoV-2 infection.⁷ Different models of care have been proposed to support safe upscaling of planned surgical volume during pandemic recovery.8, 9 However, shared global learning about the best methods to improve preparedness of surgical systems has not been done. Surgical capacity urgently needs to be upscaled to address growing backlogs of patients waiting for their planned procedures and improve preparedness for future system shocks. Solutions need to be identified for infrastructure, staffing, and care pathways that can be applied flexibly across different health systems.¹⁰ COVID-19 (an airborne pandemic) has been just one form of external stress on health systems, but provides an important learning opportunity for surgical providers and policy makers to strengthen surgical preparedness ahead of future system shocks.

Although several indices of health system preparedness and surgical capacity have been proposed, these were not designed to assess preparedness of surgery and anaesthesia services nor have they been validated against a measure of surgical capacity.11, 12, 13 Whole-health system preparedness indicators are often not applicable to surgery, and surgical capacity indicators (eg, the WHO Situational Assessment Tool, PIPES Tool, and Ethiopian Hospital Assessment Tool) are not designed to dynamically assess the response of services to external system pressure and are too complex for everyday use. This study describes the development and validation of a multinational surgical preparedness index (SPI) and framework to support elective surgery and anaesthesia services, strengthening them against future external system shocks.

Methods

This study was done in three phases (figure 1 ). First, index development. An international consultation was done with a Delphi consensus methodology to develop hospital-level preparedness indicators to support surgical service strengthening. For the purposes of this study, preparedness was defined as the ability of a hospital to maintain capacity for planned surgery during periods of external system shock. Planned surgery was defined as any operation for which the decision for surgery was made before the hospital admission during which the operation took place. Second, measurement of surgical preparedness. A cross-sectional hospital assessment study was done to assess the distribution of the total SPI score at national, subnational, and hospital levels. Third, validation of the multidomain SPI against the observed versus expected elective surgical volume during the COVID-19 pandemic; this was used as a contemporaneous example of a globally relevant system stressor.

Figure 1.

Overview of study design

IIDG=international guideline development group. SPI=surgical preparedness index.

Phase 1: development of the SPI

The international consultation was done with a diverse, multidisciplinary, expert index development group. This included surgeons, anaesthetists, critical-care doctors, nurses, and hospital managers involved in the delivery of planned surgical care across high-income country (HIC), MIC, and LIC settings. The range of people included across care roles and income setting was designed to provide breadth of perspectives during consensus rounds and fulfils typical sample size requirements for Delphi methodologies.¹⁴ A four-stage Delphi process was done within the development group to prioritise hospital-level SPIs. Consensus definitions were set a priori, and the process was done in accordance with best practice recommendations: (1) the expertise matrix was predefined, inclusive, and generalisable; (2) dropping rules were predefined; (3) a limit of two voting and two face-to-face rounds was prespecified; and (4) frequent reminders were sent to respondents to maximise the retention rate.¹⁵ The full methodology for the consultation process is described in the appendix (pp 70–73).

To explore the relevance of the surgical preparedness indicators across other external system shocks, we used a consensus ratings exercise with eight international development group members (two from high-income countries [HICs], four from MICs, and two from LICs). We defined five different external shocks: airborne pandemic, non-airborne pandemic, warfare and political instability, natural disasters, and seasonal pressures. Independent members were asked to rate the relevance (high, moderate, or low) of each indicator in their local context. Inter-rater reliability was estimated using intraclass correlation coefficient (ICC[1k]; one-way random effects, average of k raters) presented with 95% CIs.

Phase 2: preparedness of global surgery and anaesthesia systems

A hospital-level assessment of the SPI was done between June 6 and Aug 5, 2021, and data were recorded by local assessors on a centralised, encrypted Research Electronic Data Capture server hosted by the University of Birmingham, Birmingham, UK.¹⁶ COVIDSurg is a network of more than 15 000 front-line clinicians from the National Institute for Health Research (NIHR) Global Health Research Unit on Global Surgery focused on supporting data-driven decision making in perioperative care.⁷ The network facilitated distribution of the SPI assessment was sent to local clinicians and managers to complete for their hospital. Collaborators were encouraged to identify other colleagues to complete multiple assessments of the same hospital to evaluate inter-rater reliability. Any centre worldwide providing planned surgery was eligible to participate. Any postgraduate clinician or manager involved in perioperative care from any specialty background in these centres was eligible to participate. Clinicians without a temporary or permanent contract (ie, locum doctors or equivalent) and medical students were not eligible to participate.

Features of hospital assessors and hospitals were summarised overall and by country income group. To promote application and interpretation of the SPI in clinical practice, we calculated global, regional, and national distributions of SPI. We also disaggregated by hospital type, country income, COVID-19 burden, and country. Where multiple assessments were made of the same hospital, the mean was calculated first by indicator, then an overall mean index score was calculated as an aggregate mean of these means. Results presented across subgroups were calculated as the mean of hospitals' mean SPI scores in each group and presented with a 95% CI. High fidelity centre-level SPI data were presented online on a Shiny (Boston, MA, USA) application hosted on an Argonaut server at the University of Edinburgh, Edinburgh, UK. The inter-rater reliability of SPI assessment was estimated again using the ICC(1k) with 95% CIs.¹⁷

We explored the relationship between national mean SPI scores and four relevant global health indicators using generalised additive modelling fitted with a penalised cubic spline (with shrinkage). The four selected indicators were: (1) the UN's Human Development Index, which is a composite index of life expectancy, education, and per capita income (a higher Human Development Index score indicates greater development); (2) global health security index, which is an assessment of global health security capabilities (ie, a measure of whole health-system resilience) from the Johns Hopkins Center for Health Security, Baltimore, MD, USA, the Nuclear Threat Initiative (Washington DC, USA), and the Economist Intelligence Unit (London, UK; a high global health security score indicates a more resilient health system); (3) the WHO Universal Health Coverage (UHC) service coverage index, which combines 14 tracer indicators of service coverage into a single summary measure (a higher UHC index indicates greater coverage); and (4) Gini coefficient, which is a measure of population wealth inequality (a Gini coefficient of 0 expresses perfect equality; a coefficient of 1 indicates maximal inequality). Analyses were done with R Studio (version 4.1.1) packages: tidyverse, finalfit, psych, and ggplot2.

Phase 3: validation of the SPI using planned surgical volume during COVID-19

To evaluate the criterion validity of the SPI, we compared a hospital's self-assessed SPI score with its ability to maintain planned surgery capacity. This was estimated using the observed to expected planned surgical volume ratio (SVR), calculated as the ratio of each hospital's observed planned surgical volume over a 1-month assessment period against the expected planned surgical volume based on data from the same month in 2019 (the prepandemic baseline) and expressed as a percentage. Case volume data were measured from routinely collected hospital administrative data, such as theatre logbooks and electronic health-care records. A planned surgery case was defined as any planned admission for a procedure done by a surgeon in an operating theatre under general, regional, or local anaesthesia. This included procedures classified as either elective or expedited in the National Confidential Enquiry into Patient Outcome and Death classification system, but excluded urgent and immediate surgery.¹⁸ Patients undergoing surgery for any indication were eligible for inclusion, including benign disease, cancer, trauma, or obstetrics. This included day-case procedures (ie, discharged same date as operation).

Analyses were done using R Studio packages tidyverse, finalfit, lmer, and ggplot2. A complete-case analysis was preplanned if missing data were both missing at random and in a low number of samples (<5%).¹⁹ In the prestudy protocol, we planned to impute missing data using multiple imputation by chained equations based on a missing at random or missing completely at random assumption if data missingness was more than 5%. Centres with no current planned surgery volume estimate were excluded from analyses. Generalised additive models were fitted using a penalised cubic spline (with shrinkage). Models were initially fitted with a basis dimension of 10 (k). Model fit was checked using residual plots, convergence confirmed, and basis dimension choice checked. If per group estimated degrees of freedom approached basis choice minus one (k–1), then the basis dimension was increased. The link function was identify. A random-error distribution was assumed and checked on residual plots as above. To explore whether this association could be explained by confounding we created a mixed-effects linear regression model with country included as a random effect (normal distribution). We checked assumptions by exploring normality and homogeneity of variance of residuals and linearity of quantitative predictors.

Model coefficients were adjusted for predefined centre-level and country-level confounders that were identified through a scoping review of published literature and considered a priori by the international development group as likely to be clinically and causally linked to both exposure and outcome. A proposed casual model was presented in a directed acyclic graph. Covariables included country income—defined according to World Bank 2018 definitions and classified as HIC, MIC (including both upper-middle and lower-middle classifications), or LIC on the basis of annual gross domestic product per capita (US$); hospital funding (public, private, or mixed public and private); surgical service provision at the facility (planned only versus planned and unplanned); hospital location (defined by the assessor as primarily an urban, rural, or mixed urban and rural area); number of hospital beds (<50, 50–99, 100–199, 200–499, 500–999, or ≥1000); and country COVID-19 burden (low, moderate, or high) at the time of SPI assessment. The Oxford COVID-19 Government Response Tracker (OxCGRT) was used as a surrogate of the overall COVID-19 burden on a local health system at the time of the SPI assessment. The OxCGRT is a composite of 19 indicators, including measures and behavioural interventions associated with containment and closure, economic response, and health systems with an overall score range between 0 (no restrictions) and 100 (most stringent restrictions). It has been validated for use globally by showing associations with planned surgical volume,³ population SARS-CoV-2 infection rates, and Google mobile phone mobility data.²⁰ OxCGRT cutpoint scores used in previous work based on comparisons of index scores and national policy sources were used.³ Each hospital was given a classification based on the country's status at the time of assessment: low COVID-19 burden (index <20), moderate COVID-19 burden (20–60), and high COVID-19 burden (>60). The OxCGRT was preferentially used instead of SARS-CoV-2 case notification rates because of global differences in access to testing and reporting.21, 22 Subgroup analyses were presented by country income, COVID-19 burden, hospital financing, and hospital location, presented in cubic spline curves and with β coefficients generated in mixed-effects models.

Role of the funding source

The funders had no role in study design or writing of this report. The views expressed are those of the authors and not necessarily those of the National Health Service, the NIHR or the UK Department of Health and Social Care.

Results

In phase 1, the international consultation to develop the SPI indicator set involved 69 members (23 [33%] women; 46 [67%] men; 41 from HICs, 22 from MICs, and six from LICs) from 32 countries. This included front-line surgeons, anaesthetists, and critical-care doctors from the COVIDSurg and NIHR Global Health Unit on Global Surgery collaborative networks. Of 110 longlisted candidate indicators, the final index included 23 indicators across four consensus domains: facilities and consumables, staffing, prioritisation, and systems (panel ). Detailed descriptions to support hospital assessment are provided in the appendix (pp 1–3). Each indicator was scored using a Likert scale between 1 (very weak) and 5 (very strong). The scores across all 23 indicators were summed to give a total SPI score for a hospital with a range between 23 (least prepared) and 115 (most prepared). A summary of the Delphi voting rounds is presented in figure 1 and full results are reported in the appendix (pp 4–5). All eight independent raters considered the 23 indicators to have high (20 indicators) or moderate (three indicators) relevance to maintaining volume of planned surgery across all five examples of external shocks (figure 2 ) with high agreement between raters (ICC 0·76 [95% CI 0·59–0·89]).

Panel. Summary of surgical preparedness index.

Hospitals were assessed for each indicator by assessors, scored from 1 (very weak) to 5 (very strong) with an overall summary surgical preparedness index score calculated between 23 and 115. A full description of each indicator to support hospital assessment is provided in the appendix (pp 1–2).

Facilities and consumables

• 1.Availability of reserved planned surgery theatres (ring-fenced theatres)
• 2.Availability of reserved planned surgery beds (ring-fenced beds)
• 3.Availability of reserved critical care beds for planned surgery (ring-fenced critical care)
• 4.Flexibility to rearrange hospital areas to provide a segregated pathway for planned surgery (flexible areas)
• 5.Access to diagnostics and interventions to identify and treat surgical complications (managing complications)
• 6.Reliable supply of electricity (electricity supply)
• 7.Reliable supply of supplementary oxygen (oxygen supply)
• 8.Reliable supply and management of essential perioperative drugs (drug supply)
• 9.Reliable supply and management of devices and implants (device supply)
• 10.Sufficient surgical instrument and local sterilisation processes (sterilisation)
• 11.Availability of protective measures for theatre teams (protective equipment)

Staffing

• 12.Ability to redistribute staff within and between hospitals to maintain capacity (staff redistribution)
• 13.Availability of reserved teams to provide planned surgical care (ring-fenced teams)

Prioritisation

• 14.Cross-specialty patient prioritisation for surgery (patient prioritisation)
• 15.Ability to identify and cancel procedures of limited clinical value (procedure prioritisation)

Systems

• 16.Formal operational plan to continue planned surgery during external system shocks (formal plan)
• 17.Ability to do preoperative assessment in the community (preoperative assessment)
• 18.Access to routine preoperative testing for endemic and epidemic diseases (preoperative testing)
• 19.Ability to transfer patients to another hospital with greater capacity (hospital transfer)
• 20.Ability to facilitate timely discharges (timely discharge)
• 21.Social support system to facilitate safe discharge (social support)
• 22.Capacity to use telephone or video calls for outpatient appointments (remote outpatient appointments)
• 23.Capacity and capability to communicate with family members (family communication)

Figure 2.

Relevance of the surgical preparedness index to different external shocks

Independent development group members were asked to rate the relevance of each surgical preparedness indicator following five different external health-care system shocks in their local context.

In phase 2, 5375 hospital-level assessments were completed, of which 503 did not have an identifiable hospital or country or both, 118 did not complete assessment of all indicators, and 40 did not calculate an SVR. Across included facilities, the level of missingness was less than 5% for all indicators; we did a preplanned complete case analysis without imputation. 4714 complete assessments from 1632 hospitals in 119 countries, including 887 (54%) hospitals in 52 (44%) HICs, 675 (41%) hospitals in 56 (47%) MICs, and 70 (4%) hospitals in 11 (9%) LICs, were eligible for analysis in phase 2 and 3.

A summary of included hospitals both overall and by World Bank income group and the number of hospitals and assessments by country are reported in the appendix (pp 6–7). 1217 (74·6%) of 1632 hospitals assessed were public (government) funded, 196 (12·0%) were private hospitals, and 219 (13·4%) were mixed public and private. 1570 (96·2%) hospitals delivered both planned and unplanned surgery. Hospitals in urban, rural, and mixed settings, with a wide range of hospital bed numbers were included in the assessment. The median number of hospitals assessed per country was 6·0 (IQR 2·0–14·5). There was a median of 2·0 (1·0–3·0) assessments per hospital, and 764 (46·8%) hospitals had more than one assessment. In hospitals in which more than one assessment was completed, inter-rater reliability of the SPI was moderate (ICC 0·55 [95% CI 0·53–0·57]). A summary of features of hospital assessors overall and by World Bank income group are reported in the appendix (p 9). The hospital assessors were most commonly surgeons (2845 [60·4%] of 4714 assessors), although assessments were completed by a range of professionals from across all surgical disciplines. The mean SPI scores per hospital was 84·5 (95% CI 84·1–84·9) out of 115, and global distribution of the SPI are reported in figure 3 . Hospital scores ranged from 26 to 115. There was variation in the mean SPI across subgroups: HICs (88·5 [95% CI 89·0–88·0]), MICs (81·8 [82·5–81·1]), and LICs (66·8 [64·9–68·7]); moderate (81·1 [80·4–81·8) and high (87·1 [86·6–87·6) COVID-19 burden areas; public (83·0 [82·5–83·5]) and private or mixed (89·8 [88·9–90·7]) hospitals; and urban (86·1 [85·4–86·8]), rural (77·4 [74·2–80·6]), and mixed (83·7 [83·1–84·3]) settings (appendix p 11).

Figure 3.

Geographical distribution of SPI score

(A) Distribution displayed is centred around the mean value of SPI total score (84·5). Green indicates better prepared surgical systems; red indicates less prepared surgical systems. (B) Distribution of the SPI by country income group. The theoretical score range limits of the SPI were 23–115 points. The lowest mean hospital score was 26 and the highest was 115. These values are displayed at the floor and ceiling values of the x-axis. SPI=surgical preparedness index.

The mean scores (out of five) for each individual indicator, presented overall and by World Bank income group are reported in figure 4 . The highest scored indicators were electricity supply (4·38 [95% CI 4·34–4·41]), oxygen supply (4·33 [4·29–4·36]), and perioperative drugs (4·17 [4·34–4·41]). The lowest scored indicators overall were ring-fenced critical care beds (3·11 [3·07–3·17]), remote outpatient appointments (3·26 [3·21–3·32]), and formal operational plan (3·28 [3·23–3·32]). The biggest differences by indicator were seen in device supply (standardised mean difference between HICs and LICs was –1·80 points), remote outpatients (–1·63), and drug supply (–1·62).

Figure 4.

Mean ratings of hospitals across surgical preparedness indicators

Scores are a mean following ratings from 1632 participants. Indicators are ordered from highest to lowest mean score (out of 5) overall by indicator.

In the country-level analysis, greater surgical preparedness was associated with higher levels of human development, health security, and UHC, and lower levels of wealth inequality (appendix p 12). A suggested framework for assessment of the SPI and targeted, local systems strengthening initiatives is reported in the appendix (p 13) , and an online application to support longitudinal evaluation.

In phase 3, at the time of assessment, 1217 (74·6%) of 1632 hospitals had an SVR of less than 1, suggesting that they were unable to maintain usual planned surgical volume during COVID-19. Of these 625 (51·4%) hospitals were from HICs, 538 (44·2%) from MICs, and 54 (4·4%) from LICs (appendix pp 14–15). The mean SVR was 79·3% (95% CI 78·1–80·4). This varied significantly by hospital, ranging from 0·0% (doing no planned surgery) to 200·0% (doing twice as many planned surgeries than the pre-pandemic baseline). A histogram of SVR across World Bank income groups is reported in the appendix (p 14). The proposed causal model is reported in the appendix (p 16), and figure 5 shows the SPI score against the SVR, overall and across key subgroups. A linear association was observed between SPI score and SVR with a ten-point total SPI score increase associated with a 3·6% (95% CI 3·0 to 4·1; p<0·0001) increase in SVR in the mixed-effects model. Hospitals in MICs (–8·37% [95% CI –8·45 to –8·29]; p<0·0001) and LICs (–10·56% [–14·89 to –6·2]; p<0·0001) versus HICs were associated with a reduced SVR. Private (3·01% [0·12 to 5·91]; p<0·0001) and mixed public and private (3·20% [1·02 to 5·37]; p=0·0002) hospitals were both associated with increased SVR versus public hospitals. No significant associations between hospital location (urban vs rural or mixed) and SVR were observed (appendix p 10). On subgroup analysis, association between SPI score and SVR was observed in HICs (4·8% [4·1 to 5·5]; p<0·0001), MICs (2·8% [2·0 to 3·7]; p<0·0001), and LICs (3·8 [1·3 to 6·7]; p<0·0001); moderate (3·5 [2·7 to 4·2%]; p<0·0001) and high (4·1 [3·3 to 4·8]; p<0·0001) COVID-19 burden areas; public (3·6% [3·0 to 4·2]; p<0·0001) and private hospitals (4·1% [3·1 to 5·2]; p<0·0001); and in urban (4·2% [3·3 to 5·1]; p<0·0001), rural (4·9 [1·6 to 8·2]; p=0·0046), and mixed locations (3·4 [2·7 to 4·1]; p<0·0001).

Figure 5.

Association between SPI scores and hospitals' planned surgical volume ratio

Association between SPI scores and hospitals' planned surgical volume ratio overall (A) and by country income status (B), COVID-19 burden (C), hospital funding mechanism (D), number of hospital beds (E), and hospital location (F). The planned surgical volume ratio was calculated as the ratio of each hospital's observed planned surgical volume over a 1-month assessment period against the expected planned surgical volume based on data from the same month in 2019 (the prepandemic baseline) and expressed as a percentage. Shaded areas are 95% CIs. SPI=surgical preparedness index. SVR=surgical volume ratio.

Discussion

We have developed, measured, and validated a hospital-level SPI to support strengthening of elective surgery systems against external shocks. The SPI showed variability at subnational and hospital levels, identifying areas that can improve to create resilience in local surgical systems. Using COVID-19 as an example, a 10-point increase in the SPI was associated with a 3·6% increase in the planned surgical volume ratio. This relationship was robust across income settings, hospital types, and COVID-19 burdens. Hospitals with private versus public financing and in HICs were able to maintain a higher SVR than those in MICs or LICs, indicating the importance of hospital resourcing as a mediator of planned surgical throughput. Our findings suggest that the under-resourced surgical systems, identified as at risk by the Lancet Commission on global surgery,²³ will also be at greatest risk of secondary effects and delayed recovery from COVID-19. Routine SPI assessment might help to identify actionable targets for local policy, advocacy, and investment in surgery and anaesthesia service strengthening that complement existing frameworks for global health security.24, 25 Focused efforts to address surgical preparedness will be essential in addressing growing backlogs and mitigating against harm for patients awaiting surgery.

The 23 surgical preparedness indicators are easy to measure without additional resources, with moderate ICC values, and they allow local hospital teams to identify targets that are relevant to them and are actionable. There was significant variability in performance across indicators and across resource settings. For example, ring-fenced critical care beds was rated as being challenging in HIC, MIC, and LIC settings, suggesting a challenge that might be hard to surmount. Conversely, device supply, drug supply, and remote outpatients appointments were scored lower in lower-income settings, perhaps highlighting important areas for advocacy and service investment. Public hospitals and those in rural settings had lower SPI scores, highlighting vulnerable hospital types that warrant future focus.²⁶ The finding that better resourced surgical services were more resilient to system stress during SARS-CoV-2, with a more rapid recovery, aligns with other research in this area.²⁷ Country-level analysis showed consistency of the SPI with other measures of health system resilience, such as the Global Health Security index, and strong correlation with UHC service coverage and other measures of wealth equality and development. However, the SPI has strong clinical use beyond these population-level measures by allowing hospital benchmarking and highlighting areas for targeted action.

Other indices exist to address both health system preparedness and surgical capacity separately, but they do not combine the immediate need to focus on surgery at a subnational or hospital level and preparedness for external shocks.28, 29 In a review of whole-health system preparedness indicators, no index was meaningfully associated with clinical outcomes, and no surgery-specific indices were found.¹¹ Other frameworks exist to evaluate surgical capacity in isolation (the PIPES checklist,¹² WHO Situational Assessment tool,¹³ and Ethiopian Hospital Assessment tool³⁰). However, these are not designed to dynamically assess preparedness (ie, the response of services to external system pressure). They are also complex, long, and not feasible for regular application.

Our index was validated in the context of the SARS-CoV-2 pandemic, but it is likely to be generalisable beyond this setting. In a consensus exercise, independent international raters considered the indicators all to have high or moderate relevance across five example scenarios. However, the index might not have full content validity across every external shock recorded. For example, in the case of seasonal pressures (heatwaves or winter pressures) adequate temperature control in clinical areas through air conditioning or heating might be considered an important additional indicator. In addition, the relative importance of SPI indicators might change from system stressor to stressor and from country to country; for example, during the COVID-19 recovery period, staff shortages might be the primary limiting factor for delivery of planned surgery. This has been compounded over time due to burnout and staff sickness. Therefore, we consider the SPI to be a minimum core indicator set to underpin elective surgical system preparedness with relevance across a variety of scenarios. However, the SPI requires validation and a potential need for adaptation for other external stressors exists, highlighting an important area for ongoing research.

Surgery has been neglected from planning for pandemic recovery, despite being a core component of functioning health-care systems.23, 31 The SPI presents a consensus response by the international community to tackle the issue of neglect of surgery from planning. However, our study has limitations. First, this cross-sectional assessment of preparedness does not account for hospitals at different stages of the readiness–response–recovery cycle.³² Second, the index does not inform the net benefit of restoring surgery versus other hospital activities; however, surgical service preparedness is an essential component of holistic health system resilience: it strengthens other health-care processes (eg, readiness to provide oxygen)³³ and transparent prioritisation will be key in competition for restricted resources.³⁴ Third, 868 (53%) of the 1632 hospitals had only one assessor. Differences in preparedness might exist between specialties or operating theatres that are not reflected here, and we were unable to assess variability between subgroups of assessors in more details. Fourth, there was some imbalance in representation between HICs, MICs, and LICs in both indicator development and the cross-sectional assessment. However, data were collected from a large sample of 744 (46·6%) hospitals in MICs and LICs indicating generalisability. Fifth, data suggest that some hospitals in countries with lower COVID-19 burden at the time of assessment (eg, Australia and China) had a lower SPI score, but still were able to maintain their planned surgical volume. To address confounding due to COVID-19 pressures, we adjusted for this in modelling and did a subgroup analysis in high and moderate COVID-19 burden areas; no countries were classified as low COVID-19 burden. Sixth, barriers and facilitators to implementation of the SPI framework are not yet fully understood; we include an online assessment tool to support future implementation and evaluation. Seventh, our surrogate measure of COVID-19 burden (OxCGRT) did not account for government mandates to pause planned surgical care, which might have led to unmeasured confounding. Eighth, we included several plausible confounders in mixed-effects modelling supported by causal relationships presented in a directed acyclic graph. However, there might be residual unmeasured confounding or measurement error that has been unaccounted for. Ninth, our primary outcome measure was data driven and pragmatic, but represented a single cross-sectional assessment, calculated as a relative measure of surgical volume. This should be considered when applying the index to national health policy in which absolute measures of outcomes and effects might be important. Tenth, interpretation of composite measures, such as the SPI, is challenging, and we have only explored the relationship between the total index score and SVR. We are unable to judge the relative importance of individual indicators, which might vary across specialties, hospitals, and resource contexts. Future exploration of the scaling and measurement properties of the index and differential functioning of the indicators is required during future development. Future iterations of the SPI could consider normalising or transforming the scale to make its minimum and maximum values more intuitive (eg, 0–100), but this should be balanced with clinical use and interpretability of the output for local assessors. Eleventh, unplanned surgery is an important component of surgical systems, especially in LMICs in which a larger proportion of patients present to care services requiring emergency or immediate care.³⁵ The SPI has been developed and validated in planned surgery only and is not designed to be applied in emergency systems. Finally, we have not considered the safety and efficacy of the surgeries. There are likely to be differences across resource settings that might be exacerbated by the whole-health system effect of COVID-19 and should be considered in future work.3, 35

As the recovery following the COVID-19 pandemic continues to gather pace there is a need for urgent and regular (eg, annual) hospital self-assessment using the SPI. When possible, this assessment should be integrated with existing quality and safety programmes and national surgical, obstetric, and anaesthesia planning. SPI implementation alongside national surgical, obstetric, and anaesthesia planning will add resilience to national capacity building that is aleady under way. Improving preparedness is likely to strengthen planned surgical services against future external shocks and support upscaling of surgery to address growing demands. Therefore, the SPI supports a major priority area for WHO for ongoing progress towards Sustainable Development Goal 3: Health and Wellbeing.36, 37, 38 COVID-19 is just one form of external shock that puts additional pressure on planned surgical care pathways. Other epidemics, such as influenzas and Ebola virus, have had significant effects on surgical services over the past decade.³⁹ Natural phenomena associated with climate change, stresses from geopolitical instability, and conflict have also already had a substantial effect and continue to pose a substantial future threat to surgical system functioning. Surgical preparedness is a core part of the response to these stressors in minimising suffering and loss of life.40, 41 Although the SPI has been developed during the COVID-19 pandemic, it has been specifically designed to be applicable to any context of health system pressure. Other context-specific modifications (eg, to incorporate sustainable measures for climate resilience) might become necessary as use of the index expands into global surgical practice.⁴² Best processes for implementation of the SPI for longitudinal assessment of a hospital's preparedness is an urgent research area for ongoing development.

Data sharing

Anonymised data are available upon request from the writing group, and successful completion of a data sharing agreement through an Application Programming Interface linked to the REDCap data server hosted at Birmingham Clinical Trials Unit, University of Birmingham, Birmingham, UK. Summary data and a self-assessment tool are available online.

Declaration of interests

RP has received research grants or consultancy fees or both from Edwards Lifesciences, Intersurgical, and GlaxoSmithKline. JM has consulted for WHO on projects related to perioperative preparedness. TA has received consultancy fees from MSD unrelated to this work. All other members of the writing group declare no competing interests.