Early recognition and response to increases in surgical site infections using optimised statistical process control charts—The early 2RIS trial: A multicentre stepped wedge cluster randomised controlled trial

Summary

Background

Traditional approaches for surgical site infection (SSI) surveillance have deficiencies that delay detection of SSI outbreaks and other clinically important increases in SSI rates. We investigated whether use of optimised statistical process control (SPC) methods and feedback for SSI surveillance would decrease rates of SSI in a network of US community hospitals.

Methods

We conducted a stepped wedge cluster randomised trial of patients who underwent any of 13 types of common surgical procedures across 29 community hospitals in the Southeastern United States. We divided the 13 procedures into six clusters; a cluster of procedures at a single hospital was the unit of randomisation and analysis. In total, 105 clusters were randomised to 12 groups of 8–10 clusters. All participating clusters began the trial in a 12-month baseline period of control or “traditional” SSI surveillance, including prospective analysis of SSI rates and consultative support for SSI outbreaks and investigations. Thereafter, a group of clusters transitioned from control to intervention surveillance every three months until all clusters received the intervention. Electronic randomisation by the study statistician determined the sequence by which clusters crossed over from control to intervention surveillance. The intervention was the addition of weekly application of optimised SPC methods and feedback to existing traditional SSI surveillance methods. Epidemiologists were blinded to hospital identity and randomisation status while adjudicating SPC signals of increased SSI rates, but blinding was not possible during SSI investigations. The primary outcome was the overall SSI prevalence rate (PR=SSIs/100 procedures), evaluated via generalised estimating equations with a Poisson regression model. Secondary outcomes compared traditional and optimised SPC signals that identified SSI rate increases, including the number of formal SSI investigations generated and deficiencies identified in best practices for SSI prevention. This trial was registered at ClinicalTrials.gov, NCT03075813.

Findings

Between Mar 1, 2016, and Feb 29, 2020, 204,233 unique patients underwent 237,704 surgical procedures. 148,365 procedures received traditional SSI surveillance and feedback alone, and 89,339 procedures additionally received the intervention of optimised SPC surveillance. The primary outcome of SSI was assessed for all procedures performed within participating clusters. SSIs occurred after 1171 procedures assigned control surveillance (prevalence rate [PR] 0.79 per 100 procedures), compared to 781 procedures that received the intervention (PR 0·87 per 100 procedures; model-based PR ratio 1.10, 95% CI 0.94–1.30, p=0.25). Traditional surveillance generated 24 formal SSI investigations that identified 120 SSIs with deficiencies in two or more perioperative best practices for SSI prevention. In comparison, optimised SPC surveillance generated 74 formal investigations that identified 458 SSIs with multiple best practice deficiencies.

Interpretation

The addition of optimised SPC methods and feedback to traditional methods for SSI surveillance led to greater detection of important SSI rate increases and best practice deficiencies but did not decrease SSI rates. Additional research is needed to determine how to best utilise SPC methods and feedback to improve adherence to SSI quality measures and prevent SSIs.

Funding

Agency for Healthcare Research and Quality.

Research in context.

Evidence before this study

Traditional methods for surgical site infection (SSI) surveillance at acute care hospitals have major deficiencies that can lead to delayed or failed detection of important SSI rate increases. Statistical process control (SPC) methods have potential to improve surveillance of healthcare processes and outcomes such as SSIs and decrease rates of healthcare-associated infections (HAIs). We searched PubMed, ClinicalTrials.gov, and the ISRCTN registry without language restriction using the terms (“statistical process control” OR “control charts”) AND “trial” for peer-reviewed studies published before March 1, 2022. Many groups have used SPC methods to monitor the success of healthcare quality improvement interventions, often via quasi-experimental and other uncontrolled studies. However, we found only seven randomised trials that utilised SPC, and only two of these trials analysed the effect of feedback from control charts on HAI rates or surgical outcomes. The first study showed that use of SPC decreased rates of ward-acquired Staphylococcus aureus colonisation or infection among a cohort of 24 hospitals in the United Kingdom, but this decrease was not statistically significant compared to the decrease reported on control wards. The second study was performed at 40 hospitals in France and demonstrated that the implementation of control charts with feedback to surgical teams was associated with a decrease in major adverse events after gastrointestinal tract surgery. The investigators did not report the impact of the intervention on SSI rates as an individual outcome.

Added value of this study

Our study was the first randomised trial designed to assess the impact of SPC methods on SSI surveillance and SSI rates. In a stepped-wedge cluster randomised trial, we analysed 237,704 surgical procedures performed over four years within a large network of community hospitals in the Southeastern United States. The addition of optimised SPC methods to traditional SSI surveillance methods was associated with markedly greater detection of clinically important SSI rate increases and deficiencies in best practices for SSI prevention; however, improved surveillance did not lead to a decrease in SSI rates. These findings emphasise that enhanced SSI surveillance must be coupled with effective feedback of increased SSI rates to surgical personnel and implementation of perioperative practice changes to decrease risk of SSI.

Implications of all the available evidence

Our trial adds to the findings of smaller, retrospective studies that showed potential for optimised SPC surveillance to outperform traditional surveillance methods in detecting important increases in SSI rates. Furthermore, our study demonstrates the feasibility and value of prospective use of SPC surveillance for a broad range of surgical procedures performed across a large network of hospitals. Finally, our trial illustrates the potential for disconnect between performance of SSI surveillance and SSI outcomes. The lack of improvement in SSI rates in our study indicates that further research is needed to determine how to best utilise SPC methods to improve adherence to SSI quality measures and prevent SSIs.

Alt-text: Unlabelled box

Introduction

Surgical site infections (SSIs) are among the most common and costly healthcare-associated infections (HAIs) in the world.1, 2, 3 In the United States alone, over 150,000 SSIs occur each year, accounting for over $3 billion in annual healthcare expenditures.4, 5, 6 Patients in low- and middle-income countries experience even greater SSI burdens due to increased risk of SSI and the impact of associated expenditures.^7,8 In addition to increased costs, patients with SSIs incur longer hospitalisations and increased mortality compared to patients who do not develop SSIs.^4,9 As a result, hospitals devote considerable resources to SSI prevention.¹⁰ However, while hospitals have improved compliance with core process measures designed to prevent SSI, these improvements have not reliably decreased SSI rates.^11,12 Therefore, development of new strategies to prevent SSIs represents an important unmet need.

One traditional intervention found to decrease SSI risk involves the coupling of SSI surveillance and associated feedback to surgical personnel.13, 14, 15 However, the traditional approach for SSI surveillance has major deficiencies that can lead to delayed or failed detection of important SSI rate increases or outbreaks, precluding timely feedback and investigation. For example, traditional statistical techniques require aggregation of SSI data over time, delaying analysis until sufficient data accumulate (e.g., each year).^16,17 Also, comparisons to static external benchmarks, such as past SSI rates published in national or international surveys18, 19, 20 may not detect important changes in SSI rates experienced by individual hospitals.²¹ Therefore, infection prevention personnel often initiate SSI outbreak investigations months after SSI rates first increase.²²

Statistical process control (SPC) is an analytic approach that has successfully been used to decrease HAI rates^23,24 and addresses important shortcomings of traditional SSI surveillance.²¹ SPC combines time series analysis methods and graphical presentation of data to help determine in near real time whether data exhibit natural variation within probabilistic thresholds or unnatural variation representing statistically significant changes.^16,25 We previously performed a large-scale empirical optimisation study in a large network of US community hospitals that identified simultaneous use of two moving average (MA) SPC charts to be the most effective SPC strategy in this setting for detection of important SSI rate increases.^21,26 When applied retrospectively to 30 previously investigated SSI outbreaks, this chart combination identified each outbreak, typically several months earlier than traditional surveillance.²² We therefore hypothesised that prospective use of the same optimised SPC approach in addition to traditional SSI surveillance would decrease rates of SSI compared to use of traditional surveillance methods alone.

We report a stepped wedge cluster randomised trial (SW-CRT) designed to assess the impact of SSI surveillance with optimised SPC methods and feedback on SSI rates.

Methods

Study design and participants

We performed a stepped wedge cluster randomised controlled intervention trial within the Duke Infection Control Outreach Network (DICON), a community hospital network in the Southeastern United States.²⁷

We evaluated all 32 acute care DICON hospitals that had been members of DICON for at least two years and performed any of 13 types of targeted surgical procedures for potential study participation. We divided the 13 targeted surgical procedures into six types of clusters that included procedures typically performed by the same surgical specialists, including cardiothoracic, gastrointestinal, joint, obstetrics and gynecology, spine, and vascular clusters (appendix p 3). Clusters were eligible for study inclusion if they included at least 100 total procedures and three SSIs over the three calendar years prior to randomisation. All procedures performed within participating clusters were included in the study. Letters of support were required from participating hospitals prior to randomisation.

A cluster of procedures at a single hospital was the unit of randomisation and analysis, and 105 total clusters across 29 hospitals were randomised to 12 groups of 8–10 clusters. The order of the initiation of the SPC surveillance intervention was randomised and serially implemented until all 12 groups of clusters had received the intervention. Per SW-CRT design,^28,29 after a baseline period of 12 months, one group of clusters sequentially crossed over from control to intervention during each of 12 steps that occurred every three months. The study ended after the 12^th and final three-month active study period was complete (Figure 1). Clusters that were randomised to the intervention received feedback from optimised SPC surveillance and traditional surveillance methods, whereas clusters randomised to the control arm received feedback from traditional surveillance methods alone.

Figure 1.

Schematic for stepped wedge design.

Cluster randomised design was chosen to ensure that procedures associated with the same surgeons, operating room personnel, and SSI prevention interventions were grouped together to limit contamination between intervention and control clusters within the same hospital. Stepped wedge design was utilised to achieve target statistical power, which was not possible with parallel design for the number of hospitals and clusters available in the network.³⁰ We implemented the stepped wedge unidirectional crossover design because crossover from intervention to control would have required a prolonged washout period to reduce contamination from the intervention after crossover.³¹

No study interventions were performed on individual patients, and patients undergoing surgery at participating hospitals were not consented. Institutional review boards at Duke University Health System and Northeastern University determined this trial to be exempt research. The trial protocol was summarised in a peer-reviewed report,³² and the full protocol is available online (https://dcasip.medicine.duke.edu/research/clinical-trial-protocols).

Randomisation and masking

The primary investigator (DA) recruited hospitals. The statistician (YL) performed the randomisation and allocation sequence using computer-generated random numbers. Randomisation was designed to allow a maximum of one cluster per hospital to cross over to the intervention at each step. Study epidemiologists (DA and AB) were blinded to hospital identity and cluster randomisation status during weekly adjudication of signals generated by SPC surveillance. All other study and local hospital personnel except for the study coordinator (KF) were blinded to cluster randomisation status. The study coordinator (KF) assigned clusters to interventions per the randomisation scheme.

Procedures

During the baseline and active study periods, regardless of randomisation status, all clusters of procedures continued to receive traditional SSI surveillance routinely performed in DICON. Traditional surveillance included prospective data entry into the DICON Surgical Database for each surgery performed. This limited database contained 12 variables for all surgical procedures and five additional variables for procedures complicated by SSI (appendix p 4). Experienced DICON infection preventionists validated data and adjudicated SSIs based upon NHSN criteria.³³ This database was updated at least weekly at participating hospitals. DICON also sent biannual SSI reports to study hospitals that included SSI data analyses for each procedure performed, including SSI rate comparisons to similarly sized DICON hospitals. In addition, study hospitals detected potential SSI rate increases or outbreaks when concerns were reported by infection preventionists, hospital epidemiologists, surgeons, or other local hospital personnel. SSI concerns for a specific hospital and procedure identified by traditional surveillance were defined as “traditional signals.” One of three infectious diseases epidemiologists adjudicated each traditional signal by determining whether the signal and associated SSIs required “action” or “no action.” This decision was based on the number, timing, and type of SSIs; SSI rates; pathogens responsible for SSIs; and number of surgeons involved.

Throughout all 12 active study periods, clusters of procedures randomised to the intervention were monitored by traditional surveillance and the addition of optimised SPC surveillance. Optimised SPC surveillance involved use of two previously validated moving average (MA) standardised p-type charts.²⁶ The first chart used an external baseline calculated from network-wide SSI data on the procedure undergoing surveillance, and the second chart used a baseline calculated from procedure-specific data from the single hospital being analysed. Both charts utilised rolling baseline windows, control limits of ±1 standard deviation, and other chart characteristics as previously described.²² SSI data for each procedure under surveillance were plotted with monthly resolution. Any data point above the rolling upper control limit on either chart indicated detection of a potentially important SSI rate increase and was considered an “SPC signal,” which required evaluation (appendix p 17).

The optimised SPC methods were applied weekly to Surgical Database SSI data on all clusters of procedures throughout all 12 active study periods, regardless of randomisation status. The research coordinator assigned SPC signals for adjudication that were associated with new SSIs reported after the prior week's SPC analysis. Other signals that were not associated with new SSIs were excluded from adjudication. The coordinator then blinded hospital identification and sent signals requiring adjudication to study epidemiologists. One of two epidemiologists adjudicated SPC signals each week by reviewing associated SSI data and deciding whether signals required “action” or “no action,” using the same criteria employed to adjudicate traditional signals. Interrater reliability was assessed with a kappa statistic. For signals randomised to intervention and requiring action, a study epidemiologist was unblinded to the signal hospital to facilitate initial investigation. For signals randomised to control surveillance, the coordinator recorded adjudications but did not unblind epidemiologists nor take further action (appendix p 18).

In response to traditional signals or SPC signals randomised to intervention that were adjudicated to receive action, DICON and local hospital personnel evaluated signals via the same standardised approach (appendix p 18). DICON epidemiologists first reviewed detailed Surgical Database variables on all recent SSIs associated with the hospital/procedure (appendix p 4), both SPC charts generated by the hospital/procedure signal month (appendix p 17), SSI rates for the hospital and procedure responsible for the signal, surgeon-specific SSI rates, and DICON benchmark SSI rates for the same procedure. If further evaluation was deemed necessary after reviewing these additional data, the epidemiologist discussed the signal data with the appropriate DICON and local hospital infection preventionists. If indicated based on these discussions, DICON epidemiologists and infection preventionists launched formal SSI investigations. SSI investigations included multiple steps, such as analysis of detailed SSI line listings, meetings with surgical personnel, operating room observations, written investigation summaries and recommendations, and support for implementation of recommended interventions (appendix p 18).

SSIs were prospectively defined using standard NHSN definitions.³³ Complex SSIs included SSIs that were either deep-incisional or organ/space SSIs. SSIs that occurred at the same site and depth as prior infections documented to be present at time of surgery were excluded from SSI rate calculations and models. An important increase in SSI rate for a hospital/procedure was defined as presence of either a corresponding traditional or SPC signal that was adjudicated by a study epidemiologist to receive further evaluation. The signal generated a formal SSI investigation if additional investigation was pursued after discussion between a study epidemiologist and the DICON and local hospital infection preventionists. Preventability score for each SSI analysed with line listing was defined as the percentage of ten core SSI prevention perioperative best practices recommended by consensus guideline committees^1,2,4,10 that were not appropriately completed.³⁴ Best practices included four aspects of perioperative antimicrobial prophylaxis;³⁵ skin antisepsis; maintenance of normothermia; use of supplemental oxygen in patients undergoing general anaesthesia and tracheal intubation; glycemic control; use of a perioperative checklist; and, for colon surgery, use of both preoperative oral antibiotics and mechanical bowel preparation (appendix p 5).

Outcomes

The primary endpoint was prespecified as the overall SSI prevalence rate (PR), calculated as the number of SSIs per 100 procedures. SSI PR stratified by type of SSI (superficial-incisional or complex) was a secondary outcome. Other prespecified secondary outcomes compared traditional and optimised SPC signals, including number of signals and signal rate; proportion of signals adjudicated to receive further evaluation; number of formal SSI investigations generated; time from signal identification to completion of SSI investigations; timing of true positive signals; and proportion of SSIs deemed to be potentially preventable, based on deficiencies in SSI prevention best practices and preventability scores.

Statistical analysis

The sample size calculation utilised up to three years of SSI data (2011–2013) from 101 clusters of surgical procedures performed at the 29 DICON hospitals that participated in the study. These pilot data included 1622 SSIs that occurred after 154,554 total procedures (overall PR 1.05 per 100 procedures; average PR per cluster 1.33 per 100 procedures). Power was estimated via a simulation study where for each cluster, log (SSI PR) was generated from a multivariate normal distribution with four assumptions: 1) cluster-specific SSI PRs during control surveillance periods are equal to pilot data SSI PRs; 2) residual variance for log (SSI PR) of 0.76; 3) within-cluster variation of 0.36; and 4) between-cluster correlation of 0.39 in the same time period and 0.20 in different periods. Based on these parameters obtained from the pilot data, a study with 101 clusters in 29 hospitals, 12 periods, and an average of 127 procedures per cluster in each three-month period would have 90% power to detect a 25% decrease in the SSI PR for clusters randomised to SPC surveillance and feedback.

The primary outcome was analysed using generalised estimating equations with a Poisson regression model. Cluster was the unit of analysis, and the model included effects for intervention status and for each time period using indicator variables. To account for potential residual confounding, the model was also adjusted for hospital and for the cluster-level summaries of the three components of the NHSN risk index: median wound class, median American Society of Anesthesiologists Physical Status Classification System score, and mean operative duration.¹⁸ The model utilised data from all time periods, including the baseline period, which was considered to contain four three-month periods. To account for within-cluster correlation over time, exchangeable working correlation structure was used. We additionally considered negative binomial and overdispersed Poisson distributions, as well as independent and AR(1) working correlation structures; the final model was selected based on the quasi-information criterion and the mean squared error. The same approach was used to analyse the secondary outcomes of SSI PRs stratified by type of SSI, except independent working correlation structure was chosen for analysis of superficial-incisional SSIs. Finally, we analysed the effect of the intervention on the primary outcome by cluster type by including the interaction between the intervention status and cluster type in the models.

Detection and timing of traditional signals that were adjudicated as true positive signals requiring further action was compared to SPC signal detection for the same hospital/procedure combination. SPC signals that occurred within 12 months before or after true positive traditional signals were considered to have detected the same important SSI rate increase.²² Procedure months of surveillance were calculated by multiplying the sum of the number of procedure types under surveillance at each hospital by the duration of surveillance in months. For calculations of rates of SSI investigations, procedure months were subtracted for hospital/procedure combinations that were receiving active investigation and not capable of generating new SSI investigations. The remaining secondary outcomes were analysed with summary statistics.

Surgical data for all study procedures were maintained in the DICON Surgical Database, which was stored in Microsoft SQL Server 2014 (Microsoft, Redmond, WA). The dual SPC charts used in this study were applied to SSI data in the Surgical Database Data using MATLAB version R2017a (MathWorks, Natick, MA, USA). Data related to signal evaluation and SSI investigations were collected and managed using REDCap (Research Electronic Data Capture) version 10.0.30, hosted at Duke University.³⁶ Analysis was performed in SAS version 9.4 (SAS Institute Inc., Cary, NC, USA). A data monitoring committee was not used. The trial was registered at ClinicalTrials.gov (NCT03075813).

Role of the funding source

The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.

Results

Among 29 participating hospitals, all 105 clusters that met inclusion criteria were randomly assigned to one of 12 dates to implement the SPC SSI surveillance intervention. The 12-month baseline period included data collected from Mar 1, 2016, to Feb 28, 2017, and the 36 months of active study periods spanned from Mar 1, 2017, to Feb 29, 2020. SSI rate data from 6 randomised clusters were completely excluded from the analysis because SSI reporting ceased for procedures within these clusters prior to the start of the active study. Additionally, SSI rate data from 7 clusters were excluded at timepoints during active study periods when SSI reporting of numerators and denominators for associated procedures was terminated. Between Mar 1, 2016, and Feb 29, 2020, 204,233 unique patients underwent 237,704 surgical procedures at participating clusters. In total, 148,365 procedures were performed in clusters receiving traditional surveillance and feedback alone, and 89,339 procedures were performed in clusters randomised to additionally receive the intervention of optimised SPC for SSI surveillance (Figure 2, appendix p 6 and p 19).

Figure 2.

Trial profile.

SSI = surgical site infection.

Patients in both study arms had similar age and sex distributions. Distribution of the 13 procedure types, NHSN risk index, and risk index components were also similar across both groups. For the overall cohort, herniorrhaphy (n=41,840; 17.6%), knee prosthesis surgery (n=40,361; 17.0%), and Cesarean section (n=32,873; 13.8%) were performed most frequently (Table 1).

Table 1.

Baseline characteristics of patients and procedures.

	Control surveillance (n=148,365 procedures)		Intervention surveillance (n=89,339 procedures)
Number of unique patients	129,997	(88%)	74,236	(83%)
Age, yearsa	55.9	(17.8)	56.5	(17.7)
Sex
Female	81,139/129,997	(6.4%)	42,388/74,236	(57.1%)
Male	48,830/129,997	(37.6%)	31,843/74,236	(42.9%)
Missing	28/129,997	(<0.1%)	5/74,236	(<0.1%)
Number of procedures per cluster	850	(402–1666)	521	(240–1574)
Procedure type
Abdominal hysterectomy	8953	(6.0%)	4836	(5.4%)
Cardiac surgeryb	1517	(1.0%)	1036	(1.2%)
Coronary artery bypass graft	3372	(2.3%)	1902	(2.1%)
Carotid endarterectomy	1209	(0.8%)	760	(0.9%)
Cesarean section	22,046	(14.9%)	10,827	(12.1%)
Colon surgery	7723	(5.2%)	8046	(9.0%)
Herniorrhaphy	21,141	(14.2%)	20,699	(23.2%)
Hip prosthesis	19,127	(12.9%)	8995	(10.1%)
Knee prosthesis	27,790	(18.7%)	12,571	(14.1%)
Laminectomy	15,526	(10.5%)	8520	(9.5%)
Peripheral vascular bypass surgery	677	(0.5%)	535	(0.6%)
Spinal fusion	16,078	(10.8%)	9540	(10.7%)
Vaginal hysterectomy	3206	(2.2%)	1072	(1.2%)
NHSN risk index
0	59,641	(40.2%)	33,833	(37.9%)
1	73,600	(49.6%)	43,649	(48.9%)
2	14,542	(9.8%)	11,258	(12.6%)
3	582	(0.4%)	597	(0.7%)
Missing	0	(0%)	2	(<0.1%)
ASA score
1 (no systemic disease)	4969	(3.3%)	3385	(3.8%)
2 (mild systemic disease)	68,079	(45.9%)	39,121	(43.8%)
3 (severe systemic disease, not life threatening)	64,106	(43.2%)	39,899	(44.7%)
4 (severe systemic disease, life threatening)	11,056	(7.5%)	6799	(7.6%)
5 (moribund patient)	147	(0.1%)	122	(0.1%)
Missing	8	(<0.1%)	13	(<0.1%)
Wound class
Clean	103,876	(70.0%)	58,559	(65.5%)
Clean-contaminated	41,290	(27.8%)	27,525	(30.8%)
Contaminated	1605	(1.1%)	1311	(1.5%)
Dirty-infected	1577	(1.1%)	1396	(1.6%)
Missing	17	(<0.1%)	548	(0.6%)
Prolonged operative durationc	25,939	(17.5%)	18,430	(20.6%)

Data are n (%), mean (SD), n/N (%), or median (IQR). ASA = American Society of Anesthesiologists. NHSN=National Healthcare Safety Network.

^aAge is given for unique patients. Missing data for age were excluded for 8 patients (6 patients who underwent procedures that received control surveillance and 2 patients who underwent procedures that received intervention surveillance).

^bCardiac surgery included open chest procedures on the valves or septum of the heart.

^cProcedures were prolonged if the operative duration was longer than the NHSN 75^th percentile benchmark.

For clusters assigned to control SSI surveillance, a total of 1171 SSIs occurred after 148,365 procedures (PR 0.79 per 100 procedures). In comparison, for clusters that received intervention surveillance, 781 SSIs occurred after 89,339 procedures (PR 0.87 per 100 procedures; model-based PRR 1.10, 95% CI 0.94–1.30; p=0.25). For clusters receiving the intervention, SSI rates were also slightly higher than control cluster rates for complex SSIs (PRR 1.09, 95% CI 0.89–1.34; p=0.40) and superficial SSIs (PRR 1.26, 95% CI 1.00–1.58; p=0.07), but these differences were not statistically significant (Table 2). In exploratory subgroup analysis, the effect of the intervention did not differ based upon cluster type (p=0.53) (appendix p 7).

Table 2.

Primary and secondary outcomes of surgical site infection prevalence rates.

	Control surveillance			Intervention surveillance				Adjusted estimatesc
	SSIsa	Procedures	Unadjusted PRb	SSIsa	Procedures	Unadjusted PRb		PRR (95% CI)	p value
All SSIs	1171	148,365	0.79	781	89,339	0.87		1.10 (0.94–1.30)	0.25
Complex SSIsd	739	148,365	0.50	472	89,339	0.53		1.09 (0.89–1.34)	0.40
Superficial SSIs	432	148,365	0.29	309	89,339	0.35		1.26 (1.00–1.58)	0.070

PR = prevalence rate. PRR = prevalence rate ratio. SSI = surgical site infection.

^a214 SSIs that met present at time of surgery criteria were excluded, including 102 SSIs on the control arm (PR 0•07) and 112 SSIs on the intervention arm (PR 0.13).

^bPrevalence rates are given per 100 procedures performed.

^cModels included effects for time period and intervention phase, and models were adjusted for hospital, median wound class, median American Society of Anesthesiology score, and operation time score.

^dComplex SSIs included deep-incisional and organ/space SSIs.

Traditional SSI surveillance identified 38 total signals that were adjudicated over 8244 procedure-months of traditional surveillance performed during the 36 months of active study periods (signal rate: 0.06 signals per 12 procedure-months of traditional surveillance). Of the 38 total signals, 30 were adjudicated to be true positive signals (positive predictive value, 78.9%) and receive further action, including 24 signals that ultimately triggered formal SSI investigations over 7583 procedure-months capable of generating SSI investigations (investigation rate: 0.04 investigations per 12 procedure-months of traditional surveillance); four of 30 true positive signals were excluded because investigations for the associated hospital/procedure were already in progress (Figure 3). Median time from traditional signal identification to completion of the 24 SSI investigations was 185 days (IQR, 122–242 days). Of the 30 true positive traditional signals, optimised SPC methods detected 28 signals (sensitivity of SPC for detection of traditional signals, 93.3%), including detection of 21 (70.0%) signals prior to traditional surveillance detection by a median of 215 days (IQR, 72–343 days) (appendix p 9). The two traditional signals not detected by SPC were adjudicated to receive action due to the pathogen profile and surgeon-specific SSI rates, respectively.

Figure 3.

Flowcharts detailing all traditional signals (Panel A) and SPC signals on the intervention arm (Panel B) requiring adjudication.

SPC = statistical process control; SSI = surgical site infection.

Optimised SPC surveillance identified 531 total signals requiring adjudication over 4452 procedure-months of intervention surveillance (signal rate: 1.43 signals per 12 procedure-months of SPC surveillance). Of the 531 total signals, 317 were adjudicated to be true positive signals (positive predictive value, 59.7%) and require further action, including 74 signals that led to formal SSI investigations over 3918 procedure months capable of generating SSI investigations (investigation rate: 0.23 investigations per 12 procedure-months of SPC surveillance); 203 duplicate signals were excluded due to ongoing (n=163) or recently completed (n=40) investigations (Figure 3, appendix p 20). Median time from SPC signal identification to completion of the 74 SSI investigations was 163 days (IQR, 97–262 days).

Of the 98 formal SSI investigations, 96 (98.0%) investigations involved detailed line listing for 643 SSIs, including 136 (21.2%) SSIs linked to 24 traditional signals and 507 (78.8%) SSIs associated with 72 SPC signals. Compliance with ten core perioperative best practices for SSI prevention ranged from 12.9% for combined use of oral antibiotics and bowel preparation prior to colon surgery to 89.9% for appropriate perioperative antibiotic choice. The distribution of best practice deficiencies was similar for SSIs analysed during investigations initiated by traditional and SPC surveillance, and the median preventability score was 37.5 for SSIs in both groups. However, despite nearly twice as many surveillance months of traditional surveillance, the absolute number of deficiencies uncovered by SPC-generated investigations was several fold higher than deficiencies recognised during investigations linked to traditional surveillance. In fact, 120 (88.2%) of 136 SSIs analysed during investigations initiated by traditional surveillance had deficiencies in at least two best practices, compared to 458 (90.3%) of 507 SSIs linked to SPC-generated investigations (Table 3).

Table 3.

Compliance with 10 core perioperative best practices for SSI prevention for SSIs investigated due to traditional surveillance signals versus optimised SPC signals.

	Compliance with best practice
	Traditional surveillance (n=136 SSIs investigated)		Optimised SPC surveillance (n=507 SSIs investigated)
Best practice
Choice of prophylactic antibiotic(s)	129/136	(94.9%)	449/507	(88.6%)
Timing of prophylactic antibiotic(s)	111/136	(81.6%)	423/507	(83.4%)
Weight-based dose of prophylactic antibiotic(s)	108/136	(79.4%)	449/507	(88.6%)
Redosing of prophylactic antibiotic(s)a	6/12	(50.0%)	38/65	(58.5%)
Skin antisepsis with appropriate agent	120/136	(88.2%)	408/507	(80.5%)
Maintenance of perioperative normothermia	84/136	(61.8%)	383/507	(75.5%)
Operative and postoperative supplemental oxygenb	12/83	(14.5%)	77/420	(18.3%)
Postoperative glucose monitoring and control	42/136	(30.9%)	222/507	(43.8%)
Use of SSI prevention checklist	44/136	(32.4%)	151/507	(29.8%)
Prophylactic oral antibiotics and mechanical bowel preparationc	0/0	··	28/217	(12.9%)
Best practice deficiencies and preventability score
Procedures with at least 1 deficiency	134/136	(98.5%)	504/507	(99.4%)
Procedures with at least 2 deficiencies	120/136	(88.2%)	458/507	(90.3%)
Preventability scored	37.5	(25.0–50.0)	37.5	(25.0–44.4)

Data are n/N (%) or median (IQR). SPC = statistical process control. SSI = surgical site infection.

^aAnalysed for surgeries requiring redosing based on surgery duration and antibiotic(s) chosen.

^bAnalysed for surgeries requiring general anesthesia and mechanical intubation.

^cAnalysed for colon surgeries only.

^dPercentage of 10 core SSI prevention best practices that were not appropriately completed.

SSI investigations were multifaceted and, in addition to line listing, typically included meetings with local infection prevention teams (n=76/98; 77.6%). All but one investigation yielded written reports with specific recommendations for decreasing SSI risk, often including strategies to improve adherence to core best practices for SSI prevention. However, while these recommendations usually targeted infection prevention committees (n=82/97; 84.5%) and operating room leadership (n=58/97; 59.8%), surgeons performing the procedure under investigation were among the primary recipients of recommendations less than half of the time (n=42/97; 43.3%). Furthermore, direct discussion with surgeons occurred in only 16 (16.3%) investigations. While surgical teams made practice changes to follow recommendations in 73 (75.3%) investigations, all recommendations were followed for only 21 (21.6%) investigations. These characteristics of SSI investigations were similar when stratified by investigations generated by traditional surveillance signals versus optimised SPC signals (Table 4).

Table 4.

Characteristics of SSI investigations generated by traditional surveillance signals versus optimised SPC signals.

	Traditional surveillance (n=24 SSI investigations)		Optimised SPC surveillance (n=74 SSI investigations)
Procedure type investigated
Abdominal hysterectomy	4	(16.7%)	8	(10.8%)
Cardiac surgerya	0	(0%)	0	(0%)
Coronary artery bypass graft	4	(16.7%)	0	(0%)
Carotid endarterectomy	0	(0%)	0	(0%)
Cesarean section	0	(0%)	11	(14.9%)
Colon surgery	0	(0%)	20	(27.0%)
Herniorrhaphy	0	(0%)	7	(9.5%)
Hip prosthesis	5	(20.8%)	8	(10.8%)
Knee prosthesis	5	(20.8%)	9	(12.2%)
Laminectomy	2	(8.3%)	3	(4.1%)
Peripheral vascular bypass surgery	0	(0%)	2	(2.7%)
Spinal fusion	2	(8.3%)	6	(8.1%)
Vaginal hysterectomy	2	(8.3%)	0	(0%)
Primary reason for investigation
Persistent SSI rate elevation	4	(16.7%)	16	(21.6%)
Rapid SSI rate elevation	8	(33.3%)	37	(50.0%)
Elevated surgeon-specific SSI rates	10	(41.7%)	15	(20.3%)
Pathogen profile	2	(8.3%)	6	(8.1%)
Investigation actions taken by study teamb
Phone call or meeting with hospital infection prevention teams	14	(58.3%)	62	(83.8%)
Phone call or meeting with operating room staff	10	(41.7%)	10	(13.5%)
Phone call or meeting with surgeon(s)	7	(29.2%)	9	(12.2%)
Detailed line listing	24	(100%)	72	(97.3%)
Step-by-step perioperative practice review	8	(33.3%)	35	(47.3%)
In-person hospital visit	9	(37.5%)	26	(35.1%)
In-person operative room observation	5	(20.8%)	7	(9.5%)
Written recommendations provided	24	(100%)	73	(98.6%)
Type of recommendations
Perioperative process improvement	22/24	(91.7%)	72/73	(98.6%)
Ongoing surveillance and feedback	16/24	(66.7%)	45/73	(61.6%)
Education of operating room staff	1/24	(4.2%)	13/73	(17.8%)
Patient-specific intervention	3/24	(12.5%)	14/73	(19.2%)
Groups targeted by recommendations
Infection prevention committee	17/24	(70.8%)	65/73	(89.0%)
Operating room leadership	15/24	(62.5%)	43/73	(58.9%)
Surgeons	11/24	(45.8%)	31/73	(42.5%)
Hospital leadership	11/24	(45.8%)	29/73	(39.7%)
Infectious diseases clinicians	3/24	(12.5%)	3/73	(4.1%)
Pharmacy committee	0/24	(0%)	3/73	(4.1%)
Other committeec	0/24	(0%)	2/73	(2.7%)
Implementation of recommendations
All recommendations implemented	6/24	(25.0%)	15/73	(20.5%)
Some recommendations implemented	18/24	(75%)	55/73	(75.3%)
No recommendations implemented	0/24	(0%)	3/73	(4.1%)
Days from signal identification to completion of investigation	185	(122–242)	163	(97–262)

Data are n (%), n/N (%), or median (IQR). SPC = statistical process control; SSI = surgical site infection.

^aCardiac surgery included open chest procedures on the valves or septum of the heart.

^bInvestigation actions included steps that occurred after initial SSI data review and discussions with network and hospital infection preventionists, which occurred in all investigations.

^cOther committees targeted by recommendations included a performance improvement committee (n=1) and a colon SSI prevention team (n=1).

Discussion

In this large, multicentre cluster randomised trial with stepped wedge design, the addition of optimised SPC surveillance and feedback to traditional methods for SSI surveillance did not lead to decreased SSI rates within a large infection control network of community hospitals. However, compared to traditional surveillance, optimised SPC methods detected over three times as many clinically important increases in SSI rates that required detailed SSI investigations. Furthermore, SSI investigations generated by SPC signals uncovered nearly four times as many SSIs that had multiple deficiencies in best practices known to decrease SSI risk. Finally, optimised SPC detected nearly all traditional surveillance signals indicative of SSI rate increases, and SPC detection usually preceded traditional detection. In addition to excellent sensitivity and timing of detection, optimised SPC methods maintained reasonable positive predictive value.

Our findings confirm the results of the retrospective analyses we performed in the same network of hospitals that led to the selection of optimised SPC methods used in this trial.^22,26 Our prior retrospective analysis estimated that optimised SPC would have 90% sensitivity and 56% positive predictive value in detecting clinically important increases in SSI rates for hospital and procedure combinations.²⁶ Prospective evaluation in this trial yielded similar performance characteristics. Furthermore, for certain types of surgery, including colon surgery, Cesarean section, and herniorrhaphy, SSI investigations were exclusively generated by SPC signals. Use of traditional surveillance alone for these three procedure types would have led to delayed or missed detection of 38 important SSI increases that required formal investigation. Finally, investigators in this study found the workload burden of SPC surveillance to be quite manageable. For example, based on our signal and investigation rates, a hospital using these SPC surveillance methods would expect on average to adjudicate fewer than two signals per year per procedure type under surveillance. Per procedure type, these signals would generate approximately one detailed SSI investigation every four years.

To our knowledge, our study is the first prospective trial to evaluate the impact of optimised SPC methods and feedback on a primary outcome of SSI rate. Prior prospective studies have shown the potential for use of SPC to decrease rates of hospital-acquired methicillin-resistant Staphylococcus aureus²³ and to monitor for improvement in SSI rates following Cesarean section.²⁴ In addition, a recent cluster randomised trial performed at 40 hospitals in France found that use of SPC and feedback decreased rates of major adverse events following gastrointestinal tract surgery; however, SSIs were analysed with other postoperative severe complications, and the odds of this composite endpoint were not different for intervention and control hospitals.³⁷

Similarly, improved detection of clinically important SSI rate increases and identification of deficiencies in perioperative best practices for SSI prevention did not translate into reduction in SSI rates in our trial. The discrepancy between SSI detection and prevention emphasises that enhanced surveillance and associated recommendations for improvement do not necessarily result in improved outcomes.^38,39 Specifically, this dichotomy suggests that strategies for providing feedback and promoting changes in perioperative practices employed in our trial were insufficient to consistently decrease SSI risk. As a result, we identified two primary areas for improvement in our approach for SSI investigations. First, we recommend early and consistent involvement of surgeons associated with SSIs being analysed. For investigations that occurred during this study, initial discussions, line listing analyses, and formulation of recommendations often occurred without direct involvement of surgeons. Lack of direct communication likely decreased surgeon perception of the importance of the identified SSI rate increases, particularly when rate increases were identified by automated and external SPC surveillance rather than local hospital recognition of a problem. Accordingly, hospitals did not implement many of the recommendations provided to decrease SSI risk. Second, we recommend engagement of a local hospital perioperative team of designated “champions” to assist with feedback of increased SSI rates and implementation of practice changes designed to decrease SSI risk. For SSI prevention, this team would ideally include a local hospital surgeon champion, infection preventionist, and other member of the perioperative team, such as an operating room anaesthesiologist or nurse. Dedicated champions, as utilised by Duclos et al.,³⁷ could help to engage key stakeholders, interpret SPC charts and other surveillance findings, and implement practice changes adapted for local hospital strengths and resources.

Importantly, rates of SSI have declined over time in our community hospital network,⁴⁰ and in both arms of this study, overall SSI rates were less than 1%. The low risk of SSI suggests that even with improved feedback of SSI rates and adherence to best practices for SSI prevention, true decreases in overall SSI rates in a multi-hospital network would likely be difficult to detect, requiring a large study. In contrast, the rates of deficiencies in best practices for SSI prevention were much higher than SSI rates. Use of SPC as a quality tool to detect and monitor for improvement in these process measure deficiencies could promote implementation of more effective interventions designed to decrease risk of SSI. However, using SPC to monitor performance measures as surrogates for SSI rates would require near real-time assessment and documentation of compliance with best practices, which was inconsistent across our network.

This study had several limitations. First, we evaluated the addition of optimised SPC surveillance to well-established traditional SSI surveillance measures performed within a large community hospital infection control network. The impact of SPC surveillance and feedback could differ at hospitals or hospital networks with different SSI epidemiology, baseline surveillance practices, and compliance with best practices for SSI prevention. We believe that optimised SPC methods would also improve early identification of important increases in SSI rates in other settings, however, which investigators at one hospital demonstrated in a retrospective analysis.⁴¹ Second, initiating SPC surveillance and linking SPC signals to SSI data to facilitate signal adjudication required technical support at study onset. This surveillance system also required timely entry of SSI and surgical procedure data into the Surgical Database to promote early detection of SSI rate increases. Hospitals without adequate technical support or efficient electronic SSI data entry would be less likely to benefit from the enhanced detection provided by optimised SPC. Third, SSI investigations initiated by SPC signals likely prevented generation of some traditional surveillance signals that would have subsequently occurred if SSI rate increases for the same hospital and procedure had not already been detected by SPC. Nevertheless, SPC surveillance did not affect traditional surveillance for clusters randomised to the control arm, and SPC surveillance more readily identified signals in the control arm than traditional surveillance. Furthermore, early SPC signal detection among intervention clusters cannot account for the magnitude of the discrepancies between SSI investigations generated and best practice deficiencies uncovered by SPC compared to traditional surveillance. Fourth, SSI rate data from 7 clusters were excluded at time periods during the final 14 months of the study when associated hospitals terminated SSI reporting for relevant procedures; however, complete SSI data from the 92 other clusters that began the trial were analysed, decreasing the chance that the observed dropout in SSI reporting meaningfully affected the reported results.

This randomised, multicentre SW-CRT assessed the impact of optimised SPC surveillance and feedback on SSI rates within a large infection control network of community hospitals. Compared to traditional surveillance, SPC methods more frequently detected important SSI rate increases and associated deficiencies in best practices for SSI prevention; however, feedback of these data did not lead to decreases in SSI rates. Further research is needed to determine the best application of SPC methods and feedback to improve adherence to SSI quality measures and prevent SSIs.

Contributors

AWB, JCB, and DJA conceptualised the study. DJA acquired funding. DB, EPD, LRG, SSH, and CRM provided additional input on study design. AWB, KRF, and DJA curated the data. II, JCB, NN, and JS optimised the SPC methods and oversaw the weekly application of SPC charts. AWB, SSL, and DJA adjudicated traditional surveillance signals. AWB and DJA blindly adjudicated optimised SPC surveillance signals. AWB, SSL, BW, EB, LC, KLC, ALC, PP, LR, LA, and DJA performed surgical site infection investigations. YL performed the statistical analysis. AWB, YL, and DJA directly accessed and verified the underlying data. AWB, YL, and DJA were responsible for the decision to submit the manuscript for publication. AWB wrote the original draft. All authors critically reviewed the manuscript, approved the final version, and assume responsibility for the overall content of this article.

Data sharing statement

Deidentified data from the Early 2RIS trial will be made available at the time of publication. Supporting documents will be made available, including the protocol, statistical analysis plan, and data dictionary. Data and documents will be made available to investigators after approval of a proposal for data use. Proposals may be submitted following article publication and should be directed to Deverick.Anderson@duke.edu.

Declaration of interests

AWB reports grant funding from the Agency for Healthcare Research and Quality, National Institute of Allergy and Infectious Diseases of the National Institutes of Health, and the Centers for Disease Control and Prevention Epicenters Program; and participation on the Advisory Board for Medincell. DJA reports grant funding from the Agency for Healthcare Research and Quality, National Institute of Allergy and Infectious Diseases of the National Institutes of Health, and the Centers for Disease Control and Prevention Epicenters Program; royalties from UpToDate; and co-ownership of Infection Control Education for Major Sports, LLC. EPD reports participation on the Advisory Board for Crely, Inc. and board membership of the Surgical Infection Society Foundation. II, JCB, KRF, YL, and NN report grant funding from the Agency for Healthcare Research and Quality.