Comparing national point prevalence surveys of healthcare-associated infection and antimicrobial prescribing: a methodological approach to adjust for differences in case-mix

Abstract

Background:

National point prevalence surveys (PPS) of healthcare-associated infection (HAI) and antimicrobial prescribing in hospitals were conducted in 2011 and 2016 in Scotland. When comparing results of PPS, it is important to adjust for any differences in patient case-mix that may confound the comparison.

Aim:

To describe the methodology used to compare prevalence for the two surveys and illustrate the importance of taking case-mix (patient and hospital stay characteristics) into account.

Methods:

Multivariate models (clustered logistic regression) that adjusted for differences in patient case-mix were used to describe the difference in prevalence of six outcomes (HAI, antimicrobial prescribing and four devices: central vascular catheter, peripheral vascular catheter, urinary catheterisation and intubation) between the 2011 and 2016 PPS. Univariate models that did not adjust for these differences were also developed for comparison to show the importance of adjusting for confounders.

Results:

Without adjustment for case-mix, HAI and intubation prevalence estimates were not significantly different in 2016 compared with 2011 although with adjustment, the prevalence of both was significantly lower (P=0.03 and P=0.02, respectively). These associations were only identified after adjustment for confounding by case-mix.

Conclusions:

While prevalence surveys do not provide intelligence on temporal trends as an incidence-based surveillance system would, if limitations and caveats are acknowledged, it is possible to compare two prevalence surveys to describe changing epidemiology. Adjusting for differences in case-mix is essential for robust comparisons. This methodology may be useful for other countries that are conducting large, repeated prevalence surveys.

Background

Point prevalence surveys (PPS) can be a useful tool to measure and monitor the burden of healthcare-associated infection (HAI) and antimicrobial prescribing (French et al., 1989). There have been two Europe-wide PPS of HAI and antimicrobial prescribing in acute hospitals, initiated by the European Centre for Disease Prevention and Control (European Centre for Disease Prevention and Control, 2012, 2016). Unlike incidence-based surveillance programmes that tend to focus on single microorganisms or infection types, PPS can be used to measure the prevalence of all HAI types and a number of other conditions of interest, concurrently, and they are typically considered to be less resource-intensive than incidence-based surveillance (Brusaferro et al., 2006). They are therefore useful for taking stock of the current epidemiological landscape and this evidence can be used to inform local, national and EU-wide policy.

While prevalence surveys do not provide intelligence on changing temporal trends as a surveillance system based on incidence data would, if appropriate adjustments are made to account for known confounders such as demographic factors, and if limitations and caveats are acknowledged, then results from different prevalence surveys can be compared using statistical modelling to describe changes in epidemiology. Adjusting for differences in patient case-mix is a common step in statistical modelling and creates a ‘level playing field’ for comparing outcomes in different groups (e.g. in different survey years or locations with different demographics). When repeated, well-designed surveys can also provide useful data on prevalence trends and on effectiveness of infection prevention and control measures (French et al., 1989).

Scotland participated in the most recent Europe-wide hospital PPS collecting data from September to November 2016 (Health Protection Scotland, 2017) and in the previous survey during September and October 2011 (Health Protection Scotland, 2012). This paper describes the multivariate methodology used to compare the prevalence estimates of six outcomes of interest, namely HAI, antimicrobial prescribing and the use of four invasive devices while adjusting for patient case-mix, between the 2011 and 2016 Scottish hospital surveys. The results of the multivariate models that adjust for differences in the patient case-mix are presented and compared with those from the univariate, unadjusted models. The caveats and limitations of this methodology are also discussed.

Methods

Data collection

Rolling PPS were carried out in acute Scottish hospitals between September and November 2016 and in September and October 2011 using Scottish protocols (Health Protection Scotland, 2011, 2016) adapted from ECDC protocols for PPS (European Centre for Disease Prevention and Control, 2012, 2016). The same study design, inclusion/exclusion criteria and case definitions were used in both surveys, although minor changes to the surgical site infection (SSI) and pneumonia definitions were made in the most recent survey. Data pertaining to all eligible inpatients were collected by trained staff from local infection prevention and control teams and antimicrobial management teams. Data on inpatient demographics, HAI and antimicrobial status were extracted from a number of sources available on the ward at the time of survey including nursing and medical notes, temperature charts, drug charts, electronic prescribing systems, surgical notes, laboratory reports showing microbiology results, and other relevant charts, e.g. wound charts, stool charts and care plans. Full details of the study design, inclusion/exclusion criteria and case definitions for both the 2011 and 2016 surveys are described in the survey protocols (Health Protection Scotland, 2011, 2016).

Ethical approval

A Privacy Impact Assessment (PIA) was undertaken and the project was reviewed and approved by the Public Benefit and Privacy Panel for Health and Social Care (PBPP) (Application Number 1516-0599).

Data and definitions

A dataset combining patient-level data from all surveyed acute adults in the 2016 and 2011 surveys was created (n = 10,813 and n = 11,015, respectively). The combined dataset included patient case-mix data, risk factor data and binomial outcome data. The binary outcomes of interest investigated were: prevalence of HAI; antimicrobial use; use of central vascular catheter (CVC); use of peripheral vascular catheter (PVC); urinary catheterisation; and intubation. All outcomes were determined at the time the patient was surveyed. Eight variables describing patient case-mix were investigated, namely: patient age group; patient sex; McCabe score (McCabe and Jackson, 1962); whether the patient had undergone surgery since admission; length of hospital stay (from admission to survey); consultant specialty; and the ward and hospital type that the patient was in at the time of survey.

Descriptive analyses

The distribution of age in 2016 and 2011 was compared using a Mann–Whitney U test and median ages were estimated. A Pearson’s Chi-square test with a continuity correction was used to assess if there was any difference in the percentage of patients who were male and those with the most severe McCabe scores between the two surveys. All tests of independence were univariate (i.e. did not adjust for differences in patient case-mix) and were carried out in R version 3.3.1 (R Core Team, 2018) with statistical significance set at P < 0.05.

Statistical analyses comparing prevalence estimates while adjusting for patient case-mix

Six univariate models were first developed with the outcome of interest as a binary dependent variable and the categorical variable ‘year’ as the only independent variable. An odds ratio (OR) for the association between year and each outcome of interest was reported with 95% confidence intervals (CI). These models were not used to determine which variables to include in multivariate models and did not adjust for differences in the patient case-mix between surveys. They were included in this analysis to compare with the results from the multivariate models and to demonstrate the importance of adjusting for confounders.

Six multivariate models were developed to determine whether prevalence estimates of the six outcomes of interest from the 2011 and 2016 surveys were significantly different after controlling for patient case-mix. A Bonferroni correction test was initially conducted to identify any significant interactions between ‘year’ and other risk factors; however, no interactions were found to be significant at the Bonferroni-adjusted significance cut-off and hence no interactions were included in the multivariate model. Multivariate models were developed with the outcome of interest as a binary dependent variable, and the categorical variable ‘year’ plus eight patient case-mix variables as independent variables. All eight patient case-mix variables were initially included in a backward elimination approach to select the most parsimonious model for each outcome. The ‘year’ variable was retained in each model to enable the comparison between the surveys to be made. Survey weighted binomial models were used (R package ‘survey’). This model adjusts for clustering of beds within wards and wards nested within hospitals. The Wald test was used to estimate individual P values for each variable in the models. Patients with missing data were excluded (number of included patients shown in Table 1). All analyses were carried out in R (version 3.3.1).

Table 1.

Prevalence of six outcomes of interest in acute adult inpatients in Scotland, in 2011 and 2016, and results of unadjusted and adjusted models investigating differences in prevalence estimates between study years.

2011					2016				Unadjusted model results			Adjusted model results
Outcomes of interest	Patients surveyed (n)	Patients with outcome of interest (n)	Prevalence (%)	95% CI	Patients surveyed (n)	Patients with outcome of interest (n)	Prevalence (%)	95% CI	Unadjusted OR (2011 as reference year)	95% CI	P	Unadjusted OR (2011 as reference year)	95% CI	P
HAI	11,015	548	5.0	4.5–5.5	10,813	497	4.6	4.1–5.1	0.9	0.77–1.06	0.2	0.84	0.72–0.98	0.03
AM	11,012	3,653	33.2	31.8–34.6	10,869	3,878	35.7	34.2–37.2	1.12	1.02–1.22	0.02	1.11	1.02–1.21	0.01
CVC	11,022	439	4.0	3.3–4.7	10,824	482	4.5	3.7–5.2	1.11	0.86–1.43	0.42	1.01	0.78–1.29	0.95
PVC	11,002	3,551	32.3	30.5–34.1	10,803	3,924	36.3	34.3–38.3	1.19	1.06–1.34	0.004	1.25	1.14–1.38	<0.001
Urinary catheterisation	11,001	2,209	20.1	18.8–21.4	10,790	2,249	20.8	20–22.1	1.03	0.92–1.16	0.57	0.98	0.88–1.08	0.64
Intubation	11,003	135	1.2	0.8–1.6	10,823	92	0.9	0.5–1.2	0.69	0.41–1.15	0.15	0.55	0.34–0.89	0.02

AM, antimicrobial prescribing; CI, confidence interval; CVC, central vascular catheter; HAI, healthcare-associated infection; OR, odds ratio; PVC, peripheral vascular catheter.

All eight of the patient case-mix variables plus ‘year’ were included in the final models for CVC and PVC use prevalence; all variables excluding hospital class were included in the final models for antimicrobial prescribing, urinary catheterisation and intubation; and all variables excluding patient age and sex were included in the final model for HAI. For the final six models, an overall P value and an OR for the association between year and the outcome of interest, adjusted to account for confounding by patient case mix, with 95%CI (large sample approximation for the log OR) was calculated using 2011 as the reference group.

Results

A total of 10,889 acute adult inpatients in 43 hospitals were included in the 2016 survey, and 11,087 acute adult inpatients in 49 hospitals in the 2011 survey. Between the two surveys, there were differences in the demographics of the Scottish hospital population with the median age of acute adult inpatients surveyed in 2016 significantly higher than the median age in 2011 (73 years vs. 71 years, P < 0.001) and the percentage of acute adult inpatients with an ultimately fatal or rapidly fatal prognoses (quantified using the McCabe score) significantly higher in 2016 compared with 2011 (41.9% vs. 35.0%, P < 0.001). There was no difference the percentage of acute adults who were male between the two surveys (P = 0.8).

Table 1 describes the prevalence of HAI, antimicrobial prescribing and device use in 2016 and 2011. Both the unadjusted OR for each outcome and year and the adjusted OR for each outcome and year (adjusted for differences in the patient case mix between 2016 and 2011) are presented. Without adjustment for case mix, HAI and intubation prevalence estimates were not significantly different in 2016 compared with 2011; however, after adjustment, the prevalence of both was significantly lower (P = 0.03 and P = 0.02, respectively). These associations were only identified after adjustment for case-mix. After adjusting for differences in patient case-mix between the two surveys, the prevalence of both antimicrobial prescribing and PVC use was significantly higher in 2016 compared with 2011 (P = 0.01 and P < 0.001, respectively).

Discussion

This paper describes a methodology to compare the results from two national PPS conducted five years apart. The demographics of inpatient hospital populations changed in the five years between the 2011 and 2016 prevalence surveys with the results suggesting that patients in 2016 were older and sicker than in 2011. This has important implications for the potential risk of HAI, antimicrobial treatment and device use but, in addition, if such differences are not adjusted for when making comparisons between surveys, the conclusions drawn may reflect differences in patient case-mix and not accurately reflect the true epidemiological picture.

Statistical modelling techniques can be used to make comparisons of epidemiological data while accounting for confounding as a result of differences in patient case-mix providing a more robust comparison (Cairns et al., 2009). These analyses indicate that the prevalence of HAI and intubation in acute adult inpatients in 2016 was significantly lower once differences in patient case-mix were adjusted for. Without adjustments, the statistical difference in prevalence between 2011 and 2016 would not have been identified. For antimicrobial prescribing and PVC use, the significantly higher prevalence in 2016 versus 2011 remained after adjustment for differences in patient case-mix providing assurance that the reported difference was not affected by confounding. However, adjusting for case-mix differences provided more accurate measures of the magnitude of association (ORs) and level of significance. Other studies have also shown the importance in adjusting for differences in patient case-mix in order to create a level playing field for comparing prevalence estimates between hospitals (Kritsotakis, 2008; Sax et al., 2002) and countries (Kritsotakis, 2008; Zingg et al., 2017).

Prevalence surveys do not provide intelligence regarding trends nor allow the true impact of interventions to be assessed between two surveys. However, using the methodology described here may give an indication that changes in public health practices or interventions aiming to reduce HAI, antimicrobial prescribing or device use have or have not been successful, or may suggest where there is a need for additional focus. Despite quality improvement work undertaken in the intervening period between surveys (Cairns et al., 2009; Healthcare Improvement Scotland, 2017; Scottish Government, 2013), the prevalence of PVC was higher in 2016 (36.3% vs. 32.3%) and there had been no significant change in the prevalence of CVC use or urinary catheterisation since 2011. While there may be good clinical reasons for the increase in use of PVCs and important differences in the population that have not been taken into account, these data indicate that continued quality improvement work on devices remains an important strategy to reduce the risk of catheter-related infections. In addition, the higher prevalence of antimicrobial prescribing prevalence reinforces the ongoing need for effective antimicrobial stewardship.

Up to eight different patient case-mix variables were included in the multivariate models and it was possible to include this many variables owing to the sufficiently large numbers of surveyed patients and prevalence estimates in both surveys. Where prevalence is low, results must be interpreted with caution, as is the case here with prevalence of intubation. The eight case-mix variables were considered important risk factors for HAI, antimicrobial prescribing and invasive device use; data for these were readily available from the surveys, but there may be other potentially relevant factors which were not included in this analysis and thus controlled for. For example, there were some variables that could not be included in the multivariate models despite being known risk factors for the outcomes of interest. Urinary catheterisation is a known risk factor for healthcare-associated urinary tract infections; however, urinary catheterisation sits on the causal pathway from exposure to outcome (HAI) and hence its inclusion as an explanatory variable could introduce confounding into the model. In addition, urinary catheters are also often fitted to treat infection and alleviate symptoms but since the reason for and timing of catheterisation (i.e. before or after onset of infection) could not be determined from these data, this was a further reason why models did not include device use as an explanatory variable and the prevalence of HAI was not adjusted for device use.

Another limiting factor to be considered when comparing prevalence surveys is that to be comparable, all else must be equal meaning that inclusion and exclusion criteria, study population, protocols, time of year, case and category definitions used must be the same. In this analysis, the case definitions for SSI and pneumonia had changed since the 2011 survey which may have led to decreased case ascertainment of SSI and increased ascertainment of pneumonia (European Centre for Disease Prevention and Control, 2016); hence, this may have affected the overall 2016 prevalence of HAI in comparison to 2011. Lastly, it must also be acknowledged that prevalence surveys can capture the prevalence at a single point in time, which may not reflect the prevalence at other times.

Conclusion

When comparing the results of two prevalence surveys, it is important to adjust for any known factors that may have changed between surveys and therefore affected the prevalence. If confounding factors are not taken into consideration, then it is not possible to definitively conclude that any differences in prevalence estimates are true and not a result of changes in the survey population. This was shown where an unadjusted comparison of HAI and intubation prevalence estimates concluded no change between two prevalence surveys conducted five years apart but when differences in the patient case-mix were adjusted for in a multivariate model, the prevalence estimates of both were shown to be significantly lower in the most recent survey.

This paper describes a methodology for comparing the results of prevalence surveys while acknowledging a number of important limitations and caveats. It provides assurance that any differences (after adjustments were made) in prevalence between surveys are not due to differences in case-mix within the hospital population; therefore, any findings resulting from analyses using large prevalence datasets could be important in terms of setting local and national health protection priorities.