Estimating central line-associated bloodstream infection incidence rates by sampling of denominator data: A prospective, multicenter evaluation

Abstract

Background:

Large-scale, prospective, evaluation of sampling for central line–associated bloodstream infection (CLABSI) denominator data was necessary prior to National Healthcare Safety Network (NHSN) implementation.

Methods:

In a sample of volunteer hospitals from states in the Emerging Infections Program, prospective collection of CLABSI denominators (patient days, central line days [CLDs]) was performed in eligible locations for ≥6 and ≤12 consecutive months using the current NHSN method (daily collection) and also by a second data collector who sampled the denominator data 1 d/wk. The quality of the sampled data was evaluated and used to calculate estimated CLDs and CLABSI rates, which were compared with actual CLDs and CLABSI rates (daily counts).

Results:

In total, 89 locations in 66 acute care hospitals participated. Sampled data were collected as intended 88% of the time; the quality of the data was comparable with the data collected daily. In locations with higher CLDs per month (≥75), estimated CLDs and CLABSI rates were similar to actual CLDs and CLABSI rates; however, there were significant differences in actual and estimated values among locations with lower (≤74) CLDs per month.Sampling was successfully implemented, but significant differences in the accuracy of estimated CLDs and CLABSI rates, based on the actual number of CLDs per month, were noted.

Conclusion:

For locations with a higher number of CLDs per month, sampling 1 d/wk is a valid and accurate alternative to daily collection of CLABSI denominator data.

Increasing requirements for the collection and reporting of health care–associated infection (HAI) surveillance data to the National Healthcare Safety Network (NHSN)^1,2 has prioritized the need for efficient and valid data collection methods. Daily collection of HAI denominator data is reported to be a predominantly manual and burdensome process.^3-6 Although capture of data from electronic health records for NHSN reporting remains the ultimate goal,^7,8 sampling of central line days (CLDs) may provide a viable, accurate, and efficient method for collecting some HAI denominator data.^5,9,10To determine the practicality of sampling denominator data, we performed a prospective evaluation of the implementation of once-weekly sampling in a large number of acute care hospitals and inpatient care locations.

METHODS

Participants and data collection

A convenience sample of acute care hospitals performing NHSN central line–associated bloodstream infection (CLABSI) surveillance located within 10 Emerging Infections Program states was identified for participation. Eligible inpatient care locations included critical care units, step-down units, and wards collecting manual, daily CLABSI denominator data in accordance with the 2011 NHSN protocol¹¹ (the number of patients and number of patients with ≥1 central line of any type collected daily, at the same time each day, during the month, were recorded). In each participating location, a second independent data collector obtained patient days and CLDs 1 d/wk for a period of 6-12 consecutive months during 2011. Participants were instructed to select a single designated day of the week, between Monday and Friday (eg, every Thursday), on which to collect their weekly denominator data; Saturday and Sunday were not used because these days have been shown to generate the least accurate estimates of CLDs ^5,9,10 If data collection on the designated day was missed, participants were instructed to collect data on the next available day. Standardized data collection forms were provided. Each month, Emerging Infections Program project staff entered data into Microsoft Excel (Microsoft, Redmond, WA) spreadsheets for transfer to the Centers for Disease Control and Prevention (CDC).

Statistical analysis

Characteristics of participating facilities and locations and data submitted were summarized. To assess the implementation of once-weekly sampling, the proportion of days with 1 of 4 denominator data quality errors (eg, identification of days with missing or implausible values) (Table 1) was calculated and compared for daily data collection and once-weekly sampling.

Table 1.

Summary of denominator data quality errors among denominator data collected daily and sampled once weekly submitted, 89 locations

Data quality error*	Collected daily (26,618 d)	Sampled once weekly (3,807 d)
Patient days and central line days missing	318 (1.19)	77 (2.00)
Patient days missing	15 (0.06)	0 (0.00)
Central line days missing	161 (0.60)	1 (0.03)
Central line days > patient days	148 (0.56)	3 (0.08)
Total data quality errors†	642 (2.41)	81 (2.13)

NOTE. Values are number of days (%) with error identified.

^*Data quality error are mutually exclusive.

^†Comparison of daily and weekly total data quality errors (χ², P =.4372).

The accuracy of the estimated CLDs within a location was assessed using the CLD error (difference between actual and estimated CLDs), CLD percentage error (CLD error expressed as a percentage [actual CLDs – estimated CLDs/actual CLDs × 100]), and CLASBI rate error per 1,000 CLDs (difference between CLABSI rates calculated using actual and estimated CLDs). A small constant of 0.1 was added to the CLABSI rate numerator to avoid comparing zero CLABSI rates. Because the number of CLDs per month has been shown to be a significant predictor of accuracy between actual and estimated CLDs, ⁸ the accuracy of estimated CLDs was also assessed by the mean number of CLDs per location month (sum of the actual CLDs/number of months of denominator data reported).

Distributions of the CLD percentage error and CLABSI rate error were assessed using box and whisker plots showing the median, 5th, 25th, 75th, and 95th percentiles. The mean, median, and distribution of actual and estimated CLDs and CLABSI rates were compared using the paired t test, Wilcoxon signed-rank test, and Kuiper test. The Pearson χ² test was used to assess differences in proportions for categorical variables. All tests were 2 sided; P <.05 was considered statistically significant. Statistical analysis was performed using SAS version 9.3 (SAS Institute, Cary, NC).

Human subjects review

A protocol for this surveillance evaluation project was reviewed by the Office of the Director in the National Center for Emerging and Zoonotic Infectious Diseases at the CDC and was determined not to constitute human subjects research.

RESULTS

Description of participating hospitals and locations

Sixty-six acute care hospitals (median size, 159 beds; range, 12-953) submitted 6-12 consecutive months (median, 12 months) of denominator data for 89 eligible locations (median size, 15 beds; range, 2-48). Included in the analysis were a total of 876 location months of denominator data, 248,332 patient days (median, 2,166; range, 266-10,035), and 94,642 CLDs (median, 691; range, 24-3,592), with a median device utilization ratio of 0.36 (range, 0.04-0.84).

Evaluation of denominator data quality and implementation of once-weekly sampling

Four denominator data quality errors were assessed (Table 1), and for each method there was a similar proportion of days (2.4% and 2.1%) containing a data quality error (χ², P = .4372). The sampled data were collected on the designated day 88.2% of the time (3,358 of 3,807 days). The most frequent day for sampled data collection was Wednesday (43%), followed by Thursday and Tuesday (each 19%), Monday (12%), and Friday (7%). Despite instruction to avoid Saturday and Sunday, sampled data were infrequently collected on these days (0.4% of days combined).

Comparison of actual CLDs to estimated CLDs

Among all locations, there was no significant difference between the mean actual CLDs and estimated CLDs (paired t test, P = .0856) (Table 2), but there was a significant difference between the median actual CLDs and estimated CLDs (Wilcoxon signed-rank test, P = .0057) (Table 2). When stratified by the number of CLDs per month (mean CLDs per location), differences in the distribution of the CLD error and CLD percentage were observed. The lowest cut point with the greatest level of accuracy for the most locations was 75 CLDs per month (low CLD group: <75 CLDs per month vs high CLD group: ≥75 CLDs per month). The distribution of the CLD percentage error among the 2 groups was significantly different from one another (Kuiper test, P = .0029) (Fig 1). Additionally, the median CLD percentage error (−9.50; interquartile range, −17.95 to −1.35) (Fig 1) for locations in the low CLD group (n = 44) was significantly different from 0 (Wilcoxon signed-rank test, P < .0001), whereas the median CLD percentage error (0.36%; interquartile range, −5.45 to 5.58) for locations in the high CLD group (n = 45) was not (Wilcoxon signed-rank test, P <.7265).

Table 2.

Mean and percentile distribution of CLD error and CLD percentage error, 89 locations

		Percentile distribution
Measure	Mean	5th	25th	50th	75th	95th
CLD error*	−18.73	−167.62	−54.27	−25.85	19.43	136.19
CLD percentage error†	−4.09	−27.88	−10.96	−3.96	3.15	13.70

CLD, central line day.

^*CLD error is the actual CLD based on daily collection of data – estimated CLD, based on sampling of data.

^†CLD percentage error is the actual CLD – estimated CLD/actual CLD × 100.

Fig 1.

Box and whisker plot showing percentile distribution for central line day (CLD) percentage error stratified by the mean number of CLDs per month, 89 locations.

Impact of using estimated CLDs on CLABSI rates

Among all locations, the median CLABSI rate error (difference between actual and estimated CLABSI rate) was −0.014 per 1,000 CLD (interquartile range, −0.054 to 0.014 per 1,000 CLDs) and was significantly different from 0 (Wilcoxon signed rank test, P = .0147). The median CLABSI rate error (−0.034; interquartile range, −0.098 to −0.003) (Fig 2) for locations in the low CLD group was significantly different from 0 (Wilcoxon signed-rank test, P = .0068), whereas the median CLABSI rate error (0.001; interquartile range, 0.024-0.018) for locations in the high CLD group (n = 45) was not (Wilcoxon signed-rank test, P < .7265). The distribution of the CLABSI rate error per 1,000 CLDs for locations with lower CLDs per month (<75 CLDs) was significantly different than the higher CLDs per month (≥75 CLD) group (Kuiper test, P = .0006).

Fig 2.

Box and whisker plot showing percentile distribution for central line–associated bloodstream infection (CLABSI) rate error per 1,000 central line days stratified by the mean number of central line days (CLDs) per month, 87 locations (2 outlier observations with CLABSI rate errors of −7.63 and −1.3 and <10 mean CLDs per month were removed for display purposes).

DISCUSSION

Sampling CLABSI denominator data has been proposed as a method of data collection that is less burdensome than daily data collection^5,9,10; however, this approach to data collection has not been sufficiently evaluated to determine if or how it could be successfully implemented. In a large number of inpatient locations, with a wide range in CLDs and patient days, we prospectively evaluated the implementation of once-weekly sampling methods, the accuracy of estimated CLDs collected using this method, and the impact of using estimated CLDs generated from sampling on CLABSI rates per 1,000 CLDs. Our results indicate that once-weekly sampling yields the same proportion of quality errors as daily collection of denominator data. As previously suggested,¹⁰ the number of device days per location month influences the accuracy of the estimates of CLDs generated by the sampling data. We found no significant differences between the actual and estimated CLDs or between actual and estimated CLABSI rates for locations with a higher (≥75) number of CLDs per month. However, for the locations with lower (<75) CLDs per month, significant differences between actual and estimated CLDs existed. These findings support the use of sampling denominator data in certain location types with higher CLDs as a valid, less burdensome approach to CLABSI surveillance and will be used to develop a protocol for use in the NHSN.

Identifying NHSN surveillance methods that reduce data collection burden is important. However, the sampling approach to denominator data collection remains an ancillary alternative to the use of electronic health record systems for HAI data collection with submission via Clinical Document Architecture (CDA) to the NHSN.⁸ During the first half of 2014, 14% of locations submitted CLABSI denominator data to the NHSN using CDA (CDC, unpublished data, 2014). Through the Health Information Technology for Economic and Clinical Health Act and specifications for meaningful use to include collection and reporting of HAI data,^12,13 the use of CDA reporting to the NHSN is likely to increase. Manual sampling provides a short-term alternative denominator data collection approach with reduced data collection burden for CLABSI reporting to the NHSN.

There are limitations to this evaluation and the application of our findings. We were not able to validate denominator data submitted by participating locations; it is possible that differences between actual and estimated CLDs may be impacted by differences in data collection practices, and our results may therefore reflect differences larger than those from sampling alone. Participation was limited to intensive care units, step-down units, and wards. Sampling in neonatal intensive care units, specialty care areas, and oncology units was not assessed; appropriateness of this method for those location types is unknown. Although twice-weekly¹⁰ or other sampling strategies can further improve accuracy, this gain is likely to be offset by additional complexity and data collection burden.

Once-weekly sampling is a simple, less resource–intense method of denominator data collection for calculating CLABSI rates. Our findings suggest that for locations with a lower number of CLDs per month, collection of daily denominator data collection remains appropriate; however, for those with a higher number of CLDs per month, once-weekly sampling to obtain estimates of CLDs is a valid and accurate alternative to daily collection of CLABSI denominator data.