Evaluation of the First Year of National Reporting on a New Healthcare Personnel Influenza Vaccination Performance Measure by US Hospitals

Abstract

One thousand hospitals were surveyed on a new measure of health-care personnel influenza vaccination for the 2012–2013 influenza season. Facilities found it easier to collect data on employees than nonemployees; larger facilities reported more challenges than smaller facilities. Barriers may decrease over time as facilities become accustomed to the measure.

In 2007, the Joint Commission began requiring hospitals and long-term care facilities to provide annual influenza vaccinations to healthcare personnel (HCP) and to evaluate vaccination coverage as a condition of accreditation.¹ US hospitals have historically lacked uniformity with how HCP influenza vaccination coverage is measured.² In May 2012, the National Quality Forum endorsed the Centers for Disease Control and Prevention’s (CDC) HCP Influenza Vaccination Measure (NQF 0431), hereafter referred to as “HCP measure.” More than 4,000 acute care hospitals were required to report on the HCP measure starting January 1, 2013, as a part of the Centers for Medicare & Medicaid Services’ Hospital Inpatient Quality Reporting Program.

The HCP measure requires facilities to collect influenza vaccination data for 3 HCP categories: employees, licensed independent practitioners (LIPs), and adult students/trainees and volunteers. For the 2012–2013 influenza season, facilities were required to collect data on the number of vaccinations received inside or outside the facility, medical contraindications, vaccination declinations, and unknown vaccination status, along with the total number of HCP in each category for those working in the facility for 30 days or more from October 1 through March 31. The data were reported through the CDC’s National Healthcare Safety Network (NHSN), a national Internet-based surveillance system for healthcare-associated infections.³

METHODS

In August 2013, an online questionnaire was used to survey staff collecting data for NHSN at nonfederal acute care hospitals. More than 4,000 facilities were identified through the NHSN enrollment database; 3,315 facilities were included in the final sampling frame stratified by the 4 US Census Bureau Regions and 3 bed-size groups (≤100 beds, 101–300 beds, >300 beds).⁴ A simple random sample of 1,000 facilities was selected using the probability-proportional-to-size method without replacement for each stratum. Data collectors at sampled facilities were invited by email to participate in the survey and nonresponders were reminded to complete this via email and telephone. Data were analyzed in SAS, version 9.3 (SAS Institute) and an electronic spreadsheet (Excel; Microsoft). Statistical significance was set at P =.05.

RESULTS

Of the 1,000 facilities surveyed, 68% (n =680) responded; 277 facilities (41%) had 100 or fewer beds, 248 (37%) had 101–300 beds, and 150 (22%) had more than 300 beds. There were 427 facilities (63%) with policies recommending, but not requiring HCP to receive vaccination; 205 facilities (30%) reported mandatory influenza vaccination policies for all HCP. Of responding facilities, most had at least 2 years of experience collecting HCP influenza vaccination data; 508 facilities (75%) made changes to their data collection procedures to facilitate NHSN reporting. Most facilities used paper records to collect HCP vaccination status.

Most facilities had a mechanism for counting HCP; however, of the major data collection barriers, HCP not routinely present in the facility was most commonly reported (Table 1). Facilities with more than 300 beds were more likely to report major barriers to counting HCP not routinely in or rotating through the facility than smaller facilities, along with having no way to count LIPs or adult students/trainees.

TABLE 1.

Barriers Counting the Number of HCP: Overall and By Facility Bed Size

Barriers counting the number of HCP (overall)					Major barriers, by facility bed sizea
Variable	N	Not a barrier	Somewhat or moderate barrier	Major barrier	101–300 beds RR (95% CI), P value	>300 beds RR (95% CI), P value
		n (row %)	n (row %)	n (row %)
No way to count HCP
Employees	476	373 (78)b	87 (18)c	16 (3)d	1.0 (0.3–3.1), .97	1.3 (0.4–4.5), .69
LIPs	504	282 (56)b	155 (31)c	67 (13)d	1.4 (0.8–2.4), .30	2.4 (1.4–4.1), <.01
Adult students/trainees	480	248 (52)b	157 (33)c	75 (16)d	1.0 (0.6–1.9), .93	2.9 (1.8–4.7), <.01
Volunteers	469	324 (69)b	120 (26)c	25 (5)d	0.4 (0.1–1.3), .14	1.9 (0.8–4.2), .14
Required large time and/or resource commitment
	633	61 (10)	331 (52)	241 (38)	1.2 (0.9–1.5), .20	1.3 (0.9–1.6), .10
Other staff members or offices did not provide data on HCP
	583	122 (21)	298 (51)	163 (28)	1.7 (1.3–2.4), <.01	1.4 (0.9–2.1), .05
Counting HCP not routinely in facility
Employees	615	148 (24)e	271 (44)	196 (32)e	1.1 (0.9–1.5), .37	1.6 (1.2–2.1), <.01
LIPs	622	109 (18)e	255 (41)	258 (41)e	1.4 (1.1–1.7), .01	1.6 (1.3–2.0), <.01
Adult students/trainees	582	137 (24)e	255 (44)	190 (33)e	1.2 (0.9–1.6), .23	1.6 (1.2–2.1), <.01
Volunteers	577	210 (36)e	241 (42)	126 (22)e	1.2 (0.8–1.7), .48	1.8 (1.3–2.7), <.01
Counting HCP rotating through multiple facilities
Employees	529	205 (39)f	197 (37)	127 (24)g	1.2 (0.8–1.9), .28	2.0 (1.4–2.9), <.01
LIPs	522	170 (33)f	184 (35)	168 (32)g	1.4 (1.0–2.0), .04	2.1 (1.6–2.9), <.01
Adult students/trainees	494	196 (40)f	171 (35)	127 (26)g	1.7 (1.1–2.6), .02	2.8 (1.9–4.2), <.01
Volunteers	489	266 (54)f	151 (31)	72 (15)g	1.5 (0.8–2.7), .19	3.1 (1.8–5.4), <.01

NOTE. HCP, healthcare personnel; LIP, licensed independent practitioner; RR, rate ratio—percent of hospitals with 101–300 or >300 beds reporting the major barrier divided by the percent of hospitals with ≤100 beds reporting major barrier.

^aReference group is ≤100 beds.

Using the Marascuilo test, significant differences were found in the proportion of responding facilities for all pair-wise comparisons between:

^bAll HCP categories except LIPs compared with adult students.

^cEmployees to LIPs or adult students or adult volunteers.

^dAll HCP categories except LIPs compared with adult students and employees compared with volunteers.

^eAll HCP categories except employees compared with adult students.

^fAll HCP categories except employees compared with LIPs or students and LIPs compared with adult students.

^gAll HCP categories except employees compared with adult students and LIPs compared with adult students.

Collecting vaccination status data for employees was not a barrier for 396 facilities (68%), but 106 (18%) reported it as a major barrier for LIPs and 83 (15%) for adult students/trainees (Table 2). Facilities reported having barriers to reporting vaccination status due to other entities not providing data on HCP; in particular, 156 facilities (26%) reported this as major barrier for LIPs. Facilities with more than 300 beds were more likely to report major barriers to collecting vaccination status for LIPs and volunteers than smaller facilities.

TABLE 2.

Barriers Collecting HCP Vaccination Status: Overall and By Facility Bed Size, for 680 Respondents

Barriers collecting HCP vaccination status (overall)					Major barrier, by facility bed sizea
Variable	N	Not a barrier	Somewhat or moderate barrier	Major barrier	101–300 beds RR (95% CI), P value	>300 beds RR (95% CI), P value
		n (row %)	n (row %)	n (row %)
Not knowing how to get the vaccination status for HCP
Employees	582	396 (68)b	155 (27)c	31 (5)d	1.2 (0.5–2.7), .74	1.9 (0.8–4.4), .15
LIPs	588	270 (46)b	212 (36)c	106 (18)d	1.3 (0.9–2.1), .21	2.1 (1.4–3.2), <.01
Adult students/trainees	545	259 (48)b	203 (37)c	83 (15)d	1.1 (0.7–1.7), .78	1.5 (0.9–2.4), .11
Volunteers	552	333 (60)b	176 (32)c	43 (8)d	1.2 (0.6–2.7), .62	2.9 (1.4–5.9), <.01
Obtaining written documentation of vaccination given to HCP elsewhere
	627	121 (19)	364 (58)	142 (23)	1.0 (0.7–1.4), .91	1.2 (0.9–1.8), .25
Other staff members or offices did not provide data on HCP
Employees	589	228 (39)e	286 (49)f	75 (13)g	1.5 (0.9–2.4), .13	1.5 (0.8–2.6), .19
LIPs	597	138 (23)e	303 (51)f	156 (26)g	1.6 (1.1–2.2), <.01	1.5 (1.1–2.2), .02
Adult students/trainees	553	182 (33)e	282 (51)f	89 (16)g	1.3 (0.8–2.1), .26	2.0 (1.2–3.2), <.01
Volunteers	556	277 (50)e	223 (40)f	56 (10)g	1.4 (0.7–2.8), .28	2.6 (1.4–4.9), <.01
Receiving completed forms/surveys on vaccination status for HCP
Employees	620	199 (32)e	345 (56)h	76 (12)i	1.4 (0.8–2.2), .24	1.6 (0.9–2.7), .10
LIPs	606	94 (16)e	300 (50)	212 (35)i	1.6 (1.2–2.0), <.01	1.6 (1.2–2.2), <.01
Adult students/trainees	562	156 (28)e	299 (53)	107 (19)i	1.2 (0.8–1.8), .33	1.4 (0.9–2.2), .11
Volunteers	575	227 (39)e	276 (48)h	72 (13)i	1.2 (0.7–2.0), .53	1.7 (1.0–3.0), .04

NOTE. HCP, healthcare personnel; LIP, licensed independent practitioner; RR, rate ratio— percent of hospitals with 101–300 or >300 beds reporting the major barrier divided by the percent of hospitals with ≤100 beds reporting major barrier.

^aReference group is ≤100 beds.

Using the Marascuilo test, significant differences were found in the proportion of responding facilities for all pair-wise comparisons between:

^bAll HCP categories except LIPs compared with adult students.

^cEmployees compared with LIPs or students.

^dAll HCP categories except employees compared with adult volunteers and LIPs compared with adult students.

^eAll HCP categories except employees compared with adult students.

^fAll HCP categories except employees compared with LIPs or students and LIPs compared with adult students.

^gAll HCP categories except employees compared with adult students or volunteers.

^hEmployees compared with adult volunteers.

ⁱAll HCP categories except employees compared with adult volunteers.

Respondents perceived the accuracy of vaccination status data to be very or somewhat accurate for employees (n =639; 99%), LIPs (n =569; 89%), adult students/trainees (n =526; 83%), and adult volunteers (n =577; 90%). Facilities reporting major barriers to collecting data for LIPs, adult students/trainees, or adult volunteers were significantly more likely to have perceived their data to be inaccurate for these HCP categories compared with facilities reporting no major barriers.

Almost half of respondents reported that a benefit of collecting data for NHSN was that it helped communicate vaccination coverage to HCP (n = 333; 49%); more than 50% of facilities (n =401) responded that NHSN provides data for other reporting requirements, 44% (n =299) reported that it helped increase vaccination promotion efforts, 42% (n =286) reported that it helped improve the HCP tracking system, and 13% (n =88) reported no perceived benefit.

DISCUSSION

Although facilities reported data collection challenges for the HCP measure, in general, facilities found it easier to collect data on employees and adult volunteers, and smaller facilities reported fewer challenges. Factors such as size of facility, data collection methods, and facilities’ previous experience collecting data may all contribute to the ease of reporting vaccination coverage to NHSN and perceived accuracy of collected data.

Many facilities reported not having mechanisms for counting specific HCP groups; in particular, facilities had difficulty collecting data on HCP who do not routinely work in the facility. It may be difficult for data collectors to report on nonemployees because they do not have access to information owned by other entities, and the reporting facility may have no contractual obligation to report the data, which is consistent with our qualitative evaluation findings.⁴ Our findings complement other studies reporting that most organizations can collect data for only a subset of HCP, have difficulty gathering the HCP vaccination status for vaccinations received outside a facility, and have trouble assessing the number of nonemployees.^5,6

Most facilities in our evaluation used paper-based data collection mechanisms rather than electronic systems. Electronic data collection systems can increase the ease of collecting vaccination data and can provide additional benefits beyond collecting influenza vaccination coverage data: reports have found that these systems offer the flexibility of data use for other activities,^7,8 such as providing the ability to monitor vaccination coverage in real-time by location or occupation.⁹

Although most facilities in our evaluation had previous experience collecting data, they still faced a number of barriers. NHSN reporting requirements may be different than the information facilities have historically collected and more extensive than what facilities’ current data collection systems allow. A study from 2006 found that most hospitals measured influenza vaccination coverage for employees, but fewer hospitals measured coverage for nonemployee HCP or included coverage of vaccine refusal or contraindication.²

Limitations of this evaluation include not being able to determine how respondents defined data accuracy. In addition, the survey was designed to assess the experiences of data collectors in acute care hospitals; therefore, the results may not reflect the experience of reporting data to NHSN for other types of facilities.

The National Quality Forum endorsed the HCP measure with the expectation that it would drive improvement in data collection. A universal standard for HCP influenza vaccination coverage could help ease data collection and reporting for facilities; barriers to NHSN reporting may decrease as facilities develop processes and systems for collecting the necessary data. The measure has been adjusted since the first season of data collection to require facilities to submit data for any HCP working in the facility for at least 1 day; this change may result in a decrease in reported barriers to data collection because most facilities felt such a change would maintain or improve the accuracy of collected data. It is important to continue monitoring implementation of the HCP measure to ensure it is producing high-quality data that can be used to improve influenza vaccination coverage among HCP in the United States.