A Systems Approach to Improving Tdap Immunization Within 5 Community-Based Family Practice Settings: Working Differently (and Better) by Transforming the Structure and Process of Care

Abstract

Objectives. We examined how family medicine clinic physicians and staff worked in collaborative teams to implement an automated clinical reminder to improve tetanus, diphtheria, and acellular pertussis (Tdap) booster vaccine administration and documentation.

Methods. A clinical reminder was developed at 5 University of Michigan family medicine clinics to identify patients 11 to 64 years old who were in need of the Tdap booster vaccine. Quality improvement cycles were used to improve clinic care processes. Immunization rates from 2008 to 2011 were compared with rates at 4 primary care control clinics.

Results. Vaccination rates among eligible patients increased from 15.5% to 47.3% within the family medicine clinics and from 14.1% to 30.2% within the control clinics. After adjustment for covariates, family medicine patients had a higher probability of vaccination than control patients during each measurement period (0.17 vs 0.15 at baseline, 0.53 vs 0.22 during year 1, and 0.50 vs 0.30 during year 2).

Conclusions. Automated clinical reminders, when designed and implemented via a consensus-based framework that addresses the process of care, can dramatically improve provision of preventive health care.

The wise use of information and communications technology (ICT) has the potential to improve the quality and safety of health care while at the same time enhancing access and reducing waste, unnecessary delays, and administrative costs. Some degree of improvement results from use of the recent rapid and dramatic advances in moving information among a variety of clinicians and experts. Even more important over the long run, however, is the capacity of ICT to make it possible to work differently and better.—Institute of Medicine^1(p72)

Immunization schedules have grown increasingly complex in recent decades.^2–5 Automated clinical reminders (ACRs) at the point of care can help relieve this complexity by synthesizing a patient’s health history through the lens of evidence-based guidelines and presenting clinicians with patient-specific recommendations.⁶ A growing body of literature suggests that ACRs at the point of care not only can improve the quality of preventive and disease management services,^7–13 but may also enhance workflow efficiencies and lower treatment costs.^{9,10,12,14,15} Despite these potential benefits, there remains considerable variability in the use of ACRs.^10,16,17 Barriers to using ACRs include insufficient data management systems,^18–20 inadequate training or technical support,^21–24 disruptions of clinical workflows,^18,20,25,26 and excessive costs (including time) related to implementation or maintenance.^20,21,23,26

To be effective, systems designed to improve immunizations must do more than deliver ACRs; they must address the barriers just described—the structures and processes of care—and incorporate outcome data to facilitate continuous quality improvement (QI).²⁷ More specifically, they must account for and accommodate an environment’s infrastructure and available resources; attend to the process of care across the setting; accommodate the needs of clinicians, support staff, and patients; use information technology effectively; and employ real-time outcome measures to inform improvements.²⁸ In sum, the structures and processes of care must not simply be different, they must work better.

With this structure and process perspective in mind, 5 community-oriented family medicine (FM) clinics at the University of Michigan modified a point-of-care decision support system to improve administration and documentation of the tetanus, diphtheria, and acellular pertussis (Tdap) vaccine for patients 11 to 64 years of age. To accomplish this change, the clinic used a QI initiative to expand the interdependence between human resources (e.g., clinicians, support staff) and nonhuman technologies (e.g., equipment, procedures) such that administration and documentation became an automated process informed by each patient’s clinical history and treatment preferences. We hypothesized that FM patients would be more likely than patients receiving care at 4 internal medicine (IM) clinics not participating in the QI initiative to have received the booster vaccine during the intervention and follow-up years.

In this study, we contribute to the existing literature by describing the development and implementation of a prevention-oriented QI program in terms of its structures, processes, and outcomes. More specifically, we delineate how clinicians, nurses, medical assistants, and other support staff worked in collaborative and consensus-based teams to effectively use an ACR system to improve administration and documentation of the Tdap booster vaccine.

METHODS

Five University of Michigan FM clinics in and around the Ann Arbor area partnered in this initiative. Four of the university’s IM clinics located within the same geographic region were used as control sites.

Background

In 2006, the Advisory Committee on Immunization Practices²⁹ recommended that adolescents 11 to 18 years of age and adults 19 to 64 years of age receive a single dose of the Tdap vaccine for booster immunization in lieu of the tetanus and diphtheria vaccine.³⁰ Despite this recommendation, administration and documentation of Tdap booster immunizations within the 5 FM clinics remained low. As a result, a department-wide 2-year QI initiative was employed with the goal of improving performance.

Planning

Employing sequential and linked plan–do–study–act/adjust cycles,^31–33 this initiative sought to improve the data management system and clinical processes within the 5 FM clinics. Deficits in the existing Tdap surveillance and administration system were identified by clinicians, nurses, medical assistants, and support staff during department-wide and clinic-specific meetings.

Deficits were recognized in the tools and technology that facilitate care as well as clinical workflows. Identified problem areas included the absence of an ACR to trigger a review of each patient’s immunization status at every health care visit, the lack of a standardized method for documenting immunizations provided to patients outside of the university’s health system, and the absence of a uniform strategy to account for patients who might, in error, misrepresent their immunization status owing to confusion about the difference between the tetanus–diphtheria vaccine and the Tdap vaccine. Several areas related to structure and process were identified as needing improvement, including developing and implementing Tdap-specific rules and data feeds to the existing decision support system; creating Tdap-specific QI teams to increase collaboration and lessons learned both within and between clinics; employing rapid-cycle feedback, problem solving, and training to adjust (and readjust) clinic workflows; and instituting a reward system tied to clearly defined and measurable performance benchmarks.

Intervention Design

The initiative addressed both the structures and processes of care. In our context, structure refers to the physical environment and the context of care (i.e., the technology and tools used to deliver and document Tdap booster immunizations), and process refers to the actions and procedures associated with delivery and documentation of care (i.e., how immunizations were provided, workflow, and interactions within teams and between teams and the ACR system).^34,35

Structure.

In 2004, the University of Michigan’s FM department implemented a modifiable ACR system linked to a patient population management program to help facilitate the provision of evidence-based health care.⁷ This system addressed a number of evidence-based practices related to preventive care and chronic disease management. A key component of the system was its programmability; that is, ACRs could be readily tailored according to need, changes in guidelines, and clinician preferences. As a means of informing the development of a Tdap-specific ACR, clinicians, nurses, medical assistants, and support staff from each clinic were asked to provide input, including when the ACR should be triggered and how it should be tailored to best meet the clinical workflow. The goals of these discussions were to reach a general consensus on the content and format of a Tdap ACR and to maximize buy-in before its implementation.

Process.

Clinicians, nurses, medical assistants, and support staff were trained to use the newly developed Tdap ACR by means of an iterative and rapid-cycle orientation process within each clinic. Team members sought to clearly delineate their roles and tasks and to anticipate how changing roles or other system adjustments could potentially disrupt the clinic’s workflow. The ACR was developed so that clinicians could document their responses in 5 categories:

1. Tdap booster administered (during that clinic session), done (the patient reported already having received the booster), or ordered;

2. Tdap booster not indicated (e.g., contraindicated);

3. Tdap booster declined by patient;

4. Tdap booster discussed but no decision made; and

5. Tdap booster not addressed.

Importantly, the first category included all patients for whom the Tdap booster was ordered. If for any reason patients did not actually receive the booster (e.g., they left the clinic before it was administered), the ACR would be automatically retriggered at their next visit because their electronic health record (EHR) would still indicate the gap. Likewise, the first category also included patients who reported receiving their Tdap booster elsewhere. For these patients, this information was entered directly into their EHR, thus prohibiting the ACR from being retriggered during subsequent visits.

On a monthly basis, individual clinicians, patient care teams, clinic medical directors, and departmental leaders were provided status reports on the use of the ACR system, including clinicians’ responses. Collectively, the first 3 clinician response categories (administered/done/ordered, not indicated, and declined) were considered “acting on” an ACR, meaning that the clinician took direct action toward addressing the patient’s Tdap status. Providing any response to the ACR was classified as “responding to” the prompt, meaning simply that the ACR was not ignored. When the monthly data showed that a clinician was not responding to more than 20% of ACRs, a member of the department’s QI committee intervened and encouraged the clinician to review his or her Tdap-related processes with the care team. Every 3 months each clinic’s aggregate response to ACRs was reviewed by the team, and, if needed, adjustments to the workflow were made.

To help promote clinicians’ use of the ACR system, a pay-for-performance arrangement was instituted wherein clinicians could earn $1 in compensation per work relative value unit for acting on 80% or more of all ACR reminders. An additional $1 per work relative value unit was provided for responding to at least 90% of reminders.

Outcomes

Our QI study was conducted over 3 years. The Tdap ACR was developed during the 12-month baseline period (December 2008 through November 2009). The ACR was implemented between December 2009 and November 2010 (year 1), and follow-up took place from December 2010 through November 2011 (year 2). Because the processes associated with using the ACR for Tdap boosters were fully operational by year 2, there was no longer a need to continue the intensive, rapid-cycle feedback process during this time. Rather, emergent issues were addressed on an as-needed basis and in line with standard continuous QI processes within each clinic (e.g., by asking “How are we doing?” and “Can we do it better?”).³⁶ The pay-for-performance incentive was maintained throughout the follow-up year.

Documentation rates.

Tdap booster status was documented according to the proportion of patients 11 to 64 years of age who had at least 1 visit to a University of Michigan FM clinic within a given measurement period (baseline, year 1, or year 2) and whose EHR specified that they had received the booster.

ACR response rates.

We used data abstracted in real time from the ACR system during the intervention and follow-up years to calculate clinicians’ responses to the Tdap ACR. Patients for whom the Tdap booster had already been given or for whom it was administered/done/ordered were categorized as Tdap(+); patients for whom the Tdap booster vaccine was not indicated, was declined, or was discussed with no action taken were categorized as Tdap(−); and patients for whom the Tdap booster vaccine was not addressed (including cases in which the clinician did not respond to the ACR) were categorized as Tdap(NA).

Comparisons between clinics.

Tdap booster immunization rates across the 5 FM clinics were compared with rates at the 4 IM primary care clinics. Administration of the Tdap booster vaccine was measured retrospectively for each time period via de-identified information abstracted from patients’ EHRs. For each time period, 11- to 64-year-old patients who had had at least 1 encounter at 1 or more of the 9 university clinics were identified and the following data abstracted: whether the patient received the Tdap booster (including month and year of administration), number of appointments per specialty (FM and IM), year of birth, gender, race, marital status, and whether the patient received the influenza vaccine within a given 12-month measurement period (including month and year). To maximize differentiation between cohorts, we excluded patients who had appointments in both FM and IM clinics over the 3-year study period. Also excluded were patients with a record of Tdap vaccination before the start of the study. Tdap vaccination rates for eligible patients (i.e., patients without a record of Tdap vaccination at the start of each time period) were calculated for the baseline and follow-up periods and stratified by department (FM vs IM).

We conducted a conditional logistic regression analysis for discrete event time data³⁷ to analyze the effects of department at each year and covariates on Tdap administration among patients seeking care within FM or IM. We estimated marginal probabilities of vaccination for each department at each time point with a full model that included interactions of all covariates with department. The analysis was stratified by department to aid in interpretation of covariate effects. Estimates stratified by department are reported as odds ratios and 95% confidence intervals (CIs). Stata version 13 (StataCorp LP, College Station, TX) was used in conducting all of the analyses.

RESULTS

Reflecting the intervention design, results from this QI initiative are presented according to its structures, processes, and outcomes.

Structure

Informed by input from clinicians, nurses, medical assistants, and support staff, the Tdap ACR was designed to increase the number of patients receiving the Tdap booster vaccine as well as to make detailed documentation both quick and easy. Linked directly to patients’ EHRs, the ACR system automatically identified those lacking evidence that the Tdap booster had been administered; this link was activated whenever a patient initiated care at a University of Michigan FM clinic. The subsequent ACR not only prompted the clinician, nurse, or medical assistant to discuss the booster with targeted patients but enabled detailed documentation of the discussion’s outcome.

Among patients who obtained the booster outside of the university’s system, the ACR served as a reminder to document this information in the patient’s EHR. If patients did not know their booster status, they were treated as if they had not received the booster. The system also automatically identified patients with tetanus–diphtheria vaccine documentation only, thus serving as a cue to discuss with them the difference between the 2 vaccines.

Process

Although team members within each clinic tailored the Tdap-related workflow to meet the unique needs of their setting, there were key similarities in each clinic’s general approach. For example, when an ACR was activated, the medical assistant met with the patient and ascertained whether the vaccine had been administered elsewhere. If so, this information was entered into the EHR; if not, the Tdap vaccine was described and, after a discussion of contraindications and precautions, it was offered to eligible patients. If the patient agreed, the clinician ordered the vaccine in accordance with Centers for Disease Control and Prevention guidelines. The vaccine was then administered by the assistant, and administration was documented in the EHR. The ACR was closed when the clinician selected the corresponding outcome (e.g., administered/done/ordered). If a patient needing the booster did not receive it (e.g., he or she declined the booster), the ACR was automatically reactivated at the patient’s next visit.

Informed by monthly and quarterly performance data, clinic QI teams identified multiple process-related shortcomings: poor role clarification, technology-related failures, overlapping tasks, task overload, poor communication, and ineffective flow of information. Specific examples of shortcomings and their corresponding solutions are outlined in Table 1.

TABLE 1—

Problems and Solutions Identified Through Rapid-Cycle Feedback and Act/Adjust Strategies: University of Michigan Quality Improvement Initiative, 2008–2011

Process-Related Deficiency	Example Problem(s)	Example Solution(s)
Poor role clarification	Medical assistant ignoring ACR	Topic addressed/discussed during team meeting; one-on-one meeting between assistant and clinic’s quality improvement lead
Technology-related failures	Printer malfunction	Training on alternate printing sites within clinic; training on accessing information technology support services
Overlapping tasks/task overload	Clinician ignoring ACR owing to clinic back-ups/service delays	Topic addressed/discussed during team meeting; one-on-one meeting between clinician and clinic’s (or department’s) quality improvement lead; proper closing of ACR via deliberately not addressed option and action taken at subsequent encounter with patient
	Patient has multiple compelling issues that take priority over Tdap vaccination	Proper closing of ACR via deliberately not addressed option and action taken at subsequent encounter with patient
Ineffective communication/information flow	Patient agrees to Tdap and medical assistant documents patient preference, but clinician fails to place order	Development of specific criteria for standing orders; topic addressed/discussed during team meeting; one-on-one meeting between clinician and clinic’s (or department’s) quality improvement lead

Note. ACR = automated clinical reminder; Tdap = tetanus, diphtheria, and acellular pertussis vaccine.

By the end of the intervention year (year 1), process-related problems were almost always attributable to an individual’s action or inaction (i.e., human-related factors) as opposed to the ACR technology. These problems were addressed in monthly team meetings; however, if the problem persisted, the clinic’s or department’s QI lead intervened by having a one-on-one conversation with the responsible individual. On a quarterly basis, aggregate performance reports were used to help teams benchmark their progress, and comparisons between clinics helped to identify the most successful processes. Qualities associated with success included clear delineation of team members’ roles, open communication between team members, flexibility and willingness to make changes as new evidence became available, and ensuring that team members had both proper training and the requisite tools to do their job.

A subtle yet extremely important component of this intervention was development of the appropriate culture. A purposeful effort was made during development, implementation, adjustment cycles, and the evaluation to ensure that the initiative maintained a positive, cooperative, and goal-focused spirit. Process-related problems—whether stemming from workflow-related oversight or a particular individual’s mistake—were never treated as failures; instead, they were acknowledged as a normal part of learning, working differently, and striving to do better.

Outcomes

Table 2 outlines cumulative Tdap booster documentation within FM clinics as well as FM clinicians’ responses to ACRs. Over the 3-year period, the cumulative percentage of patients who received the Tdap booster more than doubled, from 33.8% at baseline to 76.8% during year 2. The biggest single-year change was observed in year 1, when the rate jumped 34.2 percentage points over baseline; by contrast, the rate jumped 8.8 percentage points from year 1 to year 2. Also as shown in Table 2, the percentage of Tdap(+) responses increased from 42.0% in year 1 to 59.0% in year 2. Moreover, the proportion of Tdap(NA) responses fell by nearly 60%, from 27.1% in year 1 to 11.0% in year 2. The percentage of Tdap(−) responses remained nearly flat between years 1 and 2, at 30.9% and 30.0%, respectively.

TABLE 2—

Tdap Booster Documentation and Family Medicine Clinicians’ Response to Automated Clinical Reminders: University of Michigan Quality Improvement Initiative, 2008–2011

	Baseline, No. or No. (%)	Intervention Year, No. or No. (%)	Follow-Up, No. or No. (%)
Tdap documentationa
Patient populationb	33 371	33 131	34 473
Patient age group, y
11–18	3 331	3 347	3 278
19–64	30 040	29 784	31 195
Patients receiving Tdap	11 264 (33.8)	22 535 (68.0)	26 486 (76.8)
Receipt of Tdap by age group, y
11–18	1 587 (47.6)	2 610 (78.0)	2 772 (84.6)
19–64	9 677 (32.2)	19 925 (66.9)	23 715 (76.0)
Clinicians’ response to automated clinical remindersc
No. of Tdap reminders	. . .	16 273	21 776
Tdap(+)d	. . .	6 828 (42.0)	12 848 (59.0)
Tdap(−)e	. . .	5 030 (30.9)	6 533 (30.0)
Tdap(NA)f	. . .	4 415 (27.1)	2 395 (11.0)

Note. Tdap = tetanus, diphtheria, and acellular pertussis vaccine.

^aBased on retrospective electronic health record data abstracted in July 2014.

^bPatients 11–64 years of age who received care at 1 or more University of Michigan family medicine clinics during the measurement period.

^cBased on data abstracted in real time directly from the automated clinical reminder system during each year of the study.

^dTdap booster already given, administered/done, or ordered.

^eTdap booster not indicated, declined by patient, or discussed but no decision made.

^fTdap booster deliberately not addressed or clinician did not respond to automated clinical reminder.

To compare Tdap booster administration at FM and IM clinics, we abstracted de-identified EHR data from 87 079 patients who received care at the 9 clinics (5 FM, 4 IM) over the 3-year study period. Of these patients, 1288 were excluded because they had received care at both an FM clinic and an IM clinic. An additional 17 877 patients were excluded because they had a prior record of Tdap administration (9985 at FM clinics and 7892 at IM clinics). Descriptive statistics for the remaining 67 914 patients are outlined in Table 3. Given the extremely large sample, statistically significant differences were detected for all demographic categories. The shapes of the population pyramids (data not shown) for FM and IM clinics were similar, with the largest FM cohort being in their 30s and early 40s and the largest IM cohort in their 40s and early 50s. As can be seen in Table 3, vaccination rates among eligible patients increased from 15.5% to 47.3% within FM clinics and from 14.1% to 30.2% within IM clinics.

TABLE 3—

Demographic Characteristics and Vaccination Statistics Among Family Medicine and Internal Medicine Patients: University of Michigan Quality Improvement Initiative, 2008–2011

Variable	Totala (n = 67 914), No., No. (%), or Mean ±SD	FM Patients (n = 39 882), No., No. (%), or Mean ±SD	IM Patients (n = 28 032), No., No. (%), or Mean ±SD
Demographic variables
Marital statusb
Married	38 088 (56.9)	20 679 (52.9)	17 409 (62.4)
Single	26 999 (40.3)	17 344 (44.4)	9 655 (34.6)
Other	1 895 (2.8)	1 062 (2.7)	833 (3.0)
Racec
White	49 251 (74.9)	27 859 (72.4)	21 392 (78.6)
Black	7 437 (11.3)	4 988 (13.0)	2 449 (9.0)
Asian	6 817 (10.4)	4 100 (10.6)	2 717 (10.0)
Other	2 215 (3.4)	1 552 (4.0)	663 (2.4)
Gender
Female	38 348 (56.5)	22 292 (55.9)	16 056 (57.3)
Male	29 566 (43.5)	17 590 (44.1)	11 976 (42.7)
Age, y
< 18	4 164 (6.1)	3 395 (8.5)	769 (2.7)
≥ 18	63 750 (93.9)	36 487 (91.5)	27 263 (97.3)
Age, y	39.3 ±13.8	37.4 ±13.8	42.0 ±13.2
Time-varying variables
No. of patients eligible for Tdap vaccined
Baseline	44 171	25 584	18 587
Intervention year	39 608	22 565	17 043
Follow-up	29 143	14 748	14 395
Patients receiving Tdap vaccination
Baseline	6 599 (14.9)	3 976 (15.5)	2 623 (14.1)
Intervention year	16 073 (40.6)	12 267 (54.4)	3 806 (22.3)
Follow-up	11 321 (38.9)	6 978 (47.3)	4 343 (30.2)
Patients receiving flu vaccination
Baseline	13 605 (30.8)	6 867 (26.8)	6 738 (36.3)
Intervention year	17 498 (44.2)	9 301 (41.2)	8 197 (48.1)
Follow-up	11 160 (38.3)	4 417 (30.0)	6 743 (46.8)
No. of visits per patient
Baseline	2.8 ±2.6	3.0 ±2.9	2.5 ±2.0
Intervention year	2.7 ±2.5	2.9 ±2.7	2.4 ±2.2
Follow-up	2.6 ±2.4	2.6 ±2.4	2.5 ±2.4

Note. FM = family medicine; IM = internal medicine; Tdap = tetanus, diphtheria, and acellular pertussis vaccine. Values are based on retrospective electronic health record data abstracted in July 2014. The sample was limited to patients having no record of a Tdap booster vaccination prior to December 1, 2008. Patients with appointments in both family medicine and internal medicine clinics over the 3-year study period were excluded. All differences between family medicine and internal medicine patients were significant at P < .001.

^aNonduplicative count of all patients over the 3-year study period without a record of having received a Tdap vaccination as of the start of the baseline period (December 1, 2008).

^bData on marital status were missing for 932 patients.

^cData on race were missing for 2194 patients.

^dEligible patients included those who had not received a previous Tdap vaccination and had an appointment during the given time frame.

After control for age, gender, marital status, race, number of health care visits in a given year, and flu shot administration, FM patients had a higher probability of vaccination than IM patients during each measurement period: 0.165 (95% CI = 0.160, 0.170) vs 0.150 (95% CI = 0.144, 0.155) at baseline, 0.534 (95% CI = 0.528, 0.541) vs 0.223 (95% CI = 0.217, 0.229) during year 1, and 0.496 (95% CI = 0.488, 0.504) vs 0.303 (95% CI = 0.296, 0.311) during year 2. Notably, whereas the probability of vaccination among FM patients during years 1 and 2 was markedly higher than the probability at baseline, a slight dip was observed in year 2. By contrast, although IM patients’ improvement was much smaller, no decline was observed in year 2.

Table 4 shows the effects of our covariates on vaccination across the 2 departments. In the case of both FM and IM patients, the odds of receiving the Tdap booster were elevated among those who received the flu vaccine in a given year, as well as among those younger than 18 years. The effect of age was much stronger among IM patients than FM patients. With the exception of IM Asian patients, Whites were more likely than others to have received the vaccine. Female IM patients had higher odds of receiving the Tdap booster than male IM patients, but the opposite was true for FM patients.

TABLE 4—

Effects of Time Period and Demographic Characteristics on Tdap Vaccination, by Department: University of Michigan Quality Improvement Initiative, 2008–2011

Variable	FM Patients (n = 38 234), ORa (95% CI)	IM Patients (n = 27 178), ORa (95% CI)
Time period
Baseline (Ref)	1.00	1.00
Intervention year	6.51 (6.22, 6.81)	1.67 (1.58, 1.77)
Follow-up	5.52 (5.25, 5.80)	2.59 (2.45, 2.74)
Marital statusb
Married (Ref)	1.00	1.00
Single	0.82 (0.79, 0.89)	0.86 (0.82, 0.90)
Other	0.77 (0.69, 0.86)	0.80 (0.71, 0.92)
Racec
White (Ref)	1.00	1.00
Black	0.84 (0.79, 0.89)	0.82 (0.76, 0.89)
Asian	0.63 (0.59, 0.67)	1.20 (1.12, 1.29)
Other	0.53 (0.48, 0.59)	0.98 (0.84, 1.13)
Age, y
< 18 (Ref)	1.00	1.00
≥ 18	0.45 (0.42, 0.49)	0.19 (0.17, 0.22)
Gender
Male (Ref)	1.00	1.00
Female	0.85 (0.82, 0.88)	1.45 (1.38, 1.52)
Receipt of flu shot
No (Ref)	1.00	1.00
Yes	2.36 (2.27, 2.45)	1.74 (1.66, 1.83)
No. of office visits	1.08 (1.07, 1.09)	0.99 (0.98, 1.01)

Note. CI = confidence interval; FM = family medicine; IM = internal medicine; OR = odds ratio; Tdap = tetanus, diphtheria, and acellular pertussis vaccine. Values are based on retrospective electronic health record data abstracted in July 2014. Odds ratios were calculated in a discrete time survival analysis via conditional logistic regression and incorporated information for all patients starting from the time they were first included in the data sample until they received the Tdap booster or until the end of the study. The sample was limited to patients with no record of a Tdap booster vaccination prior to December 1, 2008. Patients with appointments in both family medicine and internal medicine clinics over the 3-year study period were excluded.

^aCIs that do not contain a value of 1 correspond to statistically significant effects (all Ps < .001).

^bData on marital status were missing for 932 patients.

^cData on race were missing for 2194 patients.

DISCUSSION

Because ACRs do not have direct effects on either health status or disease processes, their utility stems from how well they improve quality within the milieu (structure and process) of care.^10,38 Improving structures and processes is difficult, as it requires making changes to the way clinics—including the people who work within them—are organized.^28,39 Although ACRs are an important part of this reorganization,^28,40,41 their utility lessens in the absence of flexibility and interface usability.^7,42–47 To be effective, ACRs must neither dictate care nor intrude on the clinical encounter. Rather, they need to augment clinicians’ knowledge, skills, and experience; be tailored to the unique needs of a clinic; and help facilitate provision of high-quality care while also improving efficiencies.^7,28,48,49 The Tdap ACR developed for our initiative captured these qualities, both maximizing the human–technology interface and simplifying workflow processes.

A key component of the initiative’s success stemmed from developing a consensus on the parameters for the ACR and its subsequent workflow before the ACR’s implementation; this process undercut potential resistance and facilitated a sense of commitment to achieving a shared goal. Adoption of the ACR led to successful reengineering of the broad set of clinical activities associated with the Tdap booster and to improvements in administration and documentation of the Tdap vaccine. At the end of the study (2011), 76.0% of adult FM patients 19 to 64 years of age and 84.6% of adolescent patients 11 to 18 years of age had received the Tdap vaccine. By contrast, Tdap rates among IM patients were 58.6% and 85.6%, respectively. Although IM clinics performed slightly better than FM clinics among the younger cohort, FM clinics performed much better among the older cohort. Notably, both FM and IM clinics performed markedly better than the estimated national Tdap vaccination rates of 12.5% among adults (19–64 years of age) and 78.2% among adolescents^50,51 (note that the national rate for adolescents includes those 13–17 years of age only; among FM and IM patients within the same age range, Tdap vaccination rates in year 2 were 89.1% and 92.1%, respectively).

Limitations

Our initiative involved a number of limitations. First, the 5 FM clinics were all affiliated with the University of Michigan’s FM department. As such, the clinics shared a common culture wherein research is valued, quality-improvement projects are broadly supported, and the adoption of information and communications technology is welcomed. Moreover, the FM clinics operated within an integrated health system, an institutional arrangement independently associated with successful execution of QI projects and higher performance on quality indicators.^52–54 Consequently, our findings may not be easily generalized to smaller group practice settings, independently operated clinics, or clinics not operating within an integrated network.

Second, although there was a strongly shared culture across the 5 FM clinics, small differences between clinics may have influenced some of our estimated standard errors. However, because clinic-level information was not available, we were unable to control for this potential source of heterogeneity. Given the shared culture, one can reasonably speculate that the effects of such differences on the outcomes observed would have been relatively small.

Third, the FM clinics had an existing and fully functional EHR system and a modifiable decision support system before the start of the project; hence, the clinicians, assistants, and staff were already comfortable with using the technology, enabling them to focus exclusively on the structures and processes associated with the Tdap ACR (as opposed to focusing on implementing or learning about the technology itself). Given that only about a quarter of US primary care physicians practice in settings with fully functional EHRs, and another third or so practice at sites with no EHRs,⁵⁵ implementation of ACR systems may not be feasible for many physicians.

Finally, because guideline penetration, guideline education and outreach efforts, and clinician responsiveness to new guidelines improve over time,^40,56,57 passage of time itself may have contributed to our findings. Our use of 4 IM primary care clinics within the same parent institution helped control for this potential source of bias.

Conclusions

Although our initiative focused on improving Tdap booster vaccination rates, the general methods and corresponding emphasis on structures, processes, and outcomes can be applied to many QI initiatives, including preventive care, disease screening, and chronic disease management. A key to improving administration of the Tdap booster vaccine was developing a QI culture that emphasized teamwork, cooperation, and respect. Ideas to improve the system were welcomed from all team members, whether medical directors, clinicians, medical assistants, or support staff.

By involving all key stakeholders, attending to workflows, clearly defining roles and responsibilities, and measuring progress on a regular basis, this QI initiative did much more than facilitate vaccinations and improve documentation; it expanded the level of interdependence between human resources and nonhuman technologies and changed the structure and process of care. Subsequent to this initiative, care teams within the 5 FM clinics worked not only differently but better: Tdap boosters were administered and documented automatically, as a routine part of care, thus dramatically changing and improving service provision at these clinics.