Improving Adult Immunization Practices Using a Team Approach in the Primary Care Setting

Abstract

Objectives. The objective of this study was to improve the immunization rates of primary care practices using a team approach.

Methods. Practices performed 35 random chart abstractions at 2 time points and completed a survey about immunizations at baseline and 12 months after intervention. Data were collected for the following immunizations: influenza, pneumococcal, tetanus diphtheria (Td)/tetanus diphtheria pertussis (Tdap), hepatitis A, hepatitis B, meningococcal, varicella, herpes zoster, and human papilloma virus. Between baseline and after intervention, practice teams were given feedback reports and access to an online educational tool, and attended quality improvement coaching conference calls.

Results. Statistically significant improvements were seen for Td/Tdap (45.6% pre-intervention, 55.0% post-intervention; P ≤ .01), herpes zoster (12.3% pre-intervention, 19.3% post-intervention; P ≤ .01), and pneumococcal (52.2% pre-intervention, 74.5% post-intervention; P ≤ .01) immunizations. Data also revealed an increase in the number of physicians who discussed herpes zoster and pneumococcal vaccinations with their patients (23.2% pre-intervention, 43.3% post-intervention; P ≤ .01 and 19.9% pre-intervention, 43.0% post-intervention; P ≤ .01, respectively) as well as an increase in physicians using the Centers for Disease Control and Prevention immunization schedule (52.9% pre-intervention, 88.2% post-intervention; P ≤ .02).

Conclusions. The immunization rates of the primary care practices involved in this study improved.

The need for improving quality is pervasive in the primary care setting, involving physicians, their practice teams, and administrative staff. The issue of low quality is well documented^1–3 and is not partial to any 1 disease condition.^4–15 Poor quality is a result of our medical system’s orientation to the urgent, its focus on acute and not chronic care, lack of adherence to evidence-based guidelines, and an increasing number of patients with complex medical conditions.² Quality is characterized as a systems issue rather than an individual one,¹⁶ which has led efforts to focus on the practice team. Practice teams have been shown to improve quality in primary care.^17,18 The issues with poor quality in primary care extend to the practice of adult immunizations.¹⁹ It is estimated that between 50 000 and 70 000 US adults die each year because of diseases that could be prevented by vaccination.²⁰ For example, influenza is the sixth leading cause of death for adults and contributes to at least 200 000 hospitalizations and 36 000 deaths annually.^21,22 Economic costs associated with influenza are projected to be $87.1 billion.²³

Adult vaccination guidelines, such as those published by the Centers for Disease Control and Prevention (CDC) and Advisory Committee on Immunization Practices,²⁴ are increasingly evidence-based and are a good reference for practices to measure themselves against when doing immunization practice redesign work. Although childhood vaccinations have become a public health success, adult vaccination rates are low, prompting the movement toward “lifespan immunizations.”^20,25 However, quality gaps and missed opportunities for vaccination exist between the number of patients who are recommended to receive vaccinations and those who actually receive them.^26–30 A variety of barriers at the practice, patient, economic, and social level help explain these missed opportunities. For instance, only 60% of physicians reported using CDC and Advisory Committee on Immunization Practice guidelines as their reference for adult immunizations, and most often reported recommending vaccinations at well visits compared with sick visits.³¹ Physicians also reported multiple barriers to vaccinating patients, including lack of health insurance, fear of needles, and misconception of the safety and efficacy of vaccinations.³¹ In turn, patients consistently reported that their physicians do not recommend vaccinations.^31,32

A comprehensive quality approach was considered to be more effective than mere guideline dissemination because the latter has not been shown to be successful alone in changing practice patterns.^33,34 The American College of Physicians (ACP) developed this quality improvement program to help physicians and practice teams learn about the current recommendations and best practices for adult immunization. The goal of this prospective study was to improve the immunization practices of primary care practices by using a team approach.

METHODS

Project participants were recruited from ACPNet, the ACP quality and practice improvement network. (ACPNet represented approximately 1000 practices at the time of this program.) A total of 132 practices showed a preliminary interest in participating in this project. Of this group, 20 practices enrolled, and 17 completed the project. These 17 practices each had a practice team of 4 members, bringing the total number of program participants to 68.

Each practice designed their own practice team, although program faculty recommended at least 1 front office staff and 1 nurse, medical assistant, or physician’s assistant. Most practice teams consisted of 4 individuals, which included the physician, their office manager, 1 front office staff, and 1 nurse or other allied health professional. They attended coaching conference calls as a team and were urged to meet regularly to discuss their practice improvement work. Practices completed the patient chart abstractions, and physicians completed the practice pattern survey data.

Two faculty members were contracted to develop the physician practice pattern survey, educational program, and serve as quality improvement coaches on the conference calls. Both were practicing clinicians who were also experts in the field of practice redesign and quality improvement. These consultants had previously enrolled in and successfully completed multiple ACP quality improvement programs and acted as quality improvement coaches on previous projects.

Data

Data were collected twice during this study and were gathered from 2 sources: (1) the physicians practice pattern survey and (2) patient chart abstraction form. The Likert-scale response physician practice pattern survey captured what physicians believed they were doing, whereas the patient chart abstraction tool assessed what the practice actually did.

The purpose of this unique survey was to measure physicians’ knowledge, attitudes, and behaviors regarding their immunization practices. Specifically, the 42 questions in the practice pattern survey addressed: (1) practice patterns related to adult immunization, (2) patient education, and (3) knowledge of immunization guidelines. The physician practice pattern survey was pilot tested and revised before distribution to program participants. A baseline (pre-intervention) measurement was taken at the beginning of the study and the follow-up (post-intervention) measurement was taken approximately 12 months after intervention. The reason for 2 points of measurement was to evaluate if the educational program was helpful in changing the knowledge, attitudes, and beliefs (from the practice champion’s point of view) about the care being delivered by the practice.

Practice data were randomly collected twice through a patient chart abstraction tool: the pre-intervention measurement was collected at the beginning of the study (December 2009), and the post-intervention measurement was collected after the intervention, approximately 12 months after the pre-intervention measurement (December 2010). The patient chart abstraction tool was designed to reflect the CDC adult immunization schedule. For the baseline data assessment, participants were asked to randomly select charts using simple random sampling from 12 months before the date the program began, accounting for historical control.

The patient chart abstraction tool included the following data:

1. patient demographic information (gender, risk factors, race, ethnicity, insurance),

2. type of vaccine,

3. date given,

4. if vaccinated elsewhere (date and where),

5. reason for patient not vaccinated (if discussed with patient but not administered, reason for refusal, if vaccine not in stock), and

6. the type of visit when the vaccination was given (annual, follow-up, acute).

The vaccines listed on the patient chart abstraction form included influenza, pneumococcal, tetanus diphtheria (Td)/tetanus diphtheria pertussis (Tdap), human papilloma virus, meningococcal, herpes zoster, hepatitis A, complete hepatitis B series (all 3 doses), varicella, and an option for other vaccines. The main goal of the patient chart abstraction tool was to provide a “snapshot” of the practices’ immunization practices before and after the intervention, allowing them to reflect these data throughout the project as motivators for improvement. Each practice collected data on 35 randomly selected patients from their practice who were aged 18 years or older and who had been under their care for at least 1 year. Eligibility criteria for patients under care for 1 year were determined using the data field “date of first office visit to physician.”

Confidential feedback reports based on these patient chart abstraction forms were provided to practices with information on the practice measures for adult immunization. Practices used these data to develop improvement strategies (action plans). The intervention for this project was a combination of the online educational program as well as the coaching conference calls.

Intervention

After the baseline physician practice pattern survey and patient chart abstractions were completed, participants were given access to the online educational program. The educational program included training on (quality improvement) techniques; decision support tools for practice improvement; information on the chronic care model; the plan–do–study–act (PDSA) cycle; and practice improvement tools, such as patient education materials and the CDC immunization schedule.

Conference calls with the practices aided physicians and their teams in setting goals and helped them with the development and implementation of a quality improvement strategy. Two types of conference calls were offered: beginner and advanced. The beginner conference calls were didactic in nature and offered instruction in the Chronic Care Model and the PDSA Cycle for Improvement, and how they could be applied to immunization services in the office setting. The advanced conference calls featured a more informal format, with one-on-one coaching from the program faculty. These advanced calls also offered opportunities for peer-to-peer coaching, prompting the participants to become resources for each other, sharing failures and successes of their own quality improvement work.

The calls encouraged participants to emphasize system and practice behavior rather than focusing solely on physician behavior. A prepared practice team was defined as one that had patient information, decision support, people, equipment, and time to deliver evidence-based medicine. Aside from assessing vital signs, the quality improvement experts suggested that practice teams assess a patient’s health literacy and confidence with their health care decisions, both of which are important in vaccine counseling. Delivery system redesign was another element of the intervention. This involved redefining staff roles and responsibilities, and distributing those tasks among team members to align all staff effort toward better clinical care. Clinical decision support tools (checklists, standing orders, computer-based reminders, clinical pathways) were also provided to the practices. Practices set aims (with a numerical goal and time frame), established measures (process), and selected and tested changes (PDSA cycle). Most importantly, practice teams were taught that change should be done in small steps because it was easier to accomplish, measure, and sustain.

Run charts (time series graphs) of small numbers of regularly collected immunization data were encouraged to track quality improvement progress during the program and to identify events that illustrated sustained improvement. Measurement strategies included developing visual displays of data for the practice to view, plotting key measures in time order on a regular basis, and conserving measurement resources through integration into daily work. A rule of thumb in the program was to start small; measurement should speed and not impede improvement. To accomplish this, practices collected data while performing normal work activities, such as rooming patients, preparing charts, or calling for appointment reminders. Practice champions regularly reviewed data with those who collected it, and clearly defined roles and responsibilities for data collection.

The null hypothesis for this study was that the intervention would have no impact on the immunization rates of the practice after the intervention. De-identified chart data and physician practice pattern survey data were housed in 2 databases for analysis. The unit of analysis for the patient chart abstraction was the practice, and the unit of analysis for the physician practice pattern survey data was the physician. Quantitative data analysis was performed using SPSS 19.0.1 (SPSS Inc., Chicago, Illinois). Chi-square tests and analysis of variance tests were used to analyze the patient chart abstraction and physician practice pattern survey data. All P values ≤ 0.05 were considered statistically significant. If P values were not reported, the difference was not statistically significant.

RESULTS

From the 17 practices who participated in the program, a total of 1190 patient chart abstractions were submitted. Half of the practices had an electronic medical record (EMR) system in their practice at pre-intervention, and 64.7% had an EMR at post-intervention. Pre- and post-intervention patient groups did not differ significantly in race, age, ethnicity, or gender. Most of the patients were White, non-Hispanic, and had Medicare or commercial insurance. More than half of the patients (57.8% pre-intervention, 58.7% post-intervention) had hypertension, and more than a quarter had diabetes (29.1% pre-intervention, 28.6% post-intervention; Table 1). Aside from aggregate pre- and post-intervention comparisons, the data were evaluated at the individual practice level before and after intervention to ensure there was no difference in patient populations at both time points that would affect immunization rates.

TABLE 1—

Percentage of Patients With Medical Conditions, Improving Adult Immunization Practices Using a Team Approach, 2009–2010

Medical Conditions	Pre-Intervention (n = 595), %	Post-Intervention (n = 595), %	P	Indicated Vaccinations
Asplenia	1.2	1.7	.46	Influenza, pneumoccocal, meningococcal
Chronic alcoholism	1.3	2.0	.37	Influenza, pneumoccocal
Chronic liver disease	2.2	3.9	.09	Influenza, pneumoccocal, hepatitis A, hepatitis B
Chronic lung disease	15.6	14.8	.69	Influenza, pneumoccocal
Diabetes	29.1	28.6	.85	Influenza, pneumoccocal
Heart disease	21.8	23.0	.63	Influenza, Pneumoccocal
Immunodeficiency	4.2	4.5	.78	Influenza, pneumoccocal, hepatitis B
Kidney failure	3.9	5.5	.17	Influenza, pneumoccocal
Smoker	16.6	16.5	.94	Pneumoccocal

The denominator for all reported survey data was 17. Statistically significant improvement was found in physician self-reported immunization rates for pertussis (26.47% pre-intervention, 54.41% post-intervention; P ≤ .01) and tetanus (50.00% pre-intervention, 66.18% post-intervention; P ≤ .05). The number of physicians who reported using the latest CDC adult immunization schedule on an annual basis increased significantly (52.9% pre-intervention, 88.2% post-intervention; P ≤ .02). There was a statistically significant increase in the number of physicians who reported all staff received their annual flu shot (72.06% pre-intervention, 89.71% post-intervention; P ≤ .006). Similarly, there was a statistically significant improvement seen in the number of physicians who reported that their staff was up to date with Td/Tdap vaccinations (63.24% pre-intervention, 79.41% post-intervention; P ≤ .05).

Patient Chart Abstraction Data

Data were analyzed to examine if patient vaccination rates for indicated conditions changed from before to after intervention (Tables 2 and 3). Results showed statistically significant improvements in pneumococcal vaccinations for patients with chronic lung disease (73.8%, 89.7%; P ≤ .01), diabetes (55.6%, 68.8%; P ≤ .01), and heart disease (56.3%, 86.3%; P ≤ .01). Overall vaccination rates statistically improved for pneumococcal (52.2% pre-intervention, 74.5% post-intervention; P ≤ .01), Td/Tdap (45.6% pre-intervention, 55.0% post-intervention; P ≤ .01), and herpes zoster (12.3% pre-intervention, 19.3% post-intervention; P ≤ .01).

TABLE 2—

Vaccination by Disease Conditions, Improving Adult Immunization Practices Using a Team Approach, 2009–2010

	Received Vaccination	Influenza Vaccination					Meningococcal Vaccination					Pneumococcal Vaccination
Patient Condition		Pre, %	No.	Post, %	No.	P	Pre, %	No.	Post, %	No.	P	Pre, %	No.	Post, %	No.	P
Chronic lung disease	Yes	69.9	83	77.0	74	.31	…	…	…	…	…	73.8	84	89.7	78	< .01
	No	30.1		23.0			…		…			26.2		10.3
Diabetes	Yes	75.2	161	84.9	152	.03	…	…	…	…	…	55.6	169	68.8	161	< .01
	No	24.8		15.1			…		…			44.4		31.2
Heart disease	Yes	73.5	117	83.1	124	.07	…	…	…	…	…	56.3	128	86.3	131	< .01
	No	26.5		16.9			…		…			43.7		13.7
Smoker	Yes	…	…	…	…	…	…	…	…	…	…	64.7	85	70.7	75	.42
	No	…		…			…		…			35.3		29.3
Othera	Yes	68.2	66	72.8	81	.54	0.0	23	10.0	10	.12	53.8	65	80.7	83	< .01
	No	31.8		27.2			100.0		90.0			46.2		19.3

^aOther patient conditions include asplenia, chronic alcoholism, chronic liver disease, immunodeficiency, and kidney failure.

TABLE 3—

Vaccination by Disease Conditions, Improving Adult Immunization Practices Using a Team Approach, 2009–2010

	Received Vaccination	Complete Hepatitis B Vaccination					Hepatitis A Vaccination
Patient Condition		Pre, %	No.	Post, %	No.	P	Pre, %	No.	Post, %	No.	P
Chronic liver disease	Yes	10.0	10	47.1	17	.05	11.1	9	12.5	8	.93
	No	90.0		52.9			88.9		87.5
Othera	Yes	20.9	43	37.2	43	.1	3.1	32	13.6	22	.15
	No	79.1		62.8			96.9		86.4

^aOther patient conditions include asplenia, chronic alcoholism, chronic liver disease, immunodeficiency, and kidney failure.

We examined differences pre- and post-intervention for office visits at which the vaccination was administered and found several statistically significant findings. (For each vaccination, the numerator was the number of patients vaccinated during the type of office visit; the denominator consisted of the total number of patients seen at that type of office visit, with the exclusion of those patients for whom the vaccination was noted by the physician as not indicated.) There was an increase from pre- to post-intervention (3.3% pre-intervention, 62.7% post-intervention) in hepatitis B vaccinations given at annual visits (P ≤ .01). A statistically significant improvement was seen in physicians administering the influenza vaccination at annual visits from 7.5% before intervention to 26% after intervention (P ≤ .01). In addition, there were statistically significant improvements seen in physicians administering the Td/Tdap vaccination at annual visits (from 20.1% pre-intervention to 41.7% post-intervention; P ≤ .01) and the herpes zoster vaccinations given at annual visits (10.3% pre-intervention, 35.3% post-intervention; P ≤ .04). Statistically significant improvements were seen in physicians discussing herpes zoster vaccination with their patients, from 23.2% to 43.3% (P ≤ .01). Lastly, a statistically significant improvement was seen across the 2 data points regarding physicians discussing the pneumococcal vaccination with their patients (19.9% pre-intervention, 43.0% post-intervention; P ≤ .01).

The association between practice vaccination rates and their quality improvement goals were also analyzed, with results showing that goals precipitated different improvements for various immunizations (Table 4). Practices that focused on increasing influenza vaccination rates had statistically significant improvements in influenza (66.1% pre-intervention, 77.9% post-intervention; P ≤ .01), pneumococcal (54.4% pre-intervention, 82.8% post-intervention; P ≤ .01), and Td/Tdap (39.2% pre-intervention, 56.0% post-intervention; P ≤ .01). Practices with this goal had influenza and pneumococcal rates that exceeded national immunizations rates³⁵ (68.8%, 65.6%, respectively). Practices that focused on increasing pneumococcal vaccination rates had statistically significant and higher than nationally reported rates³⁵ in influenza (61.3% pre-intervention, 71.3% post-intervention; P ≤ .01) and pneumococcal (46.3% pre-intervention, 73.2% post-intervention; P ≤ .01) vaccinations. Practices focusing on better immunization documentation saw statistically significant improvements for influenza (48.4% pre-intervention, 59.6% post-intervention; P ≤ .04), pneumococcal (29.5% pre-intervention, 58.3% post-intervention; P ≤ .01), and Td/Tdap (27.6% pre-intervention, 53.5% post-intervention; P ≤ .01). Practices that focused on patient education for vaccinations saw statistically significant improvements for pneumococcal (51.2% pre-intervention, 77.2% post-intervention; P ≤ .01) and Td/Tdap (42.7% pre-intervention, 64.0% post-intervention; P ≤ .01).

TABLE 4—

Vaccination Rates Comparison Before Versus After Intervention by Quality Improvement Goal: Improving Adult Immunization Practices Using a Team Approach, 2009–2010

Goal	Received Vaccination	Pre-Intervention Vaccination		Post-Intervention Vaccination
				%	No.	%	No.	P
Increase influenza vaccination rates	Yes	66.1	280	77.9a	276	< .01
	No	33.9		22.1
Increase Pneumoccocal vaccination rates	Yes	46.3	311	73.2a	295	< .01
	No	53.7		26.8
Increase vaccination documentation
Influenza	Yes	48.4	159	59.6	171	.04
	No	51.6		40.4
Pneumococcal	Yes	29.5	146	58.3	132	< .01
	No	70.5		41.7
Td/Tdap	Yes	27.6	170	53.5	157	< .01
	No	72.4		46.5
Increase patient education
Influenza	Yes	61.7	264	69.5a	272	.06
	No	38.3		30.5
Pneumococcal	Yes	51.2	258	77.2a	237	< .01
	No	48.8		22.8
Td/Tdap	Yes	42.7	309	64.0	289	< .01
	No	57.3		36.0

Note. Td = tetanus diphtheria; Tdap = tetanus diphtheria pertussis.

^aRefers to immunization rates higher than the national average reported through the Centers for Disease Control and Prevention or National Center for Immunization and Respiratory Diseases National Immunization 2007 Survey³⁵

Self-reported increases in immunization rates were consistent with practice data findings. Overall, results indicated that physicians improved their immunization practices (i.e., seeking out CDC adult immunization schedule, staff vaccination rates) using the team approach. Indicated vaccines for various disease conditions improved as well, evidenced by the practical application of the CDC adult immunization schedule to their practice. Additionally, practice quality improvement goals had an effect on which vaccinations improved, except for the influenza vaccination, which improved significantly regardless of the quality improvement goal set by the practice.

DISCUSSION

Orienting a practice to promote adult vaccination showed improved adult vaccination rates in this and other studies.^36–38 This required a culture shift and an investment of time for the practice, one that is becoming increasingly difficult given the current time constraints in primary care practice. All practices reported spending between 12 and 72 hours on this program over the course of approximately 12 months. Analyses revealed no significant difference at the practice level between immunization rates and the number of hours the practice spent on the program. Additionally, when the effect of EMR status on vaccine delivery was evaluated, there was no association with EMR status and improved vaccination rates. These data might help persuade smaller practices that do not have as many resources to try quality improvement, as this program showed it could be effective in paper-based practices and with minimal time allotted.

An integral part of the practice’s success was the development of the practice team. The value of the practice team meeting on a continual basis to review their quality improvement progress was vital to the success of any quality improvement work, and was demonstrated in other studies.¹⁵ Practice redesign in this program required a shifting of responsibilities among individuals in a practice in an effort to streamline activities and responsibilities and better orient the practice toward the patient’s health. The team was taught in the coaching calls to share responsibilities to accomplish a common goal. Quality improvement faculty stressed the value of a practice champion who oversaw the quality improvement work and worked to motivate the group. All physicians in this study valued this role, and reported integrating advice from the practice team into their practice improvement work. Similar to other practice teams reported in the literature,¹⁸ this program required the individuals to surrender a small percentage of their autonomy to better serve the goals of the practice as a whole. For instance in 1 practice, the physician gave up the task of screening, passing this responsibility on to the nurse, allowing the physician more time to counsel the patient on the benefits of immunization. As the burden of chronic disease is increasing in our population, physicians are managing more and more complex patients, needing more time in the clinical encounter. These complex patients are often recommended various immunizations because of the susceptibility their disease invokes on the immune system. Few guidelines exist that offer recommendations on treating multiple chronic conditions. In light of these increasing challenges, the team approach is becoming a valuable mode of delivering comprehensive care to these complex patients by re-prioritizing responsibilities among the practice team.^17,39,40

It is difficult to anticipate vaccine demand, making it harder to decide what amounts to purchase. This creates a financial risk for the practice upfront without being able to appropriately anticipate what the demand will be. A recent study evaluated physicians’ rationale behind the purchase of vaccinations and revealed that for herpes zoster, Tdap, and hepatitis B, many did not stock these vaccines because of their high inventory cost, associated low reimbursement, and inconsistent insurance coverage.⁴¹ Because only approximately 20% of internal medicine physicians carry all recommended adult vaccinations,⁴¹ it is not surprising that adult vaccination rates are subpar. Interestingly, in this study, the rates of Td/Tdap increased significantly across various measures from before to after intervention. Additionally, physicians administered herpes zoster, Tdap, and hepatitis B at more annual visits after intervention, suggesting either they made better use of their vaccine supply or increased their vaccine supply. Further data would be needed to understand this increase. Still, only a third of physicians in this study kept a supply of hepatitis B, and even fewer maintained supplies of hepatitis A, herpes zoster, human papilloma virus, and meningococcal vaccinations. Adequate supply of adult vaccinations is important to the health of the population. Therefore, it is not only imperative that physicians be reminded to recommend vaccinations, but to stock them as well.

Immunizations were fundamental to the success of health promotion of the population. However, our current system is oriented toward managing disease, with few resources targeted toward prevention. Other studies, like this one, found that a physician self-reported barrier to adult immunizations was insurance coverage.³¹ This might create a bias to refrain from suggesting immunizations to patients whom the physician suspected did not have insurance coverage for them. Another potential reason for low immunization rates among adults might be that most physicians provided immunizations at well visits, which occur annually at best. Results from this study showed that the majority of vaccinations occurred at annual well visits; a smaller percentage occurred at follow-up visits, and very few occurred at acute visits. Immunizations should be recommended at well and sick visits, requiring a restructuring of the practice visit. Integrating evidence-based guidelines into practice has not been not widely successful,^42,43 creating multiple challenges to primary care physicians. However, this study proved that the CDC immunization schedule was effectively implemented in most practices, as shown in the improvement in vaccination rates for certain disease conditions. The success of integrating guidelines into practice might be attributable to the quality improvement component of this project. Other barriers to quality improvement were numerous and included a lack of standardized metrics, no alignment between quality improvement and reimbursement, the misconception that technology equates quality, and fear of errors in relation to litigation. Overcoming these barriers require a change in the culture of medicine.

Limitations for this study included the information bias of physician self report via the survey data, as survey respondents might have under-reported or overestimated behavior. There might also have been bias depending on who the physician chose to include on the practice team as well who abstracted the data for the practice. Because not all practices had EMRs, there was potential bias in collecting data by EMR or by paper. Practices that participated in this study might have had a greater interest in quality improvement or adult immunizations than the general primary care practice. Additionally, because of the timeline of the grant, data collection was closed in late January, which had potential to limit the amount of influenza vaccinations that might have been performed by these practices through typical flu season (late March). Secular trends (public health campaigns, varying infection rates in communities over the year, vaccine availability) might also have accounted for the increase in vaccination rates as seen in these data.

Overall, the results showed that the intervention via conference call coaching and educational program had an impact on physician immunization practice patterns. All practices used a team approach, which they felt was an integral part of their success in boosting immunization practices. As the knowledge scores did not differ significantly between before and after intervention, our study showed that knowledge is necessary but not sufficient. Practice change takes a team approach, one that is invested in quality and supported to improve their practice. Because this study did not include a control group, future research may evaluate practices that use a team approach with those who do not to more concretely determine the impact of the practice team on quality improvement work. More information is needed to explain what is happening in practices that use a team that would not be happening otherwise. This information would be beneficial for small practices that may find it hard to implement the team approach.