Skip to main content
NCBI home page
BMC Public Health logoLink to BMC Public Health
. 2010 Dec 3;10:750. doi: 10.1186/1471-2458-10-750

Feasibility and impact of providing feedback to vaccinating medical clinics: evaluating a public health intervention

Nicholas Brousseau 3,4, Chantal Sauvageau 1,2,3,4,✉, Manale Ouakki 1, Diane Audet 2, Marilou Kiely 3, Colette Couture 3, Alain Paré 3, Geneviève Deceuninck 2
PMCID: PMC3017028  PMID: 21129216

Abstract

Background

Vaccine coverage (VC) at a given age is a widely-used indicator for measuring the performance of vaccination programs. However, there is increasing data suggesting that measuring delays in administering vaccines complements the measure of VC. Providing feedback to vaccinators is recognized as an effective strategy for improving vaccine coverage, but its implementation has not been widely documented in Canada. The objective of this study was to evaluate the feasibility of providing personalized feedback to vaccinators and its impact on vaccination delays (VD).

Methods

In April and May 2008, a one-hour personalized feedback session was provided to health professionals in vaccinating medical clinics in the Quebec City region. VD for vaccines administered at two and twelve months of age were presented. Data from the regional vaccination registry were analysed for participating clinics. Two 12-month periods before and after the intervention were compared, namely from April 1st, 2007 to March 31st, 2008 and from June 1st, 2008 to May 31st, 2009.

Results

Ten medical clinics out of the twelve approached (83%), representing more than 2500 vaccinated children, participated in the project. Preparing and conducting the feedback involved 20 hours of work and expenses of $1000 per clinic. Based on a delay of one month, 94% of first doses of DTaP-Polio-Hib and 77% of meningococcal vaccine doses respected the vaccination schedule both before and after the intervention. Following the feedback, respect of the vaccination schedule increased for vaccines planned at 12 months for the four clinics that had modified their vaccination practices related to multiple injections (depending on the clinic, VD decreased by 24.4%, 32.0%, 40.2% and 44.6% respectively, p < 0.001 for all comparisons).

Conclusions

The present study shows that it is feasible to provide personalized feedback to vaccinating clinics. While it may have encouraged positive changes in practice concerning multiple injections, this intervention on its own did not impact vaccination delays of the clinics visited. It is possible that feedback integrated into other types of effective interventions and sustained over time may have more impact on VD.

Background

Child vaccination is one of the most effective interventions in public health [1,2]. Vaccine coverage (VC) at a given age is a widely-used indicator for measuring the performance of vaccination programs [3]. However, there is increasing data suggesting that measuring delays in age-appropriate vaccination complements the measure of VC [4-6]. Vaccination delays (VD) can, independently of VC, increase the burden of certain childhood diseases such as pertussis and measles [7,8]. Moreover, a delay in the administration of vaccines scheduled for the age of two months is associated with poor VC at two years [3,9,10].

Across Canada, a consensus has been reached on the definition of VD, namely one month after the due date on the vaccination schedule [11]. This definition is in line with the principal studies on the subject [4,6,12-15]. However, since 2006, the Quebec Ministry of Health and Social Services has compiled VD noted in Health and Social Service Centres (Centres de santé et de services sociaux or CSSS) based on a delay of one week following the recommended age [16]. This tracking of VD is not carried out in private medical clinics, which administer about half of the vaccines for infants in the province of Quebec [3,17].

In 2008, in Quebec, using a one-month delay definition, only 21% of children received all the vaccines scheduled before the age of two years within the recommended period [3].

Providing feedback to vaccinators is one of the effective strategies for improving VC [18-24]. It involves presenting clinicians with their performance data, with or without recommendations [25]. This intervention is carried out following data collection on clinicians' performance (audit). Certain studies have shown that feedback may help reduce VD and missed vaccination opportunities [18,26]. On the other hand, the impact and feasibility of providing feedback to vaccinating medical clinics have not been documented extensively in a Canadian context [27,28].

The main objective of this study was to evaluate the feasibility and the impact on VD of providing personalized feedback to medical clinics that vaccinate infants in the Quebec City region.

Methods

Participants

The twelve medical clinics that had administered the largest number of doses of DTaP-Polio-Hib vaccine in 2007 in the Quebec City region were approached to participate in the feedback project. The vaccination registry (VAXIN), which compiles all doses administered in the Quebec City region, enabled to identify these clinics and to evaluate vaccination delays.

Intervention

In April and May 2008, a one-hour feedback session, led by a physician and a public health nurse, was carried out with the physicians, nurses and secretaries in each participating medical clinic. This feedback dealt with VD for infants at the clinic for the year 2007. Data on the proportion of doses administered without delay were presented for the first doses of three vaccines (DTaP-Polio-Hib, pneumococcal and meningococcal). Vaccination delays for each clinic were presented both in terms of the Quebec standard (one week) and the proposed Canadian standard (one month). Graphs showing the cumulative percentage of children vaccinated according to age were also presented for vaccines scheduled at 2 and 12 months, including measles, mumps and rubella vaccine (MMR). During preparation of the feedback, it became clear that certain clinics were not administering vaccines scheduled at one year during a single visit. Consequently, information on the importance of multiple injections was transmitted to these medical clinics. Ethical approval was obtained from the Comité d'éthique de la recherche du CHUQ - Centre hospitalier de l'Université Laval (project C10-11-101).

Questionnaire

At the end of each feedback session, a questionnaire on the organizational characteristics of the clinic in terms of vaccination was completed on site by a nurse or a secretary. One year following the feedback session (July 2009), the same questionnaire was completed by telephone and, when possible, with the same person.

Feasibility

The data related to the feasibility of the intervention, i.e., the material, human and financial resources required, were compiled.

Analyses of vaccination delays

In order to measure the impact of the feedback on delays associated with vaccines (1st dose of DTaP-Polio-Hib, 1st dose of pneumococcal, meningococcal and 1st dose of MMR), data from the regional vaccination registry were analyzed for the participating clinics. Two 12-month periods before and after the intervention were compared, namely from April 1st, 2007 to March 31st, 2008 and from June 1st, 2008 to May 31st, 2009. Two indicators of delay were considered: one week and one month after the due date on the vaccination schedule. Proportion comparisons were carried out using the chi-square test. The distributions of ages at vaccination, before and after the intervention, were compared using the Wilcoxon test. The analyses were carried out using the SAS 9.1 software (SAS Institute, Cary, NC).

Results

Participation

A total of ten clinics (83%), representing more than 2500 vaccinated children, agreed to participate in the feedback project. Out of the 206 physicians, nurses and secretaries working in these ten clinics, 106 participated in the feedback process (51%).

Organizational characteristics of the medical clinics

Table 1 summarizes the organizational characteristics of the clinics visited and the changes observed between the ends of each of the two observation periods. Among other characteristics, it was noted that four clinics changed their practices concerning multiple injections. For two of these four clinics, responsibilities concerning vaccination were given to nurses. Since 2002, in the province of Quebec, nurses have the possibility to vaccinate as set out in the Public Health Act [29].

Table 1.

Characteristics of clinics at the end of the second observation period and changes observed

Clinic Organizational characteristics (X = yes)
If delay noted:
Child seen very rapidly (otherwise, child referred) Possibility of vaccinating a child without appointment Possibility of extending clinic hours Possibility of making an appointment with a nurse only Multiple injections at 12 months encouraged Possibility of making an appointment on site Telephone reminder before an appointment
01 X X
02 X X
03 X X X X
04 X X X X X
05 X X X Xa, c Xb X X
06 X X X Xc Xb X
07 X
08 X X
09 Xb X X
10 X X Xb X
Open in a new tab

a During the second observation period, hiring of two nurses who vaccinate on an appointment basis.

b Unlike the end of the first observation period, multiple injections are now encouraged.

c During the second observation period, nurses began vaccinating children at 12 months on an appointment basis.

Vaccination delays

During the second observation period, 94% of the first doses of DTaP-Polio-Hib and pneumococcal vaccines were received within the one-month delay (Table 2). This proportion was 77% for meningococcal vaccine and 75% for MMR (Table 3). When a delay of one week after the scheduled vaccination date was applied, the proportion of doses received without delay was 56% for DTaP-Polio-Hib and pneumococcal vaccines, 34% for meningococcal vaccine and 33% for MMR.

Table 2.

Doses of vaccines administered without delay before and after feedback for the 10 clinics

Vaccines planned at 2 months of age and indicator Proportion without delay
Before feedback (2007-2008)		 Proportion without delay
After feedback (2008-2009)		 Difference p
1st DTaP-Polio-Hib Doses = 3297 Doses = 3519
 1 week 64.8% 55.9% - 8.9% < 0.001
 1 month 93.6% 93.6% 0.0% 0.970
1st Pneumococcal Doses = 3272 Doses = 3504
 1 week 65.0% 56.0% - 9.0% < 0.001
 1 month 94.0% 93.8% - 0.3% 0.656
Open in a new tab

Table 3.

Doses of vaccines administered without delay before and after feedback for the 10 clinics

Vaccines planned at 12 months of age and indicator Proportion without delay Before feedback (2007-2008) Proportion without delay After feedback (2008-2009) Difference p
Meningococcal Doses = 2787 Doses = 2870
 1 week 36.9% 34.2% - 2.7% 0.036
 1 month 77.0% 77.0% + 0.0% 0.973
1st MMR Doses = 2858 Doses = 2941
 1 week 33.9% 33.2% - 0.8% 0.525
 1 month 72.7% 74.9% + 2.2% 0.053
Open in a new tab

Few changes were observed before and after the feedback (Tables 2 and 3). Using the one-month delay, no statistically significant difference in VD was found between the two periods. Using the one-week delay, a statistically significant increase in VD was observed for DTaP-Polio-Hib (8.9%, p < 0.001), pneumococcal (9.0%, p < 0.001) and meningococcal (2.7%, p = 0.036) vaccines.

The median age at vaccination increased in a statistically significant manner for DTaP-Polio-Hib and pneumococcal vaccines (+ 2 days, p < 0.0001). For all vaccines, the difference in median age at vaccination before and after the feedback was nevertheless very small, consistently less than 3 days (Table 4).

Table 4.

Median age at vaccination before and after feedback for the 10 clinics

Before feedback (2007-2008) After feedback (2008-2009)
Vaccine Doses
administered 		Median age
(months) 		Doses administered Median age
(months) 		Difference
(days) 		p
1st DTaP-Polio-Hib 3297 2.14 3519 2.20 + 2 < 0.0001
1st Pneumococcal 3272 2.14 3504 2.20 + 2 < 0.0001
Meningococcal 2787 12.39 2870 12.42 + 1 0.07
1st MMR 2858 12.45 2941 12.45 0 0.65
Open in a new tab

As can be seen in Figure 1, age at vaccination for DTaP-Polio-Hib and meningococcal vaccines for the 2008-2009 period was comparable to that for the 2007-2008 period, with both curves being superimposed.

Figure 1.

Figure 1

Open in a new tab

Cumulative percentage of children vaccinated in the 10 clinics before and after the feedback intervention.

For the four clinics that began the practice of multiple injections in 2008-2009, a significant decrease in VD was observed for vaccines scheduled at 12 months. For the third dose of the pneumococcal vaccine (given at 12 months), the proportion of infants immunized within a one-month delay increased from 19.6% to 51.6% (p < 0,001) for clinic 10 and from 29.3% to 73.9% (p < 0,001) for clinic 09. The proportion of infants immunized without delay for the first dose of MMR increased from 27.4% to 67.6% (p < 0,001) for clinic 05. Finally, the proportion of infants immunized without delay for the meningococcal vaccine increased from 56.5% to 80.9% (p < 0,001) for clinic 06. Figure 2 (a and b) illustrates this situation for two of the four clinics.

Figure 2.

Figure 2

Open in a new tab

Cumulative percentage of children vaccinated before (A) and after (B) feedback for two clinics.

Feasibility

The preparation and completion of the feedback sessions required, in terms of human resources, the contribution of a physician (42 hours), two nurses (48 hours), a statistician (42 hours) and a research officer (74 hours). The cost of preparing and carrying out the meetings in the ten clinics (professional hours, materials, travel and cold meals served) was estimated at $10,000.

Discussion

The feedback process conducted in 10 of the 12 main vaccinating clinics in the Quebec City region did not decrease overall vaccination delays. A high proportion of the DTaP-Polio-Hib and pneumococcal vaccines were administered before the age of three months (94%), both before and after the intervention. The proportion of doses administered without delay was lower for meningococcal (77%) and MMR (73-75%) vaccines.

Vaccination delays observed in this study are similar or compare favourably with VD documented in other countries for MMR [13,14] or for vaccines containing certain elements of DTaP-Polio-Hib [30,31]. Nevertheless caution must be exercised in comparing VD data given the different methodologies employed.

Several reasons could explain the low impact of the intervention conducted herein. Firstly, this was an isolated intervention not combined with other types of interventional activities. Feedback that is integrated into other activities and sustained over time, such as that inspired by the Assessment, Feedback, Incentives and Exchange (AFIX) strategy, appears to be more likely to have an impact on vaccination [20,23,32]. Moreover, an increase in the number of doses administered in 2008-2009 was noted compared to 2007-2008. The pressure represented by this increased demand for vaccination could have contributed to the low impact of the feedback. As well, only half of the health professionals involved in vaccination in the clinics that were visited participated in the feedback process. The impact analyses, on the other hand, dealt with the totality of vaccination acts in the participating clinics, since the registry does not include a specific code for each vaccinator. Finally, VD presented were relatively low for some clinics and may not have generated sufficient motivation to change current practices. However, the feedback provided could have influenced more specific problematic practices resulting in high VD.

It is possible that the feedback was the cause of the decrease in VD observed for the four clinics that modified their practices concerning multiple injections. It is also possible that the introduction of the combined measles, mumps, rubella and varicella vaccine (MMR-V) administered at one year of age as well as the involvement of nurses in vaccination contributed to these changes. Two of the four clinics involved hired nurses or gave nurses additional responsibilities following the feedback. Other studies suggest that the involvement of nurses favours adherence to recommendations regarding vaccination [17]. These results are important, since the practice of multiple injections is a factor that facilitates complete immunization [33]. However, these modifications did not appear to have an impact on global analyses. A possible hypothesis is that the decrease in VD was observed in four of the smallest clinics that only gave a limited number of vaccine doses, therefore having a limited weight compared to larger clinics.

A relatively wide consensus exists concerning the definition of VD, namely one month after the date on the immunization schedule [34]. In the present study, the results using this standard did not suggest any deterioration or improvement in VD after feedback. When using a one-week standard, we observed an increase in VD for the first DTaP-Polio-Hib vaccine, for the first pneumococcal vaccine and for the meningococcal vaccine. However, these increases in VD correspond to a mean immunization time delayed by only one or two days in the year following the feedback, a difference not clinically significant. It is possible that using a one-week definition for VD may be too sensitive to very minor changes in immunization practices and lead to misinterpretations. Furthermore, using a short one-week standard for VD could demoralize clinicians because it is very difficult to reach. This issue was reported by some participants during the feedback sessions.

In Quebec, the Ministry of Health and Social Services tracks the vaccination delays associated with DTaP-Polio-Hib, pneumococcal and meningococcal vaccines. Strict surveillance of VD for DTaP-Polio-Hib vaccine is essential, as delays in its administration are associated with incomplete VC [3]. Since the VD for pneumococcal vaccine are almost identical to that for DTaP-Polio-Hib, and since it is scheduled for the same visit, the benefits associated with its surveillance appear rather limited.

A number of limitations of this study should be mentioned. A high proportion of vaccination acts are recorded in the vaccination registry used (VAXIN) [35]. However, when children move to the Quebec City region, it is possible that vaccines given previously in other regions may not be systematically reported and recorded. This phenomenon is apt to lead to an overestimation of VD, but should be consistent over time. The fact that the VD determined by this study are close to the data reported in provincial studies is nevertheless reassuring [3].

The questionnaire was completed in person after the first observation period and by telephone after the second observation period. As well, the same person was reached in only five of the ten clinics. An effort was made to limit the impact of this latter factor by formulating the questions in the same manner.

Finally, the use of a control group would have allowed for the impact of the feedback to be better identified. However, such a strategy would have been extremely difficult to apply since a large majority of doses are administered by a small number of clinics in the Quebec City region and the clinics that vaccinate the most were the ones that participated in the study. Nevertheless, we compared the VD observed during the two periods for those clinics that did not participate in the project and the same tendencies were noted.

Conclusions

While the feedback that was provided did not lead to a decrease in VD, it nevertheless appeared to have facilitated certain positive changes in practices concerning multiple injections. It is possible that feedback integrated into other types of effective interventions, sustained over time or giving certain responsibilities to vaccinators may have more impact in decreasing VD. These measures, however, would require additional resources. Efforts aimed at reducing the burden related to delayed immunization and at improving VC should be pursued in order to protect the population against diseases that can be avoided by vaccination.

Abbreviations

DTaP-Polio-Hib: Diphtheria, tetanus toxoid, acellular pertussis, inactivated poliovirus and Haemophilus influenzae type b vaccine; MMR: Measles, mumps and rubella vaccine; MMR-V: Measles, mumps, rubella and varicella vaccine; VC: Vaccine coverage; VD: Vaccination delays.

Competing interests

Competing interests for CS: research grants, honoraria and reimbursement of travel costs by the following companies: Wyeth, GlaxoSmithKline and Merck Frost. The other authors declare that they have no competing interest.

Authors' contributions

NB participated for the data collection and wrote the first draft of the manuscript. CS conceived the study, coordinated the data collection, carried out the feedback intervention, and helped in drafting the manuscript. MO and GD performed the statistical analysis and helped in drafting the manuscript. MK participated in data collection and helped in writing the first draft of the manuscript. DA, AP and CC prepared and conducted the feedback sessions for medical clinics, participated in data collection and helped in the writing of the manuscript. All authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2458/10/750/prepub

Contributor Information

Nicholas Brousseau, Email: nicholas.brousseau@ssss.gouv.qc.ca.

Chantal Sauvageau, Email: chantal_sauvageau@ssss.gouv.qc.ca.

Manale Ouakki, Email: manale.ouakki@ssss.gouv.qc.ca.

Diane Audet, Email: dianeaudet@ssss.gouv.qc.ca.

Marilou Kiely, Email: marilou_kiely@ssss.gouv.qc.ca.

Colette Couture, Email: colette.couture@ssss.gouv.qc.ca.

Alain Paré, Email: alain.pare@ssss.gouv.qc.ca.

Geneviève Deceuninck, Email: genevieve.deceuninck@ssss.gouv.qc.ca.

Acknowledgements

The authors would like to thank Ève Dubé, Geneviève Tremblay and Nicole Boulianne for their judicious comments on previous versions of this article. We would also like to thank Josiane Rivard for her technical support and Marie-France Richard for the layout of the manuscript. Finally, we would like to thank the company Wyeth for its financial support in the context of its Partnering for Protection program as well as the participating clinics, without whom this study could not have taken place. The funding body was not involved in any part of the study.

References

  1. Bos E, Batson A. Using immunization coverage rates for monitoring health sector performance: Measurement and interpretation issues. Washington DC: Human development network, The World Bank; 2000. pp. 1–21. [Google Scholar]
  2. Comité consultatif national sur l'immunisation (Ed.) Guide canadien d'immunisation. 7. Ottawa: Agence de la santé publique du Canada; 2006. p. 410. [Google Scholar]
  3. Boulianne N, Bradet R, Audet D, Deceuninck G. Enquête sur la couverture vaccinale des enfants de 1 an et 2 ans au Québec en 2008. Québec: Institut national de santé publique du Québec; 2009. pp. 1–205. [Google Scholar]
  4. Dombrowski KJ, Lantz PM, Freed GL. The need for surveillance of delay in age-appropriate immunization. Am J Prev Med. 2002;23(1):36–42. doi: 10.1016/S0749-3797(02)00442-7. [DOI] [PubMed] [Google Scholar]
  5. Luman ET, McCauley MM, Stokley S, Chu SY, Pickering LK. Timeliness of childhood immunizations. Pediatrics. 2002;110(5):935–939. doi: 10.1542/peds.110.5.935. [DOI] [PubMed] [Google Scholar]
  6. Akmatov MK, Kretzschmar M, Kramer A, Mikolajczyk RT. Timeliness of vaccination and its effects on fraction of vaccinated population. Vaccine. 2008;26(31):3805–3811. doi: 10.1016/j.vaccine.2008.05.031. [DOI] [PubMed] [Google Scholar]
  7. Shinall MC Jr, Peters TR, Zhu Y, Chen Q, Poehling KA. Potential impact of acceleration of the pertussis vaccine primary series for infants. Pediatrics. 2008;122(5):1021–1026. doi: 10.1542/peds.2007-3025. [DOI] [PubMed] [Google Scholar]
  8. The National Vaccine Advisory Committee. The measles epidemic. The problems, barriers, and recommendations. JAMA. 1991;266(11):1547–1552. doi: 10.1001/jama.266.11.1547. [DOI] [PubMed] [Google Scholar]
  9. Dietz VJ, Stevenson J, Zell ER, Cochi S, Hadler S, Eddins D. Potential impact on vaccination coverage levels by administering vaccines simultaneously and reducing dropout rates. Arch Pediatr Adolesc Med. 1994;148(9):943–948. doi: 10.1001/archpedi.1994.02170090057008. [DOI] [PubMed] [Google Scholar]
  10. Wood D, Pereyra M, Halfon N, Hamlin J, Grabowsky M. Vaccination levels in Los Angeles public health centers: The contribution of missed opportunities to vaccinate and other factors. Am J Public Health. 1995;85(6):850–853. doi: 10.2105/AJPH.85.6.850. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Canadian Immunization Registry network Task Group. Boulianne N, Hemon Y-A, Mawhinney T, Strong D, Gemmill I, Dobson S, Sartison E, Sargent M, Naus M, Tuchscherer R, Craig E, Watkins K, Schouten H. National eligible, due and overdue guidelines for immunization registries: draft recommendations from the Canadian Immunization Registry Network, Data Standards Task Group. Can Commun Dis Rep. 2004;30(6):53–59. [PubMed] [Google Scholar]
  12. Luman ET, Barker LE, Shaw KM, McCauley MM, Buehler JW, Pickering LK. Timeliness of childhood vaccinations in the United States: days undervaccinated and number of vaccines delayed. JAMA. 2005;293(10):1204–1211. doi: 10.1001/jama.293.10.1204. [DOI] [PubMed] [Google Scholar]
  13. Hull BP, McIntyre PB. Timeliness of childhood immunisation in Australia. Vaccine. 2006;24(20):4403–4408. doi: 10.1016/j.vaccine.2006.02.049. [DOI] [PubMed] [Google Scholar]
  14. Dayan GH, Shaw KM, Baughman AL, Orellana LC, Forlenza R, Ellis A, Chaui J, Kaplan S, Strebel P. Assessment of delay in age-appropriate vaccination using survival analysis. Am J Epidemiol. 2006;163(6):561–570. doi: 10.1093/aje/kwj074. [DOI] [PubMed] [Google Scholar]
  15. Grant CC, Roberts M, Scragg R, Stewart J, Lennon D, Kivell D, Ford R, Menzies R. Delayed immunisation and risk of pertussis in infants: unmatched case-control study. BMJ. 2003;326(7394):852–853. doi: 10.1136/bmj.326.7394.852. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Ministère de la Santé et des Services sociaux. Ententes de gestion, fiche descriptive des indicateurs - Fiche 1.1 FGJ. 2006.
  17. Sauvageau C, Trépanier J-P, Guay M, Landry M, Ben-Yedder N, Laflamme M, Minh Man Nguyen M, Weil A, Hudson P, Royer L, Ouakki M, Pelletier A, Boulianne N. Vaccination en clinique médicale: y a-t-il des délais? Montréal. Journées annuelles de santé publique; 2006. pp. 23–27. [Google Scholar]
  18. LeBaron CW, Chaney M, Baughman AL, Dini EF, Maes E, Dietz V, Bernier R. Impact of measurement and feedback on vaccination coverage in public clinics, 1988-1994. JAMA. 1997;277(8):631–635. doi: 10.1001/jama.277.8.631. [DOI] [PubMed] [Google Scholar]
  19. Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, Carande-Kulis VG, Yusuf HR, Ndiaye SM, Williams SM. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med. 2000;18(1S):97–140. doi: 10.1016/S0749-3797(99)00118-X. [DOI] [PubMed] [Google Scholar]
  20. LeBaron CW, Mercer JT, Massoudi MS, Dini E, Stevenson J, Fischer WM, Loy H, Quick LS, Warming JC, Tormey P, DesVignes-Kendrick M. Changes in clinic vaccination coverage after institution of measurement and feedback in 4 states and 2 cities. Arch Pediatr Adolesc Med. 1999;153(8):879–886. doi: 10.1001/archpedi.153.8.879. [DOI] [PubMed] [Google Scholar]
  21. Bordley WC, Chelminski A, Margolis PA, Kraus R, Szilagyi PG, Vann JJ. The effect of audit and feedback on immunization delivery: a systematic review. Am J Prev Med. 2000;18(4):343–350. doi: 10.1016/S0749-3797(00)00126-4. [DOI] [PubMed] [Google Scholar]
  22. Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of physician bonuses, enhanced fees, and feedback on childhood immunization coverage rates. Am J Public Health. 1999;89(2):171–175. doi: 10.2105/AJPH.89.2.171. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Melinkovich P, Hammer A, Staudenmaier A, Berg M. Improving pediatric immunization rates in a safety-net delivery system. Jt Comm J Qual Patient Saf. 2007;33(4):205–210. doi: 10.1016/s1553-7250(07)33024-9. [DOI] [PubMed] [Google Scholar]
  24. Zara S, Briss PA, Harris KW. In: The Guide to Community Preventive Services - What Works to Promote Health? Stephanie Zaza, Peter A Briss, editor. 2005. p. 544. [Google Scholar]
  25. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153(10):1423–1431. [PMC free article] [PubMed] [Google Scholar]
  26. Sabnis SS, Pomeranz AJ, Amateau MM. The effect of education, feedback, and provider prompts on the rate of missed vaccine opportunities in a community health center. Clin Pediatr (Phila) 2003;42(2):147–151. doi: 10.1177/000992280304200208. [DOI] [PubMed] [Google Scholar]
  27. Lemelin J, Hogg W, Baskerville N. Evidence to action: a tailored multifaceted approach to changing family physician practice patterns and improving preventive care. CMAJ. 2001;164(6):757–763. [PMC free article] [PubMed] [Google Scholar]
  28. Borgiel AE, Williams JI, Davis DA, Dunn EV, Hobbs N, Hutchison B, Wilson CR, Jensen J, O'Neil JJ, Bass MJ. Evaluating the effectiveness of 2 educational interventions in family practice. CMAJ. 1999;161(8):965–970. [PMC free article] [PubMed] [Google Scholar]
  29. Assemblée nationale. Projet de loi no 90 - Loi modifiant le Code des professions et d'autres dispositions législatives dans le domaine de la santé. Éditeur officiel du Québec; 2002. pp. 1–20. [Google Scholar]
  30. Kalies H, Grote V, Schmitt HJ, von Kries R. Immunisation status of children in Germany: temporal trends and regional differences. Eur J Pediatr. 2006;165(1):30–36. doi: 10.1007/s00431-005-1758-0. [DOI] [PubMed] [Google Scholar]
  31. Estimated Vaccination Coverage with Individual Vaccines by 3 Months of Age by State and Local Area US, National Immunization Survey, 2008 (consulted on October 26, 2009) 2008. http://www.cdc.gov/vaccines/stats-surv/nis/tables/08/tab04_3mo_iap.xls
  32. Massoudi MS, Walsh J, Stokley S, Rosenthal J, Stevenson J, Miljanovic B, Mann J, Dini E. Assessing immunization performance of private practitioners in Maine: impact of the assessment, feedback, incentives, and exchange strategy. Pediatrics. 1999;103(6 Pt 1):1218–1223. doi: 10.1542/peds.103.6.1218. [DOI] [PubMed] [Google Scholar]
  33. Boulianne N, Deceuninck G, Duval B. Pourquoi certains enfants sont incomplètement vaccinés à l'âge de 2 ans? Rev can santé publ. 2003;94(3):218–223. doi: 10.1007/BF03405070. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Dietz VJ, Zell ER, Stevenson J. Defining delayed immunization. Pediatr Infect Dis J. 1993;12(4):353–354. doi: 10.1097/00006454-199304000-00022. [DOI] [PubMed] [Google Scholar]
  35. Guay M, Boulianne N, Ménard S, Clouâtre A-M, Clément P, Tremblay A, Blackburn M, Lemaire J, Douville-Fradet M. Étude de validation et d'appréciation des fichiers de vaccination et de population en Estrie, en Montérégie et dans la région de Québec. Québec: Centre de recherche Hôpital Charles LeMoyne; 2005. pp. 1–70. [Google Scholar]

Articles from BMC Public Health are provided here courtesy of BMC

RESOURCES