Abstract
INTRODUCTION
In the Saskatoon Health Region (Saskatchewan), only 67.4% of children overall are fully immunized for measles, mumps and rubella (MMR) at 24 months of age, with only 43.7% of low-income children fully immunized.
METHODS
Parents of children who were behind in MMR immunizations were contacted to determine knowledge about, beliefs toward and barriers to immunization. The effectiveness of a telephone reminder system in improving immunization rates in a health region compared with a control health region was determined. Finally, the effectiveness of telephone reminders versus telephone reminders combined with home visits in improving child immunization coverage rates in low-income neighbourhoods was compared.
RESULTS
The survey was completed by 629 parents (69% response rate). Of those, 81.8% were not aware that their child was behind in immunizations. In the Saskatoon Health Region, the MMR immunization coverage increased from 67.4% to 74.0% in the first year of intervention (rate ratio = 1.10; 95% CI 1.08 to 1.12). All four neighbourhood groupings (three urban by income and one rural) had relative increases ranging from 9% to 11%. The control health region observed an immunization coverage increase from 66.5% to 69.2% in the first year (rate ratio = 1.04; 95% CI 1.01 to 1.07). The three low-income neighbourhoods with only telephone reminders had an immunization coverage rate of 48.7% (95% CI 39.5% to 57.8%). The three low-income neighbourhoods that received a telephone reminder and home visit had an immunization coverage rate of 60.5% (95% CI 52.5% to 68.6%).
CONCLUSION
Telephone reminder systems have some benefit in increasing child immunization coverage rates.
Keywords: Children, Immunization, Intervention studies
Abstract
INTRODUCTION
Dans la région sanitaire de Saskatoon (en Saskatchewan), seulement 67,4 % de l’ensemble des enfants sont entièrement immunisés contre la rougeole, la rubéole et les oreillons (RRO) à 24 mois, dont seulement 43,7 % des enfants défavorisés.
MÉTHODOLOGIE
Les chercheurs ont pris contact avec les parents d’enfants qui accusaient un retard du vaccin RRO afin de déterminer leurs connaissances, leurs croyances et les obstacles à l’égard de la vaccination. Ils ont établi l’efficacité d’un système de rappels téléphoniques pour améliorer les taux de vaccination dans une région sanitaire par rapport à une région sanitaire témoin. Enfin, ils ont comparé l’efficacité des rappels téléphoniques par rapport aux rappels téléphoniques accompagnés d’une visite à domicile pour améliorer la couverture vaccinale des enfants de quartiers défavorisés.
RÉSULTATS
Le sondage a été rempli par 629 parents (taux de réponse de 69 %). De ce nombre, 81,8 % ne savaient pas que leur enfant accusait un retard vaccinal. Dans la région sanitaire de Saskatoon, la couverture du vaccin RRO est passée de 67,4 % à 74,0 % au cours de la première année de l’intervention (ratio des taux = 1,10; 95 % IC 1,08 à 1,12). Les quatre regroupements par quartier (trois quartiers urbains répartis selon le revenu et un quartier rural) ont présenté des augmentations relatives de l’ordre de 9 % à 11 %. La région sanitaire témoin a observé une augmentation de la couverture vaccinale de 66,5 % à 69,2 % au cours de la première année (ratio des taux = 1,04; 95 % IC 1,01 à 1,07). Les trois quartiers défavorisés n’ayant reçu que des rappels téléphoniques présentaient une couverture vaccinale de 48,7 % (95 % IC 39,5 % à 57,8 %). Les trois quartiers défavorisés qui ont reçu des rappels téléphoniques et une visite à domicile présentaient une couverture vaccinale de 60,5 % (95 % IC 52,5 % à 68,6 %).
CONCLUSION
Les systèmes de rappels téléphoniques comportent certains avantages pour améliorer la couverture vaccinale des enfants.
Few measures in preventive medicine are of such proven value and as easy to implement as immunization against infectious disease (1). Accordingly, Canada is striving to achieve national childhood immunization coverage rates of 95% (1). Although this benchmark is often repeated, the attainment of the goal is more elusive. For example, a recent study from Saskatoon, Saskatchewan (2), found that only 67.9% of children overall were fully immunized for measles, mumps and rubella (MMR) at 24 months of age, with only 43.7% of low-income children fully immunized over the study period of five years.
Previous reports (3–11) indicate that low immunization coverage rates for children are associated with low socioeconomic status, urban dwelling, impoverished neighbourhoods, single-parent families, mobile populations and minority cultural status.
A meta-analysis (4) of randomized trials reviewed the efficacy of patient reminder and recall systems to improve overall immunization rates. Of the studies approved, 11 focused on the age range of birth to 24 months of age. Of these 11 studies, only one was for MMR and none included data from Canada. The overall conclusion of the meta-analysis was that telephone reminders were more effective than postcard or letter reminders in improving immunization coverage rates. However, a major limitation is that the meta-analysis only reviewed randomized trials. As such, the studies included relatively small sample sizes with volunteer selection bias and results that do not generalize well to real-world scenarios in entire populations. Despite the selection bias, only one intervention group had a child immunization coverage rate of 95%, while the intervention groups in the other 10 randomized trials obtained coverage rates ranging from 42% to 64% (4).
The purpose of the present study was to determine the causes and investigate solutions to low MMR immunization coverage rates for children at two years of age in the Saskatoon Health Region (SHR; n=298,371 in 2008). There were three objectives of the current study conducted in the SHR. The first objective was to contact parents or guardians of children who were behind in routine MMR immunizations to determine their knowledge about, beliefs toward and reported barriers to immunization. The second objective was to determine the effectiveness of a telephone reminder system in improving child immunization coverage rates in an entire health region compared with historical trends and compared with a control health region without enhanced intervention. The third objective was to use block randomization to compare the effectiveness of telephone reminders versus telephone reminders combined with home visits in improving child immunization coverage rates in low-income neighbourhoods.
METHODS
The Saskatchewan Immunization Management System (SIMS) uses vital statistics and health insurance information to create a population database to determine the number of children who have the recommended number of immunizations for their age. The database is more accurate and complete than clinical hard copies of patient files (12). It includes the name of a contact parent or guardian, the name of the child, health card number, address and contact telephone number.
In Saskatchewan, children are recommended to have two MMR immunizations by 18 months of age. Therefore, the definition of incomplete coverage is less than two MMR immunizations by 24 months of age.
For the first objective of our study, a list of names of parents or guardians was generated with children living in the SHR who did not receive two MMR immunizations by their second birthday between October 2007 and September 2008. Parents or guardians were telephoned up to five times until they were contacted (or not contacted) by an SHR research assistant who reminded the parents in English that their children were behind in all routine immunizations. The parents were then asked to consent to a short survey that would ask if they knew that their child was behind in routine immunization coverage, as well as their beliefs and self-report barriers toward immunization. The survey contained the questions that are shown in Table 1.
TABLE 1.
Knowledge of and beliefs toward childhood immunizations | n (%) | 95% CI |
---|---|---|
Do you understand the diseases that immunizations help protect against? | ||
Yes | 595 (94.7) | 91.2–98.1 |
Did you believe that your child was fully up to date with immunizations? | ||
No – did not know child was behind | 514 (81.8) | 75.8–87.7 |
My child has an appointment booked | 66 (10.5) | 4.5–16.4 |
My child just had an appointment | 27 (4.3) | 1.1–7.4 |
I do not plan to immunize my child at this time | 21 (3.3) | 0.5–6.0 |
Should children be immunized when they are sick with a minor illness like a cold? | ||
No, they should not be | 505 (80.3%) | 63.4–77.3 |
Do you believe that childhood immunizations have serious known side effects? | ||
Yes, they have known side effects | 155 (24.7) | 18.0–31.3 |
Do you believe that childhood immunizations have serious unknown side effects? | ||
Yes, they have unknown side effects | 185 (29.5) | 22.5–36.4 |
Barriers to immunization of your child | ||
Forgot to immunize | ||
Yes | 361 (57.5) | 49.9–65.0 |
Do not have time in your day | ||
Yes | 188 (29.9) | 22.8–36.9 |
Do not have an immunization clinic nearby | ||
Yes | 56 (8.9) | 4.5–13.2 |
Lack access to transportation | ||
Yes | 75 (11.9) | 6.9–16.8 |
Do not trust the medical community | ||
Yes | 38 (6.1) | 2.4–9.7 |
Personally had a bad experience with health care in the past | ||
Yes | 79 (12.6) | 7.5–17.6 |
Had a bad experience when your child was immunized | ||
Yes | 44 (7.0) | 3.0–10.9 |
Personal objections to childhood immunization | ||
Yes | 75 (11.9) | 6.9–16.8 |
If YES to personal objections, what is your objection to immunization? (n=75) | ||
Cultural/religious | 3 (4.0) | 1.0–6.9 |
Homeopathic | 12 (16.0) | 10.3–21.6 |
Safety | 60 (80.0) | 73.8–86.6 |
The second objective was to determine the effectiveness of the telephone reminder system throughout the SHR. Parents or guardians who were originally contacted between October 2007 and September 2008 had the immunization status of their child officially rechecked through SIMS three months after their initial telephone reminder to confirm immunization. Because low socioeconomic status, urban dwelling and impoverished neighbourhoods influence child immunization rates, the results were stratified by neighbourhood income. Using census data, postal codes and existing municipal boundaries for neighbourhoods, the SHR was stratified into four groups: six low-income cut-off neighbourhoods (as defined by Statistics Canada), which were contiguous; the five neighbourhoods with the highest incomes, which were also contiguous; the remaining middle-income neighbourhoods; and the rural area surrounding the city of Saskatoon (excluding on-reserve First Nations with federal responsibility) but within the SHR (2,13).
The one-year result of the intervention (telephone reminder) for children born between October 2005 and September 2006 was compared with the five-year historical average in the SHR overall and by neighbourhood grouping for children born between October 2000 and September 2005. The results from the SHR were also compared with its sister health region in Saskatchewan – the Regina Qu’Appelle Health Region (RQHR; n=250,000 in 2008) – which did not have a telephone reminder system. Data from the RQHR were collected from the same time periods with the same breakdown by neighbourhood income.
Given the very low child immunization coverage rates in Saskatoon’s six low-income neighbourhoods, an additional intervention was provided. As mentioned above, all parents or guardians of children born between October 2005 and September 2006 in the six contiguous low-income neighbourhoods received the telephone reminder. Additionally, block randomization through computer allocation was used to divide the six neighbourhoods into two blocks. Parents or guardians from three randomly selected low-income neighbourhoods were offered MMR immunization through a home visit by a public health nurse at the time of their telephone reminder.
As advocated by Rothman and Greenland (14), 95% CIs were generated in comparison with significance tests to convey more information on the magnitude and precision of the point estimates. All 95% CIs for percentages and rate ratios were computed using SPSS Statistics 17.0 (IBM Corporation, USA).
Ethics approval was obtained from the University of Saskatchewan Behavioural Research Ethics Board (BEH# 06–213).
RESULTS
Between October 2007 and September 2008, 3508 children living in the SHR who were born between October 2005 and September 2006 (two years of age during the study period) were identified. Of those children, 2597 children were up to date with two MMR immunizations by their second birthday. Conversely, 911 children were behind in at least one immunization. Of those 911 children, 787 parents or guardians (86%) could not be contacted by telephone. Of those 787 parents or guardians, 629 agreed to participate in the survey for an overall contact and response rate of 69%. Essentially, all (99%) cases of parents who could not be contacted were due to incorrect telephone numbers with no known forwarding number. Reviewing non-response bias by the four neighbourhood groupings (three urban by neighbourhood income and one rural) showed no statistically significant differences in response rates. For example, 8.3% of Saskatoon’s residents live within Saskatoon’s six low-income neighbourhoods (13). In our survey, 9.2% of the sample was from Saskatoon’s six low-income neighbourhoods. The demographics of the parents or guardians who completed the survey are listed in Table 2.
TABLE 2.
Independent variables | n (%) |
---|---|
Age, years | |
18–29 | 238 (37.9) |
30–39 | 328 (52.1) |
40 and older | 59 (9.4) |
Refused to answer | 4 (0.6) |
Sex | |
Male | 42 (6.7) |
Female | 586 (93.2) |
Refused to answer | 1 (0.2) |
Cultural status | |
First Nations or Métis | 108 (17.2) |
Caucasian | 471 (74.9) |
Other | 47 (7.5) |
Refused to answer | 3 (0.4) |
Marital status | |
Divorced/separated | 15 (2.4) |
Married/common-law | 505 (80.3) |
Single | 108 (17.2) |
Education | |
Did not complete high school | 75 (11.9) |
High school completed | 181 (28.8) |
University/technical diploma | 370 (58.9) |
Refused to answer | 2 (0.3) |
Occupation | |
Clerical/sales/service/manual/construction/farmer | 219 (34.8) |
Homemaker | 151 (24.0) |
Professional/management | 135 (21.5) |
Student/unemployed/other | 120 (19.1) |
Refused to answer | 4 (0.6) |
Neighbourhood | |
Low income | 58 (9.2) |
Middle income | 337 (53.6) |
High income | 35 (5.6) |
Rural | 198 (31.5) |
Of the 620 of 629 parents or guardians who responded to the second question, 81.8% were not aware that their child was behind in immunizations. For parents or guardians whose child was behind in immunizations, 24.7% and 29.5% believed that immunizations were associated with known or unknown side effects, respectively. As well, 80.3% believed they should not immunize their child when they have a minor illness such as a cold. The results are displayed in Table 1.
Of the parents or guardians who had the immunization status of their child officially rechecked through SIMS three months after their initial telephone reminder, 68.0% who updated immunizations had an immunization appointment occurring the next day, with all (100%) other immunization appointments occurring within the next two weeks.
Regarding barriers to childhood immunization, 57.5% of parents who knew their child was behind in immunizations stated that the biggest barrier was forgetting to immunize their child, while another 29.9% reported that they did not have enough time in their day. Other potential barriers were reported to have less impact such as not living near an immunization clinic, lack of access to transportation, lack of trust of the medical community, previous bad experiences or personal objections. The results are displayed in Table 1.
The effectiveness of the telephone call reminder system was reviewed. In the SHR, the overall child MMR immunization coverage at 24 months of age increased in an absolute sense from 67.4% (preintervention group: children born in October 2000 to September 2005) to 74.0% (intervention group: children born in October 2005 to September 2006) in the first year of intervention (10% relative increase; rate ratio 1.10; 95% CI 1.08 to 1.12). In the SHR, all four neighbourhood groupings (three urban by neighbourhood income and one rural) had relative increases ranging from 9% to 11%.
In comparison, the control health region (RQHR) observed an overall child MMR immunization coverage increase in an absolute sense from 66.5% (children born in October 2000 to September 2005) to 69.2% (children born in October 2005 to September 2006) in the first year of control intervention (4% relative increase; rate ratio 1.04; 95% CI 1.01 to 1.07). In the RQHR, the four neighbourhood groupings had differing results ranging from 5% to 13% relative increases in the three urban neighbourhood groupings to a 4% relative reduction in the rural area. The results appear in detail for the SHR and RQHR in Table 3.
TABLE 3.
Neighbourhood | Preintervention (SHR) (October 2000 to September 2005) |
Intervention (SHR) (October 2005 to September 2006) |
Rate ratio (intervention/preintervention)* | 95% CI for rate ratio | ||||
---|---|---|---|---|---|---|---|---|
n/N | % | 95% CI | n/N | % | 95% CI | |||
Low income | 536/1075 | 49.9 | 44.0–55.7 | 142/257 | 55.3 | 43.3–67.2 | 1.10 | 0.97–1.25 |
Middle income | 5892/8699 | 67.7 | 65.7–69.6 | 1425/1880 | 75.8 | 71.9–79.6 | 1.11 | 1.08–1.14 |
High income | 1148/1421 | 80.8 | 76.7–84.8 | 215/243 | 88.5 | 80.6–96.3 | 1.09 | 1.03–1.15 |
Rural | 3563/4807 | 74.1 | 71.6–76.5 | 736/901 | 81.7 | 76.6–86.7 | 1.10 | 1.06–1.14 |
Total | 11,870/17,603 | 67.4 | 66.0–68.7 | 2597/3508 | 74.0 | 71.1–76.8 | 1.10 | 1.08–1.12 |
Neighbourhood | Preintervention (RQHR) (October 2000 to September 2005) | Intervention (RQHR) (October 2005 to September 2006) | Rate ratio (intervention/preintervention) | 95% CI for rate ratio | ||||
n/N | % | 95% CI | n/N | % | 95% CI | |||
Low income | 618/1550 | 39.8 | 50.8–56.6 | 162/360 | 45.0 | 39.9–50.1 | 1.13 | 1.01–1.29 |
Middle income | 4980/7270 | 68.5 | 67.4–69.5 | 1177/1609 | 73.1 | 70.9–75.2 | 1.07 | 1.04–1.10 |
High income | 950/1165 | 81.5 | 79.3–83.7 | 193/225 | 85.7 | 80.6–89.7 | 1.05 | 0.99–1.11 |
Rural | 2148/3083 | 69.6 | 68.0–71.2 | 368/550 | 66.9 | 62.8–70.7 | 0.96 | 0.90–1.02 |
Total | 8696/13,068 | 66.5 | 65.7–67.3 | 1900/2744 | 69.2 | 67.5–70.9 | 1.04 | 1.01–1.07 |
The rate ratio includes immunization coverage rates during the intervention time period in the numerator and the immunization coverage rates before the intervention time period in the denominator
A total of 257 children born between October 2005 and September 2006 were eligible for MMR immunization in Saskatoon’s six low-income neighbourhoods. Block randomization split these six neighbourhoods into two equal blocks. The three low-income neighbourhoods that received only the telephone call reminder had a complete child MMR immunization coverage rate of 48.7% (95% CI 39.5% to 57.8%). The three low-income neighbourhoods that received a telephone call reminder along with the additional option of a home visit by a public health nurse had a complete child MMR immunization coverage rate of 60.5% (95% CI 52.5% to 68.6%). The results are shown in Table 4.
TABLE 4.
Group | n/N | % | 95% CI |
---|---|---|---|
Three low-income neighbourhoods (telephone reminder) | 56/115 | 48.7 | 39.5–57.8 |
Three other low-income neighbourhoods (telephone reminder and home visit) | 86/142 | 60.5 | 52.5–68.6 |
DISCUSSION
For parents or guardians whose children were behind in routine MMR immunization coverage, 81.8% were simply not aware that their child was behind in immunizations. With regard to barriers to childhood immunization in those who realized that an immunization was due, 57.5% simply forgot to immunize their child. These findings seem to confirm a potential benefit to reminder systems. Differences in knowledge, beliefs and barriers among parents were not analyzed in the present study.
Our study found some benefit to telephone reminders. In the SHR, the relative increase in complete child immunization coverage rates was 10% compared with 4% in the control health region (RQHR). Although the nonoverlapping CIs suggest that the results are statistically significant and might have some clinical significance, the differences were modest. However, these results (74% overall child immunization coverage rate) were consistent with the findings of the meta-analysis mentioned earlier (4) in which 10 of 11 randomized trials of patient recall systems obtained relatively low immunization coverage rates ranging from 42% to 64%.
The increases in child immunization coverage in the SHR were essentially the same in all four neighbourhood groups including urban low income, urban middle income, urban high income and rural. Although we know that it is harder to immunize children from low-income neighbourhoods with disadvantaged backgrounds, the mathematical reality is that their low immunization coverage rates should make it easier to demonstrate positive gains (3,7–11).
In the SHR, the five-year average number of children born between October 2000 and September 2005 who were fully immunized for MMR was 2374 children per year. In the following intervention year (children born between October 2005 and September 2006), 2597 children were fully immunized, for an absolute increase of 223 children. The total cost to implement the telephone reminder system was $36,000 per year or $161 per child. This was the total cost to implement all aspects of the intervention including the evaluation.
The difference in outcome between the telephone reminder system and the telephone reminder system/home visit option in Saskatoon’s six low-income neighbourhoods was not statistically significant, although the positive trend might have some limited clinical impact. Regrettably, our study was not able to replicate the 90% child immunization coverage rates obtained in Alaskan children with similar socioeconomic barriers who had access to a home visit by a public health nurse (15,16).
In Alaska, child immunization coverage rates routinely exceed 90% despite traditional barriers such as poverty, a higher proportion of parents with less formal education and remote access. High child immunization coverage rates in Alaska are attributed to the initiation of an electronic monitoring system combined with home visits by public health nurses (15,16).
In Saskatoon, only 30 parents requested immunization through a home visit (86 children in three neighbourhoods compared with 56 children in three other low-income neighbourhoods) over a one-year period for a total cost of $60,000, or $2,000 per child. It is important to note that a nurse was hired exclusively for this project and, given few requests, home immunizations could be added to regular duties of existing nurses in future interventions. However, the lack of additional success with home immunizations was related more to an inability to contact parents than to intervention refusal.
The results of the study also suggest that a public education campaign might be of benefit. In our study, 24.7% and 29.5% of parents or guardians whose children were behind in immunizations believed that immunizations were associated with known or unknown side effects. A previous study found that the risk of anaphylaxis after vaccination of 7,644,049 children and adolescents was 0.65 cases per one million doses, while the risk of death was zero cases per one million doses (17). In addition, 80.3% of parents or guardians incorrectly believed that they should not immunize their child when they have a minor illness such as a cold.
The study is an example of a real-world study design attempting to actually increase complete child immunization coverage rates in a large and diverse population. A limitation of the study is that we were not able to locate and contact 31% of the study population. Essentially, all of the cases of parents who could not be contacted were due to incorrect telephone numbers with no known forwarding number. As such, this is also the major factor influencing the effectiveness of the intervention. The only potential solution to contact more parents is to link immunization records with other agencies such as social services, provided that appropriate privacy and confidential provisions are obtained.
CONCLUSION
Telephone reminder systems appear to have some benefit in increasing child immunization coverage rates in urban and rural settings. For children living in low-income neighbourhoods, the additional benefit of home visits with telephone reminders appears to be limited.
ACKNOWLEDGEMENTS
The authors thank the Saskatoon Health Region, Regina Qu-Appelle Health Region, Saskatoon Tribal Council and the Department of Pediatrics at the University of Saskatchewan.
Footnotes
FUNDING: This research was paid for by a grant from the Canadian Institutes of Health Research.
REFERENCES
- 1.Health Canada. Canadian Immunization Guide. 6th edn. Ottawa: Health Canada; 2002. [Google Scholar]
- 2.Lemstra M, Neudorf C, Opondo J, et al. Disparity in childhood immunizations. Paediat Child Health. 2007;12:847–52. doi: 10.1093/pch/12.10.847. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Szilagyi PG, Schaffer S, Shone L, et al. Reducing geographic, racial, and ethnic disparities in childhood immunization rates by using reminder/recall interventions in urban primary care practices. Pediatrics. 2002;110:e58. doi: 10.1542/peds.110.5.e58. [DOI] [PubMed] [Google Scholar]
- 4.Jacobson VJ, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates. Cochrane Database Syst Rev. 2005;(3):CD003941. doi: 10.1002/14651858.CD003941.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18:97–140. doi: 10.1016/s0749-3797(99)00118-x. [DOI] [PubMed] [Google Scholar]
- 6.Kendrick D, Hewitt M, Dewey M, et al. The effect of home visiting programs on uptake of childhood immunization: A systematic review and meta-analysis. J Public Health Med. 2000;22:90–8. doi: 10.1093/pubmed/22.1.90. [DOI] [PubMed] [Google Scholar]
- 7.Agbley D, Campbell H. Summary of factors affecting immunization uptake levels Childhood Immunization in England: Issues from the Research. London: Health Education Authority; 1998. [Google Scholar]
- 8.Marsh GN, Channing DM. Comparison in use of health services between a deprived and an endowed community. Arch Dis Child. 1987;62:392–6. doi: 10.1136/adc.62.4.392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Reading R, Colver A, Openshaw S, Jarvis S. Do interventions that improve immunization uptake also reduce social inequalities in uptake? BMJ. 1994;308:1142–4. doi: 10.1136/bmj.308.6937.1142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Lynch M. Effect of practice and patient population characteristics on the uptake of childhood immunizations. Br J Gen Pract. 1995;45:205–8. [PMC free article] [PubMed] [Google Scholar]
- 11.Fleming DM, Charlton JR. Morbidity and healthcare utilization of children in households with one adult: Comparative observational study. BMJ. 1998;316:1572–6. doi: 10.1136/bmj.316.7144.1572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Atcheson W, Palmer C, Tuchschere R. Saskatchewan immunization management system validity study. Regina: Saskatchewan Health; 2006. [Google Scholar]
- 13.Lemstra M, Neudorf C, Opondo J. Health disparity by neighborhood income. Can J Public Health. 2006;97:435–9. doi: 10.1007/BF03405223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rothman KJ, Greenland S. Modern Epidemiology. 2nd edn. Philadelphia: Lippincott Williams and Wilkins; 1998. [Google Scholar]
- 15.Centers for Disease Control and Prevention. Vaccination coverage levels among Alaska Native Children aged 19–35 months – National Immunization Survey, United States, 2000–2001. MMWR. 2003;52:710–3. [PubMed] [Google Scholar]
- 16.Strine TW, Mokdad AH, Barker LE, et al. Vaccination coverage of American Indian/Alaska native children aged 19 to 35 months: Findings from the national immunization survey 1998–2000. Am J Public Health. 2003;93:2046–9. doi: 10.2105/ajph.93.12.2046. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bohlke K, Davis RL, Marcy SM, et al. Risk of anaphylaxis after vaccination of children and adolescents. Pediatrics. 2003;112:815–20. doi: 10.1542/peds.112.4.815. [DOI] [PubMed] [Google Scholar]