Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 May 1.
Published in final edited form as: Acad Pediatr. 2014 May-Jun;14(3):249–255. doi: 10.1016/j.acap.2014.02.003

Parent perceptions regarding novel immunization reminder/recall approaches: rural and urban differences

Alison W Saville 1, Brenda Beaty 1,2, Miriam Dickinson 1,2,3, Steven Lockhart 1, Allison Kempe 1,4
PMCID: PMC4128399  NIHMSID: NIHMS568277  PMID: 24767778

Abstract

Objectives

To assess among parents of young children: 1) preferences about the source of immunization reminder/recall (R/R) messages, 2) the degree of acceptability of different R/R modalities, and 3) factors that influence preferences, including rural and urban characteristics.

Methods

We conducted a survey among parents of children 19–35 months old who needed ≥1 immunization according to the Colorado Immunization Information System (CIIS). Equal numbers of urban and rural respondents were randomly selected. Up to four surveys were mailed to each parent who had a valid address.

Results

After removing invalid addresses, the response rate was 55% (334/607). Half of parents (49.7%) had no preference about whether the public health department or their child’s doctor sent reminders. Urban parents were more likely to prefer R/R come from their child’s doctor (46.7%) compared to rural parents (33.7%), p=.003. Mail was the preferred R/R method (57.7%), then telephone (17.0%), e-mail (12.7%), and text message (10.7%). Although not preferred, 60.1% reported it would be acceptable to receive R/R by e-mail and 46.2% by text message. Factors associated with preferring to receive R/R from their child’s doctor were urban residence and educational level of college graduate or greater.

Conclusions

A large portion of parents are willing to be reminded about vaccinations by their health department rather than their child’s provider and via novel modalities, such as e-mail or text messaging. Urbanicity and higher educational level were associated with preferring R/R come from a provider.

Keywords: immunizations, novel approaches, recall and reminder messages, parent perceptions, centralized reminder/recall, population-based reminder/recall

Introduction

Reminders, where patients are reminded of upcoming immunizations, and recalls, where patients are informed about overdue immunizations, have been shown to increase immunization rates among children and adolescents when conducted within the practice setting. 1–4 However, practices often have a difficult time conducting reminder/recall (R/R) because of barriers such as cost, competing demands in primary care practice and lack of experience in setting up systems to make R/R efficient and feasible. 1, 5–8 Nationally, data suggest that less than one in five practices are actively performing R/R for childhood vaccines.6

Due to the barriers practices face in conducting R/R, centralized approaches in which public health departments use immunization information systems (IIS) to remind or recall children at the population level have been proposed. In a recent trial, a centralized, population-based approach conducted by a public health department was shown to be more effective and cost-effective than traditional practice-based approaches.9 Furthermore, although most R/R studies have involved either mailed letters, postcards or telephone calls, newer communication technologies such as text messaging and e-mail may help decrease cost and potentially increase efficiency of the R/R process.10–16

Given the promise of these newer approaches to R/R, it is important to assess to what degree parents will be accepting of R/R messages coming from a public health department or by newer communication modalities, such as e-mail and text-messaging. Most previous research about parental attitudes and preferences regarding reminder/recall pertain to R/R modalities, 11,12, 14 with no prior assessments of preferences of the source of R/R.

Therefore, the objectives of the current study were to assess among parents of young children: 1) preferences about the source of immunization reminder/recall (R/R) messages, 2) the degree of acceptability of different R/R modalities, and 3) factors that influence preferences, including rural and urban location. Rural and urban differences were a particular focus because of differences in the way immunizations are delivered in urban and rural counties that might influence attitudes.

Methods

This study was approved by the Colorado Multiple Institutional Review Board as an expedited protocol.

Study Population and Setting

The survey sample was selected from seven counties that were randomized to the control arm of a randomized controlled trial involving 14 counties that was conducted in 2010. As previously described9 counties for the trial were selected based on similar characteristics including population size, median income, and participation in the Colorado Immunization Information System (CIIS). Frontier counties (defined as having fewer than seven people per square mile) by the US Census Bureau were excluded. Surveys were sent to a random sample of parents who had children aged 19–35 months old who needed at least one immunization according to CIIS. Children were identified as needing an immunization based on the national Advisory Committee on Immunization Practices (ACIP) recommended series of antigens to be received by the age of 19–20 months.17 Influenza was not included. A stratified random sample was identified with equal numbers of urban and rural counties to allow for comparisons.

Colorado Immunization Information System (CIIS)

The CIIS has met all functional Immunization Information System (IIS) standards set by the Centers for Disease Control and Prevention.18 Since 2004, CIIS has been populated with names and demographic information from the Colorado Electronic Birth Certificate database each week. Therefore, CIIS has the capacity to assess population-based immunization rates and to identify children who are overdue for immunizations based on recommendations from the ACIP17 by county of residence.

CIIS users in participating provider offices have access to view the consolidated record via the CIIS web application and to update immunization and demographic information. Colorado is not a mandatory reporting state, therefore not all providers giving immunizations report to CIIS. However, the percentage of Colorado children under six years of age with at least two immunization records in CIIS was 95%.19 Over 90% of public health entities, 91% of pediatric and 70% of family medicine practices in Colorado were enrolled in CIIS in 2011.19

Survey Design

The study took place January – April 2011. Survey questions were developed by the study team based on previous immunization-related survey instruments and were piloted with parents at a primary care clinic in both English and Spanish. Demographic questions are described in Table 1. Insurance status included private insurance, public insurance (Medicaid, CHP+, Tri-Care) and no health insurance. We used a four point Likert scale (strongly agree to strongly disagree) to assess parent preferences and attitudes about sources of R/R notices, see Table 2. Several discrete variable questions assessed the overall preferences for modalities (email, telephone, and text), sources of R/R, and acceptability of various modalities.

Table 1.

Characteristics of respondents (parent report)

Description Total % (n) Rural % (n) Urban % (n) P value
Gender
Male 10.9 (36) 9.1 (16) 13.0 (20) 0.27
Female 89.1 (293) 90.9 (159) 87.0 (134)
Total 100 (329) 100 (175) 100 (154)
Education
HS/GED or Less 27.9 (91) 27.3 (47) 28.6 (44) 0.81
Some College 27.9 (91) 27.3 (47) 28.6 (44)
College Grad 35.3 (115) 37.8 (65) 32.5 (50)
Advanced Degree 8.9 (29) 7.6 (13) 10.4 (16)
Total 100 (326) 100 (172) 100.1 (154)
Relationship to child
Mother* 88.8 (293) 89.1 (156) 88.4 (137) 0.59
Father 10.0 (33) 9.1 (16) 11.0 (17)
Grandparent 1.2 (4) 1.7 (3) 0.6 (1)
Total 100 (330) 99.9 (175) 100 (155)
Number of children < 19 in household
1 19.4 (64) 22.2 (39) 16.2 (25) 0.27
2 44.2 (146) 43.2 (76) 45.5 (70)
3 18.8 (62) 18.2 (32) 19.5 (30)
4 or more 17.6 (58) 16.5 (29) 18.8 (29)
Total 100 (330) 100.1 (176) 100 (154)
Regular place for healthcare
Yes 97.9 (327) 97.8 (174) 98.1 (153) 0.99
No 2.1 (7) 2.2 (4) 1.9 (3)
Total 100 (334) 100 (178) 100 (156)
Child’s health insurance
Private 53.5 (174) 49.1 (84) 58.4 (90) 0.04
Public 40.9(133) 45.7 (78) 35.8(55)
None 5.5 (18) 5.3 (9) 5.8 (9)
Total 99.9 (325) 100.1 (171) 100 (154)
Child has chronic health conditionβ
Yes 9.1 (30) 8.0 (14) 10.4 (16) 0.45
No 90.9 (299) 92.0 (161) 89.6 (138)
Total 100 (329) 100 (175) 100 (154)
Mean (sd) respondent Age in years 30.9 (6.7) 31.0 (7.3) 30.9 (6.0) 0.83
Open in a new tab
*

Note: 3 parents identified themselves as “mothers” and “male” gender

β

Does your child have a health condition or health problem that has lasted longer than 3 months?

Table 2.

Rural/Urban Differences: Attitudes about Sources of R/R Notices

Statements Location Agree strongly % (n) Agree somewhat % (n) Disagree strongly/somewhat % (n) P value
I support the HD sending reminders for children who need shots. Rural 60.8 (107) 29.0 (51) 10.2 (18) 0.27
Urban 54.5 (85) 33.3 (52) 12.2 (19)
I believe my child’s doctor is responsible for making sure my child gets shots. Rural 15.3 (27) 32.2 (57) 52.5 (93) 0.36
Urban 22.4 (35) 30.1 (47) 47.4 (74)
I believe it is the HD’s responsibility to make sure my child gets needed shots. Rural 8.0 (14) 29.3 (51) 62.7 (109) 0.59
Urban 12.9 (20) 28.4 (44) 58.7 (91)
It is not a good use of public money for the HD to send reminders to parents about their child’s shots. Rural 12.6 (22) 28.6 (50) 58.8 (103) 0.59
Urban 16.8 (26) 23.9 (37) 59.4 (92)
I think it is okay for my doctor to share information with the HD about my child, as long as it is kept private and confidential. Rural 25.0 (44) 46.6 (82) 28.4 (50) 0.69
Urban 27.3 (42) 41.6 (64) 31.9 (45)
I don’t think my doctor or the HD should remind me when my child needs a shot. Rural 7.5 (13) 9.8 (17) 82.7 (144)
Urban 6.4 (10) 12.2 (19) 81.4 (127) 0.84
My child’s doctor does a good job of reminding me when my child is in need of shots. Rural 44.6 (78) 35.4 (62) 20.0 (35) 0.28
Urban 59.0 (92) 19.9 (31) 21.1 (33)
I think reminders sent by the Health Department (HD) might help remind parents who do not have a doctor. Rural 72.7 (128) 24.4 (43) 2.9 (5) <0.01
Urban 63.5 (99) 28.2 (44) 8.3 (13)
If I had a choice, I would prefer to get a reminder from my child’s doctor rather than from the HD. Rural 37.5 (66) 38.1 (67) 24.5 (43) <0.01
Urban 50.6 (78) 33.8 (52) 15.6 (24)
I would be more likely to get shots for my child if I was reminded by my child’s doctor rather than by the HD. Rural 16.7 (29) 28.2 (49) 55.2 (96) <0.0001
Urban 31.8 (49) 33.1 (51) 35.0 (54)
Open in a new tab

Bolded statements are reverse order questions.

Survey Administration

The survey was conducted using a modified Dillman methodology for mailed surveys.20 All eligible parents received a pre-letter via mail stating the purpose and timing of the upcoming survey. All letters were addressed to the parent or guardian of the eligible child’s name. Parents self-selected which one would complete the form.

Address updating by providers in CIIS is variable so any undeliverable mail received during the first few weeks was updated, if possible, with the child’s last service provider. A paper-based, self-administered survey and a reminder postcard were then mailed to everyone with valid addresses four weeks later. Up to three mailed surveys were sent to non-responders for eight more weeks. A $5 bill was included with the first and fourth mailing. The study team continued to update inaccurate addresses throughout the survey administration period. If a child was identified as having moved out of the state or if an updated address was unable to be obtained, no additional surveys were mailed. Invalid addresses were removed from the response rate calculation, using standards developed by the American Association for Public Opinion Research (AAPOR).21

Data Analysis

Chi Square Tests and Mantel-Haenszel tests were used for categorical variables to compare preference and attitude differences between urban and rural responders. T-tests were used in comparing continuous variables. Our sample size provided 84% power to detect a 16% difference between rural and urban respondents on a dichotomous outcome.

We conducted a multivariable analysis with the outcome of preferring their child’s doctor to generate R/R messages regarding immunizations. Independent variables included in the multivariate model were: rural or urban residence, parental level of education, place of child’s medical care, length of time since child last saw doctor, and child’s health insurance. Characteristics significant at p <0.25 in bivariate analyses were tested in multivariable models, using a backwards elimination procedure in which the least significant predictor in the model was eliminated sequentially. At each step, estimates were checked to make sure other variables were not largely affected by dropping the least significant variable. This resulted in retention of only those factors that were significant at p <0.05 in the final model. Analyses were performed using SAS software, version 9.2 (SAS Institute, Cary, North Carolina).

Results

Within the seven counties, the CIIS report generated a list of 18,735 children (60% of total CIIS patient records) aged 19–35 months and in need of at least one immunization. The cohort was then stratified by urban and rural location. A total of 840 parents of children were randomly selected, with equal numbers in rural (420) and urban (420) counties. Baseline up-to-date rates were 41% for urban and 37% in rural counties. Only parents whose child was in need of at least one immunization were surveyed. Thirty-six names were removed because they had moved out of the state; 197 additional names were removed if a new address was not obtained from a provider (see Figure 1). The adjusted response rate was 55% (334/607) and was similar for rural [55% (178/323)] and urban [55% urban (156/284)] populations.

Figure 1. Flow Diagram for Study Population.

Figure 1

Open in a new tab

Mailings were discontinued (excluded) if undeliverable mail indicated the family moved out of state, or if the child’s provider could not update the address, or if the child did not have a last site of service provider in CIIS.

Reminder/recall preferred sources

Parents were asked about their preferences and attitudes about who sends R/R notices including the public health department, their child’s doctor, or either. Half of respondents (49.7%) preferred either their child’s provider or the health department to notify them when their child needed an immunization. More than a third (39.8 %) preferred being contacted by their child’s doctor, 6.8% preferred not to receive notices at all, and 3.7% percent preferred the health department contact them. There were distinct differences based on urban and rural location as shown in Figure 2, with parents in rural areas being more likely to support the involvement of the health department than those in urban areas.

Figure 2. Urban/Rural Differences Regarding Parent Preferences for Who Sends R/R Messages.

Figure 2

Open in a new tab

Overall, if your child needs a shot, who would you like to contact you?

Overall, 88.9 % of parents were supportive of the health department sending reminders for children who need immunizations (Table 2). Half of parents (50.2%) agreed strongly or somewhat that their child’s doctor was responsible for making sure their child got immunizations while 39.2% agreed somewhat or strongly that it was the health department’s responsibility. Only 17.9% of parents did not want reminders from either their child’s doctor or the health department. The majority of parents did not have issues with public money being spent on reminders or with doctors sharing data with health departments to perform reminder/recall. A majority of parents (79.5%) agreed somewhat or strongly that their child’s doctor did a good job reminding them when their child needed an immunization.

There were significant rural and urban differences for some statements with more urban parents (64.9% versus 44.9%), agreeing with the statement that they would be more likely to get their child immunized if reminded by their child’s doctor rather than the health department. More urban parents (84.4%) agreed that they preferred being reminded by their child’s doctor rather than the health department compared to rural parents (75.6%).

Reminder/recall modalities

The most preferred R/R modality by more than half of respondents (57.7%) was mail followed by telephone (17.0%), email (12.7%), text (10.7%) and other (2.0%). There were significant differences in the distribution of responses between urban and rural respondents. More urban respondents preferred text (13.5%) and email (18.4%) compared to rural respondents 8.2% and 7.5% respectively and rural respondents showed a slight preference to mail (62.9%) and telephone (18.2%) compared to urban respondents 51.8% and 15.6% respectively (p=0.01 overall).

In addition to reporting their preferred R/R method we asked parents to report if they were “okay” receiving R/R notifications in alternative ways. Ninety two percent of parents were “okay” being contacted via mail, 66.3% by telephone, 60.1% by email, and 46.2% by text message on their cell phone. Significant rural and urban differences were observed when asked about email message reminders with 66.7% of urban respondents versus 54.4% of rural respondents reporting it was okay to remind them via email message (p=0.03).

Factors associated with preferring reminder/recall by child’s doctor

A multivariate analysis was conducted with the outcome of preferring the child’s doctor send R/R notices for their child. Factors in the model associated with preferring their child’s doctor contact them were urban location and higher level of parent education, as shown in Table 3. On the contrary, parents whose child had received at least one immunization at a public health department (21% of the sample) were more likely to prefer the public health department to recall their child or had no preference for who recalled them compared to parents whose child had not received an immunization at a public health department. All other factors were not significant.

Table 3.

Factors associated with preferring child’s doctor send R/R notifications in response to following question: Overall, if your child needs a shot, who would you like to contact you?

Factors Response Biv. OR (95% CI) Multi. OR (95% CI)
Child ever received shot at a Public Health Clinic Yes 0.38 (0.20–0.71) 0.48 (0.25–0.92)
Urban Location 1.89 (1.19–3.00) 1.70 (1.04–2.80)
Education HS/GED or Less 0.33 (0.18–0.59) 0.33 (0.18–0.60)
Some college 0.45 (0.25–0.79) 0.44 (0.25–0.79)
College graduate or higher Ref. Ref.
Place of usual care (practice-type) Pediatrician Ref. Removed
Family physician 0.58 (0.36–0.95) p=0.31
Other 0.22 (0.08–0.62)
Length of time since child last saw a doctor for a checkup < 6 months Ref. Removed
6 month – <1 year 0.90 (0.54–1.48) p=0.12
1 year or more 0.28 (0.11–1.48)
Child’s health insurance Public 0.50 (0.30–0.82)
Private Ref. Removed
Other or None 0.29 (0.08–1.09) p=0.74
Open in a new tab
*

Removed=not entered into the final multivariate analysis.

Discussion

The findings of this study suggest that at least half of parents have no preference for whether the health department or their child’s doctor sends them reminder/recall notifications about their child. In general, rural parents and those with less than a college degree were more accepting of contact by the health department than urban parents. Our data also demonstrate that although most parents still prefer mailed reminder/recalls, over half consider telephone or email to be acceptable methods and roughly half are agreeable to receiving text reminders. Understanding parent perspectives about these novel approaches is especially salient given recent evidence demonstrating increased effectiveness and cost effectiveness of population-based R/R conducted by public health entities and increased access and usage of internet and cell phone technologies nationwide.

To our knowledge, there is no literature on parent preferences or acceptability about what healthcare entity should send out immunization R/R notices. Our survey showed some contradictory information on this issue. A majority agreed with a statement that they would prefer their child’s doctor remind them. However, when asked to choose who they would like to contact them for R/R, approximately half of parents reported no preference for whether they were reminded by the health department or their child’s provider. This parent data is particularly important given recent evidence that points to the effectiveness of centralized population-based R/R conducted by health departments or health networks.9,22

With at least half of parents showing no preference about what entity contacts them, it may be acceptable for health departments to assume this burden for private practices or, optimally, share the burden with a collaborative approach. For example, R/R messages could be generated using an IIS by a local or state public health agency and private physicians could provide updated patient contact (e.g. phone, address) and immunization information to health departments. Reminder/recall notifications could include both the name of the child’s physician and the public health department. This sort of collaboration exemplifies the type of public-private collaboration endorsed by a recent Institute of Medicine report, which recommends the collaboration of primary care and public health organizations to enhance population health.23 Although our survey did not address the issue of collaboration specifically, our data suggest that parents may be agreeable to and, ultimately, might prefer a more collaborative approach.

Only a few articles have looked at parent preferences regarding modalities for immunization recalls. One 2011 study found that parents preferred traditional modes of communication especially telephone or mailings, with younger and higher income parents preferring more novel approaches.12 Another recent small study of 50 parents also found general preferences for mail or phone,14 although parents identified more benefits than barriers to receiving text reminders regarding immunizations. In contrast, a 2013 study among urban primarily Latino, Spanish-speaking parents found 88% were comfortable receiving text messages about their child’s immunizations.11

Although not preferred, we found six out of 10 parents were open to being contacted by email and slightly less than half were open to receiving text messages. Access to internet and cell phone services continues to increase nationally24–28, and our data show support among many parents for their use in R/R. It is important to remember however, that rural America still has less access to these services.28, 29 Results from our study echo this disparity, with urban parents responding more favorably towards novel modalities compared to rural ones. Data from all these studies suggest that when health departments or practices consider using communication technology to perform immunization R/R, a one size fits all approach will likely be less successful than a purposeful decision taking into account urban or rural residence and demographic factors of the intended recipient, including age and socio-economic status. Acceptability of new communication technologies is also crucial to the realization of the cost-savings these technologies may provide over mail approaches, particularly by reducing the costs of printing, postage, and staff time.15, 16 Using novel R/R communication technology could reduce the cost burden among both private physicians and public health departments and could potentially motivate either entity to initiate it. In particular, text messaging seems to have the greatest potential in terms of scalability since on-going costs are only a few cents per message, which might be appealing especially to public health departments.

Our study also demonstrated substantial differences between rural and urban parents with respect to preferred sources of R/R, with rural parents being more accepting of notices sent from health departments. This difference could be explained in part because rural parents already interact more often with their local public health department. Healthcare in rural, less-populated areas in Colorado is more often collaborative, with rural providers often referring patients to public health departments for immunizations.30–36 On the other hand, urban practices are less likely to refer children to public health departments for immunizations or to work collaboratively with their public health department and, therefore, urban parents may be less familiar with the services provided and may be less accepting of reminders from them as a result.30, 31 These concepts are further supported in our study as parents whose child had received at least one immunization at a public health department were more accepting of approaches involving the public entities.

There are several strengths and limitations to the study. This was a population-based study utilizing a randomized approach to select eligible parents, thereby increasing the generalizability of the results. Our adjusted response rate was comparable to other parent surveys37 using standards developed by AAPOR. Using this method we did exclude parents with invalid addresses which may have introduced some bias. Additionally, respondents’ attitudes may differ from those who did not respond, and our sample is from one state and may not be nationally representative. In particular, rural population densities differ significantly between states and our findings may not be generalizeable to other rural areas nationally.38 In addition, urban counties in Colorado are not as densely populated as in many states with large, inner city populations and poverty rates are much lower in surveyed urban counties (8% to 13%) compared to rural counties (6% to 21%).39 Therefore, the urban-rural differences we found may not be entirely generalizable to other states. Although our sample appeared to be generally representative of the state of Colorado with respect to socioeconomic status and insurance coverage, it slightly over-represented parents with higher educational level. We oversampled rural parents which are more likely to have public insurance and to be less educated in Colorado.40–42 Our sample did indeed have lower percentages of private health insurance coverage (53.5% compared to 70.7% in the state) and higher rates of public insurance (40.9% compared to 31.1%); however, our sample also had a higher percentage of college graduates compared to the overall state (35% compared to 18%), implying some response bias. 40–42

The data presented here suggest that most parents are accepting of novel approaches such as texting and emailing for the receipt of reminder/recall messages and that they are less tied to the preference of such messages coming from their child’s primary care provider than might have been anticipated. Indeed, our findings suggest that more effective, cost-effective collaborative, population-based approaches to R/R with messages appearing to come from both the public health department and the child’s practice might be acceptable to both urban and rural parents. Given the reality that the vast majority of practices are unable or unwilling to conduct traditional R/R, the time to consider alternative, less expensive approaches seems warranted. Our data indicate that such approaches would likely satisfy the majority of parents if appropriately tailored.

What’s New?

Half of parents surveyed had no preference about whether reminder/recalls came from their child’s doctor or the public health department. Most parents preferred to receive reminder/recalls for vaccines by mail, but a large portion found email or text modalities acceptable.

Acknowledgments

The project described was supported by Award Number RC1LM010513 from the National Library of Medicine of the National Institutes of Health.

Footnotes

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Library of Medicine or the National Institutes of Health.

Material presented in part at the Pediatric Academic Societies Annual meeting Boston, Massachusetts April 28 – May 4, 2012.

The authors have no potential conflicts of interest or corporate sponsors.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Alison W. Saville, Email: Alison.Saville@ucdenver.edu.

Brenda Beaty, Email: Brenda.Beaty@ucdenver.edu.

Miriam Dickinson, Email: Miriam.Dickinson@ucdenver.edu.

Steven Lockhart, Email: Steven.Lockhart@ucdenver.edu.

Allison Kempe, Email: Allison.Kempe@childrenscolorado.org.

Reference List

  • 1.Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates: Edited 2009. Cochrane Database Syst Rev. 2009;(1):CD003941. doi: 10.1002/14651858.CD003941.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics. 1998;101(4):E3. doi: 10.1542/peds.101.4.e3. [DOI] [PubMed] [Google Scholar]
  • 3.Suh CA, Saville A, Daley MF, et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012;129(6):e1437–e1445. doi: 10.1542/peds.2011-1714. [DOI] [PubMed] [Google Scholar]
  • 4.Kempe A, Barrow J, Stokley S, et al. Effectiveness and cost of immunization recall at school-based health centers. Pediatrics. 2012;129(6):e1446–e1452. doi: 10.1542/peds.2011-2921. [DOI] [PubMed] [Google Scholar]
  • 5.Saville AW, Albright K, Nowels C, et al. Getting under the hood: exploring issues that affect provider-based recall using an immunization information system. Acad Pediatr. 2011;11(1):44–49. doi: 10.1016/j.acap.2010.12.009. [DOI] [PubMed] [Google Scholar]
  • 6.Tierney CD, Yusuf H, McMahon SR, et al. Adoption of reminder and recall messages for immunizations by pediatricians and public health clinics. Pediatrics. 2003;112(5):1076–82. doi: 10.1542/peds.112.5.1076. [DOI] [PubMed] [Google Scholar]
  • 7.Szilagyi PG, Schaffer S, Barth R, et al. Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Arch Pediatr Adolesc Med. 2006;160(2):157–163. doi: 10.1001/archpedi.160.2.157. [DOI] [PubMed] [Google Scholar]
  • 8.Pereira JA, Quach S, Heidebrecht CL, et al. Barriers to the use of reminder/recall interventions for immunizations: a systematic review. BMC Med Inform Decis Mak. 2012;12:145. doi: 10.1186/1472-6947-12-145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kempe A, Saville A, Dickinson LM, et al. Population-based versus practice-based recall for childhood immunizations: A randomized controlled comparative effectiveness trial. Am J Public Health. 2013;103(6):1116–23. doi: 10.2105/AJPH.2012.301035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA. 2012;307(16):1702–1708. doi: 10.1001/jama.2012.502. [DOI] [PubMed] [Google Scholar]
  • 11.Hofstetter AM, Vargas CY, Kennedy A, Kitayama K, Stockwell MS. Parental and provider preferences and concerns regarding text message reminder/recall for early childhood vaccinations. Prev Med. 2013 doi: 10.1016/j.ypmed.2013.04.007. [DOI] [PubMed] [Google Scholar]
  • 12.Clark SJ, Butchart A, Kennedy A, Dombkowski KJ. Parents’ experiences with and preferences for immunization reminder/recall technologies. Pediatrics. 2011;128(5):e1100–e1105. doi: 10.1542/peds.2011-0270. [DOI] [PubMed] [Google Scholar]
  • 13.Kharbanda EO, Stockwell MS, Fox HW, Rickert VI. Text4Health: a qualitative evaluation of parental readiness for text message immunization reminders. Am J Public Health. 2009;99(12):2176–2178. doi: 10.2105/AJPH.2009.161364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Ahlers-Schmidt CR, Chesser AK, Paschal AM, et al. Parent opinions about use of text messaging for immunization reminders. J Med Internet Res. 2012;14(3):e83. doi: 10.2196/jmir.1976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Car J, Gurol-Urganci I, de JT, Vodopivec-Jamsek V, Atun R. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database Syst Rev. 2012;7:CD007458. doi: 10.1002/14651858.CD007458.pub2. [DOI] [PubMed] [Google Scholar]
  • 16.Leong KC, Chen WS, Leong KW, et al. The use of text messaging to improve attendance in primary care: a randomized controlled trial. Fam Pract. 2006;23(6):699–705. doi: 10.1093/fampra/cml044. [DOI] [PubMed] [Google Scholar]
  • 17.Akinsanya-Beysolow I, Jenkins R, Meissner HC. Advisory Committee on Immunization Practices (ACIP) recommended immunization schedule for persons aged 0 through 18 years--United States, 2013. MMWR Surveill Summ. 2013;62 (Suppl 1):2–8. [PubMed] [Google Scholar]
  • 18.Centers for Disease Control and Prevention. [Accessed January 31, 2014.];Immunization Information System (IIS) Functional Standards 2013–2017. Available at URL: http://www.cdc.gov/vaccines/programs/iis/func-stds.pdf.
  • 19.Colorado Department of Public Health and Environment CIIS. Colorado Annual Immunization Progress Report for CY2011, Section IX - Immunization Information Systems. 2011 [Google Scholar]
  • 20.Dillman DA, Smyth J, Christian LM. Internet, Mail and Mixed-Mode Surveys: The Tailored Desgin Method. 3. New York, NY: John Wiley Co; 2009. [Google Scholar]
  • 21.The American Association for Public Opinion Research. [Accessed October 28, 2013.];Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. (4). 2008 Available at: URL: http://www.aapor.org/uploads/Standard_Definitions_07_08_Final.pdf.
  • 22.Szilagyi PG, Albertin C, Humiston SG, et al. A randomized trial of the effect of centralized reminder/recall on immunizations and preventive care visits for adolescents. Acad Pediatr. 2013;13(3):204–213. doi: 10.1016/j.acap.2013.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.IOM (Institute of Medicine) Primary Care and Public Health: Exploring Integration to Improve Population Health. Washington, D.C: The National Academies Press; 2012. [PubMed] [Google Scholar]
  • 24.Cisco. [Accessed May 30, 2013.];Cisco Visual Networking Index: Forecast and Methodology. 2011–2016 Available at URL: http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-481360_ns827_Networking_Solutions_White_Paper.html.
  • 25.eTForecasts. [Accessed March 28, 2013.];Internet User Forecast by Country. 2011 Available at URL: http://www.etforecasts.com/products/ES_intusersv2.htm.
  • 26.International Data Corporation. Android Expected to Reach Its Peak This Year as Mobile Phone Shipments Slow. [Accessed March 28, 2013.];According to IDC. 2012 Jun 6; Available at URL: http://www.idc.com/getdoc.jsp?containerId=prUS23523812.
  • 27.Euromonitor International. [Accessed March 28, 2013.];Mobile Phones in the US. Available at URL: http://www.euromonitor.com/mobile-phones-in-the-us/report#.
  • 28.U.S.Department of Commerce NTaIA. [Accessed March 28, 2013..];Digital Nation: Expanding Internet Usage. Available at URL: http://www.ntia.doc.gov/files/ntia/publications/ntia_internet_use_report_february_2011.pdf.
  • 29.U.S.Department of Commerce EaSANTaIA. [Accessed March 28, 2013.];Exploring the Digital Nation: Home Broadband Internet Adoption in the United States. Available at URL: http://www.ntia.doc.gov/files/ntia/publications/esa_ntia_us_broadband_adoption_report_11082010_1.pdf.
  • 30.Kempe A, Steiner JF, Renfrew BL, Lowery E, Haas K, Berman S. How much does a regional immunization registry increase documented immunization rates at primary care sites in rural colorado? Ambul Pediatr. 2001;1(4):213–216. doi: 10.1367/1539-4409(2001)001<0213:hmdari>2.0.co;2. [DOI] [PubMed] [Google Scholar]
  • 31.Renfrew BL, Kempe A, Lowery NE, Chandramouli V, Steiner JF, Berman S. The impact of immunization record aggregation on up-to-date rates--implications for immunization registries in rural areas. J Rural Health. 2001;17(2):122–6. doi: 10.1111/j.1748-0361.2001.tb00268.x. [DOI] [PubMed] [Google Scholar]
  • 32.Glazner JE, Steiner JF, Haas KJ, Renfrew B, Deutchman M, Berman S. Is reimbursement for childhood immunizations adequate? Evidence from two rural areas in colorado. Public Health Rep. 2001;116(3):219–25. doi: 10.1016/S0033-3549(04)50037-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Glazner JE, Beaty BL, Pearson KA, Berman S. The cost of giving childhood vaccinations: differences among provider types. Pediatrics. 2004;113(6):1582–7. doi: 10.1542/peds.113.6.1582. [DOI] [PubMed] [Google Scholar]
  • 34.Glazner JE, Beaty B, Berman S. Cost of vaccine administration among pediatric practices. Pediatrics. 2009;124 (Suppl 5):S492–S498. doi: 10.1542/peds.2009-1542H. [DOI] [PubMed] [Google Scholar]
  • 35.Freed GL, Cowan AE, Gregory S, Clark SJ. Variation in provider vaccine purchase prices and payer reimbursement. Pediatrics. 2008;122(6):1325–1331. doi: 10.1542/peds.2008-2038. [DOI] [PubMed] [Google Scholar]
  • 36.Freed GL, Cowan AE, Clark SJ. Primary care physician perspectives on reimbursement for childhood immunizations. Pediatrics. 2008;122(6):1319–1324. doi: 10.1542/peds.2008-2033. [DOI] [PubMed] [Google Scholar]
  • 37.Fincham JE. Response rates and responsiveness for surveys, standards, and the Journal. Am J Pharm Educ. 2008;72(2):43. doi: 10.5688/aj720243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Johnson K. Demographic Trends in Rural and Small Town America. Durham, NH: Carsey Institute, University of New Hampshire; 2006. [Accessed November 11, 2013.]. Available at URL: http://www.carseyinstitute.unh.edu/publications/Report_Demographics.pdf. [Google Scholar]
  • 39.US Census Bureau. [Accessed November 11, 2013.];American Community Survey, 5 year Estimates. Available at URL: http://factfinder2.census.gov.
  • 40.US Census Bureau. [Accessed October 17, 2013.];State and County Quick Facts: Colorado. Available at URL: http://quickfacts.census.gov/qfd/states/08000.html.
  • 41.US Census Bureau. US Bureau of Labor Statistics. [Accessed November 11, 2013.];The Annual Social and Economic Supplement to the Current Population Survey. 2012 Available at URL: http://www.census.gov/hhes/www/cpstables/032013/health/toc.htm.
  • 42.US Census Bureau. [Accessed November 11, 2013.];American Community Survey 1 Year Estimates. 2012 Available at URL: http://factfinder2.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_12_1YR_S0201&prodType=table.

RESOURCES