Abstract
We developed an electronic medical record (EMR)-based HPV vaccine decision support intervention targeting clinicians, (immunization alerts, education, and feedback) and families (phone reminders and referral to an educational website). Through telephone surveys completed by 162 parents of adolescent girls, we assessed the acceptability of the family-focused intervention and its effect on information-seeking behavior, communication, and HPV vaccine decision-making. The intervention was acceptable to parents and 46% remembered receiving the reminder call. Parents reported that the call prompted them to seek out information regarding the HPV vaccine, discuss the vaccine with friends and family, and reach a decision. Parents whose adolescent girls attended practices receiving the clinician-focused intervention were more likely to report that their clinician discussed the HPV vaccine at preventive visits. The results of this study demonstrate the acceptability and potential impact on clinical care of a comprehensive decision support system directed at both clinicians and families.
Introduction
The Human Papillomavirus (HPV) vaccine, which prevents cervical cancer, genital warts, precancerous lesions, and other conditions, has been licensed since 2006 for used in adolescent girls.1 Despite studies demonstrating the safety and efficacy of the HPV vaccine,2–4 vaccination rates remain low compared to other adolescent vaccines, with only 32% of girls aged 13–17 years fully vaccinated in 2010.5 Additionally, delay in vaccination is problematic because, as HPV is often contracted shortly following sexual initiation,6 it is important to vaccinate prior to sexual debut. In order for the vaccine to be fully effective, all three doses must be received.1 Families’ knowledge regarding the vaccine is often incomplete or inaccurate7, 8 and parents are known to have concerns regarding vaccine safety.9, 10 Further complicating efforts to promote vaccination, many parents believe that vaccine receipt may lead to earlier initiation of sexual activity,11, 12 perceive that their daughters are at low risk for contracting HPV,7, 9 and prefer to delay vaccination until the adolescent is older.9, 13 Studies also show that many clinicians are more willing to recommend the vaccine to older compared to younger girls.14, 15
To address these challenges, electronic medical record (EMR)-linked clinical decision support (CDS) provides a platform to provide a combination of educational content, reminders and feedback that may support communication regarding HPV vaccines and promote successful implementation of HPV recommendations. CDS systems have proven effective in modifying clinician behavior and improving patient outcomes by providing “clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care.”16, 17 Additionally, developing scalable and robust approaches for CDS design and implementation is among the top priorities for the American Medical Informatics Association and the Office of the National Coordinator for Health Information Technology.18
The objective of this study was to assess the acceptability of a decision support intervention, directed separately at parents and clinicians, among parents of adolescent girls and its effect on HPV vaccine communication and decision making.
Background
Studies demonstrate that clinician recommendation is one of the strongest predictors of HPV vaccination rates.19–21 Therefore, interventions that increase the likelihood that clinicians recommend the vaccine may improve immunization rates. Education for clinicians can support their ability to communicate accurate information regarding HPV vaccines and effectively address parents’ concerns. Audit and feedback, the approach of summarizing clinical performance over a specified interval, has been extensively used to deliver performance information to clinicians.22–25 Prior work has shown that this type of feedback may increase adherence to practice standards.22 Additionally, studies have shown that interventions using a multimodal approach including both education and feedback are more likely to be effective than those using a single approach to improve vaccination rates.26
The use of health IT interventions targeting families is increasing in both pediatric and adult healthcare settings, and many interventions focus on supporting families in receiving timely vaccination. Among adults, computer systems have been used to generate mailed reminders for influenza vaccination.27 For children, automatically dialed reminder calls or automated letters based on immunization data from computerized registries have proven effective and cost-effective.28–32 Additionally, a recent study found text reminders sent to parents of adolescents to be effective in increasing rates of tetanus-diptheria-acellular pertussis (Tdap) and meningococcal (MCV4) vaccination.33 However, these interventions did not provide educational content, and the approach of using automated systems to provide educational content to families to support decision-making has not been widely evaluated.
Given the health benefit of the HPV vaccine,2–4 known barriers to immunization, and low rates of receipt, novel strategies are needed to boost the initiation and completion of the vaccine series. Building upon previous work utilizing a CDS intervention for routine pediatric and influenza immunization,34, 35 our research team developed and tested a CDS intervention for adolescent vaccines that targets both parents and clinicians with the aim of improving vaccination rates among adolescent girls.
Methods
Study Setting
The clinical decision support intervention was implemented within the Children’s Hospital of Philadelphia (CHOP) Pediatric Research Consortium (PeRC), a two state, hospital-owned practice-based research network including more than 202,000 children and 33 practice sites linked by a common EMR, EpicCare (Verona, WI).
The intervention system was developed as part of a multi-site cluster randomized trial conducted at 22 of the PeRC practices: 4 urban resident teaching practices and 18 primarily suburban non-teaching practices. During the 1-year clinical trial, practices were randomized to receive clinician-focused decision support (education, immunization alerts, and feedback) or no intervention. Within each participating practice, families of girls eligible for any HPV vaccine dose during the study period were randomized to receive family-focused decision support (reminder phone calls and referral to a vaccine education website) or no intervention. The clinical trial also included decision support for tetanus, diphtheria and pertussis (Tdap) and menigoccocal conjugate (MCV) vaccines; however, the primary focus of this study was the HPV vaccine.
Intervention Development and Structure
The intervention system we developed employed multiple evidence-based strategies to influence HPV vaccine delivery and receipt in primary care. Figure 1 describes the flow of the system and how data analysis and reminder delivery were integrated with the EMR. The system supported an immunization history and reminder engine that was developed with logic from the CDC Immunization Schedules.1
Components of the Clinician-Focused Intervention
Education: We created an educational module for study clinicians, presented online and in live sessions. The module was intended to help clinicians feel comfortable discussing the HPV vaccine with families and to provide them with data to address concerns and motivate timely vaccination. The presentation included a review of published guidelines for adolescent vaccines, the epidemiology of vaccine-preventable conditions, vaccine efficacy and safety information, and local data summarizing the vaccination rates for HPV at study practices. We emphasized the importance of clinician recommendation of vaccines to families’ vaccine acceptance.
Immunization Alerts: In order to alert physicians during a visit that their patient was eligible for the HPV vaccine, we created an alert system called the Immunization Care Assistant that was embedded in the EMR. This application provided the clinician with an alert on all immunizations due and assisted the clinician by automating the order entry process. Ideally, the alert would prompt the physician to engage the parent in a discussion about the HPV vaccine, and to recommend that the adolescent receive the vaccine.
Audit and Feedback: We generated quarterly feedback reports with information on vaccination rates and delivered them to clinicians at study practices. The reports allowed each clinician to see their personal results and to compare their rates with their own clinic or the entire ambulatory network.
Components of the Family-Focused Intervention
Reminder Phone Calls: We used the EMR to identify patients who were due for any dose of the HPV vaccine. Automated, pre-recorded reminder phone calls based on this data were generated to notify parents that their child was eligible for the vaccine just prior to a scheduled clinic visit or, if the child was due and no visit was scheduled, to remind them to call the office to schedule a visit. Voicemail messages were left if the family did not answer. The reminder also included a reference to online vaccine education materials the parent could access. Recorded telephone reminders were chosen in this study rather than text-messaged or emailed reminders because the parent email address and specific permission to send an email or text message was rarely documented in the EMR at CHOP.
Parent-focused Education: Through the reminder phone calls, parents were referred to an educational website containing evidence-based, family-centered content related to the HPV vaccine.
Assessing the Acceptability of the Intervention to Parents and the Effect on Clinical Decision Making
In order to understand the impact of the decision support interventions described above on families, we conducted a nested cohort study involving telephone interviews of parents or guardians of adolescent girls enrolled in the clinical trial.
Study Population
All subjects were parents or guardians of adolescent girls, aged 11–17 years and enrolled in the randomized controlled trial, who had recently attended a preventive visit at a primary care practice participating in the study. We enrolled parents of girls who were due for HPV dose 1 only, which enabled us to assess the effect of the intervention on the decision to initiate the HPV vaccine series. We selected up to 8 families from each of the 22 study practices. We purposefully sampled an approximately equal number of families from each arm of the family-focused intervention and an equal number of families whose child had either received or had not received the first HPV vaccine dose at the preventive visit.
Survey Instrument
The survey instrument included open-ended, yes/no and Likert scale questions that explored the acceptability of the intervention, information-seeking behavior and communication between families and clinicians. The survey also included basic demographic information.
Outcomes
In this report, the primary outcome of interest was the acceptability of the parent-focused intervention. This was measured using an open-ended question regarding the subject’s first reaction to the reminder phone call. Effect of the family-focused intervention on decision making regarding HPV vaccines was assessed through open-ended and yes/no questions asking whether the subject was prompted by the call to seek information related to the HPV vaccine, where that information was sought, if the call prompted them to discuss the vaccines with others, and whether receiving the call made them more or less likely to receive the vaccine. Subjects were also asked whether they remembered the vaccine web site listed in the call, whether they visited the site, and reasons for visiting or not visiting the site. Effect of the clinician-focused intervention on the clinical encounter was assessed using multiple choice and 5-point Likert scaled questions. These items assessed whether the subject remembered their child’s clinician discussing the HPV vaccine with them at the visit, how strongly the clinician recommended the vaccine, and whether HPV vaccination was described as required or optional.
Exposures
The primary exposures were (1) receipt of a family-focused reminder call and (2) enrollment in a practice receiving the clinician-focused decision support.
Data Collection
The survey instrument used in this study was pilot-tested with 17 families in order to make minor modifications and finalize the questionnaire. Once the survey was in its final form, we conducted phone interviews with parents/guardians. Families were called shortly after the preventive care visit, and were identified using rosters created from the EMR. Demographic information including race and age of the adolescent were obtained from EMR. Verbal consent was obtained before administration of the questionnaire.
Data Analysis
Results were analyzed using descriptive statistics. Chi-squared tests or Fisher’s exact tests were conducted to compare proportions between groups. All analyses were conducted using Stata 11 (Stata-Corp, College Station, TX). This study was approved by the Children’s Hospital of Philadelphia Institutional Review Board.
Results
Study Population
Telephone interviews were conducted with 162 parents/guardians of adolescent girls age 11–17 who were due for HPV dose 1 and had a recent primary care visit (Table 1). Overall, 64% of the parents enrolled in the cohort study were white, 26% African American, and 10% other. 69% were parents of 11–13 year olds. The demographic characteristics of the adolescents in this study did not differ significantly from those in the larger clinical trial, though a slightly lower percent were in the 11–13 year age group (75% in the clinical trial, p=0.06).
Table 1.
N (%) | |
---|---|
| |
Study Population | 162 |
Race | |
White | 103 (64%) |
Black | 43 (26%) |
Other | 16 (10%) |
Age of Adolescent | |
11–13 | 112 (69%) |
14–17 | 50 (31%) |
Parent Educational Level | |
Less than College | 58 (36%) |
College Degree | 104 (64%) |
Practice Location | |
Urban | 31 (19%) |
Suburban | 131 (81%) |
Received HPV Dose 1 | 82 (51%) |
Attended Practice Receiving Clinician-Focused Decision Support | 82 (51%) |
Received Family-Focused Reminder Call | 80 (49%) |
Acceptability of Family-Focused Intervention and Effect on Decision Making
Approximately half of the subjects were parents/guardians of girls who had been randomized to the family focused reminder phone calls (80/162, 49%). Less than half of these subjects remembered receiving a reminder call that mentioned vaccines (37/80, 46%). Among those who remembered receiving the call, responses indicated the call was acceptable. The majority responded positively, with comments that they appreciated the reminder, were able to prepare their child in advance for receiving an injection, or had the opportunity to look up information before the upcoming visit. Only 2 parents reacted negatively by describing the reminder call as “pushy” or “inappropriate.”
Nearly half (16/37, 43%) of parents who remembered receiving a call about vaccines were prompted by the call to seek information related to the HPV vaccine. Parents reported seeking information online, from friends and family, and from their child’s clinician. Parents who identified as African American or other race were significantly more likely than white parents to report seeking information about the vaccine (90% vs. 26%, p=0.001).
Twenty-seven percent of parents who remembered receiving a call about vaccines said the reminder call affected the likelihood that they would have their child receive the vaccine. Two parents reported that the call prompted them to stop delaying the decision. Those with a college degree or higher (including an associates degree) were more likely than those with less than a college degree to report that the call affected their decision (37% vs. 0%, p=0.04).
Approximately half of the parents remembered hearing the study’s educational website mentioned during the call (19/37, 51%); however, none of the parents interviewed had visited the website. Despite efforts by the research team to ensure the name of the web site was clearly articulated and repeated, reasons given for not visiting the site included being unable to understand or remember the web address, having limited time or computer access, or feeling they already had enough information on the HPV vaccine to make a decision.
Effect of the Clinician-Focused Intervention on the Clinical Encounter
Overall, 77% (125/162) of parents reported that their child’s clinician discussed the HPV vaccine with them at the visit. However, parents who attended a practice that was receiving the clinician-focused decision support intervention were significantly more likely to report discussion of the vaccine at the visit (84% vs. 70%, p=0.02). There was no difference in the strength of the recommendation reported, however, with 68% of parents reporting strong clinician recommendation in both groups (p=0.9). Across both groups, a majority of parents (89%) reported that the clinician described the HPV vaccine as optional as opposed to required.
Discussion
By implementing a comprehensive vaccine decision support system directed at clinicians and families, we integrated multiple elements (clinician-focused reminders, education, and feedback as well as family-focused reminders and education) into a coherent intervention informed by EMR data.
Despite concerns that sending reminders about a potentially sensitive topic like HPV vaccination would generate controversy, the family-focused decision support was acceptable to families of adolescent girls who were due for the HPV vaccine and remembered receiving the call. There were few complaints from parents about the appropriateness of the calls, and responses suggest that the calls had an effect on the decision making process. In fact, multiple parents reported that the calls prompted specific behaviors including seeking out information, discussing the vaccine with others, and giving thought to their decision, even if they had previously procrastinated. These results suggest that, as long as a proper framework is used, family-focused decision support can be used in the context of controversial medical issues such as HPV vaccination.
We found that half of the parents surveyed remembered receiving the reminder phone call. The finding that many parents did not remember the call is not surprising, as other studies of reminder/recall interventions have found that calls may be ignored by families or not reach them due to wrong numbers or other factors.36, 37 However, nearly a third of parents who remembered the call actually reported changed behavior as a result. These results suggest that phone reminders might be an effective way of reaching families with decision support.
An additional component of the intervention in this study was the direction of families to a specific, evidence-based website. Though we hypothesized that families would visit the website listed in the reminder phone call, none of the subjects surveyed visited the site. This may be due to difficulty understanding or remembering the link to the website. It is noteworthy that parents did report that they were prompted by the calls to seek out information elsewhere--from other websites, family, friends or their child’s clinician. These results suggest that family-focused decision support might lead to more informed decision-making on the part of parents, though reminder phone calls may not be the best method of directing parents to specific educational content on the internet. Other vehicles for decision support, including text message or email reminders, may better direct parents to specific web-based educational content. However, in order to implement decision support using these methods on a large scale, information including email addresses, text message capable phone numbers and permission to send health-related messages must be documented as a part of routine practice, a workflow not yet adopted in many settings. Additionally, new patient portal systems have the potential to deliver decision support and educational material to patients.38–40 A web-based portal might deliver vaccine reminders to patients by email and include links or direct access to patient education resources tailored to specific families.
Our results suggest that aspects of the family-focused decision support intervention might work differently in parents with varied demographic characteristics. For example, non-white parents were more likely than white parents to report that the call prompted them to seek information about the HPV vaccine. Previous research has shown that African American and Hispanic caregivers in some areas may have less knowledge regarding the HPV vaccine than white caregivers;41, 42 it is possible that interventions designed to promote information-seeking might be particularly effective in this group.
We also found that college-educated parents were more likely than those with less education to report that the call affected the likelihood that they would vaccinate their child. It is possible that parents with less education preferred a greater level of clinician input. Prior research has found that caregivers with higher educational levels were more likely to report intention to seek out information about the HPV vaccine online;41 this may indicate greater confidence in their ability to seek health information on their own. Further research is needed to evaluate the impact of the decision support intervention among different subgroups of families.
This study had several limitations. First, the survey selection process was not random, but rather sought to obtain an adequate quota of complete responses. Our goal was to assess the acceptability of the phone contact process overall and among subgroups of families. Second, our survey instrument may not have captured all factors impacting the acceptability of the intervention to parents. We asked an open-ended question, but did not probe specific barriers. Additionally, we conducted the study in one health care system located in a single region of the country; as such, the study population likely does not represent the entire U.S. population. However, our results do show that the intervention was acceptable to parents of varying demographic characteristics and of younger as well as older adolescents.
Conclusion
This study used the EMR as the foundation for a multifaceted intervention to improve adolescent vaccine delivery in ambulatory settings. This approach is acceptable to families and might foster information-seeking, discussion with family and peers, and decision-making.
Acknowledgments
We would like to thank Jim Massey and PeRC for their assistance with the conduct of this study. We also thank the network of primary care physicians, the patients and their families for their contributions to clinical research. This research was conducted by the Children’s Hospital of Philadelphia under contract to the Agency for Healthcare Research and Quality, contract number HHSA 290-07-10013, Task Order 4, Rockville, MD. The authors of this article are responsible for its content. No statement may be construed as the official position of the Agency for Healthcare Research and Quality of the U.S. Department of Health and Human Services.
Footnotes
Conflict of Interest:
Drs. Fiks and Grundmeier are co-inventors of the “Care Assistant” that was used to provide clinical decision support in this study. They hold no patent on the software and to date have earned no money from this invention. Additionally, a member of Dr. Grundmeier’s family receives speaker’s fees from Merck for HPV vaccine. Prior to submission, results were independently reviewed and verified by one of the authors (RL) who had no conflict of interest.
References:
- 1.Markowitz LE, Dunne EF, Saraiya M, Lawson HW, Chesson H, Unger ER. Quadrivalent Human Papillomavirus Vaccine: Recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Recomm Rep. 2007;56(RR-2):1–24. [PubMed] [Google Scholar]
- 2.Joura EA, Leodolter S, Hernandez-Avila M, Wheeler CM, Perez G, Koutsky LA, et al. Efficacy of a quadrivalent prophylactic human papillomavirus (types 6, 11, 16, and 18) L1 virus-like-particle vaccine against high-grade vulval and vaginal lesions: a combined analysis of three randomised clinical trials. Lancet. 2007;369(9574):1693–1702. doi: 10.1016/S0140-6736(07)60777-6. [DOI] [PubMed] [Google Scholar]
- 3.Paavonen J, Jenkins D, Bosch FX, Naud P, Salmeron J, Wheeler CM, et al. Efficacy of a prophylactic adjuvanted bivalent L1 virus-like-particle vaccine against infection with human papillomavirus types 16 and 18 in young women: an interim analysis of a phase III double-blind, randomised controlled trial. Lancet. 2007;369(9580):2161–2170. doi: 10.1016/S0140-6736(07)60946-5. [DOI] [PubMed] [Google Scholar]
- 4.Villa LL, Costa RL, Petta CA, Andrade RP, Ault KA, Giuliano AR, et al. Prophylactic quadrivalent human papillomavirus (types 6, 11, 16, and 18) L1 virus-like particle vaccine in young women: a randomised double-blind placebo-controlled multicentre phase II efficacy trial. Lancet Oncol. 2005;6(5):271–278. doi: 10.1016/S1470-2045(05)70101-7. [DOI] [PubMed] [Google Scholar]
- 5.National and state vaccination coverage among adolescents aged 13 through 17 years--United States, 2010. MMWR Morb Mortal Wkly Rep. 60(33):1117–1123. [PubMed] [Google Scholar]
- 6.Winer RL, Feng Q, Hughes JP, O’Reilly S, Kiviat NB, Koutsky LA. Risk of female human papillomavirus acquisition associated with first male sex partner. J Infect Dis. 2008;197(2):279–282. doi: 10.1086/524875. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Olshen E, Woods ER, Austin SB, Luskin M, Bauchner H. Parental acceptance of the human papillomavirus vaccine. J Adolesc Health. 2005;37(3):248–251. doi: 10.1016/j.jadohealth.2005.05.016. [DOI] [PubMed] [Google Scholar]
- 8.Tiro JA, Meissner HI, Kobrin S, Chollette V. What do women in the U.S. know about human papillomavirus and cervical cancer? Cancer Epidemiol Biomarkers Prev. 2007;16(2):288–294. doi: 10.1158/1055-9965.EPI-06-0756. [DOI] [PubMed] [Google Scholar]
- 9.Dempsey AF, Abraham LM, Dalton V, Ruffin M. Understanding the reasons why mothers do or do not have their adolescent daughters vaccinated against human papillomavirus. Ann Epidemiol. 2009;19(8):531–538. doi: 10.1016/j.annepidem.2009.03.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Reiter PL, Brewer NT, Gottlieb SL, McRee AL, Smith JS. Parents’ health beliefs and HPV vaccination of their adolescent daughters. Soc Sci Med. 2009;69(3):475–480. doi: 10.1016/j.socscimed.2009.05.024. [DOI] [PubMed] [Google Scholar]
- 11.Davis K, Dickman ED, Ferris D, Dias JK. Human papillomavirus vaccine acceptability among parents of 10- to 15-year-old adolescents. J Low Genit Tract Dis. 2004;8(3):188–194. doi: 10.1097/00128360-200407000-00005. [DOI] [PubMed] [Google Scholar]
- 12.Waller J, Marlow LA, Wardle J. Mothers’ attitudes towards preventing cervical cancer through human papillomavirus vaccination: a qualitative study. Cancer Epidemiol Biomarkers Prev. 2006;15(7):1257–1261. doi: 10.1158/1055-9965.EPI-06-0041. [DOI] [PubMed] [Google Scholar]
- 13.Hughes CC, Jones AL, Feemster KA, Fiks AG. HPV vaccine decision making in pediatric primary care: a semi-structured interview study. BMC Pediatr. 2011;11:74. doi: 10.1186/1471-2431-11-74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kahn JA, Zimet GD, Bernstein DI, Riedesel JM, Lan D, Huang B, et al. Pediatricians’ intention to administer human papillomavirus vaccine: the role of practice characteristics, knowledge, and attitudes. J Adolescent Health. 2005;37(6):502–510. doi: 10.1016/j.jadohealth.2005.07.014. [DOI] [PubMed] [Google Scholar]
- 15.Riedesel JM, Rosenthal SL, Zimet GD, Bernstein DI, Huang B, Lan D, et al. Attitudes about Human Papillomavirus Vaccine among Family Physicians. J Pediatr Adolesc Gynecol. 2005;18(6):391–398. doi: 10.1016/j.jpag.2005.09.004. [DOI] [PubMed] [Google Scholar]
- 16.Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–1238. doi: 10.1001/jama.293.10.1223. [DOI] [PubMed] [Google Scholar]
- 17.Sittig DF, Ash JS, Zhang J, Osheroff JA, Shabot MM. Lessons from “Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system”. Pediatrics. 2006;118(2):797–801. doi: 10.1542/peds.2005-3132. [DOI] [PubMed] [Google Scholar]
- 18.Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A Roadmap for National Action on Clinical Decision Support. J Am Med Inform Assoc. 2007;14(2):141–5. doi: 10.1197/jamia.M2334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rosenthal SL, Weiss TW, Zimet GD, Ma L, Good MB, Vichnin MD. Predictors of HPV vaccine uptake among women aged 19–26: importance of a physician’s recommendation. Vaccine. 2011;29(5):890–895. doi: 10.1016/j.vaccine.2009.12.063. [DOI] [PubMed] [Google Scholar]
- 20.Brewer NT, Fazekas KI. Predictors of HPV vaccine acceptability: a theory-informed, systematic review. Prev Med. 2007;45(2–3):107–114. doi: 10.1016/j.ypmed.2007.05.013. [DOI] [PubMed] [Google Scholar]
- 21.Dempsey AF, Zimet GD, Davis RL, Koutsky L. Factors that are associated with parental acceptance of human papillomavirus vaccines: a randomized intervention study of written information about HPV. Pediatrics. 2006;117(5):1486–1493. doi: 10.1542/peds.2005-1381. [DOI] [PubMed] [Google Scholar]
- 22.Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care. 2006;15(6):433–436. doi: 10.1136/qshc.2006.018549. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006;(2):CD000259. doi: 10.1002/14651858.CD000259.pub2. [DOI] [PubMed] [Google Scholar]
- 24.Kanouse DE, Jacoby I. When does information change practitioners’ behavior? Int J Technol Assess Health Care. 1988;4(1):27–33. doi: 10.1017/s0266462300003214. [DOI] [PubMed] [Google Scholar]
- 25.Mugford M, Banfield P, O’Hanlon M. Effects of feedback of information on clinical practice: a review. BMJ. 1991;303(6799):398–402. doi: 10.1136/bmj.303.6799.398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(1 Suppl):97–140. doi: 10.1016/s0749-3797(99)00118-x. [DOI] [PubMed] [Google Scholar]
- 27.Hak E, van Essen GA, Stalman WA, de Melker RA. Improving influenza vaccination coverage among high-risk patients: a role for computer-supported prevention strategy? Fam Pract. 1998;15(2):138–143. doi: 10.1093/fampra/15.2.138. [DOI] [PubMed] [Google Scholar]
- 28.Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics. 1998;101(4):E3. doi: 10.1542/peds.101.4.e3. [DOI] [PubMed] [Google Scholar]
- 29.Dini EF, Linkins RW, Sigafoos J. The impact of computer-generated messages on childhood immunization coverage(2)(2) Am J Prev Med. 2000;19(1):68–70. doi: 10.1016/s0749-3797(00)00178-1. [DOI] [PubMed] [Google Scholar]
- 30.Franzini L, Rosenthal J, Spears W, Martin HS, Balderas L, Brown M, et al. Cost-effectiveness of childhood immunization reminder/recall systems in urban private practices. Pediatrics. 2000;106(1 Pt 2):177–183. [PubMed] [Google Scholar]
- 31.Gaglani M, Riggs M, Kamenicky C, Glezen WP. A computerized reminder strategy is effective for annual influenza immunization of children with asthma or reactive airway disease. Pediatr Infect Dis J. 2001;20(12):1155–1160. doi: 10.1097/00006454-200112000-00012. [DOI] [PubMed] [Google Scholar]
- 32.Szilagyi PG, Rodewald LE, Savageau J, Yoos L, Doane C. Improving Influenza Vaccination Rates in Children With Asthma: A Test of a Computerized Reminder System and an Analysis of Factors Predicting Vaccination Compliance. Pediatrics. 1992;90(6):871. [PubMed] [Google Scholar]
- 33.Stockwell MS, Kharbanda EO, Martinez RA, Lara M, Vawdrey D, Natarajan K, et al. Text4Health: Impact of Text Message Reminder-Recalls for Pediatric and Adolescent Immunizations. Am J Public Health. 2012;102(2):e15–e21. doi: 10.2105/AJPH.2011.300331. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Fiks AG, Grundmeier RW, Biggs LM, Localio AR, Alessandrini EA. Impact of clinical alerts within an electronic health record on routine childhood immunization in an urban pediatric population. Pediatrics. 2007;120(4):707–714. doi: 10.1542/peds.2007-0257. [DOI] [PubMed] [Google Scholar]
- 35.Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–169. doi: 10.1542/peds.2008-2823. [DOI] [PubMed] [Google Scholar]
- 36.Daley MF, Steiner JF, Brayden RM, Xu S, Morrison S, Kempe A. Immunization registry-based recall for a new vaccine. Ambul Pediatr. 2002;2(6):438–443. doi: 10.1367/1539-4409(2002)002<0438:irbrfa>2.0.co;2. [DOI] [PubMed] [Google Scholar]
- 37.Szilagyi PG, Schaffer S, Barth R, Shone LP, Humiston SG, Ambrose S, et al. Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Arch Pediatr Adolesc Med. 2006;160(2):157–163. doi: 10.1001/archpedi.160.2.157. [DOI] [PubMed] [Google Scholar]
- 38.Detmer D, Bloomrosen M, Raymond B, Tang P. Integrated personal health records: transformative tools for consumer-centric care. BMC Med Inform Decis Mak. 2008;8:45. doi: 10.1186/1472-6947-8-45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Ma C, Warren J, Phillips P, Stanek J. Empowering patients with essential information and communication support in the context of diabetes. Int J Med Inform. 2006;75(8):577–596. doi: 10.1016/j.ijmedinf.2005.09.001. [DOI] [PubMed] [Google Scholar]
- 40.Ross SE, Haverhals LM, Main DS, Bull SS, Pratte K, Lin CT. Adoption and use of an online patient portal for diabetes (Diabetes-STAR) AMIA Annu Symp Proc. 2006:1080. [PMC free article] [PubMed] [Google Scholar]
- 41.Hughes J, Cates JR, Liddon N, Smith JS, Gottlieb SL, Brewer NT. Disparities in how parents are learning about the human papillomavirus vaccine. Cancer Epidemiol Biomarkers Prev. 2009;18(2):363–372. doi: 10.1158/1055-9965.EPI-08-0418. [DOI] [PubMed] [Google Scholar]
- 42.Cates JR, Brewer NT, Fazekas KI, Mitchell CE, Smith JS. Racial differences in HPV knowledge, HPV vaccine acceptability, and related beliefs among rural, southern women. J Rural Health. 2009;25(1):93–97. doi: 10.1111/j.1748-0361.2009.00204.x. [DOI] [PubMed] [Google Scholar]