Abstract
Background
Uptake of the human papillomavirus (HPV) vaccine is low in Appalachian Ohio, and area with high cervical cancer rates.
Methods
We conducted a group-randomized trial among 12 counties in Appalachian Ohio randomized to receive either an HPV vaccine (intervention counties) or influenza vaccine (comparison counties) multi-level intervention (MLI). Parents (n=337) who had a daughter aged 9 to 17 years who had not received the HPV vaccine were recruited from commercial lists. Clinics (N=24) and 119 providers from these clinics were also recruited. The primary outcome was medical record confirmed receipt of the first shot of the HPV vaccine three months after receiving the intervention among daughters of parents enrolled in the study. Secondary outcomes included receipt of the first HPV vaccine shot by 6 months and changes in provider knowledge.
Results
According to medical records, 10 (7.7%) daughters of intervention participants received the first shot of the HPV vaccine within three months of being sent the intervention materials compared to 4 (3.2%) daughters of comparison group participants (p=0.061). By six months, 17 (13.1%) daughters of intervention participants received the first HPV vaccine shot compared to eight (6.5%) daughters of comparison group participants (p=0.002). Provider knowledge about HPV increased (p<0.001, from baseline to post-education).
Conclusions
The MLI increased uptake of the HPV vaccine among girls aged 9 to 17 years, however, uptake was low.
Impact
To improve HPV vaccine uptake, attention to additional levels of influence (e.g., policy, community) and more elements within levels (e.g., reminders, automated prompts) may be needed.
Keywords: Human papillomavirus (HPV), HPV vaccines, multi-level interventions, adolescents, Appalachian region
Introduction
Studies have documented the association between human papillomavirus (HPV) and six types of cancers including cancers of the cervix, oropharynx, anus, vagina, vulva, and penis (1-5). In the U.S., a quadrivalent HPV vaccine was approved for females aged 9-26 years in 2006 and for males of the same age in 2009 (6,7) that protects against two oncogenic HPV types (16 and 18) responsible for most anogenital cancers and two non-oncogenic HPV types (6 and 11) responsible for most cases of genital warts (7). A bivalent HPV vaccine was approved in 2009, for females and in 2014, a 9-valent HPV vaccine was approved for females and males (7-9).
Uptake of HPV vaccines is higher in Australia (10-12) and some countries in Western Europe (13-15) than in the U.S., with current rates of 39.7% for girls and 21.6% for boys aged 13 to 17 completing the three-dose vaccine series (16). Recommendations focus on vaccination of girls and boys 11 to 12 years of age and as early as age 9, with catch-up vaccination suggested for 13-26 year old females and 13-21 year old males (17,18). While rates of HPV vaccine uptake are universally low, vaccine uptake should be more vigorously pursued among populations that have higher cervical cancer rates. One such population(19-22) is residents of Appalachia. Appalachia is a federally-designated region of the U.S., with deficits in income, educational attainment, business growth, and access to healthcare (23-25); moreover, cancer is the number one cause of mortality in this region (26,27).
Previous research in Appalachia found negative attitudes and poor knowledge about HPV, the HPV vaccine and cervical cancer among parents (28, 29). Using community-based participatory research strategies, our team, in partnership with community members from local cancer coalitions (advocacy organizations), developed a multi-level intervention (MLI) to improve HPV vaccination among girls aged 9 to 17 in Appalachian Ohio (30). Since previous studies have mainly focused on addressing one or two levels of influence (e.g., providers or parents) (31,32), and our formative research (30) suggested that multiple levels influence the decision to get the HPV vaccine in this population, we thought that a MLI, addressing three levels most influential (health clinics, providers, and parents of girls aged 9 to 17 years), was appropriate. The theoretical framework for the intervention was guided by several theories including the Health Belief Model (33), Theory of Reasoned Action (34), Extended Parallel Process Model (35), and Organizational Developmental Theory (36). The goal of this paper is to describe and report the results of a group randomized trial (GRT) at the county-level testing the efficacy of the MLI to improve HPV vaccination among this population.
Materials and Methods
This study was conducted from 2010 to 2015. Originally, the study design called for one clinic from each county to participate and provide access to providers and patients to approach for recruitment. Clinics provided letters of support for the grant application, but when asked to provide or contact patients, clinics declined participation. The study design, therefore, was modified, as described below. CONSORT diagrams outline the number of parent participants (Figure 1) and clinics and providers recruited (Figure 2) by study arm and followed. This study was approved by the Institutional Review Board of the OSU.
Figure 1. CONSORT Diagram of Parent Participants by Intervention Group.
Figure 2. CONSORT Diagram of Clinic and Provider Participants by Intervention Group.
Sample Selection
Counties
The Ohio State University (OSU) research team and community coalition members identified 12 counties to participate in the study. Counties were pair-matched based on cervical cancer incidence rates and location. One county from each pair was randomly assigned to receive the HPV vaccine intervention (n=6) while the other county was assigned to the comparison condition (n=6), an educational intervention (attention control) about influenza (flu) and the flu vaccine.
Clinics
Coalition members and study staff identified all health clinics in each of the 12 counties. Clinics were contacted by study staff, eligibility determined, and invited those eligible to participate. Clinics were eligible if they: 1) were located in one of the 12 counties in the study; 2) provided care to girls 9 to 17 years old; and 3) either provided immunizations or referrals for immunizations. A total of 98 clinics in Ohio Appalachia were approached to participate (Figure 2). Of these, 5 were not reached despite multiple attempts via phone, 14 were ineligible, and 41 (49%) declined to participate (too busy, no time, not interested). Thus, 24 clinics agreed (10 intervention, 14 comparison) to participate fully and 14 clinics participated only in the clinic-level intervention.
Providers
To be eligible for participation, providers had to be employed at one of the 24 fully participating clinics. Potential provider participants (staff involved in educating patients and/or administering vaccines) were recruited by their clinic manager or lead physician to attend a study education session conducted at regularly scheduled meetings.
Parents
Eligible parents had to: 1) be age ≥18; 2) be able to speak, read, and write English; 3) be a resident of one of the participating 12 Ohio Appalachian counties; 4) be a parent or a legal guardian of a young girl (age 9 to 17 years) who had not received the HPV vaccine (if a parent had more than one daughter, they were asked questions about their eldest daughter); and 5) have no children who have received the HPV vaccine (as confirmed by medical record review (MRR)).
Parents were sampled using the following procedures. A master list was first obtained from Survey Sampling International (37) consisting of all households from the 12 counties that had at least one female resident aged 9 to 17 years old. A random sample of households was drawn for each county and the names and addresses were given to study staff, who then mailed a recruitment letter and informational sheet introducing the study to the selected households. Study staff made at least 10 attempts (calling at different times of the day, weekday, and weekend, as well as sending an ‘unable to contact’ letter) to reach the listed household. Once the household was reached, the interviewer asked to speak to the parent who was most involved in making health decisions for their daughter. The study was then described, questions answered, informed consent obtained, and the baseline survey administered.
Intervention
Development of the MLI has been previously described (30), however, each level is briefly detailed below.
Clinic-level Intervention
The clinic-level intervention focused on providing an environment where information about HPV vaccination was visible and readily available. We developed study specific waiting room and examination room posters, brochures, and tabletop tent cards for the HPV vaccine intervention. Study staff regularly checked in with clinic staff to answer questions and check common areas and exam rooms to see if study or other educational materials were visible (on HPV vaccination) and replenished depleted materials.
Provider-level Intervention
The educational session for providers, facilitated by a member of the research team, included a 1-hour PowerPoint presentation and handouts on the HPV vaccine, focusing on current evidence-based HPV vaccine information and strategies designed to assist physicians in discussing HPV vaccination with parents. For the comparison arm, providers were given information on the flu and flu vaccine. For the HPV vaccine education session, we modified an evidence-based tobacco cessation program (38) focused on the “5 A's” and “5 R's.” Providers completed surveys that assessed HPV vaccine knowledge and attitudes, as well as self-efficacy to talk to parents and patients about both vaccines, before and after the educational session. At 12 months following the educational session, a survey was administered to providers to assess the durability of intervention effects.
Parent-level Intervention
After the initial phone call, intervention participants were mailed a packet that included a $10 thank you gift card for completing the telephone-administered baseline survey, an educational brochure and DVD video about HPV and HPV vaccination, a magnet reminder to get the 2nd and 3rd HPV vaccine shot, and a Centers for Disease Control and Prevention (CDC) HPV vaccine information statement. The comparison group was mailed a packet that included similar items, a flu vaccine information statement from the CDC and flu information sheets from Ohio Department of Health. The packet for both groups also included a medical record release form and a self-administered survey which they were instructed to return in an enclosed self-addressed prepaid. Participants were mailed a $5 gift upon receipt of these items.
Health educators conducted an education session about the HPV vaccine (or flu vaccine to the comparison group) via telephone to reinforce the message in the educational materials regarding the need for a vaccine and addressed any vaccination barriers or questions.
Measures
Clinic-level
An administrator from each clinic was asked to complete a questionnaire measuring patient volume, revenue sources, number of insured patients, and number of providers, prior to the introduction of the provider and patient components.
Provider-level
To determine knowledge about HPV and the HPV vaccine, providers were asked about state and national cervical cancer incidence and mortality rates, their knowledge about parental concerns regarding the HPV vaccine, and what they knew about patient-level predictors of HPV vaccination. Provider knowledge (in both groups) was measured by five true/false statements with correct answers representing higher levels of knowledge. Self-efficacy to talk to patients and parents about the HPV vaccine was assessed with two items. Provider demographic characteristics (age, gender, race, ethnicity, education, and job title) were also assessed.
Parent-level
HPV Vaccination
Whether daughters received the first HPV vaccination shot within three months after mailing the intervention materials was determined by MRR. This time frame was selected as individuals usually act close to cues to action, i.e., the educational intervention, than later (33). Medical records from any clinic a parent listed on the survey or release form were obtained, regardless if the clinic was participating in the study. Shortly after receiving the health educator call and one month later, a research interviewer contacted participants and administered a short phone survey to collect information on intent to vaccinate and vaccination status, for the specific vaccine (eg, HPV vaccine in intervention group, etc.). One additional phone call six months later was conducted to collect information on HPV vaccination in both groups. A gift card for each follow-up survey was sent to the participant as a thank you for their time. Participants could receive up to $40 in gift cards for returning all forms and surveys.
Secondary outcomes included HPV vaccine uptake at 6 months and uptake of the second and third HPV vaccine shots by MRR and parental report on the six-month phone survey. Six months was chosen as this is the recommended time frame to receive the 3 shot series, thus, we also assessed receipt of the second and third HPV vaccine dose among those who received the first dose.
Parent and Daughter Demographic Characteristics
Parents provided information about their daughter's age and race, and their own age, race, ethnicity, marital status, education, employment status, household income, and health insurance on the self-administered survey. General health and smoking status were asked in the post-education survey.
Parent-reported Daughter Vaccination Coverage
Parents were asked (yes/no) at baseline if their daughter had ever received a flu, tetanus, diphtheria, and pertussis (Tdap), or meningococcal (MCV) vaccine at baseline. At the six-month survey, parents were asked about HPV and flu vaccination for their daughter.
Intention to Vaccinate a Daughter
Parents were asked at baseline and post-education if they intended to have their daughter get the HPV vaccine. Responses for this item were measured on a five-point Likert scale (1=strongly disagree to 5=strongly agree). At the six month survey, parents of unvaccinated daughters were asked (yes/no) if they intended to have their daughter get the first dose of the HPV vaccine within the next month and in the next year.
HPV Susceptibility and Severity
Intervention group participants were asked at baseline and post-education about their daughter's susceptibility to getting HPV by responding to the item, “There is a high chance that my daughter will get HPV.” To measure perceived severity, participants were asked to respond to the item, “If my daughter were to get HPV it would be a very serious threat to her health.” Responses for these two items were measured on a seven-point Likert scale (1=strongly disagree to 7=strongly agree).
Self-efficacy and Intent to Encourage Daughter to Speak with Doctor
Intervention group participants were asked at baseline and post-education about their comfort in (self-efficacy) and intention to encourage their daughter to speak to a doctor about the HPV vaccine. Social influence on a participant's intent to encourage their daughter to speak to a doctor/nurse was also assessed. Responses for these items were measured on a seven-point Likert scale (1=strongly disagree to 7=strongly agree).
Knowledge, Attitudes, and Beliefs
Intervention participants were asked at baseline and post-education about their knowledge, attitudes, and beliefs about the HPV and the HPV vaccine. They were asked to respond to 10 true or false statements about HPV and HPV vaccine knowledge, and for beliefs and attitudes about HPV and the HPV vaccine, they responded to 16 items measured on a seven-point Likert scale (1=strongly disagree to 7=strongly agree).
Statistical Methods
The primary outcome was receipt of the first HPV vaccine shot within three months of the mailing of intervention materials, as determined by MRR. The sample size for the study was chosen based on power calculations assuming an HPV vaccination rate of 28% in the comparison counties (based on estimates from respective health departments) and a 58% vaccination rate in the intervention counties, based on previous studies that used educational and provider/system interventions to increase vaccine uptake (39). The Interclass Correlation Coefficient (ICC) used was 0.07, using the formula in Donner (40) and vaccination rates from the respective county health departments. Based on these parameters, a sample size of six counties per treatment arm and 23 participants per county was chosen to provide 85% power. We expected 15% attrition from baseline to follow-up, and thus recruited an additional five participants per county to provide a final sample size of 28 per county. Loss to follow-up over six months was actually 13.8% among intervention parents and 11.7% among comparison parents (p=0.557).
Since “missingness” of medical record data was 25.3% among intervention and 23.9% among comparison groups (p=0.644), the outcome was analyzed two ways: 1) an analysis of only participants with MRR data (complete case); and 2) an intent-to-treat analysis assuming participants without MRR data did not receive the shot, as is commonly done in studies where a drop-out is considered a treatment failure (41). In each analysis, a clustered permutation test was used to determine if the outcome distribution differed by study group. The test was implemented using a two-stage approach recommended by Gail et al. (42) to account for baseline factors that were unevenly distributed across treatment arms. The same analysis strategy was used to analyze several secondary outcomes: receipt of the first HPV vaccine shot within six months based on MRR, parent-reported first shot status, and receipt of the second and third shots.
Changes in attitudes and beliefs of parents in the intervention counties were examined by comparing the proportion who agreed/strongly agreed with statements about HPV and the HPV vaccine at baseline and at the post-education survey. Change in knowledge about HPV and the HPV vaccine after receiving the video was analyzed using a Wilcoxon sign rank test. Linear mixed models were used to test for changes in provider knowledge about HPV and HPV vaccine within each arm and to compare changes across arms from baseline to the 12 month survey.
Results
Characteristics of Participants
Of the 24 participating clinics, 22 representatives completed the questionnaire about clinic characteristics (Table 1). Clinics employed, on average, two physicians, four nurses, three nurses who administer vaccines, and four support personnel. The clinics provided care for, on average, approximately 3,400 adult and 1,300 child patients. On average, 40.7% of the practice revenue was from Medicaid, followed by Medicare (14.6%), other managed care (13.0%), and other insurers (12.2%). Lastly, the average percentage of uninsured patients was 22.4% for adults and 18.8% for children. The average number of physicians in the intervention clinics was considerably higher (3.1 versus 1.2) than the comparison clinics. All other clinic-level factors were similar across arms. All clinics displayed study-specific materials as directed, and material was restocked by study staff at all clinics.
Table 1. Distribution of Clinic Characteristics and Provider Demographics by Study Condition and Total.
| Clinic Variable | Control (N=12) |
Intervention (N=10) |
Total |
|---|---|---|---|
| Size of Clinic | |||
| Number of physicians | 1.2 ± 0.6 | 3.1 ± 2.6 | 2.1 ± 2.0 |
| Number of NPs or PAs | 0.7 ± 0.7 | 1.0 ± 1.2 | 0.8 ± 0.9 |
| Number of nurses | 3.8 ± 3.0 | 4.4 ± 3.3 | 4.1 ± 3.1 |
| Number of nurses who administer vaccines | 3.4 ± 2.4 | 3.4 ± 3.4 | 3.4 ± 2.8 |
| Number of nurses who never administer vaccines | 0.3 ± 1.2 | 1.0 ± 1.5 | 0.6 ± 1.3 |
| Number of MAs, aides, technologists | 1.1 ± 1.1 | 2.4 ± 2.3 | 1.7 ± 1.8 |
| Number of other support personnel | 3.3 ± 3.3 | 5.8 ± 4.5 | 4.4 ± 4.0 |
| Patient volume | |||
| Number of adult patients seen last year | 2841.4 ± 3811.7 | 4408.4 ± 5449.4 | 3401.1 ± 4322.6 |
| Number of children patients seen last year | 1647.0 ± 2040.9 | 825.2 ± 330.3 | 1318.3 ± 1610.2 |
| Percentage of practice revenue driven from | |||
| Medicare | 13.8 ± 14.3 | 15.7 ± 13.4 | 14.6 ± 13.4 |
| Other managed care | 14.0 ± 14.4 | 11.7 ± 14.3 | 13.0 ± 13.8 |
| Medicaid | 47.7 ± 30.6 | 30.2 ± 26.1 | 40.7 ± 29.3 |
| Other insurers | 13.4 ± 12.0 | 10.2 ± 9.6 | 12.2 ± 10.8 |
| Percentage of patients uninsured | |||
| Adult population | 14.4 ± 23.0 | 33.0 ± 32.1 | 22.4 ± 27.8 |
| Non-adult population | 16.2 ± 19.2 | 22.6 ± 22.6 | 18.8 ± 20.2 |
| Provider Variable | Control (N=62) | Intervention (N=57) | Total |
| Age (M ± SD) | 49.4 ± 13.4 | 49.0 ± 12.8 | 49.2 ± 13.1 |
| Gender | |||
| Male | 3 (4.8%) | 5 (8.8%) | 8 (6.7%) |
| Female | 58 (93.6%) | 52 (91.2%) | 110 (92.4%) |
| Hispanic | |||
| No | 62 (100%) | 56 (98.3%) | 118 (99.2%) |
| Yes | 0 (0%) | 1 (1.7%) | 1 (0.8%) |
| Race | |||
| White | 58 (93.6%) | 55 (96.5%) | 113 (95.0%) |
| Other | 4 (6.5%) | 2 (3.5%) | 6 (5.0%) |
| Highest level of education completed | |||
| High school | 5 (8.1%) | 3 (5.3%) | 8 (6.7%) |
| RN | 8 (12.9%) | 10 (17.5%) | 18 (15.1%) |
| Technical school | 12 (19.4%) | 7 (12.3%) | 19 (16.0%) |
| Associate degree | 12 (19.4%) | 14 (24.6%) | 26 (21.9%) |
| College Degree | 7 (11.3%) | 6 (10.5%) | 13 (10.9%) |
| Professional Degree | 13 (21.0%) | 16 (28.1%) | 29 (24.4%) |
| Title | |||
| Nurse | 31 (50.8%) | 26 (45.6%) | 57 (48.3%) |
| Physician | 6 (9.8%) | 5 (8.8%) | 11 (9.3%) |
| Other | 23 (37.7%) | 26 (45.6%) | 49 (41.5%) |
| Years of practice if nurse or doctor | 23.2 ± 13.9 | 24.0 ± 12.1 | 23.6 ± 13.0 |
Note: 2 out of the 24 clinics did not fill out the questionnaire regarding the clinic characteristics.
Data presented are mean ± SD.
Providers in the intervention counties were more likely to have a professional degree (28.1% versus 21.0%) and have a title other than physician or nurse (45.6% versus 37.7%). All other provider-level demographic variables were similar between groups. Loss to follow-up over 12 months was 33.3% among intervention and 24.2% among comparison providers (p=0.270).
The demographic characteristics of parent and child participants (n=337) are presented in Table 2. The mean age was 43.5 years, and the majority (98.5%) were white, representative of the Ohio Appalachian region. Most parents reported being married or living as a couple (89.3%), being employed full or part-time (63.4%), having health insurance (91.1%), annual household incomes <$70,000/year (63%), and at least some college education (71.2%). Parents in the intervention group were, on average, two years older (42.5 versus 44.5). All other demographic factors were similar across study arms for both parents and daughters.
Table 2. Demographic Characteristics Reported by Parent and Child Participants by Study Group (N=337).
| Variable | Flu Vaccine (Comparison) (N=163) n (%) |
HPV Vaccine (Intervention) (N=174) n (%) |
Total (N=337) |
|---|---|---|---|
| Parent variables | |||
| Age (M ± SD) | 42.5 ± 6.6 | 44.5 ± 6.6 | 43.5 ± 6.7 |
| Female Gender | 151 (92.6) | 160 (92.0) | 311 (92.3) |
| Non-Hispanic White | 160 (98.2) | 171 (98.3) | 332 (98.5) |
| Marital Status | |||
| Married/Living as Married | 146 (89.6) | 155 (89.1) | 301 (89.3) |
| Other | 17 (10.4) | 19 (10.9) | 36 (10.7) |
| Highest level of education completed | |||
| High School or less | 45 (27.6) | 52 (29.9) | 97 (28.8) |
| Some College/Associate's degree | 61 (37.4) | 61 (35.1) | 122 (36.2) |
| College degree/Graduate degree | 57 (35.0) | 61 (35.0) | 118 (35.0) |
| Employment status | |||
| Full/Part-time | 95 (58.3) | 118 (67.8) | 213 (63.4) |
| Disabled/Unemployed/Retired | 67 (40.7) | 56 (32.2) | 123 (36.6) |
| Annual household income | |||
| <$30K | 23 (20.5) | 23 (18.0) | 46 (19.2) |
| $30K-$69,999 | 51 45.5) | 54 (42.2) | 105 (43.8) |
| $70K+ | 38 (34.0) | 51 (39.8) | 89 (37.0) |
| Health insurance | |||
| No | 14 (8.6) | 16 (9.2) | 30 (8.9) |
| General health | |||
| Excellent/Very good | 80 (53.7) | 86 (55.4) | 166 (54.6) |
| Good | 44 (29.5) | 46 (29.7) | 90 (29.6) |
| Fair/Poor | 25 (16.8) | 23 (14.9) | 48 (15.8) |
| Smoking status | |||
| Never | 96 (64.4) | 105 (68.2) | 201 (66.3) |
| Former | 27 (18.1) | 29 (18.8) | 56 (18.5) |
| Current | 26 (17.5) | 20 (13.0) | 46 (15.2) |
| Years since Pap test | |||
| 0-3 | 117 (86.0) | 124 (89.2) | 241 (87.6) |
| 4+ | 19 (14.0) | 15 (10.8) | 34 (12.4) |
| Child variables | |||
| Age (M ± SD) | |||
| White, non-Hispanic | |||
| Ever received Flu vaccine | 96 (58.9) | 90 (51.7) | 186 (55.2) |
| Received Tdap Vaccine | 130 (94.9) | 147 (94.8) | 277 (94.9) |
| Received MCV Vaccine | 34 (24.8) | 44 (28.4) | 78 (26.7) |
| County Clinic as Usual Source of Care | 30 (18.4) | 20 (11.4) | 50 (14.8) |
Totals may not sum to stated sample size due to missing data. Tdap=combined Tetanus, Diptheria and Pertusiss vaccines); MCV=Meningococcal vaccine
Twenty parents in the intervention counties listed a participating intervention county clinic as their usual source of care and 30 parents in the comparison counties likewise listed a clinic in the comparison counties. None of the parents listed a clinic from the opposite group's county as a source of care. Of parents responding to the question at 6 months (147/155), 39% reported discussing the HPV vaccine with a healthcare provider, 44% reported they visited a healthcare provider but did not discuss the HPV vaccine, and 17% did not visit a healthcare provider since the baseline survey. Moreover, only 5.7% of parents reported visiting one of the participating clinics during the study.
HPV Vaccination Outcomes
Of the 337 parents enrolled in the study, we were able to obtain MRR data on the outcome for 254 (75%) daughters. The proportion of daughters with MRR data did not differ by study group (76.1% comparison, 74.7% intervention, age-adjusted p=0.644). Among the participants with MRR data, 10 (7.7%) daughters of intervention participants received the first dose of the HPV vaccine within three months compared to 4 (3.2%) daughters of comparison participants (age-adjusted p=0.061) (Table 3). Vaccine uptake estimates were lowered when it was assumed that daughters missing MRR data did not have the shot (age-adjusted p=0.067).
Table 3. Receipt of the First Dose of the HPV Vaccine.
| Outcome | Method | Received Shot | Flu Vaccine (Comparison) n (%) |
HPV Vaccine (Intervention) n (%) |
p-value | Adjusted p-valuea |
|---|---|---|---|---|---|---|
| MRR (within 3 months) | C.C. | Yes | 4 (3.2) | 10 (7.7) | 0.045 | 0.061 |
| No | 120 (96.8) | 120 (92.3) | ||||
| Null Imputation | Yes | 4 (2.5) | 10 (5.8) | 0.056 | 0.067 | |
| No | 159 (97.5) | 164 (94.2) | ||||
| MRR (within 6 months) | C.C. | Yes | 8 (6.5) | 17 (13.1) | 0.003 | 0.002 |
| No | 116 (93.5) | 113 (86.9) | ||||
| Null Imputation | Yes | 8 (4.9) | 17 (9.8) | 0.003 | 0.002 | |
| No | 155 (95.1) | 157 (90.2) | ||||
| Parent-Reported (within 6 months) | C.C. | Yes | 13 (9.0) | 26 (17.3) | 0.003 | 0.009 |
| No | 131 (91.0) | 124 (83.7) | ||||
| Null Imputation | Yes | 13 (8.0) | 26 (14.9) | 0.005 | 0.011 | |
| No | 150 (92.0) | 148 (85.1) |
Note: C.C. = complete case, Null Imputation = imputation assuming drop-out = no shot. MRR = medical record review.
Adjusted for age of parent.
Using MRR data, 17 (13.1%) daughters of intervention participants received the first HPV vaccine shot within 6 months compared to eight (6.5%) daughters of comparison participants (p=0.002). Parent-reported HPV vaccinations at 6 months was also significantly higher among daughters of intervention participants (17.3% versus 9.0%, p=0.009). Results were similar for each outcome when we assumed those with missing data did not receive the vaccine.
Among those who received the first shot by 6 months according to MRR, 12 (70.6%) daughters of intervention participants and three (37.5%) daughters of comparison participants received the second dose (p=0.119). Among those who received the second HPV vaccine shot according to MRR, six (50%) daughters of intervention participants and two (66.7%) daughters of comparison participants received the third dose (p=0.524).
For intervention parents who completed the baseline and post-education survey (n=155), 60 (38.7%) agreed/strongly agreed with the statement “I intend to encourage my daughter to get the HPV vaccine” before the intervention which significantly increased to 91 (58.7%) following the post-education survey (p<0.001). At six months, 124 parents of unvaccinated daughters in the intervention group provided information about intent. Of those 124, 40 (32.3%) indicated that they intend to have their daughter receive the first dose of the HPV vaccine within the next year. Parents who did discuss the HPV vaccine with their healthcare provider were more likely (OR=3.43, 95% CI=1.19-9.87) to have their daughter receive the HPV vaccine than those who did not discuss the HPV vaccine with their healthcare provider.
Parent Knowledge, Attitudes, and Beliefs
Among the 155 intervention participants who responded to questions about viewing the intervention materials, 139 (89.7%) reported that they watched the video and 146 (94.2%) reported that they read the brochure and vaccine informational statement. Intervention group parents averaged 9.4 ± 1.0 correct answers to HPV and HPV vaccine knowledge questions (out of ten) on the post-education survey, which was significantly higher than their knowledge score at baseline (7.4 ± 2.1, p<0.001 from Wilcoxon sign-rank test) and the baseline score of comparison group parents (7.3 ± 1.9, p=0.001 from clustered permutation test). Several belief and attitude items about HPV and the HPV vaccine also changed between the baseline and post-education surveys (See Table 4).
Table 4. Beliefs and Attitudes about HPV and the HPV Vaccine Reported by Parent Participants from the Intervention Counties.
| Variable | Pre-Intervention | Post-Intervention | p-value | ||
|---|---|---|---|---|---|
| Yesa n (%) | Noa n (%) | Yesa n (%) | Noa n (%) | ||
| There is a high chance that my daughter will get HPV | 15 (9.7) | 140 (90.3) | 29 (18.7) | 126 (81.3) | <0.01 |
| If my daughter were to get HPV it would be a severe threat to her health | 97 (63.0) | 57 (37.0) | 109 (70.8) | 45 (29.2) | 0.07 |
| I would be comfortable encouraging my daughter to talk to her doctor/nurse about the HPV vaccine | 121 (78.1) | 34 (21.9) | 136 (87.7) | 19 (12.3) | 0.01 |
| I would be comfortable talking to my daughter's doctor/nurse about the HPV vaccine | 150 (96.8) | 5 (3.2) | 152 (98.1) | 3 (1.9) | 0.63 |
| The HPV vaccine would prevent my daughter from getting certain types of HPV | 105 (96.8) | 50 (32.3) | 138 89.0) | 17 (11.0) | <0.01 |
| I intend to encourage my daughter to get the HPV vaccine | 60 (38.7) | 95 (61.3) | 91 (58.7) | 64 (41.3) | <0.01 |
| My family would want me to encourage my daughter to talk to her doctor/nurse about the HPV vaccine | 72 (46.8) | 82 (53.3) | 100 (64.9) | 54 (35.1) | <0.01 |
| Most people where I am from (besides my family) would want me to encourage my daughter to talk to her doctor/nurse about the HPV vaccine | 66 (42.9) | 88 (57.1) | 79 (51.3) | 75 (48.7) | 0.08 |
| The cost of the HPV vaccine prevents me from getting it for my daughter | 7 (4.5) | 148 (95.5) | 2 (1.3) | 153 (98.7) | 0.13 |
| My daughter does not need the HPV vaccine because she is not sexually active | 40 (25.8) | 115 (74.2) | 27 17.4) | 128 (82.6) | 0.047 |
| It is hard/difficult to find a place that offers/gives the HPV vaccine in my community | 5 (3.2) | 150 (96.8) | 4 (2.6) | 151 (97.4) | 1.00 |
| My daughter's doctor/nurse has not recommended the HPV vaccine for my daughter | 61 (39.9) | 92 (60.1) | 69 (45.1) | 84 (54.9) | 0.20 |
| I am worried about the safety of the HPV vaccine | 73 (47.1) | 82 (52.9) | 61 (39.4) | 94 (60.6) | 0.065 |
| I am concerned about the HPV vaccine because it is too new | 73 (47.1) | 82 (52.9) | 65 (41.9) | 90 (58.1) | 0.22 |
| The HPV vaccine gives girls a license to have sex | 7 (4.5) | 148 (95.5) | 1 (0.7) | 154 (99.4) | 0.07 |
| I am concerned about the shot being painful for my daughter | 16 (10.3) | 139 (89.7) | 12 (7.7) | 143 (92.3) | 0.45 |
Yes= strongly agree, agree; No=slightly agree, not sure, slightly disagree, disagree, strongly disagree
Provider-level Outcomes
Prior to the in-clinic education session, intervention group providers averaged 4.4 correct answers to HPV and HPV vaccine questions out of five (N=57, SD=0.7) which significantly (p<0.001) increased to an average of 4.9 correct out of five following the education session (N=55, SD=0.3). The average number of correct responses by these providers decreased to 4.6 (N=37, SD=0.7) at 12 months, which was still significantly better than their baseline knowledge (p=0.035), but was similar to the 12 month change observed in the comparison group (p=0.87). Self-efficacy to talk to parents and patients about the HPV vaccine in the intervention group was similar at the baseline (89% for both questions) and 12 month surveys (83% to talk to parents and 92% to talk to patients).
Discussion
The goal of this study was to test the efficacy of a MLI to improve the uptake of the HPV vaccine among Appalachian girls aged 9 to 17 years old. The primary outcome, receipt of the first HPV vaccine shot within three months, was more common in the intervention group though vaccination in both groups was very low (less than 10%). These numbers nearly doubled by 6 months to 13.1% but still remained low, making it was difficult to assess completion of the second and third shots among those who received the first shot.
MLIs are gaining more interest as a way to reduce disparities among underserved populations (46,47). While these interventions have been tested for smoking cessation and healthy diet, none have been directed at HPV vaccine uptake among girls. Perkins (31) found that a provider-focused intervention at two federally qualified community health centers consisting of six to eight education sessions, feedback about providers' and practices' HPV vaccination rates, and quality improvement incentives (credits for maintaining board certification), significantly increased HPV vaccine initiation among females in the active intervention period compared to comparison clinics, however, differences did not remain significant at the 6 month post-intervention period. Two studies directed at parents have also shown positive effects. Aragones (43) used a using a nonequivalent group design to test an education session plus text-messaging intervention compared to an education session intervention among Mexican-American parents with a child who needed the HPV vaccine. Based on parental report, there was an 88% series completion rate among those receiving the first HPV vaccine dose in the education plus text messaging group compared to 40% in the education only group (p=0.004). Parra-Medina (44) tested a promotora outreach, education, and navigation program for HPV vaccination among Hispanic women with a daughter who did not receive the HPV vaccine. Compared to the brochure-only parent participants, those parents who received the promotora navigation program were more likely to complete the HPV vaccine series. Thus, it appears from these studies that more intensive, multi-component interventions directed at either clinicians and/or parents are effective in increasing vaccination rates.
The current study, while demonstrating a greater effect in the intervention compared to the comparison group, did not have a large effect on the uptake of the HPV vaccine. Due to difficulties in accessing patients at the clinics where the intervention was delivered, parents were recruited from county-level commercial lists. Thus, few daughters of parents recruited visited the participating clinics, resulting in the small effect we found. Moreover, while knowledge about HPV and HPV vaccine increased among providers in participating intervention clinics, the impact of this increase on vaccination rates was not able to be assessed because of study design limitations. However, data indicate that the majority of intervention parents (83% among respondents) did have an opportunity to discuss HPV vaccination with a provider during the study period. This did not result in increased uptake of the vaccine, again perhaps due to the small percent of participants who visited study clinics (5.7%).
Among parents who did not vaccinate their daughter, only 32% intended to have their daughter vaccinated in the next year. This suggests that additional levels of influence, e.g., policy (in the form of school mandates) or societal level (changing norms to accept the vaccine) should be included in future MLI studies. Another possible reason for the low uptake could have been that the intervention may require a “lag time” for effects on behaviors to be realized (45,46) .
This study has several strengths. We used a GRT design which allowed for comparisons of outcomes across similar groups. Moreover, the primary outcome, receipt of the first HPV vaccine shot, was assessed by MRR and showed that participant self-reports overestimated the receipt of the shot in both groups. Our study used MRR as the gold standard, as we have found MRR to be reliable in the past (47,48), as have other studies (49,50). The MLI tested was developed with significant input from providers, parents, and relevant community members, and thus, was culturally appropriate and addressed the misconceptions and questions parents and providers have about HPV, the HPV vaccine, and cervical cancer. The intervention also changed knowledge, attitudes, and beliefs about HPV and the HPV vaccine at both the provider and parent level.
There were, however, several weaknesses. Changes in rules at participating clinics in relation to HIPAA interpretation and enforcement limited our ability to recruit parents from participating clinics, as initially planned. We also were not able to “tease out” which of the levels of the MLI had an effect, an inherent weakness of MLI's, in general. More in-depth process evaluation in future studies may assist in teasing out effects of each component of MLI's. In addition, we were limited in what behavioral determinants we intervened on and did not address provider communication which has been addressed in other interventions (31, 51). Another weakness is that the sample only included parents of adolescent girls as at the time the study began the HPV vaccine was only approved for girls. Future studies should examine how parents of adolescent boys and girls respond to interventions to improve HPV vaccine uptake. Lastly, most participants were white and living in Ohio Appalachia, limiting generalizability across other parts of the country.
Conclusion
In summary, a MLI designed with input from the Appalachian community, increased uptake of the HPV vaccine among girls aged 9 to 17 years compared to a comparison group; however, the increase was much lower than expected. The results suggest that future studies should focus on patients within clinics where all levels of the planned MLI can be implemented. Additional levels of influence and components within each level should be tested, such as using the features of electronic medical record systems and automated reminder letters and/or phone calls, as some studies have found successful (52,53), or interventions focused on provider communication strategies and other behavioral determinants impacting parental actions. Efforts to address low uptake of the HPV vaccine among this population need to continue.
Acknowledgments
Financial Support: This study was funded by a grant from the National Cancer Institute (P50 CA105632, E.D. Paskett) and the Behavioral Measurement Shared Resource at The Ohio State University Comprehensive Cancer Center (P30CA016058, M.A. Caligiuri).
Footnotes
Conflict of Interest Statement: E.D. Paskett has received research grants from Merck Sharp & Dohme Corp. P.L. Reiter has received research grants from Merck Sharp & Dohme Corp. and Cervical Cancer-Free America, via an unrestricted educational grant from GlaxoSmithKline. These funds were not used to support this research study.
References
- 1.Schiffman M, Hildesheim A. Cervical cancer. In: Schottenfeld D, Fraumeni J, editors. Cancer Epidemiology and Prevention. 3. New York: Oxford; 2006. pp. 1044–67. [Google Scholar]
- 2.Mayne S, Morse D, Winn D. Cancers of the oral cavity and pharynx. In: Schottenfeld D, Fraumeni J, editors. Cancer Epidemiology and Prevention. 3. New York: Oxford; 2006. pp. 674–96. [Google Scholar]
- 3.Frisch M, Melbye M. Anal cancer. In: Schottenfeld D, Fraumeni J, editors. Cancer Epidemiology and Prevention. 3. New York: Oxford; 2006. pp. 830–40. [Google Scholar]
- 4.Madeline M, Daling J. Cancers of the vulva and vagina. In: Schottenfeld D, Fraumeni J, editors. Cancer Epidemiology and Prevention. 3. New York: Oxford; 2006. pp. 1068–74. [Google Scholar]
- 5.Wideroff L, Schottenfeld D. Penile cancer. In: Schottenfeld D, Fraumeni J, editors. Cancer Epidemiology and Prevention. 3. New York: Oxford; 2006. pp. 1166–72. [Google Scholar]
- 6.Quadrivalent human papillomavirus vaccine recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Morb Mortal Wkly Rep. 2007;56:1–24. [PubMed] [Google Scholar]
- 7.Markowitz LE, Dunne EF, Saraiya M, Chesson HW, Curtis CR, Gee J, et al. Human papillomavirus vaccination: recommendations of the Advisory Committee on Immunization Practices (ACIP) MMWR Recomm Rep. 2014;63:1–30. [PubMed] [Google Scholar]
- 8.U.S. Food and Drug Administration (FDA) [Accessed October 26, 2015];Vaccines, blood, and biologics: Human papillomavirus vaccine. Available: www.fda.gov/biologicsbloodvaccines/vaccines/approvedproducts/ucm172678.htm. Updated January 29, 2015.
- 9.Recommendations on the use of quadrivalent human papillomavirus vaccine in males- Advisory Committee on Immunization Practices (ACIP), 2011. MMWR Morb Mortal Wkly Rep. 2011;60:1705–8. [PubMed] [Google Scholar]
- 10.Brotherton JM, Fridman M, May CL, Chappell G, Saville AM, Gertig DM. Early effect of the HPV vaccination programme on cervical abnormalities in Victoria, Australia: an ecological study. Lancet. 2011;377:2085–92. doi: 10.1016/S0140-6736(11)60551-5. [DOI] [PubMed] [Google Scholar]
- 11.Donovan B, Franklin N, Guy R, Grulich AE, Regan DG, Ali H, et al. Quadrivalent human papillomavirus vaccination and trends in genital warts in Australia: analysis of national sentinel surveillance data. Lancet Infect Dis. 2011;11:39–44. doi: 10.1016/S1473-3099(10)70225-5. [DOI] [PubMed] [Google Scholar]
- 12.Brotherton JM, Liu B, Donovan B, Kaldor JM, Saville M. Human papillomavirus (HPV) vaccination coverage in young Australian women is higher than previously estimated: independent estimates from a nationally representative mobile phone survey. Vaccine. 2014;32:592–7. doi: 10.1016/j.vaccine.2013.11.075. [DOI] [PubMed] [Google Scholar]
- 13.Marty R, Roze S, Bresse X, Largeron N, Smith-Palmer J. Estimating the clinical benefits of vaccinating boys and girls against HPV-related diseases in Europe. BMC Cancer. 2013;13:10–21. doi: 10.1186/1471-2407-13-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Limia A, Pachón I. Coverage of human papillomavirus vaccination during the first year of its introduction in Spain. Euro Surveill. 2011;16:1–4. [PubMed] [Google Scholar]
- 15.Hopkins TG, Wood N. Female human papillomavirus (HPV) vaccination: global uptake and the impact of attitudes. Vaccine. 2013;31:1673–9. doi: 10.1016/j.vaccine.2013.01.028. [DOI] [PubMed] [Google Scholar]
- 16.Rahman M, Laz TH, McGrath CJ, Berenson AB. Provider recommendation mediates the relationship between parental human papillomavirus (HPV) vaccine awareness and HPV vaccine initiation and completion among 13-to 17-year-old U.S. adolescent children. Clinical Pediatr. 2015;54:371–5. doi: 10.1177/0009922814551135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.American College of Obstetricians and Gynecologists. Human papillomavirus vaccination. Committee opinion No. 47. Obstet Gynecol. 2010;116:800–3. doi: 10.1097/AOG.0b013e3181f680c8. [DOI] [PubMed] [Google Scholar]
- 18.Centers for Disease Control and Prevention (CDC) FDA Licensure of bivalent human papillomavirus vaccine (HPV2, Cervarix) for use in females and updated HPV vaccination recommendations from the Advisory Committee on Immunization Practices (ACIP) MMWR Morb Mortal Wkly Rep. 2010;59:626–9. [PubMed] [Google Scholar]
- 19.Hariri S, Unger ER, Sternberg M, Dunne EF, Swan D, Patel S, Markowitz LE. Prevalence of genital human papillomavirus among females in the United States, the National Health and Nutrition Examination Survey, 2003-2006. J Infect Dis. 2011;204:566–73. doi: 10.1093/infdis/jir341. [DOI] [PubMed] [Google Scholar]
- 20.Reiter PL, Katz ML, Ruffin MT, Hade EM, DeGraffenreid CR, Patel DA, et al. HPV prevalence among women from Appalachia: results from the CARE project. PLoS One. 2013;8:e74276. doi: 10.1371/journal.pone.0074276. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Horner MJ, Altekruse SF, Zou Z, Wideroff L, Katki HA, Stinchcomb DG. U.S. geographic distribution of prevaccine era cervical cancer screening, incidence, stage, and mortality. Cancer Epidemiol Biomarkers Prev. 2011;20:591–9. doi: 10.1158/1055-9965.EPI-10-1183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Reiter PL, Fisher JL, Hudson AG, Tucker TC, Plascak JJ, Paskett ED. Assessing the burden of HPV-related cancers in Appalachia. Human Vaccin Immunother. 2013;9:90–6. doi: 10.4161/hv.22389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Pollard K, Jacobsen LA. The Appalachian region: a data overview from the 2006-2010 American Community Survey. 2012 Available at: http://www.arc.gov/assets/research_reports/PRB-DataOverview-2012.pdf.
- 24.PDA, Inc., and the Cecil B. Sheps Center for Health Services Research. Health care costs and access disparities in Appalachia. 2012 Available at: http://www.arc.gov/assets/research_reports/healthcarecostsandaccessdisparitiesinappalachia.pdf.
- 25.Halverson JA. An analysis of disparities in health status and access to health care in the Appalachian region. 2004 Available at: http://www.arc.gov/research/researchreportdetails.asp?REPORT_ID=82.
- 26.Wingo PA, Tucker TC, Jamison PM, Martin H, McLaughlin C, Bayakly R, et al. Cancer in Appalachia, 2001-2003. Cancer. 2008;112:181–92. doi: 10.1002/cncr.23132. [DOI] [PubMed] [Google Scholar]
- 27.Blackley D, Behringer B, Zheng S. Cancer mortality rates in Appalachia: descriptive epidemiology and an approach to explaining differences in outcomes. J Community Health. 2012;37:804–13. doi: 10.1007/s10900-011-9514-z. [DOI] [PubMed] [Google Scholar]
- 28.Reiter PL, Katz ML, Paskett ED. Correlates of HPV vaccination among adolescent females from Appalachia and reasons why their parents do not intend to vaccinate. Vaccine. 2013;31:3121–5. doi: 10.1016/j.vaccine.2013.04.068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ruffin MT, Hade EM, Gorsline MR, DeGraffinreid CR, Katz ML, Kobrin SC, Paskett ED. Human papillomavirus vaccine knowledge and hypothetical acceptance among women in Appalachia Ohio. Vaccine. 2012;30:5349–57. doi: 10.1016/j.vaccine.2012.06.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Katz ML, Paskett ED. The process of engaging members from two underserved populations in the development of interventions to promote the uptake of the HPV vaccine. Health Promot Pract. 2015;16:443–53. doi: 10.1177/1524839914559776. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Perkins RB, Zisblatt L, Legler A, Trucks E, Hanchate A, Gorin SS. Effectiveness of a provider-focused intervention to improve HPV vaccination rates in boys and girls. Vaccine. 2015;33:1223–9. doi: 10.1016/j.vaccine.2014.11.021. [DOI] [PubMed] [Google Scholar]
- 32.Fu LY, Bonhomme LA, Copper SC, Joseph JG, Zimet GD. Educational interventions to increase HPV vaccination acceptance: a systematic review. Vaccine. 2014;32:1901–20. doi: 10.1016/j.vaccine.2014.01.091. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Janz NK, Becker MH. The health belief model: a decade later. Health Educ Q. 1984;11:1–47. doi: 10.1177/109019818401100101. [DOI] [PubMed] [Google Scholar]
- 34.Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall; 1980. [Google Scholar]
- 35.Witte K. Fear control and danger control: a test of the extended parallel process model (EPPM) Commun Monogr. 1994;61:113–34. [Google Scholar]
- 36.French WL, Bell CH. Organizational development: behavioral sciences interventions for organization improvement. Englewood Cliffs, NJ: Prentice-Hall; 1973. [Google Scholar]
- 37.Survey Sampling International [Internet] Shelton, CT. c2015 [updated 2015; cited 2016 Feb 8]. Available from: https://www.surveysampling.com/about/
- 38.Fiore MC, Jaen CR, Baker TB, Bailey WC, Benowitz NL, Curry SJ, et al. Treating tobacco use and dependence: 2008 update. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service; 2008. [Google Scholar]
- 39.Briss PA, Rodewald LE, Hinman AR, Shefer AM, Strikas RA, Bernier RR, et al. Reviews of the evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community and Preventive Services. Am J Prev Med. 2000;18:97–140. doi: 10.1016/s0749-3797(99)00118-x. [DOI] [PubMed] [Google Scholar]
- 40.Donner A, Klar N. Design and analysis of cluster randomization trials in health research. New York: Oxford University Press; 2000. [Google Scholar]
- 41.Lichtenstein E, Glasgow RE. Smoking cessation: what have we learned over the last decade. J Consult Clin Psychol. 1992;60:518–27. doi: 10.1037//0022-006x.60.4.518. [DOI] [PubMed] [Google Scholar]
- 42.Gail MH, Mark SD, Carroll RJ, Green SB, Pee D. On design considerations and randomization based inference for community intervention trials. Stat Med. 1996;15:1069–92. doi: 10.1002/(SICI)1097-0258(19960615)15:11<1069::AID-SIM220>3.0.CO;2-Q. [DOI] [PubMed] [Google Scholar]
- 43.Aragones A, Bruno DM, Ehrenberg M, Tonda-Salcedo J, Gany FM. Parental education and text messaging reminders as effective community based tools to increase HPV vaccination rates among Mexican American children. Prev Med Rep. 2015;2:554–8. doi: 10.1016/j.pmedr.2015.06.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Parra-Medina D, Morales-Campos DY, Mojica C, Ramirez AG. Promotora outreach, education and navigation support for HPV vaccination to Hispanic women with unvaccinated daughters. J Cancer Educ. 2015;30:353–9. doi: 10.1007/s13187-014-0680-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Rosen LJ, Noach MB, Winickoff JP, Hovell MF. Parental smoking cessation to protect young children: a systematic review and meta-analysis. Pediatrics. 2012;129:141–52. doi: 10.1542/peds.2010-3209. [DOI] [PubMed] [Google Scholar]
- 46.Berwick DM. Disseminating innovations in health care. JAMA. 2003;289:1969–75. doi: 10.1001/jama.289.15.1969. [DOI] [PubMed] [Google Scholar]
- 47.Paskett ED, McLaughlin JM, Reiter PL, Lehman AM, Rhoda DA, Katz ML, et al. Psychosocial predictors of adherence to risk-appropriate cervical cancer screening guidelines: A cross sectional study of women in Ohio Appalachia participating in the Community Awareness Resources and Education (CARE) project. Prev Med. 2010;50:74–80. doi: 10.1016/j.ypmed.2009.09.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Krok-Schoen JL, Young GS, Pennell ML, Reiter PL, Katz ML, Post DM, et al. Testing Interventions to Motivate and Educate (TIME): A multi-level intervention to improve colorectal cancer screening. Prev Med Rep. 2015;2:306–13. doi: 10.1016/j.pmedr.2015.04.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Chao C, Slezak JM, Coleman KJ, Jacobsen SJ. Papanicolaou screening behavior in mothers and human papillomavirus vaccine uptake in adolescent girls. Am J Public Health. 2009;99:1137. doi: 10.2105/AJPH.2008.147876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Vanderpool RC, Cohen EL, Crosby RA, Jones MG, Bates W, Casey BR, et al. “1-2-3 Pap” intervention improves HPV vaccine series completion among Appalachian women. J Commun. 2013;63:95–115. doi: 10.1111/jcom.12001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Davis TC, Fredrickson DD, Kennen EM, Humiston SG, Arnold CL, Quinlin MS, et al. Vaccine risk/benefit communication: effect of an educational package for public health nurses. Health Educa Behav. 2006;33:787–801. doi: 10.1177/1090198106288996. [DOI] [PubMed] [Google Scholar]
- 52.Fiks AG, Grundmeier RW, Mayne S, Song L, Feemster K, Karavite D, et al. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics. 2013;131:1114–24. doi: 10.1542/peds.2012-3122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Suh CA, Saville A, Daley MF, Glazner JE, Barrow J, Stokley S, et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012;129:e1437–45. doi: 10.1542/peds.2011-1714. [DOI] [PubMed] [Google Scholar]