The Process of Engaging Members From Two Underserved Populations in the Development of Interventions to Promote the Uptake of the HPV Vaccine

Abstract

We describe a community-engaged research process used to develop multilevel interventions (caregivers, providers, system) to improve the uptake of human papillomavirus vaccine among adolescents by partnering with members from two underserved populations in Ohio. We began by conducting focus groups to better understand the knowledge and attitudes of caregivers and providers about the human papillomavirus vaccine and to develop teams of community members to assist with development of the interventions. The process continued with conducting writing sessions to determine the content and format of the interventions, and initial feedback was obtained during review sessions to refine the interventions prior to implementation. Using this approach, we were able to consider contextual factors that made the interventions more acceptable and relevant to members of the priority populations. Challenges included development and maintenance of a team of community members to participate in the entire intervention development process, rejection of ideas presented by academic researchers, the need to balance community members’ suggestions with what was known from evidence-based research, and the time, cost, and effort associated with partnering with community members. The benefits, however, outweigh the challenges associated with using a community-engaged research process to develop interventions aimed at reducing cancer disparities among underserved populations.

INTRODUCTION

Human papillomavirus (HPV) is the most common sexually transmitted infection (STI) in the United States (Dunne & Park, 2013; Weinstock, Berman, & Cates, 2004). Genital HPV prevalence has been reported to be 53.1% among heterosexual males (ages 18-70 years) and 42.5% among females (ages 14-59 years; Hariri et al., 2011; Nyitray et al., 2011). The prevalence of oral HPV is 6.9% among males and females (ages 14-69 years; Gillison et al., 2012). Persistent infection with oncogenic HPV types (mostly 16 and 18) can cause cervical, vaginal, vulvar, anal, penile, and oropharyngeal cancers (Bosch et al., 2013). Nononcogenic HPV types 6 and 11 cause more than 90% of genital warts (Camenga et al., 2013; Garland et al., 2009). In addition to gender and age, HPV prevalence varies by race, education, poverty level, and history of sexual behaviors (Hariri et al., 2011; Nyitray et al., 2011).

As a consequence of disparities in HPV infection rates, cervical cancer rates are higher among women from minority and low-income populations including African Americans, Hispanics, and women living in rural Appalachia (American Cancer Society [ACS], 2012, 2013a; Centers for Disease Control and Prevention [CDC], 2002; Reiter et al., 2013; Wingo et al., 2008; Yabroff et al., 2005). Reasons for cervical cancer disparities are numerous and complex and have been identified at multiple levels (individual, provider, system; ACS, 2013b; del Carmen & Avila-Wallace, 2013; Yabroff et al., 2005; Zapka, Taplin, Ganz, Grunfeld, & Sterba, 2012). In addition to differences in HPV prevalence rates, other factors play a role in cervical cancer disparities, including poor patient–provider communication, lack of access to health care such as cancer screening, cultural factors including social norms, and poor economic resources (del Carmen & Avila-Wallace, 2013; Friedman & Shepeard, 2007; Reiter et al., 2013; Wewers, Katz, Fickle, & Paskett, 2006; Yabroff et al., 2005; Zapka et al., 2012).

In the United States, the Advisory Committee on Immunization Practices (ACIP) currently recommends the quadrivalent HPV vaccine for adolescents to prevent cervical, vaginal, and vulvar cancer among females and to prevent genital warts and anal cancer among females and males (Markowitz et al., 2014). A bivalent HPV vaccine has been approved for the prevention of cervical cancer among females (Markowitz et al., 2014). Currently, the ACIP recommends a three-dose HPV vaccine series, with the second dose administered 1 to 2 months after the first dose and the third dose 6 months after the first dose. The ACIP recommends routine HPV vaccination for adolescents ages 11 to 12 years, but the vaccine is approved and may be administered as young as age 9, with catch-up vaccinations for ages 13 to 26 years (Markowitz et al., 2014).

HPV vaccination rates in the United States are lower than the Healthy People 2020 objective of 80% coverage among adolescent females (U.S. Department of Health & Human Services, 2010). HPV vaccine data from the National Immunization Survey-Teen provide the following estimated HPV vaccine rates: (1) receipt of ≥1 HPV vaccine dose, (2) receipt of ≥3 HPV vaccine doses, and (3) HPV 3-dose series completion (receipt of 3 doses among those who had at least on HPV vaccine dose with at least 24 weeks between the first dose and the interview date). The three HPV vaccine rates (≥1 dose, ≥3 doses, series completion) for 13- to 17-year-old females in 2013 were 57.3%, 37.6%, and 70.4% (CDC, 2014), respectively. Male HPV vaccination rates (≥1 dose, ≥3 doses, series completion) were lower with 34.6%, 13.9%, and 48.3% in the same year and among the same age-group (CDC, 2014). HPV vaccine ≥1 dose rates are higher among Black and Hispanic males and females compared to Whites; however, HPV 3-dose series completion rates are lower among minority females and males (CDC, 2014). HPV vaccination initiation and completion rates are similar among Appalachian (40.8% and 27.7%, respectively) and non-Appalachian females (43.6% and 25.3%, respectively; Reiter, Katz, & Paskett, 2012).

Based on known cervical cancer disparities and the potential for the HPV vaccine to assist with reducing disparities among minority and underserved populations, increasing HPV vaccination rates is one strategy to address the cervical cancer burden experienced by these population groups. Engaging community members to identify barriers and facilitators to HPV vaccination in order to inform educational curriculum and educational programs has been previously reported (Barnack-Tavlaris, Garcini, Sanchez, Hernandez, & Navarro, 2013; Brawner et al., 2013; Katz et al., 2009). Additionally, although HPV vaccine interventions to increase adolescent HPV vaccination rates have been reported, studies have been limited by one or more of the following: small sample size, inclusion of only one level (e.g., parents), HPV vaccine intention was measured instead of vaccine uptake, only completion of the second and third dose was targeted, and/or did not include community members in the development of the intervention (Cassidy, Braxter, Charron-Prochownik, & Schlenk, 2014; Fiks et al., 2013; Kharbanda, Stockwell, Fox, & Ricker, 2009; Moss, Reiter, Dayton, & Brewer, 2012; Reiter, Stubbs, Panozzo, Whitesell, & Brewer, 2011; Vanderpool et al., 2013).

To address low HPV vaccination rates among adolescents, cancer disparities researchers partnered with community members from Appalachian Ohio and Columbus, Ohio, to develop multilevel interventions to improve HPV vaccination rates among vaccine-eligible adolescents. We used this process since community-engaged research (CEnR) has been shown to be effective in addressing health disparities by developing culturally appropriate interventions that resonate with community members and improve the success of the interventions (Gehlert & Coleman, 2010; Israel et al., 2005). We also thought it was important to develop multilevel interventions to incorporate different components aimed at multiple behavioral influences on adolescent HPV vaccination, including caregivers, providers, and health care systems (Bosch et al., 2013).

The purpose of this article is to describe the CEnR process used with two distinct underserved population groups for the development of multilevel interventions to improve HPV vaccine uptake. We believe that the description of this interactive and iterative process will be useful for researchers and public health practitioners who want to develop health promotion and disease prevention interventions among different underserved populations in the future.

METHOD

The Process

Researchers partnered with caregivers (parents and grandparents) and health care providers to develop the theoretically based interventions. To make this a productive partnership and to minimize time commitment from community members, the intervention development process (Tables 1 and 2) was similar for both rural and urban population groups and was completed in several steps over a 1-year period.

TABLE 1. Summary of Intervention Development Process With caregivers.

Site	Step 1: Focus Groups	Step 2: Writing Sessions	Step 3: Review Sessions	Final Intervention Components
Appalachian  Ohio	Reviewed HPV vaccine awareness, knowledge, and barriers to HPV vaccination Team building Suggested video (DVD) format supplemented by brochure and reminders	DVD script developed and edited including scenes, setting, and characters Development and decisions made about print material content and format Refrigerator magnet suggested as a 2nd and 3rd dose reminder	Minor edits to brochure	DVD Brochure
Columbus,  Ohio	Reviewed HPV vaccine awareness, knowledge, and barriers to HPV vaccination Team building Suggested interactive computer program supplemented by brochure and reminders Suggested comic book about HPV vaccine for young adolescents	Interactive computer program narrative developed and edited including scenes and characters to include in embedded video clips Development and decisions made about print material content and format Agreed with refrigerator magnet being the 2nd and 3rd dose reminder	Minor edits to brochure Suggestion to add additional language options for materials	Interactive computer program Brochure

NOTE: HPV = human papillomavirus.

TABLE 2. Summary of Intervention Development Process With Health Care Providers.

Site	Step 1: Focus Group and Writing Sessions	Step 2: Review Sessions	Final Intervention Components
Appalachian  Ohio	Data to include: current HPV-associated cancer and HPV vaccine rates, tips to address vaccine barriers Ability of the program to be viewed at different times Provide current and relevant scientific papers Periodic newsletters to provide HPV vaccine updates	All materials were acceptable	PowerPoint presentation Current scientific papers Clinic: posters and reminder magnets Newsletters
Columbus,  Ohio	Data to include: current HPV-associated cancer and HPV vaccine rates, tips to address vaccine barriers, and communication strategies for limited health literacy Ability of the program to be viewed at different times Provide current and relevant scientific papers Periodic newsletters to provide HPV vaccine updates	Suggestion to add more pictures and increasing the speed of the PowerPoint presentation	PowerPoint presentation Current scientific papers Clinic: posters and reminder magnets Newsletters

NOTE: HPV = human papillomavirus.

Caregivers participated in focus groups so we could better understand their knowledge and attitudes about HPV and the HPV vaccine. Approximately 3 to 4 months later, two writing sessions were conducted, with 1 month between them. A review session was conducted approximately 6 months later, after development of a draft of intervention materials.

The process for health care providers was somewhat similar except that the first step combined focus group discussion to understand barriers to recommending the HPV vaccine and an intervention writing development process into one session. Approximately 6 months later, providers participated in a session to review a draft of intervention materials.

Development of the HPV vaccine intervention in Appalachian Ohio began prior to approval of the vaccine for males, and therefore the intervention focused on female adolescents. Sessions in Appalachia were held in 2010-2011. The HPV vaccine intervention in Columbus, Ohio, began after the HPV vaccine was approved for use among males, and consequently the intervention included both female and male adolescents. Sessions in Columbus were conducted in 2011-2012. Each step of the CEnR process used is described, followed by differences that emerged from the rural and urban populations. Participants (caregivers and providers) received a $25 gift card per hour of participation, and refreshments were served at all sessions. The study was approved by the Institutional Review Board of The Ohio State University.

Population Description

Residents of Appalachian Ohio are mostly White, have less education, and have increased poverty rates compared to residents of non–Appalachian Ohio (Ludke & Obermiller, 2012; Pollard & Jacobsen, 2013; Wewers et al., 2006). Community members from Gallia County, Ohio, participated in the intervention development, and the sessions were conducted in the county health department, which serves a mostly low-income population.

Columbus is the capital of Ohio, the 15th largest city in the United States, and located in Franklin County. Community members participated in the intervention development sessions conducted in one federally qualified health center (FQHC) serving mostly a minority and low-income population.

Caregivers

Step 1: Caregiver Focus Groups

There were three purposes for conducting the initial focus groups of caregivers: (1) documentation of knowledge, attitudes, and acceptability of the HPV vaccine; (2) generation of ideas for the intervention content and format; and (3) development of rapport among participants to build a team to work together during the intervention development process. Caregivers were recruited by hanging flyers in the health department and in the FQHC. The initial focus groups lasted 1 to 2 hours.

The first half of the focus group discussions, facilitated by a trained member of the research team, followed an interview guide and included a discussion of participants’ knowledge and attitudes about HPV and the HPV vaccine, including barriers to vaccination, sources of health information, and preferred communication channels for health-related issues. Next, because some caregivers were not aware of HPV, the focus group facilitator briefly explained that HPV is the most common STI in the United States and provided additional information about the HPV vaccine. The second half of the session was spent on brainstorming for ideas about the content and format of the intervention to improve HPV vaccination rates among adolescents.

To facilitate the discussion about potential intervention components, caregivers reviewed publically available HPV vaccine print materials and provided input about the content and design. For example, caregivers suggested that the print material include information for parents about how to talk to their children about the HPV vaccine. Caregivers also suggested a question and answer format for the print material since a caregiver may not think of every question to ask the provider. Caregivers also thought that some information was missing from other print materials. For example, they suggested including information about whether an individual should receive the HPV vaccine if that person has already been diagnosed with HPV. Additionally, the group focused on other options for ways to deliver the educational material, including videos, a website, Facebook, phone apps, text messages, group versus individual sessions, and so on.

The caregiver focus group in Appalachian Ohio consisted of nine parents (including two fathers) and one grandmother (legal guardian). Eight of the caregivers (including one father and one grandmother) from the focus groups participated in future intervention development and review sessions. The Appalachian caregivers suggested that a short video (DVD) would be the best format supplemented with a brochure and some type of second and thirrd HPV vaccine dose reminder. Caregivers eliminated any Internet component because many individuals living in rural Ohio do not have access to high-speed Internet connection. The rural caregivers suggested that the intervention be delivered at the health center.

In Columbus, 18 parents (one father) and one grandmother (legal guardian) were in the initial focus groups. A team of five female caregivers participated in the intervention development sessions, and four female caregivers reviewed materials. The caregivers in Columbus, Ohio, suggested that the format for the intervention should be more technology-driven including some type of interactive computer program that included embedded video clips. These caregivers thought that a short video would be boring to watch and would not engage most caregivers. The urban caregivers thought the intervention material should be delivered to caregivers in the health centers.

Step 2: Caregiver Writing Groups

Approximately 1 month later, the first caregiver writing group session was held and lasted about 3 hours. The same facilitator presented several potential story lines for the video or computer program that were suggested by the research team.

In Appalachia, caregivers thought that two story lines had potential for the video and worked together to blend both stories into one narrative focused on a family.

In Columbus, caregivers expressed different opinions about the story depending if the intervention was for caregivers of female versus male adolescents. Caregivers suggested that the story directed toward caregivers of female adolescents should include two female friends, since there are many single mothers who make health care decisions. Caregivers thought the story about male adolescents, however, should include a male role model, either a father or a male family member/friend, because their perspective would be important to consider in the decision about male adolescents receiving the vaccine.

The remaining time during the second writing session in both geographic locations was focused on developing the sequence of the scenes, graphics, and characters to include in the video or computer program. As each suggestion was made, the facilitator would write information on large sticky notes that were placed on the walls. These notes would be referred to as the groups made decisions about the story and characters. Once these initial decisions were made, the discussion focused on more detailed information including the location of the scenes, what the characters should be wearing, and so on. During this writing session, the group also began discussion about the format of the print materials, voicing their opinions about colors, size, and pictures. For example, caregivers requested the brochure be small so it would fit in a purse or pocket and to list trusted web-sites for individuals who wanted additional information from reliable sources. In addition, caregivers also began to explore different names for the project.

Approximately 1 month later, the second writing session was held and lasted 2 to 3 hours. The facilitator presented the draft of the script for each scene of the video or computer program. These drafts were written by members of the research team and addressed the relevant constructs of several behavioral theories (theory of reasoned action, extended parallel process model, and the PACE communication system; Ajzen & Fishbein, 1980; Cegala, McClure, Marinelli, & Post, 2000; Witte, 1994). As each scene was presented, caregivers assisted with editing the script to make the characters more realistic, often telling the facilitator that people in their community “do not talk like” what was written in the script. Further suggestions included what the characters should be doing in each scene (e.g., young female adolescent should be texting) and what music to include in the programs. Additionally, caregivers reviewed different suggestions for the name of the project and made final decisions about print materials and an HPV vaccine second and third dose reminder.

Step 3: Caregiver Review Session

Drafts of the intervention materials were presented to the caregivers in 1-hour review sessions. Overall, the caregivers were satisfied with the developed intervention materials. Caregivers had minor suggestions at this time, and their feedback was incorporated into the final products. For example, caregivers from Appalachia suggested adding a graphic in the video when the physician discusses HPV prevalence rates. Another example is when the caregivers in Columbus suggested that the brochure include a message to talk to your doctor or nurse about receiving the HPV vaccine if someone already had been diagnosed with HPV.

Health Care Providers

Step 1: Provider Focus Groups/Writing Sessions

There were five female health care providers (nurses) who participated in the 1- to 2-hour session in Appalachia. In rural Appalachian Ohio, nurses are the most common health care provider in county health departments. In Appalachia, the provider sessions were conducted immediately following work in the conference room at the health center.

In Columbus, Ohio, 27 health care providers (physicians, nurses, medical assistants) participated in the sessions. The urban providers were mostly females; however, three male providers participated in the sessions. The provider sessions were conducted during lunch periods and were arranged with assistance from the health center director who communicated to all providers about the sessions.

Health care providers in both rural and urban populations suggested a similar format for the intervention for providers. The suggestions for the provider intervention focused on having the ability to conduct the program in a group or individual setting because of the limited possibilities to include the entire staff in one session in an active health center. Providers thought that a PowerPoint presentation with current scientific articles about the HPV vaccine would be the type of activity that would be acceptable to most providers. Furthermore, providers suggested that a periodic newsletter providing updated HPV vaccine information would serve as a reminder and be a beneficial component of the program.

Providers recommended that the presentation include the following: data about cervical cancer rates, information about HPV prevalence and HPV vaccination rates, tips on how to address common HPV vaccine barriers, and ways to communicate the importance of vaccinating young adolescents. Providers in Columbus also discussed the limited health literacy among parents and HPV vaccine–eligible patients. Finally, providers made suggestions about additional intervention components, including a reminder system, posters, and brochures.

Step 2: Provider Review Sessions

Review sessions were conducted among providers in both Ohio locations and lasted 1 hour. In addition to reviewing the PowerPoint presentation and the handouts for the provider intervention, the four female providers in Appalachia also reviewed the caregiver and clinic components of the intervention. The rural providers reported that all intervention components were acceptable, and they thought the DVD for the caregivers represented a typical encounter providers have with caregivers and adolescents about the HPV vaccine.

Review sessions in Columbus included 15 providers (3 males) who reviewed the PowerPoint presentation and handouts for the provider intervention, the brochure for caregivers, and the clinic components of the intervention. The providers did not review the interactive computer program due to limited time, but the format of program was explained and examples were discussed. The components of the intervention were acceptable; however, many of the providers thought the PowerPoint presentation was not exciting to watch. In addition to the embedded video clips demonstrating the teach-back methodology, providers suggested adding more pictures and increasing the speed of the presentation. The suggested modifications to the presentation were made prior to pilot testing the intervention in a feasibility study.

Community Differences

Although both Appalachia and Columbus, Ohio, locations serve low-income populations, several important differences emerged using the CEnR process. Two or three of the nine caregivers in the initial focus groups conducted in Appalachia expressed concern that the HPV vaccine may be perceived by their female adolescents as a license to have sex, that some caregivers are not willing to discuss an STI with younger adolescents, and apprehension about including the sexually transmitted nature of HPV as a part of the program. These concerns were addressed by including specific content within the video’s narrative and that the intervention was directed toward the caregivers who could decide what HPV vaccine information to share with their children. In addition, near the end of the video, the featured parents gave a testimonial focused on the myth that the HPV vaccine may cause sexual promiscuity.

Conversely, caregivers from Columbus commented that sexual initiation was early among adolescents, and therefore, the fact that HPV is an STI should be included in the program. Additionally, the urban caregivers expressed a desire to have more comprehensive sexual disease education as a part of the program and to place an emphasis on safe sexual practices postvaccination informing adolescents that the HPV vaccine only protects against HPV and not other STIs or pregnancy.

Another difference was that Appalachian caregivers wanted HPV vaccine information presented to them and they would present relevant and appropriate information about HPV and the HPV vaccine to their adolescents. Caregivers from Appalachia did not believe that the intervention should include information targeted specifically toward adolescents. On the other hand, caregivers in Columbus expressed a desire for age-appropriate HPV and HPV vaccine information to be given to young adolescents as a way to initiate an HPV vaccine discussion. They suggested that HPV vaccine information could be presented in a comic book format for young adolescents to introduce basic facts about the importance of receiving the HPV vaccine to remain healthy.

One more difference between rural and urban caregivers was that caregivers in Appalachia suggested a short video would be the best format to engage caregivers, whereas caregivers from Columbus suggested an interactive computer program. Caregivers expressed that an interactive computer program would provide an engaging experience, and they believed the narrative could be accomplished by including short video clips within the program. The flexibility of this format also provided the ability for the research team to tailor the program based on the child’s gender, age, race, and ethnicity. This was important because caregivers thought there were substantive differences between the decision-making process for caregivers of female compared to male adolescents. Therefore, the interactive computer program provided the opportunity for caregivers to view a story line that was more consistent with their family’s composition and more relevant to the decision they had to make for their adolescent.

Although both caregiver groups believed that it is important to ensure consistency between the story line and typical lives of the priority population, the differences in the daily experiences of people living in Appalachia and Columbus emerged during the intervention development process. For example, Appalachian culture was described by caregivers and health care providers as having an emphasis on family, family life, and family relationships. Therefore, the video’s narrative followed a mother’s and father’s decision to vacci-nate their young daughter after their older daughter had been diagnosed and treated for cervical cancer. Alternatively, the urban caregivers suggested including two female friends in the computer program about the HPV vaccine decision for a female adolescent since many single mothers rely on input from their friends or neighbors. By ensuring that the narrative component of the intervention reflected the experience common to the priority populations, both HPV vaccine interventions will more fully engage caregivers in the HPV vaccine educational process, thereby initiating HPV vaccine discussions with their child’s health care providers, and potentially improving HPV vaccination rates.

Health care providers in rural and urban settings had many of the same experiences when recommending the HPV vaccine for adolescents. Providers reported that HPV vaccine–hesitant caregivers’ concerns have been focused on the newness and the safety of the vaccine. Providers in the urban setting reported that caregiver hesitancy about the HPV vaccine because of the young age of the child (e.g., not sexually active) was less of a concern than providers from a rural setting. Providers’ perception that the HPV vaccine is sensitive topic for caregivers because it prevents an STI is important and has been previously reported (Healy, Montesinos, & Middleman, 2014). This issue underscores the importance of developing interventions aimed at multiple levels to develop more effective ways to communicate about HPV-associated diseases and the HPV vaccine to improve HPV vaccination rates (U.S. Department of Health & Human Services, 2014).

DISCUSSION

The value of conducting CEnR among minority and underserved populations has been documented in the past (Gehlert & Coleman, 2010; Israel et al., 2005; Israel et al., 2010; Minkler, 2005) and is illustrated by the multilevel HPV vaccine interventions that were developed by partnering with two distinct population groups within Ohio. Members from these populations had the same perspective on the importance of keeping their children healthy, yet differed on the content and format of the intervention developed to achieve that goal. Both HPV vaccine interventions developed by this iterative process have the potential to contribute to improving adolescent vaccination rates within their respective communities. The similarities and the differences among the members from these underserved population groups illustrate the importance of involving community members from populations in the development of behavioral interventions and how contextual factors shape cancer prevention and control research. The influence of history and the social norms of a community are critical considerations as is the local relevance of the public health problem when developing interventions to reduce cancer disparities.

The CEnR process for the described intervention development began prior to the report of many interventions to increase the uptake of the HPV vaccine. Most reported that HPV vaccine interventions were based on behavioral theory and completed formative data collection (Brawner et al., 2013; Cassidy et al., 2014; Paiva, Lipschitz, Fernandez, Redding, & Prochaska, 2014). However, the intervention development generally did not describe input or mentioned limited participation (formative data) from members of the community (Brawner et al., 2013; Cassidy et al., 2014; Paiva et al., 2014; Vanderpool et al., 2013). Additionally, most of the interventions were aimed at one level, had small sample sizes, or relied on self-report of vaccination (Brawner et al., 2013; Cassidy et al., 2014; Kharbanda et al., 2009; Moss et al., 2012; Paiva et al., 2014;Vanderpool et al., 2013). Two studies describe HPV vaccine interventions aimed at multiple levels (Fiks et al., 2013; Reiter et al., 2011). One multilevel study did not describe community input (Fiks et al., 2013) and the other study had input from a community advisory board; however, it did not measure vaccination rates (Reiter et al., 2011). A review of HPV vaccine educational interventions has also documented similar methodological limitations in reported studies to date (Fu, Bonhomme, Cooper, Joseph, & Zimet, 2014). Although diverse population groups may suggest different components for an intervention targeting the same health behavior, the process described in this article is based on our experience working with community members to address different cancer disparities issues and is easily transferred to other communities for developing cancer prevention and control interventions. An example of this experience is that we had the caregivers help edit the script instead of trying to have the caregivers actually participate in the writing of the scripts included in the interventions.

Several important lessons were learned by the intervention development process and can be used in future intervention development. First, we think that having a group of about six community members emerge as a “team” to work on developing the interventions was key to the successful process. This size group provides an opportunity for different viewpoints to be expressed, yet is small enough so that consensus can be reached and progress made in a timely fashion. An important aspect of the team-building process was managing the groups to ensure that all members of the team had a voice during the intervention development process. Therefore, we think it is essential that the research team member has skills in facilitating group sessions, familiarity with the community, and experience working with members of the priority population. Second, work completed by the academic researchers between sessions is critical so that community members can provide useful feedback and alternative ideas. The iterative process with community members proved to be very valuable. Third, the cancer researchers cannot be wedded to any specific intervention component when partnering with community members. This was evident on several occasions during the process. For example, caregivers combined elements of two suggested story lines into one narrative for the video, edited the scripts that were drafted for the narratives making the characters more realistic and acceptable to the priority population, chose bright colors for the print materials, or insisted the brochure be kept small enough to put in a purse or pocket. Fourth, academic researchers have to balance what the community members suggest and what is known by a significant body of research or what might offend some members of the community. For example, some Columbus, Ohio, caregivers thought a picture of genital warts should be on the cover of the brochure to scare adolescents. In this case, the facilitator of the session made a convincing case that placing a picture of genital warts on the brochure cover might not be acceptable to some members of the community. A solution to this difference in opinion emerged during the group discussion; genital warts can be viewed by caregivers who chose to see the picture within the computer program. Fifth, we think it is critical to have community members review intervention materials prior to pilot testing. The urban providers suggested that the automated PowerPoint presentation include more pictures and move at a faster speed compared to the rural providers. Changes made at this stage of the intervention development process may be addressed in a timely fashion prior to final production.

The value of using CEnR has been documented in the past few decades, especially when conducting research among minority and underserved populations (Gehlert & Coleman, 2010; Israel et al., 2005; Israel et al., 2010; Minkler, 2005). The benefits of partnering with community members to conduct research or to develop interventions outweighs the many challenges and time investment of this process (Frazier, Massingale, Bowen, & Kohler, 2013; Israel et al., 2005, Minkler, 2005). The primary outcome of the described process was the development of two multilevel HPV vaccine interventions that considered contextual factors and built on these factors to develop interventions that are culturally relevant in content (information included, appearance of characters, etc.) and format (e.g., interactive computer program) and more likely to be acceptable to the priority populations.

There are several limitations in this study. Small groups of caregivers and providers were used in both settings, and they may not reflect the viewpoints of all caregivers and providers from their communities. However, many of the issues about the HPV vaccine raised by the caregivers and health care providers in the initial focus groups have been raised by caregivers and providers in previous focus group studies (Katz et al., 2009; Reiter, Oldach, Randle, & Katz, 2014). Furthermore, some caregivers and providers did not participate in all intervention development and review sessions and that could also bias the findings. Although each of the sessions was scheduled in advance, a few individuals did not show for the group meetings.

Additional limitations include that the intervention process was conducted and completed in Appalachia prior to the start of the process in Columbus, Ohio. Although the sessions in Appalachia occurred only 1 year before the process in the urban setting, this difference may have influenced participants’ input. This time difference was also the reason that the HPV vaccine intervention was focused only on female adolescents for the Appalachian population and on female and male adolescents for the Columbus, Ohio, population. This difference due to the changes in the HPV vaccine recommendations may have led to contrasts between populations.

In spite of the time difference, however, the providers from both populations had extensive experience working with caregivers from their communities, the caregivers had lived in their respective communities for a period of time, and all caregivers and health care providers were engaged in the intervention development process. Finally, we do not know if the developed interventions are effective in improving adolescent HPV vaccination rates yet. The HPV vaccine intervention in Appalachian Ohio is currently being tested in a county-level randomized trial, and the intervention in Columbus, Ohio, is being pilot tested for feasibility in one urban FQHC.

CONCLUSIONS

We believe that caregivers and health care providers provided valuable input into the development of HPV vaccine interventions that will assist in making the interventions relatable and more acceptable to the priority populations. Although we think that a multilevel intervention is important to improve HPV vaccination rates among adolescents, the development of a multilevel intervention compared to a single-level intervention is more complex, costs more, and involves more time and effort. We think the value of the community–academic partnership is critical and that the additional time and energy expended are important to successfully address cancer disparities among different populations.