Taking an HPV vaccine research-tested intervention to scale in a clinical setting

Abstract

Research tested interventions are seldom ready for wide spread use. Successful intervention adaptation to clinical settings demands an iterative process with target audience feedback. We describe the adaptation process of implementing an NCI research tested HPV vaccine intervention, Women's Stories, to a community clinic context (Planned Parenthood). Five phases are described for the adaptation of content and the development of a health kiosk intervention delivery system: (a) informant interviews with the target audience of young adult, predominantly African-American women, (b) translating HPV vaccine decision narratives into prevention messages, (c) health kiosk interface design, (d) conducting a usability study of the health kiosk intervention product, and (e) conducting a waiting room observational study. Lessons learned and challenges in adapting prevention interventions to clinical settings are discussed.

Taking evidence-based prevention programs to scale involves fast-tracking a series of steps that serve critically important goals for advancing the translation of science into clinical practice settings.

Implications

Practice: Successfully taking prevention interventions to scale in clinical settings involves (a) integrating the preventive health intervention into the existing clinic system, and, (b) devoting considerable thought and time to the intervention delivery platform system and designing the platform to reflect people’s preferences for receiving health information.

Policy: Our research has some clear policy implications that more patient education about covered prevention services should occur at the health plan level, and health plans should be required to provide this education in a clear and culturally sensitive fashion.

Research: Future research should consider the importance of including observational studies when adapting prevention interventions to clinic settings to maximize the potential for true intervention adoption by the intended audience.

INTRODUCTION

Research-tested interventions are seldom ready for widespread use [1]. Successful intervention design demands a user-centered and iterative approach [2] that often requires additional development to ready programs to be taken to scale. Here, we describe the process of adapting a National Cancer Institute (NCI) research-tested intervention program (RTIP) for HPV vaccination, “Women’s Stories,” [3] for implementation in Planned Parenthood (PP) community clinics. PP was chosen because their clinic model offers a scalable infrastructure that is available in every state and they serve underserved populations who are most in need of access to preventive services. In contrast to the “build it and they will come model” commonly used in public health [4], we argue for starting with the end user and integrating the intervention into their existing system. In this manner, if evidence supports taking the intervention to scale, dissemination is built into the original design. This approach stems from a combination of product development and community-based participatory research practices [5, 6]. We first offer a description of the original NCI-designated RTIP, provide a description of the adaptation phases, and conclude with a summary of challenges encountered and lessons learned.

Women’s stories: The HPV project intervention

The original intervention encouraged human papillomavirus (HPV) vaccination among young adult college women ages 18–26 using video-based vaccine decision stories or narratives. The intervention was developed based on Narrative Engagement Theory (NET) [7], which emphasizes eliciting health content from and with the target audience. A video was developed portraying five prototypical vaccine decision stories. The stories include a susceptibility story, a self-efficacy re-enactment, a story that speaks to the safety of the HPV vaccine, a cue-to-act dorm room discussion re-enactment, and a physician narrative disclosing a personal mother–daughter conversation. The intervention was tested in a randomized controlled trial at one university where it nearly doubled vaccination and was subsequently designated by NCI as an RTIP [3, 8].

After the successful initial college study resulting in the RTIP, the developers sought to adapt the intervention to reach underserved women who had lower rates of HPV vaccination and higher rates of late-stage cancer [9–11]. This led to a partnership with PP that started with Planned Parenthood of Southeastern Pennsylvania (PPSP), which serves a diverse clientele through 12 clinics [12]. Initial collaboration began with one PPSP clinic that serves predominantly African-American women many of whom are Muslim. Moreover, the PPSP community clinic serves young adult women who may or may not be attending college. Thus, initial efforts were focused on adapting the intervention content to resonate with these young adult women. Informant interviews were conducted with the intended target audience to identify relevant vaccine decision narratives that would be perceived as authentic with local community members who attend the PPSP clinic.

Adapting the intervention to a community clinic context

The adaptation process consisted of five phases. The first two phases focussed on adapting intervention content, whereas the latter three phases involved adapting the intervention to the clinic setting.

PHASE I: INFORMANT INTERVIEWS

Procedure and data analysis

Informant interviews (N = 26) were conducted in person with PPSP clients ages 18–26 at PPSP onsite counseling offices to elicit vaccine decision narratives and identify motivations that led to vaccinating. Both vaccinated and unvaccinated women were recruited from PPSP clinic waiting rooms asking women if they would participate in an interview prior to their clinic visit to talk about their vaccine decision. Women received $20 as compensation for their time. Only women were recruited given that PPSP predominantly serves women and not men. An experienced African-American interviewer recruited women on-site at the clinic and conducted the interviews aimed at eliciting decision stories to understand motivations and barriers to HPV vaccination among young adult women. Women were also asked for feedback about how best to design a health kiosk that would be placed in the waiting room to educate women about HPV vaccination (pictures were shown of a health kiosk to prompt discussion). Interviews were audio-recorded and transcribed verbatim with identifying names removed and pseudonyms replacing names. Data were analyzed initially using cultural grounding methods [13, 14] identifying emergent codes derived directly from the data. Subsequently, the research team coded the data at a second, more abstract level for emergent prototypical decision stories using prior established methods [15]. The research team additionally used a constant comparison method [16] to discuss and compare prototypical decision stories. Details of the Phase I interviews are provided in a separate manuscript [17].

Findings

Eight vaccine decision stories were identified as prototypical among African-American young adult women attending PPSP: (a) HPV (un)awareness, (b) wanting to stay healthy, (c) practitioners not mentioning HPV, (d) keeping female reproductive parts healthy, (e) including men in vaccine messages, (f) experiencing an abnormal Pap smear, and (g) cancer stories.

An example decision story among a vaccinated woman illustrated the importance of cues women take from their clinicians: “I’ve been talking to my primary care provider since I’ve become sexually active. She told me different ways to prevent…just different things from happening. She gave me information about the HPV vaccine. She said it was good to get. You never know who has it. It’s good to be aware. She told me about it preventing cancer. She told me a little bit not a lot.” Most unvaccinated women had not received HPV vaccine recommendations from their clinicians.

Interviews also elicited additional feedback about familiarity with and use of health kiosks in waiting rooms and receptivity to receiving vaccine reminder messages on smartphones. Women were more familiar with food kiosks used in gas stations or coffee shops and less familiar with health kiosks but expressed a willingness and interest in using health kiosks in the waiting room.

Clinic staff interviews

Two staff were interviewed to share their understanding of how the HPV burden is perceived among their patients and how HPV vaccine communication is handled at their clinic. They noted one of the biggest problems is a lack of education among PP clientele. Patients are unaware of HPV and lack an understanding of the severity of HPV if it advances. Interest in the vaccine commonly occurs after being exposed to HPV. Staff acknowledged that the biggest problem was the lack of opportunity to educate patients prior to exposure to infection, including men who come in for routine screening. Obtaining insurance coverage was noted as another major obstacle. The vaccine is too expensive for patients to pay out of pocket and obtaining insurance coverage, which often must be sought during or immediately after a clinic visit (on the spot), can be insurmountable. For example, sometimes patients are seen during clinic times when insurance offices are closed (Saturdays, or weekday evening hours). Other times, patients are simply unwilling to wait for the time it takes to receive insurance approval. Furthermore, while the Affordable Care Act (ACA) covers HPV vaccination for women aged 18–26 and men aged 18–21, insurance stipulates that vaccination is covered only when delivered by the primary care provider in many cases. PPSP is often not the primary care provider and consequently, patients experience insurance barriers. This in turn contributes to PPSP practitioners not routinely mentioning or recommending HPV vaccination.

PHASE II: TRANSLATING DECISION NARRATIVES INTO PREVENTION MESSAGES

Developing the intervention videos

Prototypical vaccine decision narratives were translated into four scripts creating composite narratives using previously established procedures [18]. The research team discussed the identified prototypical scripts (from Phase I data analysis) that would lend themselves to the intervention with consideration of practical elements; for example, that stories needed to be told and delivered within short 1 min video stories delivered on a health kiosk in a waiting room, the use of multiple stories from which women could choose, and the inclusion of men in at least one story. Also, a balance was needed between ensuring engaging stories that were authentic and kept women’s attention while keeping fidelity with medical accuracy. Given the low awareness of HPV, it was collaboratively decided that one of the four decision scripts needed to be tailored to an audience with little or no HPV knowledge. Thus, one decision story was developed for “beginners” who had either never heard of HPV or knew little about it. The decision story was delivered as a monologue in which a young woman retells a story about her cousin who was unaware of HPV and the vaccine. The monologue retells how the cousin initially declined vaccination because she did not know what HPV was, how the cousin learned about cancer risks associated with HPV from her sister, and that she reconsidered and eventually vaccinated against HPV [Fig. 1].

Fig. 1.

Screenshot of woman who delivers monologue & screen description

The remaining three scripts convey the following: (a) learning about the real risk and potential consequences of HPV through a conversation between two female friends (kitchen conversation), (b) learning about HPV and cancer risk for men through a conversation between a male and female friend (park bench conversation), and (c) learning about doctors’ strong support for vaccination through a physician strongly endorsing HPV vaccination to a young women during a well visit when she shares she is considering becoming sexually active. Each narrative ends with reinforcement messages that exemplify the themes: Be protected, Ask your doctor about the HPV shot, Talk to your friends about getting vaccinated for HPV [Fig. 2].

Fig. 2.

Intervention graphic of 4 HPV vaccine decision stories

Piloting scripts

The four scripts were pilot tested with 12 female PPSP clients aged 18–26. Women were recruited from the waiting room, screened for eligibility, read scripts, completed a survey that assessed their level of engagement, answered interviewer questions about what they did and did not like, and provided suggestions for improvement. Women completed a nine-item measure of engagement with items drawn from the Narrative Engagement Scale [19] and from the Audience Engagement Scale [20]. Responses used a five-point Likert scale ranging from strongly disagree to strongly agree. The survey captured the extent to which women found the scripts interesting and believable and to what extent scripts led to personal reflection and critical thinking about vaccination. The interviewer also asked women to indicate what they did and didn’t like about each script with suggestions for improvement.

Findings

Findings are summarized in Table 1, and based on them, further edits were made to scripts to reflect findings from the target audience.

Table 1.

Summary of quantitative and qualitative script feedback (N = 12)^a

Means and standard deviations for script ratings
	Video 1, What is HPV?	Video 2, Effects of HPV	Video 3, Talk w/your doctor	Video 4, Men can get HPV too
I enjoyed reading this script.	4.8 (0.5)	4.2 (0.7)	4.6 (0.7)	4.7 (0.5)
The script was boring.	1.5 (0.7)	2.2 (1.3)	1.6 (0.9)	1.6 (0.7)
The script was realistic.	4.7 (0.5)	4.4 (0.7)	4.8 (0.5)	4.5 (0.9)
The script was NOT believable.	1.4 (0.5)	2.0 (1.1)	1.6 (0.7)	1.3 (0.5)
I think this script would be effective as a short video on a kiosk for other women like me.	4.1 (1.4)	3.7 (1.3)	4.3 (1.2)	4.3 (1.1)
This script made me think about the importance of getting the HPV vaccine.	4.6 (0.8)	4.7 (0.5)	4.6 (0.7)	4.6 (0.5)
This script made me think about the importance of talking to a doctor about the HPV vaccine.	4.5 (0.7)	4.7 (0.5)	4.6 (0.7)	4.6 (0.5)
This script made me think about the importance of talking to other people about the HPV vaccine.	4.4 (0.8)	4.5 (0.5)	4.4 (0.7)	4.6 (0.5)
This script made me think about what might happen if I did not get the HPV vaccine.	4.6 (0.7)	4.7 (0.5)	4.4 (0.8)	4.4 (0.7)
Qualitative Findings from Surveys.
Seemed like a real conversation.
It’s something that would actually happen.
It grabs your attention.
Information is practical and helpful.
Messages emphasize protecting the boyfriend, messages include men.
Stories provide model for how to talk with men about HPV vaccination.
Scripts are honest, straightforward.
I learned something.
Reinforcement messages showed the importance of talking with your doctor.
The scripts tell you what happens when you get HPV and the potential seriousness.
Messages were not preachy. They were clear and easy to understand.
Messages promote and encourage asking questions.

^aQuantitative ratings were on a five-pt scale from strongly disagree (1) to strongly agree (5).

PHASE III: HEALTH KIOSK INTERFACE DESIGN

Procedure

The next step was to design an interactive, engaging interface. The research and technology development teams worked collaboratively to create a prototype. Input not only from the research team but from Phase I informant interviews was integrated into the interface design. Women in Phase I had been shown a picture of a prototype health kiosk and were asked to comment how the kiosk could be designed to engage women in the waiting room.

Findings and product revision

Women wanted the kiosk to be visually informative with clear signage about what the kiosk was about. Women also said that bright colors should be used to attract attention and that the kiosk screen and menu should be interactive. Finally, given that many women were less knowledgeable about female reproductive anatomy, they wanted a female reproductive graphic to explain where the cervix is and how HPV can affect the cervix. The research team consequently designed the kiosk interface to include an interactive female anatomy graphic that allowed users to learn more about how HPV affects the cervix by interfacing with the graphic and having information cartoon bubbles shoot out from the graphic [Fig. 3].

Fig. 3.

Kiosk interface of female anatomy with talk bubble

Touchscreen fact bubbles (like cartoon bubbles) were generated from the anatomy when touching the kiosk screen. A user-friendly interface allowed women to navigate freely between the videos and the health information based on user-centered design research showing that user satisfaction is higher with this kind of design [21]. User interfaces (kiosk screens) that allow users to navigate freely (as opposed to being directed to information) through the menu and self-select what information and the order in which users choose to view and read information results in higher user satisfaction [22].

A key challenge was signage to bring users to the kiosk. A balance was needed to avoid the potential stigma of using an STD-related kiosk while at the same time clearly conveying to users the kiosk’s purpose. A group decision involving the entire study team was made to have the signage focus on women’s health (signage was placed above the kiosk screen) avoiding potential stigma to use the kiosk (due to possible embarrassment if labeled as HPV) and to avoid causing any potential psychological damage [Fig. 4].

Fig. 4.

Women’s stories health kiosk in planned parenthood clinic waiting room

PHASE IV: USABILITY STUDY

Procedure

A usability study was conducted (N = 16) at the clinic by an independent researcher. Forty-one women were approached with 16 women meeting criteria and volunteering. In a private office, they used the kiosk navigating its interface and the female anatomy graphic followed by watching all four videos. Feedback was obtained about content, ease of use, and audio. Participants also completed a survey measuring usability metrics (see Table 2).

Table 2.

Usability study testing health kiosk interface (N = 16)^a

Means and standard deviations for overall kiosk usability ratings
I thought the kiosk was easy to use.	4.8 (0.4)
Navigating the kiosk screens to get to information I wanted was easy.	4.7 (0.6)
I would imagine that most people would learn to use this kiosk very quickly.	4.8 (0.5)
I felt very confident using the kiosk.	4.5 (0.6)
I would need help using this kiosk.	1.6 (0.8)
I felt comfortable using the kiosk.	4.8 (0.6)
It was easy to find the information I needed.	4.8 (0.8)
The organization of information on the kiosk screens is clear.	4.8 (0.4)
I like the interactive nature of this kiosk.	4.7 (0.5)
This kiosk has all the functions and capabilities I expected it to have.	4.6 (0.6)
Overall, I am satisfied with this kiosk.	4.8 (0.4)
Navigating through this health kiosk makes me feel that I can easily make an appointment to vaccinate.	4.7 (0.6)
I know how I can schedule an appointment to get vaccinated for HPV as a result of the information on this kiosk.	4.6 (0.7)
Qualitative feedback on health kiosk.
Kiosk is useful and instructions are clear.
All four videos well liked, with the video with the man and woman talking (park bench conversation) the most popular, followed by the kitchen conversation among friends.
Female anatomy graphic: “good to see where HPV is.”
Interactive graphic should highlight parts of the body when information comes up.
Clinic staff suggested using a tinted privacy screen.
Two women viewed the kiosk screen at an angle that made viewing difficult.
This type of health kiosk would be appropriate for a PP women’s health care clinic. We are all women here for the same reason.
Audio is fine; indicated they would not feel uncomfortable listening to it in the waiting room; three participants indicated they would prefer headphones.
No one used a QR code to receive vaccine reminder texts on their phones.

PP planned parenthood.

^aQuantitative ratings were on a five-pt scale from strongly disagree (1) to strongly agree (5).

Findings

The kiosk was easily implemented, experienced no technical problems, and was evaluated positively by women using it. It was anticipated that women would use the kiosk in the clinic waiting room and benefit from the experience.

PHASE V: WAITING ROOM OBSERVATIONAL STUDY

Procedure

A waiting room observational study was conducted over 2 days to evaluate actual kiosk use. The kiosk was placed in the waiting room, and a research team member observed kiosk use and general behavior over 2 days.

Findings

Several problems were observed during this phase. First, the kiosk screen went into “sleep mode” and patients thought the kiosk was off. Second, routine, normative waiting room procedures included checking in at the front desk and then sitting down. Most patients sat quietly and used their cell phones or watched the TV while waiting. One woman briefly glanced at the kiosk, but most women did not notice it. This was the case during both low and high volume times. Thus, no women utilized the kiosk on the first day.

To better understand this situation, eight women were approached and asked why they did not use the kiosk and what would motivate them to use it. Responses included “I didn’t notice it,” “I didn’t know what it was about,” and “It didn’t look like it was on” but said they would use it if staff prompted them. On day two of observation, the receptionist prompted women to use the kiosk during check-in. Even with prompting, the kiosk was not used. While this was surprising given the prior day’s feedback, observations and interviews with staff revealed several considerations. Some women were already vaccinated, giving them little motivation to walk up to the kiosk. Also, the prompt by the receptionist came at the end of lengthy discussions about forms patients needed to complete. Finally, a PPSP research assistant spent an additional third day more directly prompting clients to use the kiosk. She explained that PPSP was testing the kiosk and looking for feedback. This approach was successful in getting clientele to use the kiosk.

LESSONS LEARNED AND CHALLENGES IN ADAPTING PREVENTION INTERVENTIONS

Taking evidence-based intervention (prevention) programs to scale involves fast-tracking a series of steps that serve critically important goals for not only advancing science but for advancing the translation of science into clinical practice settings.

Lessons learned from adapting Women’s Stories: The HPV project

Whereas content adaption was a smooth process given our well-established methods [23], the delivery platform and clinic setting posed several challenges. The most notable challenge was overcoming clients’ normative waiting room behavior despite their positive feedback and assurances they would use the kiosk. This lesson underscores the importance of including observational studies when adapting an intervention to a new setting, as what people say they are interested in and willing to do does not necessarily translate into actual behavior in real-world settings.

Questions about appropriate platforms for delivering prevention interventions in the clinic setting is a reoccurring theme [24]. The pace at which digital technologies and intervention platforms change challenges effective and practical intervention development [25]. For successfully adapting a prevention intervention to a new setting, evaluating all implementation phases ensures the greatest likelihood of successful adoption. It also allows program developers to explain which aspects of adaptation prove to be obstacles, facilitators, or neither. Because of this study, our plans include integrating the intervention into established clinical practices; in particular, integrating the intervention into check-in and exam room procedures to insure its use. We are also considering transforming the intervention into an independent e-learning module and an app that can be accessed on smartphones and tablets at the discretion of the individual.

Translational research is the study of how research findings are translated into programs, policy, and practice [26]. The “Women’s Stories” intervention is an excellent example of developing an evidence-based program to be used practically in the health care context. Within implementation science, the nature and quality of how an intervention gets implemented is of great importance, and two primary issues in implementation quality are fidelity and adaptation. Manualized interventions such as educational programs implemented by a health educator often suffer because the implementer does not teach the program as intended (i.e., lack of program fidelity), by leaving material out (“no video today!”), embellishing on material (insert fear message), or making adaptations to fit an intended audience [27]. The use of technology to deliver the “Women’s Stories” intervention overcomes these issues by delivering the program consistently each time without unnecessary revisions. Moreover, the need for adaptation to end users is not necessary because the end users were included in the design of the intervention.

Nevertheless, as the results of this study make evident, key questions regarding the sustainable uptake and adoption of this kind of intervention still remain. What are the most efficient means of getting the most patients exposed to the intervention in the clinical setting, while simultaneously considering the constraints of clinic resources such as time and money? If the target audience does not access the technology or use it correctly, fidelity is moot. A next step for “Women’s Stories” involves integrating the intervention into clinic check-in procedures thus, ensuring exposure, but what impact will this have on patient wait times? These and other questions are essential to answer as we continue in our translational efforts.

Conflicts of Interest: S.H. is affiliated with REAL Prevention, LLC and will profit from marketing the intervention. A.E.R. has no financial disclosures. M.H. is owner of REAL Prevention, LLC that will profit from marketing the intervention. M.M. is owner of REAL Prevention, LLC that will profit from marketing the intervention. R.B. has no financial disclosures. G.Z. has been an investigator on an investigator-initiated HPV research funded by Merck and Roche, received travel support from Merck to present research findings at a scientific meeting, and an honorarium from Sanofi Pasteur for participation in an adolescent immunization working group. W.D.E. has no financial disclosures. F.X.M., owner of St. Andrews Development has no financial disclosure but does have an arrangement with REAL Prevention to profit from marketing the HPV intervention described in this manuscript.

Compliance with Ethical Standards

Primary Data: This manuscript has not been previously published, either in whole or in part, nor have the findings been posted online.

Authors’ Contributions: S.H. contributed to the conception and design of the study, data collection, analysis, interpretation, and drafted the manuscript. A.E.R. contributed to planning the observational study, data collection, analysis, and interpretation and revised the article for important intellectual content. M.H. contributed to the conception and design of the study, data collection, and revised the article for important intellectual content. M.M. contributed to the conception and design of the study, and revised the article for important intellectual content. R.B. contributed to the design of the study and reviewed and revised the article for important intellectual content. G.Z. contributed to the design of the study and reviewed and revised the article for important intellectual content. All authors read and approved the final version of the manuscript. D.E. contributed to data collection of the usability study. F.M. contributed to the research team discussion, design and production of the health kiosk.

Ethical Approval: The study has been approved by the Tanglewood Research Human Subjects Protection Committee to be in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. No animal research was involved in this study. The welfare of animals used for research must be respected.

Informed Consent: Informed consent was obtained from all individual participants included in the study.