Adolescent pregnancy prevention interventions increasingly use video, especially video delivered remotely over the Internet.1,2 This use of video brings substantial benefits but also costs, which are detailed below based on our experiences with a randomized controlled trial in multiple sites, including both rural and urban settings.
Our team conducted a multisite randomized controlled trial (clinical trials NCT02049710) to evaluate the interactive video intervention Seventeen Days relative to an interactive video control on safe driving, measuring behavior six months after intervention.3 An evaluation of an earlier version of the intervention found increased sexual abstinence and reduced condom failures.4 That evaluation was conducted in person with participants who made appointments for study visits where they watched video and reported their behavior using paper-and-pencil measures.
We aimed to capitalize on the Internet to deliver the intervention in more locations and implement more sensitive measures of behavior. To allow adolescents access at locations of their choosing, we provided them with a Web login to view the study site from anywhere. Follow-up procedures included monitoring of login data, and reminders for those who had not logged in recently that they could return any time to watch more content. For outcome measures, we adapted the event-specific timeline follow-back calendar5 for an online environment, with the intention of collecting data specific to sexual events and partners. Thus, we began our evaluation with hopes for rich behavioral data measuring the effects of a high-dosage intervention. Unfortunately, things did not work out that way.
BENEFITS AND LIMITS OF USING VIDEO
Video has the benefit of delivering high-quality engaging content with fidelity and flexibility, but it is one-way and asocial. Traditionally, recruiters and facilitators make a personal connection with the adolescents that can be key to maintaining enthusiasm over repeated interactions, and participants develop accountability to a person who might be disappointed if they neglect the program. After consent into our evaluation, most subsequent communication was by text messaging. Although this was our participants’ explicit preference, the abbreviated and relatively anonymous nature of text messages makes then easy to ignore, and they rarely enhance relationships with the research team. In the absence of a personal connection, remote delivery of content lacks the social forces that encourage people to be “good citizens” and do what is expected of them.
This dynamic is ubiquitous. Consider voting: decades of efforts to make voting easier by instituting early or absentee voting have partially backfired, reducing voter turnout, apparently because of reduced social pressure to vote, including missing out on the prospects of social interactions at the polls.6 These same processes are relevant for delivering behavioral interventions, particularly considering adolescents’ strong social drive. Interactions with classmates or facilitators provide their own incentives to participate. Without that social component—no matter how appealing and useful the content—participants’ motivation may be insufficient for the intervention’s appeal to sustain engagement.
TECHNOLOGY LIMITATIONS
In our study, variability in bandwidth created inequities across populations. The perceived ease of implementation by Internet led us to engage numerous research sites—rural and urban. But many clinic settings presented constraints, such as institutional restrictions to broadband wifi, so that Internet access was restricted at most settings to cellular hotspots. Rural areas in particular had such variable or limited cellular service that streaming even low-bandwidth video was impossible. This allowed many participants to complete baseline measures and be randomized to condition, but not watch the video.
Additionally, adolescents’ adoption of new technologies was not always in sync with projections. When we proposed this intervention, video on personal phones was uncommon. We anticipated providing a novel education alternative for use in clinics. As the intervention ramped up, recruiters were increasingly competing with potential participants’ own cell phones, providing social interactions that filled any downtime in clinic, and making the prospect of our educational video less attractive.
Evolution of technology created further barriers. When creating our Web site, Flash was the medium of choice for video, but other platforms emerged soon after. In particular, exploding sales of iPhones, which do not play Flash, meant that for many adolescents their only means of accessing the Internet did not support our video. Despite our efforts to encourage other Internet access—including clinic visits and free public and commercial wifi hotspots, the video was no longer convenient for them.
PARTICIPANT BURDEN
Our online evaluation measures were sometimes too burdensome for participants to complete easily, especially on phones. Our calendar’s interface was cumbersome and lengthy to complete, especially for high-risk participants with many sexual events to report. This likely contributed to participants’ neglect in providing comprehensive event-specific responses on key outcomes, particularly condom use. Additionally, early in the study, substantial data were lost because of programming bugs. These factors combined to produce noisy data that failed to produce much insight into behaviors.
These experiences increased participant burden to the degree that many did not receive the video intervention at all. Once participants had completed baseline measures, they often lacked time or motivation to continue the program. Project personnel encouraged participants to get through the measures on site, then to watch the video later on their own. Unfortunately, this left participants alone to begin the intervention, without any positive impression of the video, and a negative association of a cumbersome behavioral event calendar, leaving them unmotivated to return to the program. Many were randomized (automatically) to a condition, but never exposed to the intervention, resulting in extremely low levels of content delivery, diluting intervention exposure. Only 61% completed the core dosage, but a large proportion of those (55%) watched additional video content on their own, suggesting that participants found the program engaging once they started it.
These low dosage rates and noisy behavioral data left little power to detect any effects. Examining the primary outcome from the earlier evaluation, sexual abstinence, the recent evaluation did show means that were directionally consistent—with abstinence levels increasing among those assigned to Seventeen Days relative to controls—but statistical tests did not reveal these effects to be significant at either the 3-month (P = .501) or 6-month (P = .178) time points.
LESSONS FOR THE FUTURE
Video and Internet technologies provide opportunities to deliver content, but also increase the competition for audience attention. To use these tools wisely, we must incorporate key assets of traditional interventions, especially social connections. Creating and maintaining connectedness with adolescents and fostering responsibility and engagement may help overcome the anonymity that accompanies technology.
The quality of the content matters, but only if the audience has a chance to see it. The entertainment industry devotes funds for promotion at a level that is many orders of magnitude higher than what public health efforts can afford. In a randomized controlled trial it is not possible to promote one program over the other, and so we were unable to entice participants with previews. Among participants and other audiences, we have found that once people start watching Seventeen Days, they love it. However, without structured and committed times and places, it is challenging to motivate adolescents to turn off their own social media and instead participate in education or research.
ACKNOWLEDGMENTS
This publication was made possible by grant number 1TP1AH0000400 from the Office of Adolescent Health, US Department of Health and Human Services.
REFERENCES
- 1.Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health. 2009;30:273–292. doi: 10.1146/annurev.publhealth.031308.100235. [DOI] [PubMed] [Google Scholar]
- 2.Tuong W, Larsen ER, Armstrong AW. Videos to influence: a systematic review of effectiveness of video-based education in modifying health behaviors. J Behav Med. 2014;37(2):218–233. doi: 10.1007/s10865-012-9480-7. [DOI] [PubMed] [Google Scholar]
- 3.Eichner J, Salaway J, Smith-Jones J, McCall R. Evaluation of Seventeen Days in Ohio, Pennsylvania, and West Virginia. Findings from the replication of an evidence-based teen pregnancy prevention program. Pittsburgh, PA: University of Pittsburgh, Office of Child Development. Available at: http://www.hhs.gov/ash/oah/oah-initiatives/evaluation/grantee-led-evaluation/grantees-2010-2014.html. Accessed September 30, 2016.
- 4.Downs JS, Murray PJ, Bruine de Bruin W, Penrose J, Palmgren C, Fischhoff B. Interactive video behavioral intervention to reduce adolescent females’ STD risk: a randomized controlled trial. Soc Sci Med. 2004;59(8):1561–1572. doi: 10.1016/j.socscimed.2004.01.032. [DOI] [PubMed] [Google Scholar]
- 5.Sobell LC, Sobell MB. Measuring Alcohol Consumption. Totowa, NJ: Humana Press; 1992. Timeline follow-back; pp. 41–72. [Google Scholar]
- 6.Burden BC, Canon DT, Mayer KR, Moynihan DP. Election laws, mobilization, and turnout: The unanticipated consequences of election reform. Am J Pol Sci. 2014;58(1):95–109. [Google Scholar]