Outsmart HPV: Acceptability and short-term effects of a web-based HPV vaccination intervention for young adult gay and bisexual men

Abstract

Background

Effective interventions to promote human papillomavirus (HPV) vaccination are needed, particularly among populations at increased risk of HPV-related disease. We developed and pilot tested a web-based intervention, Outsmart HPV, to promote HPV vaccination among young gay and bisexual men (YGBM).

Methods

In 2016, we recruited a national sample (n=150) of YGBM ages 18-25 in the United States who had not received any doses of HPV vaccine. Participants were randomized to receive either standard HPV vaccination information (control) or population-targeted, individually- tailored content (Outsmart HPV intervention). We assessed between group differences in HPV vaccination attitudes and beliefs immediately following the intervention using multiple linear regression.

Results

There were no differences in HPV vaccination attitudes, beliefs and intentions between groups at baseline. Compared to participants in the control group, intervention participants reported: greater perception that men who have sex with men are at higher risk for anal cancer relative to other men (b=0.30); greater HPV vaccination self-efficacy (b=0.18); and fewer perceived harms of HPV vaccine (b=−0.23) on posttest surveys (all p<.05). Overall, intervention participants reported high levels of acceptability and satisfaction with the Outsmart HPV intervention (all >4.4 on a 5-point scale).

Conclusions

Findings from this study provide preliminary support for a brief, tailored web- based intervention in improving HPV vaccination attitudes and beliefs among YGBM. An important next step is to determine the effects of Outsmart HPV on HPV vaccine uptake.

Clinical trials registration

ClinicalTrials.gov identifier NCT02835755

Introduction

Oncogenic human papillomavirus (HPV) types (mainly types 16 and 18) can cause cancers of the anus, oropharynx, and penis among males, and nononcogenic HPV types (mainly types 6 and 11) cause genital warts.[1, 2] Compared to their heterosexual counterparts, gay and bisexual, and other men who have sex with men (MSM) have higher rates of HPV infection and HPV-related disease, including genital warts and anal cancer.[3–5] For example, over 50% of HIV-negative MSM are estimated to have a current HPV infection,[5] and anal cancer incidence is significantly higher among this population with an estimated 14-69 cases per 100,000 population compared to only 2 cases per 100,000 among heterosexual men.[4, 6]

HPV vaccination is approved by the Food and Drug Administration to prevent HPV-related disease, including anal cancer and genital warts. The Advisory Committee on Immunization Practices (ACIP) currently recommends routine vaccination for males ages 11-12 and catch-up vaccination for ages 13-21.[7] Importantly, routine HPV vaccination is recommended for MSM, including those who identify as gay or bisexual or who intend to have sex with men, through age 26. The vaccine series currently consists of 3 doses administered over a 6 month period for those who initiate the vaccine series after turning age 15, with 2 doses for those who initiate the vaccine series before age 15.[7] Despite recommendations, many age-eligible male adolescents are not receiving HPV vaccine, with just 50% of male adolescents in the U.S. having received any doses,[8] making efforts to increase vaccination among young adults important.

Given the known HPV-related disparities, it is important to examine the factors that may influence HPV vaccination among age-eligible young gay and bisexual men (YGBM). Previous research finds that YGBM have modest knowledge about HPV and HPV vaccine, though their willingness to get HPV vaccine is high.[9–12] However, only about 21% of YGBM ages 18-26 in the United States have received any doses of HPV vaccine and fewer than 10% have received all 3 doses.[12–15] YGBM have reported numerous barriers to HPV vaccination including: low perceived risk of HPV; concerns about vaccine side effects; concerns about sexual orientation disclosure to a provider; and lack of a provider recommendation to get vaccinated.[9, 12, 14, 16–18]

The President’s Cancer Panel Annual Report declared HPV vaccination a public health priority and identified the development of effective interventions—including Internet-based and mobile health interventions—to communicate about HPV and HPV vaccine as 1 of 5 high-priority research areas.[19] Technology-based interventions have effectively reached and affected the health behaviors (e.g., safer sex) of YGBM in past research.[20–22] However, to our knowledge, no HPV vaccination interventions for this population have yet been tested. We developed and pilot-tested Outsmart HPV, a brief, web-based intervention for YGBM. In this article, we report on the intervention’s acceptability to YGBM and its short-term effects on HPV vaccination knowledge, attitudes, and beliefs.

Methods

Participants and procedures

We recruited participants via paid Facebook advertisements (“ads”) in Facebook users’ News Feeds during July and September 2016.[23] All Facebook ads linked potential participants to the Outsmart HPV website or the project’s Facebook page (which in turn directed potential participants to the website). Once on the Outsmart HPV website, potential participants first completed a brief eligibility screener. Men between the ages of 18–25 years who resided in the U.S., self- identified as either gay or bisexual, and had not received any doses of HPV vaccine were eligible to participate. After completing the screener, eligible participants provided informed consent, created a website username/password, and completed the baseline (pretest) survey. Participants were then randomized into either the intervention or control group (described below) using a 1:1 allocation ratio until the target sample size for this pilot study (n=150) was reached. Immediately after viewing either intervention or control materials, participants were asked to complete a follow-up (posttest) survey. Participants received a $40 gift card for completing pretest and posttest assessments. The present analysis includes data from 141 participants (68 intervention, 73 control) who completed both surveys (a 94% retention rate). The Institutional Review Board at the Ohio State University approved all study procedures.

Study materials

Materials for both conditions were built in collaboration with the Center for Health Communications Research (CHCR) at the University of Michigan and delivered via a mobile-friendly website accessible by desktop/laptop, tablet computer, or smartphone. The thematic design of the project website, including the logo, color scheme, and intervention content (e.g., imagery, infographics, messages) were designed and refined with input from a group of YGBM (n=10) who initially evaluated the website and intervention content for acceptability and relevance. We made revisions based on this input and then had 3 YGBM examine the revisions before finalizing the website and other study materials.

Intervention Condition

The protection-motivation theory (PMT) formed the theoretical underpinning of our intervention.[24] Participants in the intervention group received population-targeted (i.e., information that applies specifically to YGBM), individually-tailored content about HPV and HPV vaccination. Tailored content was based on participants’ baseline assessment. The intervention, Outsmart HPV, included four, sequential sections (Figure 1). Each section included infographics and other visual formats. The four sections were:

1. “Learn about HPV” provided population-targeted information about HPV prevalence, transmission, and HPV-related diseases among YGBM.

2. “Learn about the Vaccine” provided information about HPV vaccine recommendations for YGBM and vaccine effectiveness, and individually-tailored testimonials (based on men’s relationship status and whether they had a regular healthcare provider) illustrating men’s reasons for getting vaccinated.

3. “Get Answers” used a question and answer format to provide information about barriers and concerns about HPV and HPV vaccination frequently reported by YGBM in past research.[9–11, 14, 18, 25] All questions and answers were available to all intervention participants. However, the website tailored the ordering of content for this section by prioritizing the presentation of information based on participants’ baseline surveys. For example, the potential barriers and concerns about HPV vaccine endorsed at baseline (e.g., reasons for not vaccinating) appeared at the top of this section, highlighting this relevant content.

4. The final section, “Get Vaccinated,” provided content about the logistics of getting the vaccine. This included resources for accessing HPV vaccine such as a link to an LGBT-friendly provider directory,[26] information about cost and health insurance, and skills-building strategies for talking with a provider about getting vaccinated.[27] Participants were then prompted to create a customized “Vaccination Plan” which included: self-identified HPV vaccination motivations; a goal date for getting their first shot; and tailored next steps for getting vaccinated.

Figure 1.

Screenshot of the Outsmart HPV intervention homepage showing the 4 sequential sections

Following completion of the posttest survey, intervention participants could choose to print or save a PDF document of their vaccination plan and summary information about HPV and HPV vaccine, and were able to log back into the website to view intervention content again.

Control Condition

Participants in the control group received standard information about HPV and HPV vaccine that was closely modeled after the Vaccine Information Statement (VIS) for HPV vaccine [28] (created by the Centers for Disease Control and Prevention) that was in use during our study. We formatted the VIS content to match the color and font scheme of the project website. We modeled the control group content after the VIS because healthcare providers are required to give a copy of the VIS to patients before vaccination and it provides easy-to-understand information about HPV vaccine that is publicly available.[28] Similar to the intervention group, following completion of the posttest survey, control group participants could choose to print or save a PDF version of this content following completion of the posttest survey, and had the ability to log back into the project website to view this content again.

Measures

HPV vaccination constructs

Pre- and posttest surveys included identical items drawn from existing literature to assess key PMT constructs.

Knowledge about HPV was based on the number of correct responses to 5 true/false statements, including knowledge that HPV vaccine is recommended for all men who have sex with men ages 18-26. We assessed participants’ perceived severity of HPV-related disease, perceived likelihood of getting HPV-related disease, and perceptions of anal cancer risk among MSM relative to other men. We used items from a previously validated scale [29] to assess participants’ perceptions of HPV vaccine effectiveness, potential HPV vaccine harms, and barriers to getting HPV vaccine. Survey items also examined perceived positive social/community norms (i.e., if other YGBM are getting HPV vaccine), self-efficacy to talk with a provider about HPV vaccine and to get vaccinated, and intentions to get HPV vaccine in the next 6 months. All belief and attitude items used Likert-type response scales coded so that higher values indicate greater levels of that construct. Participants also indicated reasons why they had not received any doses of HPV vaccine.

Acceptability

Eight agree-disagree items on the posttest survey assessed acceptability of study materials across both conditions.[22] Questions asked about participants’ perceptions about the: quality of information, quality of the system, and usefulness. All items were coded so that higher values indicate greater acceptability.

Demographics and health-related characteristics

The baseline survey collected information on a range of demographic and health-related characteristics (Table 1). We assessed participants’ disclosure of sexual orientation to their healthcare provider and perceptions of discrimination from a healthcare provider due to their sexual orientation, since these constructs may be related to healthcare access and quality.[30] We assessed participants’ electronic health (e-health) literacy [31, 32] because the ability to find and use health information on the Internet may affect participants’ understanding of study materials.

Table 1.

Participant characteristics, by study group

	Intervention (n=68)	Control (n=73)	p
	n (%)	n (%)
Demographics
Sexual identity			0.97
Bisexual	11 (16.2)	12 (16.4)
Gay	57 (83.8)	61 (83.6)
Age (years)			0.61
18–21	26 (38.2)	31 (42.5)
22–25	42 (61.8)	42 (57.5)
Race/ethnicity			0.93
White, non-Hispanic	41 (60.3)	41 (56.2)
African American or Black, non-Hispanic	8 (11.8)	11 (15.1)
Other race, non-Hispanic	4 (5.9)	5 (6.9)
Hispanic	15 (22.1)	16 (21.9)
Relationship status			0.29
Othera	52 (76.5)	61 (83.6)
In partnership, married, or civil union	16 (23.5)	12 (16.4)
Education level			0.59
Some college or less	44 (64.7)	44 (60.3)
College degree or higher	24 (35.3)	29 (39.7)
Annual income			0.01
Less than $50,000	45 (66.2)	63 (86.3)
$50,000 or more	23 (33.8)	10 (13.7)
Healthcare
Health insurance			0.38
None	11 (16.2)	15 (20.6)
On parents’ insurance	34 (50.0)	28 (38.4)
Insures self	23 (33.8)	30 (41.1)
Has a regular healthcare provider			0.82
No	33 (48.5)	34 (46.6)
Yes	35 (51.5)	39 (53.4)
Had a routine medical check-up in last year			0.95
No	36 (52.9)	39 (53.4)
Yes	32 (47.1)	34 (46.6)
Ever received an HPV vaccination recommendation from a healthcare provider			0.90
No	67 (91.8)	62 (91.2)
Yes	6 (8.2)	6 (8.8)
Disclosed sexual orientation to a healthcare providerb			0.57
No	12 (34.3)	11 (28.2)
Yes	23 (65.7)	28 (71.8)
Ever perceived discrimination from healthcare provider			0.38
No	56 (82.4)	64 (87.7)
Yes	12 (17.7)	9 (12.3)
Health literacy
Electronic health literacyc, mean (SD)	4.04 (0.77)	3.89 (0.92)	0.25
Sexual health
Age at sexual debut			0.81
Younger than 18 years	34 (50.0)	38 (52.1)
18 years or older	34 (50.0)	35 (48.0)
Lifetime number of sexual partners			0.80
11 or less	35 (51.5)	36 (49.3)
12 or more	33 (48.5)	37 (50.7)
HIV status			0.21
Negative	63 (92.7)	71 (97.3)
Positive	5 (7.4)	2 (2.7)
Ever had an STD			0.971
No	56 (82.4)	58 (79.5)
Yes	12 (17.7)	15 (20.6)
Reasons for not receiving HPV vaccined
Not knowing enough about the vaccine	26 (38.2)	31 (42.5)	0.61
Lack of a doctor recommendation	23 (33.8)	25 (34.3)	0.96
Thinking the vaccine was only for females	20 (29.4)	20 (27.4)	0.79
Not having been to a doctor lately	16 (23.5)	24 (32.9)	0.22
Not knowing where to get the vaccine	14 (20.6)	24 (32.9)	0.10
Cost	17 (25.0)	13 (17.8)	0.30

Note. HPV= human papillomavirus; SD=standard deviation; HIV=human immunodeficiency virus; STD=sexually transmitted disease. Percentages may not total 100 due to rounding. P- values represent findings of analyses assessing differences between intervention and control groups using chi-square analyses for categorical variables and independent samples t-tests for continuous variables

^aOther=never married, divorced, separated, or widowed

^bAsked only of participants who reported having a regular healthcare provider (n=74)

^c4 item scale, α=0.86; possible range 1-5 with higher values indicating greater electronic health (e-health) literacy

^d4 Showing the 6 most common responses overall. Participants could select more than one reason from a predefined list, and each reason was coded as a separate dichotomous variable. All other reasons were endorsed by fewer than 20% of participant.

Analysis

We first examined if baseline demographic and health-related characteristics differed between intervention and control groups due to chance imbalance using chi-square tests for categorical variables and independent samples t-tests for continuous variables. We then determined the effects of the intervention using multiple linear regression. Each outcome (at posttest) was regressed on study group (intervention vs. control), controlling for the baseline measure of the outcome and any potential residuals confounders (i.e., demographic and health-related variables that differed (p<.10) between study groups at baseline). We also calculated Cohen’s d to assess the magnitude of intervention effects. All analyses were conducted using Stata version 14.0 (Statacorp, College Station, TX) and two-tailed statistical tests with a critical alpha of .05 unless otherwise noted.

Results

Participant characteristics

Table 1 provides data on baseline demographic and health-related characteristics, including reasons why participants had not yet received HPV vaccine. The majority (84%) of participants self-identified as gay. Most were ages 22-25 (60%), non-Hispanic white (58%), and not married or living with a partner (80%). Most participants had some form of health insurance, either through a parent (44%) or self (38%), but fewer than half had a routine medical check-up in the last year (47%). About half (52%) of participants did not have a regular healthcare provider and, among those who did almost a third (31%) had not disclosed their sexual orientation to their provider. Fewer than 9% reported having received an HPV vaccination recommendation from a healthcare provider. Intervention and control groups were equivalent on all baseline characteristics, except for income; a greater proportion of participants in the intervention than in the control group had an annual income of at least $50,000 (34% vs. 14%, p=.01).

HPV vaccination knowledge, attitudes, and beliefs

As shown in Table 2, intervention and control groups had similar HPV vaccination knowledge, attitudes, and beliefs at baseline (all p>.05). Overall, participants had moderately high knowledge about HPV and HPV vaccine (mean [M]=0.72, standard deviation [SD]=0.32), and believed that getting an HPV-related disease would be severe (M=3.32, SD=0.74). However, they also perceived a relatively low likelihood of getting an HPV-related disease (M=2.30, SD=0.63), reported moderate HPV vaccine-related harms (M=3.40, SD=0.75), and expressed moderate agreement that HPV vaccine was effective (M=2.76, SD=0.83). On average, participants had moderate intentions to get vaccinated within the next 6 months (M=3.33, SD=0.88).

Table 2.

Baseline HPV and HPV vaccination knowledge, attitudes, and beliefs, by study group

	Intervention (n=68)		Control (n=73)		p
	Mean	(SD)	Mean	(SD)
Knowledge of HPV and HPV vaccinea	0.75	(0.31)	0.70	(0.32)	0.30
Perceived severity of HPV-related diseaseb	3.31	(0.66)	3.33	(0.81)	0.85
Perceived likelihood of HPV-related diseasec	2.30	(0.65)	2.29	(0.61)	0.88
Perception that MSM are at higher risk of anal cancer relative to other mend	3.65	(0.99)	3.67	(1.00)	0.89
Perceived effectiveness of HPV vaccinee	2.71	(0.78)	2.82	(0.87)	0.44
Perceived harms of HPV vaccined	3.35	(0.88)	3.44	(0.60)	0.50
Perceived barriers to getting HPV vaccinef	2.69	(0.91)	2.71	(0.90)	0.93
Perceived positive social norms of HPV vaccination among gay and bisexual mend	2.88	(0.95)	2.82	(0.84)	0.28
HPV vaccination self-efficacyf	3.94	(0.77)	4.11	(0.73)	0.69
Intent to get HPV vaccine in next 6 monthsd	3.34	(0.78)	3.33	(0.97)	0.95

Note. HPV=human papillomavirus; SD=standard deviation; MSM=men who have sex with men P-values represent findings of analyses assessing differences between intervention and control groups using independent samples t-tests.

^aProportion of correct responses 5 true/false items about HPV and HPV vaccine

^b3 item scale, α=0.79; each item used a 4-point response scale ranging from 1=“not at all [serious]” to 4=“very [serious]”

^c3 item scale, α=0.81; each item used a 4-point response scale ranging from 1=“no chance” to 4=“high chance”

^dSingle item using a 5-point response scale ranging from 1=“strongly disagree” to 5=“strongly agree”

^e2 item scale, α=0.82; each item used using a 4-point response scale ranging from 1=“not at all” to 4=“a lot”

^f2 item scale, α=0.73; each item used a 5-point response scale ranging from 1=“strongly disagree” to 5=”strongly agree”

Table 3 compares intervention and control groups for study outcomes on the posttest survey controlling for the pretest measure of each outcome and income (which differed between groups at baseline). Intervention participants reported higher mean levels of relative perceived risk of anal cancer among MSM compared to other men (4.38 vs. 4.10, b=0.34, p<.05, d=0.30) and greater HPV vaccination self-efficacy (4.42 vs. 4.32, b=0.15, p<.05, d=0.18). Intervention participants also had lower perceived harms of HPV vaccine than control group participants (3.75 vs. 4.05, b=−0.34, p<.05, d=−0.23). In addition, findings indicated a trend toward greater intentions to get vaccinated in the next 6 months among intervention participants (4.04 vs. 3.82, b=0.26, p<.10, d=0.21), which while not statistically significant, suggests a meaningful effect of the intervention on vaccination intent.

Table 3.

Short-term effects of the Outsmart HPV intervention on HPV and HPV vaccine knowledge, attitudes, and beliefs

	Intervention (n=68)	Control (n=73)
	Mean	(SD)	Mean	(SD)	b	p	Cohen’s d
Knowledge of HPV and HPV vaccinea	0.94	(0.17)	0.94	(0.14)	0.00	0.95	0.00
Perceived severity of HPV-related diseaseb	3.48	(0.58)	3.42	(0.66)	0.10	0.56	0.06
Perceived likelihood of HPV-related diseasec	2.62	(0.65)	2.54	(0.64)	0.12	0.67	0.04
Perception that MSM are at higher risk of anal cancer relative to other mend	4.38	(0.81)	4.10	(0.84)	0.34	0.02	0.30
Perceived effectiveness of HPV vaccinee	3.26	(0.67)	3.27	(0.76)	−0.01	0.68	0.05
Perceived harms of HPV vaccined	3.75	(1.08)	4.05	(0.66)	−0.34	0.01	−0.23
Perceived barriers to getting HPV vaccinef	2.30	(1.00)	2.32	(0.88)	−0.02	0.94	−0.01
Perceived positive social norms of HPV vaccination among gay and bisexual mend	3.69	(1.01)	3.52	(1.02)	0.17	0.39	0.13
HPV vaccination self-efficacyf	4.42	(0.63)	4.32	(0.66)	0.15	0.04	0.18
Intent to get HPV vaccine in next 6 monthsd	4.04	(0.89)	3.82	(0.82)	0.26	0.08	0.21

Note. HPV= human papillomavirus; SD=standard deviation; MSM=men who have sex with men Table shows HPV and HPV vaccination knowledge, attitudes, and beliefs at posttest. Coefficients (b) and P-values come from regression models of posttest scores controlling for the baseline level of the outcome and adjust for baseline differences in income between study groups.

^aProportion of correct responses 5 true/false items about HPV and HPV vaccine

^b3 item scale, α=0.79; each item used a 4-point response scale ranging from 1=“not at all [serious]” to 4=“very [serious]”

^c3 item scale, α=0.81; each items used a 4-point response scale ranging from 1=“no chance” to 4=“high chance”

^dSingle item using a 5-point response scale ranging from 1=“strongly disagree” to 5=“strongly agree”

^e2 item scale, α=0.82; each item used using a 4-point response scale ranging from 1=“not at all” to 4=“a lot”

^f2 item scale, α=0.73; each item used a 5-point response scale ranging from 1=“strongly disagree” to 5=“strongly agree”

Acceptability

As shown in Table 4, participants in both study groups reported their study materials were highly acceptable, with mean scores for all acceptability measures greater than a 4.00. Participants in the intervention (vs. control) group more strongly endorsed that their materials were easy to understand (4.72 vs. 4.42, p<.01, d=0.49), and there was a trend toward participants in the intervention group more strongly endorsing that their materials were accurate (4.51 vs. 4.32, p<.10, d=0.28). The direction of findings and effect sizes for other measures suggest that the intervention was equally or more acceptable than the control condition, though none of these comparisons reached statistical significance.

Table 4.

Mean acceptability ratings for study materials, by study group

	Intervention (n=68)		Control (n=73)		p	Cohen’s d
	Mean	(SD)	Mean	(SD)
Information quality
The information was…
easy to understand	4.72	(0.48)	4.42	(0.71)	0.005	0.49
credible	4.49	(0.68)	4.38	(0.64)	0.36	0.17
important	4.69	(0.50)	4.64	(0.48)	0.57	0.10
accurate	4.51	(0.63)	4.32	(0.72)	0.08	0.28
relevant to me	4.60	(0.55)	4.53	(0.60)	0.48	0.12
System quality
I like how [materials] looked.	4.62	(0.52)	4.48	(0.63)	0.16	0.24
It is easy to use.	4.57	(0.61)	4.56	(0.53)	0.90	0.02
It loads all the text and graphics quickly.	4.57	(0.58)	4.64	(0.48)	0.43	−0.13
It is easy to go back and forth between pages.	4.53	(0.66)	4.38	(0.66)	0.19	0.23
It is easy to locate information.	4.54	(0.56)	4.41	(0.60)	0.17	0.22
Usefulness
It improved my ability to make healthier choices.	4.46	(0.63)	4.37	(0.66)	0.43	0.14

Note. HPV= human papillomavirus; SD=standard deviation

All items were on a 5-point agree-disagree scale (1=strongly disagree, 5=strongly agree). P-values represent results from independent samples t-tests.

Discussion

Vaccinating YGBM against HPV is critical for protection against HPV infection and future HPV-related disease, including anogenital cancers. We developed and pilot-tested Outsmart HPV, an innovative web-based intervention for YGBM. Findings provide preliminary support for the intervention having positive short-term effects. At posttest, we saw positive effects of the intervention on several attitudes and beliefs, including those related to the risk of anal cancer among MSM, HPV vaccination self-efficacy, and perceived harms of HPV vaccine. These positive effects are likely attributable to the intervention’s focus on: (a) population-targeted information about HPV and HPV-related diseases among YGBM, which could have affected beliefs related to risk of disease; (b) skills-building strategies and resources for getting HPV vaccine, which could have affected self-efficacy; and (c) information that addresses specific barriers and concerns about HPV vaccination, including those related to participants’ perceived harms. We also found a trend with a medium effect size for the intervention positively affecting men’s intention to get HPV vaccine, which provides further support that tailored interventions can improve intentions.[33, 34] Addressing these attitudes and beliefs is potentially important to increasing HPV vaccination among YGBM, as they are key constructs of the PMT, and have been associated with HPV vaccine uptake in past work.[12] A critical next step is to determine Outsmart HPV’s impact on increasing actual HPV vaccine uptake among YGBM.

Participants in both study conditions improved on all assessed constructs between pretest and posttest, except perceived severity of HPV-related disease, and that several of these changes were similar between study groups. Similarities may be due in part to the conservative study design used for this pilot study—evaluating two competing HPV vaccination interventions rather than comparing Outsmart HPV to a no-treatment control group—thus making it harder to detect intervention effects. Yet, even with the inclusion of this strong control group and a modest pilot study sample size, our intervention showed statistically significant effects on several constructs associated with HPV vaccination. This suggests that it is indeed beneficial for HPV vaccine interventions to move beyond providing basic education to increase knowledge about HPV and HPV vaccine, to include additional components. As multi-component interventions may be more effective than those taking a single-pronged approach,[8] Outsmart HPV provides more than just education and bundled multiple components (e.g., skills-building strategies, targeted and tailored content, resources for getting vaccinated), that can affect key drivers of vaccination.

Participants in both study groups, but particularly in the intervention group, reported high acceptability their study materials. Of note, we found that participants in the intervention more strongly endorsed that their materials were easy to understand, as well as support for these participants more strongly endorsing that their materials were accurate and that the system appearance and usability were of higher quality. Findings concerning understandability are especially notable since control materials were closely modelled after the VIS for HPV vaccine, which is designed to be easy to understand for individuals considered to have “low literacy.”[35] However, the VIS presents information solely through text, whereas Outsmart HPV was designed to be less-text heavy and present information through a variety of formats (e.g., infographics). These more visual formats may have led to the observed differences in acceptability, as adolescents and young adults have a preference for, and may have better recall of, the graphical presentation of information compared to text only.[36, 37] We developed the infographics and other visual formats through extensive and iterative formative research, including input from YGBM at different stages of the intervention’s development.

Strengths and limitations

Study strengths include a national sample similar in demographic and health-related characteristics to YGBM in other studies [23] and a rigorous randomized controlled design. Limitations include self-reported measures which may be subject to response bias. However, past research has shown young adults can accurately their vaccination status with a only a 2% net bias of self-reported HPV vaccination data (which we used to determine study eligibility) compared to medical records.[38] Although effects on most outcomes moved in favor of the intervention, given the pilot nature of our study, our ability to detect these effects with statistical precision was limited by our modest sample size. We also do not know whether the observed short-term effects will be maintained over time or associated with actual HPV vaccine uptake. Finally, we recruited participants through Facebook since recent national data show it is the most popular social media site among young adults (ages 18-29) who are online, with 88% reporting use in 2016[39]; future research should consider recruiting YGBM through other social media sites that are growing in popularity and may have different audiences (e.g., Instagram, Twitter, Grindr, MiGente).

Conclusion

HPV vaccination remains a public health priority. The current pilot study is the first to demonstrate the acceptability and short-term effects of a web-based HPV vaccination intervention specifically for YGBM. Results demonstrate positive changes in several attitudes and beliefs which have previously been associated with HPV vaccination, though additional research is needed to assess intervention effects on vaccine uptake. Findings support continued efforts to pursue population-targeted, individually-tailored and web-based strategies to address the low levels of HPV vaccine uptake among YGBM. Future research may also adapt this approach to other priority populations at increased risk of either HPV-related disease or undervaccination.

Abbreviations

HPV

human papillomavirus

MSM

men who have sex with men

YGBM

young gay and bisexual men