eHealth Literacy and Intervention Tailoring Impacts the Acceptability of a HIV/STI Testing Intervention and Sexual Decision Making among Young Gay and Bisexual Men

Abstract

We assessed whether young men who have sex with men’s acceptability with the online Get Connected! intervention and subsequent sexual health decision making were influenced by their baseline eHealth literacy (high vs. low competency) and intervention tailoring (tailored or non-tailored intervention condition). Participants (n=127) were on average 21 years old, 55% non-Hispanic white, and used the Internet 1–3 hours a day (54%). Compared to the high eHealth literacy/tailored intervention group: 1) those in the low eHealth literacy/tailored intervention condition and participants in the non-tailored intervention condition (regardless of eHealth literacy score) reported lower intervention information quality scores; 2) those in the low eHealth literacy/non-tailored intervention group reported lower intervention system quality scores and that the intervention had less influence on their sexual health decision making. Future similar intervention research should consider how eHealth literacy might influence participants’ abilities to navigate intervention content and integrate it into their sexual decision making.

Introduction

Online interventions have been shown to modify a variety of health behaviors (Webb, Joseph, Yardley, & Michie, 2010), although literature reviews note wide variation in their effectiveness (Morrison, Yardley, Powell, & Michie, 2012). Systematic reviews generally find that tailoring informational content to individual users improves outcomes in traditional (Noar, Benac, & Harris, 2007) and eHealth (Morrison et al., 2012) interventions. Tailoring content is intended to ease the delivery and relevance of the information presented to participants. However, little is known about the effects of tailoring on users with varying levels of eHealth literacy, defined as the degree to which one navigates, understands and applies electronic health information (Norman & Skinner, 2006). Tailoring may benefit users with moderate or high levels of eHealth literacy given their existing skills to navigate specific health information and their familiarity with electronic devices (e.g., web, smartphones). Conversely, users with lower eHealth literacy may have difficulty navigating and integrating tailored information into their behavioral repertoire given the cognitive demands of simply navigating the interface and the physical device (Clark & Mayer, 2016). Given the increasing popularity of eHealth interventions to address the HIV/STI prevention and care needs of vulnerable populations that include youth, racial/ethnic minorities, and men who have sex with men (Hightow-Weidman, Muessig, Bauermeister, Zhang, & LeGrand, 2015; Sullivan, Jones, Kishore, & Stephenson, 2015), it is vital that the programs developed are sensitive to issues of eHealth literacy. Therefore, the purpose of this study is to investigate the intersection of intervention tailoring and eHealth literacy on participants’ experience of an online HIV/STI (sexually transmitted infection) testing intervention, called Get Connected!, and their subsequent sexual health decision making.

Get Connected! is an online intervention developed to increase HIV/STI testing among gay, bisexual, and other men who have sex with men (MSM) between 15 and 24 years of age (referred to hereafter as young MSM or YMSM) living in Metro Detroit. One hundred thirty YMSM were randomized to the Get Connected! intervention or to a control condition. The intervention is described in greater detail elsewhere (Bauermeister et al., 2015); briefly, Get Connected! employed tailoring algorithms based on key characteristics of participants (e.g., age, race/ethnicity, relationship status, sexual identity) to tailor imagery (e.g., Black YMSM were shown pictures of similar other Black young men) and intervention content (e.g., participants who reported never having tested for HIV received messages promoting testing, while those who had been tested received messages reinforcing their testing behaviors). The intervention included content on the facts about HIV/STIs transmission and care; participants’ motivations, values and strengths regarding STI testing; barriers to getting HIV/STI tested; and, a HIV/STI test locator where YMSM could seek out HIV/STI care in the region. Those assigned to the control condition did not receive any tailored content and had access only to the HIV/STI test locator. At 30-day follow-up, YMSM in the Get Connected! tailored intervention condition believed the intervention to be more credible than men in the non-tailored control condition, and more reported getting testing than those in the non-tailored control condition (Bauermeister et al., 2015).

For the present study, we sought to answer the following two research questions: 1) What is the effect of intervention tailoring and eHealth literacy on acceptability ratings of the Get Connected! intervention?; 2) What is the effect of intervention tailoring and eHealth literacy on using the Get Connected! intervention for sexual health decisions related to the primary outcome (i.e., HIV and STI testing)? Results may be used to guide future research in online health interventions by informing how to optimize information tailoring by considering users’ eHealth literacy proficiency.

Methods

Participants

To be eligible for the study, participants must have self-reported that they were between the ages of 15–24 (inclusive), cis-male (i.e., assigned sex at birth as male and self-identifies as male), reside in the five counties included in the larger Southeast Michigan region, and having had sex with a male partner in the prior 6 months. Participants were recruited at Southeast Michigan LGBTQ pride celebrations, by distributing palm cards with information about the study at various Southeast Michigan bars and clubs that cater to YMSM, via Facebook ads, and print and online ads in a Michigan-based LGBTQ social magazine. Persons interested in the study called a toll free number to verify their eligibility. Out of the 130 YMSM who were enrolled and completed the baseline assessment, 104 YMSM completed the 30-day follow-up questionnaire. Participants were reimbursed $30 in VISA e-gift cards ($20 for completing the baseline and intervention; $10 for completing the 30-day follow-up) for their participation.

Study Procedures

Eligible participants were given a unique identifier code to enter the Get Connected! website and were then prompted to create a question that only they would be able to answer and then provide the response. Upon subsequent re-entry to the site, participants were given their personalized question; their correct response to the question served as their password and allowed them to return to the section they had most recently visited. After logging into the system, eligible participants consented, received information on the study’s logistics (including study confidentiality procedures, the overall goals of the study, and incentive payment procedures), and completed the baseline assessment. Next, participants were randomized using a 2:1 ratio into the tailored Get Connected! intervention condition or to the non-tailored control condition. At the 30-day follow-up, participants were asked about their recent HIV risk behaviors. Primary HIV testing outcomes of trial are published elsewhere (Bauermeister et al., 2015). All study procedures, including a waiver of parental permission, were approved by the University of Michigan Institutional Review Board.

Measures

eHealth Literacy

eHealth literacy was measured using an adapted, shortened 6-item version of the eHealth Literacy Scale (eHEALS; (Norman & Skinner, 2006). eHEALS asks respondents about their knowledge of online health resources including where to locate them, how to find them, how to use information once obtained, and comfort with assessing the quality of online health information on a 5-point Likert scale (1= Strongly Disagree; 5=Strongly Agree). We computed a sum score based on participants’ answers; higher scores indicate greater eHealth literacy (α=0.94).

We used a median split used to dichotomize youth into either lower (score = 0–17) or higher (score = 18–24) eHealth literacy groups. A median split was used since there is not an accepted cut-off score differentiating higher and lower online health seeking competency with respect to the eHEALS scale, and this approach was used in a prior study among African American women (Blackstock et al., 2016).

Intervention Acceptability

Across both conditions, we ascertained participants’ overall satisfaction with the intervention with three items. Two items (e.g., Overall, I’m very satisfied with Get Connected!, I would recommend Get Connected to friends) were answered using a 7-point scale (1=Strongly Disagree;7=Strongly Agree). The last item (How likely would you be to continue using Get Connected if it were available) was answered using a 7-point scale (1=Very Unlikely; 7=Very Likely). We created a mean composite score from these three items, where higher scores indicate greater satisfaction (α=0.75).

We also adapted factors from the Information Systems Success Model (ISSM) proposed by DeLone & McLean (1992; 2003) to assess users’ perception of the information quality (4 items; α=0.86), system quality (6 items; α=0.85), and perceived usefulness of the intervention (6 items; α=0.95). Information quality refers to users’ perceptions of the quality of the information on the intervention, such as whether they perceived the information to be accurate and clear; system quality refers to users’ perceptions of how easy the intervention was to navigate and its technical responsiveness; and perceived usefulness reflects how the intervention was perceived by participants to impact their health behaviors. Each item could be answered using a 7-point scale (1=Strongly Disagree; 7=Strongly Agree). We computed a mean score for each of these three domains.

Sexual Health Decision Making

At the 30-day follow-up, participants were asked to indicate whether they had used the sexual health information they received in their assigned version of the intervention during their sexual decision making. We measured four specific sexual health decisions: evaluating their personal risk for HIV/STIs; educating others about HIV/STIs; deciding whether to get tested for HIV; and deciding whether to get tested for STIs. Participants could answer each statement with a 4-point scale (1=Never; 4=Most of the time).

Demographic Characteristics

Participants were asked to report their age (in years) and highest level of educational attainment (8^th grade or less, some high school, graduated high school/GED, technical school, associate degree, some college, college, some graduate school, graduate school). Education was then dichotomized to indicate whether a participant had completed high school (1) or not (0). We measured race/ethnicity using the following categories: Black/African American, White/Caucasian, Hispanic/Latino, American Indian/Alaskan Native, Asian, Native Hawaiian/Pacific Islander, and Other Race. For the purposes of our analysis, we dichotomized participants’ race/ethnicity as a racial/ethnic minority (0) or white (1). Finally, men were asked how many hours per day they usually spend on the Internet outside of school or work responsibilities (no hours; less than an hour; 1 to 3 hours; 4 to 6 hours; 7 to 9 hours; 10 to 12 hours; 13 to 15 hours; 16 hours or more). The number of hours spent online for reasons other than school or work responsibilities were collapsed into: less than one hour a day (1), one to three hours a day (2), four to six hours a day (3), or seven or more hours a day (4).

Data Analytic Strategy

In order to test for differences based on intervention arm and eHealth literacy, we categorized participants into one of four groups: 1) Tailored Intervention with High eHealth Literacy; 2) Tailored Intervention with Low eHealth Literacy; 3) Non-tailored Control with High eHealth Literacy; 4) Non-tailored Control with Low eHealth Literacy. We then tested for group differences across the four intervention acceptability domains and the four sexual health decision-making indicators using a series of linear regression analyses. All regression models were adjusted for age, race/ethnicity, education, and the number of hours youth used the Internet for non-school or work related activities. The level of significance was set at p<.05 for all analyses.

Results

Demographic Characteristics

Youth were on average 21 years of age, and were overall 55% non-Hispanic white, had more than a high school education (76%), and used the Internet outside of school and work 1–3 hours per day (54%). There were no significant differences in demographic characteristics by intervention arm/eHealth literacy group (see Table 1). The mean score for the modified eHEALS scale was 17.7 (range 6–24).

Table 1.

Demographic Characteristics by Tailoring and eHealth Literacy Group

	Overall (n=130)	Tailored Intervention		Non-Tailored Control
		High Literacy (n=52)	Low Literacy (n=32)	High Literacy (n=25)	Low Literacy (n=18)
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	Test Statistic
Age	21.1 (2.2)	21.5 (2.1)	21.0 (2.2)	20.9 (2.3)	19.94 (2.4)	F(3,122)=2.34
eHEALS	17.7 (3.9)	20.3 (2.6)	14.0 (3.0)	19.4 (2.3)	14.7 (3.1)	F(3,123)=45.02***
	% (n)	% (n)	% (n)	% (n)	% (n)
% White	55.1 (70)	51.9 (27)	59.4 (19)	44.0 (11)	72.2 (13)	FE=0.29ns
% High School	75.6 (96)	80.8 (42)	71.9 (23)	72.0 (18)	72.2 (13)	X2(3, 127)=1.28 ns
Internet Use/Day
< 1 hour	10.2 (13)	9.6 (5)	18.8 (6)	0 (0)	11.1 (2)	FE=0.16ns
1–3 hours	54.3 (69)	59.6 (31)	46.9 (15)	48.0 (12)	61.1 (11)
4–6 hours	24.1 (31)	19.2 (10)	31.3 (10)	28.0 (7)	22.2 (4)
7+ hours	11.0 (14)	11.5 (6)	3.1 (1)	24.0 (6)	5.6 (1)

Notes. FE = Fisher’s exact test (p-value reported); ^ns=not significant;

^*p<.05;

^**p<.01;

^***p<.001

Intervention Acceptability

Table 2 shows the results for the regression analyses examining the impact of intervention arm/eHealth literacy group on acceptability ratings of the Get Connected! intervention when adjusted for age, race/ethnicity, education, and hours of Internet use per day.

Table 2.

Estimated Effect of Tailoring and eHealth Literacy on Intervention Acceptability

	Overall Satisfaction		Information Quality		System Quality		Perceived Usefulness
	b	SE	b	SE	b	SE	b	SE
eHealth Literacy by Condition a
Tailored Intervention with Low Literacy	−.07	.20	−.49**	.17	−.54*	.18	−.21	.24
Non-Tailored Control with Low Literacy	−.31	.25	−.68**	.22	−.41	.23	−.42	.31
Non-Tailored Control with High Literacy	−.31	.21	−.42*	.19	−.17	.20	−.33	.27
Age	−.06	.04	−.04	.04	.005	.04	−.09	.05
Race/Ethnicity b	−.52**	.16	−.27	.14	−.34*	.15	−.37	.20
Completion of High School c	−.32	.21	.14	.18	−.29	.20	−.17	.26
Internet Use	.17	.10	.14	.09	.15	.09	.26*	.13
Constant	7.35***	.89	7.02***	.80	6.60***	.85	7.59***	1.09
Omnibus Test	F(7,113)=3.27**		F(7,110)=3.06**		F(7,109)=3.56**		F(7,110)=2.05*
R-Squared	16.86%		16.30%		18.59%		11.52%

^aParticipants in the Tailored Intervention Condition with High eHealth Literacy served as referent group

^bRacial/ethnic minorities served as referent group

^cYMSM without a high school education served as referent group

^*p<.05;

^**p<.01;

^***p<.001

We observed no differences in overall satisfaction with the intervention across tailoring and eHealth literacy groups, after controlling for the sociodemographic covariates. Non-Hispanic White participants, however, reported less satisfaction with the intervention than racial/ethnic minorities. We found no association between overall satisfaction and age, high school completion, and Internet use per day.

We observed differences across tailoring and eHealth literacy groups for information quality. Compared to participants in the Tailored Intervention with High eHealth Literacy group, we noted that participants in the tailored intervention condition with low eHealth literacy reported poorer rating scores to the intervention’s information quality. Similarly, both Non-tailored Control condition groups reported poorer information quality, irrespective of their eHealth literacy, compared to youth with high eHealth literacy in the tailored condition. We found no association between information quality and age, race/ethnicity, high school completion, and Internet use per day.

Participants with lower eHealth literacy and assigned to the tailored intervention condition scored the system quality as poorer than those in the high eHealth literacy tailored intervention condition. We observed no differences in system quality scores between the non-tailored control condition, irrespective of their eHealth literacy, compared to the Tailored Intervention with High eHealth Literacy group. Non-Hispanic White participants rated the system quality lower than racial/ethnic minorities. We found no association between system quality and age, high school completion, and Internet use per day.

We observed no group differences in participants’ ratings of the intervention’s perceived usefulness. Participants who reported greater Internet use per day were more likely to find the intervention useful. We found no association between perceived usefulness and age, high school completion, and race/ethnicity.

Sexual Health Decision Making

Table 3 shows the results for the regression analyses examining the impact of each intervention/online health seeking competency group on use of the Get Connected! intervention for sexual health decision making when adjusted for age, race/ethnicity, education, and hours of Internet use per day.

Table 3.

Estimated Effect of Tailoring and eHealth Literacy on Sexual Health Decision Making

In which of the following ways did you use the sexual health information that you received in GC? Did you use the information to…	Evaluate your personal risk for HIV/STIs?		Educate others about HIV/STIs?		Decide whether to get tested for HIV?		Decide whether to get tested for STIs?
	b	SE	b	SE	b	SE	b	SE
eHealth Literacy by Condition a
Tailored Intervention with Low Literacy	.09	.28	.18	.26	.21	.26	.28	.27
Non-Tailored Control with Low Literacy	−.88*	.34	−1.08***	.31	−.80*	.32	−.75*	.32
Non-Tailored Control with High Literacy	.12	.29	−.23	.27	.05	.28	−.28	.27
Age	−.06	.06	−.06	.05	−.11*	.06	−.16**	.06
Race/Ethnicity b	−.37	.23	−.69**	.21	−.51*	.21	−.44*	.22
Completion of High School c	−.46	.29	−.18	.27	.11	.28	−.16	.28
Internet Use	.008	.13	−.09	.12	−.07	.13	.13	.13
Constant	4.55***	1.27	4.69***	1.17	5.27***	1.19	5.86	1.19
Omnibus Test	F(7,93)=2.57*		F(7,93)=4.13***		F(7,93)=2.35***		F(7,92)=3.25**
R-Squared	16.22%		23.71%		15.01%		19.80%

^aParticipants in the Tailored Intervention Condition with High eHealth Literacy served as referent group

^bRacial/ethnic minorities served as referent group

^cYMSM without a high school education served as referent group

^*p<.05;

^**p<.01;

^***p<.001

Compared to participants in the Tailored Intervention with High eHealth Literacy group, we found that participants with low eHealth literacy in the non-tailored control condition were less likely to have evaluated their personal risk for HIV/STIs at the 30-day follow-up. We observed no other differences in personal risk for HIV/STIs by tailoring and eHealth literacy, nor by age, high school completion, race/ethnicity, or Internet use per day.

Participants with low eHealth literacy in the non-tailored control condition were less likely to educate others about HIV/STIs than youth with high eHealth literacy and in the tailored intervention. Non-Hispanic Whites were less likely to educate others about HIV and STIs than racial/ethnic minorities.

Compared to participants in the Tailored Intervention with High eHealth Literacy group, we found that participants with low eHealth literacy in the non-tailored control condition were less likely to report deciding to get tested for HIV and STI, respectively, at the 30-day follow-up. Older participants were less likely to report deciding whether to get tested for HIV and STIs, respectively. Non-Hispanic Whites were less likely to note report thinking about getting tested for HIV and STIs, respectively, than racial/ethnic minorities. We observed no other differences by tailoring and eHealth literacy groups, age, high school completion, or Internet use per day.

Discussion

Tailoring has been posited to improve the user experience by increasing relevance of intervention content and heighten motivation for behavior change (Noar et al., 2007). Online interventions provide ample opportunities to tailor content; however, a user’s level of eHealth literacy may play an important role in determining how intervention content is navigated and, ultimately, incorporated into his or her health behaviors. We sought to examine how youths’ eHealth literacy and intervention tailoring impacted their experience of the online Get Connected! HIV testing intervention and their subsequent sexual decision making. eHEALS scores in the current sample ranged from 6 to 24, with two-fifths of youth reporting lower online health seeking competency. Overall, YMSM’s ratings of the intervention and their subsequent sexual health decision making varied as a function of their baseline eHealth literacy and whether they received tailored intervention content.

A complex picture emerged of the combined effect of tailoring and eHealth literacy on participants’ acceptability ratings of the Get Connected! intervention. The presentation of intervention content (information, resources, and a HIV/STI test locator) in the tailored condition employed images and content that reflected each participants’ demographic characteristics (e.g., age, race). Although we did not observe differences across these demographic characteristics on participants’ eHEALS scores, participants in the tailored Get Connected! intervention condition reported higher intervention system quality (a measure of how easy or hard the intervention was to navigate) if they had high eHealth literacy. As such, youth who entered into the tailored intervention with more skills to navigate the complex information appeared to do so more than peers with fewer skills. The same pattern, however, was not found in other intervention acceptability domains. Given that both intervention conditions had the same purpose (i.e., to encourage HIV/STI testing among YMSM), we were not surprised by the absence of group differences in participants’ overall satisfaction or perceived usefulness. However, compared to youth with high eHealth literacy and who were assigned to the tailored intervention condition, participants in the three other groups rated the information quality (e.g., their perception of how accurate or clear the information was) of their assigned intervention condition as lower. Taken together, these findings suggest that there may be complex interactions between intervention tailoring and eHealth literacy on young MSM’s ability to navigate an online intervention and their appraisal of the quality of the content presented to them. In light of these findings, future research examining how to account for varying levels of eHealth literacy during the design of the online intervention might be warranted.

The effect of eHealth literacy and intervention tailoring was also related to youth’s beliefs about how they used the information in the Get Connected! intervention to inform their subsequent sexual health decision making. Compared to participants with high eHealth literacy who received the tailored intervention, youth who were assigned to the non-tailored control condition and who had low eHealth literacy were less likely to report using the information in the Get Connected! intervention to evaluate their personal risk for HIV/STIs, educate others about HIV/STIs, and decide whether to get tested for HIV and STIs. Interestingly, we found no other differences between youth with high eHealth literacy in the tailored and the two other groups (i.e., low eHealth literacy in the tailored group; high eHealth literacy in the control group). The absence of differences may be attributable to participants’ ability to understand and use the information provided in the tailored intervention condition to make sexual health decisions, irrespective of their eHealth literacy (as could be the case for participants in the tailored intervention condition and who had low eHealth literacy), or to supplement the missing information from other online sources if they have high eHealth literacy (as could be the case for YMSM with high eHealth literacy in the control condition). These findings suggest that the intersection of eHealth literacy and exposure to tailored content may be vital to consider when researchers evaluate risk reduction interventions, as some participants’ perceived gains may differ from other peers based on the amount of tailoring and their eHealth literacy skills.

Although YMSM are often portrayed as competent in e-technologies and early adopters of new technologies (Hightow-Weidman, Muessig, Bauermeister, Zhang, & LeGrand, 2015), our data indicate that they still experience significant variation in their competency to navigate and interpret online health information. In our multivariate analyses, we were particularly surprised by the racial/ethnic variation in users’ satisfaction with the intervention and use of intervention content to sexual health decision-making. Racial and ethnic minority youth reported higher acceptability scores across several domains and that they were able to use the information in Get Connected! to a greater degree in their subsequent decision making than their non-Hispanic White counterparts. Given the high incidence of new HIV cases among racial/ethnic minority YMSM in the United States (Centers for Disease Control and Prevention, 2015), these findings show promise of the Get Connected! intervention for this segment of the population. On the other hand, differences across sociodemographic characteristics underscores the importance of considering audience segmentation and cultural characteristics during the design and implementation of eHealth interventions for diverse groups of YMSM.

Limitations

There were a number of limitations that are important to consider when interpreting results. First, findings from the study are based on a community sample of YMSM from the Detroit and surrounding areas; the generalization of these findings is limited due to the small sample size and the recruitment and survey methods employed. The extent to which these findings apply to the larger population of YMSM is unknown. Moreover, given the pilot nature of our trial, our ability to detect these effects with statistical precision was limited by our small sample size and short follow-up period. In future scaled-up versions of the intervention, we intend to have a larger sample size, as well as a greater number of follow-up periods, in order to examine how eHealth literacy and tailoring affects YMSM’s perceptions of the intervention and their sexual decision-making. It is also important to note that we evaluated two competing HIV interventions in our study design (e.g., tailored intervention vs non-tailored control) without a no-treatment control group. Although the inclusion of a no-treatment control group (e.g., an intervention focused on another topic such as diet and exercise) could have allowed us to discern how eHealth literacy and tailoring affected the Get Connected! intervention differentially based on a topic’s content, we felt that withholding referrals to care to our population would be unethical given their vulnerability to HIV and STIs. Finally, given the number of associations examined, we may have increased the propensity for Type I errors; future research should seek to replicate the current findings.

Conclusions

Researchers creating online HIV testing interventions for YMSM should be sensitive to eHealth literacy, as it appears to be associated with their experience of the intervention and their subsequent sexual health decision making. Thus, it may be important to provide additional online health literacy training in eHealth interventions to support users who may feel less competent navigating the information. Alternatively, if tailoring is vital for the delivery of content, researchers may wish to consider creating tailoring algorithms around low eHealth literacy and examine whether this feature helps to improve overall system acceptability and support sexual decision making. Future research examining how to integrate design and functionality features to overcome eHealth literacy variability among YMSM populations may be warranted.