Post-test comparison of HIV test knowledge and changes in sexual risk behaviour between clients accessing HIV testing online versus in-clinic

Abstract

Objective

Internet-based HIV testing offers the potential to address privacy-related barriers to testing and increase frequency of testing but may result in missed opportunities related to sexual health education and prevention that typically occur in face-to-face encounters. In this study, we assessed the HIV test knowledge and sexual risk behaviour of clients testing for HIV through GetCheckedOnline, an internet-based sexually transmitted and bloodborne infection testing platform inclusive of HIV testing, in comparison to clients testing through a large sexual health clinic.

Methods

We concurrently recruited GetCheckedOnline clients and clinic clients from Vancouver, Canada, over the course of a 10-month period during 2015–2016. Participants completed baseline and 3-month questionnaires, anonymous and online. A six-item score was used to estimate knowledge of HIV test concepts typically conveyed during an HIV pretest encounter in a clinic. We used multiple regression to estimate associations between testing modality (online vs clinic based) and two outcomes—HIV test knowledge and change in condom use pre/post-test—with adjustment for relevant background factors.

Results

Among 352 participants, online testers demonstrated higher HIV post-test knowledge than clinic-based testers (mean score 4.65/6 vs 4.09/6; p<0.05); this difference was reduced in adjusted analysis (p>0.05). Men who have sex with men, clients with a university degree, those who have lived in Canada >10 years and English speakers had higher HIV post-test knowledge (p<0.05). Eighteen per cent of online testers and 10% of clinic-based testers increased condom use during the 3 months post-test (p>0.05).

Conclusions

In this comparative study between online and clinic-based testers, we found no evidence of decreased HIV test knowledge or decreased condom use following HIV testing through GetCheckedOnline. Our findings suggest that with careful design and attention to educational content, online testing services may not lead to missed opportunities for HIV education and counselling.

Background

Internet-based (or online) testing for sexually transmitted and bloodborne infections (STBBI) is considered a convenient, private and low-barrier alternative to clinic-based testing. Relatively few online STBBI testing programmes have included HIV tests.^1–4 Online HIV testing offers the potential to reach individuals who have never tested before or who face barriers to testing access (eg, due to HIV-related stigma or inaccessibility of in-person testing), and to increase test frequency among individuals who are at risk of acquiring HIV due to engagement in high-risk behaviours.⁵ At the same time, however, online testing may result in missed opportunities for education and prevention that often occur during face-to-face clinical encounters.^{6 7}

The concern that online HIV testers will fail to receive important educational messages stems from long-standing guidelines for HIV screening, which recommend that HIV testing encounters include: explanation of the meanings of positive and negative results (in particular the window period and its effect on interpretation of a negative result), an assessment of the client’s preparedness to receive a positive test result, information about the implications of a positive result (ie, mandatory reporting) and the provision of counselling—all of which may vary by country or setting.^8–10 Current guidelines also emphasise not mandating behavioural counselling as a requirement of testing in order to reduce barriers to testing for clients unwilling or unprepared to participate in counselling.^{9 10}

While missed opportunities regarding HIV education are perceived as a potential risk of online testing, to our knowledge, there is no empirical evidence to support or refute this assumption.⁸ Face-to-face clinical counters no doubt offer a quality of interaction that cannot easily be replicated online; however, cost and capacity pressures limit the amount of time clinicians can spend with clients, which in turn often makes it difficult for providers to address all aspects of sexual health in a single visit.^{11 12} In addition, individual clients will have distinct learning styles (eg, visual vs auditory), oral and written language abilities, and abilities to retain information during face-to-face clinical encounters.^{13 14} The risk of missed educational opportunities could be mitigated through careful and thorough design of online services that attends to educational messages that clients would otherwise receive in a face-to-face clinical encounter.

As described in detail elsewhere, GetCheckedOnline.com (GCO, http://getcheckedonline.com) is an online STBBI testing service that was developed and launched in 2014 in British Columbia (BC), Canada.^{4 6} The objectives of this programme include increasing uptake and frequency of testing (leading to earlier diagnosis), by reaching populations with higher prevalence of STBBIs—many of whom face barriers to accessing testing.⁵

In this study, we assessed the HIV test knowledge and sexual risk behaviour of clients testing for HIV through GCO in comparison to clients testing in-person through the Provincial STBBI Clinic. The purpose was to understand whether clients testing through GCO may be missing opportunities for education and counselling provided during clinic-based interactions with testing providers. Given the extensive efforts to incorporate key pretest HIV messages into the website described below, we hypothesised that there would be no post-test differences in HIV test knowledge or pre/post-test changes in sexual risk behaviour between online and clinic-based testers.

Methods

Development of educational messages in GCO

GCO (http://getcheckedonline.com) is an online testing service for HIV, chlamydia, gonorrhoea, syphilis and hepatitis C, launched in September 2014 in Vancouver, Canada. Development of the intervention is described in detail elsewhere.⁴ Receiving an STBBI test through GCO involves five steps: (1) create an account; (2) complete an assessment of sexual history and risk; (3) consent to testing and print laboratory requisition form; (4) present in person to provide specimens (blood, urine, swabs) at a designated collection centre; and (5) receive results, online if all are negative, or by phone if any result is positive or indeterminate. Educational information about HIV was accessible through GCO at all online steps. During step 3, clients reviewed a page summarising key HIV concepts and then provided informed consent (online supplementary figure S1). In step 5, clients receiving negative HIV test results were shown information about the meaning of positive and negative results, including information about window periods.

The educational content about HIV testing on GCO was developed by nurses and health educators working at the BC Centre for Disease Control (BCCDC) and was based on key concepts necessary for an individual to provide informed consent for HIV testing. In addition to reviewing existing testing and consent guidelines, we interviewed the following individuals to elicit detailed feedback about best practices for delivering relevant HIV information through GCO: clinicians who work in sexual health clinics,⁸ individuals working with professional practice regulatory bodies,⁴ authors of HIV testing guidelines, ethicists¹⁵ and end users.^{7 15 16} Once educational content was finalised, we incorporated it into an online informed consent page. We then pilot-tested prototypes of the online informed consent page with potential end users to understand how best to deliver this information in order to achieve adequate informed consent for HIV testing online.¹⁵ The following information was specifically incorporated into the informed consent page: information about the tests clients will receive; benefits and harms of testing; follow-up of positive STBBI results by public health nurses; descriptions of test window periods; implications of positive and negative results; electronic storage of test results; availability of support services; and STBBI disease information and prevention messages (online supplementary figure S1).¹⁵ Online materials were written with a Flesch-Kincaid readability score at or below grade 8.^{17 18} At the time of this study, GCO was only offered in English.

Comparator group: clinic-based versus online testing modalities

The comparator group for GCO were clients who accessed HIV testing at the Provincial STBBI Clinic at the BCCDC, a public health clinic in Vancouver, British Columbia, that delivers approximately 13 000 STBBI testing encounters per year. Both GCO and the Provincial STBBI Clinic are publicly funded, free of cost to the user and allow clients to test using a pseudonym (ie, no identification is required). The Provincial STBBI Clinic is staffed by specially trained certified STI practice registered nurses and physicians who routinely incorporate STBBI risk assessments and counselling into STBBI testing visits; in this regard, the Provincial Clinic offers more extensive pretest education than the average HIV testing encounter. Provincial Clinic staff were involved in the development of GCO educational materials—described above—thus ensuring that online educational content mirrored those messages typically delivered in-clinic.

Study design

Design and recruitment details for this study are described in a separate report published concurrently in this issue.⁵ A baseline questionnaire (completed +2 weeks after their test), described in our parallel publication,⁵ contained questions related to HIV testing knowledge (outcome 1) and sexual risk behaviour, in addition to questions on experiences of testing barriers (online supplementary text S1). In the present study, we also drew on a follow-up questionnaire, administered 3 months after their test to detect any change in condom use behaviour since the baseline (outcome 2; online supplementary text S1). While both GCO and Provincial Clinic clients may have received pretest information both before and on the day of testing, some GCO clients may have had a longer time lapse between receipt of pretest information and the date of testing because they may not have gone to the lab to provide specimens (step 4, described above) immediately after viewing the informed consent page and printing the requisition (step 3) (online supplementary figure S2). For the present study, the sample was restricted to HIV-negative participants who received an HIV test (90% of total survey respondents). All study participants provided informed consent before undergoing any study activities.

Measurement

In the absence of an existing HIV knowledge scale, we conducted a pilot study to develop a measure of knowledge related to key HIV testing-related concepts, based on existing clinical guidelines (online supplementary text S2).9 10 19–24 An HIV test knowledge score was calculated by summing the number of correct responses in the six-item questionnaire (outcome 1, baseline survey). In addition, the questionnaire included a behavioural measure as a proxy for HIV sexual risk: ‘Have you had anal or vaginal sex without condoms in the past 3 months?: yes/no’ (outcome 2). The condom use question was asked during the baseline and 3-month surveys to describe self-reported pretest and post-test sexual behaviour, respectively.

The following explanatory variables were included in analysis. The primary comparison of interest was testing modality, that is, receiving an HIV test via GCO or at the Provincial STBBI Clinic. We used directed acyclic graphs to identify the minimally sufficient set of background variables to control for confounding of the effects of testing modality on both outcomes of interest (HIV knowledge and changes in sexual risk behaviours), as shown in figure 1.²⁵ Pretest HIV test knowledge was not measured; therefore, we selected background covariates to serve as a proxy for this construct. Specifically, we hypothesised that gay, bisexual and other men who have sex with men (gbMSM) would be more likely to use GCO due to promotional activities directed at gbMSM during the study period, and that gbMSM would have higher levels of HIV test knowledge, given the high prevalence of HIV and testing rates among communities of gay and bisexual men.²⁶ gbMSM status, age and HIV testing history were conceived as common causes of both outcomes, and choice of testing modality. Level of education completed, language spoken at home (English vs other) and immigration history (immigrated to Canada in previous 10 years) were conceived as having potential causal effects on pretest knowledge about HIV testing.

Figure 1.

Conceptual model for analysis of HIV test knowledge and condom use by testing modality: online versus clinic.

Analysis

First, we compared characteristics of online and clinic-based testers using t-test and χ² test, as appropriate (p<0.05 statistically significant). Second, we used multiple regression models to evaluate the associations between testing modality and each of the outcomes. HIV test knowledge (outcome 1) at baseline was analysed as a continuous six-point variable. Change in condom use—that is, an increase in condom use, as would be expected in the context of effective HIV test counselling related to behaviour change (outcome 2)—was calculated as reporting condomless anal sex or condomless vaginal sex (CAS/CVS) at baseline (during the 3 months before the test) but not reporting CAS/CVS at follow-up (during the 3 months following the test). Linear regression was used to evaluate the relationship between testing modality and HIV test knowledge, before and after adjustment for background covariates, in a sample restricted to those who completed the HIV knowledge items. Poisson regression with robust variance estimation was used to evaluate the relationship between testing modality and change in condom use, before and after adjustment for background covariates, in a sample restricted to those who completed CAS/CVS questions at both time points. Zou’s method estimated the relative risk (RR) of increased condom use.²⁷ 95% CIs were considered statistically significant if they excluded 0 for outcome 1 (β), and if they excluded 1 for outcome 2 (RR). All analyses were conducted in R V.3.4.1; exploratory factor analysis (described above) was performed using the psych package, and modified Poisson regression was performed using the geepack package.

Results

A total of 68 eligible GCO clients and 284 eligible clinic clients completed the study. Online testers were older than clinic-based testers (mean: 36 years vs 33 years), more online testers were gbMSM (43% vs 17%) and fewer immigrated to Canada in the previous 10 years (4% vs 16%) (all comparisons statistically significant; table 1). (These differences are consistent with those reported in the parallel publication describing differences in testing-related barriers between online and in-clinic testers, though the exact numbers differ due to differences in the analytic sample—ie, present analyses are restricted to HIV-negative clients who received an HIV test.)

Table 1.

Characteristics of survey respondents by testing modality, n=352

Characteristic	Level	Online clients n=68	Clinic clients n=284	P values
Age: median, mean (range)		35, 36.2 (19–59)	30, 32.7 (19–75)	0.01
Gender identity	Man	43 (0.632)	136 (0.479)	0.03
	Woman	22 (0.324)	143 (0.504)	–
	Transgender*	2 (0.029)	2 (0.007)	–
	Genderqueer or non-binary	1 (0.015)	1 (0.004)	–
	No response	0	2 (0.007)	–
Man who has sex with men	Yes	29 (0.426)	47 (0.165)	< 0.001
Educational attainment	University degree	35 (0.515)	149 (0.525)	0.99
Immigration history	Immigrated to Canada in the last 10 years	3 (0.044)	44 (0.155)	0.03
Language spoken at home	English	61 (0.897)	260 (0.915)	0.81
HIV/STI testing history	First-time tester	4 (0.059)	41 (0.144)	0.09
Number of sex partners, last 3 months (at baseline): median, mean (range)		2, 4.1 (0–45)	2, 3.1 (0–40)	0.23

The table includes all survey respondents eligible for either analysis: HIV knowledge score (eligibility: HIV negative, received HIV test at last visit and completed HIV knowledge items) or condomless anal sex or condomless vaginal sex (CAS/CVS) (eligibility: completed CAS/CVS questions at +2 weeks and +3 months’ surveys). Continuous variables compared using t-test; categorical variables compared using χ² test of most prevalent category.

Bold values indicate statistical significant differences between testing modalities, P<0.05.

*Includes transgender respondents who also reported a binary gender identity.

In unadjusted analysis, online testers demonstrated higher HIV post-test knowledge at baseline (mean score: 4.65, 95% CI 4.36 to 4.93 vs 4.09, 95% CI 3.94 to 4.23) (table 2). This difference reduced and became non-significant in adjusted analysis. gbMSM and clients with a university degree, who have lived in Canada >10 years and speak English at home all had a significantly higher post-test HIV test knowledge following adjustment. Lower HIV test knowledge scores were observed for first-time testers compared with individuals previously tested, however this difference also reduced and became non-significant in adjusted analysis (data not shown).

Table 2.

HIV test knowledge score by testing modality and covariates, n=321

Variable	Level	n	HIV knowledge score (mean, 95% CI)	Unadjusted β (95% CI)	Adjusted β (95% CI)
Modality	Online	62	4.65 (4.36 to 4.93)	0.56 (0.23 to 0.90 )	0.29 (−0.05 to 0.63)
	Clinic	259	4.09 (3.94 to 4.23)	Referent	Referent
Age (continuous)			–	0.02 (0.01 to 0.03 )	0.01 (−0.01 to 0.02)
Man who has sex with men	Yes	70	4.67 (4.36 to 4.99)	0.62 (0.29 to 0.93 )	0.53 (0.20 to 0.85 )
	No	251	4.06 (3.92 to 4.20)	Referent	Referent
Educational attainment	University degree	170	4.31 (4.13 to 4.48)	0.24 (−0.03 to 0.51)	0.29 (0.03 to 0.55 )
	Less than university degree	151	4.07 (3.86 to 4.27)	Referent	Referent
Immigration history	Immigrated to Canada in the last 10 years	44	3.52 (3.15 to 3.90)	−0.78 (−1.16 to 0.40 )	−0.51 (−0.91 to 0.11 )
	Lived in Canada >10 years	277	4.30 (4.16 to 4.44)	Referent	Referent
Language spoken at home	English	291	4.26 (4.12 to 4.40)	0.73 (0.27 to 1.18 )	0.49 (0.03 to 0.95 )
	Other language	30	3.53 (3.08 to 3.99)	Referent	Referent
HIV/STI testing history	First-time tester	43	3.63 (3.21 to 4.04)	−0.65 (−1.04 to 0.26 )	−0.36 (−0.75 to 0.02)
	History of testing	278	4.28 (4.14 to 4.42)	Referent	Referent

Bold values indicate statistical significant differences between testing modalities, P<0.05.

At baseline, self-reported condomless sex in the past 3 months was high within both groups (71% of online testers and 69% of clinic testers). Condom use increased among clients using both modalities by the 3-month survey; 18% of online testers and 10% of clinic-based testers increased condom use (ie, reported CAS/CVS pretest but not post-test) (table 3). The unadjusted RR comparing the two modalities was 1.87 (95% CI 0.94 to 3.74). None of the hypothesised factors shown in figure 1 were associated with a statistically significant change in condom use.

Table 3.

Condomless anal or vaginal sex (CAS/CVS) by testing modality and covariates, n=271

Variable	Level	n	CAS/CVS 3 months pretest (proportion)	CAS/CVS 3 months post-test (proportion)	Increase in condom use* (proportion)	Unadjusted RR† (95% CI)	Adjusted RR† (95% CI)
Modality	Online	55	0.709	0.600	0.182	1.87 (0.94 to 3.74)	1.67 (0.80 to 3.47)
	Clinic	216	0.694	0.694	0.097	Referent	Referent
Age (continuous)			–	–	–	1.02 (0.99 to 1.06)	1.02 (0.99 to 1.05)
Man who has sex with men	Yes	59	0.627	0.525	0.169	1.71 (0.85 to 3.43)	1.36 (0.64 to 2.91)
	No	212	0.717	0.717	0.099	Referent	Referent
HIV/STI testing history	First-time tester	31	0.677	0.613	0.129	1.15 (0.43 to 3.06)	1.45 (0.54 to 3.88)
	History of testing	240	0.700	0.683	0.112	Referent	Referent

*Binary variable indicating report of CAS/CVS during 3 months pretest but no CAS/CVS during 3 months post-test.

†RR: change in condom use associated with covariate, estimated using Zou’s modified Poisson regression.

RR, relative risk.

Discussion

In this comparative study between online and clinic-based testers, we found no evidence of decreased HIV test knowledge or increased sexual risk behaviour following testing for HIV through GCO. In fact, online testers demonstrated significantly higher HIV test knowledge after their test, though this difference was removed after adjustment for potential confounding variables, suggesting that online HIV testing may initially have higher uptake among population subgroups who are already informed about the HIV testing process (ie, gbMSM, and those with higher educational attainment, who speak English as a first language, or who were born in Canada). To our knowledge, no other studies of online STBBI testing services have addressed this research question.

With the exception of age and testing history, our hypothesised background covariates were significantly associated with HIV knowledge score in adjusted analyses, further supporting the construct validity of our HIV knowledge measure. Knowledge scores were significantly higher among gbMSM, individuals with a university degree, English speakers and long-term residents of Canada. Conversely, non-English speakers, recent immigrants and individuals with lower educational attainment demonstrated lower knowledge of HIV test concepts, suggesting the need to specifically adapt and tailor information for these clients, as has also been identified in qualitative research related to GCO.^{7 8 28} Future development of GCO will include translation into other languages common in British Columbia, as well as considering alternative methods for conveying information, such as videos.

We did not observe any significant differences between pretest and post-test reports of condomless sex by testing modality, or by age, gbMSM status or testing history. Our findings here are in accordance with prior research where the effectiveness of behavioural counselling during HIV test visits in reducing an individual’s sexual risk of HIV acquisition or transmission has been demonstrated for HIV-positive clients but not for those who test negative.²⁹

We acknowledge the following limitations of our study. We were unable to assess pretest HIV knowledge due to the GCO and provincial STBBI service pathways (ie, consent to be contacted for research in the Provincial STBBI Clinic is collected at the time of the pretest encounter). Prior studies on the effectiveness of different pretest and post-test HIV counselling interventions on HIV knowledge are typically based on randomised controlled designs with assessment of both preintervention and postintervention knowledge. Individually randomised designs, however, are generally not appropriate for evaluations of public health digital interventions like GCO, which have already been implemented. While we have adjusted for variables hypothesised to affect pretest knowledge, there may be other unmeasured characteristics confounding our results relating to HIV knowledge, or that further explain the relatively high levels of test knowledge observed among online testers. However, our use of a comparative clinic which routinely offers comprehensive pretest counselling (the ‘gold standard’) is a strength of our study, considering our finding of higher knowledge among online testers. We also may not have had sufficient sample size to detect significant differences, particularly for testing history, which was not significantly associated with HIV knowledge in our final adjusted analysis. In a sensitivity analysis among first-time testers, there was no difference in post-test HIV test knowledge between online and clinic-based testers; however, the sample size was limited (n=4 first-time testers online; n=39 first-time testers in-clinic). That testing history was statistically associated with HIV knowledge scores in unadjusted analysis suggests this is a clinically important difference and that additional resources within the GCO service for first-time testers may be needed. Finally, while we examined two potential missed opportunities associated with online STBBI testing (ie, HIV testing-related education and counselling), others remain to be examined; for example, qualitative interviews with GCO users, published in a separate report in this issue, highlighted the identification of co-occurring mental health and substance use concerns (eg, anxiety, depression, addiction) as another potential missed opportunity in online versus in-person clinical STBBI testing encounters.³⁰

This study adds to a small but growing body of knowledge regarding the impacts and role of online STBBI testing services in comparison to clinical testing services; few comparative studies between online and clinic-based testers have been performed.⁵ Our findings suggest that with careful design and attention to educational content, online testing services in which individuals access HIV testing and receive negative HIV results online may not lead to missed opportunities for HIV education and counselling. In fact, online platforms offer an opportunity to standardise health education messages. Users of GCO may consistently receive specific messages—for example, those explaining window periods in relation to negative tests—compared with clients accessing testing in-clinic, where language, communication styles and timing may vary between providers or from visit to visit. The results of this study and others related to GCO^{4 7 15} underscore the importance of thorough consultation—with end users, clinicians and other sexual health stakeholders—and clinical service integration in order to implement online sexual health services that reduce barriers to testing while simultaneously delivering critical STBBI prevention messages at the time of testing.

Key messages.

• Internet-based HIV testing reduces barriers related to matters of privacy and convenience but may result in missed opportunities for education typically provided in clinical encounters.

• Among 352 adults completing HIV tests, we found no evidence of decreased HIV test knowledge or condom use among online testers, relative to clinic-based testers.

• Careful and deliberate design of online educational content can ensure that internet-based HIV and STI testing platforms adequately address clients’ HIV/STI prevention needs.