Accuracy of Un-supervised Versus Provider-Supervised Self-administered HIV Testing in Uganda: A Randomized Implementation Trial

Abstract

Unsupervised HIV self-testing (HST) has potential to increase knowledge of HIV status; however, its accuracy is unknown. To estimate the accuracy of unsupervised HST in field settings in Uganda, we performed a non-blinded, randomized controlled, non-inferiority trial of unsupervised compared with supervised HST among selected high HIV risk fisherfolk (22.1 % HIV Prevalence) in three fishing villages in Uganda between July and September 2013. The study enrolled 246 participants and randomized them in a 1:1 ratio to unsupervised HST or provider-supervised HST. In an intent-to-treat analysis, the HST sensitivity was 90 % in the unsupervised arm and 100 % among the provider-supervised, yielding a difference 0f −10 % (90 % CI −21, 1 %); non-inferiority was not shown. In a per protocol analysis, the difference in sensitivity was −5.6 % (90 % CI −14.4, 3.3 %) and did show non-inferiority. We conclude that unsupervised HST is feasible in rural Africa and may be non-inferior to provider-supervised HST.

Introduction

Knowledge of one’s HIV status is a critical step in the path toward HIV prevention and care. Despite the advances made in the field of HIV prevention and care, many people cannot access these services because they are unaware of their HIV serostatus. In sub-Saharan Africa, it has been an estimated that 36 % of people have never been tested for HIV [1]. Uganda is reported among the countries where lack of knowledge of HIV status is the limiting factor to getting people into prevention and care programs [1]. Traditionally, HIV counseling and testing (HCT) has been administered by health care providers in the clinic, home-based and mobile HCT outreaches. The effectiveness of these strategies has been hampered by barriers like the lack of privacy, stigma, disclosure issues, poor male involvement, as well as long distances to health facilities [2–4].

A more effective response to the global HIV epidemic necessitates alternative and multiple strategies to improve on knowledge of HIV serostatus. Novel and efficient approaches like oral HIV Self-Testing (HST) provides promising alternatives to clinic and provider-based HIV screening programs. Self-testing has the potential to reach more clients of previously unknown HIV status [5, 6]. This may be the case for fisherfolk in Uganda who have documented high HIV prevalence levels of 26.7–28.8 % [7–9].

Although oral self-testing has been promising in some settings, its effectiveness has not yet been established, especially in the setting of developing countries with high HIV prevalence. Studies conducted among health workers in Kenya [10] and an urban population in Malawi [5] found HST feasible, but the accuracy of HST among lay users remains a concern [11]. This concern is especially acute in field settings in rural populations where lay persons lack familiarity with medical devices [12]. Despite its potential, the accuracy of self-testing in un-supervised, field conditions in a high HIV risk population is unknown.

To estimate the accuracy of un-supervised, self-administered oral HIV self-testing in field settings, we performed a non-blinded, randomized controlled, non-inferiority trial among high-risk fisherfolk in three fishing communities in Uganda.

Methods

Study Participants

The study was conducted between July 10 and September 13, 2013 in three fishing communities around Lake Edward, western Uganda. Research assistants screened local residents who presented to the research camp for HIV testing. These individuals were eligible for the study if they were between 18 and 49 years, at high risk for acquiring HIV infection, and lived or worked in the study community for at least 3 months prior to enrollment. High risk for HIV infection was defined as sexually active clients with: a history of unprotected intercourse with one or more partners of unknown HIV sero-status within the past 3 months, new sex partners in the past 3 months, symptoms of sexually transmitted infections (STIs) in the same period, commercial sex activity, or being in a known HIV discordant partnership. Any one of the high risk scenarios in addition to other two eligibility criteria above was sufficient for study entry. All participating clients provided written, informed consent and received adequate pre and posttest HIV counseling and referral services.

Randomization

Eligible participants were randomly assigned to one of two testing groups (Fig. 1): unsupervised oral HST followed by rapid HIV testing OR provider-supervised oral HST followed by rapid HIV testing. Randomization was stratified by fishing village and done with a 1:1 ratio. The random allocation schedule was computer-generated using Stata^® version 11. The individualized assignment code sheets were placed in opaque, individualized, sealed envelopes, which were stacked in batches of random blocks with uneven block sizes not exceeding 8 assignments. At the time of enrollment, each participant would select a sealed envelope in consecutive order and open it to reveal the testing assignment. Because of the nature of HIV testing, study participants and study personnel could not be blinded to the intervention. However, steps were taken to reduce reporting bias by ensuring that the interviewers were unaware of the HIV testing results at the time of the exit interviews and by blinding the laboratory staff to the viral and molecular endpoints.

Fig. 1.

Enrollment and Randomization

Study Procedures

All enrolled participants received pre-test HIV counseling and a brief, 10 min demonstration of how to use the oral self HIV test kit. The 10-min demonstration familiarized participants with the contents of the test kit which included printed instructions in English indicating how the test is conducted. These instructions and illustrations were briefly re-read to them in the local language (Runyankore). In the provider-supervised oral HST arm, research staff (i.e. the provider) supervised the participant performing the oral HIV self-testing in the research clinic; once this test was completed and the client had recorded the result, a confirmatory HIV test was performed along with an exit interview. In the unsupervised HST arm, clients performed the oral HIV self-testing in private without supervision from the provider. Participants were asked to conduct the self-test at home (or in a convenient private location), develop and read the results guided by the illustrated instructions, and then return to the researcher within 12–72 h for confirmatory rapid HIV assay results (from HIV rapid tests on a blood sample collected at baseline) and an exit interview. Subjects who completed the unsupervised oral HST were asked to interpret and record their results as one of three outcomes: may have HIV/preliminary positive; don’t have HIV/preliminary negative; and test not working/invalid.

The oral HIV self-testing was done using the OraQuick^® lnHome Rapid HIV-1/2 Antibody Test (Orasure Technologies) self-test kit according to manufacturer’s instructions. Clinic-based HIV testing was done using a serial algorithm of rapid HIV assays that is standard of care and approved by the Ugandan Ministry of Health. The algorithm included Determine (Abbot Laboratories), STAT-PAK (Chembio Diagnostic Systems Inc) and Unigold (Trinity Biotech plc) as tie breaker [13]. For quality control, all HIV seropositives and 10 % of HIV seronegative samples were retested with Western Blot and HIV p24 ELISA at the MBN Molecular Laboratory in Kampala. Proper pre and posttest counseling was provided to all study volunteers so that they understood that ALL positive oral HIV test results will be subject to confirmation, both in the field and at the MBN molecular reference laboratory.

Research assistants were trained on the specific procedures included in this protocol. As part of assuring quality of HIV testing, we assessed inter-rater reliability of the three research assistants using known HIV seropositive and seronegative samples. The Cohens kappa statistic estimated to evaluate the reliability of test administration across the research assistants was high (κ = 0.989, p = 0.00).

The exit questionnaire was administered after all testing was done. It assessed some secondary study outcomes such as performance errors, requests for help, and linkage to care.

Study Outcomes

The primary outcome was the difference in accuracy of HST, comparing the unsupervised testing to the provider-supervised HIV testing. Accuracy in this analysis is defined as the sensitivity of the diagnostic test procedure. We estimated the sensitivity, specificity as well as positive and negative likelihood ratios, comparing the oral HIV self-test to the current Uganda National standard of HIV screening (rapid HIV testing in such field conditions).

Secondary outcomes included difference in accuracy among first time and repeat HIV testers; the observed and reported error rates; proportions who requested for extra help with the test beyond the standard pre-test demonstration; return for test result revalidation and exit interviews; and finally linkage to HIV prevention, treatment and care services. We determined the above secondary outcomes as our measures of implementation effectiveness of oral self HIV testing compared to the standard of care (rapid provider administered testing) conducted in field conditions of Uganda.

Statistical Analysis

We used a one-sided design to test non-inferiority between groups, specifically to test the hypothesis that the accuracy of unsupervised self-administered HIV testing is objectively non-inferior to that of provider administered testing in field settings of Uganda. To reduce variability between subjects, conditional tests were employed to evaluate the differences in accuracy between the new and standard diagnostic procedures since tests conducted on the same subjects are correlated.

The sample size was calculated to detect non-inferiority at 95 % accuracy of self-testing in the provider supervised HIV self-testing arm with a non-inferiority margin of −15 %, a 10 % significance, and 95 % power. A total sample of 220 participants was calculated, although we enrolled an adjusted sample of 246 participants giving us >95 % power to detect non-inferiority, defined as a difference in sensitivity of −15 % using a one-sided Wald asymptotic test. The non-inferiority limit was conservatively set at −15 %, guided by summarized findings of the few studies that examined accuracy of self-tests among lay users in field settings [11]. Data were entered (after consistency checks and cleaning) into a Microsoft Access (version 2010) database using Epi-Info™ 7, then exported to Stata^® (version 11) and SAS^® (version 9.3) for analyses.

The primary analysis was by intention to treat (ITT), including all participants in their randomized group. We assumed that the clients who had oral HST and didn’t return for re-validation of their results were HIV negative. For the analysis of the primary study outcome, we compared the differences in the sensitivity and specificity within individuals between the two arms. Sensitivity was defined as the conditional probability that the oral HST was positive given that the standard rapid HIV test algorithm was also positive. Specificity was defined as the conditional probability that the oral HST was negative given that by the standard rapid HIV test algorithm was also negative. A one sided Wald asymptotic test was used to assess for non-inferiority. The confidence interval for the difference was based on the Wald asymptotic method, at an alpha level of 0.05 corresponding to 90 % confidence limits. Because six participants from the self-administered arm failed to return to report results and complete an exit interview, we performed a per-protocol analysis in which the data from these participants were removed from the analysis.

No compensation or incentives were provided for participating in this study. This trial was approved and registered by the University of Georgia IRB, the Uganda National AIDS/HIV Research Ethics Committee (NARC) and the Uganda National Council of Science and Technology (UNCST) respectively, ID number HS 1409.

Results

Of 329 screened participants, 246 (74.5 %) were enrolled, with 83 ineligible on basis of consent, age, and low risk for HIV (Fig. 1). The two study groups were similar in terms of age, sex, marital status, education, monthly income, HIV transmission risk, and previous testing for HIV (Table 1). The study population was predominantly male, currently married or married in the past, and had less than primary school education. Less than half of the participants had tested for HIV in the past 12 months, but nearly 80 % had tested at some point in the past. The group had evidence of high risk behaviors. Most participants did not have a regular sex partner and did not use condoms. Over half had signs of STI within the past 3 months. Most participants perceived themselves at risk for HIV infection with 57 (46.3 %) reported high-risk, 32 (26.0 %) medium risk, 22 (17.9 %) low risk, and 12 (9.8 %) perceived themselves at no risk for HIV acquisition at all. There were no differences in HIV risk perception across study arms (p = 0.276) (Data not shown). Overall, 33 individuals (13.4 %) tested seropositive using the rapid HIV test algorithm (Table 2).

Table 1.

Baseline characteristics of study participants by study arm

Participant Characteristics	Unsupervised oral self HIV test N = 123	Provider supervised oral self HIV test N = 123	p value	Total N (%)
Sex, n (%)
Male	64 (52.1)	77 (62.6)	0.1217	141 (57.3)
Female	59 (47.9)	46 (37.4)		105 (42.7)
Age (median IQR)	28 (23–32)	27 (22–32)	0.2865	28 (23–32)
Marital status, n (%)
Never married	20 (16.3)	28 (22.8)	0.1322	48 (19.5)
Currently married	63 (51.2)	68 (55.3)		131 (53.3)
Ever married	40 (32.5)	27 (21.9)		67 (27.2)
Education, n (%)
No Education or Lower	78 (63.4)	63 (51.2)	0.1142
Primary	15 (12.2)	24 (19.5)		141 (57.3)
Primary complete	17 (13.8)	26 (21.1)		39 (15.9)
Lower secondary	13 (10.6)	10 (8.1)		43 (17.5)
Post-secondary				23 (9.4)
Monthly Income (USDa, Median IQR)	23.6 (7.9–59.0)	35.4 (11.8–59.0)	0.2362	31.5 (17.9–59.0)
HIV transmission risk factor
Non regular sex partner (n,%)
None	74 (60.7)	79 (64.2)	0.4209	53 (62.4)
1	38 (31.2)	30 (24.4)		68 (27.8)
>1	10 (8.2)	14 (11.4)		24 (9.8)
Condom use with non-regular sex partners (n/N, %)
0	30 (69.8)	34 (84.9)	0.3542	64 (76.2)
≥1	13 (30.2)	7 (17.1)		20 (23.8)
Sexually transmitted infection in past 3 months
Present	68 (55.3)	70 (56.9)	0.792	138 (56.1)
Absent	55 (44.7)	53 (43.1)		108 (43.9)
Circumcision status (men only, N = 141; no/total number, %)
Circumcised	36/65 (55.4)	49/76 (64.5)	0.2276	85/141 (60.3)
Circumcised	29/65 (44.6)	27/76 (35.5)		56/141 (39.7)
HIV test in past 12 months (n, %)
Yes	58 (47.9)	49 (39.8)	0.2025	107 (43.9)
No	63 (52.1)	74 (60.2)		137 (56.2)
Lifetime history of HIV testing
Ever	97 (78.9)	96 (78.1)	0.877	193 (78.5)
Never	26 (21.1)	27 (21.9)		53 (21.5)
HIV test preference of rapid test (n,%)
Finger prick sample	18 (14.6)	16 (13.0)	0.729b	34 (13.8)
Oral (mouth) sample	63 (51.2)	72 (58.5)		135 (54.9)
Blood sample (venipuncture)	39 (31.7)	33 (26.8)		72 (29.3)
Urine sample	3 (2.4)	2 (1.6)		5 (2.0)
Would you buy oral-self HIV test kit if locally available?
Definitely yes	84 (68.3)	95 (77.2)	0.466b	179 (72.8)
Probably yes	30 (24.4	21 (17.1)		51 (20.7)
Maybe	5 (4.1)	2 (1.6)		7 (2.9)
Probably not	3 (2.4)	3 (2.4)		6 (2.4)
Definitely not	1 (0.8)	2 (1.6)		3 (1.2)
At what cost would you find it affordable?
USD (median, IQR)	1.9 (0.8–1.9)	1.9 (1.2–3.9)	0.8562	1.9 (0.8–3.9)

^a1USD ($) is equivalent to 2,543 Uganda Shillings (UGX), September 2013

^bFishers exact 2-sided test

Table 2.

Diagnostic accuracy of oral self-HIV testing (rapid HIV testing as gold standard)

Participant characteristics	Unsupervised oral self HIV test N = 123	Provider supervised oral self HIV test N = 123	p value	Total N (%)
HIV test results
Oral self HIV testa (n = 240)
Positive	18 (15.4)	13 (10.6)	0.2945b	31 (12.9)
Negative	98 (83.8)	109 (88.6)	0.2615	207 (86.3)
Indeterminate	1 (0.8)	1 (0.8)		2 (0.8)
Rapid HIV testing
Positive	20 (16.3)	13 (10.6)		33 (13.4)
Negative	103 (83.7)	110 (89.4)		213 (86.6)
Indeterminate	–	–		–

Sensitivity: (31/33)100 = 93.9 % (78.4–98.9), specificity: (207/213)100 = 97.2 % (93.9–98.9), positive likelihood ratio: 100 (25.1–398.5), negative likelihood ratio: 0.06 (0.02–0.23)

^aSix participants had no results

^bFishers exact two-sided test

All 246 HIV tests conducted were included in the primary ITT analysis (Fig. 2). Overall, using the oral HIV self-test, 18 participants (15.4 %) reported testing positive for HIV infection in the self-administered arm whereas 13 tested positive (10.6 %) in the provider-supervised arm. Using the rapid HIV test algorithm, 20 participants (16.3 %) tested positive for HIV infection in the self-administered arm whereas 13 tested positive (10.6) in the provider-supervised arm. The sensitivity and specificity of the oral HIV self-test was 90 and 95.2 %, respectively, in the self-administered arm (Fig. 2), whereas the sensitivity and specificity were 100 and 99.1 %, respectively, in the provider supervised arm. The prevalence of HIV infection did not differ by study arm (p = 0.26).

Fig. 2.

Implementation effectiveness measures of HIV self-testing in Uganda, 2013, by study arm, using a serial rapid HIV algorithm as the gold standard

The absolute difference in sensitivity was 10 % between the two study arms. The null hypothesis that the difference in accuracy between arms was greater than or equal to the lower equivalence margin was rejected (p = 0.0001); however, the lower one sided 90 % confidence limit fell below the lower equivalence boundary of −15 % (Fig. 2). Therefore, we were unable to show non–inferiority of unsupervised HST when using the oral HIV test kit. During the study, six participants failed to return to report self-test results; based on the confirmatory test sample, one of these six was HIV seropositive. In a per protocol analysis that excludes these six participants, the difference in sensitivity between the two arms was 5.6 %, lower than the 10 % observed in the ITT analysis. Moreover, the lower limit of the 90 % confidence interval was −14 %, indicating that the unsupervised oral HST was not inferior to the provider administered oral test.

Accuracy among the 53 first time testers was high (100 %) and not different across study arms (Table 3). Participants in the unsupervised arm were less likely to return for test revalidation and exit interview (p = 0.038). No errors were reported on exit interview for self-testers, however errors were observed almost one fifth of participants in the provider-supervised arm (24 participants, 19.5 %). The errors observed most often were incorrect swabbing of the upper and lower gums for a suitable mucosal exudate sample (12, 9.8 %), touching the flat pad (6, 4.9 %), and spills of developer fluid (5, 4.1 %). Overall, most participants (181, 75.4 %) reported that the oral HST was very easy to conduct, and strongly agreed that they would recommend it to a friend or family member (159, 66.5 %). However, among unsupervised testers, at least 29 (23.6 %) found some form of additional help necessary, mostly with using a timer (15, 12.2 %). Among the provider-supervised arm, a higher number of participants (51, 41.5 %) requested some form of help when testing. The majority in this arm (27, 21.9 %) were also unable to use a timer. Differences in request for help were not statistically significant across arms. All participants found HIV to be seropositive were referred for care and treatment. However, only 25 out of the 33 (75.7 %) HIV positives identified were able to provide samples for CD4 test to facilitate quick uptake into care programs.

Table 3.

Implementation effectiveness measures for self HIV Testing in Uganda

Secondary outcomes	Unsupervised oral self HIV test N = 123	Provider supervised oral self HIV test N = 123	Proportion difference (%) (Exact, 90 % CI)	Total (overall)	p value
Sensitivity (n/N,%)
First time testers	6/6 (100)	7/7 (100)	0	53 (21.5)	–
Repeat testers (>12 months)	12/13 (92.3)	6/6 (100)	−7.7 (−37.7–38.8)	193 (78.5)	0.6635
Return rate (test validation and exit interview) (N, %)	117 (95.1)	123 (100)	−4.9 (−10.5–0.6)	240 (95.8)	0.0382
Help requesteda (N, %)	29 (23.6)	51(41.5)	−17.9 (−29.7–5.5)	240 (95.8)	0.1708
Observed performance error rate (N, %)	(N/A)	24 (19.5)	–	123 (50)	–
Linkage to Careb
CD4 cell counts (N; median, IQR)
Total (median, IQR)	N = 16; 418 (127–672)	N = 9; 551 (387–822)	–	N = 25; 452 (348–810)	0.1374
Among clients CD4 ≤500	N = 10; 251 (68–410)	N = 4; 369 (328–439)		N = 14; 349 (101–410)	0.0596

^aAt least some form of help found necessary or help actually requested.

^bAll 33 HIV positives referred for HIV care and treatment. CD4 Samples derived from 25 consenting participants

Discussion

In this un-blinded, randomized, non-inferiority trial of unsupervised self-administered oral HIV testing, we found that the sensitivity of unsupervised oral HST was satisfactory, 90 %, but we could not demonstrate non-inferiority when compared to provider-supervised oral HST in a conservative intention-to-treat analysis. The absolute difference in test sensitivity between the two arms was large (10 %) and the lower boundary of the 90 % confidence limit fell below the pre-stated −15 % non-inferiority margin. In the per protocol analysis, however, the difference in test sensitivity was reduced (5.6 %), and we were able to demonstrate non-inferiority.

Although these findings do not provide convincing evidence that individuals from an African rural setting can accurately test themselves without the supervision of a health care provider, the preponderance of evidence from this study supports the use of unsupervised HST. In both the conservative ITT and per protocol analysis, the point estimate of difference in test sensitivity between the two study arms fell within the pre-specified boundary of non-inferiority. The wide confidence intervals in the ITT analysis suggest that a larger sample size would have provided greater precision to the estimate and allowed us to conclude non-inferiority. The test sensitivity with unsupervised HST was very good, identifying 90 % or more HIV-infected persons. Finally, our findings were close to the 91.7 % sensitivity claimed by the oral test kit manufacturers [14].

Although our finding is promising, the observed sensitivity of oral HIV testing falls below what has been previously reported. In an urban community setting in Malawi, the sensitivity of the test was 99.2 % [5]. In a clinical setting in Singapore, the sensitivity was estimated to be 97.4 % [15]. Previous studies that have assessed accuracy of oral HIV testing in comparison to blood based tests have shown that oral tests perform very well with accuracy close to 100 % [16–19]. Since we found that the sensitivity of HST improved to 100 % when clients were supervised, we believe that higher accuracy can be achieved once individuals are sufficiently trained. During the conduct of this trial, no adequate “training” was provided to participants on how to use the self-test kit beyond the 10 min demonstration provided by the kit package insert, re-read in the participant’s local language. Supervised self-testing improved on the confidence of the self-testers; however we realize this may not be feasible or practical every time an HIV self-test kit is used in real life.

Two other findings may affect the widespread use of unsupervised oral HST. First, we observed a high error rate when participants performed the oral test in the clinic. If we assume that these same errors are occurring when self-testing is performed unsupervised, then these errors may have affected our estimate of sensitivity. Again, greater attention to training before testing may be needed to optimize the use of the oral kits. Second, we observed that 5 % of our participants in the unsupervised test arm did not return to report the test result or complete the exit interview. Of concern, one of these individuals was HIV seropositive. Because of the confidential manner in which the tests were performed, the study health providers were not able to tract this individual. Further education and counseling may be needed to motivate individuals who self-test outside of the clinic setting to return for appropriate counseling and care. Overall, most participants who tested HIV seropositive were linked to care, thereby indicating the potential value of this type of rapid testing.

The strengths of this study lie in the randomization, which minimized potential confounding effects between testing groups. The study was set in a high prevalence, rural area of Africa where many individuals with undiagnosed HIV infection reside. It provides insights into the potential value of oral HIV self-testing and provides guidance on how to improve upon test performance in the field. Among the weaknesses of this study, differences in proportion may be affected by the underlying test response rate in the control group. In our study, this wasn’t an issue because we achieved 100 % response (all persons) in the control arm. Because sensitivity and specificity depend on the selected diagnostic thresholds of a test method, they have to be considered simultaneously as outcome indices [20]. We therefore report both sensitivity and specificity as measures of accuracy. The tests used in this trial will not detect acute HIV infection because they are antibody tests; so it is possible any participant with acute HIV infection would be missed, even with our confirmatory test. Our exclusion criteria eliminated certain people who had not had sex in the past 3 months, were older than 50, did not have symptomatic STIs, were unaware of or misinformed about their partner’s HIV sero-status, or may have been infected with HIV. Moreover, we focused on sexual transmission and did not assess transmission that may have been due to other mechanisms, like injection drug use. With these exclusions, the HIV prevalence that we measured in this study may be an overestimate of the HIV prevalence in the general population of these fisherfolks. We could not adequately measure acceptability as the HIV testing experience was high (few first time testers). Because of this we also did not objectively assess disclosure.

In conclusion, our study showed that unsupervised HST is feasible and can achieve high sensitivity and specificity for HIV infection, even in a rural setting among individuals at high risk for HIV infection. Although we did not show non-inferiority of unsupervised HST, we believe that improved counseling and training prior to unsupervised HST would enhance the test accuracy. Our findings provide evidence that unsupervised oral HST should be evaluated further as a way to improve access to HIV testing and open new paths to care in an African rural setting.