Point-of-Care Cepheid Xpert HIV-1 Viral Load Test in Rural African Communities Is Feasible and Reliable

Abstract

Routine monitoring of HIV-1 RNA or viral load (VL) in patients on antiretroviral therapy (ART) is important, but there are multiple impediments to VL testing in resource-constrained settings. An accurate point-of-care (POC) HIV-1 VL test could alleviate many of these challenges. We compared the performance of the Cepheid Xpert HIV-1 VL assay against the laboratory-based Abbott m2000sp/m2000rt assay (Abbott assay). ART-naive individuals participating in the Botswana Combination Prevention Project in 20 communities provided EDTA-blood specimens during household surveys. Both the POC Xpert HIV-1 VL and Abbott assays were performed on specimens sampled from 277 individuals. We found a high correlation between the Xpert HIV-1 VL and Abbott assay results (r² = 0.92; P < 0.001). The overall mean difference in the HIV-1 RNA values obtained by Xpert HIV-1 VL assay and Abbott assay was 0.34 log₁₀ copies/ml (95% confidence interval [CI], 0.26 to 0.40 log₁₀ copies/ml) (P < 0.001). Using a clinically relevant level of 1,000 copies/ml as a threshold, agreement was 90.6% (95% CI, 87.9 to 93.1%), with a sensitivity of 98.6% (95% CI, 97.2 to 100%). The two methods agreed on their detectability of HIV-1 RNA (>40 copies/ml) at 97.1% (95% CI, 95.5 to 98.7%), with a sensitivity of 99.6% (95% CI, 97.2 to 100%). The POC Cepheid Xpert HIV-1 VL assay showed high agreement and accuracy with a laboratory-based method of HIV-1 RNA testing. The POC Xpert HIV-1 VL assay tended to overestimate HIV-1 VL, although the difference was below a clinically relevant threshold of 0.5 log₁₀ copies/ml. The POC Cepheid Xpert HIV-1 VL assay is a promising tool for monitoring patients on ART in southern Africa.

INTRODUCTION

Virologic monitoring of patients on antiretroviral therapy (ART) is a cornerstone of effective clinical care for the individuals, helping to monitor and support adherence, decrease drug resistance, and minimize costs and adverse clinical outcomes associated with unnecessary switching of antiretroviral (ARV) regimens. Botswana has successfully initiated antiretroviral therapy in over 87% of the citizens living with HIV (1). The magnitude of this scale-up necessitates routine HIV-1 RNA testing for monitoring treatment failure and adherence (2–5).

An increasing number of resource-poor countries are including viral load testing within their revised guidelines, a policy which conforms with the World Health Organization's recommendations for universal testing and treatment (6, 7). In resource-limited settings, especially in rural areas, laboratory-based HIV-1 RNA testing remains challenging for many reasons, among them insufficient access to laboratory facilities, cold-chain management shortcomings, and inability to consistently provide for sample transportation (8–10). Delays in the reporting of results may also negatively impact the HIV treatment cascade (11–13). Point-of-care (POC) viral load testing has the potential to alleviate these challenges, particularly in rural and remote communities.

Despite the development of several POC viral load technologies over the last decade (14–24), there is a shortage of commercially available tests. In this study, we evaluated the performance of the POC Cepheid Xpert HIV-1 viral load test in rural communities in Botswana in order to provide additional data to policy makers regarding the use of this assay in decentralized HIV treatment programs. The results obtained by the Xpert HIV-1 viral load assay were compared to those obtained by the laboratory-based Abbott m2000sp/m2000rt assay.

MATERIALS AND METHODS

Study participants.

This study was performed as a cross-sectional substudy within the Botswana Combination Prevention Project (BCPP; registration no. NCT01965470 at ClinicalTrials.gov). The overall design of BCPP is described elsewhere (1). Aggregate viral load data, pooled across arms, on study participants are presented here. A total of 302 ART-naive HIV-infected individuals residing in 20 rural communities in Botswana (and taking part in BCPP) provided written informed consent and took part in the study. The Botswana institutional review board (IRB) (Human Resource Development Council [HRDC]) and CDC IRB approved the BCPP study as well as this POC HIV-1 RNA assessment substudy.

POC testing of HIV-1 RNA.

Venous blood was collected in EDTA tubes in households (during the household survey) from September 2015 to May 2016. The collected blood was transported to a mobile clinic (portacamp) that was stationed in the community, and the blood was processed within 4 h of collection. The POC Xpert HIV-1 viral load testing was also performed in these mobile community-based clinics. One milliliter of plasma was used as input in the quantitative POC viral load test, the Cepheid Xpert HIV-1 viral load (catalog no. GXHIV-VL-CE-10), according to the manufacturer's instruction. The Xpert HIV-1 viral load is a test that automates the test process, including RNA extraction, purification, reverse transcription, and cDNA real-time quantification in a single cartridge within 90 min. The quantification range of the Xpert HIV-1 viral load test is 40 to 10,000,000 copies/ml (1.6 log₁₀ to 7.0 log₁₀).

Laboratory-based testing of HIV-1 RNA load.

The Abbott m2000sp/m2000rt assay (Abbott assay) was used as a comparator for evaluation of the Xpert HIV-1 viral load test. The Abbott assay was performed at the Botswana-Harvard AIDS Institute Partnership HIV Reference Laboratory (BHHRL) in Gaborone, Botswana, according to the manufacturer's instructions. The BHHRL was accredited by the South African National Accreditation System for HIV-1 viral load testing and maintains certification in Rush University's Virology Quality Assurance Program (25). The minimum level of HIV-1 RNA detection by Abbott assay is the same as that by POC Xpert HIV-1 viral load assay, 40 copies/ml (or 1.6 log₁₀ copies/ml).

Specimen flow.

A total of 302 participants were enrolled in this study. No data were generated for six samples due to quality control (QC)-related errors during POC testing. Three of these errors were related to internal quality control failure, and the other 3 were related to hardware error. A subset of 19 samples were tested by a single assay due to a limited specimen volume. A total of 277 specimens were tested by both the POC test and Abbott assay.

Statistical methods.

We assume that missing data (insufficient plasma volume and QC errors) were missing completely at random. Correlation analysis was performed using the Spearman rank test. To determine differences between two viral load assays, we used the Bland-Altman plot method. The concordance was considered good when the differences are within the limits of two standard deviations of the mean. Passing-Bablok regression was used to test for systematic and proportional differences using the “BlandAltmanLeh” and “mcr” packages in R 3.3.0 (26–28). Passing-Bablok regression is insensitive to outliers and assumes that measurement errors in both variables have the same distribution. For all analysis, 95% confidence intervals are computed using bootstrapping to account for clustering within communities. We used 1,000 replicates where communities were sampled with replacement. The clinically relevant difference between two viral load measurements was considered at 0.5 log₁₀ copies/ml, as described previously (29–31). We used the Cohen's kappa coefficient, a statistic which measures interrater agreement for qualitative values between two assays. It is considered to be a more robust measure than simple percent agreement calculation, since it takes into account the agreement occurring by chance. The kappa coefficient indicates a satisfying agreement between 0.6 and 0.8, and an excellent agreement above 0.8.

It should be noted that assay comparison implies a comparison of results obtained by POC Xpert HIV-1 VL assay performed in the mobile clinic to those obtained by the laboratory-based Abbott m2000sp/m2000rt assay, and it does not imply comparison of the assays. Each assay was performed in a natural setting, so there are substantial differences between the performance environments of the two assays (different conditions, methods, times, personnel, etc.).

RESULTS

Results from both the POC Xpert HIV-1 viral load test and the Abbott m2000sp/m2000rt assay were available for 277 participants. We observed a significant correlation between the HIV-1 RNA levels obtained by these two assays, using the nonparametric Spearman rank test (r² = 0.92; P < 0.001). With Pearson correlation, the results of the two assays also correlated highly linearly (r = 0.94; P < 0.001; Fig. 1). In Fig. 1, the estimated regression line (shown in blue) is compared to the diagonal dashed line (shown in red) that represents a hypothetically ideal match between two assays. Using Passing-Bablok regression, we observed significant systematic bias (intercept = 0.20; 95% confidence interval [CI], 0.05 to 0.32) and proportional bias (slope = 1.04; 95% CI, 1.02 to 1.07).

FIG 1.

Passing-Bablok regression plot comparing POC Cepheid Xpert HIV-1 viral load assay against the Abbott m2000sp/m2000rt assay.

The similarities between the two assays were evaluated by the Bland-Altman plot method (Fig. 2). The overall mean difference in the HIV-1 RNA values obtained by POC and Abbott assay was 0.34 log₁₀ copies/ml (95% CI, 0.16 to 0.51 log₁₀ copies/ml) (overall POC mean [standard deviation {SD}], 3.98 log₁₀ copies/ml [1.10 log₁₀ copies/ml] versus overall Abbott mean [SD], 3.64 log₁₀ copies/ml [1.05 log₁₀ copies/ml]). Although statistically significant (P < 0.001), the mean difference between the POC and Abbott assays was below the generally accepted clinically relevant difference of 0.5 log₁₀ copies/ml (29–31). A significant difference between the two assays (i.e., >2 standard deviations of the mean) was observed for 4.7% (n = 13) of the samples.

FIG 2.

Bland-Altman plot comparing POC Cepheid Xpert HIV-1 viral load assay against the Abbott m2000sp/m2000rt assay.

Using a threshold of 40 copies/ml (i.e., 1.6 log₁₀ copies/ml) for detectable HIV-1 RNA load, the two assays were in agreement at 97.1% (95% CI, 95.5 to 98.7%; Table 1), with a sensitivity of 99.6% (95% CI, 99.2 to 100%). Among the 7 individuals with detectable viral load using the POC Xpert HIV-1 viral load test (≥40 copies/ml) and undetectable viral load using the Abbott test (<40 copies/ml), the mean log₁₀ HIV-1 RNA load (Q1, Q3) was 1.94 (1.87, 2.01). This corresponded to a mean difference of 0.34 log₁₀ copies/ml with detectability threshold of 1.6 log₁₀, which is not clinically relevant. Only one individual had an HIV-1 RNA load higher than 0.5 log₁₀ of the detectability limit. The only individual with undetectable POC Xpert HIV-1 viral load test had an Abbott quantification of 1.82 log₁₀ HIV-1 RNA load.

TABLE 1.

Agreement on detection of HIV-1 RNA between POC Cepheid Xpert HIV-1 viral load test and the Abbott m2000sp/m2000rt assay at two detection levels, 1.6 log₁₀ copies/ml and 3.0 log₁₀ copies/ml

Detection threshold for Xpert HIV-1 VL assay	Abbott assay detectability by HIV-1 RNA level
	1.6 log10 copies/ml		3.0 log10 copies/ml
	No. undetectable	No. detectable	No. undetectable	No. detectable
Undetectable	16	1	206	23
Detectable	7	253	3	45
Total	23	254	209	68

Using a threshold of 1,000 copies/ml (i.e., 3.0 log₁₀ copies/ml) for clinical monitoring of ART, the assays were in agreement for 90.6% (95% CI, 87.9 to 93.1%; Table 1), with a sensitivity 98.6% (95% CI, 97.2 to 100%). Among the 23 individuals with discordant results and high viral load with the POC Xpert HIV-1 viral load assay (Table 1), the mean (Q1, Q3) difference between the two tests was 0.66 log₁₀ copies/ml (0.44, 0.66 log₁₀ copies/ml). This difference was higher than 0.5 log₁₀ in 14 individuals (61%). Among the 3 individuals with discordant results and lower viral load with the POC Xpert HIV-1 viral load assay, the differences were 0.46, 0.5, and 0.89 log₁₀ copies/ml. Further analysis showed that the agreement is higher than 80% at any threshold used.

To assess whether agreement remains constant or depends on any particular threshold level, we calculated the level of agreement between the two assays against a range of HIV-1 RNA load measurements from 1.6 to 6.0 log₁₀ copies/ml (Fig. 3A). The highest level of disagreement between the two assays occurred for HIV-1 RNA load between 3.5 and 4.5 log₁₀ copies/ml. However, the level of agreement was >80% across thresholds. Figure 3B shows the Cohen's kappa as a measure of agreement between POC and Abbott assays for different thresholds of the HIV RNA load. For all nonextreme HIV RNA load values, the Cohen's kappa indicates satisfying (above 0.6) or excellent (above 0.8) agreement between methods.

FIG 3.

Level of agreement between POC Cepheid Xpert HIV-1 viral load assay and the Abbott m2000sp/m2000rt assay according the viral load detection threshold. (A) Percentage of agreement between two assays. (B) Kappa statistics.

A sensitivity analysis was performed to further investigate differences of agreement between the two assays at a threshold of 3.0 log₁₀ copies/ml (1,000 copies/ml). For low HIV-1 RNA load (≤3.0 log₁₀ copies/ml), statistically significant systematic (intercept = −0.63; 95% CI, −0.76 to −0.49) and proportional biases (slope = 1.39; 95% CI, 1.31 to 1.48) were observed (Fig. 4A). For high HIV-1 RNA load (>3.0 log₁₀ copies/ml), significant systematic bias (intercept = 0.36; 95% CI, 0.04 to 0.62) but no proportional bias (slope = 1.0; 95% CI, 0.95 to 1.07) was found (Fig. 4B). Overall, the POC assay tended to overestimate HIV-1 RNA load compared to the Abbott assay. In one case, HIV-1 RNA load was determined to be undetectable by POC assay but was borderline detectable by Abbott assay at 66 copies/ml.

FIG 4.

Passing-Bablok regression for POC Cepheid Xpert HIV-1 viral load assay and the Abbott m2000sp/m2000rt assay for low (≤3.0 log₁₀ copies/ml) (A) and high (>3.0 log₁₀ copies/ml) (B) HIV-1 viral load.

DISCUSSION

We found that POC Cepheid Xpert HIV-1 viral load testing was feasible in rural communities (after home-based collection of venous blood), and that the Xpert HIV-1 viral load results obtained in this setting correlated with the Abbott assay with a high level of agreement across a broad range of HIV-1 RNA values. We observed a slight overestimation of viral load by the Xpert HIV-1 viral load assay, which is consistent with other studies (21–24).

It is important to note that underestimation of HIV-1 RNA at low levels with the Abbott m2000rt assay was reported previously (29, 32). For example, while the overall correlation between the Abbott RealTime assay and the Roche Amplicor assay was 0.90, it was only 0.45 at a low level of HIV-1 RNA (32). According to a meta-analysis of the performance of HIV-1 RNA technologies, there is a tendency of the Abbott RealTime HIV-1 assay to underestimate HIV-1 RNA in plasma (29). Assuming that the Abbott assay underestimates HIV-1 RNA at low levels, it is possible that POC Xpert HIV-1 viral load results are more accurate at the low end of HIV-1 RNA distribution.

The primary goal of this study was to evaluate the performance of the Xpert HIV-1 viral load assay across the broad range of HIV-1 RNA values. In addition, we assessed the performance of the Xpert HIV-1 VL at two binary cutoffs, 40 copies/ml and 1,000 copies/ml. While we observed a high level of agreement across the range of more than 6 log₁₀ copies/ml, some values around the assessed thresholds were misclassified in binary analysis. Most mismatches were within 0.5 log_10, which may be attributable to a natural variability associated with HIV-1 RNA measurements. Due to the small sample size around the assessed thresholds, the binary results obtained in this study should be taken cautiously. A dedicated study addressing the performance of the Xpert HIV-1 VL assay for the detection of HIV-1 RNA at the specified threshold among virologically suppressed individuals (e.g., individuals receiving ART) is warranted.

This study has some limitations. To preserve the integrity of data collected for an ongoing randomized trial, only limited aggregate information related to the characteristics of the study participants has been presented. To assess the performance of the Xpert HIV-1 VL test across a broad range of HIV-1 RNA values, we enrolled only ART-naive individuals. It might be necessary to evaluate the performance of the Xpert HIV-1 viral load assay among patients on ART in rural communities in southern Africa. This should assess the feasibility of the Xpert HIV-1 VL assay to be used for monitoring virologic suppression among individuals receiving ART. Finally, we used mobile clinics equipped with simple centrifuges for real-time processing of whole blood that was collected in households. However, this could be challenging in some settings where centrifugation facilities are not readily available. Finger-prick and whole-blood versions of the Xpert HIV-1 viral load assay are still under development. The current version of the Xpert HIV-1 viral load assay requires 1,000 μl of plasma, and 19 samples did not have adequate volume to test using both platforms. This could be challenging for pediatric blood draws.

The major implementation lesson learned during the study is that quantification of HIV-1 RNA in rural southern Africa communities is feasible. Major challenges were related to initial steps, such as the collection of venous blood and its processing in a timely manner. This study benefitted from the extensive field network developed and maintained by the BCPP study (1). Nevertheless, smooth performance of the Xpert HIV-1 VL assay required proper training. This was evident from errors which occurred primarily at the beginning of the study, which could be attributable, at least partially, to the operator's mistake. In particular, a strict requirement that the input plasma volume for Xpert HIV-1 VL must be 1,000 μl caused a few errors at the early stage of the study.

We conclude that Xpert HIV-1 viral load testing can be considered a feasible and reliable option for viral load monitoring, including in small rural communities in southern Africa. Same-day viral load could allow for immediate assessment of virologic failure, allow for triaging of patients for adherence counseling, and reduce loss to follow-up. The multiplexing functionality of the POC Cepheid Xpert assays could be an additional advantage for cost-effective diagnostics of other pathogens.

Funding Statement

This study was supported by the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) through the Centers for Disease Control and Prevention (CDC) under the terms of cooperative agreement U01 GH000447. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the CDC. The funders had no role in the study design, data collection, analysis, interpretation, writing, or submission of the manuscript for publication.