Pooled Nucleic Acid Testing to Detect Antiretroviral Treatment Failure in Mexico

Abstract

Background

Similar to other resource-limited settings, cost restricts availability of viral load monitoring for most patients receiving antiretroviral therapy in Tijuana, Mexico. We evaluated if a pooling method could improve efficiency and reduce costs while maintaining accuracy.

Methods

We evaluated 700 patient blood plasma specimens at a reference laboratory in Tijuana for detectable viremia, individually and in 10 × 10 matrix pools. Thresholds for virologic failure were set at ≥500, ≥1000 and ≥1500 HIV RNA copies per milliliter. Detectable pools were deconvoluted using pre-set algorithms. Accuracy and efficiency of the pooling method were compared with individual testing. Quality assurance (QA) measures were evaluated after 1 matrix demonstrated low efficiency relative to individual testing.

Results

Twenty-two percent of the cohort had detectable HIV RNA (≥50 copies/mL). Pooling methods saved approximately one third of viral load assays over individual testing, while maintaining negative predictive values of >90% to detect samples with virologic failure (≥50 copies/mL). One matrix with low relative efficiency would have been detected earlier using the developed QA measures, but its exclusion would have only increased relative efficiency from 39% to 42%. These methods would have saved between $13,223 and $14,308 for monitoring this cohort.

Conclusions

Despite limited clinical data, high prevalence of detectable viral loads and a contaminated matrix, pooling greatly improved efficiency of virologic monitoring while maintaining accuracy. By improving cost-effectiveness, these methods could provide sustainability of virologic monitoring in resource-limited settings, and incorporation of developed QA measures will most likely maximize pooling efficiency in future uses.

INTRODUCTION

When available, routine viral load monitoring is recommended to monitor for failure of antiretroviral therapy (ART)^1–3 and to prevent the development and transmission of drug-resistant HIV.⁴ In most resource-limited settings, viral load monitoring is either not performed or not recommended because of prohibitively high costs.^4–8 Instead, detailed historical, hematological, immunological, and clinical monitoring is used to monitor for ART failure,^5,9,10 but multiple studies have shown that these methods are ineffective.^4,8,9,11 To overcome these obstacles and improve quality of HIV care, more cost-effective approaches to monitor for virologic failure are urgently needed.

Although Mexico has a relatively low adult HIV prevalence of 0.3%,¹² there is considerable regional variability in HIV/AIDS incidence.¹³ The state of Baja California has the highest incidence of AIDS among Mexico’s 32 states, second only to the Distrito Federal (Mexico City),¹⁴ and the third highest AIDS-related mortality rate at 8.9 deaths per 100,000 inhabitants.¹⁵ Tijuana is the largest city in Baja California and has a concentrated epidemic with an estimated HIV prevalence of 0.8% among residents between the ages of 15 and 49 years.¹³ Similar to other resource-limited settings, routine viral load monitoring is prohibitively expensive in Baja California, and currently, only 1 laboratory in the entire state, Laboratorio de Soluciones Genéticas (LSG), has viral load testing capabilities. Therefore, viral load monitoring is not readily available to the 1100 patients who are estimated to be receiving ART in Baja California (Don Diego Guerena, personal communication with LSG, 2009).

Recently, quantitative pooled nucleic acid testing methods have been used in a clinical cohort in the United States to determine proof-of-principle of effectiveness and cost savings for virologic monitoring during ART.¹⁶ These methods were found to be more than 40% more efficient than individual viral load testing while maintaining excellent accuracy in a population with a prevalence of ART failure of 23%.¹⁶ These methods, however, have not been evaluated for this purpose in a resource-limited setting, where they could potentially lower costs to make routine virologic monitoring more available to patients. Therefore, we determined the efficiency and accuracy of the 10 × 10 matrix platform method in detecting the presence of HIV RNA in patient blood plasma at a reference laboratory in Tijuana, México.

METHODS

Study Population

This study was approved by the University of California, San Diego Human Research Protection Program and was conducted at LSG, a reference laboratory in Tijuana, Baja California, México. Blood plasma specimens that were previously tested by individual viral load assays and were being stored for quality control and internal validation purposes were used in the present study.

Nucleic Acid Testing Assay

All nucleic acid testing was performed using a locally developed one-step real-time quantitative reverse transcription polymerase chain reaction (qRT-PCR) assay using primers and probes based on the 5′ long-terminal repeat region of the HIV-1 genome and Ambion AgPath-ID reagents (Applied Biosystems, Inc, Foster City, CA). Quantification was performed using 7300 Sequence Detection System software (Applied Biosystems, Inc., Foster City, CA). This assay is approved by Mexican National Laboratories and presented the opportunity to evaluate pooling methods under different assay conditions, since previous methods used the ultrasensitive Amplicor HIV-1 Monitor assay (Roche Molecular Diagnostic Systems, Pleasanton, CA).¹⁶ HIV-1 RNA was extracted from pooled blood plasma specimens using the QIAamp Viral RNA Purification Spin Protocol (Qiagen, Valencia, CA).

To validate the local qRT-PCR assay, 36 viral load assays were performed using standard solutions provided by the Centers for Disease Control and Prevention. The lower limit of detection was determined by observing the percentage of positive replicates using the local assay to screen standard solutions with viral loads ranging from 25 to 1,250,000 HIV RNA copies per milliliter. The average viral load results from the local assay and variation coefficients using 4 of these standard solutions (500, 10,000, 100,000, and 1,000,000 copies/mL) were compared with those obtained using the COBAS Amplicor HIV-1 Version 1.5 Assay, a U.S. Food and Drug Administration–approved commercial assay (Roche Molecular Diagnostic Systems, Pleasanton, California, USA).

Based on simulated¹⁷ and clinical data,¹⁶ a 10 × 10 matrix screening method for the presence of HIV-1 RNA in blood plasma was compared with viral load testing of individual samples. Samples were selected in groups of 100 for inclusion in the matrix based on chronological order of acquisition at the reference laboratory. Samples were excluded from the analysis only if there was insufficient volume for inclusion in the pooling platform. Two laboratory technicians, who were blinded to the individual viral load testing results, constructed the pools and performed nucleic acid testing. Matrix pools were constructed as previously described.¹⁶ RNA extraction and qRT-PCR were performed on 600 microliters of each pool as described above. Algorithm thresholds for virologic failure of an individual sample were defined a priori as ≥500, ≥1000 and ≥1500 HIV RNA copies per milliliter, translating to thresholds of ≥50, ≥100, and ≥150 HIV RNA copies per milliliter to define a positive pool. These thresholds were chosen based on the dilutional effect of combining 10 individual samples in a pool and the assay lower limit of detection, as any threshold below 500 copies per milliliter would have been expected to significantly decrease efficiency. A search and test algorithm implemented in a web-based program (http://mepac.ucsd.edu) that incorporates individual and pooled viral load values into the resolution of positive pools was used, as previously described.¹⁶

Efficiency, Accuracy, and Cost Savings

Test characteristics, including relative efficiency compared with individual testing and accuracy, were determined for the matrix platform at each testing threshold. Relative efficiency was defined as the percentage of assays saved by pooling methods compared with individual testing. Accuracy was calculated as negative predictive values, and each algorithm threshold was evaluated for its ability to detect virologic failure at levels of >50, >100, >250, and >500 HIV RNA copies per milliliter. Cost savings were estimated based on the relative efficiency of the matrix platform at each threshold, and the cost of the single locally developed pRT-PCR viral load assay at LSG.

Evaluation for Contamination

Given the nature of sample pooling with the risk of contamination during PCR and of human error, contamination of pools can occur as has been reported previously with pooled nucleic acid testing.¹⁸ Procedures to screen for contamination that can impact relative efficiency and costs were therefore evaluated. Because the web-based calculator (http://mepac.ucsd.edu) was not initially programmed to screen for contamination, each matrix was deconvoluted assuming that no contamination was present when determining primary test characteristics. However, quality assurance (QA) measures designed to detect and resolve matrix contamination before and during deconvolution were implemented in the web-based calculator during the study period and are described in detail in the Supplementary Methods (Supplemental Digital Content 1, http://links.lww.com/QAI/A108).²⁵

Role of the Funding Source

The funders had no role in study design, in the collection, analysis, and interpretation of data, or in the writing of the report.

RESULTS

Validation of Locally Developed qRT-PCR Assay

Using the locally developed qRT-PCR assay, 35 of 36 (97%) replicates of the standardized plasma containing 50 HIV RNA copies per milliliter were detected. Using the standard solution of 40 copies per milliliter, the assay showed a 95% positivity rate (34 of 36 replicates). Using standard solutions of 500, 1000, 100,000 and 1,000,000 copies per milliliter, variation coefficients of 26%, 37%, 21%, and 8%, respectively, were obtained for the locally developed real-time qRT-PCR, compared with 48%, 31%, 27%, and 28%, respectively, for the COBAS Amplicor assay (Roche Molecular Diagnostic Systems, Pleasanton, CA). Based on the results of these validation experiments, the working lower limit of detection of the qRT-PCR assay was 50 HIV RNA copies per milliliter.

Cohort Characteristics

A total of 700 patient blood plasma samples collected in 2009 were included in the analysis. Although no clinical data were available, more than 95% of the samples were estimated by LSG staff to be from patients receiving ART. Although the aim of the pooling method is to screen for virologic failure of ART, the study design included all samples available at LSG tested in chronological order of acquisition to determine if the proposed method could be used at a reference laboratory, such as LSG, that receives samples from patients both on and off of ART. The absence of treatment data and adherence measures were expected to lower efficiency of pooling methods, as they could screen for those at highest risk of demonstrating detectable viremia and allow for exclusion of those patient samples from the pools. However, because most patients in the cohort were on treatment, pooling methods were still expected to demonstrate favorable efficiency compared with individual testing despite the lack of historical data at LSG.

Consistent with published reports of virologic failure rates during ART in resource-limited settings,^19–21 22% of the samples had detectable HIV RNA at ≥50 copies per milliliter on individual testing. However, because not all individual samples were collected during ART, the range of individual viral loads was quite large (136–2,500,000 copies per milliliter), with median viral load of 60,350 copies per milliliter and 20% of the cohort having a viral load of at least 1,500 copies per milliliter. Handling of a few samples with very large viral loads among samples with intermediate viral load may be problematic in the pooling methods because of the large assay variability associated with very large viral loads.

Test Characteristics of Matrix Method

The 700 patient samples were included in 1 of 7 10 × 10 matrices. Relative efficiency was determined by comparing the number of assays required to screen for detectable viremia with pooled testing to the total number of samples screened. All but one of the matrices demonstrated a positive relative efficiency, meaning that assays were saved by pooling compared with individual viral load testing. One matrix demonstrated relative efficiencies of −3% and −1% using lower (500 HIV RNA copies/mL) and higher (1000 and 1500 copies/mL) thresholds of virologic failure using the original version of the search and test algorithm in the web-based calculator.

Based on these results, we developed QA measures for identifying potentially contaminated matrices. These methods are described in detail in the Supplementary Methods (Supplemental Digital Content 1, http://links.lww.com/QAI/A108).

Briefly, the QA measures follow a stepwise process. First, contamination is suspected if a row pool and column pool are exceedingly high and not accounted for by the viral load of the individual sample at the intersection of the very high row and column pools. The second check determines if the row and column pool assay errors came from the same distribution. The third check is similar to the first check to identify an exceedingly high row or column pool viral load, but not both, and recommends reconstitution of the affected row or column. Given that these QA measures were implemented post hoc, we were unable to evaluate their exact effects on relative efficiency and accuracy but suspect that the contaminated matrix would have been detected very early during deconvolution before efficiency was significantly compromised (see Supplemental Digital Content 1, http://links.lww.com/QAI/A108).

The average relative efficiencies of all seven of the matrices were 33% and 36% using lower (500 copies/mL) and higher (1000 and 1500 copies/mL) thresholds to define virologic failure. In other words, using the higher cut-offs for virologic failure, 36% fewer assays were required to screen the cohort than were used for individual viral load testing. The sensitivities and negative predictive values of the 10 × 10 matrix method to detect individual viral loads of >50, >500, >1000, and >1500 copies per milliliter were all around 60% and 90%, respectively, at all algorithm thresholds used (Table 1). Since individual viral load results were used to resolve positive pools, the specificity and positive predictive values were all 100% at all thresholds to detect any level of viremia.

TABLE 1.

Test Characteristics of 10 × 10 Matrix Pooling Platform Compared With Individual Viral Load Testing

	Sensitivity (%)		Negative Predictive Value (%)
	All Matrices	Matrices 1–6	All Matrices	Matrices 1–6
VF = 500 copies per milliliter
>50 copies/mL	60%	55%	90%	88%
>500 copies/mL	61%	55%	90%	88%
>1,000 c/mL	62%	56%	90%	89%
>1,500 copies/mL	62%	56%	91%	89%
VF = 1500 copies/mL
>50 copies/mL	59%	53%	89%	88%
>500 copies/mL	59%	53%	89%	88%
>1,000 copies/mL	60%	54%	90%	89%
>1,500 copies/mL	61%	55%	90%	89%

VF, virologic failure cut-off point.

Because most individual specimens included in the contaminated matrix would have been retested, only 1 positive specimen would have been missed using the pooling procedure, resulting in individual sensitivities and negative predictive values of more than 95%. When the contaminated matrix was excluded, the average relative efficiencies increased to 42% with a slight decrease in overall sensitivity (around 55% for all thresholds, Table 1). Because individual retesting was not performed in real time, turnaround time could not be determined; however, 22% of the samples were deemed undetectable on the first day of testing since they fell in negative pools.

DISCUSSIONS

Pooled nucleic acid testing was shown to theoretically decrease the costs of virologic monitoring of patients on ART in a clinical cohort in the United States.¹⁶ In the present study, we show that these methods can be applied in resource-limited settings where viral load testing is currently not available to the majority of patients on ART. A 10 × 10 matrix pooling method was evaluated in a Mexican cohort of 700 patients, which is over 4 times the number of patients previously screened by these methods. Despite a relatively high prevalence of detectable viremia (≥50 HIV RNA copies/mL) of 22%, the pooling platform saved more than 33% of the viral load assays required to monitor patients individually.

Based on the estimated cost of US $57 per individual viral load assay at LSG, $13,223 to $14,308 could have potentially been saved by employing pooling methods compared with individual viral load testing. Some of these savings would be diminished by the cost of the extra technician time required for the constitution and deconvolution of matrices, and we were unable to determine what effect, if any, this had on the cost of pooled testing in the absence of a formal cost-effectiveness analysis. However, it is likely that the technician time saved by decreasing the number of viral load assays needed to screen the same number of patients for virologic failure would offset any additional costs of constituting and deconvoluting the pools, as in the United States study.¹⁶

There are several notable differences between our study in Tijuana and the San Diego cohort study. Our study was performed at a reference laboratory, where patients were referred for viral load testing by providers with no accompanying clinical data, whereas the San Diego cohort was a selected group of patients on first-line ART for at least 6 months.¹⁶ Although most of the patient samples included in this study are thought to be from patients on ART, some were most likely not, as indicated by relatively high individual viral loads in this cohort (18% had HIV RNA levels of at least 4 log₁₀ copies/mL). This likely lowered pooling efficiency by increasing the overall prevalence of detectable viremia and need for individual retesting, and indeed the relative efficiencies observed in our study were lower than those observed previously,¹⁶ although they are in agreement with simulations described by May et al,¹⁷ with relative efficiencies of around 30% for a prevalence of virologic failure of 22%. Additionally, the very high viral loads in some samples could have led to a decrease in efficiency in their respective pools.

Despite these limitations, the fact that pooling still showed considerable relative efficiency compared to individual viral load testing in this study indicates that these methods can be useful even in cohorts with higher prevalence of virologic failure and reference laboratories where patient treatment status may not be known. Although we were unable to determine at which point pooling efficiency would fall below 25% based on this single data set, pooling is unlikely to be useful once the prevalence of virologic failure reaches 25% based on the previous simulations.¹⁷ The incorporation of basic clinical information such as treatment status and the use of adherence measures^22,23 to exclude those patients at highest risk of virologic failure are likely to increase overall efficiency of pooling methods in future applications.

Further, the San Diego cohort study used the Roche Ultrasensitive Amplicor viral load assay, a commercial Food and Drug Administration–approved test,¹⁶ whereas our group used a locally developed real-time qRT-PCR assay that was validated against the COBAS Amplicor HIV-1 Version 1.5 Assay and approved by the Mexican National Laboratories. Therefore, we show that pooled nucleic acid testing can be applied to different viral load assays. By combining pooling methods with assays designed for specific geographic locations, both efficiency and sensitivity can be maximized in the local setting. The additional savings from the use of low-cost technologies combined with those from pooling could make viral load monitoring feasible in most resource-constrained settings.

Although pooled viral load testing results in a decrease in sensitivity due to the dilutional effect of combining multiple individual samples, we show that pooled testing combined with algorithms that incorporate the quantitative results of the viral load assay maintained excellent accuracy as measured by negative predictive value. Regardless of the threshold used to define virologic failure, the matrix pooling method demonstrated average negative predictive values of around 90% to predict levels of viremia as low as 50 copies per milliliter, which is well below the minimum amount of viremia required for most commercial genotypic assays (ie, drug resistance tests).²⁴

One issue that arose during this study was contamination, which can occur during the pooling, RNA extraction, or PCR steps, and will decrease the relative efficiency and cost savings and increase the turnaround time of results. After encountering a matrix with low relative efficiency during this study, QA measures were incorporated into the web-based calculator to detect and manage potential contamination before and during pool deconvolution. Because these algorithms were implemented post hoc and require either pool reconstitution or retesting of individual samples, their exact effects on relative efficiency and other test characteristics could not be determined in this retrospective study; however, we suspect that the contaminated matrix would have been detected very early in the deconvolution process before the relative efficiency was compromised had these measures been in place. Although further evaluation of these QA measures is needed in future studies, they are the first to address the important issue of contamination and will likely improve overall pooling efficiency and quality control in future uses.

In conclusion, we demonstrate that nucleic acid testing on pooled blood samples, using a search and test algorithm to resolve positive pools and screen for contamination, can reduce the cost of monitoring for virologic failure of ART in resource-limited settings by up to one third compared with individual viral load monitoring. We further confirm that these methods can maintain excellent accuracy even in populations with high levels of detectable viremia. By reducing the costs associated with virologic monitoring through pooling methods and the use of low-cost technologies such as locally developed assays, viral load monitoring may be made available to many people who are currently not receiving it. This could maximize first-line ART in areas where second-line options are limited and prevent prolonged virologic failure and the development of drug resistant HIV in areas of high disease burden.