Weak Agreement between Antivirogram and PhenoSense Assays in Predicting Reduced Susceptibility to Antiretroviral Drugs

Qari and colleagues (3) in 2002 reported a comparative analysis of two commercial phenotypic assays (Antivirogram and PhenoSense) for human immunodeficiency virus type 1 (HIV-1) drug resistance testing. The Antivirogram (1) and PhenoSense (2) assays are both recombinant virus-based assays to measure the drug susceptibilities of patient-derived HIV protease or reverse transcriptase sequences and were developed by Virco (Belgium) and ViroLogic, Inc. (South San Francisco, Calif.), respectively. In the paper, the authors collected viruses from 50 clinical specimens and measured their susceptibility to 12 to 15 antiretroviral drugs using both assays. Based on 529 pairs of observations, they reported a 91.5% concordance by binary classification of sensitive versus reduced susceptibility. They concluded that “the Antivirogram and PhenoSense assays correlate well, despite the use of different testing strategies.”

Because of the lack of quantitative data in their paper on the actual correlation coefficients for the two phenotypic assays, we downloaded resistance estimates (increases [n-fold] in 50% inhibitory concentrations relative to that for the wild type) for the samples used by Qari et al. (3) from the Stanford HIV drug resistance database (4). Despite the high binary concordance obtained with the samples that they tested, correlations between the resistance values were surprisingly poor for many of the drugs (Table 1). For some drugs, such as delavirdine (DLV), the two assays had high Pearson correlation coefficients, but these high correlations were due to a small number of highly resistant samples. We have focused, therefore, on Spearman rank correlation coefficients (r_s) since they are less sensitive to extreme values. For the 14 drugs, r_s values ranged from 0.018 for saquinavir (SQV) to 0.551 for lamivudine (3TC), with a median of 0.290. Spearman's rank correlation test is a statistical test in which the null hypothesis is that two variables are uncorrelated. Performing a separate test for each drug, we failed to reject this null hypothesis for eight drugs using a significance value of 0.05 (Table 1).

TABLE 1.

Correlation between Antivirogram and PhenoSense phenotypic assays for 14 antiretroviral drugs

Drug classa	Drugb	No. of observations in Stanford databasec	Correlation coefficient	Rank correlation coefficientd
PI	APV	19	0.336	0.332
	IDV	37	0.064	0.040
	NFV	37	0.541	0.448*
	RTV	37	0.285	0.173
	SQV	36	0.143	0.018
NNRTI	DLV	35	0.897	0.267
	EFV	37	0.801	0.312*
	NVP	36	0.651	0.347*
NRTI	3TC	33	0.964	0.551*
	ABC	33	0.419	0.528*
	AZT	32	0.386	0.237
	d4T	33	0.235	0.109
	ddC	33	0.431	0.317*
	ddI	32	0.086	0.062

^aPI, protease inhibitor; NNRTI, nonnucleoside reverse transcriptase inhibitor; NRTI, nucleoside reverse transcriptase inhibitor.

^bAPV, amprenavir; IDV, indinavir; NFV, nelfinavir; RTV, ritonavir; SQV, saquinavir; DLV, delavirdine; EFV, efavirenz; NVP, nevirapine; 3TC, lamivudine; ABC, abacavir; AZT, zidovudine; d4T, stavudine; ddC, zalcitabine; ddI, didanosine.

^cFor some drugs, a small fraction of data in the original paper is not available in the Stanford database. No drug resistance data for adefovir is available in the Stanford database.

^d*, P < 0.05 for the Spearman's rank correlation test.

Furthermore, from Table 1 of the original paper, we noticed that only a small fraction of the samples were classified as having “reduced susceptibility” by both assays. We argue that because of the high fraction of drug-susceptible viruses in their study, their overall 91.5% concordance result is somewhat misleading. (Given frequencies of susceptible viruses of 89.6 and 91.7% for the Antivirogram and PhenoSense assays, respectively, one would expect 83.0% concordance by chance). In clinical settings it is most important to have high prediction accuracy when the virus has reduced susceptibility to drugs. This argues for a closer examination of samples predicted to have reduced susceptibility on the basis of Qari et al.'s cutoff values (3). Of the 44 samples that are classified as having reduced susceptibility by the Antivirogram assay, 17 are reported as sensitive by the PhenoSense assay, resulting in a concordance of only 61.4% (95% confidence interval [CI], 45.5 to 75.6%). Of the 55 samples that are classified as having reduced susceptibility by the PhenoSense assay, 28 are reported as sensitive by the Antivirogram assay, resulting in a concordance of only 49.1% (95% CI, 35.4 to 62.9%).

We conclude that resistance estimates from the Antivirogram and PhenoSense assays do not correlate well for the data shown by Qari et al. (3). Although these assays may correlate better for data sets that include more resistant viruses, in the absence of data showing that one test is better than the other, we suggest that classifications of reduced susceptibilities resulting from these phenotypic assays be interpreted with caution.