Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Jun 10.
Published in final edited form as: J Math Stat. 2019 Apr 3;15(1):55–64. doi: 10.3844/jmssp.2019.55.64

Comparing the Diagnostics Accuracy of CD4+ T-Lymphocyte Count and Percent as a Surrogate Markers of Pediatric HIV Disease

Musie Ghebremichael 1,*, Haben Michael 2, Jack Tubbs 3, Elijah Paintsil 4
PMCID: PMC6557447  NIHMSID: NIHMS1024366  PMID: 31186621

Abstract

The percentage CD4+ T-lymphocytes is used to monitor pediatric HIV disease. However, in resource-limited settings, enumerating the percentage of CD4+ T-lymphocytes is hampered by the lack of laboratory infrastructure and trained technicians. In this paper, we investigated the performances of the percentage and absolute CD4+ T-lymphocytes as markers of pediatric HIV disease progression using data from HIV-infected children enrolled through the Yale Prospective Longitudinal Pediatric Cohort study. A Lehmann family of Receiver Operating Characteristic (ROC) curves were used to estimate and compare the performance of the two biomarkers in monitoring pediatric HIV disease progression. The area under the ROC (AUC) curve and its empirical estimator have previously been used to assess the performance of biomarkers for a cross-sectional data. However, there is a paucity of literature on the AUC for correlated longitudinal biomarkers. Previous works on the estimation and inference of the AUC for longitudinal biomarkers have largely focused on independent biomarkers or failed to consider the effect of covariates. The Lehmann approach allowed us to estimate the AUC of the aforementioned correlated longitudinal biomarkers as functions of explanatory variables. We found that the overall performance of the two biomarkers was comparable. The area under the ROC curves for CD4+ T cell count and percentage were 0.681 [SE = 0.029; 95% CI: 0.624–0.737] and 0.678 [SE = 0.024; 95% CI:0.630–0.725], respectively. Our results suggest that absolute CD4+ T-lymphocyte counts could be used as a proxy for percentage of CD4+ T-lymphocytes in monitoring pediatric HIV in resource-limited settings.

Keywords: HIV, CD4+ T Cells, Receiver Operating Characteristics (ROC) Curve, Lehman Family of ROC Curves

Introduction

HIV infects CD4+ T cells and leads to cell death and subsequent depletion of CD4 cells. The progressive depletion of CD4+ T cells results in an increased risk for the development of opportunistic infections, Acquired Immune Deficiency Syndrome (AIDS) and death (Nishanian et al., 1998; Pantaleo et al., 1993; Vlahov et al., 1998). Antiretroviral therapy (ART) suppresses HIV viral replication and leads to maintenance or increase in CD4+ T cells, thereby restoring immune function. In resource-rich countries, the standard of care for monitoring ART is the routine laboratory monitoring of CD4+ T cell count and HIV viral load (Hammer et al., 2006; Mellors et al., 1997). In the US, CD4+ T cell count and HIV viral load are measured at the time of diagnosis and at least every 3–4 months on ART. These laboratory services and frequency of measurements are not routinely available in many resource-limited settings (Diagbouga et al., 2003). The WHO recommendation for ART monitoring in low and middle income countries has evolved from CD4+ T cell monitoring every six months and viral load testing only when the capacity exists to routine viral load monitoring being the preferred monitoring approach to diagnose and confirm treatment failure (WHO, 2010; 2015). There is a preponderance of evidence to suggest that viral load monitoring is more sensitive in detecting patients with ART failure than clinical or immunological (CD4+ T cells) monitoring (Hosseinipour et al., 2009; Rawizza et al., 2011; Reynolds et al., 2009). However, if viral load is not routinely available, the WHO recommends that CD4+ T count and clinical monitoring should be used to diagnose treatment failure (WHO, 2015). This later recommendation is based on the fact that routine laboratory monitoring, particularly viral load determination, is not feasible in most resource-limited countries due to cost, lack of technical expertise and lack of infrastructure (Taiwo and Murphy, 2008). Moreover, there are instances where patients on ART may have suppressed viral load without CD4+ T cell recovery. Hence, CD4+ T cell assessment is the most popular ART monitoring tool and will continue to play an important role in ART monitoring, particularly, where viral load monitoring is not available (Ford et al., 2015).

Due to the well-known large natural decline and variation in absolute CD4+ T cell numbers during early childhood (Stein et al., 1992; Wade and Ades, 1998), the percentage of these cells is used in children, particularly in those less than 5 years of age (Hughes et al., 1994). We previously reported that in a pediatric HIV cohort, the changes in percentage CD4+ T cells with treatment were similar to that of absolute CD4+ T cell count, regardless of age (Paintsil et al., 2008). In many resource-limited settings, clinicians rely on clinical and immunologic criteria to identify children failing first-line therapy (Jittamala et al., 2009; Ruel et al., 2011). CD4+ T cells are enumerated with flow cytometry technique based on either dual or single platform technologies (Mandy et al., 2003). The dual-platform technology determines absolute T-lymphocyte numbers using two different instruments (a hematology analyzer and a flow cytometer). The absolute CD4+ T cell number is the product of three laboratory measurements: The white blood cell count, the percentage of white blood cells that are lymphocytes (determined by hematology analyzer) and the percentage of lymphocytes that are CD4+ (determined by flow cytometry). The single-platform technology is designed to determine both absolute and percentage lymphocyte subset values using a single tube. Because of the high cost of equipment, reagents and maintenance of the single and dual platform technologies, most resource-limited settings use relatively less expensive and technically less complex devices for CD4 enumeration (Lyamuya et al., 1996; Taiwo and Murphy, 2008). However, most of these devices provide the absolute CD4+ T cell counts and not the percentage of CD4+ T cells, thus they are not pediatric friendly (Diagbouga et al., 2003). There is a need for further research to determine whether absolute CD4+ T cell count can be used in place of percentage CD4+ T cells in managing HIV-infected children of all ages especially in areas where available devices for CD4 enumeration do not measure percentage CD4+ T cells (Diagbouga et al., 2003).

The objective of our study was two-fold: (1) To estimate the predictive accuracy of absolute CD4+ T count and CD4+ T percentage in monitoring pediatric HIV disease progression and (2) compare performances of the two markers. HIV disease progression was assessed by the proportion of detectable viremia defined as any detectable viral load above the limit of detection offered by the assay (Paintsil et al., 2016). We analyzed longitudinal data from HIV-infected children enrolled in the Yale Prospective Longitudinal Pediatric HIV Cohort study comprising of children born to HIV infected mothers in the greater New Haven area, since 1985. We assessed the performance of the two markers in a longer follow-up duration period spanning different treatment strategy eras (i.e., pre-highly active antiretroviral therapy (HAART) and HAART) and also provide evidence to support the use of absolute CD4+ T cell count in HIV-infected children, regardless of age.

Statistical Methods

Receiver Operating Curve (ROC) and Area Under ROC

Let Di denotes the detectable viremia status for the ith subject. D is a binary random variable and takes the values 0 or 1 depending on the frequency of detectable viremia episodes. D=0 for individuals who had detectable viremia during less than or equal to 50% of their clinic visits (infrequent episodes of detectable viremia). For individuals who had detectable viremia during more than 50% of their clinic visits (frequent episodes of detectable viremia), D takes the value 1. Suppose Y1 and Y2 are continuous positive random variables and represent the absolute CD4+ T cell count and percent of an individual, respectively. Let Sy1 and Sy2 denote the survival functions for Y1 and Y2 in individuals with frequent episodes of detectable viremia; and Gy1 and Gy2 are the corresponding functions for the group with infrequent episodes of detectable viremia. The sensitivity (true positive rate) and 1-specificity (false positive rate) of the kth (k = 1, 2) marker at cut-off t are given by [Ykt|D=1]=Syk(t) and 1[Yk<t|D=0]=[Ykt|D=0]=Gyk(t). The Receiver Operating Curve (ROC) is a plot of Gyk(t) (x-coordinate) versus Syk(t) (y-coordinate) for every possible threshold t. That is, the ROC curve is the entire set of possible values of Gyk and Syk obtained by dichotomizing Yk with different values of t ϵ(−∞, +∞).

The ROC curve has many advantages over single measurements of sensitivity and specificity. ROC curve incorporates both of these measures and gives a full picture of the performance of the classifier marker at different thresholds. Furthermore, ROC curves show the trade-offs between these two measures as the threshold changes. While the ROC curve gives a full picture of the performance of the classifier at different thresholds, a quantitative summary index frequently used as an overall assessment of classifier performance is the area under the ROC curve (AUC) which is given by 01ROC(u)du. The area under the ROC curve is interpreted as the average value of sensitivity (specificity) for all possible values of specificity (sensitivity) and corresponds to the probability of correctly identifying subjects with frequent (+) and infrequent (−) episodes of detectable viremia. That is, AUC is equal to the probability (Y+k>Yk). For a given threshold t, the sensitivity which corresponds to specificity z0=1Gyk(t) is Syk(t)=Syk(Gyk1(1z0)). From this, we can write the AUC as 01Syk(Gyk1(1z0))dz0=Syk(t)dGyk(t)=(Y+k>Yk). Thus, the AUC is a measure of the separation in location between Syk and Gyk.

ROC Curves for Correlated Longitudinal Markers

The area under the ROC (AUC) and its empirical estimator have previously been studied for a cross-sectional data (Hanley and McNeil, 1982; Venkatraman and Begg, 1996). However, there is a paucity of literature on the AUC for correlated longitudinal markers. Obuchowski proposed a nonparametric approach for the estimation and inference of AUC of two correlated longitudinal markers (Obuchowski, 1997). However, her approach did not consider the effect of covariates. Covariates should be incorporated in the ROC curve analyses as they affect the performance of diagnostic markers (Pepe, 2003). In this paper, we used the Lehmann model (referred as the proportional hazards model) (Gönen and Heller, 2010; Holling et al., 2012) to estimate and compare the performance of absolute CD4 count and CD4 percentage as markers of pediatric HIV disease progression. Using the Lehman approach, the association between sensitivity and 1-specificity can be modeled as:

Syk(t)=Gyk(t)θ (1)

where, 0 < θ ≤ 1 is measure of marker performance with values close to zero indicating higher diagnostic accuracy. Thus, the Lehmann family of ROC curve can be written using the equation y = xθ and the corresponding AUC is 01xθdx=11+θ. Moreover, from Equation (1), logSyk(t)logGyk(t)=θ which is equivalent to the Cox Proportional Hazards (Cox PH) model used in survival analyses (Cox, 1972). The connection between the Lehmann and Cox model will enable us to estimate θ which in turn allows us to obtain the corresponding ROC and AUC. The Cox PH model states that covariates have proportional effect on the hazard function of failure time distribution. That is, if hi(t) denotes the hazard for the ith individual at time t, the Cox PH model can be written as hi(t)=h0(t)eβ1xi1++βpxip. h0 denotes the hazard function for an individual whose covariates all have values of 0 and (xi1,…, xip) are set of p covariates.

Let Yijk denotes values of the kth marker (k = 1, 2) for the ith (i = 1, 2,…,n) patient at the jth (j = 1, 2,…, ni) time. The response vector for the kth marker is Yk=(Y11k,Y12k,,Y1n1k,Y21k,Y22k,,Y2n2k,,Yn1k,Yn2k,,Ynnnk). Yk is the kth marker values of all individuals stacked into a single response vector. The hazard functions for Yk are defined as hs(y)=limΔy0P[y<Y<y+Δy|Yy,D=s]Δy, for s = 0, 1. To estimate the area under the ROC curve of each longitudinal marker, Cox PH model was used with the marker values Yk as dependent variable and the detectable viremia status indicator D as independent variable:

h1(y|D)=h0(y)eβ1D. (2)

Note that h1(y,D=1)h1(y,D=0)=eβ1 and using Lehmann model (1) the marker performance measure θ=e1β. An estimate of β^1 can be obtained using Cox’s partial maximum likelihood estimation procedure available in many statistical software packages (Cox, 1975). By the invariance property of maximum likelihood estimators θ is estimated by θ^=eβ^1. Repeated marker measurements made on the same individual are correlated. Thus, we used the robust sandwich variance estimator in order to obtain the correct inferences and variance of β^1 (Wei et al., 1989). We then used the Delta method to obtain approximate variance for θ^. That is, we approximated variance of θ^ by θ^2σβ^12, where σβ^12 is variance of β^1. Similarly, the variance for the AUC was estimated by θ^2σβ^12(1+θ^)4.

The Lehmann (Cox PH) model can be also used to obtain covariate-adjusted ROC curves. We present here the effect of a single covariate but the model can be easily extended to include multiple covariates. Suppose X2 denotes a covariate that may affect the performance of each marker and β2 denotes the corresponding regression coefficient. Then the adjusted Cox PH model is:

h1(y|D,X2)=h0(y)eβ1D+β2X2+β3DX2 (3)

In the above model, covariate effect is assessed using the regression coefficient of the interaction term. Using the connection between the Lehman (1) and Cox (3) models, the covariate-adjusted marker performance parameter is θ(x2)=eβ1+β3x2 (note that h1(y,D=1,x2)h1(y,D=0,x2)=eβ1+β3x2). Thus, the covariate-adjusted ROC curve is y(x,x2)=xθ(x2).

The objective of our study was twofold: Estimate and compare the performance of the two biomarkers of HIV disease progression. As described above, Cox PH models were used in assessing the performance of each longitudinal biomarker in the absence or presence of covariates. In this section, we evaluate the performance of the two markers by comparing their corresponding ROC curves. Let Y = (Y1, Y2) be the combined marker values, i.e., the absolute and percent CD4+ T cells of all patients are stacked into a single response vector. Let C define a dummy variable with C = 0 for k = 1 (CD4+ T cells absolute count) and C = 1 for k = 2 (CD4+ T cells percent). Then, using the combined marker values, the Cox PH model can be written as:

h1(y|D,C)=h0(y)eβ1D+β2C+β3CD. (4)

For the first marker, h1(y,D=1,C=0)h1(y,D=0,C=0)=eβ1 and for the second marker h1(y,D=1,C=1)h1(y,D=0,C=1)=eβ1+β3. These show that the ROC curves of the two markers are the same when β3 = 0. Thus, the performance of the two markers can be compared by testing the null hypotheses Ho: β3 = 0. The statistic for testing Ho is computed as T=β^32/σβ^32, where σβ^32 is the robust sandwich variance of β^3. Under Ho, T has an asymptotic chi-squared distribution with one degree of freedom (Wald chi-squared statistic).

Simulation Study

We ran a large set of simulation studies to evaluate the performance of the Lehmann model. In each simulation, the response for the ith (i = 1, 2,…, n) patient at the jth (j = 1, 2,…, n) time (i.e., Yij) was generated from a Weibull distribution with scale parameter 1 for the population with infrequent episodes of detectable viremia. For the population with frequent episodes of detectable viremia, Yij followed a Weibull distribution with scale parameter 1/θ1/s for a given choice of shape parameter s. We have considered various values of the Weibull shape parameter s. The Clayton copula with dependence parameter c was used to model the correlation structure of the ith patient response vector Yi=(Yi1,Yi2,,Yini). We generated 5,000 independent data sets and model 2 was applied to the generated 5,000 independent data sets to obtain β^1 and σβ^32. For each data set, the asymptotic confidence interval of β11 = log(θ)) was constructed using (β^1qσβ^1,β^1+qσβ^1), where q = Φ−1(1-α/2) and Φ is the standard normal cumulative distribution function. A significance level α = 0.05 corresponding to the nominal confidence level 0.95 was considered. The empirical coverage probability was obtained by computing the proportion of times the confidence interval contained the true β1.

The results presented here are representative of the many simulations performed. Figure 1 presents results for ni = 5 (i = 1, 2,…,n) and c = 2, i.e., five repeated observations per person with Clayton copula dependence parameter = 2. Various sample sizes, values of the Weibull shape parameter and values of β1 were considered. The true AUC value was obtained from β1 using Equation (1), i.e., AUC=11+θ where θ=e1β. The results in Figure 1 show that the empirical coverage rates of the nominal 1-α = 0.95 confidence intervals improve from approximately 0.9 when the sample size is 10 to approximately 0.94 when the sample size is 40. When the sample size is small, the coverage is somewhat poorer for higher true AUC values. The shape parameter does not appear to affect strongly the coverage in this design.

Fig. 1:

Fig. 1:

Coverage probabilities for the AUC obtained by the Lehman model

Data Analyses

Study Population and Measures

The study population was comprised of HIV-infected children enrolled through the Yale Prospective Longitudinal Pediatric Cohort study at Yale-New Haven Hospital. All the children either had mothers already known to be HIV-1 seropositive during pregnancy or at the time of delivery or were discovered to be infected with HIV-1 after presenting with an AIDS defining illness. In the analyses contained herein, we used longitudinal data collected from 104 HIV-infected children from January 1996, when plasma HIV-1 RNA quantification became available, to November 2013. The study participants were seen and examined at the pediatric clinic every 2–3 months and more frequently as necessary. Routine blood tests, HIV-1 RNA quantification and CD4+ T-lymphocyte counts and percentages were done every 2–3 months to follow HIV disease progression. HIV-1 RNA quantification was done using the Amplicor Monitor test (Roche Diagnostic Systems, Inc., Branchburg, New Jersey, USA). CD4+ T-lymphocyte levels were quantified using a standard dualplatform flow cytometry technology. The study protocol was reviewed and approved by Institutional Review Board at the Yale School of Medicine.

The outcome variable was viral blip defined as any detectable viral load above the limit of detection offered by the assay (Paintsil et al., 2008). The children were categorized into two groups based on the frequency of detectable viral load episodes: (1) Infrequent episodes of detectable viremia (children who had detectable viremia during less than 50% of their clinic visits) and (2). frequent episodes of detectable viremia (children who had detectable viremia during more than 50% of their clinic visits). The predictor variables included time varying (absolute CD4+ T-lymphocyte count and CD4+ T-lymphocyte percentage) and time-invariant (e.g., gender, race, age at study entry, age at HIV diagnosis, caregiver type, history of AIDS defining illness, time since HIV diagnosis, antiretroviral treatment status and CDC clinical staging of HIV infection) covariates.

Results

The demographic and HIV disease characteristics of the study participants are described in Table 1. Continuous characteristics were summarized using median and Inter Quartile Range (IQR); categorical characteristics were summarized using frequency and percent (%). Wilcoxon rank sum and Fisher’s exact tests were used to compare continuous and categorical patients characteristics, respectively, between subjects with infrequent and frequent episodes of detectable viremia. The median age of the study participants at study entry and the time of diagnosis were 7.83 (IQR = 5.02–11.98) and 1.79 (0.44–4.91) years, respectively. Majority of the participants (81%, n = 86) had frequent episodes of detectable viremia. Fifty four percent were males, 59% were African Americans and 57% had biological parents as caregivers. The majority of the participants had congenital HIV (93%), moderate to severe CDC classification (65%) and other comorbidities (55%). As previously reported (Paintsil et al., 2016), there were no statistically significant differences in patient characteristics between patients with frequent and infrequent episodes of detectable viremia.

Table 1:

Characteristics of study participants stratified by frequency of detectable viremia

Detectable viremia
Study covariates All (N = 104) Infrequent (N = 18) Frequent (N = 86) P-value
Baseline CD4/mm3 436 (125–725) 550 (26–820) 420 (140–663) 0.48
Baseline VL (Log10) 4.79 (4.13–5.32) 4.60 (2.60–5.50) 4.79 (4.20–5.18) 0.85
Age at study entry (years) 7.83 (5.02–11.98) 7.74 (3.70–13.07) 7.83 (5.20–11.76) 0.92
Age at diagnosis (years) 1.79 (0.44–4.91) 2.04 (0.21–5.32) 1.79 (0.52–4.91) 0.82
Time since Diagnosis (years) 5.14 (2.32–7.35) 3.03 (0.60–8.88) 5.42 (2.52–7.32) 0.40
Gender 1.00
Female 48 (46%) 8 (44%) 40 (47%)
Male 56 (54%) 10 (56%) 46 (53%)
Race/Ethnicity 0.80
Black 61 (59%) 10 (56%) 51 (59%)
Other 43 (41%) 8 (44%) 35 (41%)
Other Illness 1.00
No 43 (45%) 7 (47%) 36 (44%)
Yes 53 (55%) 8 (53%) 45 (56%)
Caregiver 0.79
Parent 59 (57%) 11 (61%) 48 (56%)
Other 44 (43%) 7 (39%) 37 (44%)

Source: Paintsil et al., J AIDS Clin Res. 2016 Apr; 7(4). pii: 565

Figure 2 displays CD4+ T cell count and percentage trajectories by frequency of detectable viremia status. The average trend lines in Figure 2 show that CD4+ T cell count and percentage values for both groups slowly increased from baseline values and decreased at later time points over the study period. Moreover, Fig. 2 shows that children with infrequent episodes of detectable viremia had higher CD4+ T-cell counts and percentage over time compared with those with frequent episodes of detectable viremia.

Fig. 2:

Fig. 2:

Trajectories of CD4+ T Cell lymphocytes by Frequency of Detectable Viremia Status (LOESS Lines): Absolute CD4+ T-cells (Left), Percent CD4+ T-cells (Right)

The association between episodes of detectable viremia and the two longitudinal biomarkers of interest was examined using model (2). More specifically, the model was used to construct the ROC curves of the two markers and compute the corresponding areas under the ROC curves. Figure 3 displays the ROC curves for the two biomarkers. The overall diagnostic accuracy of the two biomarkers were comparable. Both biomarkers had about a 68% probability of correctly distinguishing a frequent from an infrequent episodes of detectable viremia patient. The area under the ROC curves for CD4+ T cell count and percentage were 0.681 [SE = 0.029; 95% CI: 0.624–0.737] and 0.678 [SE = 0.024; 95% CI:0.630–0.725], respectively. We compared the two ROC curves by testing the null hypotheses Ho: β3 = 0 against Ha: β3 ≠ 0 in model 4. The two-sided Wald P-value was 0.59 and this shows that there was no statistically significant difference between the ROC curves of the two biomarkers.

Fig. 3:

Fig. 3:

ROC curves for absolute and percent CD4+ T cells

Given the greater variability of absolute CD4+ T-lymphocyte counts compared with percentage among children under 5 years of age (Paintsil et al., 2008), we investigated whether the performance of the two biomarkers will vary by age. Using model (3), we have shown that the performance of each biomarker did not change with age although there was a slight improvement in older patients. The area under the ROC curve for CD4+ T cell absolute count in younger (≤5 years) and older patients (5+ years) were 0.702 [95% CI: 0.647–0.757] and 0.739 [95% CI: 0.606–0.873], respectively; and the corresponding two sided Wald P-value was 0.650. Similarly, the area under the ROC curve for CD4+ T cell percent in younger and older patients were 0.677 [95% CI: 0.624–0.729] and 0.772 [95% CI: 0.706–0.837], respectively; and the corresponding two-sided Wald P-value was 0.097. Figure 4 display the ROC curves of the two biomarkers for the two age groups.

Fig. 4:

Fig. 4:

ROC Curves for Absolute and Percent CD4+ T Cells: Younger Patients (Left), Older Patients (Right)

Conclusion

In this paper, we developed model based AUC to estimate and compare the performance of absolute CD4+ T-lymphocyte count and CD4+ T-lymphocyte percentage. We found that the cross validated AUC for CD4+ T-lymphocyte count and percentage were approximately 0.681 (0.678) with the corresponding 95% confidence intervals [0.624–0.737] ([0.630–0.725]). Furthermore, there was no statistically significant difference between the AUC of the two biomarkers. This finding is of special interest with regard to the control of pediatric HIV in resource-limited settings where HIV prevalence is highest. Due to the variations in CD4+ T-lymphocyte numbers, the CD4+ T-lymphocyte percentage is traditionally used to monitor HIV disease progression and the effects of HAART in children (Paintsil et al., 2016). However, due to the high cost of equipment and lack of adequate laboratory infrastructure, most laboratories in resource-limited settings use simple protocols that enumerate only the absolute CD4+ T-lymphocyte counts. Our findings demonstrate that absolute CD4+ T-lymphocyte count could be as useful as the percentage CD4+ T-lymphocytes in monitoring pediatric HIV in resource-limited settings regardless of age.

Acknowledgment

The authors are grateful for the financial support from the Harvard University Center for AIDS Research (HU CFAR NIH/NAIDS P30-AI 060354). The authors thank the children and families who participated in the study.

Footnotes

Ethics

This article is original and contains unpublished material. The corresponding author confirms that all of the other authors have read and approved the manuscript and no ethical issues involved.

References

  1. Cox DR, 1972. Regression models and life tables (with discussion). J. R Statist. Soc. B, 34: 187–202. DOI: 10.1111/j.2517-6161.1972.tb00899.x [DOI] [Google Scholar]
  2. Cox DR, 1975. Partial likelihood. Biometrika, 62: 269–276. DOI: 10.1093/biomet/62.2.269 [DOI] [Google Scholar]
  3. Diagbouga S, Chazallon C, Kazatchkine MD, Van de Perre P and Inwoley A et al. , 2003. Successful implementation of a low-cost method for enumerating CD4+ T lymphocytes in resource-limited settings: The ANRS 12–26 study. Aids, 17: 2201–2208. DOI: 10.1097/00002030-200310170-00008 [DOI] [PubMed] [Google Scholar]
  4. Ford N, Meintjes G, Pozniak A, Bygrave H and Hill A et al. , 2015. The future role of CD4 cell count for monitoring antiretroviral therapy. Lancet Infect. Dis, 15: 241–247. DOI: 10.1016/S1473-3099(14)70896-5 [DOI] [PubMed] [Google Scholar]
  5. Gönen M and Heller G, 2010. Lehmann family of ROC curves. Med. Decis. Mak, 30: 509–517. DOI: 10.1177/0272989X09360067 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Hammer SM, Saag MS, Schechter M, Montaner JS and Schooley RT et al. , 2006. Treatment for adult HIV infection: 2006 recommendations of the International AIDS Society-USA panel. Jama, 296: 827–843. DOI: 10.1001/jama.296.7.827 [DOI] [PubMed] [Google Scholar]
  7. Hanley J and McNeil B, 1982. The meaning and use of the area under a Receiver Operating Characteristic (ROC) curve. Radiology, 143: 29–36. DOI: 10.1148/radiology.143.1.7063747 [DOI] [PubMed] [Google Scholar]
  8. Holling H, Böhning W and Böhning D, 2012. Meta-analysis of diagnostic studies based upon SROC-curves: A mixed model approach using the Lehmann family. Stat. Modell, 12: 1–29. DOI: 10.1177/1471082X1201200403 [DOI] [Google Scholar]
  9. Hosseinipour MC, van Oosterhout JJ, Weigel R, Phiri S and Kamwendo D et al. , 2009. The public health approach to identify antiretroviral therapy failure: High-level nucleoside reverse transcriptase inhibitor resistance among Malawians failing first-line antiretroviral therapy. Aids, 23: 1127–1134. DOI: 10.1097/QAD.0b013e32832ac34e [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hughes MD, Stein DS, Gundacker HM, Valentine FT and Phair JP et al. , 1994. Withinsubject variation in CD4 lymphocyte count in asymptomatic human immunodeficiency virus infection: Implications for patient monitoring. J. Infect. Dis, 169: 28–36. DOI: 10.1093/infdis/169.1.28 [DOI] [PubMed] [Google Scholar]
  11. Jittamala P, Puthanakit T, Chaiinseeard S and Sirisanthana V, 2009. Predictors of virologic failure and genotypic resistance mutation patterns in Thai children receiving non-nucleoside reverse transcriptase inhibitor-based antiretroviral therapy. Pediatr. Infect. Dis. J, 28: 826–830. DOI: 10.1097/INF.0b013e3181a458f9 [DOI] [PubMed] [Google Scholar]
  12. Lyamuya EF, Kagoma C, Mbena EC, Urassa WK and Pallangyo K et al. , 1996. Evaluation of the FACScount, TRAx CD4 and Dynabeads methods for CD4 lymphocyte determination. J. Immunol. Meth, 195: 103–112. DOI: 10.1016/0022-1759(96)00094-4 [DOI] [PubMed] [Google Scholar]
  13. Mandy FF, Nicholson JK and McDougal JS, 2003. Guidelines for performing single-platform absolute CD4+ T-cell determinations with CD45 gating for persons infected with human immunodeficiency virus. Centers for Disease Control and Prevention. MMWR Recomm. Rep, 52: 1–13. [PubMed] [Google Scholar]
  14. Mellors JW, Munoz A, Giorgi JV, Margolick JB and Tassoni CJ et al. , 1997. Plasma viral load and CD4+ lymphocytes as prognostic markers of HIV-1 infection. Ann. Intern. Med, 126: 946–954. DOI: 10.7326/0003-4819-126-12-199706150-00003 [DOI] [PubMed] [Google Scholar]
  15. Nishanian P, Taylor JM, Manna B, Aziz N and Grosser S et al. , 1998. Accelerated changes (inection points) in levels of serum immune activation markers and CD4+ and CD8+ T cells prior to AIDS onset. J. Acquir. Immune Defic. Syndr. Hum. Retrovirol, 18: 162–170. DOI: 10.1097/00042560-199806010-00008 [DOI] [PubMed] [Google Scholar]
  16. Obuchowski N, 1997. Nonparametric analysis of clustered ROC curve data. Biometrics, 53: 567–578. DOI: 10.2307/2533958 [DOI] [PubMed] [Google Scholar]
  17. Paintsil E, Ghebremichael M, Romano S and Andiman WA, 2008. Absolute CD4+ T-lymphocyte count as a surrogate marker of pediatric human immunodeficiency virus disease progression. Pediatr. Infect. Dis. J, 27: 629–635. DOI: 10.1097/INF.0b013e3181693892 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Paintsil E, Martin R, Goldenthal A, Bhandari S and Andiman W et al. , 2016. Frequent episodes of detectable viremia in HIV treatment-experienced children is associated with a decline in CD4+ T-cells over time. J. AIDS Clin. Res, 7: 565–565. DOI: 10.4172/2155-6113.1000565 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Pantaleo G, Graziosi C and Fauci AS, 1993. New concepts in the immunopathogenesis of human immunodeficiency virus infection. N Engl. J. Med, 328: 327–335. DOI: 10.1056/NEJM199302043280508 [DOI] [PubMed] [Google Scholar]
  20. Pepe MS, 2003. The Statistical Evaluation of Medical Tests for Classification and Prediction. 1st Edn., Oxford University Press, New York, ISBN-10: 0198509847, pp: 302. [Google Scholar]
  21. Rawizza HE, Chaplin B, Meloni ST, Eisen G and Rao T et al. , 2011. Immunologic criteria are poor predictors of virologic outcome: Implications for HIV treatment monitoring in resource-limited settings. Clin. Infect. Dis, 53: 1283–1290. DOI: 10.1093/cid/cir729 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Reynolds SJ, Nakigozi G, Newell K, Ndyanabo A and Galiwongo R et al. , 2009. Failure of immunologic criteria to appropriately identify antiretroviral treatment failure in Uganda. AIDS, 23: 697–700. DOI: 10.1097/QAD.0b013e3283262a78 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Ruel TD, Kamya MR, Li P, Pasutti W and Charlebois ED et al. , 2011. Early virologie failure and the development of antiretroviral drug resistance mutations in HIV-infected Ugandan children. J. Acquir. Immune Defic. Syndr, 56: 44–50. DOI: 10.1097/QAI.0b013e3181fbcbf7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Stein DS, Korvick JA and Vermund SH, 1992. CD4+ lymphocyte cell enumeration for prediction of clinical course of human immunodeficiency virus disease: A review. J. Infect. Dis, 165: 352–363. DOI: 10.1093/infdis/165.2.352 [DOI] [PubMed] [Google Scholar]
  25. Taiwo BO and Murphy RL, 2008. Clinical applications and availability of CD4+ T cell count testing in sub-Saharan Africa. Cytometry B Clin. Cytom, 74: S11–18. DOI: 10.1002/cyto.b.20383 [DOI] [PubMed] [Google Scholar]
  26. Venkatraman E and Begg C, 1996. A distribution-free procedure for comparing receiver operating characteristic curves from paired experiment. Biometrika, 83: 835–848. DOI: 10.1093/biomet/83.4.835 [DOI] [Google Scholar]
  27. Vlahov D, Graham N, Hoover D, Flynn C and Bartlett JG et al. , 1998. Prognostic indicators for AIDS and infectious disease death in HIV-infected injection drug users: Plasma viral load and CD4+ cell count. Jama, 279: 35–40. DOI: 10.1001/jama.279.1.35 [DOI] [PubMed] [Google Scholar]
  28. Wade AM and Ades AE, 1998. Incorporating correlations between measurements into the estimation of age-related reference ranges. Stat. Med, 17: 1989–2002. DOI: [DOI] [PubMed] [Google Scholar]
  29. Wei LJ, Lin DY and Weissfeld L, 1989. Regression analysis of multivariate incomplete failure time data by using marginal distributions. J. Am. Stat. Assoc, 84: 1065–1073. DOI: 10.1080/01621459.1989.10478873 [DOI] [Google Scholar]
  30. WHO, 2010. Antiretroviral drugs for treating pregnant women and preventing HIV infection in infants: recommendations for public health approach. World Health Organization. [PubMed] [Google Scholar]
  31. WHO, 2015. WHO early release guidelines on when to start antiretroviral therapy and on pre-exposure prophylaxis for HIV. World Health Organization; Geneva. [PubMed] [Google Scholar]

RESOURCES