Abstract
Background
Early initiation of antiretroviral therapy (ART) has been shown to reduce mortality among perinatally HIV-infected infants, but availability of virologic testing remains limited in many settings.
Methods
We collected cross-sectional data from mother-infant pairs in three primary care clinics in Lusaka, Zambia to develop predictive models for HIV infection among infants age <12 weeks. We evaluated algorithm performance for all possible combinations of selected parameters using an iterative approach. In primary analysis, we identified the model with the highest combined sensitivity and specificity.
Results
Between July 2009 and May 2011, 822 eligible HIV-infected mothers and their HIV-exposed infants were enrolled; of these, 44 (5.4%) infants were diagnosed with HIV. We evaluated 382,155,260 different parameter combinations for predicting infant HIV infection. The algorithm with highest combined sensitivity and specificity required 5 of the following 7 parameter thresholds : infant CD8% > 22, infant CD4% ≤ 44, infant weight-for-age Z score ≤ 0, infant CD4 ≤ 1600 cells/μL, infant CD8 > 2200 cells/μL, maternal CD4 ≤ 600 cells/μL, and mother not currently on ART for HIV treatment. This combination had a sensitivity of 90.3%, specificity of 78.4%, positive predictive value (PPV) of 22.4%, negative predictive value (NPV) of 99.2%, and area under the curve (AUC) of 0.844.
Conclusion
Predicting HIV infection in HIV-exposed infants in this age group is difficult using clinical and immunologic parameters. Expansion of PCR capacity in resource-limited settings remains urgently needed.
Keywords: HIV, early infant diagnosis, presumptive diagnosis, predictive model, Zambia
Diagnosis of HIV and early initiation of antiretroviral therapy (ART) dramatically improve survival among infected infants.1 However, as few as 28% of HIV-exposed infants worldwide undergo HIV testing within the first two months of life.2 The implementation of early infant diagnosis programs has proven challenging, particularly in high-prevalence, low-resource settings where the majority of perinatal HIV transmission occurs.3, 4
Definitive diagnosis of HIV in children less than 18 months of age requires virologic assays, typically polymerase chain reaction (PCR) testing to detect viral DNA or RNA.5-8 Because infants may retain maternal immunoglobulin-G antibodies for more than a year after birth, conventional HIV antibody tests have limited utility for accurately determining positive infection status.9-14 Unfortunately, PCR testing is expensive and, in most developing countries, available only through specialty laboratories. It is technically complex, requires staff with special training and technical skill, and can be considerably more costly than other diagnostic tests.5, 15 Even when access is available, the geographic distances between clinical facilities and specialty laboratories often result in significant delays in accurate reporting of results.3, 4 Thus, most countries with large populations of HIV-infected children have extremely limited capacity for timely infant diagnosis during early infancy.
METHODS
To address this gap in implementation science, we sought to develop a non-virologic algorithm to predict HIV infection among exposed infants less than12 weeks of age. We conducted a cross-sectional study across three primary care clinics in Lusaka, Zambia. Prevention of mother-to-child HIV transmission (PMTCT) services provided by the Ministry of Health have been previously described.16-18 Briefly, women are offered “optout” HIV testing at their first antenatal visit. Those identified as HIV-infected are screened for ART eligibility by clinical staging and CD4 testing. Women meeting criteria for long-term HIV treatment – CD4 cell counts < 350 cells/μL and/or World Health Organization (WHO) clinical stage 3 or 4 – are prescribed it immediately. Those who are not eligible for ART are prescribed short-course zidovudine and peripartum nevirapine. At the time of study enrollment, the Zambian national PMTCT guidelines had not yet incorporated maternal or infant antiretroviral prophylaxis during breastfeeding. Following delivery, infants are scheduled for a six-week postpartum visit, where dried blood spots are collected for HIV DNA PCR testing.
Cross-sectional data collection
HIV-infected mothers and their infants were recruited from each facility’s maternal and child health (or “under-5”) department. Candidates were deemed eligible if they met the following criteria: documented maternal HIV infection, infant age ≤60 weeks, and willingness to participate in the study. The primary study population comprised infants under 12 weeks of age; infants age 12 to 60 weeks were separately enrolled to validate the non-virologic algorithm for older age groups and are not included in this report. Although maternal and infant antiretroviral drug use for perinatal HIV prophylaxis was permitted, we excluded infants who had already initiated ART for HIV treatment.
After explaining the study in detail and obtaining written informed consent, we collected detailed information about the mother and infant, including demographic characteristics, medical history, physical examination, and laboratory testing. All mothers and infants underwent WHO staging as part of this study. Weight-for-age Z scores were calculated using established algorithms from the World Health Organization.19 Maternal tests included a complete hematologic panel and CD4 count. Infant specimens were drawn for CD4 count and percentage, CD8 count and percentage, complete hematologic panel, and rapid HIV antibody test. Consistent with the standard of care in Zambia at the time, a single HIV DNA PCR test was performed to determine infant HIV status. All results were reported to participants at a scheduled visit two to four weeks later. Where appropriate, referrals were made to an HIV care and treatment department co-located on the same facility grounds.
For the present analysis, we sought to enroll at least 800 HIV-exposed infants under 12 weeks of age and their mothers. This sample size was based on the anticipated precision of algorithm performance measures. The first 500 enrollees – all from a single clinic – were included in the “model-building” phase of the study (see below). The remaining enrollees, approximately 300 mother-infant pairs, provided temporal validation of our initial findings.
Model-building
With data from the first 500 participants, we constructed models to predict infant HIV infection, defined by a single positive HIV DNA PCR. We first identified maternal and infant characteristics associated in univariate analysis with infant HIV infection. Those associations with statistical thresholds of p ≤ 0.01 were included in further analysis. We used this rigorous threshold for statistical significance to prioritize those with greatest association to infant HIV infection, while limiting the potential number of variables included in our final models. For each of these characteristics, we then established pre-defined increments for assessment, an approach that retained discriminatory power while minimizing the potential for “over-fitting.”20 We next iteratively dichotomized the population of interest using multiple cut-off values for each variable. We evaluated all permutations of factors across all combinations of selected variables. For each combination, we constructed a two-by-two table that cross-tabulated the algorithm’s categorization of infants as either HIV-infected or -uninfected, against the gold standard measure of HIV DNA PCR. The performance for these combinations were then assessed by calculating sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operative characteristic curve (AUC). We examined every increment for each field’s pre-defined range and across every possible number of included fields.
Selecting and validating predictive models
In our primary analysis, the algorithm with the highest combined sensitivity and specificity was declared the best-performing, consistent with receiver-operator characteristic approaches.21, 22 Because AUC can be approximated by adding sensitivity and specificity and dividing by two,23 this best-performing algorithm prioritized AUC as well. In our secondary analysis, we sought to maximize sensitivity first, followed by specificity. We reasoned that such an algorithm could have an important role for triaging HIV-exposed infants. Those identified as screen-positive could be started on empiric therapy or referred for further virologic testing. Assuming that the negative predictive value was acceptably high, those identified as screen-negative could potentially defer virologic testing until after the cessation of breastfeeding under close clinical monitoring.
We identified the best-performing algorithm for each of these approaches, stratified by the number of model parameters. These algorithms were temporally validated using our independently recruited study population of approximately 300 mother-infant pairs, and the performance measures of sensitivity, specificity, PPV, NPV, and AUC were calculated. All analyses were performed using SAS version 9.1 (SAS Institute, Cary, NC) and R software version 2.4.1 (http://www.r-project.org). We obtained ethical review approvals from the University of North Carolina at Chapel Hill (Chapel Hill, NC, USA), the University of Alabama at Birmingham (Birmingham, AL, USA) and the University of Zambia (Lusaka, Zambia).
RESULTS
Between July 2009 and May 2011, we enrolled 822 eligible HIV-infected mothers and their HIV-exposed infants under 12 weeks of age. Overall, 44 (5.4%) infants were diagnosed with HIV by DNA PCR testing. Most participating mothers reported use of PMTCT services during the antenatal, intrapartum, and postpartum periods. At time of enrollment, the vast majority of infants (93%) were still exclusively breastfed. Other medical, demographic, and laboratory characteristics are shown in Table, Supplemental Digital Content 1, http://links.lww.com/INF/B337. Of these 822 mother-infant pairs, 779 (95%) had complete data and were included in the model building (n= 481) and validation (n= 298) components of the study.
We performed univariate analyses to determine predictors for infant HIV infection, using both maternal and infant parameters (TABLE 1). Seven characteristics met our statistical threshold for inclusion in the predictive models: infant weight-for-age Z score, infant CD4 count, infant CD8 count, infant CD4 percentage, infant CD8 percentage, maternal CD4 count, and current ART status of the mother. An eighth characteristic, infant CD4:CD8 ratio, also met the defined statistical threshold but was ultimately excluded because its component parts were already considered. We used our iterative approach to generate and evaluate 382,155,260 separate models for predicting infant HIV infection within this age group. The range and parameter thresholds used in constructing these models are shown in Table, Supplemental Digital Content 2, http://links.lww.com/INF/B338.
Table 1.
Univariate predictors of HIV infection among the first 500 infants enrolled in the “model-building” phase of the study
| HIV-infected infants (N=31) |
HIV-uninfected infants (N=469) |
||||
|---|---|---|---|---|---|
| n | Value | n | Value | P * | |
| Infant Characteristics | |||||
| Age (days) | 31 | 44 (44, 51) | 469 | 44 (44, 44) | 0.28++ |
| Weight-for-age Z score | 31 | −0.6 (−1.3, 0.1) | 465 | −0.0 (−1.1, 0.6) | <0.01++ |
| WHO clinical stage 3 or 4 | 31 | 1 (3.2%) | 467 | 2 (0.4%) | 0.05** |
| CD4 count (cells/μL) | 31 | 1317 (984, 2136) | 455 | 1795 (1381, 2328) | <0.01++ |
| CD4 percentage | 31 | 30.0 (23.0, 37.0) | 455 | 40.0 (33.0, 45.0) | <0.01++ |
| CD8 count (cells/μL) | 31 | 1324 (921, 2218) | 454 | 933 (628, 1401) | <0.01++ |
| CD8 percentage | 31 | 26.0 (24.0, 34.0) | 454 | 20.0 (16.0, 25.0) | <0.01++ |
| CD4:CD8 ratio | 31 | 1.3 (0.7, 1.6) | 454 | 2.0 (1.4, 2.8) | <0.01++ |
| Hemoglobin (g/dL) | 30 | 11.1 (9.5, 12.2) | 443 | 11.3 (10.5, 12.1) | 0.23++ |
| Maternal Characteristics | |||||
| Age (years) | 31 | 27 (23, 31) | 469 | 28 (24, 33) | 0.17++ |
| WHO clinical stage 3 or 4 | 31 | 1 (3.2%) | 468 | 38 (8.1%) | 0.33** |
| CD4 count (cells/μL) | 31 | 387 (188, 525) | 469 | 469 (328, 631) | <0.01++ |
| Hemoglobin (g/dL) | 30 | 11.7 (10.5, 13.0) | 458 | 12.1 (11.2, 13.0) | 0.21++ |
| Not currently on HIV treatment | 31 | 26 (83.9%) | 469 | 287 (61.2%) | 0.01** |
Continuous variables are shown as medians with interquartile ranges (IQR) noted in parentheses.
Chi-square test;
Wilcoxon Rank Sum Test
In our primary analysis, we sought to maximize the combined value of sensitivity and specificity. We examined the best-performing algorithms by this standard according to the number of model parameters (TABLE 2). The highest combined sensitivity and specificity was observed when any 5 of the following 7 parameter thresholds were met: infant CD8% > 22, infant CD4% ≤ 44, infant weight for age Z score ≤ 0, infant CD4 count ≤ 1600 cells /μL, infant CD8 count > 2200 cells /μL, maternal CD4 count ≤ 600 cells /μL, and mother not currently on ART. Overall model performance showed a sensitivity of 90.3% (95%CI: 74.2%-98.0%), specificity of 78.4% (95%CI: 74.4%-82.2%), PPV of 22.4% (95%CI: 15.4%-30.7%), NPV of 99.2% (95%CI: 97.6%-99.8%), and AUC of 0.844 (95%CI: 0.788, 0.900). When we applied this algorithm to our temporal validation cohort, we found that sensitivity was 41.7% (95%CI: 15.2%-72.3%), specificity was 82.9% (95%CI: 78.0%-87.0%), PPV was 9.3% (95%CI: 3.1%-20.3%), NPV was 97.1% (95%CI: 94.2%-98.8%), and AUC was 0.623 (95%CI: 0.475, 0.770).
Table 2.
Best-performing parameter combinations when prioritized by highest combined sensitivity and specificity in the model-building phase of the study, stratified by number of parameters in the model
| Model-building phase | Temporal validation phase | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|||||||||
| Parameters in model (N) |
Criteria met |
Criterion thresholds for best- performing algorithm |
Sens (%) | Spec (%) | PPV (%) | NPV (%) | Sens (%) | Spec (%) | PPV (%) | NPV (%) |
| 1 | 1 | Infant CD8% > 22 | 80.6 (62.5 - 92.5) |
65.3 (60.7 - 69.7) |
13.8 (9.1 - 19.7) |
98.0 (95.7 - 99.3) |
50.0 (21.1 - 78.9) |
68.2 (62.4 - 73.5) |
6.2 (2.3 - 13.0) |
97.0 (93.6 - 98.9) |
|
| ||||||||||
| 2 | 2 | Infant CD8% > 22, Infant weight-for-age Z-score < 0.4 |
77.4 (58.9 - 90.4) |
76.9 (72.7 - 80.7) |
18.8 (12.4 - 26.6) |
98.0 (96.0 - 99.2) |
33.3 (9.9 - 65.1) |
79.0 (73.8 - 83.6) |
6.3 (1.7 - 15.2) |
96.6 (93.4 - 98.5) |
|
| ||||||||||
| 3 | 2 | Infant CD8% > 22, Infant weight-for-age Z-score < 0.4, Mom CD4 count < 100 |
80.6 (62.5 - 92.5) |
76.2 (72.0 - 80.1) |
18.9 (12.6 - 26.7) |
98.3 (96.3 - 99.4) |
33.3 (9.9 - 65.1) |
78.7 (73.5 - 83.3) |
6.2 (1.7 - 15.0) |
96.6 (93.3 - 98.5) |
|
| ||||||||||
| 4 | 3 | Infant CD8% > 22, Infant weight-for-age Z-score < 0, Infant CD4 count < 1600, Mother not on ART |
87.1 (70.2 - 96.4) |
74.9 (70.6 - 78.8) |
19.3 (13.1 - 26.8) |
98.8 (97.0 - 99.7) |
41.7 (15.2 - 72.3) |
78.7 (73.5 - 83.3) |
7.6 (2.5 - 16.8) |
97.0 (93.9 - 98.8) |
|
| ||||||||||
| 5 | 4 | Infant CD8% > 22, Infant weight-for-age Z-score ≤ 0, Infant CD4 count ≤ 1600, Maternal CD4 count ≤ 600, Mother not on ART |
83.9 (66.3 - 94.5) |
80.2 (76.2 - 83.8) |
22.6 (15.3 - 31.3) |
98.6 (96.8 - 99.6) |
41.7 (15.2 - 72.3) |
83.9 (79.1 - 88.0) |
9.8 (3.3 - 21.4) |
97.2 (94.2 - 98.9) |
|
| ||||||||||
| 6 | 4 | Infant CD8% > 22, Infant weight-for-age Z-score ≤ 0, Infant CD4 count ≤ 1600, Infant CD8 count > 2200, Maternal CD4 count ≤ 600, Mother not on ART |
90.3 (74.2 - 98.0) |
77.1 (72.9 - 80.9) |
21.4 (14.7 - 29.4) |
99.1 (97.5 - 99.8) |
41.7 (15.2 - 72.3) |
81.5 (76.5 - 85.8) |
8.6 (2.9 - 19.0) |
97.1 (94.1 - 98.8) |
|
| ||||||||||
| 7 * | 5 | Infant CD8% > 22, Infant CD4% ≤ 44 Infant weight-for-age Z-score ≤ 0, Infant CD4 count ≤ 1600, Infant CD8 count > 2200, Maternal CD4 count ≤ 600, Mother not on ART |
90.3 (74.2 - 98.0) |
78.4 (74.4 - 82.2) |
22.4 (15.4 - 30.7) |
99.2 (97.6 - 99.8) |
41.7 (15.2 - 72.3) |
82.9 (78.0 - 87.0) |
9.3 (3.1 - 20.3) |
97.1 (94.2 - 98.8) |
Best-performing algorithm in model-building phase. Sens = sensitivity; Spec = specificity, PPV = positive predictive value, NPV = negative predictive value, 95%CI = 95% confidence interval. Due to space constraints, we were unable to include the area under the curve (AUC) for each of the listed algorithms in this table; however, AUC is approximated by the equation (sensitivity + specificity) / 2.
For our secondary “triage” model, an algorithm meeting 4 of the following 6 parameter thresholds performed best: infant CD8% > 22, infant CD4% ≤ 48, infant weight for age Z score ≤ −0.2, infant CD4 count ≤ 2600 cells /μL, infant CD8 count > 1000 cells /μL, and mother not currently on ART (TABLE 3). Model performance demonstrated a sensitivity of 100% (95%CI: 88.8%–100%), specificity of 54% (95%CI: 49.3%-58.7%), PPV of 13.0% (95%CI: 9.0%-18.0%), NPV of 100% (95%CI: 98.5%-100%), and AUC of 0.769 (95%CI: 0.747, 0.793). Validation of the model resulted in a sensitivity of 83.3% (95%CI: 51.6%-97.9%), specificity of 58.4% (95%CI: 52.4%-64.2%), PPV of 7.8% (95%CI: 3.8%-13.8%), NPV of 98.8% (95%CI: 95.8%-99.9%), and AUC of 0.709 (95%CI: 0.595, 0.822).
Table 3.
Best-performing parameter combinations when sensitivity is first prioritized, then followed by specificity, stratified by number of parameters in the model
| Model-building phase | Temporal validation phase | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
|||||||||
| Parameters in model (N) |
Criteria met |
Criterion thresholds for best- performing algorithm |
Sens (%) | Spec (%) | PPV (%) | NPV (%) | Sens (%) | Spec (%) | PPV (%) | NPV (%) |
| 1 | 1 | Infant CD4% ≤ 48 | 100.0 (88.8 - 100.0) |
16.2 (12.9 - 20.0) |
7.6 (5.2 - 10.6) |
100.0 (95.1 - 100.0) |
91.7 (61.5 - 99.8) |
20.3 (15.8 - 25.4) |
4.6 (2.3 - 8.1) |
98.3 (90.9 - 100.0) |
|
| ||||||||||
| 2 | 1 | Infant CD8% > 22, Mother not on ART |
100.0 (88.8 - 100.0) |
26.0 (22.0 - 30.3) |
8.5 (5.9 - 11.9) |
100.0 (96.9 - 100.0) |
83.3 (51.6 - 97.9) |
33.6 (28.1 - 39.4) |
5.0 (2.4 - 9.0) |
98.0 (92.8 - 99.8) |
|
| ||||||||||
| 3 | 2 | Infant CD8% > 22, Infant CD4% ≤ 48, Mother not on ART |
100.0 (88.8 - 100.0) |
36.2 (31.8 - 40.9) |
9.7 (6.7 - 13.6) |
100.0 (97.8 - 100.0) |
75.0 (42.8 - 94.5) |
43.7 (37.9 - 49.7) |
5.3 (2.4 - 9.8) |
97.7 (93.3 - 99.5) |
|
| ||||||||||
| 4 | 2 | Infant CD4% > 26, Infant weight-for-age Z-score ≤ −0.2, Infant CD8 count > 1000, Mother not on ART |
100.0 (88.8 - 100.0) |
44.2 (39.6 - 48.9) |
11.0 (7.6 - 15.2) |
100.0 (98.2 - 100.0) |
75.0 (42.8 - 94.5) |
59.4 (53.5 - 65.2) |
7.2 (3.3 - 13.2) |
98.3 (95.0 - 99.6) |
|
| ||||||||||
| 5 | 3 | Infant CD8% > 22, Infant weight-for-age Z-score ≤ −0.2, Infant CD4 count ≤ 2600, Infant CD8 count > 1000, Mother not on ART |
100.0 (88.8 - 100.0) |
50.7 (45.9 - 55.4) |
12.3 (8.5 - 16.9) |
100.0 (98.4 - 100.0) |
83.3 (51.6 - 97.9) |
55.9 (50.0 - 61.8) |
7.4 (3.6 - 13.1) |
98.8 (95.6 - 99.9) |
|
| ||||||||||
| 6 * | 4 | Infant CD8% > 22, Infant CD4% ≤ 48, Infant weight-for-age Z-score ≤ −0.2, Infant CD4 count ≤ 2600, Infant CD8 count > 1000, Mother not on ART |
100.0 (88.8 - 100.0) |
54.0 (49.3 - 58.7) |
13.0 (9.0 - 18.0) |
100.0 (98.5 - 100.0) |
83.3 (51.6 – 97.9) |
58.4 (52.4 - 64.2) |
7.8 (3.8 - 13.8) |
98.8 (95.8 - 99.9) |
|
| ||||||||||
| 7 | 4 | Infant CD8% > 22, Infant CD4% ≤ 48, Infant weight-for-age Z-score ≤ −0.2, Infant CD4 count ≤ 2600, Infant CD8 count > 1000, Mom CD4 count ≤ 50, Mother not on ART |
100.0 (88.8 - 100.0) |
53.8 (49.0 - 58.5) |
13.0 (9.0 - 17.9) |
100.0 (98.5 - 100.0) |
83.3 (51.6 - 97.9) |
58.4 (52.4 - 64.2) |
7.8 (3.8 - 13.8) |
98.8 (95.8 - 99.9) |
Best-performing algorithm overall. Sens = sensitivity; Spec = specificity, PPV = positive predictive value, NPV = negative predictive value, 95%CI = 95% confidence interval. Due to space constraints, we were unable to include the area under the curve (AUC) for each of the listed algorithms in this table; however, AUC is approximated by the equation: (sensitivity + specificity) / 2.
DISCUSSION
In this study, we evaluated more than 380 million separate cut-point combinations of seven commonly available parameters to predict HIV infection among infants less than 12 weeks of age. Our optimal combination demonstrated reasonable performance when compared to the gold standard of HIV DNA PCR; however, its sensitivity was not confirmed by temporal validation. These findings are discouraging and highlight inherent difficulties in diagnosing infant HIV without virologic testing. By themselves, clinical and immunologic factors are simply not good enough for use in programmatic settings.
Numerous studies have examined the validity of surrogate markers for predicting HIV infection among infants. Performance has varied greatly, though certain trends have emerged. Clinical algorithms, for example, generally demonstrate high specificity at the expense of sensitivity.24-27 In contrast, laboratory markers such as infant CD4 percentage show higher sensitivity but lower specificity at studied thresholds.27, 28 Combination algorithms that include both types of information generally perform better than when either is considered alone, with sensitivities at 70-90% accompanied by specificities at 40-50%.27, 29 In this context, our best-performing model (sensitivity 90%, specificity 78%) compared very favorably to other published studies. This study was designed specifically for the purpose of identifying a non-virologic diagnostic algorithm for HIV infection in infants. A key strength of our design was the provision made for model validation. We first enrolled 500 mother-infant pairs from a single primary care facility to obtain data from which to construct our models. We then expanded recruitment to two additional facilities (i.e., three in total) to test the performance of our best algorithms. In the validation phase, the performance of our primary algorithm declined considerably. Sensitivity dropped from 90% to 42%, although specificity remained relatively stable (78% vs. 83%). The smaller validation sample – along with lower HIV transmission rates – may have contributed to model instability. With only 13 HIV-infected infants, misclassification of even a single event could profoundly affect performance. It is also possible, however, that our original model may have been over-fitted and its findings not applicable across different populations, even ones that appeared similar. These results emphasize the importance of model validation as part of algorithm development,20, 30 a process seldom included in analogous studies seeking to predict infant HIV infection.
In secondary analysis, we considered a predictive algorithm optimized for patient triage. The best-performing algorithm within these predetermined parameters had 100% sensitivity, 54% specificity, 13% PPV, and 100% NPV. Applied to a hypothetical cohort of 1,000 HIV-exposed infants and assuming an infant infection rate of 6% (similar to that of this study), 432 could theoretically be triaged from further virologic testing based on a very high certainty of a negative status. Such a strategy could have important implications for resource-constrained settings, where HIV DNA PCR testing may be limited by cost or where geographical distances between the clinic and specialty laboratory may lead to delayed results reporting. Although sensitivity was reduced in our validation exercise, the role of triaging algorithms appears promising and should be considered further.
Our study was designed to include both clinical findings and laboratory screening in the model building process. We emphasized medical information from the infant, though maternal parameters deemed to be readily available in most clinical settings (e.g., maternal ART status, maternal CD4) were considered. Interestingly, several infant laboratory tests – but very few clinical characteristics – were found to be associated with infant infection in our initial univariate selection of model parameters. Laboratory-intensive algorithms, such as the ones proposed here, may be constrained by the availability of their component tests. While we acknowledge this potential limitation, most of our proposed laboratory tests (e.g., CD4 testing, hemoglobin) are performed as part of routine HIV care and treatment. Across the nine provinces in Zambia, for example, the Ministry of Health has certified 161 laboratories with CD4 capacity (personal communication, C. Moyo), in contrast to only three with HIV DNA PCR testing capacity. CD8 testing, perhaps the one listed assay that is not routinely performed, can be conducted on most flow cytometry instruments with only modest increases in reagent costs.
In the urban Lusaka district, where a robust public sector PMTCT program has been in place for nearly a decade,16 we observed relatively low HIV transmission rates. Among the 500 infants in the model-building phase, the prevalence of HIV was 6.0%; among the 322 infants in the validation phase, it was slightly lower at 4.2%. The wide availability of effective antiretroviral prophylaxis regimens undoubtedly contributed to this success. However, we also recognize the selection biases inherent to this primary care setting. Since symptomatic infants are typically referred to the University Teaching Hospital, the country’s only tertiary care center, this population is underrepresented in our cross-sectional study. Similar work conducted in settings of higher HIV transmission (e.g., secondary or tertiary care institutions, rural programs with lower service coverage) could yield different results.
We used an iterative method to determine the optimal thresholds of model parameters to predict infant HIV infection.31 This technique was computationally intensive, requiring the generation and evaluation of numerous algorithm combinations. However, this approach has several advantages over more commonly used techniques such as tree-based recursive partitioning models. Our methodology permitted optimization of varying performance measures for different clinical scenarios. We were able to pre-determine clinically relevant increments and cut-points for our algorithms. Our final algorithm format – which resembles a checklist rather than the flow charts of tree-based models – may be more clinically intuitive for frontline providers. In many African countries, initial HIV testing for exposed infants typically occurs around six weeks of life, a practice recommended by the WHO.32 We restricted our analysis to infants less than 12 weeks of age and, as such, our results are most applicable to populations infected perinatally or in the early breastfeeding period. We recognize that these results may not extend to older infants who likely acquire HIV through breastfeeding. As part of this study, however, we enrolled infants between 12 to 60 weeks to evaluate predictive models within this older age range. We are planning similar analyses to determine whether non-virologic algorithms may perform better in this population.
In this study, we sought to address a notable gap in the implementation of comprehensive pediatric AIDS mitigation programs in Africa: the timely diagnosis of HIV-infected infants in the months following delivery. Although our analysis yielded promising algorithms for predicting infant HIV infection, performance did not hold up in validation. We believe a predictive approach may have utility in settings with limited diagnostic capacity, particularly in a triaging role. Ultimately, however, PCR – or other molecular technologies that test for the virus itself – will likely be required. Efforts to increase their access must continue urgently.
Supplementary Material
Acknowledgements
The authors acknowledge the Zambian Ministry of Health for consistent and high-level support of operations research in the context of HIV program expansion.
Funding source: The work reported herein was funded by a PMTCT operations research grant from the Elizabeth Glaser Pediatric AIDS Foundation. Additional investigator salary and trainee support was provided by the National Institutes of Health (D43-TW001035) and a Clinical Scientist Development Award from the Doris Duke Charitable Foundation (2007061). The authors report no conflicts of interest to disclose.
Footnotes
LISTING OF SUPPLEMENTAL DIGITAL CONTENT Supplemental_digital_content_1.doc Supplemental_digital_content_2.doc
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
REFERENCES
- 1.Violari A, Cotton MF, Gibb DM, et al. Early antiretroviral therapy and mortality among HIV-infected infants. N Engl J Med. 2008;359:2233–2244. doi: 10.1056/NEJMoa0800971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Joint United Nations Programme on HIV/AIDS (UNAIDS) Global HIV/AIDS response: epidemic update and health sector progress towards universal access, progress report 2011. World Health Organization; Geneva: 2011. [Google Scholar]
- 3.Ciaranello AL, Park JE, Ramirez-Avila L, Freedberg KA, Walensky RP, Leroy V. Early infant HIV-1 diagnosis programs in resource-limited settings: opportunities for improved outcomes and more cost-effective interventions. BMC Med. 2011;9:59. doi: 10.1186/1741-7015-9-59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Creek TL, Sherman GG, Nkengasong J, et al. Infant human immunodeficiency virus diagnosis in resource-limited settings: issues, technologies, and country experiences. Am J Obstet Gynecol. 2007;197:S64–71. doi: 10.1016/j.ajog.2007.03.002. [DOI] [PubMed] [Google Scholar]
- 5.Sherman GG, Cooper PA, Coovadia AH, et al. Polymerase chain reaction for diagnosis of human immunodeficiency virus infection in infancy in low resource settings. Pediatr Infect Dis J. 2005;24:993–997. doi: 10.1097/01.inf.0000187036.73539.8d. [DOI] [PubMed] [Google Scholar]
- 6.Nesheim S, Lee F, Kalish ML, et al. Diagnosis of perinatal human immunodeficiency virus infection by polymerase chain reaction and p24 antigen detection after immune complex dissociation in an urban community hospital. J Infect Dis. 1997;175:1333–1336. doi: 10.1086/516464. [DOI] [PubMed] [Google Scholar]
- 7.Kovacs A, Xu J, Rasheed S, et al. Comparison of a rapid nonisotopic polymerase chain reaction assay with four commonly used methods for the early diagnosis of human immunodeficiency virus type 1 infection in neonates and children. Pediatr Infect Dis J. 1995;14:948–954. doi: 10.1097/00006454-199511000-00005. [DOI] [PubMed] [Google Scholar]
- 8.Comeau AM, Pitt J, Hillyer GV, et al. Early detection of human immunodeficiency virus on dried blood spot specimens: sensitivity across serial specimens. Women and Infants Transmission Study Group. J Pediatr. 1996;129:111–118. doi: 10.1016/s0022-3476(96)70197-x. [DOI] [PubMed] [Google Scholar]
- 9.Moodley D, Bobat RA, Coutsoudis A, Coovadia HM. Predicting perinatal human immunodeficiency virus infection by antibody patterns. Pediatr Infect Dis J. 1995;14:850–852. doi: 10.1097/00006454-199510000-00006. [DOI] [PubMed] [Google Scholar]
- 10.Cunningham CK, Charbonneau TT, Song K, et al. Comparison of human immunodeficiency virus 1 DNA polymerase chain reaction and qualitative and quantitative RNA polymerase chain reaction in human immunodeficiency virus 1-exposed infants. Pediatr Infect Dis J. 1999;18:30–35. doi: 10.1097/00006454-199901000-00009. [DOI] [PubMed] [Google Scholar]
- 11.Steketee RW, Abrams EJ, Thea DM, et al. Early detection of perinatal human immunodeficiency virus (HIV) type 1 infection using HIV RNA amplification and detection. New York City Perinatal HIV Transmission Collaborative Study. J Infect Dis. 1997;175:707–711. doi: 10.1093/infdis/175.3.707. [DOI] [PubMed] [Google Scholar]
- 12.Delamare C, Burgard M, Mayaux MJ, et al. HIV-1 RNA detection in plasma for the diagnosis of infection in neonates. The French Pediatric HIV Infection Study Group. J Acquir Immune Defic Syndr Hum Retrovirol. 1997;15:121–125. doi: 10.1097/00042560-199706010-00004. [DOI] [PubMed] [Google Scholar]
- 13.Chantry CJ, Cooper ER, Pelton SI, Zorilla C, Hillyer GV, Diaz C. Seroreversion in human immunodeficiency virus-exposed but uninfected infants. Pediatr Infect Dis J. 1995;14:382–387. doi: 10.1097/00006454-199505000-00009. [DOI] [PubMed] [Google Scholar]
- 14.Louisirirotchanakul S, Kanoksinsombat C, Likanonsakul S, Sunthornkachit R, Supanit I, Wasi C. Patterns of anti-HIV IgG3, IgA and p24Ag in perinatally HIV-1 infected infants. Asian Pac J Allergy Immunol. 2002;20:99–104. [PubMed] [Google Scholar]
- 15.Ginsburg AS, Miller A, Wilfert CM. Diagnosis of pediatric human immunodeficiency virus infection in resource-constrained settings. Pediatr Infect Dis J. 2006;25:1057–1064. doi: 10.1097/01.inf.0000243157.16405.f0. [DOI] [PubMed] [Google Scholar]
- 16.Stringer EM, Sinkala M, Stringer JS, et al. Prevention of mother-to-child transmission of HIV in Africa: successes and challenges in scaling-up a nevirapine-based program in Lusaka, Zambia. AIDS. 2003;17:1377–1382. doi: 10.1097/01.aids.0000060395.18106.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Chi BH, Chintu N, Lee A, Stringer EM, Sinkala M, Stringer JS. Expanded Services for the Prevention of Mother-to-Child HIV Transmission: Field Acceptability of a Pilot Program in Lusaka, Zambia. J Acquir Immune Defic Syndr. 2007;45:125–127. doi: 10.1097/QAI.0b013e318050d28f. [DOI] [PubMed] [Google Scholar]
- 18.Killam WP, Tambatamba BC, Chintu N, et al. Antiretroviral therapy in antenatal care to increase treatment initiation in HIV-infected pregnant women: a stepped-wedge evaluation. AIDS. 2010;24:85–91. doi: 10.1097/QAD.0b013e32833298be. [DOI] [PubMed] [Google Scholar]
- 19.World Health Organization [Accessed June 4, 2012];Child growth standards: software and macros. http://www.who.int/childgrowth/software/en/
- 20.Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605. doi: 10.1136/bmj.b605. [DOI] [PubMed] [Google Scholar]
- 21.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845. [PubMed] [Google Scholar]
- 22.Zou KH, O’Malley AJ, Mauri L. Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation. 2007;115:654–657. doi: 10.1161/CIRCULATIONAHA.105.594929. [DOI] [PubMed] [Google Scholar]
- 23.Zhang DD, Zhou XH, Freeman DH, Jr., Freeman JL. A non-parametric method for the comparison of partial areas under ROC curves and its application to large health care data sets. Stat Med. 2002;21:701–715. doi: 10.1002/sim.1011. [DOI] [PubMed] [Google Scholar]
- 24.Jaspan HB, Myer L, Madhi SA, et al. Utility of clinical parameters to identify HIV infection in infants below ten weeks of age in South Africa: a prospective cohort study. BMC Pediatr. 2011;11:104. doi: 10.1186/1471-2431-11-104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Horwood C, Liebeschuetz S, Blaauw D, Cassol S, Qazi S. Diagnosis of paediatric HIV infection in a primary health care setting with a clinical algorithm. Bull World Health Organ. 2003;81:858–866. [PMC free article] [PubMed] [Google Scholar]
- 26.Iliff P, Ntozini R, Nathoo K, Piwoz E, Moulton L, Humphrey J. Making a working clinical diagnosis of HIV infection in infants in Zimbabwe. Trop Med Int Health. 2008;13:1459–1469. doi: 10.1111/j.1365-3156.2008.02178.x. [DOI] [PubMed] [Google Scholar]
- 27.Inwani I, Mbori-Ngacha D, Nduati R, et al. Performance of clinical algorithms for HIV-1 diagnosis and antiretroviral initiation among HIV-1-exposed children aged less than 18 months in Kenya. J Acquir Immune Defic Syndr. 2009;50:492–498. doi: 10.1097/QAI.0b013e318198a8a4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Rouet F, Inwoley A, Ekouevi D, et al. CD4 percentages and total lymphocyte counts as early surrogate markers for pediatric HIV-1 infection in resource-limited settings. J Trop Pediatr. 2006;52:346–354. doi: 10.1093/tropej/fml024. [DOI] [PubMed] [Google Scholar]
- 29.Peltier CA, Omes C, Ndimubanzi PC, et al. Validation of 2006 WHO prediction scores for true HIV infection in children less than 18 months with a positive serological HIV test. PLoS One. 2009;4:e5312. doi: 10.1371/journal.pone.0005312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144:201–209. doi: 10.7326/0003-4819-144-3-200602070-00009. [DOI] [PubMed] [Google Scholar]
- 31.Liu KC, Mulindwa J, Giganti MJ, et al. Predictors of CD4 eligibility for antiretroviral therapy initiation among HIV-infected pregnant women in Lusaka, Zambia. J Acquir Immune Defic Syndr. 2011;57:e101–105. doi: 10.1097/QAI.0b013e31821d3507. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.World Health Organization . Antiretroviral therapy for HIV infection in infants and children; recommendations for a public health approach - 2010 revision. WHO Press; Geneva, Switzerland: 2010. [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.