Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jun 1.
Published in final edited form as: Pediatr Infect Dis J. 2014 Jun;33(6):608–616. doi: 10.1097/INF.0000000000000214

Prognosis of Children with HIV-1 Infection Starting Antiretroviral Therapy in Southern Africa: A Collaborative Analysis of Treatment Programs

Mary-Ann Davies 1, Margaret May 2, Carolyn Bolton-Moore 3, Cleophas Chimbetete 4, Brian Eley 5, Daniela Garone 6, Janet Giddy 7, Harry Moultrie 8, James Ndirangu 9, Sam Phiri 10, Helena Rabie 11, Karl Technau 12, Robin Wood 13, Andrew Boulle 1, Matthias Egger 1,14, Olivia Keiser 14, for the IeDEA Southern Africa (IeDEA-SA) Collaboration
PMCID: PMC4349941  NIHMSID: NIHMS540908  PMID: 24378936

Abstract

Background

Prognostic models for children starting antiretroviral therapy (ART) in Africa are lacking. We developed models to estimate the probability of death during the first year receiving ART in Southern Africa.

Methods

We analyzed data from children ≤10 years old who started ART in Malawi, South Africa, Zambia or Zimbabwe from 2004–2010. Children lost to follow-up or transferred were excluded. The primary outcome was all-cause mortality in the first year of ART. We used Weibull survival models to construct two prognostic models: one with CD4%, age, WHO clinical stage, weight-for-age z-score (WAZ) and anemia and one without CD4%, because it is not routinely measured in many programs. We used multiple imputation to account for missing data.

Results

Among 12655 children, 877 (6.9%) died in the first year of ART. 1780 children were lost to follow-up/transferred and excluded from main analyses; 10875 children were included. With the CD4% model probability of death at 1 year ranged from 1.8% (95% CI: 1.5–2.3) in children 5–10 years with CD4% ≥10%, WHO stage I/II, WAZ ≥−2 and without severe anemia to 46.3% (95% CI: 38.2–55.2) in children <1 year with CD4% <5%, stage III/IV, WAZ< −3 and severe anemia. The corresponding range for the model without CD4% was 2.2% (95% CI: 1.8–2.7) to 33.4% (95% CI: 28.2–39.3). Agreement between predicted and observed mortality was good (C-statistics=0.753 and 0.745 for models with and without CD4% respectively).

Conclusion

These models may be useful to counsel children/caregivers, for program planning and to assess program outcomes after allowing for differences in patient disease severity characteristics.

Keywords: Mortality, HIV-1, children, sub-Saharan Africa, antiretroviral


Despite increased access to antiretroviral therapy (ART) for HIV-infected children in low-income settings, mortality remains high. In 2010, an estimated 230000 children died of AIDS in sub-Saharan Africa.1 While many deaths occur in untreated patients,2 mortality remains high during the first year of ART, especially for children starting therapy with advanced disease.35 Knowing the short term prognosis associated with particular disease severity characteristics is important for individual children initiating ART and their caregivers, as well as for clinicians and for program planning. Further, comparison of actual mortality outcomes with predictions from a prognostic model that is generalizable across settings may be useful for benchmarking the quality of health care provision. While models of pediatric pre-ART mortality have been developed for high and low-income settings and used to inform decisions regarding treatment initiation,69 prognostic models of mortality on ART have to date only been developed for adults.1012

The characteristics associated with mortality in children starting ART have been well described.35,1320 However, the combined power of different disease severity markers to predict mortality and the absolute mortality risk associated with these markers remains unknown. Young children and those with low CD4% or advanced clinical disease are at high risk of morbidity and mortality.3,4,13,18,20 HIV-1 RNA level and anemia are also independent mortality risk factors, although HIV-RNA is less predictive than CD4.13,15,21 However, in low income settings, measurement of many of these prognostic factors, including CD4, is often unavailable.22 The International epidemiologic Databases to Evaluate AIDS Southern Africa (IeDEA-SA) Collaboration includes data from children starting ART at 11 treatment programs in a range of settings in four countries.3,20 We aimed to use these data to develop a prognostic model that estimates the cumulative probability of death at 3, 6 and 12 months after starting ART according to age, and prognostic factors commonly measured in resource-limited settings. Separate models were developed for settings with and without access to CD4% at ART initiation.

METHODS

Treatment programs

IeDEA-SA is a regional collaboration of ART programs, which is part of a larger international network.23 Data are collected at ART initiation and follow-up visits and regularly transferred to data centers at the Universities of Cape Town, South Africa, and Bern, Switzerland. All sites have ethical approval to collect data and participate in IeDEA-SA. This analysis was based on data from 11 programs in four countries, including eight clinics in three provinces in South Africa (Red Cross Children’s Hospital, Khayelitsha and Gugulethu ART Programs and Tygerberg Academic Hospital, Western Cape; McCord Hospital and Hlabisa HIV Treatment and Care Program, Kwazulu-Natal; Harriet Shezi Children’s Clinic and Rahima Moosa Mother and Child Hospital, Gauteng) and one program each in Zambia (Ministry of Health and Centre for Infectious Disease Research in Zambia program [MoH-CIDRZ]; Lusaka), Malawi (Lighthouse Trust Clinic at Kamuzu Central Hospital; Lilongwe) and Zimbabwe (Newlands Clinic; Harare).

Inclusion criteria

All HIV-infected, ART-naïve (except for antiretrovirals to prevent vertical transmission) children who initiated treatment with ≥3 antiretrovirals at age ≤10 years between 1 January 2004 and 31 January 2010 were eligible. Children with <1 year of potential follow-up (ART initiation <1 year prior to site database closure) were excluded. Children were excluded from the main analysis if, within 6 months of starting ART, they were lost to follow-up (LTFU) or transferred out (TFO) to a different treatment site. As clinic visits may be up to 6 months apart, LTFU was defined as the last visit being ≥270 days before database closure.

Outcomes and prognostic models

We used an intention-to-continue-treatment analysis, ignoring treatment changes and interruptions. The outcome was all-cause mortality during the first year on ART. Follow-up was censored on the earliest of: date of death, last visit date +90 days in children LTFU/TFO, or 1 year after ART initiation (Supplementary Figure S1). The 11 cohorts were grouped into five geographic regions as some cohorts had small patient numbers. Three regions were provinces within South Africa where ART programs at different sites are coordinated by provincial Departments of Health (Western Cape, Kwazulu-Natal and Gauteng). The fourth region comprised two cohorts with similar patient characteristics in Malawi and Zimbabwe, while the MoH-CIDRZ cohort (Zambia) comprised the fifth region. Treatment initiation criteria at these sites were based on WHO and national guidelines at the time. In South Africa, the recommended first-line regimen was 2 nucleoside reverse transcriptase inhibitors (NRTI) plus either lopinavir/ritonavir (children <3 years or <10kg) or efavirenz (children > 3 years and >10kg). In Malawi, Zimbabwe and Zambia 2 NRTIs plus nevirapine was the recommended first-line for all children.

The following prognostic variables measured at ART initiation were considered for inclusion in a prognostic model and associations with mortality were explored using Kaplan-Meier survival curves in the pre-specified categories of age (<1 year, 1 year, 2–4 years, 5–10 years); WHO Clinical Stage (I/II compared to III/IV); CD4% (<5, 5–9.9, 10–14.9; ≥15); weight-for-age z-score (WAZ) calculated using WHO 2006 standards (<−3.00; −3.00 to −2.01, −2.00 to −1.01 and ≥ −1.00 standard deviations below mean)24 and anemia defined using CDC classification that incorporates both hemoglobin and age.25 Apart from age and gender, data on prognostic variables were not recorded for all patients. Missing data were modeled using multiple imputation by chained equations, with 25 imputed datasets.2629 The following variables were used in imputation equations: cohort, sex, age, WHO Stage, CD4%, WAZ, hemoglobin, interactions between age, CD4% and WAZ, survival time and mortality indicator. Log or square root transformations were used for non-normally distributed variables. Weibull proportional hazards models were used to explore crude and adjusted associations between prognostic variables and mortality.

A set of candidate models (with and without CD4%) were selected using the Akaike Information Criterion (AIC). These were flexible parametric survival models30 with spline smoothing of the baseline hazard to model the steep mortality during the first 3 months of treatment. We used a system of leave-one-out cross-validation to select the most generalizable models with and without CD4%.31,32 This method fits the model using data from four regions and tests discrimination of predictions of the model when applied to the omitted fifth region. This was repeated sequentially rotating the omitted region. Discrimination was assessed using the D-statistic (averaged across the imputed datasets) which measures the prognostic separation between the survival distributions for two independent prognostic groups.32 We calculated the D-statistic for the model fitted on the four regions and applied to the omitted region (Dtest) and compared this to the D-statistic for the model when coefficients were re-estimated using data only from the omitted region (Dr). The difference (Dr−Dtest) is a measure of the degradation in model fit and discrimination when applied to independent data compared with when applied to the data used to estimate the model coefficients. Models with a low AIC score, high Dtest and low Dr−Dtest were favored.

Concordance between predicted mortality and observed mortality for the final selected models was assessed using Harrell’s C-statistic (0.5 = agreement expected by chance; 1 = perfect agreement). The explained variation (R2) of the final selected models was calculated.33,34 Model calibration was assessed by comparing Kaplan-Meier curves of observed mortality with curves predicted from the model for groups with poor to good prognosis, and for each region. All analyses were conducted in STATA version 12.0 (STATA Corporation, College Station, Texas).

Sensitivity analyses

In sensitivity analyses we included children LTFU/TFO within six months of starting treatment. We examined Kaplan-Meier estimates of one-year mortality first censoring their follow-up at the last visit date +90 days, and then assuming 30% and 50% mortality in those LTFU/TFO with time to death randomly assigned based on the distribution in those not LTFU/TFO. Finally, we developed models including children LTFU/TFO, censoring follow-up time at the last visit date +90 days.

RESULTS

In the 11 treatment programs, 12655 children started ART. 1780 children (14%) were LTFU or TFO within 6 months of ART start and excluded from the main analyses. The main analysis included 10875 children (70% ≥2 years old) with 10204 child-years of follow-up (Table 1). Most children had advanced disease at ART initiation (72% WHO Clinical Stage III/IV; median [IQR] CD4%: 13% [8–19]). There was considerable between-region heterogeneity in age and disease severity; the percentage of children <1 year ranged from 3% to nearly 30%; the percentage with WHO Stage III/IV disease ranged from approximately 60% to >90%. There was substantial missing data on many covariates; anemia and CD4% were not reported for 37% (range across regions: 18–88%) and 32% (16–56%) of children respectively. The estimated cumulative mortality by 1 year after ART start was 8.6% (95% CI: 8.0–9.2) (range across regions: 5.8%–10.5%).

Table 1.

Pediatric antiretroviral therapy (ART) programs with number of patients, characteristics at ART initiation, follow-up duration, status by 1 year after ART initiation and Kaplan-Meier estimates of mortality by 1 year after ART initiation.

Region Western Cape Gauteng Kwazulu-Natal Malawi & Zimbabwe Zambia Total
Number of children 1505 (14%) 2015 (19%) 791 (7%) 706 (6%) 5858 (54%) 10875
Sex
Female 720 (48%) 997 (49%) 396 (50%) 364 (52%) 2883 (49%) 5360 (49%)
Male 785 (52%) 1018 (51%) 395 (50%) 342 (48%) 2975 (51%) 5515 (51%)
Age group
< 1 year 441 (29%) 306 (15%) 74 (9%) 23 (3%) 583 (10%) 1427 (13%)
1 year 290 (19%) 282 (14%) 128 (16%) 71 (10%) 1008 (17%) 1779 (16%)
2–4 years 412 (27%) 629 (31%) 224 (28%) 219 (31%) 1809 (30%) 3293 (30%)
5–10 years 362 (24%) 798 (40%) 365 (46%) 393 (56%) 2458 (40%) 4376 (40%)
WHO Stage1
I/II 241 (18%) 461 (41%) 98 (19%) 37 (6%) 1746 (31%) 2583 (28%)
III/IV 1078 (82%) 669 (59%) 428 (81%) 538 (94%) 3964 (69%) 6677 (72%)
Not reported 186 (12%) 885 (44%) 265 (34%) 131 (18%) 148 (3%) 1615 (15%)
Anemia1
Mild/none 903 (83%) 209 (87%) 464 (85%) 196 (87%) 3736 (78%) 5508 (80%)
Moderate 114 (10%) 18 (8%) 54 (10%) 18 (8%) 590 (12%) 794 (12%)
Severe 72 (7%) 13 (5%) 31 (6%) 11 (5%) 461 (10%) 588 (9%)
Not reported 416 (28%) 1775 (88%) 242 (31%) 481 (68%) 1071 (18%) 3985 (37%)
Median (IQR) CD4% 13 (9 to 19)% 11 (7 to 15)% 13 (8 to 17)% 14 (9 to 20)% 14 (9 to 20)% 13 (8 to 19)%
Not reported 460 (31%) 313 (16%) 230 (29%) 397 (56%) 2080 (36%) 3480 (32%)
Median (IQR) WAZ −1.9 (−3.5 to −0.8) −2.0 (−3.1 to −1.1) −1.4 (−2.5 to −0.5) −1.8 (−2.9 to −0.8) −2.2 (−3.3 to −1.1) −2.0 (−3.2 to −1.0)
Not reported 537 (36%) 385 (19%) 168 (21%) 213 (30%) 319 (5%) 1622 (15%)
Median (IQR) log10HIV-RNA (copies/ml) 5.56 (4.97 to 6.09) 5.23 (4.69 to 5.79) 4.52 (3.81 to 5.26) 5.09 (4.62 to 5.45) 5.27 (4.69 to 5.84)
Not reported 430 (29%) 516 (26%) 459 (58%) 578 (82%) 5856 (>99%) 7839 (72%)
Year of ART initiation
2004 487 414 47 78 261 1287
2005 572 884 123 188 812 2579
2006 433 706 144 101 994 2378
2007 13 11 158 80 1244 1506
>2008 0 0 319 259 2547 3125
Follow-up (child years) 1377.9 1920.5 754.0 671.7 5480.2 10204.3
AT 1 YEAR2
% in follow-up 86.4 92.3 92.0 89.7 87.3 88.6
% transferred 3.3 0.8 0.3 2.5 0 0.8
% LTFU 0.7 1.1 1.3 2.3 3.7 2.5
% deceased 9.6 5.8 6.4 5.5 9.0 8.1
1-year mortality rate
K-M estimate (95% CI) 10.5 (8.9 to 12.4) 6.0 (5.0 to 7.2) 6.8 (5.1 to 8.9) 5.8 (4.2 to 7.9) 9.6 (8.8 to 10.5) 8.6 (8.0 to 9.2)

IQR: interquartile range; LTFU: loss to follw-up; K-M: Kaplan-Meier

1

Note that percentages for particular categories of this variable are calculated with the denominator being the number of children in whom this variable was reported.

2

Note that LTFU and transferred outcomes refer to children LTFU or transferred between 6 and 12 months after anti retroviral therapy (ART) start as children LTFU/TFO within 6 months of ART start were excluded from the main analysis.

The crude and adjusted associations between prognostic variables and mortality are shown in Table 2 (a). In adjusted analyses there was a significant interaction between age and both CD4% (p=0.028) and WAZ (p=0.002). The effect of an increase in either of these variables had a greater impact on reducing mortality risk in older children compared to younger children (Table 2 [b]), although mortality overall was lower for older children.

Table 2(a).

Prognostic variables and mortality for children starting ART and not LTFU or TFO during the first 6 months of treatment in 11 treatment programs in Southern Africa. Note: results shown only for imputed data; results for complete cases were similar.

Analysis based on imputed data (25 multiple imputation datasets)

Variable Available data N(%) missing Person years N deaths Crude mortality HR
n=10875
Adjusted mortality HR
n=10875

HR (95% CI) p-value HR (95% CI) p-value
Age (years) 877

 < 1 1427 1221 260 1 1
 1 1779 1588 261 0.74 (0.62 to 0.88) 0.001 0.72 (0.39 to 1.33)1 0.291
 2–4 3293 3152 187 0.27 (0.23 to 0.33) <0.001 0.31 (0.17 to 0.57)1 <0.001
 5–10 4376 4243 169 0.18 (0.15 to 0.22) <0.001 0.28 (0.17 to 0.46)1 <0.001

Sex

 Male 5515 5178 438 1 1
 Female 5360 5026 439 1.04 (0.91 to 1.18) 0.598 1.06 (0.93 to 1.21) 0.372

Stage 1615 (15)

 Less advanced 2583 2503 111 1 1
 Advanced 6677 6179 650 2.21 (1.80 to 2.71) <0.001 1.93 (1.58 to 2.37) <0.001

CD4 percent 3480 (32)

 less than 5 854 766 114 1 1
 5 to 9 1588 1479 143 0.72 (0.57 to 0.91) 0.006 0.84 (0.51 to 1.38)2 0.490
 10 to 14 2036 1936 141 0.59 (0.47 to 0.75) <0.001 0.57 (0.34 to 0.96)2 0.035
 ≥ 15 2917 2733 238 0.64 (0.52 to 0.79) <0.001 0.59 (0.38 to 0.92)2 0.021

Anemia 3985 (36)

 Severe 588 509 104 1 1
 Moderate 794 710 113 0.80 (0.62 to 1.03) 0.080 0.83 (0.64 to 1.08) 0.165
 Mild/none 5508 5206 401 0.43 (0.35 to 0.53) <0.001 0.61 (0.49 to 0.75) <0.001

Weight-for-age z-score 1622 (15)

 ≤ − 3 2715 2373 451 1 1
 −3 to −2 1975 1852 163 0.51 (0.43 to 0.61) <0.001 0.90 (0.62 to 1.29)3 0.560
 −2 to −1 2258 2191 87 0.27 (0.21 to 0.33) <0.001 0.83 (0.57 to 1.21)3 0.332
 ≥ −1 2305 2251 70 0.21 (0.17 to 0.27) <0.001 0.52 (0.35 to 0.80)3 0.003

HR: hazard ratio; CI: confidence interval

1

HRs are shown for a child with CD4 percent <5%;

2

HRs are shown for a child < 1 year of age;

3

HRs are shown for a child <1 year of age with CD4 percent <5%

Table 2(b).

Hazard ratios for the associations between age, CD4% and weight-for-age z-score (WAZ) adjusted for these variables as well as WHO Clinical Stage and sex, taking into account the interactions between age and CD4% and age and WAZ for data with missing covariate values modeled using multiple imputation (N=10875)

AGE WAZ CD4% <5 CD4% 5–9.9 CD4% 10–14.9 CD4% ≥ 15
< 1 year < −3 1 0.87 (0.52 to 1.44) 0.60 (0.36 to 1.00) 0.62 (0.40 to 0.98)
−3 to −2 0.90 (0.62 to 1.29) 0.78 (0.41 to 1.45) 0.54 (0.29 to 0.98) 0.56 (0.32 to 0.97)
−2 to −1 0.83 (0.57 to 1.21) 0.72 (0.38 to 1.36) 0.50 (0.26 to 0.95) 0.52 (0.29 to 0.92)
≥ −1 0.52 (0.35 to 0.80) 0.45 (0.24 to 0.87) 0.31 (0.16 to 0.60) 0.33 (0.18 to 0.59)
1 year < −3 0.88 (0.47 to 1.65) 0.60 (0.35 to 1.02) 0.73 (0.43 to 1.22) 0.49 (0.30 to 0.80)
−3 to −2 0.66 (0.34 to 1.31) 0.45 (0.26 to 0.81) 0.55 (0.31 to 0.98) 0.37 (0.22 to 0.63)
−2 to −1 0.31 (0.15 to 0.67) 0.22 (0.11 to 0.43) 0.26 (0.12 to 0.51) 0.18 (0.09 to 0.33)
≥ −1 0.26 (0.11 to 0.57) 0.17 (0.09 to 0.35) 0.21 (0.11 to 0.41) 0.14 (0.07 to 0.27)
2–4 years < −3 0.50 (0.27 to 0.92) 0.43 (0.25 to 0.73) 0.26 (0.15 to 0.46) 0.32 (0.19 to 0.52)
−3 to −2 0.21 (0.11 to 0.42) 0.18 (0.10 to 0.33) 0.11 (0.06 to 0.15) 0.13 (0.07 to 0.24)
−2 to −1 0.09 (0.04 to 0.20) 0.08 (0.04 to 0.16) 0.05 (0.02 to 0.10) 0.06 (0.03 to 0.12)
≥ −1 0.10 (0.05 to 0.22) 0.09 (0.04 to 0.17) 0.05 (0.03 to 0.11) 0.06 (0.03 to 0.12)
5–9 years < −3 0.41 (0.25 to 0.68) 0.20 (0.11 to 0.35) 0.14 (0.08 to 0.27) 0.14 (0.08 to 0.26)
−3 to −2 0.22 (0.12 to 0.40) 0.11 (0.06 to 0.20) 0.07 (0.04 to 0.15) 0.08 (0.04 to 0.15)
−2 to −1 0.12 (0.06 to 0.23) 0.06 (0.03 to 0.11) 0.04 (0.02 to 0.08) 0.04 (0.02 to 0.08)
≥ −1 0.08 (0.04 to 0.18) 0.04 (0.02 to 0.09) 0.03 (0.01 to 0.06) 0.03 (0.01 to 0.06)

The two final models selected by internal-external cross-validation included age (in four categories), clinical stage (two), WAZ (three) and anemia (two), with one model additionally including CD4% (in three categories, Table 3). There were thus 144 risk groups (model with CD4%) or 48 risk groups (model without CD4%). The C-statistics over the entire first year on ART were 0.753 and 0.745 for the models with and without CD4% respectively. Concordance was lower when restricting to the second six months on ART (C-statistics of 0.708 and 0.705 for models with and without CD4% respectively). The R2 values were 32.1% and 31.1% for the whole first year and 23.6% and 23.9% for the second six months for the models with and without CD4% respectively. As hemoglobin was imputed for a large proportion of children, model diagnostics were recalculated restricted to children in whom hemoglobin was recorded. This resulted in slightly higher C-statistics (0.762 [with CD4%] and 0.754 [no CD4%]) and R2 (33.8% [with CD4%] and 32.9% [no CD4%]) for both models with very little difference between the two models.

Table 3.

Adjusted mortality hazard ratios (HR) in the selected best models with CD4% and withoutCD4%



Model with CD4% Model without CD4%


Variable Adjusted mortality HR (95% CI) p-value Adjusted mortality HR (95% CI) p-value


Age
 <1 year 1 1
 1 year 0.78 (0.66 to 0.93) 0.006 0.78 (0.66 to 0.93) 0.005
 2 to 4 years 0.34 (0.28 to 0.41) <0.001 0.35 (0.29 to 0.43) <0.001
 5 to 10 years 0.22 (0.18 to 0.27) <0.001 0.25 (0.2 to 0.3) <0.001
WHO Clinical Stage
 I or II 1 1
 III or IV 1.39 (1.13 to 1.71) 0.002 1.39 (1.13 to 1.72) 0.002
CD4%
 <5% 1 not in model
 5–9.9% 0.69 (0.54 to 0.87) 0.002 not in model
 ≥10% 0.56 (0.45 to 0.68) <0.001 not in model
Weight-for-age z-score
 <−3 1 1
 −3 to −2 0.66 (0.55 to 0.79) <0.001 0.63 (0.53 to 0.76) <0.001
 ≥ −2 0.35 (0.29 to 0.42) <0.001 0.33 (0.27 to 0.4) <0.001
Anemia
 Severe 1 1
 Mild/moderate/none 0.71 (0.57 to 0.88) 0.002 0.7 (0.57 to 0.87) 0.001


HR: hazard ratio; CI: confidence interval

For both models, predicted mortality closely followed observed mortality for five prognostic groups of children with approximately 20% of deaths in each group (Figure 1[a]). Within each region predicted and observed mortality for groups of children with different prognosis were also similar (Figure 1[b]), indicating generalizability of the models. The majority of children (57% [model with CD4%] and 58% [model without CD4%] were in the group with a good prognosis and one-year mortality <5%. Predicted mortality from the model with CD4% closely approximated observed mortality for all regions except Kwazulu-Natal (observed>predicted) and Malawi & Zimbabwe (observed<predicted) (Figure 1[c]). Predicted mortality from the model without CD4% only approximated observed mortality closely for CIDRZ (figure 1[b]). This was the only region where <20% of hemoglobin values were missing. In other regions, hemoglobin may not have been well imputed, particularly in Gauteng and Malawi & Zimbabwe where >60% of values were missing.

Figure 1.

Figure 1

Figure 1

Figure 1

Figure 1 (a): Cumulative mortality predicted from prognostic models (dashed lines) compared with Kaplan-Meier observed mortality (solid lines) for five prognostic groups of children commencing antiretroviral therapy, ranging from worst to best prognosis. Prognostic groups were defined by ranking the children in order of mortality risk according to their risk factors. Mortality risk was estimated by the linear predictor of the prognostic model. The cut-points for each prognostic group were determined so that each group contained approximately 20% of deaths and were ordered from low to high risk. Number of patients in each group (%) is shown.

Figure 1 (b): Cumulative mortality predicted from prognostic models (dashed lines) compared with Kaplan-Meier observed mortality (solid lines) for each region for three prognostic groups of children commencing antiretroviral therapy, ranging from worst to best prognosis. Prognostic groups were defined as for figure 1 (a) but only three prognostic groups were used due to the smaller number of children for individual regions. Each group contains approximately one third of deaths.

Figure 1 (c): Cumulative mortality predicted from prognostic models (solid lines) compared with Kaplan-Meier observed mortality for each region (dashed lines) for patients with the most commonly occurring values of prognostic variables. Patients were age 5–10 years with WHO Stage III/IV disease, CD4≥10% and weight-for-age z-score −3 to −2.

Children age <1 year in clinical stage III/IV with WAZ <−3, severe anemia and CD4% <5 had the highest predicted cumulative mortality at one year (46.3%) and children age 5–10 years in stage I/II with WAZ ≥ −2, no severe anemia and CD4% ≥10 the lowest mortality (1.8%). Predictions for all combinations of prognostic variables are shown in appendix Table S1. The 1780 children LTFU/TFO within the first six months of treatment were younger and had more advanced disease compared to those included (appendix Table S2). In sensitivity analyses that included these children, Kaplan-Meier one-year mortality estimates ranged from 7.2% (censoring children LTFU/TFO at last visit date +90 days) to 11.6% and 14.4% (assuming 30% and 50% mortality in those LTFU/TFO respectively). When developing the prognostic models including all children (censoring children LTFU/TFO at last visit date +90 days) the predicted cumulative one-year mortality for the best and worst prognostic groups ranged from 1.72% to 39.0% (CD4% model) and from 2.1% to 27.7% (model without CD4%). The corresponding C-statistics were 0.741 and 0.733 and R2 values were 29.5% and 28.3% respectively, which were very similar to the main analysis.

DISCUSSION

Main findings

These prognostic models for one year mortality in children commencing ART are generalizable with good discrimination and prognostic separation. The majority of children (>55%) starting ART and remaining in care have a one year mortality risk of ≤5%, with 6% of children having >20% predicted risk of dying. For predicting mortality on ART, low cost prognostic markers such as WAZ and anemia may be almost as good as CD4%.

Overall mortality, loss to follow-up and transfer out

There was substantial heterogeneity between regions in overall mortality rate, disease characteristics and age. Crude mortality was highest in the Western Cape which includes the only exclusively tertiary care treatment sites and has the highest proportion of children <2 years initiating treatment. These sites rapidly transfer children to primary care once they are getting better (12.7% TFO from Western Cape tertiary care sites between six and twelve months on ART)35 which may result in over-estimating mortality. We excluded children LTFU/TFO within six months of starting ART to reduce bias by underascertainment of deaths if these patients had been included and censored. This does not completely remove bias as children classified as LTFU may have died before meeting the LTFU definition and mortality might be higher in children LTFU compared to children remaining in care.11,36,37 Our sensitivity analysis including children LTFU/TFO within six months of ART start showed that assumptions about their mortality impact on estimated overall mortality, and their exclusion may result in underestimated predicted mortality at the program level. Indeed, excluded children were more likely to have characteristics associated with mortality compared to those included in the main analysis.

Comparison to pre-ART prognostic model

While predictors of mortality pre-ART and on ART have been examined in developing and wealthy countries, this is the first prognostic model for children on ART.6,7,9,13,18,38 A similar model for pre-ART mortality has been developed in children from high income settings.8 A weighted score incorporating weight percentile, WHO stage, symptoms, general health rating, total lymphocyte count, packed cell volume and albumin predicted mortality well with a C-statistic of 0.852 (higher than the values of 0.753 and 0.745 for our models with and without CD4% respectively). The study also showed that CD4% can be replaced by simpler measures to predict pre-ART mortality and so could be applied to resource-limited settings where CD4% is not routinely available. However, as expected from high income settings, children had less advanced disease. Furthermore, the mothers of all children participated in a randomized trial and the model may not be applicable to routine care in resource-limited settings.

Comparison with adult model

Our pediatric model compares favorably with the adult model for resource-limited settings (higher R2 and C-statistic).11 This is probably due to the powerful prognostic value of age in children. Mortality declines rapidly with increasing age in all children, irrespective of HIV-infection. In the context of ART eligibility in a perinatally HIV-infected child only after “waiting” for disease severity criteria to be met, the age at therapy start is a proxy for rate of disease progression since birth and thus a strong prognostic factor.3 The model predictions may therefore not be applicable to children <5 years who start ART without clinical/immunological progression as recommended in the WHO guidelines.39,40 The Children with HIV Early Antiretroviral Therapy (CHER) trial demonstrated a better prognosis in infants starting ART before disease progression.41

Utility of the models

The likely mortality of a patient with a particular set of characteristics can be determined using the supplementary tables in this paper. This may be useful for guiding clinicians and patients regarding prognosis, and for risk stratification. In a similar way the HIV Pediatric Prognostic Markers Collaborative Study (HPPMCS) online calculator for pre-ART mortality has been used to guide decisions on when to start ART in Europe.6 Our models are useful for program planning, and their generalizability makes them useful for comparing outcomes across different programs after adjusting appropriately for different patient disease severity.

Strengths and limitations

To our knowledge this is the first pediatric prognostic model of mortality on ART and is based on a very large cohort across a range of settings. The large number of missing values for some variables (e.g. hemoglobin) is a limitation. In particular, this limited our ability to determine whether a model based on hemoglobin alone was as good as including CD4% and hemoglobin for prognostic purposes. Due to imputation of a large proportion of hemoglobin values for all regions except Zambia, there was misfit of predicted mortality at the level of the region for the model without CD4% (Figure 1[c]]). However, there was little difference in measures of fit when restricting to patients in whom hemoglobin was measured, and the fit of the models with and without CD4% were comparable in Zambia for which <20% of hemoglobin values were imputed. In addition, associations with mortality were similar if missing values were imputed or a complete case analysis was performed. This, together with the cross validation, underlines the robustness of our findings. The majority of children in this analysis started ART with advanced disease, hence we were unable to examine mortality for higher values of CD4% and WAZ. Nevertheless, the model predictions are applicable to most children starting ART as the majority of children in this region still commence ART with advanced disease.42,43

Despite the fact that many sites from different settings are represented, the good fit between predicted and observed mortality for different prognostic groups (figure 1[a]) may be driven by the large MoH-CIDRZ region where concordance at a regional level was also good. Discrepancies between predicted and observed mortality for other regions may be due to differences in LTFU and mortality ascertainment, proportion of imputed data, differences in prognostic variables not measured or included in the models or differences in background mortality, access to health services and models of care. Other factors such as nutrition supplements, co-trimoxazole prophylaxis, co-infections including malaria, first-line regimen, adherence, HIV-1 RNA and exposure to vertical transmission prevention regimens may be associated with outcomes in children.15,4447 However, these factors are often not easily available. There is a trade-off between models which would be more accurate but less applicable and useful in general health care settings in low-income countries. Poor availability of any of the variables in our models would limit their utility, hence developing models both with and without CD4 percent. This may be increasingly important if programs phase out CD4 monitoring with increasing emphasis on universal ART for children irrespective of CD4 values and on routine HIV-RNA monitoring. In this respect, the poor availability of hemoglobin values is a concern, however likely reflects failure to record rather than measure these values. Many of the sites that had low proportions of hemoglobin recorded have reasonable access to laboratory testing or point of care hemoglobinometers and it is likely that hemoglobin values would be available for a clinician wanting to use them for prognostication. In addition all children came from largely urban regions and all sites had medical record systems available, limiting generalizability to less well-resourced cohorts and primary care facilities where pediatric ART increasingly occurs.

Supplementary Material

1

Acknowledgments

Sources of support:

We received funding support for the study from the Eunice Kennedy Shriver National Institute of Child Health and Human Development and the National Institute of Allergy and Infectious Diseases (Grant Number 2U01AI069924-06; PIs: Egger and Davies). Margaret May is funded by the UK Medical Research Council (grant MR/J002380/1). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. HM has received grant funding from GSK/ViiV and Tibotec for investigational products unrelated to the current manuscript. KT has been paid by Abbott for teaching activities unrelated to this manuscript.

We thank the children whose data were used in this analysis, as well as their caregivers. We also thank all staff at participating sites for providing patient care and preparation of data contributed to the IeDEA-SA collaboration. Thanks to Nicola Maxwell & Fritz Keiser for data management and Morna Cornell and Claire Graber for project management. This study was supported by the National Institutes of Health (National Institute of Allergy and Infectious Diseases and the Eunice Kennedy Shriver National Institute of Child Health and Human Development; grant number 2U01AI069924; PIs: Egger and Davies). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

IeDEA Southern Africa Steering Group

Maureen Wellington, Newlands Clinic, Harare, Zimbabwe; Brian Eley, Red Cross Children’s Hospital, Cape Town, South Africa; Christiane Fritz, SolidarMed Zimbabwe, Zimbabwe; Kathryn Stinson, Khayelitsha ART Programme and Médecins Sans Frontières, Cape Town, South Africa; Janet Giddy, McCord Hospital, Durban, South Africa; Matthew Fox, Themba Lethu Clinic, Helen Joseph Hospital, Johannesburg, South Africa; Sabine Heinrich, SolidarMed Lesotho; Christopher Hoffmann, Aurum Institute for Health Research, South Africa; Harry Moultrie, Wits Reproductive Health and HIV Institute, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg and Harriet Shezi Children’s Clinic, Chris Hani Baragwanath Hospital, Soweto, South Africa; James Ndirangu, Hlabisa HIV Treatment and Care Programme, South Africa; Sabrina Pestilli, SolidarMed Mozambique, Mozambique; Sam Phiri, Lighthouse Clinic, Lilongwe, Malawi; Hans Prozesky, Tygerberg Academic Hospital, Stellenbosch, South Africa; Benjamin Chi, Center for Infectious Disease Research in Zambia, Zambia; Karl Technau, Empilweni Service and Research Unit, Rahima Moosa Mother and Child Hospital, University of the Witwatersrand, Johannesburg, South Africa; Robin Wood, Gugulethu and Masiphumelele ART Programmes and Desmond Tutu HIV Centre, Cape Town, South Africa.

Footnotes

Conflicts of interest: No conflicts of interest were declared by the other authors.

References

  • 1.WHO. GLOBAL HIV/AIDS RESPONSE. Epidemic update and health sector progress towards Universal Access. [Accessed July 20, 2012];Progress Report 2011. 2011 Available at: http://whqlibdoc.who.int/publications/2011/9789241502986_eng.pdf.
  • 2.Anglaret X, Minga A, Gabillard D, et al. AIDS and non-AIDS morbidity and mortality across the spectrum of CD4 cell counts in HIV-infected adults before starting antiretroviral therapy in Cote d’Ivoire. Clin Infect Dis. 2012;54:714–723. doi: 10.1093/cid/cir898. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Davies M, Keiser O, Technau K, et al. Outcomes of the South African National Antiretroviral Treatment (ART) programme for children - The IeDEA Southern Africa Collaboration. S Afr Med J. 2009;99:730–737. [PMC free article] [PubMed] [Google Scholar]
  • 4.Bolton-Moore C, Mubiana-Mbewe M, Cantrell RA, et al. Clinical outcomes and CD4 cell response in children receiving antiretroviral therapy at primary health care facilities in Zambia. JAMA. 2007;298:1888–1899. doi: 10.1001/jama.298.16.1888. [DOI] [PubMed] [Google Scholar]
  • 5.Sutcliffe CG, van Dijk JH, Bolton C, Persaud D, Moss WJ. Effectiveness of antiretroviral therapy among HIV-infected children in sub-Saharan Africa. Lancet Infect Dis. 2008;8:477–489. doi: 10.1016/S1473-3099(08)70180-4. [DOI] [PubMed] [Google Scholar]
  • 6.Dunn D HIV Paediatric Prognostic Markers Collaborative Study Group. Short-term risk of disease progression in HIV-1-infected children receiving no antiretroviral therapy or zidovudine monotherapy: a meta-analysis. Lancet. 2003;362:1605–1611. doi: 10.1016/s0140-6736(03)14793-9. [DOI] [PubMed] [Google Scholar]
  • 7.Dunn D, Woodburn P, Duong T, et al. Current CD4 cell count and the short-term risk of AIDS and death before the availability of effective antiretroviral therapy in HIV-infected children and adults. J Infect Dis. 2008;197(3):398–404. doi: 10.1086/524686. [DOI] [PubMed] [Google Scholar]
  • 8.Patel K, Weinberg GA, Buchacz K, et al. Simple Pediatric AIDS Severity Score (PASS): a pediatric severity score for resource-limited settings. J Acquir Immune Defic Syndr. 2006;43(5):611–617. doi: 10.1097/01.qai.0000242454.97650.72. [DOI] [PubMed] [Google Scholar]
  • 9.Cross Continents Collaboration for Kids (3Cs4kids) Analysis and Writing Committee. . Markers for predicting mortality in untreated HIV-infected children in resource-limited settings: a meta-analysis. AIDS. 2008;22:97–105. doi: 10.1097/01.aids.0000302262.51286.a5. [DOI] [PubMed] [Google Scholar]
  • 10.Egger M, May M, Chene G, et al. Prognosis of HIV-1-infected patients starting highly active antiretroviral therapy: a collaborative analysis of prospective studies. Lancet. 2002;360:119–129. doi: 10.1016/s0140-6736(02)09411-4. [DOI] [PubMed] [Google Scholar]
  • 11.May M, Boulle A, Phiri S, et al. Prognosis of patients with HIV-1 infection starting antiretroviral therapy in sub-Saharan Africa: a collaborative analysis of scale-up programmes. Lancet. 2010;376:449–457. doi: 10.1016/S0140-6736(10)60666-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.May M, Sterne JA, Sabin C, et al. Prognosis of HIV-1-infected patients up to 5 years after initiation of HAART: collaborative analysis of prospective studies. AIDS. 2007;21:1185–1197. doi: 10.1097/QAD.0b013e328133f285. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.The KIDS-ART-LINC Collaboration. Low risk of death, but substantial program attrition, in pediatric treatment cohorts in sub-Saharan Africa. J Acquir Immune Defic Syndr. 2008;15:523–531. doi: 10.1097/QAI.0b013e31818aadce. [DOI] [PubMed] [Google Scholar]
  • 14.Fassinou P, Elenga N, Rouet F, et al. Highly active antiretroviral therapies among HIV-1-infected children in Abidjan, Cote d’Ivoire. AIDS. 2004;18:1905–1913. doi: 10.1097/00002030-200409240-00006. [DOI] [PubMed] [Google Scholar]
  • 15.Eley B, Davies MA, Apolles P, et al. Antiretroviral treatment for children. S Afr Med J. 2006;96:988–993. [PubMed] [Google Scholar]
  • 16.Davies M, Egger M, Keiser O, Boulle A. Paediatric antiretroviral treatment programmes in sub-Saharan Africa: a review of published clinical studies. Afr J AIDS Res. 2009;8:329–338. doi: 10.2989/AJAR.2009.8.3.9.930. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Peacock-Villada E, Richardson BA, John-Stewart GC. Post-HAART outcomes in pediatric populations: comparison of resource-limited and developed countries. Pediatrics. 2011;127:e423–441. doi: 10.1542/peds.2009-2701. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Walker AS, Prendergast AJ, Mugyenyi P, et al. Mortality in the year following antiretroviral therapy initiation in HIV-infected adults and children in Uganda and Zimbabwe. Clin Infect Dis. 2012;55:1707–1718. doi: 10.1093/cid/cis797. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Brady MT, Oleske JM, Williams PL, et al. Declines in mortality rates and changes in causes of death in HIV-1-infected children during the HAART era. J Acquir Immune Defic Syndr. 2010;53:86–94. doi: 10.1097/QAI.0b013e3181b9869f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Fenner L, Brinkhof M, Keiser O, et al. Early mortality and loss to follow-up in HIV-infected children starting antiretroviral therapy in Southern Africa. J Acquir Immune Defic Syndr. 2010;54:524–532. doi: 10.1097/QAI.0b013e3181e0c4cf. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Panel on Antiretroviral Therapy and Medical Management of HIV-Infected Children. [Accessed September 1, 2012];Guidelines for the Use of Antiretroviral Agents in Pediatric HIV Infection. 2011 Available at http://aidsinfo.nih.gov/ContentFiles/PediatricGuidelines.pdf.
  • 22.Musoke PM, Young AM, Owor MA, et al. Total lymphocyte count: not a surrogate marker for risk of death in HIV-infected Ugandan children. J Acquir Immune Defic Syndr. 2008;49:171–178. doi: 10.1097/QAI.0b013e318183a92a. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Egger M, Ekouevi DK, Williams C, et al. Cohort Profile: The international epidemiological databases to evaluate AIDS (IeDEA) in sub-Saharan Africa. Int J Epidemiol. 2012;41:1256–64. doi: 10.1093/ije/dyr080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.WHO. [Accessed November 23, 2008];The WHO Child Growth Standards. 2006 Available at: http://www.who.int/childgrowth/en/
  • 25.Division of AIDS. Division of AIDS Table for Grading the Severity of Adult and Pediatric Adverse Events Version 1.0, December 2004. [Accessed January 2, 2012];Clarification August 2009. 2009 Available at: http://rsc.tech-res.com/Document/safetyandpharmacovigilance/Table_for_Grading_Severity_of_Adult_Pediatric_Adverse_Events.pdf.
  • 26.Royston P. Multiple imputation of missing values. Stata J. 2004;4:227–241. [Google Scholar]
  • 27.Royston P. Multiple imputation of missing values: further update of ice, with am emphasis on interval censoring. Stata J. 2009;7:445–464. [Google Scholar]
  • 28.Sterne J, White I, Carlin J, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;29:b2393. doi: 10.1136/bmj.b2393. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011;30:377–399. doi: 10.1002/sim.4067. [DOI] [PubMed] [Google Scholar]
  • 30.Royston P, Parmar MK. Flexible parametric proportional-hazards and proportional-odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects. Stat Med. 2002;21:2175–2197. doi: 10.1002/sim.1203. [DOI] [PubMed] [Google Scholar]
  • 31.May M, Royston P, Egger M, Justice AC, Sterne JA. Development and validation of a prognostic model for survival time data: application to prognosis of HIV positive patients treated with antiretroviral therapy. Stat Med. 2004;23:2375–2398. doi: 10.1002/sim.1825. [DOI] [PubMed] [Google Scholar]
  • 32.Royston P, Parmar MK, Sylvester R. Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer. Stat Med. 2004;23:907–926. doi: 10.1002/sim.1691. [DOI] [PubMed] [Google Scholar]
  • 33.Royston P, Sauerbrei W. A new measure of prognostic separation in survival data. Stat Med. 2004;23:723–748. doi: 10.1002/sim.1621. [DOI] [PubMed] [Google Scholar]
  • 34.Royston P. Explained variation for survival models. Stata J. 2006;6:83–96. [Google Scholar]
  • 35.Eley B, Nuttall J. Antiretroviral therapy for children: challenges and opportunities. Ann Trop Paediatr. 2007;27:1–10. doi: 10.1179/146532807X170448. [DOI] [PubMed] [Google Scholar]
  • 36.Braitstein P, Songok J, Vreeman RC, et al. “Wamepotea” (they have become lost): outcomes of HIV-positive and HIV-exposed children lost to follow-up from a large HIV treatment program in western Kenya. J Acquir Immune Defic Syndr. 2011;57:e40–46. doi: 10.1097/QAI.0b013e3182167f0d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Weigel R, Hochgesang M, Brinkhof MW, et al. Outcomes and associated risk factors of patients traced after being lost to follow-up from antiretroviral treatment in Lilongwe, Malawi. BMC Infect Dis. 2011;11:31. doi: 10.1186/1471-2334-11-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Leroy V, Malateste K, Rabie H, et al. Outcomes of antiretroviral therapy in children in Asia and Africa: a comparative analysis of the IeDEA pediatric multiregional collaboration. J Acquir Immune Defic Syndr. 2013;62:208–219. doi: 10.1097/QAI.0b013e31827b70bf. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.WHO. Antiretroviral therapy for HIV infection in infants and children: Towards universal access. [Accessed October 19, 2010];Recommendations for a public health approach: 2010 revision. 2010 Available at: http://www.who.int/hiv/pub/paediatric/infants2010/en/index.html. [PubMed]
  • 40.WHO. [Accessed September 12, 2013];Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. 2013 http://www.who.int/hiv/topics/treatment/technical/en/index.html.
  • 41.Violari A, Cotton MF, Gibb DM, et al. Early antiretroviral therapy and mortality among HIV-infected infants. N Engl J Med. 2008;359:2233–2244. doi: 10.1056/NEJMoa0800971. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Fatti G, Bock P, Eley B, Mothibi E, Grimwood A. Temporal trends in baseline characteristics and treatment outcomes of children starting antiretroviral treatment: an analysis in four provinces in South Africa, 2004–2009. J Acquir Immune Defic Syndr. 2011;58:e60–67. doi: 10.1097/QAI.0b013e3182303c7e. [DOI] [PubMed] [Google Scholar]
  • 43.Davies M, Phiri S, Wood R, et al. Temporal trends in the characteristics of children at antiretroviral therapy initiation in Southern Africa: The International epidemiologic Databases to Evaluate AIDS Southern Africa (IeDEA-SA) Collaboration. PLoS ONE. 2013 doi: 10.1371/journal.pone.0081037. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Davies M, Moultrie H, Eley B, et al. Virologic failure and second-line antiretroviral therapy in children in South Africa: The IeDEA Southern Africa collaboration. J Acquir Immune Defic Syndr. 2011;56:270–278. doi: 10.1097/QAI.0b013e3182060610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Duggan C, Fawzi W. Micronutrients and child health: studies in international nutrition and HIV infection. Nutr Rev. 59(11):358–369. doi: 10.1111/j.1753-4887.2001.tb06963.x. [DOI] [PubMed] [Google Scholar]
  • 46.Walker AS, Mulenga V, Ford D, et al. The impact of daily cotrimoxazole prophylaxis and antiretroviral therapy on mortality and hospital admissions in HIV-infected Zambian children. Clin Infect Dis. 2007;44(10):1361–1367. doi: 10.1086/515396. [DOI] [PubMed] [Google Scholar]
  • 47.Violari A, Lindsey JC, Hughes MD, et al. Nevirapine versus ritonavir-boosted lopinavir for HIV-infected children. N Engl J Med. 2012;366:2380–2389. doi: 10.1056/NEJMoa1113249. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES