Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Jan 14.
Published in final edited form as: AIDS. 2018 Jan 14;32(2):189–204. doi: 10.1097/QAD.0000000000001683

Neuropsychological performance in African children with HIV enrolled in a multi-site anti-retroviral clinical trial

Michael J BOIVIN a,#, Linda BARLOW-MOSHA b, Miriam CHERNOFF c, Barbara LAUGHTON d, Bonnie ZIMMER e, Celeste JOYCE f, Mutsa BWAKURA-DANGAREMBIZI g, Mmule RATSWANA h, Nasreen ABRAHAMS i, Lee FAIRLIE h, Hermien GOUS h, Portia KAMTHUNZI j, Katie McCARTHY k, Itziar FAMILIAR-LOPEZ l, Patrick JEAN-PHILLIPPE l, Joan COETZEE d, Avy VIOLARI f, Mark COTTON d, Paul PALUMBO m; the IMPAACT P1104s Study Teaml
PMCID: PMC5736412  NIHMSID: NIHMS917332  PMID: 29112069

Abstract

Objective & Design

Children with HIV infection (HIV+) are at neuropsychological risk, but few studies have evaluated this at multiple sites in low and middle income countries (LMICs). We compared neuropsychological outcomes at enrollment (> 5 yrs age) among HIV+, HIV-uninfected perinatally-exposed (HEU), and HIV unexposed (HU) children from 4 sub-Saharan countries.

Methods

IMPAACT P1060 compared Nevirapine (NVP) versus Lopinavir/Ritonavir (LPVr)-based ART in HIV-infected children 6 to 35 months of age. This study (P1104s) enrolled P1060 children at 5–11 years of age and evaluated their neuropsychological performance over 2 years using the KABC-II, TOVA, BOT-2, and parent-reported BRIEF . Cohorts were compared using GEE least-squares means adjusted for site, child age and gender, and personal and social characteristics for child and caregiver.

Results

611 (246 HIV+, 183 HEU, 182 HU) of the 615 enrolled at 6 sites (South Africa [3], Zimbabwe, Malawi, Uganda) were available for analysis. Mean age was 7.2 years, 48% male, 69% in school. Unadjusted and adjusted comparisons were consistent. HIV+ children performed significantly worse than HEU and HUU on KABC-II, TOVA, BOT-2 (P< 0.001), but not on the BRIEF. HUU and HEU cohorts were comparable on cognitive outcomes. HIV+ children initiated on ARV treatment before one year of age had significantly better only in BOT-2 total motor proficiency compared to those started after.

Conclusions

Significant cognitive deficits were documented among HIV+ children. Earlier HIV treatment, neuropsychological monitoring and rehabilitative interventions are needed. Subsequent testing for 2 more years will help evaluate how HIV infection and exposure affect the developmental trajectory.

Keywords: HIV, HIV exposed uninfected, Neuropsychology, Attention, Memory, Learning, Executive Function, Africa, Child Development

INTRODUCTION

Children with HIV infection (HIV+) are at neuropsychological risk, with studies showing an association of HIV with cognitive and motor dysfunction in perinatally infected children.[1] [2] However, few studies have evaluated neuropsychological outcomes among perinatally infected children who received combination antiretroviral treatment (ARV treatment) prior to 3 years of age in resource-poor settings. While ARV treatment has increased survival of HIV infected children into adolescence and young adulthood, alone it is insufficient to reverse the neurodevelopmental consequence of HIV infection.[3]

The purpose of this observational multicenter longitudinal study is to compare neuropsychological outcomes cross-sectionally between the perinatally HIV+, HEU and HUU children in sub-Saharan Africa with language and cultural differences. HUU children without a history of significant risk for brain injury were included as a reference group in order to better understand the neuropsychological risk from HIV infection versus exposure but not infection. We present the neuropsychological performance of HIV+ children compared to age-matched HEU and HUU controls, after statistically adjusting for a number of descriptive factors that may influence neuropsychological outcomes. By doing so, we address a key gap in the research literature, by evaluating whether neuropsychological function in African children with HIV is diminished at school age (5 to 12 yrs) even ARV treatment is initiated at an early age (< 3 yrs) with good clinical monitoring and treatment support.

METHODS

Children were recruited into three cohorts: (1) HIV+ children (N=246) participating in P1060 Version 5.0; (2) HEU (N=183) and (3) HU (N=182). All P1060 children (HIV+) were eligible for enrollment in the present study with over 95% participating. These were HIV status-verified and medically well-characterized children from 5 to 11 yrs of age at enrollment in the present follow-up observational study. These were perinatally infected children from a randomized controlled trial (RCT) comparing ARV nevirapine versus Lopinavir/Ritonavir upon HIV diagnosis in infancy or very early childhood.[46] Neuropsychological comparison (Table 1) between the ARV nevirapine versus Lopinavir/Ritonavir for the children with HIV will be presented in a separate report to be published at a later time. By the time of enrollment in the present observational follow-up study, most HIV children in the NVP arm had been switched to 2nd line cART and 96% were virally suppressed at the time of baseline neuropsychological assessment. In the present study, HIV+ children had good immunological and virological status, with over 95% having CD4% ≥ 25% and viral load ≤ 400 copies/mL (Table 3). However, most of the HIV+ children in the present study presented with major signs of disease at the time of enrollment in the ARV treatment clinical trial in early childhood, and therefore were documented to have had these prior occurrences when enrolled in the present follow-up study (WHO Stage I=38 (15%); Stage II=58 (24%), Stage III=137 (56%); Stage IV=13 (5%)) (Table 3).

Table 1.

Core models for the analysis of the association between HIV status and neuropsychological outcomes

Test Key Outcome Covariates*
Kaufman Assessment Battery for Children (KABC-II) Mental Processing Index standard score (MPI; mpisc) Child is in school (NPW0051 Q7 present school year>0; inschool), WHO-body mass index z-score (who_bmiz), Caregiver completed high school (hsgrad), Major source of household income includes a social grant (grant), household residential zone (curzone), household has working refrigerator (fridge), MICS disability score (dis_score)
Bruininks–Oseretsky Test of Motor Proficiency (BOT-2) Total point score (totalsc) Child is in school (NPW0051 Q7 present school year>0; inschool), WHO-height z-score (who_htz), caregiver completed high school (hsgrad), household residential zone (curzone), household has working refrigerator (fridge) MICS disability score (dis_score), MICS development score (dev_score)
Behavior Rating Inventory of Executive Function (BRIEF) Global Executive Composite (GEC; gects) WHO-body mass index z-score (who_bmiz), caregiver completed at least of one year of trade, college or other specialized learning (higheduc), household residential zone (curzone), caregiver anxiety (anxiety), caregiver depression (depression), MICS disability score (dis_score), MICS development score (dev_score).
Test of Variables of Attention (TOVA) D Prime (dprime) Child is in school (NPW0051 Q7 present school year>0; inschool), Major source of household income includes a social grant (grant), household residential zone (curzone), MICS development score (dev_score)
*

All models also included cohort, age at study entry in years, sex and clinical site

Table 3.

HIV disease characteristics and ARV regimen at entry

Characteristic Total (N=246)
Age at ARV initiation (years), (Median, IQR) 1.2 (0.7–2.1)
Age at ARV initiation (months)
 0–<6 mons 3 (1%)
 6–<12 mons 105 (43%)
 12–<18 mons 31 (13%)
 18–<24 mons 38 (15%)
 24–<30 mons 35 (14%)
 30+ mons 34 (14%)
# years on ARVs prior to entry, (Median, IQR) 5.6 (4.9–6.6)
ARV regimen classification (classes)
 NRTI 1 (0%)
 NRTI + NNRTI 78 (32%)
 NRTI + PI 165 (67%)
 PI 2 (1%)
ARV regimen classification (HAART)
 HAART w/ PI 164 (67%)
 HAART w/out PI 78 (32%)
 Non-HAART ARV 4 (2%)
WHO disease stage
 Clinical stage I 38 (15%)
 Clinical stage II 58 (24%)
 Clinical stage III 137 (56%)
 Clinical stage IV 13 (5%)
CD4 % (Median, IQR) 39 (34–43)
 0–<15% 1 (0%)
 15%–<25% 6 (2%)
 25%+ 239 (97%)
HIV-1 RNA (cp/ml; Median IQR) 400 (400–400)
 <=400 cp/ml 235 (96%)
 >400 – 5000 cp/ml 8 (3%)
 >5000 cp/ml 3 (1%)
P1060 treatment arm
 NVP 126 (51%)
 LPV/r 120 (49%)
P1060 strata: NVP Exposed 86 (35%)
HIV subtype
 A1 24 (10%)
 C 192 (84%)
 D 11 (5%)
 R10 1 (0%)
 R43 1 (0%)
 Not reported 17

1 HIV-1 RNA censored at 400 copies/ml regardless of assay used

Sample sizes for the HEU and HU cohorts were determined on the basis of sample size needed for 80% statistical power calculated using a prior study comparing HIV, HEU, HUU Ugandan cohorts for the principal Kaufman Assessment Battery for Children, 2nd edition (KABC-II; Mental Processing Index or MPI), Tests of Variables of Attention (TOVA; D prime score) and Bruininks-Oseretsky Test, 2nd edition (BOT-2) of motor proficiency total score.[1] Enrollment of the HUU children took place at vaccination and outpatient treatment clinics at each study site, consisting of a convenience sample age matched to the P1060 HIV+ children.[6] These children were medically screened for prior hospitalizations that could involve brain injury (e.g., cerebral malaria, meningitis, head trauma) or severe malnutrition, and also excluded if they screened positive for any developmental disability. HEU children were born to HIV+ mothers recruited from the same households, extended families, or neighborhoods/communities as the P1060 participants. HEU children were not excluded on the basis of prior medical history or indication of developmental delay. The HIV status of the mother at the time of birth was verified in the medical record.

Enrollment took place from October, 2013 to mid-December, 2014. The six study sites were: 1) Wits RHI Shandukani clinic, Johannesburg, South Africa (Johannesburg RSA; principal local languages of Sesotho and Zulu); 2) Chris Hani HIV Unit, Soweto, South Africa (Soweto RSA; principal local languages of Sesotho and Zulu); 3) Family Clinical Research Unit, Cape Town, South Africa (Tygerberg RSA; principal local languages of Xhosa, Afrikaans, English); 4) Kamuzu Central Hospital HIV clinic (Lilongwe Malawi; principal local language of Chichewa); 5) Makerere University – Johns Hopkins University Clinic Mulago National Referral Hospital (Kampala Uganda; principal local language of Luganda); and 6) Parirenyatwa General Hospital (Harare Zimbabwe; principal local language of Shona).

The institutional review board (IRB) approval for this study was obtained from the human subjects’ protection in research regulatory committee (Institutional Review Board or IRB) at each study site, and where applicable the corresponding ministry of health in the host country, and the university partner in the United States for each study site. Informed consent was obtained from parents or primary caregivers with additional assent from children >7 years based on country regulations.

Participants were assessed with a neuropsychological assessment battery of tests at entry and at two yearly follow-up visits. The present study reports only the cross-sectional comparisons of the baseline assessment among the study cohorts. A longitudinal comparison among cohorts of all three assessments over the two-year study period will be presented separately in paper to be published at a later time. Additional data were collected regarding the demographic and socioeconomic status of the household, the child’s anthropomorphic measures, illness history and medications. These are listed in Table 2. Caregivers responded the Hopkins Checklist for Depression/Anxiety (HSCL-25), Behavior Rating Inventory of Executive Function (BRIEF) school-age, UNICEF-sponsored Multiple Indicators Cluster Survey, 4th edition (MICS4) Questionnaire for Child Development, and MICS4 Child Disability Questionnaire. Testers were blinded to the child’s exposure group. Table 1 lists the key outcomes for each of the tests described below.

Table 2.

Child, Caregiver and Household Characteristics at Entry by Cohort

Cohort
Characteristic Total (N=611) HIV-infected (N=246) HIV exposed, uninfected (N=183) HIV unexposed, uninfected (N=182) P-Value
Male sex (n,%) 290 (47.5%) 111 (45.1%) 95 (51.9%) 84 (46.2%) 0.347 (a)
Race <.001 (a)
 Black African 568 (93.0%) 242 (98.4%) 176 (96.2%) 150 (82.4%)
 Coloured/White/Other 43 (7.0%) 4 (1.6%) 7 (3.8%) 32 (17.6%)
Age in years, (Median, IQR) 6.9 (6.2, 8.1) 7.0 (6.3, 7.8) 6.9 (6.1, 8.5) 6.7 (6.1, 8.5) 0.963 (b)
<6 yrs 112 (18.3%) 41 (16.7%) 36 (19.7%) 35 (19.2%) 0.872 (a)
6–<7 yrs 211 (34.5%) 83 (33.7%) 64 (35.0%) 64 (35.2%)
>=7 yrs 288 (47.1%) 122 (49.6%) 83 (45.4%) 83 (45.6%)
In School at testing 404 (68.8%) 165 (70.2%) 119 (66.5%) 120 (69.4%) 0.707 (a)
Unknown/NA 24 11 4 9
School grade
 0 (not in school) 183 (31.2%) 70 (29.8%) 60 (33.5%) 53 (30.6%) 0.007 (a)
 1 174 (29.6%) 85 (36.2%) 45 (25.1%) 44 (25.4%)
 2 114 (19.4%) 52 (22.1%) 30 (16.8%) 32 (18.5%)
 3 58 (9.9%) 19 (8.1%) 18 (10.1%) 21 (12.1%)
 4 51 (8.7%) 8 (3.4%) 23 (12.8%) 20 (11.6%)
 5 7 (1.2%) 1 (0.4%) 3 (1.7%) 3 (1.7%)
WHO BMI z-score, (Median, IQR) −0.09 (−0.66, 0.54) −0.18 (−0.76, 0.43) 0.00 (−0.60, 0.66) −0.09 (−0.65, 0.60) 0.081 (b)
WHO weight z-score, (Median, IQR) −0.52 (−1.10, 0.09) −0.74 (−1.34, −0.21) −0.24 (−0.93, 0.46) −0.41 (−0.93, 0.27) <.001 (b)
WHO height z-score, (Median, IQR) −0.73 (−1.35, 0.04) −1.06 (−1.68, −0.45) −0.51 (−1.05, 0.30) −0.36 (−1.09, 0.22) <.001 (b)
MICS Disability score (%), (Median, IQR) 0 (0, 10) 5 (0, 10) 0 (0, 10) 0 (0, 10) <.001 (b)
MICS Development score (%), (Median, IQR) 78.9 (63.2, 89.5) 76.3 (57.9, 89.5) 78.9 (63.2, 89.5) 78.9 (68.4, 84.2) 0.150 (b)
Caregiver is biological mother 572 (93.6%) 209 (85.0%) 181 (98.9%) 182 (100.0%) <.001 (a)
Caregiver HIV status
 HIV-uninfected 193 (31.9%) 10 (4.2%) 1 (0.5%) 182 (100.0%) <.001 (a)
 HIV-infected 412 (68.1%) 230 (95.8%) 182 (99.5%) 0 (0.0%)
 Not reported 6 6 0 0
Caregiver 5 years or more 563 (92.1%) 223 (90.7%) 169 (92.3%) 171 (94.0%) 0.451 (a)
Caregiver Anxiety score, (Median, IQR) 0.4 (0.2, 1.0) 0.4 (0.2, 0.9) 0.5 (0.1, 1.1) 0.5 (0.2, 1.1) 0.620 (b)
Caregiver Depression, (Median, IQR) score 0.6 (0.3, 1.3) 0.7 (0.3, 1.3) 0.5 (0.2, 1.3) 0.6 (0.3, 1.1) 0.512 (b)
Residential zone
 Rural 109 (17.8%) 51 (20.7%) 29 (15.8%) 29 (15.9%) 0.634 (a)
 Peri-urban 268 (43.9%) 103 (41.9%) 81 (44.3%) 84 (46.2%)
 Urban 234 (38.3%) 92 (37.4%) 73 (39.9%) 69 (37.9%)
Socio-economic index, Median (IQR) 6.0 (4.6, 7.6) 6.0 (4.4, 7.6) 5.7 (4.8, 7.4) 6.2 (4.6, 7.9) 0.322 (b)
(a)

Chi-Square Test

(b)

Kruskal-Wallis Test

Socio-economic index (0–10) is a composite of water and fuel sources, having a refrigerator, income characteristics, caregiver work and education status.

A high score represents a better socio-economic status.

Kaufman Assessment Battery for Children (second edition)

The KABC-II was the principal test for cognitive ability outcomes.[7] It has also been validated in the sub-Saharan African context,[811] and previously adapted for use in pediatric HIV research in the present study site countries of Uganda and South Africa.[1, 1217] Using the Luria model for neuropsychological assessment within the KABC-II, the primary outcome variables were the global scores of Sequential Processing (memory), Simultaneous Processing (visual-spatial processing and problem solving), Learning (immediate and delayed memory), Planning (executive reasoning), Delayed recall, nonverbal index (NVI) (subtests not dependent on the understanding of instructions in English) and Mental Processing Index (MPI) (a composite of the principal cognitive performance domains). The KABC offers the option of scoring using time points, but these were not used in the present assessment.

Bruininks-Oseretsky Test of Motor Proficiency, 2nd edition (BOT-2)

The BOT-2 is one of the most comprehensive instruments for motor assessment,[18] previously used for pediatric HIV assessment in Uganda as motor impairment that often accompanies HIV in children.[1, 19] The short form of this test includes two or three items pertaining to fine motor precision, fine motor integration, manual dexterity, upper-limb coordination, bilateral coordination, balance, running speed and agility, and strength. These are combined into total composite score of motor proficiency, standardized by age and gender using American norms. This total standard score for motor proficiency was the only outcome for this test used in the present analyses.

Test of Variables of Attention (TOVA)

The TOVA is a computerized visual continuous performance test used in the diagnosis and monitoring of children and adults with attention deficit disorders (www.tovatest.com).[20] This test consists of the rapid (tachistoscopic) presentation of a large geometric square on the computer screen with a smaller dark box either in the upper position (signal) or lower position (non-signal). The child is asked to press a switch held in the preferred hand as quickly as possible in response to the signal (measuring vigilance attention), but to withhold responding to the non-signal (measuring impulsivity). Following spoken instructions in the local language and practice trials, the TOVA takes about 11 minutes for children 5 to 5.5 years of age, and 22 minutes to administer for children 5.5 years of age and older. The TOVA had been adapted for pediatric HIV research in Uganda.[1, 13, 15, 19, 21]

The primary outcome variables were response time variability (a sensitive indication of inattention), response time, percent commission errors (impulsivity), percent omission errors (inattention), an attention deficit – hyperactivity disorder (ADHD) index score (missed signals in proportion to incorrect responses to non-signal), and a signal detection measure of overall test performance called D Prime (correct “hits” to signal in proportion to correct non-responses to non-signal).

Behavior Rating Inventory for Executive Function (BRIEF) school-age

The BRIEF-SA (6 to 18 yrs) has 86 items that are completed by the parent or guardian and evaluates behavioral and cognitive behavior problems related to disruption of executive functions of the brain.[22] It has previously been adapted for use in pediatric HIV research in Uganda.[23, 24] The BRIEF was translated into the principal local languages at all study sites (with permission of the publisher including approval of the back translation). The eight scales form two broad indexes, Behavior Regulation (BRI) with three scales, Metacognition Index (MI) with five scales, and these are combined into a Global Executive Composite Score (GEC). The higher the score, the more day-to-day behavior problems related to executive function as reported by the parent/caregiver.

Hopkins Symptoms Checklist (25-items) for Depression/Anxiety (HSCL-25)

The HSCL-25 is used to assess severity of caregiver depression (15 items) and anxiety (10 items).[25, 26] The HSCL-25 has been used in studies of emotional well-being of caregivers of Ugandan children with or affected by HIV.[23, 27] The HSCL was translated in the principal local languages at each of the study sites and read out loud to the mother or principal caregiver in a private setting.

Multiple Indicators Cluster Survey, 4th round (MICS4) Questionnaire for Child Development and for Child Disability

It is crucial to control for quality of home environment whenever measuring developmental and neuropsychological outcomes in at-risk children. At enrollment, we used portions of the Multiple Indicators Cluster Survey (4th round) (MICS4) administered to the principal caregiver of the child. We used the Early Childhood Development portion of the Questionnaire for Children Under 5 (17 items) as a measure of quality of child development environment. We also used the MICS4 Child Disability Questionnaire, derived from the Ten Question Questionnaire (TQQ),[28, 29] in screening HUU children for eligibility as a control group in the present study.

Socio-Economic Status (SES)

Using an assessment of SES previously used in pediatric research in Uganda,[30] information on other members of the household, parental/caregiver status and their education and occupation, physical quality of home environment (e.g., electricity, water source), material possessions (e.g. working refrigerator), and source of income.

Medical History and Physical Development

Medical history and anthropometric measurements (weight, height, mid-upper arm circumference) were collected at the study visit. Anthropometric measures were standardized using WHO norms. The medical history questionnaire included questions on health status (targeted diagnoses, signs/symptoms) and was collected at the clinic before neuropsychological assessment to help ensure that the child was well enough to test.

Statistical Analysis

Descriptive statistics were used to summarize caregiver and child characteristics for all three cohorts, as well as HIV disease status for the HIV+ cohort. Comparability across study cohorts was tested using the Kruskal-Wallis non-parametric tests and chi-square tests. Linear regression analyses using generalized estimating equations (GEE models) were performed to assess differences among study cohorts, first without adjustment followed by adjustment for clinical site, age and sex (partially adjusted models), and finally adjusting for personal and family characteristics (fully adjusted models). Least squares mean estimates by cohort were computed for unadjusted, partly adjusted (adjusted for sex, age, clinical site) and fully adjusted models.

The associations between each potential confounder and each outcome measure were first assessed using unadjusted GEE models. Final regression models were developed in two steps. Initially, all potential confounders with univariate p-values < 0.20 were included in a multivariable model. Clinical site, age and sex were included in these models regardless of the univariate p-values. We excluded sites by cohort interactions based on our previous findings, which suggested their magnitude would not greatly influence interpretation. If more than one related measure was significant in the univariate analysis (e.g., for those related to fuel - ranking of fuel, access to electricity, number of fuel sources), we selected one out of the group for the multivariable model. In such cases, in addition to selecting by ease of interpretation, we also performed multiple regressions including the related factors to assess which had the smallest p-value (most highly significant).

All covariates for which the p-values remained < 0.20 were retained in the final multivariable model, along with cohort, clinical site, age and sex. Backward selection was used until only covariates with p < 0.20 remained in addition to age, sex, cohort and site. A core multivariable model for each test domain was developed in this way for key outcomes and then subsequently used on the remaining outcomes in the domain. For the KABC-II, the key outcome was the mental processing index (MPI), although a separate model was run for the nonverbal test index (NVI). The D-Prime score was the key outcome for the TOVA, and the standardized total motor proficiency score was the only outcome for the BOT-2. The key outcome for BRIEF was the global executive composite (GEC). Table 1 has a list of the principal outcomes for each test along with the covariates from Table 2 that were retained in the adjusted analyses for each outcome for Tables 4 through 6.

Table 4.

Unadjusted and Adjusted Least Squares Means

HIV/negative HIV/exposed HIV/infected

Test Outcome Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted Adj Group diffs

Mean (95% CI) Mean (95% CI) Mean (95% CI) Mean (95% CI) Mean (95% CI) Mean (95% CI) P-Value
KABC_ KABC Sequential 86.52 (84.81,88.23) 85.09 (83.09,87.09) 85.46 (83.79,87.14) 84.28 (82.41,86.15) 79.36 (77.88,80.84) 79.09 (77.39,80.79) <0.001
KABC Simultanous 79.52 (77.79,81.24) 78.07 (76.02,80.12) 78.90 (77.07,80.73) 77.79 (75.78,79.80) 75.24 (73.65,76.83) 74.97 (72.97,76.97) 0.01
KABC Learning 89.87 (87.48,92.25) 87.83 (84.91,90.75) 88.32 (85.88,90.77) 87.24 (84.99,89.48) 82.22 (80.24,84.20) 81.79 (79.39,84.18) <0.001
KABC Planning 80.83 (78.00,83.66) 72.75 (65.35,80.15) 76.81 (73.85,79.77) 69.80 (61.50,78.11) 71.81 (69.99,73.63) 67.44 (61.50,73.39) 0.01
KABC Delayed recall 89.63 (87.16,92.10) 88.57 (86.03,91.11) 89.83 (87.52,92.15) 89.39 (87.35,91.42) 83.15 (81.17,85.13) 83.85 (81.67,86.02) <0.001
KABC Nonverbal index 79.02 (77.32,80.72) 78.82 (77.10,80.54) 78.37 (76.68,80.06) 78.02 (76.27,79.77) 72.05 (70.61,73.49) 72.69 (71.11,74.27) <0.001
KABC Mental processing index 80.89 (79.18,82.60) 79.25 (77.40,81.09) 79.14 (77.50,80.78) 78.12 (76.43,79.81) 73.18 (71.86,74.50) 72.86 (71.24,74.48) <0.001
KABCR KABC Conceptual thinking raw score 13.05 (12.26,13.84) 12.57 (11.81,13.33) 12.84 (12.01,13.68) 12.60 (11.88,13.33) 10.78 (10.04,11.52) 11.02 (10.26,11.77) <0.001
KABC Story completion raw score 6.13 (5.38,6.87) 5.75 (5.09,6.41) 5.34 (4.75,5.93) 5.10 (4.56,5.63) 4.10 (3.75,4.45) 4.25 (3.78,4.71) <0.001
KABC Pattern reasoning raw score 6.33 (5.60,7.06) 5.82 (5.20,6.44) 6.07 (5.36,6.77) 5.79 (5.22,6.37) 4.48 (3.94,5.02) 4.62 (4.00,5.25) <0.001
KABC Face recognition raw score 11.83 (11.28,12.38) 11.23 (10.65,11.82) 11.40 (10.82,11.99) 10.91 (10.29,11.53) 10.29 (9.75,10.83) 10.35 (9.73,10.97) 0.03
KABC Hand movements raw score 7.09 (6.68,7.51) 6.81 (6.37,7.25) 7.29 (6.88,7.70) 7.07 (6.69,7.45) 6.05 (5.73,6.36) 5.97 (5.60,6.34) <0.001
BOT-2 BOT-2: Total score 52.56 (51.44,53.69) 51.96 (50.79,53.14) 52.71 (51.58,53.84) 51.80 (50.67,52.93) 48.38 (47.30,49.47) 48.59 (47.45,49.72) <0.001
TOVA TOVA ADHD 0.27 (−0.11,0.65) 0.13 (−0.33,0.58) 0.51 (0.11,0.90) 0.43 (−0.02,0.87) −0.51 (−0.91,−0.10) −0.60 (−1.05,−0.16) <0.001
TOVA Response time variability (msec) 238.49 (228.49,248.49) 242.50 (232.24,252.76) 234.73 (225.27,244.20) 241.07 (232.06,250.08) 263.39 (253.44,273.34) 264.56 (254.70,274.42) <0.001
TOVA Response time (msec) 642.36 (622.55,662.17) 650.36 (631.66,669.06) 647.90 (624.12,671.67) 660.76 (640.80,680.72) 690.92 (670.69,711.15) 689.65 (669.95,709.35) <0.001
TOVA Commission (%) 8.97 (8.03,9.92) 9.23 (8.19,10.28) 9.19 (8.34,10.05) 9.71 (8.76,10.67) 10.20 (9.33,11.07) 10.55 (9.53,11.56) 0.09
TOVA Omission (%) 12.12 (10.04,14.20) 13.49 (11.14,15.85) 12.67 (10.77,14.58) 14.34 (12.40,16.28) 19.72 (17.13,22.30) 20.60 (18.00,23.19) <0.001
TOVA D Prime 2.89 (2.75,3.02) 2.78 (2.64,2.92) 2.83 (2.70,2.96) 2.71 (2.58,2.85) 2.47 (2.33,2.60) 2.41 (2.29,2.54) <0.001
TOVA D-prime standard score 87.89 (86.16,89.61) 86.96 (84.93,88.98) 88.26 (86.43,90.09) 86.88 (84.88,88.89) 82.54 (80.74,84.34) 81.60 (79.59,83.60) <0.001
BRIEF BRIEF BRI 51.83 (50.21,53.46) 52.38 (50.86,53.90) 51.24 (49.47,53.01) 52.12 (50.54,53.70) 53.00 (51.38,54.63) 52.28 (50.85,53.72) 0.97
BRIEF MI 49.76 (48.24,51.28) 50.78 (49.29,52.28) 51.87 (50.12,53.62) 53.20 (51.51,54.88) 52.09 (50.45,53.72) 51.63 (50.15,53.12) 0.07
BRIEF GEC 50.55 (49.02,52.08) 51.43 (50.00,52.85) 51.52 (49.79,53.26) 52.80 (51.23,54.38) 53.19 (51.51,54.87) 52.53 (51.04,54.03) 0.34

Note: Multivariate models adjust for effects with univariable p-values < 0.20

Table 6.

Selected Results, Impact of Treatment Initiation

Adjusted Means

Outcome Initiated at < 12 months Initiated at 12+ months Contrast P-value

Mean (95% CI) Mean (95% CI) Contrast (95% CI) P-Value
KABC Nonverbal index 72.68 (70.09,75.28) 72.47 (70.25,74.69) 0.21 (−3.07,3.49) 0.90
KABC Mental processing index 74.53 (72.25,76.81) 73.52 (71.66,75.37) 1.01 (−1.75,3.77) 0.47
BOT-2: Total score 49.25 (47.46,51.04) 47.85 (46.11,49.60) 1.39 (−0.89,3.67) 0.23
BRIEF GEC 50.75 (47.58,53.91) 55.01 (51.73,58.29) −4.26 (−8.15,−0.37) 0.03
TOVA ADHD −0.85 (−1.60,−0.11) −0.13 (−0.75,0.48) −0.72 (−1.61,0.18) 0.12
TOVA D Prime 2.42 (2.21,2.62) 2.55 (2.35,2.74) −0.13 (−0.40,0.14) 0.35
TOVA D-prime standard score 82.17 (78.98,85.35) 84.06 (80.91,87.22) −1.89 (−6.49,2.70) 0.42

Note: Multivariate models adjust for site, sex, age, rural/urban zone, careigver relation to child, socioeconomic index

Note: Contrast is computed as initiated at < 12 months minus initiated at 12+ months

A final GEE regression analysis compared HIV children with early (at < 12 months of age) versus later antiretroviral medication initiation, controlling for site, age, sex and various family characteristics. Tests of statistical significance were two-sided and, unless noted, 5% error rates were used for hypothesis testing. SAS version 9.2 and 9.4 were used for this analysis. Plots were generated using SAS version 9.4.

RESULTS

A total of 615 participants were enrolled at the six research sites, with 246 in the HIV+ cohort, 185 HEU and 184 HUU cohorts. However, 611 participants (246 HIV, 183 HEU and 182 HU) were eligible for this baseline analysis with a median age of 6.9 years (interquartile range [IQR], 6.2–8.1 years). Enrollment by study site is in the Supplemental table. Four HIV+ children were excluded from the present analyses because they could not complete the test battery (one child was deaf/mute, two were behaviorally uncooperative or disruptive, and one had significant neuromotor disability). When evaluating floor scores (scores at or below the lower limit) for each the principal outcomes for each neuropsychology test; there were fewer than 3% of participants with floor scores. Aside from the TOVA, which had roughly a 94% completion rate, over 98% of tests were completed on the other tests (KABC-II, BOT-2, BRIEF). Neither validity nor completeness proved a problem in these data. Given the very high level of available valid and complete measures for our principal outcome measures, there was no need to impute for missing scores on the basis of cohort mean values for a given measure. The probability estimates for both the adjusted and unadjusted between-cohort comparisons in Tables 4 through 6 were adjusted according to the appropriate degrees of freedom in order to evaluate statistical significance and effect size.

Table 2 shows personal and family characteristics by study cohort. The groups are balanced by sex and age. HIV+ children tended to have caregivers with less schooling and fewer have siblings enrolled. Fewer of the participants in the HIV+ cohort have caregivers who are biological mothers. As expected from the study design, primary caregivers are largely infected with HIV in the HIV+ and HEU cohorts, in contrast to the HUU cohort. Their immunological and virological status at study entry is very good, with over 95% having CD4% ≥ 25% and viral load ≤ 400 copies.

The WHO classification is progressive, and most participants entered the ARV clinical trial study in early childhood with severe HIV disease. Up to 9% of study participants reported a serious illness prior to entry in the present study, with the most prevalent across cohorts being premature birth. Participants in the HIV cohort reported severe malaria, low birth weight, tuberculosis and malnutrition, among others. A fair number (39%) of HIV+ participants reported Cozole/Trimethoprim use (39%). Other medications were reported for fewer than 2% of each study cohort. Weight and height z-scores (standardized using WHO norms based on age and gender) were significantly lower for the HIV+ children compared to the HEU and HUU cohorts (Table 2). The disability scores were higher for the HIV+ compared to the HEU and HUU cohorts. However, the MICS4 child development environment scores were comparable.

In the between-group adjusted comparisons presented below and in Tables 4 through 6, the covariates for which we adjusted from Table 2 were dependent those which were retained based on the significance of their loadings in the initial stage of the stepwise regression analyses. Unadjusted and adjusted mean scores and differences among cohorts are available in Tables 4 through 6. Only significant P values are reported in this narrative so as to make the presentation of the results less cumbersome and to conserve space. In adjusted between-group differences, HIV+ children performed significantly more poorly than both the HUU and HEU cohorts on all the global scales of the KABC-II (Tables 4 and 5). These included sequential processing (working memory) (p<0.001), learning (p<0.001), delayed recall (p<0.001), planning (reasoning) (p<0.01), and the composite scores of these four domains (mental processing index (MPI) (p<0.001)), as well as the composite for those subtests not involving language comprehension (nonverbal index (NVI) (p<0.001)). There were no significant differences between the HEU and HUU cohorts on any of the KABC-II global measures (Table 4). Between-group differences among the groups were consistent across all six study sites (HIV < HEU = HU), but overall performance on the KABC-II, irrespective of HIV exposure group, differed significantly between sites (Figures 2 and 3), resulting at times in significant group by site interaction effects. Among the KABC-II adjusted raw score comparisons for individual subtests not always included in the global scale measures, HIV+ children performed worse than the HEU and HUU cohorts on the planning/reasoning domain subtests of conceptual thinking, story completion, pattern reasoning and on the sequential processing (working memory) subtest of hand movements (p<0.001).

Table 5.

Unadjusted and Adjusted Pairwise Contrasts

HIV/exposed vs. negative HIV/exposed vs. positive HIV/negative vs. positive

Test Outcome Unadjusted Adjusted Unadjusted Adjusted Unadjusted Adjusted

Mean (95% CI) Mean (95% CI) Mean (95% CI) Mean (95% CI) Mean (95% CI) Mean (95% CI)
KABC_ KABC Sequential −1.06 (−3.45,1.34) −0.81 (−3.05,1.43) 6.10 (3.86,8.33) 5.19 (2.93,7.44) 7.15 (4.89,9.42) 6.00 (3.72,8.28)
KABC Simultaneous −0.62 (−3.14,1.89) −0.28 (−2.69,2.13) 3.66 (1.23,6.08) 2.82 (0.52,5.11) 4.28 (1.93,6.63) 3.10 (0.91,5.30)
KABC Learning −1.54 (−4.96,1.87) −0.60 (−3.62,2.43) 6.10 (2.96,9.25) 5.45 (2.75,8.15) 7.64 (4.54,10.74) 6.04 (3.12,8.96)
KABC Planning −4.02 (−8.12,0.07) −2.95 (−6.55,0.66) 5.00 (1.52,8.48) 2.36 (−1.93,6.65) 9.02 (5.65,12.39) 5.30 (1.75,8.86)
KABC Delayed recall 0.20 (−3.18,3.59) 0.82 (−1.93,3.57) 6.69 (3.64,9.73) 5.54 (3.00,8.08) 6.48 (3.32,9.65) 4.72 (1.93,7.52)
KABC Nonverbal index −0.65 (−3.05,1.75) −0.80 (−3.04,1.44) 6.32 (4.10,8.54) 5.33 (3.22,7.44) 6.97 (4.74,9.19) 6.13 (4.05,8.21)
KABC Mental processing index −1.75 (−4.12,0.62) −1.13 (−3.29,1.03) 5.96 (3.85,8.06) 5.26 (3.27,7.26) 7.71 (5.55,9.87) 6.39 (4.41,8.37)
KABC Conceptual thinking raw score −0.21 (−1.36,0.94) 0.03 (−0.83,0.90) 2.06 (0.95,3.18) 1.58 (0.70,2.47) 2.27 (1.19,3.35) 1.55 (0.69,2.41)
KABC Story completion raw score −0.79 (−1.74,0.16) −0.65 (−1.40,0.10) 1.24 (0.55,1.92) 0.85 (0.27,1.43) 2.03 (1.20,2.85) 1.50 (0.86,2.15)
KABC Pattern reasoning raw score −0.27 (−1.28,0.74) −0.02 (−0.71,0.66) 1.58 (0.70,2.47) 1.17 (0.48,1.87) 1.85 (0.94,2.76) 1.20 (0.54,1.85)
KABC Face recognition raw score −0.43 (−1.23,0.37) −0.32 (−0.97,0.32) 1.11 (0.32,1.91) 0.56 (−0.12,1.25) 1.54 (0.77,2.31) 0.88 (0.23,1.54)
KABC Hand movements raw score 0.20 (−0.39,0.78) 0.26 (−0.22,0.74) 1.24 (0.73,1.76) 1.10 (0.65,1.55) 1.05 (0.53,1.57) 0.84 (0.37,1.30)
BOT-2 BOT-2: Total score 0.15 (−1.45,1.75) −0.16 (−1.63,1.30) 4.33 (2.76,5.90) 3.21 (1.72,4.70) 4.18 (2.61,5.75) 3.37 (1.91,4.84)
TOVA TOVA ADHD 0.24 (−0.31,0.78) 0.30 (−0.22,0.82) 1.01 (0.45,1.57) 1.03 (0.52,1.54) 0.77 (0.22,1.33) 0.73 (0.22,1.23)
TOVA Response time variability (msec) −3.76 (−17.52,10.01) −1.43 (−12.79,9.93) −28.66 (−42.39,−14.92) −23.49 (−34.66,−12.31) −24.90 (−39.01,−10.79) −22.06 (−33.97,−10.14)
TOVA Response time (msec) 5.53 (−25.41,36.48) 10.40 (−12.34,33.14) −43.02 (−74.24,−11.81) −28.89 (−52.26,−5.51) −48.56 (−76.87,−20.24) −39.29 (−60.38,−18.19)
TOVA Commission (%) 0.22 (−1.06,1.49) 0.48 (−0.75,1.71) −1.01 (−2.23,0.21) −0.84 (−2.05,0.38) −1.23 (−2.51,0.06) −1.32 (−2.52,−0.12)
TOVA Omission (%) 0.55 (−2.27,3.38) 0.85 (−1.58,3.27) −7.04 (−10.25,−3.83) −6.26 (−9.10,−3.41) −7.59 (−10.91,−4.27) −7.10 (−9.97,−4.24)
TOVA D Prime −0.06 (−0.25,0.13) −0.07 (−0.22,0.09) 0.36 (0.17,0.55) 0.30 (0.14,0.46) 0.42 (0.22,0.61) 0.37 (0.20,0.54)
TOVA D-prime standard score 0.37 (−2.14,2.89) −0.07 (−2.45,2.31) 5.72 (3.16,8.29) 5.29 (2.78,7.80) 5.35 (2.86,7.84) 5.36 (2.86,7.86)
BRIEF BRIEF BRI −0.59 (−3.00,1.81) −0.26 (−2.34,1.82) −1.76 (−4.17,0.64) −0.16 (−2.18,1.86) −1.17 (−3.47,1.13) 0.10 (−1.93,2.12)
BRIEF MI 2.11 (−0.21,4.43) 2.42 (0.35,4.48) −0.22 (−2.61,2.18) 1.57 (−0.51,3.65) −2.33 (−4.56,−0.09) −0.85 (−2.83,1.13)
BRIEF GEC 0.97 (−1.34,3.29) 1.38 (−0.59,3.35) −1.66 (−4.08,0.76) 0.27 (−1.77,2.31) −2.64 (−4.91,−0.36) −1.11 (−3.09,0.87)

Note: Multivariate models adjust for effects with univariable p-values < 0.20

For the TOVA, HIV+ children performed poorly compared to the HEU and HUU cohorts on all outcomes pertaining to vigilance attention (percent omission errors, response time latency, response time variability, ADHD index, and D prime) (p<0.001) (Tables 4 and 5). The cohorts did not differ significantly on the principal outcome for impulsivity (percent commission errors). There was no difference in the TOVA measures between the HEU and HUU children. These between-group differences were consistent for all of the study sites (no significant between-group by site interaction effects), although as with the KABC-II, there were significant overall TOVA performance differences between the study sites (see supplemental table).

Overall, HIV+ children had significantly lower mean scores on the adjusted BOT-2 performance than the HEU and HUU controls (p<0.001) (Table 4). There was no difference between the HEU and HUU children. Although not significantly so, all the BRIEF global index scores tended to be higher (more behavior problems) for the HIV cohort (Tables 4 and 5).

An important consideration in the present study was whether earlier initiation of ARV treatment for the HIV+ children was protective for neuropsychological outcomes. In Table 6 we compare HIV+ children initiated on treatment before one year of age, to those initiated on treatment after one-year of age. We limited this comparison to just the standardized (age adjusted using American norms) global performance measures for our neuropsychological tests. The only significant difference was for the BRIEF global executive composite (GEC) composite (p=0.03), with an advantage in executive function behavior evaluations for children initiating ARV treatment before one year of age. In a subsequent paper we plan to do more thorough analyses of neuropsychological outcomes on the basis of ARV treatment arms and clinical response and parameters for the HIV+ cohort in our study, who all participated in the IMPAACT P1060 RCT.[4, 5, 31] Among selected KABC, TOVA, BOT-2 and BRIEF outcomes, only the BRIEF GEC score differed at study entry by age at HIV medication treatment initiation (Table 6).

DISCUSSION

The present study is distinctive in that it used the same assessment protocol in six sub-Saharan Africa study sites in four countries and enrolled children and caregivers in ten different languages. Although overall neuropsychological performance differed across sites, exposure group differences on our neuropsychological outcomes were remarkably consistent across all 6 sites, representing a greater level of rigor and reproducibility than documented in “single-site” studies. Furthermore, the present study enrolled HIV+ children from the same ARV trial protocol, initiated on treatment at diagnosis (from 3 months to 3 years of age). This provided for an exceptionally well characterized and cared for cohort that was virally suppressed and clinically stable at the time of neuropsychological assessment. A comparison of neuropsychological outcomes on the basis of ARV trial study arm for the present study children with HIV will follow in a separate publication.

The present findings filled a key gap in the research literature by characterizing the neuropsychological status of the ARV trial children at school-age, who were all initiated on an ARV treatment program at an early age (< 3 yrs) with careful clinical and adherence monitoring following treatment initiation, until the present assessment. We did so with a cross-sectional comparison of neuropsychological function of age-matched perinatally-exposed HIV, HEU, and HUU cohorts. Additional longitudinal assessment comparisons of these cohorts across three time points over a two-year period will follow in a separate publication. Findings from the first assessment comparison are presented here.

Despite the challenges of enrolling appropriate control groups for each study site, the study sites successfully enrolled and completed a 3 to 4 hr. battery of tests which included children < 6 years of age, not yet in school, from low-resource settings, and successfully completed the baseline tests. In this study, HIV+ children performed poorly on the KABC-II including sequential processing, simultaneous processing, learning, planning, delayed recall, nonverbal index, and MPI compared to HIV negative controls. Similar findings were reported with significantly poor performance in executive function tasks, particularly in terms of processing speed,[1, 32, 33] memory,[2] and attention.[1, 19] Early studies among younger HIV+ children have also described lower visual–spatial processing scores, which impact on reading, writing and learning in adolescence.[3]

The present HIV+ participants had received ARV treatment at a young age (< 3 years) and received continuous clinical and medical monitoring and support in a well-resourced clinical trial program (IMPAACT P1060 clinical trial) up until the present study assessment. Nevertheless, the HIV+ cohort still performed significantly worse than their HEU and HUU counterparts on all major neuropsychological outcomes pertaining to cognitive, attention, motor ability. These findings are similar to our published findings comparing HIV (ARV naïve school-age Ugandan children with less advanced HIV disease) and HUU Ugandan children using the KABC-II, TOVA, and BOT-2.[1] However, the neuropsychological performance differences among our exposure groups are more extensive and more significant, particularly for the HIV+ children. This is likely because the HIV+ children came into P1104s baseline assessment proportionately with more advanced HIV disease than those studied by Ruel et al. (2013) where children were ARV naïve and not yet ARV treatment eligible. In terms of physical growth, HIV+ children in this study were also more stunted and wasted then the HEU and HUU children, which, in the African context, is predictive of the more pervasive neurodevelopmental effects of HIV disease in early childhood.[34, 35]

Further support of this conclusion is provided in a recently published cohort study of 1383 children 6-year-old to 8-year-old in KwaZulu-Natal, South Africa.[17] Using the KABC-II as one of the tests for providing cognitive performance outcomes, the authors evaluated the association of demographic variables (area of residence, sex, pre-school education, HIV status, height for age and haemoglobin level) and family variables (socioeconomic status, maternal and paternal level of education), with children’s cognitive performance. Area of residence, height-for-age, and paternal level of education were all statistically significant factors affecting cognitive test scores, whereas HIV status, sex and SES were not. They concluded that at-risk children in impoverished rural areas with low cognitive scores tended to be stunted (low height-for-age scores), lacked pre-school education and were younger, and that parents’ educational level also was important. These factors could overshadow HIV status per se in a highly impoverished rural African-based study population.[17]

These conclusions are consistent with another recent comparison of HIV to HUU an HEU children in an impoverished rural area of Uganda.[14] Using the KABC-II as main outcome, these investigators obtained significantly poorer performance in the nonverbal index composite (NVI) among HIV+ children compared to their HEU and HUU counterparts. These differences were driven largely by differences in the sequential processing (working memory) and learning global domains. Early initiation of ARV, however, combined with subsequent years of schooling resulted in better KABC-II sequential processing outcomes for the HIV children.[14] An earlier KABC assessment of Ugandan HIV (ARV naïve), HEU, and HUU children noted little difference among these cohorts except perhaps in the domain of simultaneous processing.[36] This is in contrast to a study of HIV, HEU, and HUU cohorts in the Democratic Republic of the Congo (DRC), which observed significantly poorer performance for HIV children in KABC sequential and simultaneous processing and in motor proficiency,[37] although these neurodevelopmental deficits could be mitigated by ARV treatment and supportive medical care in the first few years of life.[38, 39]

The neuropsychological performance of the HEU and HUU children did not differ significantly for our principal study outcomes, unlike other studies where such differences have been noted.[37, 39, 40] Standards of care varied during pregnancy for women with HIV varied among our country sites, as well as the type of ARV treatment used for the prevention of mother-to-child transmission (PMTCT) of HIV infection. However, the HEU dyads enrolled for our comparison cohort across all six of our study sites were from other clinical studies where the HIV status of mother and child were perinatally documented. These women tended to receive prenatal care throughout most of their pregnancy at our study clinics. Therefore, they were probably receiving a better quality of medical care for ante- and post-natal maternal health conditions than might typically be available for pregnant mothers with HIV served at such large public hospitals in the urban setting (e.g., maternal ARV treatment and support, maternal and child micronutrient supplements, prompt diagnosis and treatment for anemia, symptomatic and asymptomatic malaria, supportive care for continued breast feeding, co-trimoxazole prophylactic for HEU children in the first months of life). Such supportive care, although the standard of care in all of our study settings, are not typically available and could well have boosted the health and early growth and development of our HEU children than would have been the case in other neurodevelopmental studies comparing HEU to HUU cohorts. These aspects of our HEU cohort enrollment could limit the extent to which our present neuropsychological findings for the HEU and HUU children should be generalized to the general population in such African settings.

The present study provides conclusive evidence that African children with HIV are at significant neuropsychological risk, even with early ART treatment initiation and careful medical support. These findings support the conclusions by Boivin, Ruisenor-Escudero, and Familiar-Lopez (2016) in their review.[41] They also documented that although the more severe forms of HIV-associated neurologic deficits are reduced following cARV treatment, neurocognitive and behavioral problems can persist and deepen at school age despite effective and sustained plasma viral suppression from an early age. These conclusions were also supported in a review by Laughton and colleagues (2016),[3] and suggest that behavioral interventions are needed in combination with medical treatment and care in order to fully address the neurodevelopmental needs of children and adolescents in Africa living with HIV. To illustrate, early childhood caregiver training programs can enhance the developmental milieu of the child with HIV and lead to improved cognitive and social development.[4244] For school-age children, computerized cognitive rehabilitation training can improve attention, working memory, and problem solving skills for children with HIV.[15, 21, 43]

For future studies with our present study cohorts, our study team is presently proposing combining such behavioral interventions with newer cARV medications proven to be effective in penetration of the central nervous system in achieving viral suppression as early as possible in the developing brain. Dolutegravir (DTG), an integrase inhibitor approved for children over 6 years of age, has good CNS penetration effectiveness (CPE), exceeding that of LPV/r. [45, 46] Maraviroc (MVC) is a CCR-5 inhibitor, preventing HIV attachment to CCR-5 expressing macrophages and CD4+ T-cells. MVC has a CPE of ‘3’, equivalent to EFV and when added to suppressive ART, was associated with neurocognitive improvement in 2 prospective adult studies. [47, 48] MVC also has an anti-inflammatory effect which can enhance neural function diminished by neural inflammation in HIV disease. This is evidenced by the preliminary findings that MVC can improve neurocognition in adult HIV patients with advanced disease. [49] Likewise, recent work in mice suggests that CCR-5 inhibition also increases plasticity and learning independent of effects on HIV and inflammation. [50]

Our study team is proposing in future work that combining DTG and MVC with computerized cognitive rehabilitation training in school-age children with HIV may provide a means by which neuropsychological impairment can, in part, be remediated. Likewise, initiating DTG and MVC as early as possible with infected infants and combining this with follow-up early childhood development (ECD) caregiver training for cognitive and nutritional enrichment in the home may better buffer children against neuropsychological impairment at school age. Such cARV and behavioral intervention combination treatment plans, initiated as early as possible, should become the new standard of care in the treatment of pediatric HIV disease.

Conclusion

Significant cognitive deficits were documented for the HIV cohort across sites. Earlier HIV treatment, neuropsychological monitoring and rehabilitative interventions are needed. In the present study, we adapted our neuropsychological test battery and caregiver questionnaires to differing cultural contexts at six study sites in four different sub-Saharan countries and across at least ten different local languages. Despite the cross-cultural implementation challenge, our preliminary findings clearly establish the feasibility, sensitivity, and robust nature of our neuropsychological outcomes for the principal study aims. Extending the assessment further to 48 and 96 weeks post-baseline assessments will provide a between-exposure group and between treatment arm comparisons of the developmental trajectory over three yearly time points. This should bridge the gap in documenting long term effects of HIV and ARV treatment on neurodevelopmental outcomes. Finally, using newer more highly CNS penetrant cARV drugs (e.g., DTG, MVC) should be combined with caregiver-based cognitive and nutritional enrichment interventions in the home. These can also be transitioned to computerized/tablet/mobile devices for cognitive enrichment at school-age. Such combination ARV/behavioral interventions for children with HIV in resource-constrained settings should initiated as early as possible in children’s development, in the hopes of preventing further neuropsychological injury from pediatric HIV disease.

Supplementary Material

Supplemental Data File _.doc_ .tif_ pdf_ etc.__1
Supplemental Data File _.doc_ .tif_ pdf_ etc.__2
Supplemental Data File _.doc_ .tif_ pdf_ etc.__3
Supplemental Data File _.doc_ .tif_ pdf_ etc.__4
Supplemental Data File _.doc_ .tif_ pdf_ etc.__5

Acknowledgments

Overall support for the International Maternal Pediatric Adolescent AIDS Clinical Trials (IMPAACT) Network was provided by the National Institute of Allergy and Infectious Diseases (NIAID) of the National Institutes of Health (NIH) under Award Numbers UM1AI068632 (IMPAACT LOC), UM1AI068616 (IMPAACT SDMC) and UM1AI106716 (IMPAACT LC), with co-funding from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the National Institute of Mental Health (NIMH). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Drs. E. Pim Brouwers (NIH/NIMH) and Sonia Lee (NIH/NICHD) served as protocol advisors the research leadership team for P1104s for their respective NIH institutes. We gratefully acknowledge their expertise and counsel during the study. The authors also acknowledge the protocol administrative support provided by J.L. Ariansen from FHI360, as well as the support provided by Dr. Elizabeth Petzold throughout the protocol approval, finalization and study initiation process.

Funding: Overall support for the International Maternal Pediatric Adolescent AIDS Clinical Trials (IMPAACT) Network was provided by the National Institute of Allergy and Infectious Diseases (NIAID) of the National Institutes of Health (NIH) under Award Numbers UM1AI068632 (IMPAACT LOC), UM1AI068616 (IMPAACT SDMC) and UM1AI106716 (IMPAACT LC), with co-funding from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and the National Institute of Mental Health (NIMH).

Footnotes

Conflicts of Interest: None of the authors have any conflicts of interest to report.

Ethics in Human Subject Research: This study was conducted in accordance with the Declaration of Helsinki accords for the protection of human subject participants in research.

M.J.B. as protocol chair designed and directed the study and co-wrote the complete first draft of the manuscript.

L.B-W was lead investigator at the Kampala Uganda study site and co-wrote the complete first draft of the manuscript.

M.C. directed all statistical analyses and constructed the tables for this manuscript, contributed to and approved the final version of the manuscript.

B.L. was lead investigator at the Stellenbosch South Africa study site and advised on assessment methodology for this study, as well as the validation process and quality assurance of the assessment process.

B.Z. directed the development of the Case Report Forms and all phases of data management and validation checks.

C.J. directed the neuropsychological assessments at the Soweto South Africa study site and contributed to and approved the final version of the manuscript.

M. B-D. was lead investigator at the Harare Zimbabwe study site and contributed to and approved the final version of the manuscript.

M.R. was responsible for all of the neuropsychological testing at the Johannesburg South Africa study site and approved the final version of the manuscript.

N.A. directed patient recruitment and supervised protocol management at the Soweto South Africa study site and approved the final version of the manuscript.

L.F. was lead investigator at the Johannesburg South Africa study site and contributed to and approved the final version of the manuscript.

H.G. directed patient recruitment and supervised protocol management at the Johanesburgh South Africa study site and approved the final version of the manuscript.

P.K. was lead investigator at the Lilongwe Malawi study site and supervised protcol management and approved the final version of the manuscript.

K.M. was responsible for protocol oversight, amendment approvals and coordination, enrollment management, and contributed to and approved the final version of the manuscript.

I.F-L. coordinated quality assurance and assessment for the neuropsychological assessment center based at the Uganda study site and contributed to and approved the final version of the manuscript.

P.J-P. served as NIAID scientific advisor to this study and contributed to protocol design and approval and contributed to and approved the final version of this manuscript.

J.C. served as project Human Subjects Ethics and Research advisor, protocol advisor, and enrollment advisor for the study and approved the final version of the manuscript.

A.V. was lead investigator at the Soweto South Africa study site, served as protocol and scientific advisor, and contributed to and approved the final version of the manuscript.

M.C.C. was lead investigator at the Stellenbosch South Africa study site, served as protocol and scientific advisor, and contributed to and approved the final version of the manuscript.

P.P. was principal investigator for the P1060 RCT study for the study children with HIV, was a principal contributor to the study design, protocol design and implementation, grant writing and funding approval, contributed to and approved the final version of the manuscript.

References

  • 1.Ruel TD, Boivin MJ, Boal HE, Bangirana P, Charlebois E, Havlir DV, et al. Neurocognitive and motor deficits in HIV-infected Ugandan children with high CD4 cell counts. Clin Infect Dis. 2012;54(7):1001–1009. doi: 10.1093/cid/cir1037. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Phillips N, Amos T, Kuo C, Hoare J, Ipser J, Thomas KG, et al. HIV-Associated Cognitive Impairment in Perinatally Infected Children: A Meta-analysis. Pediatrics. 2016;138(5) doi: 10.1542/peds.2016-0893. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Laughton B, Cornell M, Boivin M, Van Rie A. Neurodevelopment in perinatally HIV-infected children: a concern for adolescence. J Int AIDS Soc. 2013;16:18603. doi: 10.7448/IAS.16.1.18603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Barlow-Mosha L, Angelidou K, Lindsey J, Archary M, Cotton M, Dittmer S, et al. Nevirapine- Versus Lopinavir/Ritonavir-Based Antiretroviral Therapy in HIV-Infected Infants and Young Children: Long-term Follow-up of the IMPAACT P1060 Randomized Trial. Clin Infect Dis. 2016;63(8):1113–1121. doi: 10.1093/cid/ciw488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Lindsey JC, Hughes MD, Violari A, Eshleman SH, Abrams EJ, Bwakura-Dangarembizi M, et al. Predictors of virologic and clinical response to nevirapine versus lopinavir/ritonavir-based antiretroviral therapy in young children with and without prior nevirapine exposure for the prevention of mother-to-child HIV transmission. Pediatr Infect Dis J. 2014;33(8):846–854. doi: 10.1097/INF.0000000000000337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Palumbo P, Lindsey JC, Hughes MD, Cotton MF, Bobat R, Meyers T, et al. Antiretroviral treatment for children with peripartum nevirapine exposure. N Engl J Med. 2010;363(16):1510–1520. doi: 10.1056/NEJMoa1000931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Kaufman AS, Kaufman NL. Manual for the Kaufman Assessment Battery for Children. 2. Circle Pines, MN: American Guidance Service Publishing/Pearson Products Inc; 2004. [Google Scholar]
  • 8.Bangirana P, Musisi S, Allebeck P, Giordani B, John CC, Opoka RO, et al. A preliminary investigation of the construct validity of the KABC-II in Ugandan children with prior cerebral insult. African Health Sciences. 2009;9(3):186–192. [PMC free article] [PubMed] [Google Scholar]
  • 9.Giordani B, Boivin MJ, Opel B, Dia Nseyila D, Diawaku N, Lauer RE. Use of the K-ABC with children in Zaire, Africa: An evaluation of the sequential-simultaneous processing distinction within an intercultural context. International Journal of Disability, Development and Education. 1996;43(1):5–24. [Google Scholar]
  • 10.Jansen P, Greenop K. Factor analysis of the Kaufman Assessment Battery for Children assessed at 5 and 10 years. South African Journal of Psychology. 2008;38(2):355–365. [Google Scholar]
  • 11.Ochieng CO. Meta-Analysis of the Validation Studies of the Kaufman Assessment Battery for Children. International Journal of Testing. 2003;3(1):77–93. [Google Scholar]
  • 12.Boivin MJ, Bangirana P, Tomac R, Parikh S, Opoka RO, Nakasujja N, et al. Neuropsychological benefits of computerized cognitive rehabilitation training in Ugandan children surviving cerebral malaria and children with HIV. BMC Proceedings. 2008;2(Suppl 1):P7. [Google Scholar]
  • 13.Boivin MJ, Nakasujja N, Sikorskii A, Opoka RO, Giordani B. A Randomized Controlled Trial to Evaluate if Computerized Cognitive Rehabilitation Improves Neurocognition in Ugandan Children with HIV. AIDS Res Hum Retroviruses. 2016;32(8):743–755. doi: 10.1089/aid.2016.0026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Brahmbhatt H, Boivin M, Ssempijja V, Kagaayi J, Kigozi G, Serwadda D, et al. Impact of HIV and Atiretroviral Therapy on Neurocognitive Outcomes Among School-Aged Children. J Acquir Immune Defic Syndr. 2017;75(1):1–8. doi: 10.1097/QAI.0000000000001305. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Giordani B, Novak B, Sikorskii A, Bangirana P, Nakasujja N, Winn BW, et al. Designing and evaluating Brain Powered Games for cognitive training and rehabilitation in at-risk African children. Global Mental Health. 2015;2(e6):1–14. doi: 10.1017/gmh.2015.5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Wyhe KSv, Water Tvd, Boivin MJ, Cotton MF, Thomas KGF. Cross-cultural assessment of HIV-Associated Neurocognitive Impairment using the Kaufman Assessment Battery for Children: A systematic review. Journal of the International AIDS Society. 2017 doi: 10.7448/IAS.20.1.21412. in press. [DOI] [PMC free article] [PubMed]
  • 17.Ajayi OR, Matthews G, Taylor M, Kvalsvig J, Davidson LL, Kauchali S, et al. Factors associated with the health and cognition of 6-year-old to 8-year-old children in KwaZulu-Natal, South Africa. Trop Med Int Health. 2017;22(5):631–637. doi: 10.1111/tmi.12866. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Bruininks RH, Bruininks BD. BOT2: Bruininks-Oseretsky Test of Motor Proficiency. 2. Minneapolis, MN: Pearson Assessments; 2005. [Google Scholar]
  • 19.Boivin MJ, Ruel TD, Boal HE, Bangirana P, Cao H, Eller LA, et al. HIV-subtype A is associated with poorer neuropsychological performance compared with subtype D in antiretroviral therapy-naive Ugandan children. AIDS. 2010;24(8):1163–1170. doi: 10.1097/qad.0b013e3283389dcc. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Greenberg LM. Universal Attention Disorders. Los Alamitos, CA: 1993. The T.O.V.A. (Version 6.X) (Computer Program) [Google Scholar]
  • 21.Boivin MJ, Busman RA, Parikh SM, Bangirana P, Page CF, Opoka RO, et al. A pilot study of the neuropsychological benefits of computerized cognitive rehabilitation in Ugandan children with HIV. Neuropsychology. 2010;24(5):667–673. doi: 10.1037/a0019312. [DOI] [PubMed] [Google Scholar]
  • 22.Gioia GA, Isquith PK, Guy SC, Kenworthy L. Behavior Rating Inventory of Executive Function® (BRIEF®) Lutz, FL: Psychological Assessment Resources (PAR); 2003. [Google Scholar]
  • 23.Familiar I, Nakasujja N, Bass J, Sikorskii A, Murray S, Ruisenor-Escudero H, et al. Caregivers’ depressive symptoms and parent-report of child executive function among young children in Uganda. Learn Individ Differ. 2016;46:17–24. doi: 10.1016/j.lindif.2015.01.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Familiar I, Ruisenor-Escudero H, Giordani B, Bangirana P, Nakasujja N, Opoka R, et al. Use of the Behavior Rating Inventory of Executive Function and Child Behavior Checklist in Ugandan children with HIV or a history of severe malaria. J Dev Behav Pediatr. 2015;36(4):277–284. doi: 10.1097/DBP.0000000000000149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. The Hopkins Symptom Checklist (HSCL): a self-report symptom inventory. Behav Sci. 1974;19(1):1–15. doi: 10.1002/bs.3830190102. [DOI] [PubMed] [Google Scholar]
  • 26.Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. The Hopkins Symptom Checklist (HSCL). A measure of primary symptom dimensions. Mod Probl Pharmacopsychiatry. 1974;7(0):79–110. doi: 10.1159/000395070. [DOI] [PubMed] [Google Scholar]
  • 27.Familiar I, Murray S, Ruisenor-Escudero H, Sikorskii A, Nakasujja N, Boivin MJ, et al. Socio-demographic correlates of depression and anxiety among female caregivers living with HIV in rural Uganda. AIDS Care. 2016:1–5. doi: 10.1080/09540121.2016.1191609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Durkin MS, Gottlieb CA, Maenner MJ, Cappa C, Loaiza E. UNICEF, editor. Monitoring child disability in developing countries: results from the Multiple Indicator Cluster Surveys. New York: UNICEF; 2008. [Google Scholar]
  • 29.Durkin MS, Wang W, Shrout PE, Zaman SS, Hasan ZM, Desai P, et al. Evaluating a ten questions screen for childhood disability: reliability and internal structure in different cultures. J Clin Epidemiol. 1995;48(5):657–666. doi: 10.1016/0895-4356(94)00163-k. [DOI] [PubMed] [Google Scholar]
  • 30.Bangirana P, John CC, Idro R, Opoka RO, Byarugaba J, Jurek AM, et al. Socioeconomic predictors of cognition in Ugandan children: Implications for community based interventions. PLoS ONE. 2009;4(11):e7898. doi: 10.1371/journal.pone.0007898. doi:7810.1371/journal.pone.0007898. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Violari A, Lindsey JC, Hughes MD, Mujuru HA, Barlow-Mosha L, Kamthunzi P, et al. Nevirapine versus ritonavir-boosted lopinavir for HIV-infected children. N Engl J Med. 2012;366(25):2380–2389. doi: 10.1056/NEJMoa1113249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Koekkoek S, de Sonneville LMJ, Wolfs RFW, Licht R, Greelen SPM. Neurocognitive function profile in HIV-infected school-age children. European Journal of Paediatric Neurology. 2008;12:290–297. doi: 10.1016/j.ejpn.2007.09.002. [DOI] [PubMed] [Google Scholar]
  • 33.Smith R, Chernoff M, Williams PL, Malee KM, Sirois PA, Kammerer B, et al. Impact of HIV severity on cognitive and adaptive functioning during childhood and adolescence. Pediatr Infect Dis J. 2012;31(6):592–598. doi: 10.1097/INF.0b013e318253844b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Abubakar A, Holding P, Newton CR, van Baar A, van de Vijver FJ. The role of weight for age and disease stage in poor psychomotor outcome of HIV-infected children in Kilifi, Kenya. Dev Med Child Neurol. 2009;51(12):968–973. doi: 10.1111/j.1469-8749.2009.03333.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Abubakar A, Van Baar A, Van de Vijver FJ, Holding P, Newton CR. Paediatric HIV and neurodevelopment in sub-Saharan Africa: a systematic review. Trop Med Int Health. 2008;13(7):880–887. doi: 10.1111/j.1365-3156.2008.02079.x. [DOI] [PubMed] [Google Scholar]
  • 36.Bagenda D, Nassali A, Kalyesubula I, Sherman B, Drotar D, Boivin MJ, et al. Health, neurologic, and cognitive status of HIV-infected, long-surviving, and antiretroviral-naive Ugandan children. Pediatrics. 2006;117(3):729–740. doi: 10.1542/peds.2004-2699. [DOI] [PubMed] [Google Scholar]
  • 37.Boivin MJ, Green SD, Davies AG, Giordani B, Mokili JK, Cutting WA. A preliminary evaluation of the cognitive and motor effects of pediatric HIV infection in Zairian children. Health Psychol. 1995;14(1):13–21. doi: 10.1037//0278-6133.14.1.13. [DOI] [PubMed] [Google Scholar]
  • 38.Van Rie A, Dow A, Mupuala A, Stewart P. Neurodevelopmental trajectory of HIV-infected children accessing care in Kinshasa, Democratic Republic of Congo. J Acquir Immune Defic Syndr. 2009;52(5):636–642. doi: 10.1097/QAI.0b013e3181b32646. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Van Rie A, Mupuala A, Dow A. Impact of the HIV/AIDS epidemic on the neurodevelopment of preschool-aged children in Kinshasa, Democratic Republic of the Congo. Pediatrics. 2008;122(1):e123–128. doi: 10.1542/peds.2007-2558. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Malee KM, Tassiopoulos K, Huo Y, Siberry G, Williams PL, Hazra R, et al. Mental health functioning among children and adolescents with perinatal HIV infection and perinatal HIV exposure. AIDS Care. 2011;23(12):1533–1544. doi: 10.1080/09540121.2011.575120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Boivin MJ, Ruisenor-Escudero H, Familiar-Lopez I. CNS impact of perinatal HIV infection and early treatment: the need for behavioral rehabilitative interventions along with medical treatment and care. Curr HIV/AIDS Rep. 2016;13(6):318–327. doi: 10.1007/s11904-016-0342-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Bass JK, Opoka R, Familiar I, Nakasujja N, Sikorskii A, Awadu J, et al. Randomized controlled trial of caregiver training for HIV-infected child neurodevelopment and caregiver well-being. AIDS. 2017 doi: 10.1097/QAD.0000000000001563. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Boivin MJ, Bangirana P, Nakasuja N, Page CF, Shohet C, Givon D, et al. A year-long caregiver training program to improve neurocognition in preschool Ugandan HIV-exposed children. J Dev Behav Pediatr. 2013;34(2):269–278. doi: 10.1097/DBP.0b013e318285fba9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Boivin MJ, Bangirana P, Nakasujja N, Page CF, Shohet C, Givon D, et al. A year-long caregiver training program improves cognition in preschool Ugandan children with human immunodeficiency virus. J Pediatr. 2013;163:1409–1416. doi: 10.1016/j.jpeds.2013.06.055. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Letendre S, Marquie-Beck J, Capparelli E, Best B, Clifford D, Collier AC, et al. Validation of the CNS Penetration-Effectiveness rank for quantifying antiretroviral penetration into the central nervous system. Arch Neurol. 2008;65(1):65–70. doi: 10.1001/archneurol.2007.31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Letendre SL, Mills AM, Tashima KT, Thomas DA, Min SS, Chen S, et al. ING116070: a study of the pharmacokinetics and antiviral activity of dolutegravir in cerebrospinal fluid in HIV-1-infected, antiretroviral therapy-naive subjects. Clin Infect Dis. 2014;59(7):1032–1037. doi: 10.1093/cid/ciu477. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Ndhlovu LC, Umaki T, Chew GM, Chow DC, Agsalda M, Kallianpur KJ, et al. Treatment intensification with maraviroc (CCR5 antagonist) leads to declines in CD16-expressing monocytes in cART-suppressed chronic HIV-infected subjects and is associated with improvements in neurocognitive test performance: implications for HIV-associated neurocognitive disease (HAND) J Neurovirol. 2014;20(6):571–582. doi: 10.1007/s13365-014-0279-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Gates TM, Cysique LA, Siefried KJ, Chaganti J, Moffat KJ, Brew BJ. Maraviroc-intensified combined antiretroviral therapy improves cognition in virally suppressed HIV-associated neurocognitive disorder. AIDS. 2016;30(4):591–600. doi: 10.1097/QAD.0000000000000951. [DOI] [PubMed] [Google Scholar]
  • 49.Francisci D, Falcinelli E, Baroncelli S, Petito E, Cecchini E, Weimer LE, et al. Potential anti-inflammatory effects of maraviroc in HIV-positive patients: a pilot study of inflammation, endothelial dysfunction, and coagulation markers. Scand J Infect Dis. 2014;46(6):466–470. doi: 10.3109/00365548.2014.898332. [DOI] [PubMed] [Google Scholar]
  • 50.Zhou M, Greenhill S, Huang S, Silva TK, Sano Y, Wu S, et al. CCR5 is a suppressor for cortical plasticity and hippocampal learning and memory. eLife. 2016:5. doi: 10.7554/eLife.20985. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Data File _.doc_ .tif_ pdf_ etc.__1
Supplemental Data File _.doc_ .tif_ pdf_ etc.__2
Supplemental Data File _.doc_ .tif_ pdf_ etc.__3
Supplemental Data File _.doc_ .tif_ pdf_ etc.__4
Supplemental Data File _.doc_ .tif_ pdf_ etc.__5

RESOURCES