Using neuronal extracellular vesicles and machine learning to predict cognitive deficits in HIV

Abstract

Our objective was to predict HIV-associated neurocognitive disorder (HAND) in HIV-infected people using plasma neuronal extracellular vesicle (nEV) proteins, clinical data and machine learning. We obtained 60 plasma samples from 38 women and 22 men, all with HIV infection and 40 with HAND. All underwent neuropsychological testing. nEVs were isolated by immunoadsorption with neuron-specific L1CAM antibody. High-mobility group box 1 (HMGB1), neurofilament light (NFL) and phosphorylated tau-181 (p-T181-tau) proteins were quantified by ELISA. Three different computational algorithms were performed to predict cognitive impairment using clinical data and nEV proteins. Of the 3 different algorithms, Support Vector Machines performed the best. Applying 4 different models of clinical data with 3 nEV proteins, we showed that selected clinical data and HMGB1 plus NFL, best predicted cognitive impairment with an area under the curve value of 0.82. The most important features included CD4 count, HMGB1 and NFL. Previous published data showed nEV p-T181-tau was elevated in Alzheimer’s disease (AD) and in this study p-T181-tau had no importance in assessing HAND but may actually differentiate it from AD. Machine learning can access data without programming bias. Identifying a few nEV proteins plus key clinical variables can better predict neuronal damage. This approach may differentiate other neurodegenerative diseases and determine recovery after therapies are identified.

Introduction

The increased availability of “omics” technology has increased the number of new and potentially useful biomarkers of disease. At the same time, machine and statistical learning methods have supplied biologists with new tools to analyze these big datasets. In this context, machine learning for biomarker discovery and response to therapy involves using computational models, including clinical and epidemiological data plus any combination of biomarkers.

HIV-associated neurocognitive disorder (HAND) is a disorder of heterogeneous etiology, reflecting the effects of viral infection, immunologic response, medication usage, comorbidities and genetics (Antinori et al, 2007). The presence and severity of HAND has been measured with the global deficit score (GDS), a compilation of extensive neuropsychological test scores (Carey et al, 2004). An inexpensive, objective, noninvasive biomarker for cognitive impairment in HIV infection is needed, not only for diagnosis but for monitoring potential recovery of neuronal health after treatment or cure. Ideally, biomarkers would not only serve to predict cognitive dysfunction but differentiate it from common age-related impairments, such as Alzheimer’s disease (AD).

Elevated cerebrospinal fluid (CSF) biomarkers of immune activation (neopterin, β2-microglobulin and soluble CD14), neuronal injury (neuron filament light, NFL), and glial response (S100B) have been described in HIV-positive individuals with suppressed plasma and CSF viral loads (Du Pasquier et al, 2013; Eden et al, 2016; McGuire et al, 2015; Pemberton and Brew, 2001; Peterson et al, 2014; Price et al, 2013; Yilmaz et al, 2008). Recently CSF NFL was shown to have value in differentiating a number of neurological conditions (Bridel et al, 2019). While CSF is thought to be the closest to the neuropathology, it requires an invasive procedure and, like plasma, has a complex protein profile derived from a variety of different cell types.

Small extracellular vesicles (EVs) that include exosomes are released from cells under physiologic and pathologic conditions and may serve as neuroinflammatory biomarkers for various neurodegenerative diseases, including HIV infection (Gupta and Pulliam, 2014; Hu et al, 2016; Thery et al, 2002). Exosomal contents reflect the parent cell’s state at the time of secretion, packaging proteins, lipids and nucleic acids to influence target cells (Dalvi et al, 2017). Neuronal EVs (nEVs) cross the blood brain barrier (BBB) and can be identified by their expression of neuron-specific cell surface antibody, L1 cell adhesion molecule (L1CAM) (Goetzl et al, 2015). Recent work from us and others have characterized plasma EV abundance and nEV cargo from HIV-infected individuals, and the association with HAND subcategories (Chettimada et al, 2018; Dalvi et al, 2017; Pulliam et al, 2019; Sun et al, 2019). We reported that protein targets associated with neuronal damage, including high mobility group box1 (HMGB1) and NFL were elevated and p-T181-tau was decreased in nEVs from HIV-infected people with cognitive impairment (Sun et al, 2019).

Machine learning is a highly effective method for both prediction and feature selection. This study aimed to use machine learning to determine the predictive value of using several nEV proteins with select clinical variables to better identify cognitive impairment in HIV infection.

Materials and methods

Participants

Frozen plasma from 60 subjects was obtained from the National NeuroAIDS Tissue Consortium and HIV Neurobehavioral Research Program. The plasma included 20 neuropsychologically normal (NPN) people and 40 testing asymptomatic (ANI) or mild neurocognitive disorder (MND). The ANI and MND groups were combined as neuropsychologically impaired (NPI). Participants were characterized using medical, neuropsychological, and laboratory variables as described previously (Sun et al, 2019).

Participants completed a comprehensive neuropsychological testing battery including 7 domains and a self-report Beck depression survey. A GDS based on the neuropsychological test scores was calculated. Additional measurements, including age and education (Table 1) were determined. Exclusion criteria included recent infection at blood draw, other neurological illnesses, severe depression, psychoactive drug use or history of head injury.

Table 1.

Characteristics of individuals

HIV Status	HIV+		P value
Diagnosis	NPN	NPI
N			0.636
Women	14	24
Men	6	16
Ethnicity			0.302
Asian	0	1
Black	2	11
Caucasian	11	20
Hispanic/other	7	8
Age	46.1 (8.4)	47.8 (10.4)	0.510
Education	13.6 (2.6)	12.4 (2.9)	0.123
Viral load	UD	UD
CD4	686.9 (243.9)	549.9 (364.6)	0.092
GDS	0.092 (0.089)	0.78 (0.41)	< 0.0001

Number of individuals (N) and ethnicity are presented as counts, using Pearson’s chi-squared tests; age, education, CD4 and GDS are presented as mean (S.D.), using Welch’s t test. NPN = Neuropsychologically Normal; NPI = Neuropsychologically Impairment; GDS = Global Deficit Score; UD = undetectable.

Isolation of nEV from plasma

nEVs were isolated as previously described (Sun et al, 2017; Sun et al, 2019). Briefly, insoluble particles in plasma were removed by centrifugation at 1000g for 10min, and plasma fibril and coagulation proteins were removed by adding 100μl thromboplastin to 250μl plasma for 1hr followed by centrifugation at 3000g for 20min. Total exosomes were isolated by adding 126μl ExoQuick (Systems Biosciences, Palo Alto, CA) to each sample in the presence of protease and phosphatase inhibitors. Pellets of total exosomes were resuspended and incubated with biotinylated-L1CAM monoclonal antibody (Thermo-Fisher Scientific, Inc., Waltham, MA). Labeled exosomes were captured with streptavidin-labeled-resin (Thermo-Fisher). The exosome-resin complexes were precipitated and washed 5 times with PBS. Pure neuronal exosomes were released from the resin beads into solution with 100μl of 50mM Glycine-HCl (pH3) and neutralized with 10μl Tris-HCl (pH8). Exosomes were lysed using mammalian protein extraction reagent (Thermo-Fisher) in the presence of proteinase and phosphatase inhibitors and bovine serum albumin. The lysates were stored at −80C until use.

Characterization of nEVs and protein quantification by ELISA

nEV size and counts were determined by a nanoparticle tracking system (NTA), NanoSight LM10 instrument (Malvern Instruments, Malvern, UK) with a 405nm laser-equipped sample chamber, as described previously (Tang et al, 2016). The samples were diluted in PBS to 10⁸-10⁹ particles/mL. Camera shutter speed, camera gain and software detection threshold were manually adjusted for optimal detection and kept consistent during all sample analyses. Each sample was analyzed three times with three recordings of 30 seconds each. The modes were reported for particle sizes due to the highly skewed distributions. The mean of the triplicate recordings was reported for size and concentration. Protein concentrations in nEVs were analyzed using commercial ELISA kits, apoptosis-linked gene 2-interacting protein X (ALIX) (Lifeome-Cusabio, Oceanside, CA), synaptophysin (SYN) (American Research Products, Inc., Waltham, MA), HMGB1 (LifeSpanBioSciences, Inc., Seattle, WA), NFL (MyBioSource, San Diego, CA) and p-T181-tau (Fujirebio US, Malvern, PA). ELISAs were performed per manufacturer’s instructions and normalized to ALIX as previously described (Sun et al, 2019). All standards and samples were assayed in duplicate. Plates were analyzed using a SpectraMax M5 plate reader and SoftMax v5 software (Molecular Devices, San Jose, CA).

Machine learning methodology

To ensure proper data processing by machine learning algorithms, several data transformations were performed. Education was trichotomized (0 = 6–10, 1 = 11–15, and 2 = 16–20 years). Age was grouped as young = 32–44 or old = 49–69 years old. Categorical features of age-group, sex and ethnicity were encoded with one-hot encoding. Missing values were imputed using K-Nearest Neighbors imputation method. Clinical variables including CD4, age-group, ethnicity, sex and education entered the models. GDS was colinear with prediction labels and thus omitted from the models. To classify between NPI and NPN samples, we employed a variety of supervised-learning methods that give a good representation of proven classification success (tree-based, clustering, ensembling, support vector).

K-Nearest Neighbors (KNN) is an algorithm that attempts to cluster or classify newly introduced data against an existing training set using a K parameter and a chosen distance metric. The K parameter is set by the practitioner and represents the amount of training set points to consider in proximity to the new point during the classification process. We used an ensemble version of KNN that leverages the high variance of low K, KNN models (K=1). This is an attempt to use individual KNN models as weak learners to build a more robust estimator.

Because of its performance on smaller data sets, Support Vector Machines (SVM) was a good candidate for this data set. In its simplest form, SVM uses the most optimal data points (also called support vectors) to determine the best separating boundary (also called a hyperplane) to separate data points for proper classification. In complex applications, kernel functions are used with SVMs to work with non-linear boundaries or higher dimensional data sets. We chose Support Vector Machines due to its success across scientific studies concentrating on “-omics” type data.

AdaBoost leverages an ensemble of weak learners in order to come to an optimal estimate. The premise behind AdaBoost is to build weak learners in sequential order with each subsequent model learning from the misclassifications of the previous model. The application of AdaBoost used decision trees as the weaker learners but many other algorithms can be used with AdaBoost.

Ensemble KNN was used specifically to leverage high variance predictions of individual KNN models. For tree-based algorithms we tested Random Forests and Adaptive boosting of decision trees with the later yielding better results for our given data set.

To properly evaluate the performance of these algorithms, we used repeated (10x) 6-fold cross validation (CV). We found this to be an optimal balance of train and test size (for N=60) to allow for adequate training and evaluation. Employing the repeated CV allowed for a more exhaustive estimation of model performance due to more perspectives. Within the main CV loop, data leakage between test folds was precluded and we analyzed the results of each model instantiated in CV.

Statistical analyses and data cleaning

Exploratory data analysis and data cleaning were performed using Python 3.7.3, Anaconda distribution. Test for normality and group-wise comparisons were performed with Shapiro-Wilk and Kruskal-Wallis tests, respectively. Tests for independence between each categorical variable and the binary response variable were performed with chi-square tests. Descriptive statistical analysis for group-wise comparisons were performed using R 3.5.3. Continuous variables were compared with a Student’s t test or analysis of variance and Tukey HSD post hoc tests. Log transformation was applied when necessary to improve normality. Categorical variables were compared with chi-squared tests. Pearson’s coefficient was used for correlation analysis.

Results

Participants

No differences were seen between NPN and NPI for sex, ethnicity, age, education or CD4 counts (Table 1). GDS was significantly higher in NPI (mean 0.78, CI=0.65–0.91) compared to NPN (mean 0.092, CI = 0.051–0.13, p<0.0001). All participants were treated with antiretroviral therapy (ART) and demonstrated undetectable plasma HIV viral RNA at the time of blood draw.

nEV proteins

To characterize nEVs, we quantified ALIX protein concentrations as well as concentration of EVs using NTA, which is considered the gold standard. We found that NTA was highly correlated with ALIX, making it easier to use this marker for normalization of ELISA results (Fig. 1a). In addition, we tested for the neuronal protein SYN and found it to be enriched in nEVs over total EVs (Fig. 1b) (Sun et al, 2019) strongly suggesting a neuron as the cell of origin. We previously published that HMGB1, NFL and p-T181-tau had group differences between HIV-infected people with cognitive impairment versus normal cognition and men versus women (Fig. 2) (Sun et al, 2019). HMGB1 was elevated in impaired versus cognitively normal people with men driving the increase (Fig. 2a). NFL was altered in HIV infection but only significantly elevated in men with cognitive impairment (Fig. 2b) and decreased in women. p-T181-tau showed no difference between HIV-infected individuals with cognitive impairment and normal cognition (Fig. 2c); however, this biomarker is significantly elevated in nEVs from patients with AD (Kapogiannis et al, 2019) and may differentiate HIV cognitive impairment from AD. We selected these 3 proteins to analyze in our machine learning algorithms.

Fig. 1.

Characterization of nEVs. (a) Correlation of nEV concentration as determined by NTA (particles/ml) and ALIX quantified by ELISA using Pearson’s correlation with 95% confidence interval (in grey shade). (b) nEVS were highly enriched in synaptophysin (SYN) compared to total EV by ELISA using a Mann-Whitney test. N= 40, Women (ο) and Men (♦) (revised from (Sun et al, 2019)).

Fig. 2.

Proteins signaling neuronal damage in nEVs. Neuronal EVs were isolated and analyzed by ELISA for (a) HMGB1, N=56, (b) NFL, N=58 and (c) p-T181-tau, N=56. Data are the ratio of pg/ml of each analyte over ALIX (pg/ml). NPN, HIV+ neuropsychologically normal; NPI, HIV+ neuropsychologically impaired. Student’s t test. Women (ο), Men (♦) (revised from (Sun et al, 2019)).

Model evaluation

After repeated (10x) 6-fold cross validation with three machine learning algorithms using variant models that included HMGB1, NFL, p-T181-tau, and clinical features (sex, ethnicity, age, education, CD4 count), SVM generated the best performance in a model consisting of clinical parameters plus HMGB1. All models were evaluated with aggregated Area Under the Curve (AUC) per cross validation instance. Confidence intervals were calculated via bootstrap aggregation of cross validated AUCs (Fig. 3a). For each algorithm, we compared 7 models: Model 1 (M1): Clinical parameters only; Model 2 (M2): M1+ NFL; Model 3 (M3): M1 + HMGB1; Model 4 (M4): M1 + NFL + HMGB1; Model 5: M1 + p-T181-tau; Model 6: M1 + HMGB1 + p-T181-tau; and Model 7: M1 + HMGB1 + NFL + p-T181-tau. Adding p-T181-tau did not improve the performance in models 5–7. We dropped these models and focused on models 1–4. The best performing model was using SVM and M3 with an AUC of 0.82 (±0.16) (Fig. 3b). Adding NFL in model 4 did not add to the prediction value; however, NFL did add to the AUC using Ensemble KNN (Fig. 3c) and AdaBoost (Fig. 3d).

Fig. 3.

Comparison of machine learning algorithms and models. Receiver-operating characteristics (ROC) curve for four different models. Area under the curve (AUC) for the diagnostic performance of the presence of NPI for clinical model (M1) and combined with biomarkers (M2-M4). (a) comparison of 3 algorithms for Model 4 clinical + HMGB1 + NFL, (b) comparison of 4 models with ensemble KNN, (c) comparison of the 4 models with SVM and (d) comparison of the 4 models with AdaBoost. The combined models (M4) outperformed the clinical model predicting NPI (paired t test p <0.05) in all 3 algorithms.

To further understand the contribution of each variable in the model, we performed feature importance analysis (Fig. 4). Because SVM showed the best performance with M3 (Fig. 4a), we evaluated this combination but also included M4 (adding NFL) (Fig. 4b) which added predictive value in the other algorithms. The results showed CD4, HMGB1 and NFL each contributed over 10% of the performance (Fig. 4a,b). The probability of NPI varied nonlinearly with both HMGB1 and NFL (Fig. 4c). Nonetheless, above 0.8ng/ml, a higher concentration of HMGB1 and NFL increased the probability of NPI with steep inflections at 3.2 for HMGB1 and 1.4 for NFL. The lower tail may be due to inaccurate measurement at low concentrations due to limited amount of plasma or small sample size and a few NPI patients with lower nEV HMGB1 and NFL concentrations. Predictably, higher CD4 counts lower the probability of NPI, although highly variable. p-T181-tau did not predict cognitive impairment in HIV infection (Fig. 4c).

Fig. 4.

Feature importance of the best models and probability of cognitive impairment. (a) Permutation importance of the best performing model M3, SVM support vector classifier with HMGB1 and clinical features and (b) M4, SVM support vector classifier with HMGB1, NFL and clinical features. (c) Partial dependency plot between response variable (probability between NPI versus NPN) on y-axis and feature values of CD4, HMGB1, NFL, and p-T181-tau on the x-axis. The greatest value swing is between NFL; the response variable with NFL values above 1.6 influences the likelihood of neurocognitive impairment.

Discussion

We show using clinical data and nEV protein markers, cognitive impairment in HIV+ subjects can be predicted with reasonable accuracy. The addition of nEV biomarkers improves predictive ability. It was not our intention to suggest that these few protein nEV biomarkers are the best targets. On the contrary, we used 2 common biomarkers of neurodegeneration as a benchmark for comparison with future novel biomarkers.

In a recent study in HIV infection, plasma NFL levels were decreased after starting ART, still increased with age and were comparable to CSF NFL (Anderson et al, 2018). HMGB1 protein is an alarmin that is released from necrotic cells and serves as a danger signal to the immune system (Fang et al, 2012). Both active and passive release of HMGB1 from cells can initiate inflammation. HMGB1 is found in the CSF of HIV-infected subjects on ART, suggesting that in spite of treatment, CNS inflammation continues (Gougeon et al, 2017). Taken together, damaged neurons associated with cognitive impairment are releasing increased levels of HMGB1 in nEVs. p-tau-181 is used in many neurodegenerative diseases to herald neuronal damage. An increase in p-tau-181, often with amyloid beta, is a hallmark of AD (Dayon et al, 2017; Santangelo et al, 2019). In our nEV modeling analysis, p-tau-181 did not predict or confirm HAND but may be a good target to differentiate AD from HAND in aging individuals.

Machine learning with nEV targets was recently used to predict AD (Kapogiannis et al, 2019). A large cohort of subjects who developed AD and controls was analyzed longitudinally for several nEV markers. The best model to predict AD included age, sex, nEV diameter, p-tau-231, pY-IRS-1, pSer312-IRS-1 and p-tau-181 with an AUC of 0.8.

To our knowledge, the only machine learning study with HAND, was a recent study using random forest algorithm (Tu et al, 2020). The investigators acquired a substantial array of clinical and demographic variables plus neuropsychological testing. HIV dementia, viral load and polypharmacy were included and the most important predictive variables were CD4+ T cell count with current alcohol use and polypharmacy (Tu et al, 2020). The AUC for HAND was 0.87. Our present study excluded subjects with polypharmacy, opportunistic infections and detectable viral load. Our results strongly suggest that even without these significant co-morbidities, impairment can be differentiated.

The limitation of the present study is the small sample size and limited nEV biomarkers. Machine learning provides an excellent assessment of predicting multiple clinical and laboratory features in view of the nonlinear relationships and to provide feature importance among these predictors. Ultimately, the goal is to identify more nEV protein targets to assess neuroinflammation and neuronal health, improve the predictive value and differentiate HAND from other age-defining illnesses that may affect people living with HIV. Additionally, when treatments are available for cognitive impairment, repeat testing for these nEV biomarkers may signal restoration of neuronal health. Going forward, if more or better proteins that signal neuronal damage are identified, the abundance of diverse clinical variables may not be as relevant.