Highlights
-
•
Networks strengths predicted global cognitive performance in PWH.
-
•
Model generalized to data from an independent PWH sample.
-
•
Network strengths in PWH with HAND were different from either controls or PWH without HAND.
-
•
Network strengths may serve as a potential biomarker to assist HAND diagnosis.
Keywords: Magnetic resonance imaging, HIV, Machine Learning, Neurodegenerative Diseases, Cognitive function, Connectome-based predictive modeling
Abstract
Global cognitive performance plays an important role in the diagnosis of HIV-associated neurocognitive disorders (HAND), yet to date, there is no simple way to measure global cognitive performance in people with HIV (PWH). Here, we performed connectome-based predictive modeling (CPM) to pursue a neural biomarker of global cognitive performance in PWH based on whole-brain resting-state functional connectivity. We built a CPM model that successfully predicted individual differences in global cognitive performance in the training set of 67 PWH by using leave-one-out cross-validation. This model generalized to both 33 novel PWH in the testing set and a subset of 39 PWH who completed a follow-up visit two years later. Furthermore, network strengths identified by the CPM model were significantly different between PWH with HAND and without HAND. Together, these results demonstrate that whole-brain functional network strengths could serve as a potential neural biomarker of global cognitive performance in PWH.
1. Introduction
Combination antiretroviral therapy (cART) is widely available, however, HIV-associated neurocognitive disorders (HAND) remain highly prevalent among people with HIV (PWH) (Heaton et al., 2010, Sacktor et al., 2016). Compared to the pre-cART era, antiretroviral treatment has significantly reduced the prevalence of the most severe form of HAND, HIV-associated dementia (HAD), but the prevalence of mild forms of HAND has remained largely unchanged or even increased (Heaton et al., 2011). Although in general PWH with HAND may experience milder symptoms, they are still facing an increased risk of cART non-adherence, virologic failure, and mortality (Sacktor et al., 2016). Given that age is a major risk factor for HAND and that the HIV population is aging rapidly (Sacktor et al., 2016), it is important to better understand the mechanism and the progression of HAND.
Global deficit score (GDS) is a widely-used method for summarizing the level of cognitive impairment in PWH (Carey et al., 2004). Specifically, it considers both the number of impaired domains as well as the severity of deficits in a comprehensive neuropsychological (NP) battery across seven different cognitive domains (e.g., learning, memory, processing speed, etc.). An individual test score from each cognitive domain (i.e., T-score) is first converted to a deficit score, then averaged across all domains to obtain a GDS. Although GDS has been shown to be sensitive to mild HIV-associated cognitive impairment in PWH (Carey et al., 2004), it faces three empirical challenges. First, it takes hours to administer and score a comprehensive seven-domain NP battery, which generates a GDS. Second, due to the known practice effects in NP testing, it is hard to track changes in GDS over a short retest interval (Duff, 2014), i.e., during the course of a clinical trial study. Third, differences in language, educational background, and culture are barriers for testing in diverse populations.
These practical challenges could be greatly ameliorated by a potential neural biomarker of global cognitive performance in PWH, which can further track changes in global cognitive performance for early HAND intervention. We hypothesized that a well-established connectome-based predictive modeling (CPM) (Finn et al., 2015, Rosenberg et al., 2016, Shen et al., 2017) can be used to identify network connections (i.e., functional connectivity, FC) associated with global mean T-score (GMT), an index of global cognitive performance. GMT was chosen over GDS as it is more suitable for predictive modeling and these two scores are highly correlated, see methods for details. CPM is a fully cross-validated, data-driven machine learning approach that has been applied to either resting-state FC or FC during a sustained attention task to derive a model that predicts sustained attentional performance in novel participants (Rosenberg et al., 2016). This sustained attention connectome-based predictive model has been generalized to predict severity of attention-deficit/hyperactivity disorder (ADHD) symptoms in children (Rosenberg et al., 2016), recall in a reading task (Jangraw et al., 2018), and inhibitory control in aging adults (Fountain-Zaragoza et al., 2019).
CPM has been shown to have great potential in predicting cognitive abilities in healthy adults (Finn et al., 2015, Jiang et al., 2020, Rosenberg et al., 2016). However, it has not been applied to the field of PWH yet. In this study, we apply CPM to identify network connections that can predict global cognitive performance in a training set of 67 PWH using leave-one-out cross-validation (LOOCV), then we test whether the trained CPM model could be generalized to a testing set of 33 new PWH, a 24-month follow-up visit consisting of 39 PWH, and 40 demographically-comparable controls. Last, we investigate the impact of HAND status on network connections identified by CPM in PWH.
2. Methods and materials
2.1. Participants
In total, 104 PWH and 40 demographically-comparable HIV-negative controls were recruited from the greater Washington, D.C. metropolitan area. Participants were screened via telephone interview, followed by an onsite screening visit to ensure the following inclusion/exclusion criteria were met: aged from 41 to 70 years old; able to speak and understand English; had more than seven years of education; had no MRI contraindications such as claustrophobia or metal implants; no illicit substance use within the past three months (mandatory urine toxicology tests was performed during each visit); and no other major neurological and psychiatric disorders (i.e., stroke, loss of consciousness for more than 30 min, or other HIV-unrelated neurological disorders). The study protocol was approved by Georgetown University’s Institutional Review Board and written informed consent from each participant was obtained prior to enrollment. Four PWH were excluded due to visible brain anomalies (n = 2) or having GMT scores more than three standard deviations away from the group mean (n = 2), see quality control section in Supplemental Materials. Hence, the final dataset reported here includes a total of 100 PWH and 40 demographically-comparable controls. HIV disease-related information for the 100 PWH included: current CD4 count 679.5, 482 (median, interquartile range (IQR)); CD4 nadir 194.5, 307 (median, IQR); estimated HIV disease duration 26.1, 9.2 years (median, IQR); 82.4% of PWH were virally suppressed (lower than 20 copies/ml); 98% of PWH were on stable cART, 25% of PWH were diagnosed with HAND (see Table S1 for details). Among the 100 PWH, 39 participated in a follow-up visit (Visit 2) about two years after their first visit (Visit 1), and the data from Visit 2 were used as one of the validating samples. The protocols of Visits 1 and 2 were the same. Controls only had Visit 1 data.
2.2. Neuropsychological testing
To assess neurocognitive function across seven domains, i.e., information processing speed, verbal fluency, learning, memory, executive function, working memory, and motor abilities, that are known to be affected in PWH, the following neuropsychological tests were administered (Table S1): Brief Visuospatial Memory Test-Revised (BVMT-R) (Benedict et al., 1996); WAIS-III Digit Symbol Coding Test, Letter-Number Sequencing, and Symbol Search Subtest (Heaton et al., 2003, Wechsler, 1997); Grooved Pegboard Test (Heaton et al., 1991, Kløve, 1963); Hopkins Verbal Learning Test-Revised (HVLT-R) (Benedict et al., 1998); Controlled Oral Word Association Test (COWAT) (Benton et al., 1983, Gladsjo et al., 1999); Trail Making Test A and B (Heaton et al., 1991); Wide Range Achievement Test (WRAT) 4 Reading (Wilkerson and Robertson, 2006); Stroop Task (Golden and Freshwater, 1978); Wisconsin Card Sorting Test-64 (WCST) (Kongs et al., 2000).
For each participant, a T-score for each cognitive domain was calculated separately using a normative database (Blackstone et al., 2012, Carey et al., 2004), then a GMT, a global cognitive measure, was obtained by averaging the T-scores from each of the seven domains. In addition, a GDS was obtained (Blackstone et al., 2012, Carey et al., 2004) and the Lawton and Brody Activities of Daily Living (ADL, 1969) index was computed to diagnose HAND using the standard Frascati guideline (Antinori et al., 2007). GMT but not GDS was used in this study for model training and testing because PWH with normal cognitive performance would have the same value (0) in GDS even though their GMTs are different (this would lead to much smaller variance, which is not desirable for predictive modeling). The GMT in 100 PWH ranged from 36.6 to 60.5, and was normally distributed.
2.3. MRI acquisition and preprocessing
Structural MRI and resting-state functional MRI (fMRI) were acquired at the Center for Functional and Molecular Imaging at Georgetown University Medical Center using a 3-Tesla Siemens Magnetom Trio with a 12-channel head coil or Prisma-Fit scanner with a 20-channel head coil. The acquisition parameters were exactly the same between the two scanners. The potential effects of data acquisition from different scanners were controlled using the ComBat method (Johnson et al., 2007, Yu et al., 2018). Yu et al. 2018 demonstrated that ComBat removed site effects and enhanced the power to find the age effect in functional connectivity. In addition, previous work from our lab found minimal scanner effects on functional connectivity using two different methods (ComBat or adding the scanner as a covariate) (Yang et al., 2021).
High-resolution T1-weighted images were acquired with 3D-MPRAGE using the following parameters: 1 × 1 × 1 mm3 resolution, TR/TE = 1900/2.52 ms, flip angle = 9°, 160 contiguous 1 mm sagittal slices, FoV = 256x256 matrix. One run of resting state fMRI images was acquired with an echo-planar sequence using the following parameters: flip angle = 90°, TR/TE = 2040/29 ms, FoV = 64 × 64 matrix, 35 interleaved axial slices (4 mm thick, no gap; 3.2 × 3.2 mm2 in plane resolution). There were 264 acquisitions, and the first 5 acquisitions were discarded from analysis to allow for magnetization stabilization.
The Computational Anatomy Toolbox (CAT, version 12.6) (www.neuro.uni-jena.de/cat/) and the CONN functional connectivity toolbox (https://www.nitrc.org/projects/conn/) were used for preprocessing and analyzing structural and functional MRI data, respectively. Default preprocessing procedures in the CAT and CONN software were applied. Standard structural MRI preprocessing in CAT consisted of correction for bias-field inhomogeneities, denoising, skull-stripping, segmentation, and corrections for partial volume estimation. Resting-state functional images were first preprocessed in SPM12 (https://www.fil.ion.ucl.ac.uk/spm/). The preprocessing of functional MRI data included slice-timing correction, realignment, coregistration to structural volume, normalization based on structural normalization parameters obtained from CAT12, outlier identification, and smoothing with an 8 mm FWHM. Normalized images were then processed following the standard CONN pipeline (Whitfield-Gabrieli and Nieto-Castañón, 2012). The temporal processing in CONN included movement regression, removal of signals from CSF and white matter, band passing [0.01 0.1] Hz, detrending, and a structural aCompCor strategy. Additional quality control can be found in Supplemental Materials.
2.4. CPM training and prediction
We first divided 100 PWH into a training set (67 PWH) and a testing set (33 PWH) by using stratified sampling based on race and sex. The purpose of the stratified sampling was to make sure the demographics of these two sets were comparable, as there is considerable population heterogeneity in PWH (Marty et al., 2018, Rubin et al., 2019). For those 39 PWH who completed Visit 2, 24 of their Visit 1 data fell into the training set, while 15 fell into the testing set.
Prior to applying the CPM methods (Shen et al., 2017), we first defined network nodes using a 268-node functional brain atlas (shen268 atlas) (Shen et al., 2013). Next, for each participant, pairwise Pearson correlation coefficients of averaged BOLD signals between every two nodes were calculated and Fisher-transformed. This resulted in a 268 × 268 symmetric connection matrix. To assess the relevance of connections to GMT, the following analysis pipeline was performed. First, partial correlation between each connection and GMT was performed across participants in the training set, controlling for demographics (age, education, sex, and race). The resulting r values were statistically thresholded at different p values, e.g. from 0.01 to 0.001 with 0.001 increments, and from 0.001 to 0.0001 with 0.0001 increments. Those thresholded r values were further separated into a positive tail (connection strength positively correlated with GMT) and a negative tail (connection strength negatively correlated with GMT). The threshold p < 0.0001 was chosen because it provided the best prediction performance for left-out subjects (see next paragraph) in the training set (please note this threshold was chosen solely on the training set and was determined before applying the model to the three validation samples). Positive network strength was used to characterize each participant’s summed connections in the positive tail (i.e., positive network strength was calculated by adding up all the Fisher-transferred correlation coefficients z values of FCs that were significantly positively correlated with GMT). Likewise, negative network strength was calculated by summing the z values of the connections in the negative tail (i.e., negative network strength was the sum of all the z values of FCs that were significantly negatively correlated with GMT).
To ensure the internal validity of this procedure, a LOOCV method was applied. During each loop, predictive network strengths were defined and separated into positive and negative networks using data from 66 PWH (out of 67 PWH in the training set). Next, simple linear regression models (see models 1–6 below) were constructed using network strengths to predict GMT. Last, these models were used to predict the one left-out patient’s GMT based on his or her own network strengths (Model 1–3) or network strengths plus demographics (Models 4–6). There were a total of 67 loops, and each of the 67 PWH was left out once. Given that GMT is corrected for demographics while network strengths are not, including demographics in the linear model might enhance the prediction of GMT (Model 4–6).
Model 1.
Model 2.
Model 3.
Model 4.
Model 5.
Model 6.
Pearson correlations between the observed and the predicted GMT of all 67 PWH were used to evaluate predictive power. Statistical significance of these correlations was achieved by permutation testing with 5000 randomly shuffled samples, i.e., GMT values and demographics were shuffled across PWH 5000 times and then the whole CPM procedure was re-run based on those shuffled samples to obtain a null distribution of correlation coefficients between observed and predicted GMT values.
To test the generalization of the CPM approach, we first identified connections that appeared more than 53 times (80%) in the 67 loops to form positive and negative networks. These connections are depicted in Fig. 3. Then the six regression models were reconstructed in the whole training set (67 PWH) before being applied to predict GMT of three data samples: 40 controls (Visit 1), 33 PWH in the testing set (Visit 1), and the Visit 2 data of 39 PWH. The data in these three samples were not used to train the models nor to select parameters. Pearson correlations between observed and predicted GMT were used to assess predictive power for each of the three samples separately. Statistical significance of these correlations was achieved by permutation testing with 5000 randomly shuffled samples of the training set (to obtain the null distribution of r values in testing sets, Visit 2, and controls, separately). See Fig. 1 for the flowchart of CPM training, prediction, and validation.
2.5. Statistical analyses
The statistical analyses were conducted in SPSS 25.0 (Chicago, IL), and MATLAB 2018b (Math Works, Natick, MA). All statistical analyses were two-tailed, and controlled for age, education (number of years), sex, and race.
Contingency χ2 tests and ANCOVA were used to examine group differences in demographics across four samples (see Table 1). As our sample of participants was predominantly African American (AA), race was defined as a dichotomous variable: AA (1), non-AA (0).
Table 1.
Training set | Testing set | Visit 2 | Controls | p-value | |
---|---|---|---|---|---|
Age | 56.6 (7.1) | 56.2 (5.3) | 58.1 (6.7) | 54.6 (7.1) | n.s.1 |
Education | 14.5 (3.1) | 13.9 (2.9) | 13.7 (3.5) | 14.6 (2.7) | n.s. |
Sex (Female%) | 23.9% | 24.2% | 23.1% | 35.0% | n.s. |
Race (AA2%) | 64.2% | 63.6% | 82.1% | 60.0% | n.s. |
Global Mean T | 48.7 (5.9) | 48.9 (5.3) | 49.7 (6.9) | 49.57 (7.54) | n.s. |
GDS3 | 0.3 (0.3) | 0.3 (0.3) | 0.3 (0.4) | 0.39 (0.47) | n.s. |
Num of Participants | 67 | 33 | 39 | 40 |
Note: data depicted as mean (standard deviation). 1. n.s. a non-significant difference was found across groups; 2. AA, denotes African Americans; 3. GDS, denotes global deficit score. PWH: People with HIV.
We also examined the relationship between network strengths (selected using the training set) and GDS using Visit 1 data from 100 PWH, after adjusting for age, education, sex, and race. ANCOVA was used to assess the effect of HAND diagnoses on positive and negative network strength with age, education, sex, and race as nuisance covariates.
3. Results
There were no significant differences in demographics, GDS, and GMT across the four samples (Table 1). Among PWH, 25 out of 100 (25%) were diagnosed with HAND. Eleven controls had a GDS higher than 0.5 (7 of them had >= 1 GDS, and removing these 7 controls did not change the study conclusion). These controls were not excluded as they met all the inclusion criteria mentioned in the Method and Materials. Detailed information about each sample can be found in Table S1.
3.1. CPM results from the training set (Visit 1 of PWH, n = 67)
By using the LOOCV method, all six regression models performed reasonably well in predicting GMT in unseen PWH and reached statistical significance with 5000 permutation tests (permutation p < 0.05, see Fig. 2). The best linear regression model was the one that took positive network strength, negative network strength, and demographics into account (Model 6).
Six positive connections and one negative connection appeared more than 53 times (80%) in the 67 LOOCV loops (Fig. 3) and were selected for validation analysis. Detailed information about these seven connections can be found in Table S2. Briefly, the positive network included 6 FCs: FCs between the right cerebellum (cerebellum lobe 8) and the left posterior insula, the right posterior insula, the right putamen, the right postcentral gyrus; and FCs between the right postcentral gyrus and the right fusiform gyrus, the left parahippocampal gyrus. The negative network only included 1 FC between the right orbital prefrontal and the left middle temporal lobe.
3.2. CPM validation with the testing set (Visit 1 of PWH, n = 34), Visit 2 (PWH, n = 39), and controls (Visit 1, n = 40)
The six positive and one negative FCs in Fig. 3 were used to reconstruct the six regression models using data from the entire training set (n = 67). Then the six models were used to predict GMT of three unseen data sets: the testing set, Visit 2 data from PWH, and controls. All results are listed in Table 2. The CPM model trained with demographics, positive and negative networks was able to predict GMT of the testing set as well as Visit 2 data from the 39 PWH (permutation p < 0.05, see Table 2, Fig. S1, and Fig. S2. Results stayed the same when adding disease duration as a covariate, see Table S3). In contrast, the model could not be generalized to predict GMT of demographically-comparable controls (at least p greater than 0.4, see Table 2), suggesting that the model was specific to PWH. An additional receiver operating characteristic (ROC) curve analysis was performed to identify HAND diagnoses in the testing set, we found that by using predicted GMT in model 6, 0.6923 sensitivity and 0.7133 specificity were achieved (see Fig. S3).
Table 2.
Regression models |
Training set |
Testing set |
Visit 2 |
Controls |
||||
---|---|---|---|---|---|---|---|---|
r | pp | r | pp | r | pp | r | p | |
Model 1 | 0.350 | 0.047* | 0.166 | 0.112 | 0.318 | 0.037* | −0.137 | 0.400 |
Model 2 | 0.382 | 0.040* | 0.024 | 0.228 | 0.276 | 0.051 | 0.006 | 0.972 |
Model 3 | 0.464 | 0.012* | 0.187 | 0.167 | 0.359 | 0.032* | −0.111 | 0.497 |
Model 4 | 0.449 | 0.013* | 0.303 | 0.079 | 0.296 | 0.043* | 0.079 | 0.626 |
Model 5 | 0.461 | 0.010* | 0.256 | 0.126 | 0.266 | 0.081 | 0.105 | 0.520 |
Model 6 | 0.578 | 0.002** | 0.359 | 0.033* | 0.347 | 0.018* | 0.072 | 0.661 |
Note: pp : the p-value based on a 5000 permutation test. p: the p values for predicting controls’ GMT were based on correlation analysis as the permutation test was not conducted due to poor prediction. * denotes p < 0.05, ** denotes p < 0.01.
3.3. Network strengths, GDS, and HAND diagnosis
As GDS and GMT were highly correlated, the network strengths should be correlated with GDS as well. Indeed, although networks were defined only on the training set of 67 PWH, both positive and negative network strengths strongly correlated with GDS in the 100 PWH (consisting of both the training set and testing set, see Fig. 4). The correlation was stronger with the positive than the negative network strength.
In addition, ANCOVA analysis was conducted to test the difference of positive/negative network strength across three groups: PWH with HAND diagnoses, PWH without HAND diagnoses (non-HAND), and controls, with age, education, sex, and race as covariates. For positive network strength, significant group differences were found (F(1,133) = 10.33, p < 0.0001). Post-hoc analyses revealed significant differences between PWH with HAND and PWH without HAND (F(1,94) = 26.1, p < 5 × 10−6, see Fig. 5, left panel), and between PWH with HAND and controls (F(1,59) = 4.8, p < 0.05, see Fig. 5, left panel), but not between PWH without HAND and controls (p greater than 0.1). For negative network strength, significant group differences were found (F(1,133) = 4.26, p < 0.05). Post-hoc analyses demonstrated a significant difference between PWH with HAND and PWH without HAND (F(1,94) = 9.5, p < 0.01, see Fig. 5 right panel), but not between controls and either PWH group (all p values greater than 0.1).
4. Discussion
In this study, we successfully applied the connectome-based machine learning approach, CPM, to predict global cognitive performance measured by GMT in PWH using network connections at rest. The CPM model trained with 67 PWH was able to generalize to 33 novel PWH and a follow-up visit from 39 PWH, but not to demographically-comparable controls. In addition, network strengths were significantly different between PWH with HAND and without HAND. Taken together, network strengths may serve as a neural biomarker for global cognitive performance (measured by GMT) in PWH and complementary evidence of global cognitive deficit (defined by GDS), although further replication and refinement are needed before application in clinical settings.
CPM has been shown to be a valid and reliable approach to investigate brain-behavior relationships (Beaty et al., 2018, Finn et al., 2015, Jiang et al., 2020, Rosenberg et al., 2016, Shen et al., 2017). Motivated and supported by the emerging evidence that individual differences in functional connectivity patterns are reliable across time (Noble et al., 2019), CPM and similar data-driven approaches have made significant progress in elucidating functional connection-based biomarkers of complex cognitive abilities, including fluid intelligence (Finn et al., 2015), sustained attention (Rosenberg et al., 2016), intelligence quotient (Jiang et al., 2020), and creativity (Beaty et al., 2018). Moreover, a recent study has further shown that resting-state FC patterns are capable of predicting cognitive impairment in an older population (Lin et al., 2018), demonstrating that CPM has a lot of potential in predicting cognitive abilities in clinical settings. To our knowledge, the current study is the first study that has applied CPM to the population of PWH.
The current model, which generalizes across two samples, represents an important first step towards finding a neural biomarker of global cognitive performance in PWH. In particular, this CPM model performed reasonably well when applied to PWH from the second visit two years after the first visit, suggesting that it can be a useful tool to track global cognitive function over time. It is especially useful for multiple tests under short retest intervals, which is hampered by practice effects when using the GDS approach.
Although only a few connections (6 in the positive network and 1 in the negative network) have been identified by the model, it does not imply that these 7 functional connections form the essential neural basis of global cognitive performance in PWH. Rather, it may reflect necessary subprocesses that are vital for brain function in PWH but not in controls, or degree of neural injury that is specific to PWH. This might explain why CPM models trained with PWH were able to predict global cognitive performance of unseen PWH but not demographically-comparable controls. Future studies are needed to test this hypothesis.
Out of the connections identified by the CPM model, the right cerebellum lobe 8 is the main node identified by CPM. In the past two decades, there has been growing evidence that the cerebellum is involved in nearly all the aspects of human behaviors, apart from its well-established role in motor function (Guell and Schmahmann, 2020, King et al., 2019, Schmahmann, 2019, Schmahmann et al., 2019, Schmahmann and Sherman, 1998, Stoodley et al., 2012, Stoodley and Schmahmann, 2009). Specifically, while sensorimotor processing is predominantly located in the cerebellar anterior lobe, the posterior lobe plays an important role in cognitive and emotional processing (Schmahmann, 2019). One exception to this functional division along the anterior-posterior axis is the cerebellar lobe 8, which is the second representation of motor functions but is also involved in various cognitive functions (Guell and Schmahmann, 2020, Schmahmann, 2019, Schmahmann et al., 2019). For example, a recent study demonstrated a functional map of the cerebellum using a comprehensive, multi-domain task battery during fMRI (King et al., 2019). Based on this map, the cerebellar region identified by the CPM model in the current study (located in lobe 8, centered at 23.4, −59.3, −52.1, MNI coordinates) is related to divided attention and motor planning, which are known to be affected in PWH (Marcotte et al., 2006, Moore et al., 2006) and are proposed to be key components of global cognitive performance. In addition, the cerebellum is relevant to health and diseases. For example, the cortico-cerebellar network has been shown to be involved in the aging process in the brain, and disruption to this network is related to cognitive impairment in healthy, elderly adults (Bernard et al., 2013). Indeed, PWH with HAND in our study showed reduced cortico-cerebellar connections as compared to both PWH without HAND and demographically-comparable healthy controls.
In addition to the cerebellum, CPM identified nodes such as bilateral insula, right putamen, and right post central gyrus as also part of the somatosensory network (Seitzman et al., 2020). That is, given their roles in sensorimotor processing, especially information processing speed, one of the bases of cognition (Salthouse, 1996), slow processing speed might partially account for cognitive abnormalities in PWH (Fellows et al., 2014).
The negative network included a connection between the right orbital prefrontal cortex and the left middle temporal lobe. Given that PWH with HAND have higher negative network strength than PWH without HAND, it might reflect a compensatory mechanism that PWH with HAND need more brain regions to be involved in global cognition.
In the present study, we found that the CPM model based on resting state FC could reliably predict global cognitive function (GMT) in PWH (Fig. 2) and differentiate PWH with HAND from PWH without HAND (Fig. S3), suggesting that alterations in spontaneous neural activity might be a useful predictor of cognitive performance and cognitive impairment in PWH. Our fMRI finding is in line with previous neuroHIV studies using magnetoencephalography (MEG) technique (Groff et al., 2020, Lew et al., 2018, Spooner et al., 2020, Wiesman et al., 2018). For instance, Wiesman et al. (2018) found that the spontaneous neural activity in the gamma band differentiated PWH from controls, whereas that in the alpha band differentiated PWH with HAND from controls and PWH without HAND. Given that similar predictions can be made across the present fMRI study and above-mentioned MEG studies, the findings using resting state fMRI techniques and the findings with MEG might share some common neural bases, i.e., neural oscillations in the alpha band, which dominates neural activity at the resting state (de Munck et al., 2007). This assumption is in accordance with previous studies, which found resting-state FC is more related with alpha oscillations than with any other frequency bands (Mantini et al., 2007, Marino et al., 2019, de Pasquale et al., 2010). Thus, future MEG-fMRI studies are needed to directly examine the link between resting state FCs and spontaneous neural activity in the alpha band.
The current study has several advantages and also clinical implications. Compared to previous studies applying different machine learning approaches (e.g., Gaussian process regression, support vector regression) to predict GMT from brain structures (Underwood et al., 2018) or graph properties (Chockanathan et al., 2019, Chockanathan and et al., 2018) without testing generalizability, our study established a CPM model that has stronger generalizability and better interpretability (this is one aspect of CPM, see comparisons between CPM and other machine learning approaches in Shen et al., 2017). In addition, traditional correlation/regression analyses tend to overfit the data and thus have low generalizability (Shen et al., 2017). By using an unbiased, data-driven approach based on whole-brain functional connections, the CPM model has a chance to capture distributed connections that are potential signatures of a complex index of cognitive performance in PWH. In terms of clinical implications, the CPM model could be used as a supplementary method to track global cognitive performance, for example, as a surrogate marker of treatment effectiveness in clinical trials. It is time-efficient (10-min scan time vs. hours of NP battery), and can be administered several times under short retest intervals. More importantly, it is a promising avenue to apply real-time fMRI neurofeedback to change the connection-behavior relationship identified by the CPM model.
Several limitations should be noted. First, although our sample of PWH had reasonably good diversity in terms of demographics, e.g., age ranged from 41 to 70 years old, caution should be taken in generalizing our results to populations that have different demographic backgrounds. Second, the visit 2 sample used in the current study cannot be treated as a completely independent validation sample, as visit 1 data of 24 out of 39 PWH from the visit 2 sample were included in the training set. The purpose of including the visit 2 sample is to show that the relationship between network strengths and GMT were reliable across time (when there is no sign for cognitive impairment). Thus, network strengths identified by the current study have the potential to track changes of global cognitive performance. Third, we have a relatively small sample size, and thus future studies are needed to replicate our study using a larger sample, and out-of-sample replications and model refinement are needed before this neural biomarker can be used in clinical settings.
In conclusion, we demonstrated that functional connectivity at rest might be a good neural biomarker of global cognitive performance in people with HIV, especially for longitudinally tracking changes in cognitive performance. This neural biomarker might be considered as complementary, easy to obtain evidence of cognitive impairment aside from GDS. Future work might examine therapy-induced changes in functional connectivity as a biomarker of treatment efficacy in evaluating new therapies for HAND.
5. Data/code availability statement
The processed data from this study are available on request from X.J. The raw data are not publicly available due to personally identifiable information that could compromise the privacy of research participants. Code is available upon request from F.N.Y.
CRediT authorship contribution statement
Fan Nils Yang: Conceptualization, Methodology, Writing - original draft. Shiva Hassanzadeh-Behbahani: Conceptualization, Writing - review & editing, Investigation. Margarita Bronshteyn: Resources, Investigation. Matthew Dawson: Resources, Investigation. Princy Kumar: Resources. David J. Moore: Writing - review & editing, Resources. Ronald J. Ellis: Writing - review & editing, Resources. Xiong Jiang: Writing - review & editing, Supervision, Validation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
We thank all participants for their time and participation and the assistance for patient care from the Georgetown University Clinical Research Unit (GU-CRU), which has been supported by Grant # UL1TR000101 (previously UL1RR031975) through the Clinical and Translational Science Awards Program (CTSA).
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.nicl.2021.102677.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
References
- Antinori A., Arendt G., Becker J.T., Brew B.J., Byrd D.A., Cherner M., Clifford D.B., Cinque P., Epstein L.G., Goodkin K., Gisslen M., Grant I., Heaton R.K., Joseph J., Marder K., Marra C.M., McArthur J.C., Nunn M., Price R.W., Pulliam L., Robertson K.R., Sacktor N., Valcour V., Wojna V.E. Updated research nosology for HIV-associated neurocognitive disorders. Neurology. 2007;69(18):1789–1799. doi: 10.1212/01.WNL.0000287431.88658.8b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beaty R.E., Kenett Y.N., Christensen A.P., Rosenberg M.D., Benedek M., Chen Q., Fink A., Qiu J., Kwapil T.R., Kane M.J., Silvia P.J. Robust prediction of individual creative ability from brain functional connectivity. Proc. Natl. Acad. Sci. 2018;115(5):1087–1092. doi: 10.1073/pnas.1713532115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Benedict R.H.B., Schretlen D., Groninger L., Brandt J. Hopkins Verbal Learning Test – Revised: Normative Data and Analysis of Inter-Form and Test-Retest Reliability. Clin. Neuropsychol. 1998;12(1):43–55. doi: 10.1076/clin.12.1.43.1726. [DOI] [Google Scholar]
- Benedict R.H.B., Schretlen D., Groninger L., Dobraski M., Shpritz B. Revision of the Brief Visuospatial Memory Test: Studies of normal performance, reliability, and validity. Psychol. Assess. 1996;8:145–153. doi: 10.1037/1040-3590.8.2.145. [DOI] [Google Scholar]
- Benton, A.L., Hamsher, K., Sivan, A.B., 1983. Controlled oral word association test (COWAT). Multiling. Aphasia Exam. 3rd Ed Iowa City IA AJA Assoc.
- Bernard J.A., Peltier S.J., Wiggins J.L., Jaeggi S.M., Buschkuehl M., Fling B.W., Kwak Y., Jonides J., Monk C.S., Seidler R.D. Disrupted cortico-cerebellar connectivity in older adults. NeuroImage. 2013;83:103–119. doi: 10.1016/j.neuroimage.2013.06.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blackstone K., Moore D.J., Franklin D.R., Clifford D.B., Collier A.C., Marra C.M., Gelman B.B., McArthur J.C., Morgello S., Simpson D.M., Ellis R.J., Atkinson J.H., Grant I., Heaton R.K. Defining Neurocognitive Impairment in HIV: Deficit Scores versus Clinical Ratings. Clin. Neuropsychol. 2012;26(6):894–908. doi: 10.1080/13854046.2012.694479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carey C.L., Woods S.P., Gonzalez R., Conover E., Marcotte T.D., Grant I., Heaton R.K. Predictive Validity of Global Deficit Scores in Detecting Neuropsychological Impairment in HIV Infection. J. Clin. Exp. Neuropsychol. 2004;26(3):307–319. doi: 10.1080/13803390490510031. [DOI] [PubMed] [Google Scholar]
- Chockanathan U., DSouza A.M., Abidin A.Z., Schifitto G., Wismüller A. Automated diagnosis of HIV-associated neurocognitive disorders using large-scale Granger causality analysis of resting-state functional MRI. Comput. Biol. Med. 2019;106:24–30. doi: 10.1016/j.compbiomed.2019.01.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chockanathan, U., DSouza, A.M., Abidin, A.Z., Schifitto, G., Wismüller, A., 2018. Identification and functional characterization of HIV-associated neurocognitive disorders with large-scale functional MRI. Proc. SPIE-- Int. Soc. Opt. Eng. 10575. https://doi.org/10.1117/12.2293888. [DOI] [PMC free article] [PubMed]
- de Munck J.C., Gonçalves S.I., Huijboom L., Kuijer J.P.A., Pouwels P.J.W., Heethaar R.M., Lopes da Silva F.H. The hemodynamic response of the alpha rhythm: an EEG/fMRI study. NeuroImage. 2007;35(3):1142–1151. doi: 10.1016/j.neuroimage.2007.01.022. [DOI] [PubMed] [Google Scholar]
- Duff K. One week practice effects in older adults: Tools for assessing cognitive change. Clin. Neuropsychol. 2014;28(5):714–725. doi: 10.1080/13854046.2014.920923. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fellows R.P., Byrd D.A., Morgello S. Effects of information processing speed on learning, memory, and executive functioning in people living with HIV/AIDS. J. Clin. Exp. Neuropsychol. 2014;36(8):806–817. doi: 10.1080/13803395.2014.943696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Finn E.S., Shen X., Scheinost D., Rosenberg M.D., Huang J., Chun M.M., Papademetris X., Constable R.T. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat. Neurosci. 2015;18(11):1664–1671. doi: 10.1038/nn.4135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fountain-Zaragoza S., Samimy S., Rosenberg M.D., Prakash R.S. Connectome-based models predict attentional control in aging adults. NeuroImage. 2019;186:1–13. doi: 10.1016/j.neuroimage.2018.10.074. [DOI] [PubMed] [Google Scholar]
- Gladsjo J.A., Schuman C.C., Evans J.D., Peavy G.M., Miller S.W., Heaton R.K. Norms for letter and category fluency: demographic corrections for age, education, and ethnicity. Assessment. 1999;6(2):147–178. doi: 10.1177/107319119900600204. [DOI] [PubMed] [Google Scholar]
- Golden, C.J., Freshwater, S.M., 1978. Stroop color and word test.
- Groff B.R., Wiesman A.I., Rezich M.T., O'Neill J., Robertson K.R., Fox H.S., Swindells S., Wilson T.W. Age-related visual dynamics in HIV-infected adults with cognitive impairment. Neurol. - Neuroimmunol. Neuroinflammation. 2020;7(3):e690. doi: 10.1212/NXI.0000000000000690. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guell X., Schmahmann J. Cerebellar Functional Anatomy: a Didactic Summary Based on Human fMRI Evidence. The Cerebellum. 2020;19(1):1–5. doi: 10.1007/s12311-019-01083-9. [DOI] [PubMed] [Google Scholar]
- Heaton R.K., Clifford D.B., Franklin D.R., Woods S.P., Ake C., Vaida F., Ellis R.J., Letendre S.L., Marcotte T.D., Atkinson J.H., Rivera-Mindt M., Vigil O.R., Taylor M.J., Collier A.C., Marra C.M., Gelman B.B., McArthur J.C., Morgello S., Simpson D.M., McCutchan J.A., Abramson I., Gamst A., Fennema-Notestine C., Jernigan T.L., Wong J., Grant I. HIV-associated neurocognitive disorders persist in the era of potent antiretroviral therapy: CHARTER Study. Neurology. 2010;75(23):2087–2096. doi: 10.1212/WNL.0b013e318200d727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heaton R.K., Franklin D.R., Ellis R.J., McCutchan J.A., Letendre S.L., LeBlanc S., Corkran S.H., Duarte N.A., Clifford D.B., Woods S.P., Collier A.C., Marra C.M., Morgello S., Mindt M.R., Taylor M.J., Marcotte T.D., Atkinson J.H., Wolfson T., Gelman B.B., McArthur J.C., Simpson D.M., Abramson I., Gamst A., Fennema-Notestine C., Jernigan T.L., Wong J., Grant I. HIV-associated neurocognitive disorders before and during the era of combination antiretroviral therapy: differences in rates, nature, and predictors. J. Neurovirol. 2011;17(1):3–16. doi: 10.1007/s13365-010-0006-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heaton R.K., Grant I., Matthews C.G. Comprehensive Norms for an Expanded Halstead-Reitan Battery: Demographic Corrections, Research Findings, and Clinical Applications. Psychological Assessment. Resources. 1991 [Google Scholar]
- Heaton, R.K., Taylor, M.J., Manly, J., 2003. Demographic effects and use of demographically corrected norms with the WAIS-III and WMS-III, in: Clinical Interpretation of the WAIS-III and WMS-III. Elsevier, pp. 181–210.
- Jangraw D.C., Gonzalez-Castillo J., Handwerker D.A., Ghane M., Rosenberg M.D., Panwar P., Bandettini P.A. A functional connectivity-based neuromarker of sustained attention generalizes to predict recall in a reading task. NeuroImage. 2018;166:99–109. doi: 10.1016/j.neuroimage.2017.10.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jiang R., Calhoun V.D., Fan L., Zuo N., Jung R., Qi S., Lin D., Li J., Zhuo C., Song M., Fu Z., Jiang T., Sui J. Gender Differences in Connectome-based Predictions of Individualized Intelligence Quotient and Sub-domain Scores. Cereb. Cortex. 2020;30:888–900. doi: 10.1093/cercor/bhz134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson W.E., Li C., Rabinovic A. Adjusting batch effects in microarray expression data using empirical Bayes methods. Biostatistics. 2007;8:118–127. doi: 10.1093/biostatistics/kxj037. [DOI] [PubMed] [Google Scholar]
- King M., Hernandez-Castillo C.R., Poldrack R.A., Ivry R.B., Diedrichsen J. Functional boundaries in the human cerebellum revealed by a multi-domain task battery. Nat. Neurosci. 2019;22(8):1371–1378. doi: 10.1038/s41593-019-0436-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- H. Kløve Grooved pegboard 1963 Lafayette Lafayette Instrum.
- Kongs S.K., Thompson L.L., Iverson G.L., Heaton R.K. PAR Lutz; FL: 2000. Wisconsin Card Sorting Test-, 64 Card Version: WCST-64. [Google Scholar]
- Lew B.J., McDermott T.J., Wiesman A.I., O'Neill J., Mills M.S., Robertson K.R., Fox H.S., Swindells S., Wilson T.W. Neural dynamics of selective attention deficits in HIV-associated neurocognitive disorder. Neurology. 2018;91(20):e1860–e1869. doi: 10.1212/WNL.0000000000006504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin Q., Rosenberg M.D., Yoo K., Hsu T.W., O’Connell T.P., Chun M.M. Resting-State Functional Connectivity Predicts Cognitive Impairment Related to Alzheimer’s Disease. Front. Aging Neurosci. 2018;10 doi: 10.3389/fnagi.2018.00094. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mantini D., Perrucci M.G., Del Gratta C., Romani G.L., Corbetta M. Electrophysiological signatures of resting state networks in the human brain. Proc. Natl. Acad. Sci. U. S. A. 2007;104(32):13170–13175. doi: 10.1073/pnas.0700668104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marcotte T.D., Lazzaretto D., Cobb Scott J., Roberts E., Woods S.P., Letendre S., the HNRC Group Visual attention deficits are associated with driving accidents in cognitively-impaired HIV-infected individuals. J. Clin. Exp. Neuropsychol. 2006;28(1):13–28. doi: 10.1080/13803390490918048. [DOI] [PubMed] [Google Scholar]
- Marino M., Arcara G., Porcaro C., Mantini D. Hemodynamic Correlates of Electrophysiological Activity in the Default Mode Network. Front. Neurosci. 2019;13 doi: 10.3389/fnins.2019.01060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marty L., Cazein F., Panjo H., Pillonel J., Costagliola D., Supervie V. Revealing geographical and population heterogeneity in HIV incidence, undiagnosed HIV prevalence and time to diagnosis to improve prevention and care: estimates for France. J. Int. AIDS Soc. 2018;21(3):e25100. doi: 10.1002/jia2.25100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moore, D.J., Masliah, E., Rippeth, J.D., Gonzalez, R., Carey, C.L., Cherner, M., Ellis, R.J., Achim, C.L., Marcotte, T.D., Heaton, R.K., Grant, I., Group, the H., 2006. Cortical and subcortical neurodegeneration is associated with HIV neurocognitive impairment. AIDS 20, 879–887. https://doi.org/10.1097/01.aids.0000218552.69834.00. [DOI] [PubMed]
- Noble S., Scheinost D., Constable R.T. A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis. NeuroImage. 2019;203:116157. doi: 10.1016/j.neuroimage.2019.116157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- de Pasquale F., Della Penna S., Snyder A.Z., Lewis C., Mantini D., Marzetti L., Belardinelli P., Ciancetta L., Pizzella V., Romani G.L., Corbetta M. Temporal dynamics of spontaneous MEG activity in brain networks. Proc. Natl. Acad. Sci. 2010;107(13):6040–6045. doi: 10.1073/pnas.0913863107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rosenberg M.D., Finn E.S., Scheinost D., Papademetris X., Shen X., Constable R.T., Chun M.M. A neuromarker of sustained attention from whole-brain functional connectivity. Nat. Neurosci. 2016;19(1):165–171. doi: 10.1038/nn.4179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rubin L.H., Saylor D., Nakigozi G., Nakasujja N., Robertson K., Kisakye A., Batte J., Mayanja R., Anok A., Lofgren S.M., Boulware D.R., Dastgheyb R., Reynolds S.J., Quinn T.C., Gray R.H., Wawer M.J., Sacktor N. Heterogeneity in neurocognitive change trajectories among people with HIV starting antiretroviral therapy in Rakai. Uganda. J. Neurovirol. 2019;25(6):800–813. doi: 10.1007/s13365-019-00768-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sacktor N., Skolasky R.L., Seaberg E., Munro C., Becker J.T., Martin E., Ragin A., Levine A., Miller E. Prevalence of HIV-associated neurocognitive disorders in the Multicenter AIDS Cohort Study. Neurology. 2016;86(4):334–340. doi: 10.1212/WNL.0000000000002277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Salthouse T.A. The processing-speed theory of adult age differences in cognition. Psychol. Rev. 1996;103:403–428. doi: 10.1037/0033-295x.103.3.403. [DOI] [PubMed] [Google Scholar]
- Schmahmann J.D. The cerebellum and cognition. Neurosci. Lett. The Cerebellum in Health and Disease. 2019;688:62–75. doi: 10.1016/j.neulet.2018.07.005. [DOI] [PubMed] [Google Scholar]
- Schmahmann J.D., Guell X., Stoodley C.J., Halko M.A. The Theory and Neuroscience of Cerebellar Cognition. Annu. Rev. Neurosci. 2019;42(1):337–364. doi: 10.1146/annurev-neuro-070918-050258. [DOI] [PubMed] [Google Scholar]
- Schmahmann J.D., Sherman J.C. The cerebellar cognitive affective syndrome. Brain J. Neurol. 1998;121(Pt 4):561–579. doi: 10.1093/brain/121.4.561. [DOI] [PubMed] [Google Scholar]
- Seitzman B.A., Gratton C., Marek S., Raut R.V., Dosenbach N.U.F., Schlaggar B.L., Petersen S.E., Greene D.J. A set of functionally-defined brain regions with improved representation of the subcortex and cerebellum. NeuroImage. 2020;206:116290. doi: 10.1016/j.neuroimage.2019.116290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shen X., Finn E.S., Scheinost D., Rosenberg M.D., Chun M.M., Papademetris X., Constable R.T. Using connectome-based predictive modeling to predict individual behavior from brain connectivity. Nat. Protoc. 2017;12(3):506–518. doi: 10.1038/nprot.2016.178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shen X., Tokoglu F., Papademetris X., Constable R.T. Groupwise whole-brain parcellation from resting-state fMRI data for network node identification. NeuroImage. 2013;82:403–415. doi: 10.1016/j.neuroimage.2013.05.081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spooner R.K., Wiesman A.I., O’Neill J., Schantell M.D., Fox H.S., Swindells S., Wilson T.W. Prefrontal gating of sensory input differentiates cognitively impaired and unimpaired aging adults with HIV. Brain Commun. 2020;2 doi: 10.1093/braincomms/fcaa080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stoodley C., Schmahmann J. Functional topography in the human cerebellum: A meta-analysis of neuroimaging studies. NeuroImage. 2009;44(2):489–501. doi: 10.1016/j.neuroimage.2008.08.039. [DOI] [PubMed] [Google Scholar]
- Stoodley C.J., Valera E.M., Schmahmann J.D. Functional topography of the cerebellum for motor and cognitive tasks: an fMRI study. NeuroImage. 2012;59(2):1560–1570. doi: 10.1016/j.neuroimage.2011.08.065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Underwood J., Cole J.H., Leech R., Sharp D.J., Winston A. Multivariate pattern analysis of volumetric neuroimaging data and its relationship with cognitive function in treated HIV-disease. J. Acquir. Immune Defic. Syndr. 2018;1999(78):429–436. doi: 10.1097/QAI.0000000000001687. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wechsler D. Psychological Corporation; 1997. WAIS-3., WMS-3: Wechsler adult intelligence scale, Wechsler memory scale: Technical manual. [Google Scholar]
- Whitfield-Gabrieli S., Nieto-Castanon A. Conn: A Functional Connectivity Toolbox for Correlated and Anticorrelated Brain Networks. Brain Connect. 2012;2(3):125–141. doi: 10.1089/brain.2012.0073. [DOI] [PubMed] [Google Scholar]
- Wiesman A.I., O’Neill J., Mills M.S., Robertson K.R., Fox H.S., Swindells S., Wilson T.W. Aberrant occipital dynamics differentiate HIV-infected patients with and without cognitive impairment. Brain. 2018;141:1678–1690. doi: 10.1093/brain/awy097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilkerson G.S., Robertson G.J. Lutz FL Psychol. Assess; Resour: 2006. WRAT4 wide range achievement test professional manual. [Google Scholar]
- Yang F.N., Bronshteyn M., Flowers S.A., Dawson M., Kumar P., Rebeck G.W., Turner R.S., Moore D.J., Ellis R.J., Jiang X. Low CD4+ cell count nadir exacerbates the impacts of APOE ε4 on functional connectivity and memory in adults with HIV. AIDS. 2021;35(5):727–736. doi: 10.1097/QAD.0000000000002840. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yu M., Linn K.A., Cook P.A., Phillips M.L., McInnis M., Fava M., Trivedi M.H., Weissman M.M., Shinohara R.T., Sheline Y.I. Statistical harmonization corrects site effects in functional connectivity measurements from multi-site fMRI data. Hum. Brain Mapp. 2018;39(11):4213–4227. doi: 10.1002/hbm.v39.1110.1002/hbm.24241. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.