Abstract
Patients with multiple sclerosis (MS) display reduced structural connectivity among brain regions, but the pathogenic mechanisms underlying network disruption are still unknown. We aimed to investigate the association between the loss of diffusion-based structural connectivity, measured with graph theory metrics, and magnetic resonance (MR) markers of microstructural damage. Moreover, we evaluated the cognitive consequences of connectivity changes. We analysed the frontoparietal network in 102 MS participants and 25 healthy volunteers (HV). MR measures included radial diffusivity (RD), as marker of demyelination, and ratios of myo-inositol, N-acetylaspartate and glutamate+glutamine with creatine in white (WM) and grey matter as markers of astrogliosis, neuroaxonal integrity and glutamatergic neurotoxicity. Patients showed decreased global and local efficiency, and increased assortativity (p < 0.01) of the network, as well as increased RD and myo-inositol, and decreased N-acetylaspartate in WM compared with HV (p < 0.05). In patients, the age-adjusted OR of presenting abnormal global and local efficiency was increased for each increment of 0.01 points in RD and myo-inositol, while it was decreased for each increment of 0.01 points in N-acetylaspartate (the increase of N-acetylaspartate reduced the risk of having abnormal connectivity), all in WM. In a multiple logistic regression analysis, the OR of presenting abnormal global efficiency was 0.95 (95% confidence interval, CI: 0.91–0.99, p = 0.011) for each 0.01 increase in N-acetylaspartate, and the OR of presenting abnormal local efficiency was 1.39 (95% CI: 1.14–1.71, p = 0.001) for each 0.01 increase in RD. Patients with abnormal efficiency had worse performance in attention, working memory and processing speed (p < 0.05). In conclusion, patients with MS exhibit decreased structural network efficiency driven by diffuse microstructural impairment of the WM, probably related to demyelination, astroglial and neuroaxonal damage. The accumulation of neuroaxonal pathological burden seems to magnify the risk of global network collapse, while demyelination may contribute to the regional disorganization. These network modifications have negative consequences on cognition.
Keywords: Structural connectivity, Spectroscopy magnetic resonance, Frontoparietal network, Multiple sclerosis, Cognition
Highlights
-
•
The pathogenic mechanisms underlying structural network disruption in MS are unknown.
-
•
The microstructural damage of the white matter contributes to network impairment.
-
•
The worsening of global efficiency is associated with markers of neuroaxonal burden.
-
•
Local efficiency impairment is related to MR measures of demyelination.
-
•
The described network modifications contribute to cognitive dysfunction.
1. Introduction
Multiple sclerosis (MS) is a chronic inflammatory condition of the central nervous system (CNS) that can affect physical and cognitive functioning. CNS injury in MS is the consequence of a complex interaction between the immune system, glia and neurons (Reich et al., 2018). The resulting tissue damage is characterised by demyelination, neuroaxonal impairment, microglial activation, astrocytic disturbances and glutamatergic excitotoxicity, among others (Haider et al., 2016; Kutzelnigg et al., 2005; Kutzelnigg and Lassmann, 2014). Although the focal demyelinated plaques are considered the pathological hallmark of the disease, diffuse pathology is present in normal-appearing white matter (WM) and grey matter (GM) (Kutzelnigg et al., 2005; Kutzelnigg and Lassmann, 2014).
Several advanced magnetic resonance (MR) techniques have been proved to be sensitive to the microstructural changes of the CNS in MS. Radial diffusivity (RD) obtained from diffusion MR imaging (dMRI) has the advantage of being closely related to demyelination (Song et al., 2005), compared with lesion volume in conventional T2-weighted sequences (Filippi et al., 2012), and is able to detect modifications in tissue integrity early in the disease (Llufriu et al., 2014b). However, RD may be modified by axon impairment or inflammation too (Wheeler-Kingshott and Cercignani, 2009). Furthermore, MR spectroscopy can also provide valuable quantitative information of the focal and diffuse microstructural changes in MS. This technique quantifies the concentration of several metabolites including N-acetylaspartate+N-acetylaspartylglutamate (tNAA), an aminoacid present in neurons that has been used as a marker of neuroaxonal integrity and mitochondrial function (Su et al., 2013). Besides, myo-inositol (mI) originates from astrocytes and can increase when astrogliosis or astroglial hypertrophy occurs (Fisher et al., 2002), while glutamate + glutamine (Glu + Gln; abbreviated as Glx) concentrations mainly depends on neuronal integrity and is part of the neurotransmitter pool (Ciccarelli et al., 2014). In patients with MS, several reports have shown that tNAA is decreased while RD, mI and Glx are increased in lesions and normal-appearing brain areas (Chard et al., 2002; Ciccarelli et al., 2014; Llufriu et al., 2014a; Sarchielli et al., 1999). Although those changes cannot be linked to a single histological substrate, they are sensitive markers to detect in vivo neuroaxonal damage, myelin loss, astrogliosis and glutamatergic neurotoxicity respectively.
The MS pathology may impact on network connectivity disrupting the flow of information among brain regions (Shu et al., 2016). Studies of the functional and dMRI-based structural connectome, using graph theoretical analyses, have shown an impairment of the optimal balance between integration and segregation of the network components (Sporns, 2013). Specially, patients present worse network efficiency as well as reorganization changes (Kocevar et al., 2016; Liu et al., 2016; Llufriu et al., 2017; Rocca et al., 2016; Shu et al., 2011). Structural connectivity modifications have been related to the presence of focal WM lesions (He et al., 2009; Shu et al., 2011).
Network connectivity changes seem to contribute to physical and cognitive dysfunction (Li et al., 2013; Llufriu et al., 2017; Rocca et al., 2016; Shu et al., 2011). In particular, attention, working memory and processing speed, which are essential for maintaining everyday life activities, are impaired since the early stages of the illness (Chiaravalloti and Deluca, 2008; Schulz et al., 2006) and have been associated with the integrity of large-scale neural networks connecting frontal and parietal areas among others (Llufriu et al., 2017). Actually, the frontoparietal network (FPN), which connects heteromodal association cortical areas, is a relevant circuit that supports new and highly-demanding cognitive tasks (Harding et al., 2015; Kamali et al., 2014; Parlatini et al., 2016).
Although dMRI studies have shown the disruption of the structural cerebral network in MS patients, the impact of microstructural tissue impairment on the reduction in the connectivity properties remains to be elucidated. The study of the FPN, an area of high prevalence of MS damage (Filippi and Rocca, 2012), could boost our understanding of these mechanisms. Therefore, the main aim of this study was to assess the association between the disruption of structural network connectivity and MR markers of demyelination, neuroaxonal impairment, astrogliosis and glutamatergic changes. Moreover, we evaluated the repercussions of the FPN damage on attention, working memory and processing speed.
2. Material and methods
2.1. Participants
Patients with relapsing-remitting or secondary progressive MS aged between 18 and 65 years old were consecutively recruited at the MS Unit in Hospital Clinic of Barcelona. They must be free from relapses in the last 30 days, and had no history of any other neurologic or psychiatric condition. Global disability was evaluated using the Expanded Disability Status Scale (EDSS) (Kurtzke, 1983). We included a cohort of 102 patients with diagnosis of MS according to 2010 McDonald criteria (Polman et al., 2011) and 25 healthy volunteers (HV). The HVs had neither cognitive complains nor neurologic or psychiatric problems. The Ethics Committee of the Hospital Clinic of Barcelona approved the study and all participants signed an informed consent.
2.2. Cognitive assessment
Attention, working memory and processing speed were only assessed in patients with the three seconds-Paced auditory serial addition test and the Symbol digit modalities test (Boringa et al., 2001). Raw values were stratified by age and education to create z-scores for each test, and a mean score of them (zAttention) was calculated (Sepulcre et al., 2006). Patients were considered impaired in the assessed functions if they scored below −1.5 in zAttention.
2.3. Magnetic resonance acquisition and processing.
2.3.1. Structural and diffusion magnetic resonance imaging acquisition
MR images were acquired on a 3T Magnetom Trio (SIEMENS, Erlanger, Germany) scanner using a 32 channel phased-array head coil, and included 3D-Magnetization Prepared Rapid Acquisition Gradient Echo (MPRAGE) structural, 3D-T2 fluid attenuated inversion recovery (FLAIR) and diffusion-weighted imaging (DWI) sequences. The 3D-structural image had the following acquisition parameters: repetition time = 1800 ms; echo time = 3.01 ms; inversion time = 900 ms; 240 sagittal slices with 0.94 mm isotropic voxel size and 256 × 256 matrix size. The 3D-T2 FLAIR sequence parameters were: repetition time = 5000 ms; echo time = 304 ms; inversion time = 1800 ms; 192 sagittal slices with 0.94 mm isotropic voxel size and 256 × 256 matrix size. High Angular Resolution Diffusion Imaging (HARDI) data was acquired with repetition time = 14,800 ms; echo time = 103 ms; 100 contiguous axial slices; 1.5 mm isotropic voxel size; 154 × 154 matrix size; b value = 1000 s/mm2; 60 diffusion encoding directions and a single baseline image acquired at 0 s/mm2. Also, field map images were generated to correct the distortions caused by field inhomogeneities (echo time 1/echo time 2 = 4.92/7.38 ms, with the same slice prescription, slice thickness and field of view as the HARDI sequence).
WM lesions were segmented in the 3D-MPRAGE sequence using 3D-FLAIR images as reference to improve identification and delineation of the lesions with ITK-SNAP v3.4 toolkit (http://www.itksnap.org/pmwiki/pmwiki.php) or Jim7 software (http://www.xinapse.com/j-im-7-software/). The absolute agreement between both methods measured with the intraclass correlation coefficient was 0.94 (95% confidence interval, CI: 0.61–0.99), indicating excellent reliability. Afterwards, lesion inpainting was applied to the 3D-MPRAGE image to improve segmentation and registration in patients (Battaglini et al., 2012).
2.3.2. Reconstruction of the frontoparietal network
The studied FPN included the rostral branch of the superior longitudinal fasciculus and the cingulate gyrus (Parlatini et al., 2016). A set of 11 regions of interest (ROIs) per hemisphere were selected as nodes for the network reconstruction comprising the frontal (superior and middle frontal) and parietal regions (inferior and superior parietal, supramarginal and precuneus), cingulate cortex (caudal anterior, posterior and the isthmus of the cingulate), and subcortical regions (thalamus and striatum). Cortical GM regions were parcellated with the Desikan-Killiany atlas (Desikan et al., 2006) from Freesurfer 5.3 (https://surfer.nmr.mgh.harvard.edu/), and subcortical GM regions were segmented using the FIRST tool (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FIRST). HARDI images were denoised (Veraart et al., 2016) and corrected for geometric distortions and head motion. The structural connectome was obtained using probabilistic streamlines by high-order fiber orientation distributions and anatomically-constrained tractography framework (Smith et al., 2012) provided in the MRtrix3 software package (http://www.mrtrix.org/). This approach preserves the principal fiber orientation directions, improving tract reconstructions in complex structural architecture and in areas of low FA, such as focal MS lesions. Thus, it may overcome the premature stop of the tracking process when the FA falls below a threshold (Martínez-Heras et al., 2015) (Supplementary Fig. 1). The WM and lesions mask were registered to undistorted HARDI images applying boundary-based registration (Greve and Fischl, 2009) in order to define the tractography seeding, and later on, generate a set of three million streamlines. The default step size = 0.75; curvature = 45° and fiber orientation distribution amplitude threshold = 0.1 were used. Then, the 22 segmented ROIs were used to define the nodes of the network, and connections between them were obtained. Anatomical exclusion criteria postprocessing (Martínez-Heras et al., 2015) was applied to minimize the number of anatomically aberrant connections originated from the tractography (Fig. 1). Afterwards, the 231 connections linking pairs of nodes were represented by a fractional anisotropy-weighted network adjacency matrix for each subject.
Network connectivity was described with graph theoretical analyses from the Brain Connectivity Toolbox (https://sites.google.com/site/bctnet/) and included measures of segregation: local efficiency (the average of the inverse of the shortest path length in the network computed on node neighbourhoods), clustering coefficient (the fraction of node's neighbours that are neighbours of each other), and transitivity (the ratio of triangles to triplets in the network); integration: global efficiency (the average of the inverse of the shortest path length in the network); centrality: betweenness centrality (the fraction of all shortest path in the network that contains a given node), and brain resilience: assortativity (a correlation coefficient between the degrees of all nodes on two opposite ends of a link) (Rubinov and Sporns, 2010). Nodal strength (the sum of weights of links connected to the node) was also obtained.
Finally, RD mean value from the FPN pathway was computed as measure of demyelination in the network.
Connectivity metrics and RD values greater or lesser than 1.5 × interquartile range from the median in the group of patients and in the HVs, were considered outliers and were removed from the analyses.
2.3.3. Magnetic resonance spectroscopy
Two-dimensional chemical shift imaging (2D-CSI) spectroscopy images were obtained with a point-resolved spectroscopy sequence: repetition time = 4000 ms; echo time = 30 ms; slice thickness = 12 mm; field of view = 160 mm; and volume of interest = 80 mm × 80 mm × 12 mm. The 2D-CSI water unsuppressed data set was acquired with the same sequence parameters. The volume of interest (VOI) covered 76.8 cm3 with a set of 8 × 8 voxels, over the corpus callosum, at the cingulate cortex, and was fully involved in the FPN (Fig. 1). The VOI did not include ventricular cerebrospinal fluid in any participant.
Metabolites concentrations were quantified using LCModel (v6.3-1J, Provencher, 1993) and expressed as ratios of tNAA/creatine + phosphocreatine (tCr), mI/tCr and Glx/tCr to provide information on axonal damage, astrogliosis and glutamate + glutamine density, respectively. Only high-quality spectra, defined as signal-to-noise ratio >15, Cramer-Rao lower bounds <15% and full width at half maximum of metabolites <0.07 (Kreis, 2004), was included in the analyses. SIENAX (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/SIENA), performed after lesion filling, was used to obtain WM and GM maps in order to measure the percentage of their content within each spectroscopic voxel. Then, the metabolic estimates for each tissue type were extrapolated from the endpoints of linear regression models in R Software (https://www.r-project.org/) as previously described (Azevedo et al., 2014; Srinivasan et al., 2006) (Fig. 2). Metabolic outliers from each group of participants were removed from the regression models.
2.4. Statistical analysis
Differences between groups in demographic, cognitive data, connectivity values and MR markers of tissue damage were studied using Student's t-test for independent samples or Chi-square tests, as convenient.
We used receiver operating characteristics (ROC) curve analyses to estimate the optimal cut-offs of each connectivity metric that discriminates HV and MS patients using the Youden J statistic (Youden, 1950). Then, patients were categorized as having normal or abnormal network integrity using the estimated cut-offs.
The probability of the association between abnormal network integrity and MR markers was estimated using logistic regression analyses including age as a covariate. We only evaluated as outcome the network metrics that were different between MS participants and HV. Similarly, we only studied MR markers that were different between groups, in order to understand the impact of MS pathology in the decrease of network connectivity. We applied a transformation in all predictors variables (value∗100) to quantify estimated change in age-adjusted odds ratio (OR) of abnormal network connectivity associated per 0.01 units of each MR marker. Then, we ran multiple regression analysis including MR markers and age as predictors in a single model for each outcome. Finally, we evaluated the relationship between connectivity impairment and cognitive performance through Student t-test for independent samples and Chi-square test in order to assess whether the connectivity changes were clinically meaningful.
Statistical analyses were done using STATA v14 for Mac. Significance level was set at two-sided p < 0.05.
3. Results
Clinical, demographic and cognitive data from the participants are summarised in Table 1. Patients with MS and HV were similar in age and gender. Twenty five (25%) patients displayed impaired performance in zAttention. The impaired group had higher percentage of patients with secondary progressive form of the disease and worse EDSS (see Supplementary Table 1).
Table 1.
Multiple sclerosis patients (n = 102) | Healthy volunteers (n = 25) | P-value | |
---|---|---|---|
Female, n (%) | 72 (70.6) | 13 (52) | 0.08 |
Age, years | 42.0 (10.4) | 40.3 (11.7) | 0.50 |
Type of multiple sclerosis | |||
Relapsing remitting | 91 (89.2) | n.a. | n.a. |
Secondary progressive | 11 (10.8) | n.a. | n.a. |
Disease duration, years | 9.5 (9.1) | n.a. | n.a. |
EDSS, median (range) | 2.0 (0–6.5) | n.a. | n.a. |
Disease modifying therapy, n (%) | 28 (27.5) | n.a. | n.a. |
zAttention | −0.5 (1.7) | – | – |
Values expressed as mean (standard deviation). EDSS = Expanded Disability Status Scale; n.a. = not applicable.
Two sample t-test with unequal variance for age and Chi-squared for sex.
Median (range) number of voxels in the 2D-CSI excluded due to low quality per participant was 1 (0–35), similar in HV and patients (p = 0.42).
3.1. Differences in network connectivity and MR data between patients and healthy volunteers
MS patients showed a statistically significant reduction in global and local efficiency and increase of assortativity of the network compared with HV (p < 0.01, Table 2). Such changes were not different between women and men (p > 0.05). The ROC curves relative to global and local efficiency, and assortativity in patients and HV are reported in the Supplementary material (Supplementary Fig. 2). The area under the curve (AUC) was 0.67 (0.56–0.79) and the cut-off maximising the sensitivity and specificity of global efficiency in discriminating patients with MS from HV in our cohort was 0.38, with a sensitivity of 79% and specificity of 51%. The AUC of local efficiency was 0.75 (0.64–0.85) with a cut-off of 0.43, sensitivity of 71% and specificity of 69%. For assortativity, the AUC was 0.67 (0.57–0.77) with a cut-off of −0.12, sensitivity of 61% and specificity of 71%. In our cohort, 70 (69%), 51 (50%) and 62 (61%) patients had abnormal global efficiency, local efficiency and assortativity, respectively.
Table 2.
Multiple sclerosis patients |
Healthy volunteers |
P-value | |||
---|---|---|---|---|---|
n | Mean (SD) | n | Mean (SD) | ||
Network connectivity properties | |||||
Nodal strength | 100 | 6.62 (0.82) | 25 | 6.74 (0.75) | 0.493 |
Transitivity | 100 | 0.33 (0.03) | 25 | 0.34 (0.03) | 0.221 |
Global efficiency | 100 | 0.38 (0.03) | 24 | 0.40 (0.02) | 0.002 |
Local efficiency | 101 | 0.41 (0.02) | 24 | 0.43 (0.02) | <0.001 |
Assortativity | 101 | −0.11 (0.04) | 24 | −0.13 (0.03) | 0.003 |
Clustering coefficient | 100 | 0.34 (0.03) | 25 | 0.35 (0.02) | 0.090 |
Betweenness centrality | 100 | 5.76 (1.49) | 25 | 6.16 (1.44) | 0.225 |
Magnetic resonance markers of tissue damage | |||||
RD in WM | 98 | 0.65 (0.04) | 22 | 0.61 (0.02) | <0.001 |
tNAA/tCr in WM | 101 | 1.80 (0.16) | 25 | 1.88 (0.14) | 0.019 |
tNAA/tCr in GM | 102 | 1.50 (0.13) | 25 | 1.55 (0.13) | 0.107 |
mI/tCr in WM | 99 | 0.78 (0.08) | 25 | 0.72 (0.06) | <0.001 |
mI/tCr in GM | 101 | 0.76 (0.06) | 25 | 0.75 (0.05) | 0.892 |
Glx/tCr in WM | 100 | 1.18 (0.10) | 23 | 1.21 (0.06) | 0.134 |
Glx/tCr in GM | 102 | 1.71 (0.15) | 25 | 1.73 (0.08) | 0.455 |
GM = Grey matter; Glx = glutamate + glutamine ratio with total creatine; mI = myo-inositol ratio with total creatine; tNAA/tCr = total N-acetylaspartate ratio with total creatine; RD = radial diffusivity; SD = standard deviation; WM = white matter. Average radial diffusivity is expressed in units of mm2/s × 10−3.
Two sample t-test with unequal variance for all variables.
Moreover, patients displayed higher RD and mI/tCr, and lower tNAA/tCr concentrations in the network WM, in comparison to HV (p < 0.05, Table 2). We did not observe significant differences in the metabolite concentrations at GM between patients and HV (Table 2).
3.2. Association between MR markers and connectivity status
We evaluated the individual predictive value of RD, mI/tCr and tNAA/tCr, in WM, on the connectivity status. The age-adjusted OR of having abnormal global efficiency was 1.28 for every 0.01 increase in RD, 1.09 for every 0.01 increase in mI/tCr and 0.92 (reduction in 8% of the probability) for every 0.01 increase in tNAA/tCr. Similarly, the age-adjusted OR of presenting abnormal local efficiency was 1.50 for every 0.01 increase in RD, 1.10 for each 0.01 increase in mI/tCr and 0.94 for every 0.01 increase in tNAA/tCr (p < 0.01). On the contrary, there was no association between MR markers and the presence of an abnormal assortativity (Table 3).
Table 3.
MR marker | Abnormal global efficiency |
Abnormal local efficiency |
Abnormal assortativity |
||||||
---|---|---|---|---|---|---|---|---|---|
n | OR (95% CI) | P-value | n | OR (95% CI) | P-value | n | OR (95% CI) | P-value | |
RD in WM | 98 | 1.28 (1.12–1.47) | <0.001 | 98 | 1.50 (1.24–1.81) | <0.001 | 97 | 1.12 (0.99–1.25) | 0.069 |
mI/tCr in WM | 97 | 1.09 (1.02–1.15) | 0.005 | 98 | 1.10 (1.03–1.17) | 0.007 | 98 | 0.98 (0.92–1.03) | 0.406 |
tNAA/tCr in WM | 99 | 0.92 (0.89–0.96) | <0.001 | 100 | 0.94 (0.90–0.97) | <0.001 | 100 | 1.00 (0.97–1.02) | 0.792 |
CI = confidence interval; mI = myo-inositol ratio with total creatine; tNAA/tCr = total N-acetylaspartate ratio with total creatine; RD = radial diffusivity; WM = white matter.
MR data was transformed (value∗100) to obtain age-adjusted ORs to quantify estimated change in the odds of abnormal network functionality associated per 0.01 units of each MR marker. All models had Likelihood ratio test with P-value <0.001 except for those including metabolites as predictor and assortativity as outcome.
In the multiple logistic regression analysis including RD, mI/tCr, tNAA/tCr in WM and age, only tNAA/tCr and RD were associated with global and local efficiency respectively. Thus, the OR of presenting abnormal global efficiency was 0.95 (95% CI: 0.91–0.99) times higher for every 0.01 increase in tNAA/tCr (p = 0.011) and the OR of having an abnormal local efficiency was 1.39 (95% CI: 1.14–1.71, p = 0.001) for each 0.01 increase in RD.
3.3. Cognitive performance in patients with abnormal connectivity
Patients with abnormal global efficiency had lower zAttention than those with normal efficiency [mean (SD) z-score of −0.95 (1.76) versus 0.05 (1.35), p = 0.002] and higher proportion of subjects with impaired performance (34% versus 12%, p = 0.01). Moreover, patients with abnormal local efficiency showed lower zAttention than those with normal local efficiency [z-score of −0.70 (1.71) versus 0.01 (1.40), p = 0.048], while the proportion of participants with zAttention impairment tended to be higher (29% versus 13%, p = 0.008).
4. Discussion
This study shows that the reduction in network connectivity in patients with MS is associated with microstructural modifications in the brain. Specifically, network efficiency was influenced by the levels of RD, tNAA/tCr and mI/tCr, suggesting that demyelination, neuroaxonal and astroglial damage in WM are key contributors to the loss of structural connectivity in MS. Importantly, the risk of having an abnormal global efficiency reaches as the MR marker of neuroaxonal burden progresses, and the probability of presenting an abnormal local efficiency increases as the measure of demyelination worsens. The impairment of the FPN global efficiency has negative consequences in cognition. These results highlight the relevance of the MS diffuse tissue damage beyond focal lesions.
FPN involves highly connected regions that play a key role in global information integration. The cost-efficiency of its components raises the vulnerability of the network to damage (Bullmore and Sporns, 2012; Crossley et al., 2014). We previously demonstrated that connections involved in the FPN are impaired in patients with MS and contribute to low performance in attention and executive functions (Llufriu et al., 2017). Here, we focused our analysis on the rostral branch of the superior longitudinal fasciculus and the cingulate gyrus due to its susceptibility to MS damage and the feasibility to be studied with 2D-CSI without large contamination of the ventricular cerebrospinal fluid. We found a reduction of global and local efficiency of the reconstructed FPN in more than half of the included patients. Changes in global efficiency could be associated with damage of long-range connections that slow down the transfer of information. Meanwhile, modifications in local efficiency could be related to impairment of short-range connections between nearby regions driving to a more randomized configuration of the network and, therefore, increasing wiring costs. On the other hand, the increase in assortativity, which had been described mainly in progressive courses of the disease (Kocevar et al., 2016), contributes to network segregation and worse resilience. All in all, although our analysis was specific of a particular brain circuit, the present results are similar to the disruption of the whole brain network observed in previous reports (Kocevar et al., 2016; Li et al., 2013; Llufriu et al., 2017; Shu et al., 2011).
The main aim of this work was to understand the pathophysiological processes underlying the changes in structural network connectivity in MS. MR data provides in vivo valuable information on the tissue microstructure, but with limited histological specificity. We analysed MR markers of demyelination, neuroaxonal impairment, astrogliosis and glutamate + glutamine concentrations in WM and GM at the FPN, involving lesions and normal-appearing areas. We observed that demyelinating damage, assessed through mean RD, contributed to the impairment of network connectivity properties, supporting previous studies that highlighted the relevance of WM lesions on network disruption (He et al., 2009; Shu et al., 2011). The fact that high levels of RD had a larger impact on local rather than global connectivity would be concordant with a predominant effect of lesions on short-range connections (He et al., 2009), and supports the pathological correlate of more severe demyelination in plaques (Kutzelnigg and Lassmann, 2014). However, future studies combining computational models with empirical structural data are needed to understand the relationship between the presence of lesions and the regional disorganization of brain networks.
Furthermore, astrogliosis and neuroaxonal impairment, measured by myo-inositol and total N-acetylaspartate concentrations, were associated with decreased network efficiency in MS. Astrocytes have been described to be powerful controllers of synapse formation, function, plasticity and elimination, and reactive astrocytes in brain injury express molecules that affect synapse formation and, thus, impair neuronal connectivity (Eroglu and Barres, 2010). Moreover, axonal degeneration can be found in demyelinating lesions and in normal-appearing tissue, and has been related to the loss of myelin-derived trophic support and to immune-mediated injury (Criste et al., 2014; Kutzelnigg and Lassmann, 2014). It is considered one of the most important pathological mechanisms in the accumulation of disability (De Stefano et al., 1995; Llufriu et al., 2014a; Mathiesen et al., 2006). In fact, this study pointed out that the concentration of N-acetylaspartate in the WM was the main risk factor for having an abnormal global efficiency, suggesting that the accumulation of neuroaxonal pathological burden in long-range connections enlarge the risk of network collapse in MS (Schoonheim et al., 2015).
A valuable aspect of the analysis is that the MR markers provided information from the diffuse microstructural changes in WM. Although larger lesion volume inside the FPN increased the adjusted odds of presenting abnormal connectivity (OR 1.002 (95% CI: 1.001–1.004) and 1.003 (95% CI: 1.001–1.006), p < 0.01, for global and local efficiency), the association between connectivity changes and microstructural biomarkers was stronger than with network lesion volume (See Supplementary Table 2). These findings support the relevance of the microstructural damage beyond lesions on network integrity in MS.
The collapse of the studied part of the FPN due to MS damage seems to have negative consequences on cognition. Indeed, we found that patients with impaired network efficiency displayed worse attention, working memory and processing speed. These findings validate the previous reports showing the relationship between MS structural network impairment and cognitive decline and reinforce the role of disconnection mechanisms in the evolution of disability (Dineen et al., 2009). It is important to have in mind that our approach was limited to the analysis of a given network, which facilitated to reach the main aim of the study, but did not evaluate other important circuits involved in attention and working memory such as connections linking the FPN with other relevant networks in the temporal, occipital and deep GM (Dineen et al., 2009; Harding et al., 2015; Llufriu et al., 2017). In addition, other relevant aspects may contribute to the patient's cognitive functioning like intellectual enrichment during life and lifestyle variability (Mollison et al., 2017) that were outside the scope of this study.
The present work has strengths and limitations. The reconstruction of the rostral part of the FPN is challenging, even more in the presence of MS lesions. To partially overcome this issue, we applied new high-order probabilistic tractography methods and anatomical exclusion criteria that improve tracking results in the healthy brain and in patients with MS (Llufriu et al., 2017; Martínez-Heras et al., 2015) (Supplementary Fig. 1). We focused on a selected network, however, results on the relation between tissue disruption and connectivity changes could be extrapolated to other networks. Although RD values are widely used as surrogate metrics of demyelination, results must be taken in caution as previous findings have suggested that this measure is also sensitive to axonal injury or inflammation (Klawiter et al., 2011). Newer advanced MR techniques such as density imaging or myelin water fraction that seem to be more specific of the underlying damage, should confirm the present findings. Finally, spectroscopy markers of tissue damage in GM were not different between patients and HV, similar to other studies (Llufriu et al., 2014a). Although cortical pathology may exist in this cohort of mild-to-moderate disabled patients, the spectroscopy technique used here and the presence of partial volume effects could influence our results.
5. Conclusions
The current study shows that patients with MS exhibit decreased structural network efficiency driven by microstructural impairment of the WM. Neuroaxonal damage, astrogliosis and demyelination, measured by advanced MR techniques, contribute to network disruption. Specifically, the accumulation of neuroaxonal pathological burden substantially increases the risk of global network collapse while demyelination seems to be a key factor for the impairment of local connectivity, pointing out the relevance of diffuse MS damage beyond plaques. The impairment of the FPN has negative consequences on attention, executive functions and processing speed. Further studies evaluating the changes in structural and functional brain connectivity along the disease are needed to shed light the dynamics of network failure and its clinical consequences.
The following are the supplementary data related to this article.
Acknowledgments
Acknowledgements
Contract grant sponsor: This work has been funded by a Proyecto de Investigacion en SaludPI15/00587, S.L., A.S., integrated in the Plan Estatal I+D+I and co-funded by Instituto de Salud Carlos III-Subdireccion General de Evaluacion and the Fondo Europeo de Desarrollo Regional (FEDER); the CERCA Programme of the Generalitat de Catalunya; Red Española de Esclerosis Múltiple (REEM) (RD16/0015/0002, RD16/0015/0003, RD12/0032/0002, RD12/0060/01-02); TEVA SLU. The following authors receive grants from the Instituto de Salud Carlos III: S.L. (JR14/00015), E.ML. (JR16/00006), N.SV·(FI16/00251). M.A. (FI16/00168), I.Z. (CM16/00113). M.S. receives a grant from the Generalitat de Catalunya (SLT002/16/00354). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
We are grateful to Alberto Prats-Galino for his support in the reconstruction of the FPN and Helena Plana for her help in the quantification of the metabolites.
Conflict of interest
The authors have no conflicts of interest to declare.
References
- Azevedo C.J., Kornak J., Chu P., Sampat M., Okuda D.T., Cree B.A., Nelson S.J., Hauser S.L., Pelletier D. In vivo evidence of glutamate toxicity in multiple sclerosis. Ann. Neurol. 2014;76:269–278. doi: 10.1002/ana.24202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Battaglini M., Jenkinson M., De Stefano N. Evaluating and reducing the impact of white matter lesions on brain volume measurements. Hum. Brain Mapp. 2012;33:2062–2071. doi: 10.1002/hbm.21344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boringa J.B., Lazeron R.H., Reuling I.E., Adèr H.J., Pfennings L., Lindeboom J., de Sonneville L.M., Kalkers N.F., Polman C.H. The brief repeatable battery of neuropsychological tests: normative values allow application in multiple sclerosis clinical practice. Mult. Scler. 2001;7:263–267. doi: 10.1177/135245850100700409. [DOI] [PubMed] [Google Scholar]
- Bullmore E., Sporns O. The economy of brain network organization. Nat. Rev. Neurosci. 2012;13:336–349. doi: 10.1038/nrn3214. [DOI] [PubMed] [Google Scholar]
- Chard D.T., McLean M.A., Parker G.J.M., MacManus D.G., Miller D.H. Reproducibility of in vivo metabolite quantification with proton magnetic resonance spectroscopic imaging. J. Magn. Reson. Imaging. 2002;15:219–225. doi: 10.1002/jmri.10043. [DOI] [PubMed] [Google Scholar]
- Chiaravalloti N.D., Deluca J. Cognitive impairment in multiple sclerosis. Lancet Neurol. 2008;7:1139. doi: 10.1016/S1474-4422(08)70259-X. [DOI] [PubMed] [Google Scholar]
- Ciccarelli O., Barkhof F., Bodini B., De Stefano N., Golay X., Nicolay K., Pelletier D., Pouwels P.J.W., Smith S.A., Wheeler-Kingshott C.A.M., Stankoff B., Yousry T., Miller D.H. Pathogenesis of multiple sclerosis: insights from molecular and metabolic imaging. Lancet Neurol. 2014 doi: 10.1016/S1474-4422(14)70101-2. [DOI] [PubMed] [Google Scholar]
- Criste G., Trapp B., Dutta R. Axonal loss in multiple sclerosis: causes and mechanisms. Handb. Clin. Neurol. 2014;122:101–113. doi: 10.1016/B978-0-444-52001-2.00005-4. [DOI] [PubMed] [Google Scholar]
- Crossley N.A., Mechelli A., Scott J., Carletti F., Fox P.T., Mcguire P., Bullmore E.T. The hubs of the human connectome are generally implicated in the anatomy of brain disorders. Brain. 2014;137:2382–2395. doi: 10.1093/brain/awu132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Stefano N., Matthews P.M., Antel J.P., Preul M., Francis G., Arnold D.L. Chemical pathology of acute demyelinating lesions and its correlation with disability. Ann. Neurol. 1995;38:901–909. doi: 10.1002/ana.410380610. [DOI] [PubMed] [Google Scholar]
- Desikan R.S., Ségonne F., Fischl B., Quinn B.T., Dickerson B.C., Blacker D., Buckner R.L., Dale A.M., Maguire R.P., Hyman B.T., Albert M.S., Killiany R.J. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage. 2006;31:968–980. doi: 10.1016/j.neuroimage.2006.01.021. [DOI] [PubMed] [Google Scholar]
- Dineen R.A., Vilisaar J., Hlinka J., Bradshaw C.M., Morgan P.S., Constantinescu C.S., Auer D.P. Disconnection as a mechanism for cognitive dysfunction in multiple sclerosis. Brain. 2009;132:239–249. doi: 10.1093/brain/awn275. [DOI] [PubMed] [Google Scholar]
- Eroglu C., Barres B.A. Regulation of synaptic connectivity by glia. Nature. 2010;468:223–231. doi: 10.1038/nature09612. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Filippi M., Rocca M. The neurologist's dilemma: MS is a grey matter disease that standard clinical and MRI measures cannot assess adequately – No. Mult. Scler. J. 2012;18:557–558. doi: 10.1177/1352458512443995. [DOI] [PubMed] [Google Scholar]
- Filippi M., Rocca M.A., Barkhof F., Brück W., Chen J.T., Comi G., DeLuca G., De Stefano N., Erickson B.J., Evangelou N., Fazekas F., Geurts J.J., Lucchinetti C., Miller D.H., Pelletier D., Popescu B.F.G., Lassmann H., Attendees of the Correlation between Pathological MRI findings in MS workshop Association between pathological and MRI findings in multiple sclerosis. Lancet Neurol. 2012;11:349–360. doi: 10.1016/S1474-4422(12)70003-0. [DOI] [PubMed] [Google Scholar]
- Fisher S.K., Novak J.E., Agranoff B.W. Inositol and higher inositol phosphates in neural tissues: homeostasis, metabolism and functional significance. J. Neurochem. 2002;82:736–754. doi: 10.1046/j.1471-4159.2002.01041.x. [DOI] [PubMed] [Google Scholar]
- Greve D.N., Fischl B. Accurate and robust brain image alignment using boundary-based registration. NeuroImage. 2009;48:63–72. doi: 10.1016/j.neuroimage.2009.06.060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haider L., Zrzavy T., Hametner S., Höftberger R., Bagnato F., Grabner G., Trattnig S., Pfeifenbring S., Brück W., Lassmann H. The topograpy of demyelination and neurodegeneration in the multiple sclerosis brain. Brain. 2016;139:807–815. doi: 10.1093/brain/awv398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harding I.H., Yücel M., Harrison B.J., Pantelis C., Breakspear M. Effective connectivity within the frontoparietal control network differentiates cognitive control and working memory. NeuroImage. 2015;106:144–153. doi: 10.1016/j.neuroimage.2014.11.039. [DOI] [PubMed] [Google Scholar]
- He Y., Dagher A., Chen Z., Charil A., Zijdenbos A., Worsley K., Evans A. Impaired small-world efficiency in structural cortical networks in multiple sclerosis associated with white matter lesion load. Brain. 2009;132:3366–3379. doi: 10.1093/brain/awp089. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kamali A., Flanders A.E., Brody J., Hunter J.V., Hasan K.M. Tracing superior longitudinal fasciculus connectivity in the human brain using high resolution diffusion tensor tractography. Brain Struct. Funct. 2014;219:269–281. doi: 10.1007/s00429-012-0498-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klawiter E.C., Schmidt R.E., Trinkaus K., Liang H.-F., Budde M.D., Naismith R.T., Song S.-K., Cross A.H., Benzinger T.L. Radial diffusivity predicts demyelination in ex vivo multiple sclerosis spinal cords. NeuroImage. 2011;55:1454–1460. doi: 10.1016/j.neuroimage.2011.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kocevar G., Stamile C., Hannoun S., Cotton F., Vukusic S., Durand-Dubief F., Sappey-Marinier D. Graph theory-based brain connectivity for automatic classification of multiple sclerosis clinical courses. Front. Neurosci. 2016;10:1–11. doi: 10.3389/fnins.2016.00478. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kreis R. Issues of spectral quality in clinical 1H-magnetic resonance spectroscopy and a gallery of artifacts. NMR Biomed. 2004;17:361–381. doi: 10.1002/nbm.891. [DOI] [PubMed] [Google Scholar]
- Kurtzke J.F. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS) Neurology. 1983;33:1444–1452. doi: 10.1212/wnl.33.11.1444. [DOI] [PubMed] [Google Scholar]
- Kutzelnigg A., Lassmann H. Handbook of Clinical Neurology. 2014. Pathology of multiple sclerosis and related inflammatory demyelinating diseases; pp. 15–58. [DOI] [PubMed] [Google Scholar]
- Kutzelnigg A., Lucchinetti C.F., Stadelmann C., Brück W., Rauschka H., Bergmann M., Schmidbauer M., Parisi J.E., Lassmann H. Cortical demyelination and diffuse white matter injury in multiple sclerosis. Brain. 2005;128:2705–2712. doi: 10.1093/brain/awh641. [DOI] [PubMed] [Google Scholar]
- Li Y., Jewells V., Kim M., Chen Y., Moon A., Armao D., Troiani L., Markovic-Plese S., Lin W., Shen D. Diffusion tensor imaging based network analysis detects alterations of neuroconnectivity in patients with clinically early relapsing-remitting multiple sclerosis. Hum. Brain Mapp. 2013;34:3376–3391. doi: 10.1002/hbm.22158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu Y., Wang H., Duan Y., Huang J., Ren Z., Ye J., Dong H., Shi F., Li K., Wang J. Functional brain network alterations in clinically isolated syndrome and multiple sclerosis: a graph-based connectome study. Radiology. 2016 doi: 10.1148/radiol.2016152843. [DOI] [PubMed] [Google Scholar]
- Llufriu S., Kornak J., Ratiney H., Oh J., Brenneman D., Cree B.A., Sampat M., Hauser S.L., Nelson S.J., Pelletier D. Magnetic resonance spectroscopy markers of disease progression in multiple sclerosis. JAMA Neurol. 2014;71:840. doi: 10.1001/jamaneurol.2014.895. [DOI] [PubMed] [Google Scholar]
- Llufriu S., Martinez-Heras E., Fortea J., Blanco Y., Berenguer J., Gabilondo I., Ibarretxe-Bilbao N., Falcon C., Sepulveda M., Sola-Valls N., Bargallo N., Graus F., Villoslada P., Saiz A. Cognitive functions in multiple sclerosis: impact of gray matter integrity. Mult. Scler. 2014;20:424–432. doi: 10.1177/1352458513503722. [DOI] [PubMed] [Google Scholar]
- Llufriu S., Martinez-Heras E., Solana E., Sola-Valls N., Sepulveda M., Blanco Y., Martinez-Lapiscina E.H., Andorra M., Villoslada P., Prats-Galino A., Saiz A. Structural networks involved in attention and executive functions in multiple sclerosis. NeuroImage Clin. 2017;13:288–296. doi: 10.1016/j.nicl.2016.11.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martínez-Heras E., Varriano F., Prčkovska V., Laredo C., Andorrà M., Martínez-Lapiscina E.H., Calvo A., Lampert E., Villoslada P., Saiz A., Prats-Galino A., Llufriu S. Improved framework for Tractography reconstruction of the optic radiation. PLoS One. 2015;10 doi: 10.1371/journal.pone.0137064. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mathiesen H.K., Jonsson A., Tscherning T., Hanson L.G., Andresen J., Blinkenberg M., Paulson O.B., Sorensen P.S. Correlation of global N-acetyl aspartate with cognitive impairment in multiple sclerosis. Arch. Neurol. 2006;63:533. doi: 10.1001/archneur.63.4.533. [DOI] [PubMed] [Google Scholar]
- Mollison D., Sellar R., Bastin M., Mollison D., Chandran S., Wardlaw J., Connick P. The clinico-radiological paradox of cognitive function and MRI burden of white matter lesions in people with multiple sclerosis: a systematic review and meta-analysis. PLoS One. 2017;12 doi: 10.1371/journal.pone.0177727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parlatini V., Radua J., Dell'Acqua F., Leslie A., Simmons A., Murphy D.G., Catani M., Thiebaut De Schotten M. Functional segregation and integration within fronto-parietal networks. NeuroImage. 2016;146:367–375. doi: 10.1016/j.neuroimage.2016.08.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Polman C.H., Reingold S.C., Banwell B., Clanet M., Cohen J.A., Filippi M., Fujihara K., Havrdova E., Hutchinson M., Kappos L., Lublin F.D., Montalban X., O'Connor P., Sandberg-Wollheim M., Thompson A.J., Waubant E., Weinshenker B., Wolinsky J.S. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann. Neurol. 2011;69:292–302. doi: 10.1002/ana.22366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Provencher S.W. Estimation of metabolite concentrations from localized in vivo proton NMR spectra. Magn. Reson. Med. 1993;30:672–679. doi: 10.1002/mrm.1910300604. [DOI] [PubMed] [Google Scholar]
- Reich D.S., Lucchinetti C.F., Calabresi P.A. Multiple Sclerosis. N. Engl. J. Med. 2018;378:169–180. doi: 10.1056/NEJMra1401483. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rocca M.A., Valsasina P., Meani A., Falini A., Comi G., Filippi M. Impaired functional integration in multiple sclerosis: a graph theory study. Brain Struct. Funct. 2016;221:115–131. doi: 10.1007/s00429-014-0896-4. [DOI] [PubMed] [Google Scholar]
- Rubinov M., Sporns O. Complex network measures of brain connectivity: uses and interpretations. NeuroImage. 2010;52:1059–1069. doi: 10.1016/j.neuroimage.2009.10.003. [DOI] [PubMed] [Google Scholar]
- Sarchielli P., Presciutti O., Pelliccioli G.P., Tarducci R., Gobbi G., Chiarini P., Alberti A., Vicinanza F., Gallai V. Absolute quantification of brain metabolites by proton magnetic resonance spectroscopy in normal-appearing white matter of multiple sclerosis patients. Brain. 1999;122(Pt 3):513–521. doi: 10.1093/brain/122.3.513. [DOI] [PubMed] [Google Scholar]
- Schoonheim M.M., Meijer K.A., Geurts J.J.G. Network collapse and cognitive impairment in multiple sclerosis. Front. Neurol. 2015;6:82. doi: 10.3389/fneur.2015.00082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schulz D., Kopp B., Kunkel A., Faiss J.H. Cognition in the early stage of multiple sclerosis. J. Neurol. 2006;253:1002–1010. doi: 10.1007/s00415-006-0145-8. [DOI] [PubMed] [Google Scholar]
- Sepulcre J., Vanotti S., Hernández R., Sandoval G., Cáceres F., Garcea O., Villoslada P. Cognitive impairment in patients with multiple sclerosis using the Brief Repeatable Battery-Neuropsychology test. Mult. Scler. J. 2006;12:187–195. doi: 10.1191/1352458506ms1258oa. [DOI] [PubMed] [Google Scholar]
- Shu N., Liu Y., Li K., Duan Y., Wang J., Yu C., Dong H., Ye J., He Y. Diffusion tensor tractography reveals disrupted topological efficiency in white matter structural networks in multiple sclerosis. Cereb. Cortex. 2011;21:2565–2577. doi: 10.1093/cercor/bhr039. [DOI] [PubMed] [Google Scholar]
- Shu N., Duan Y., Xia M., Schoonheim M.M., Huang J., Ren Z., Sun Z., Ye J., Dong H., Shi F.-D., Barkhof F., Li K., Liu Y. Disrupted topological organization of structural and functional brain connectomes in clinically isolated syndrome and multiple sclerosis. Sci. Rep. 2016;6:29383. doi: 10.1038/srep29383. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith R.E., Tournier J.-D., Calamante F., Connelly A. Anatomically-constrained tractography: improved diffusion MRI streamlines tractography through effective use of anatomical information. NeuroImage. 2012;62:1924–1938. doi: 10.1016/j.neuroimage.2012.06.005. [DOI] [PubMed] [Google Scholar]
- Song S.-K., Yoshino J., Le T.Q., Lin S.-J., Sun S.-W., Cross A.H., Armstrong R.C. Demyelination increases radial diffusivity in corpus callosum of mouse brain. NeuroImage. 2005;26:132–140. doi: 10.1016/j.neuroimage.2005.01.028. [DOI] [PubMed] [Google Scholar]
- Sporns O. Network attributes for segregation and integration in the human brain. Curr. Opin. Neurobiol. 2013;23:162–171. doi: 10.1016/j.conb.2012.11.015. [DOI] [PubMed] [Google Scholar]
- Srinivasan R., Cunningham C., Chen A., Vigneron D., Hurd R., Nelson S., Pelletier D. TE-averaged two-dimensional proton spectroscopic imaging of glutamate at 3 T. NeuroImage. 2006;30:1171–1178. doi: 10.1016/j.neuroimage.2005.10.048. [DOI] [PubMed] [Google Scholar]
- Su K., Bourdette D., Forte M. Mitochondrial dysfunction and neurodegeneration in multiple sclerosis. Front. Physiol. 2013;4:169. doi: 10.3389/fphys.2013.00169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Veraart J., Fieremans E., Novikov D.S. Diffusion MRI noise mapping using random matrix theory. Magn. Reson. Med. 2016;76:1582–1593. doi: 10.1002/mrm.26059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wheeler-Kingshott C.A.M., Cercignani M. About “axial” and “radial” diffusivities. Magn. Reson. Med. 2009;61:1255–1260. doi: 10.1002/mrm.21965. [DOI] [PubMed] [Google Scholar]
- Youden W.J. Index for rating diagnostic tests. Cancer. 1950;3:32–35. doi: 10.1002/1097-0142(1950)3:1<32::aid-cncr2820030106>3.0.co;2-3. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.