Diagnostic Accuracy Using Diffusion Tensor Imaging in the Diagnosis of ALS: A Meta-analysis

Abstract

Rationale and Objectives

A number of studies have reported decreases in fractional anistropy (FA) in amyotrophic lateral sclerosis using diffusion tensor imaging (DTI). The purpose of this study was to perform a meta-analysis in order to estimate the diagnostic test accuracy measures of DTI for the diagnosis of amyotrophic lateral sclerosis (ALS).

Materials and Methods

We searched MEDLINE (1966–April 2011), EMBASE (1999–April 2011), CINAHL (1999–April 2011), and Cochrane (2005–April 2011) databases to identify studies that measured FA in ALS subjects. Human, single-center studies using a DTI region of interest (ROI) or tractography techniques were used to compare FA values along the brain corticospinal tracts between ALS subjects and healthy controls. There were no language restrictions. Independent extraction of articles by 2 authors using predefined data fields including study quality indicators. We identified 30 case-control studies that used region of interest or tractography DTI techniques. We applied binormal receiver operative characteristic (ROC) curve analysis to assign specificity and sensitivity for each study. We applied the bivariate mixed-effects regression model using the Markov Chain Monte Carlo Simulation to calculate summary estimates for the sensitivity and specificity. We used the metan module in Stata, version 11.0, to calculate the area under the ROC curve, diagnostic odds ratio and the test effectiveness summary estimates.

Results

The pooled sensitivity was 0.65 (95% CI 0.61–0.69); the pooled specificity, 0.67 (95% CI 0.63–0.72); the pooled diagnostic odds ratio, 1.88 (95% CI 1.46–2.30); the pooled test effectiveness, 1.04 (95% CI 0.81–1.27); and the pooled area under the ROC curve, 0.76 (95% CI 0.71–0.81). Subanalyses comparing magnetic resonance imaging (MRI) field strength (1.5T vs. 3.0T) and brain location (corticospinal tract average vs. internal capsule) revealed no significant differences in the test accuracy measures. Reference standard used for the diagnosis of ALS was the El Escorial criteria. There was at least moderate heterogeneity between the studies. True study quality is uncertain.

Conclusion

The discriminatory capability of DTI to make a diagnosis of ALS is only modest. There were no significant differences in the diagnostic test accuracy summary estimates with respect to MRI field strength or brain location.

Amyotrophic lateral sclerosis (ALS), a neurodegenerative condition of the corticospinal tract and spinal anterior horn cells, presents with varying degrees of lower motor neuron (LMN) and upper motor neuron (UMN) signs (1,2). The incidence of ALS is estimated to be 1.5–2.7 per 100,000 (3), with a uniformly fatal outcome. Mean survival of 2–4 years after initial diagnosis has been reported (4). Delay between onset of symptoms and diagnosis reaches one year (5), which in part is due to an absence of reliable biomarkers. Thus, objective diagnostic tests to establish and corroborate extent of UMN disease are needed (6).

Diffusion tensor imaging (DTI), an advanced neuroimaging technique, quantifies the local microenvironmental characteristics of water diffusion and evaluates the integrity of white matter fiber tracts (7). DTI holds promise as a potential biomarker to detect the pathologic changes in ALS, particularly UMN involvement (8). One of the key measurements provided by DTI is the fractional anisotropy (FA), which provides a scalar measurement of the degree of water diffusion (9). When the integrity of the white matter tracts is compromised by disease processes that interfere with water diffusion directionality, the FA is reduced (10,11). Over the past decade, a number of published studies have described decreases in the FA of the brain’s corticospinal tract in ALS. However, the vast majority of the publications do not report the test accuracy of DTI to differentiate between diseased and nondiseased populations. In particular, the diagnostic value of DTI in patients presenting with possible or “early” signs of ALS is unknown. If the diagnostic test accuracy measures of this advanced neuroimaging technique are adequate, DTI could be used in the clinical workup of suspected ALS similar to electromyography. The purpose of this study was to aggregate the reported data across the different studies and to estimate summary diagnostic test accuracy measures of DTI in the diagnosis of ALS using standard meta-analysis techniques. Our secondary aims were to investigate study-specific sources of heterogeneity in test accuracy measures.

METHODS

Study Search Strategy

We performed a computerized search to identify relevant articles in MEDLINE (1966–April 2011), EMBASE (1999–April 2011), CINAHL (1999–April 2011), and Cochrane databases (2005–April 2011). For the MEDLINE search, we used the following combination of keywords: [amyotrophic lateral sclerosis or Lou Gehrig’s disease to include all subheadings] and [magnetic resonance imaging or diagnostic imaging or diffusion magnetic resonance imaging or diffusion tensor imaging or fractional anisotropy or anisotropy to include all subheadings]. For the EMBASE search, we used the following combination of keywords: [amyotrophic lateral sclerosis to include all subheadings] and [nuclear magnetic resonance imaging or diagnostic imaging or diffusion tensor imaging or fractional anisotropy to include all subheadings]. For the CINAHL and Cochrane database searches, we used the search words: amyotrophic lateral sclerosis, Lou Gehrig’s disease, imaging, diagnostic imaging, magnetic resonance imaging, diffusion, fractional anisotropy, and diagnostic tests. The results were imported into RefWorks and duplicate references purged. There were no language restrictions. We combined this strategy with a manual search of reference lists from identified articles.

Study Selection

We included a study if it satisfied the following criteria: single-center human studies using DTI region of interest (ROI) or tractography techniques to compare FA values along the brain corticospinal tracts between ALS subjects and healthy controls (HC). The study had to be of an original patient group; follow-up articles based on the same patient group were identified and excluded. Publications that used only voxel based morphometry analysis methods and case studies with fewer than 4 subjects were also excluded. Where data were missing or only graphs provided, we emailed the listed corresponding author twice over a period of 2 weeks for the relevant information.

Data Extraction

Two authors (B.R.F., M.P.) independently evaluated each abstract for inclusion. Next, we obtained full publications for further assessment and data extraction. Two authors (B.R.F., M.P.) independently reviewed each article and reached a final consensus for inclusion. Two investigators (B.R.F., M.P.) abstracted the information from the eligible articles: author, journal, year of publication, number of subjects, demographic and clinical characteristics of subjects, method of FA analysis including brain location, magnetic resonance imaging (MRI) field strength, and the respective mean and standard deviation of fractional anisotropy values for ALS subjects and healthy controls.

Assessment of Methodologic Quality

Two reviewers (B.R.F., M.P.) independently assessed the quality of each study according to the QUADAS criteria (12). These criteria assessed that there were: adequate index test description, adequate reference standard likely to correctly classify disease, adequate reference test description, blinded interpretation of index test, no differential verification bias, no incorporation bias, no partial verification bias, representative spectrum of diseased patients who will receive the test in practice, selection criteria clearly described, short period between index and reference tests and whether uninterpretable test results were reported. Each of these criteria were scored as “yes,” “no,” or “unclear.” In particular, studies that only studied probable or definite diagnosis of ALS by El Escorial criteria (13) were not considered to have a representative disease spectrum. Any disagreements in abstracted results between the reviewers were resolved by discussion and consensus.

Statistical Analysis

Study level analysis

The statistical analysis occurred at the group level because the studies compared average FA values between ALS subjects and HCs. We applied a binormal receiver operating characteristic (ROC) curve analysis to assign cutoff values for the specificity and sensitivity based on the cumulative probability function of the standard normal distribution (14). We assumed equal variance between diseased and nondiseased groups in calculating the threshold for definition of positive test results (15). The specificity and sensitivity values were then estimated. The ROC curve and the area under the binormal ROC curve were then calculated for each study using the respective cutoff values for the specificity and sensitivity. The diagnostic odds ratio and test effectiveness (16) were determined for each study. The 95% confidence intervals for each diagnostic test measure were also calculated.

Meta-analysis model

We applied the bivariate mixed-effects regression model using the Markov Chain Monte Carlo simulation and Gibbs sampling to calculate summary estimates for the sensitivity and specificity (17). For logit transforms of sensitivity and specificity, we assumed independent binomial distributions. We chose Bayesian methodology because in the presence of sparse data, likelihood methods fail to converge (18). Further, when parameters are on their boundary space, estimates of between study correlations are inaccurate (19). To allow the estimates to depend solely on data and provide equivalence to likelihood estimates, we used noninformative priors. Convergence was monitored using trace plots (20). We used the metan module in Stata, version 11.0, to calculate the summary estimates for the area under the ROC curve (AUC), diagnostic odds ratio, and the test effectiveness. The data were intrinsically weighted by using the individual study variances. In addition, subgroup analyses comparing MRI field strength (1.5T versus 3.0T) and brain location (corticospinal tract average versus internal capsule) were performed to generate summary sensitivity, specificity, AUC, diagnostic odds ratio, and test effectiveness values for each subgroup.

Assessment of Heterogeneity

Heterogeneity was assessed using the quantity I² as defined by I² = 100% × (Q – df)/Q, where I² is a measure of consistency across studies, Q is Cochran’s heterogeneity statistic, and df is the degrees of freedom. Higgins et al propose describing I² values of 25%, 50%, and 75% as low, moderate, and high between-trial heterogeneity (21). We obtained the I² values as output of the metan program.

Testing for Publication Bias

Publication bias was evaluated using a scatterplot of the area under the ROC curve versus the corresponding study’s standard error measurement, which should have a symmetric distribution when publication bias is not present. A linear regression of area under the ROC curve versus standard error measurement was also used for publication bias assessment, with P < .05 indicating significant asymmetry.

RESULTS

Study Selection

Numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, is given in Figure 1. The search yielded 1148 literature citations for potential inclusion. We excluded 1088 of the studies because they were not relevant, because they did not involve DTI or humans, or because they were review based articles or letters to the editor. Of the 60 studies that underwent full publication review, another 30 studies were excluded; reasons included only graphical depiction of data, non-ROI/tractography based analysis, not corticospinal tract data, reanalysis of previously reported patient cohort, case study based, or not relevant.

Figure 1.

Flowchart illustrates the selection of studies. DTI, diffusion tensor imaging; ROI, region of interest.

Study Characteristics

The 30 evaluated studies involved 561 subjects diagnosed with ALS and 530 HCs (22–51). The studies were published between 1999 and 2011 (Table 1). Nineteen studies were performed on a 1.5T MRI, 1 study on a 2.9T MRI, 9 studies on 3.0T MRI, and 1 study on a 4.0T MRI. Eighteen studies were performed in Europe, 6 in North America, 5 in Asia, and 1 in South America. We extracted FA values from the corticospinal tract region in 12 studies, the posterior limb of the internal capsule in 14 studies, the brainstem in 3 studies, and the right centrum semiovale in 1 study. The 12 studies reporting values from the corticospinal tract used varying methods to calculate an average corticospinal tract fractional anisotropy. Five of the studies used tractography (ie, fiber tracking techniques) to delineate the corticospinal tract to directly calculate an average of the resolved fiber tracks. Two of the studies using tractography to delineate the corticospinal tract to guide the placement of multiple ROIs along the corticospinal tract and calculate an overall average. The other five studies placed ROIs along the corticospinal tract using visual inspection methods, all of which included the posterior limb of the internal capsule in order to calculate an overall average. Almost all of the studies averaged values from the right and left sides; in studies in which the right and left side values were reported separately, we used the average FA value of the right and left sides and the corresponding pooled standard deviations. All studies except one based the diagnosis of ALS on the El Escorial criteria.

TABLE 1.

Individual Study Characteristics

Author	Year	Origin	MRI	b-Value	Number of Directions	DTI Analysis Method
Agosta et al (23)	2007	Italy	1.5T	900	12	ROI–visual
Agosta et al (24)	2009	Italy	1.5T	Unclear	Unclear	Tractography
Aoki et al (25)	2005	Japan	1.5T	1000	13	ROI–tractography
Bartels et al (26)	2008	Germany	2.9T	1000	24	ROI–tractography
Blain et al (27)	2007	UK	1.5T	1300	64	ROI–visual
Blain et al (28)	2011	UK	1.5T	1300	64	Tractography
Ciccarelli et al (29)	2006	UK	1.5T	1150	54	Tractography
Cosotinni et al (30)	2005	Italy	1.5T	1000	25	ROI–visual
Cosottini et al (31)	2010	Italy	1.5T	1000	31	ROI–visual
Ellis et al (32)	1999	UK	1.5T	620	7	ROI–visual
Filippini et al (33)	2010	UK	3.0T	1000	60	Tractography
Garcia et al (34)	2007	Brazil	1.5T	1000	12	ROI–visual
Graham et al (35)	2004	UK	1.5T	1200	54	ROI–visual
Hong et al (36)	2008	S. Korea	3.0T	1000	25	ROI–tractography
Ito et al (37)	2008	Japan	3.0T	700	6	ROI–tractography
Karlsborg et al (38)	2004	Denmark	1.5T	550	6	ROI–visual
Lombardo et al (39)	2009	Italy	1.5T	1000	25	ROI–visual
Metwalli et al (40)	2010	USA	3.0T	1000	64	ROI–visual
Nair et al (41)	2010	USA	3.0T	1000	30	ROI–visual
Nelles et al (42)	2008	Germany	3.0T	600	16	Tractography
Pyra et al (43)	2010	Canada	1.5T	1000	6	ROI–visual
Roccatagliata et al (44)	2009	Italy	1.5T	1000	51	ROI–visual
Sage et al (45)	2007	Belgium	3.0T	800	16	Tractography
Schimrigk et al (46)	2007	Germany	1.5T	1000	6	ROI–tractography
Senda et al (47)	2009	Japan	3.0T	700	6	ROI–tractography
Valsasina et al (48)	2007	Italy	1.5T	900	12	ROI–visual
Wang et al (49)	2006	USA	3.0T	1000	12	ROI–tractography
Wong et al (50)	2007	Canada	1.5T	1000	6	ROI–visual
Woolley et al (51)	2011	USA	4.0T	800	6	ROI–visual
Yin et al (52)	2004	China	1.5T	1000	25	ROI–visual

MRI, magnetic resonance imaging; ROI, region of interest; ROI–tractography, used tractography generated fiber tracts to aid placement of ROIs; ROI–visual, placed ROIs manually using visual inspection of structural imaging; Tractography, used tractography techniques to generate fiber tracts to directly calculate fractional anisotropy values; UK, United Kingdom; USA, United States of America.

Diagnostic Test Performance Indexes and Summary Estimates

Figures 2 and 3 show the summary ROC curve and the forest plot of the AUC values. For the overall 30 studies, the pooled sensitivity was 0.65 (95% CI 0.61–0.69); the pooled specificity, 0.67 (95% CI 0.63–0.72); the pooled diagnostic odds ratio, 1.88 (95% CI 1.46–2.30); the pooled test effectiveness, 1.04 (95% CI 0.81–1.27); and the area under the ROC curve, 0.76 (95% CI 0.71–0.81).

Figure 2.

Area under the summary ROC curve for DTI. Each circle represents an individual study result. The diamond in the center represents intersection of the summary sensitivity and specificity, the inner dark gray circle represents the 95% confidence interval of the summary sensitivity and specificity, and the outer light gray circle represents the 95% predicted interval. SENS, sensitivity; SPEC, specificity; SROC, summary receiver operating characteristic; AUC, area under the curve.

Figure 3.

Forest plot of AUC for diagnosis of ALS using DTI-generated fractional anisotropy values. The center point represents the point estimated AUC for the respective study and the horizontal line, the 95% confidence interval (CI) for the respective study. The vertical broken line represents the pooled AUC and the boundaries of the hollow diamond represent the 95% CI of the pooled results. nd, number of diseased; meand, mean fractional anistropy (FA) of diseased; sdd, standard deviation FA of diseased controls; nh, number of healthy controls; meanh, mean FA of healthy controls; sdh, standard deviation FA of healthy controls; AUC, area under the curve.

Effect of Field Strength on Diagnostic Performance (1.5T versus 3.0T)

Figure 4a shows the forest plot of the AUC values for the 1.5T and 3.0T studies, Table 2 displays the sensitivity and specificity measures, and Table 3 shows the diagnostic odds ratio and the test effectiveness measures. There were no significant differences in respective pooled diagnostic test accuracy measures between 1.5T versus 3.0T studies.

Figure 4.

AUC Forest plots for (a) 1.5T and 3.0T MRI field strength and (b) corticospinal tract and internal capsule. The center point represents the point estimated AUC for the respective study and; the horizontal line, the 95% confidence interval (CI) for the respective study. The vertical broken line represents the pooled AUC and the boundaries of the hollow diamond represent the 95% CI of the pooled results. nd, number of diseased; meand, mean fractional anistropy (FA) of diseased; sdd, standard deviation FA of diseased; nh, number of healthy controls; meanh, mean FA of healthy controls; sdh, standard deviation FA of healthy controls; AUC, area under the curve.

TABLE 2.

Sensitivity and Specificity Measures

	Sensitivity	Specificity
Overall	0.65 (0.61–0.69)	0.67 (0.63–0.72)
1.5T	0.65 (0.59–0.71)	0.70 (0.61–0.74)
3.0T	0.70 (0.61–0.73)	0.71 (0.57–0.80)
CST	0.68 (0.63–0.74)	0.72 (0.68–0.76)
IC	0.65 (0.54–0.73)	0.66 (0.60–0.73)

CST, corticospinal tract; IC, internal capsule.

TABLE 3.

Diagnostic Odds Ratio and Test Effectiveness Measures

	Diagnostic Odds Ratio	Test Effectiveness
Overall	1.88 (1.46–2.30)	1.04 (0.81–1.27)
1.5T	2.03 (1.45–2.61)	1.12 (0.80–1.44)
3.0T	1.92 (1.36–2.48)	1.06 (0.75–1.37)
CST	2.18 (1.56–2.81)	1.20 (0.86–1.55)
IC	1.93 (1.29–2.56)	1.06 (0.71–1.41)

CST, corticospinal tract; IC, internal capsule.

Effect of Brain Location on Diagnostic Performance (Corticospinal Tract Average versus Internal Capsule)

Figure 4b shows the forest plot of the AUC values for studies reporting corticospinal tract (CST) average measurements compared to those reporting internal capsule (IC) measurements. Table 2 displays the sensitivity and specificity measures, and Table 3 shows the diagnostic odds ratio and the test effectiveness measures. There were no significant differences in the respective pooled diagnostic test accuracy measures between CST versus IC studies.

Assessment of Methodological Quality

Study quality scores ranged from 5 to 10 of a possible total score of 11 (Fig 5). The majority of studies did not incorporate ALS patients with a representative spectrum of disease and studied only patients with probable or definite diagnosis of ALS by the El Escorial criteria. Eighteen of the 30 studies also did not explicitly state the inclusion and/or the exclusion criteria for the participants. Many of the studies did not clearly state that a short period was used between recruitment and the MRI scans. Only 11 of 30 studies explicitly stated that the investigators evaluating the DTI data were blinded to the clinical status of the subject. The majority of the studies also did not clearly comment on whether there were uninterpretable test results.

Figure 5.

Study quality scores. Graph illustrates study quality based on QUADAS criteria, expressed as a percent of studies meeting each criterion.

Assessment of Heterogeneity

The study-specific AUCs for the 30 studies demonstrated at least moderate heterogeneity (I² = 70%; P < .001). The subgroup analysis of the AUC for the MRI field strengths of 1.5T demonstrated a greater degree of heterogeneity (I² = 71%; P < .001) compared to the 3.0T subgroup (I² = 46%; P = .061). Similar degrees of heterogeneity were demonstrated in the subgroup analyses by brain location (CST [I² = 70%; P < .001], IC [I² = 59%; P = .002]).

Publication Bias

There was significant publication bias (bias coefficient = −4.14; P < .001). As seen in Figure 6, there was an asymmetric distribution of the data with a tendency of the studies with a lower standard error to have a higher area under the ROC curve.

Figure 6.

Assessing publication bias. The funnel plot horizontal axis expresses treatment effect, in this instance, measured by area under the receiver operating characteristic curve (AUC). The vertical axis expresses study size, as measured by standard error (SE). Studies with larger standard errors have a wider confidence interval from smaller sample size. The graphed vertical line represents the observed mean AUC and the dashed lines represent the 95% confidence interval limits for the expected distribution for published studies. The points represent the observed distribution of the published studies. Visual inspection of the plot demonstrates the presence of publication bias, with many studies outside the 95% confidence interval limits. Further, the plot demonstrates that studies with smaller study size (ie, larger standard errors) have lower test performance (ie, AUC).

DISCUSSION

In the diagnostic workup of ALS, standard neuroimaging techniques are typically used to exclude other neurological diseases that can cause both UMN and LMN neurological signs such as a compressive lesion involving the upper cervical spinal cord. MRI can also demonstrate subtle changes in ALS patients such as T2 hypointensity in the precentral gyrus (52) and T2 hyperintensity in the corticospinal tract (53,54). However, these findings are nonspecific and lack diagnostic validity (55–58). As a result, advanced neuroimaging techniques, such as DTI, have been investigated to further our understanding of ALS pathophysiology and establish improved disease markers (2,6,48,59).

The ALS DTI literature comprises a series of case-control studies with relatively small populations predominately focused on determining a significant difference between diseased and nondiseased populations (ie, exploratory phase of clinical test development). We conducted what we believe is the first meta-analysis to assess the test accuracy measures of DTI to diagnose ALS by aggregating those small studies. In our meta-analysis, we assessed the sensitivity, specificity, diagnostic odds ratio, test effectiveness, and area under the ROC curve in case-control studies using DTI comparing ALS subjects to healthy controls. Part of the scope of this metaanalysis is to investigate the potential of DTI to progress along the test validation phases.

The overall summary measures of DTI to diagnose ALS yield the relatively low sensitivity of 0.65 and specificity of 0.67 which is reflected in the relatively small area under the ROC curve value of 0.76. To place these results in clinical use context, early test accuracy studies of MRI in the context of multiple sclerosis and herniated nucleus pulposus-caused nerve compression yielded area under the ROC curve values of 0.82 and 0.81–0.84, respectively (60,61). The test effectiveness results also reflected the limited discriminatory capacity of DTI in the setting of ALS. The average difference of the means between diseased and nondiseased groups is just over one standard deviation indicating that there is significant overlap in the FA value distribution curves between the two groups. The diagnostic odds ratio was slightly under 2, indicating that ALS subjects had approximately twice the odds of a positive test result compared to HCs. To provide implications of these results for clinical decision-making, we used Bayesian techniques to calculate the posttest probability after positive and negative DTI results with different pretest probabilities in Figure 7. For example, for an individual with an ALS pretest probability of a 0.50 based on clinical criteria, the posttest probability of ALS is 0.66 with a positive test results and the post-test probability of ALS is 0.34 with a negative test result.

Figure 7.

Posttest probabilities after diffusion tensor imaging (DTI) for hypothetical populations with different prevalence of disease according to Bayes theorem.

It is interesting to note that we did not find differences between the test accuracy measures and either MRI field strength or brain region studied. There has been discussion in the literature with respect to ROI placement for DTI measures in the setting of ALS. Although differences in ROI placement likely generate heterogeneity between studies, it does not appear to significantly impact disease discrimination. Higher field strength magnets are generally considered more desirable in terms of improved signal to noise ratio; however, our subanalysis reveals no difference in diagnostic test performance measures between 1.5T and 3.0T, although the heterogeneity of the studies was considerably lower for the 3.0T studies suggesting greater reliability of the summary accuracy of 3.0T studies.

There are several limitations to our study. None of the studies used the gold standard of reference (autopsy) for confirmation of ALS. The reference test in 29 of 30 publications was based on the El Escorial criteria; the El Escorial criteria have been shown in clinicopathological studies to have a high level of diagnostic concordance with histopathological findings of ALS in subjects with definite or probable ALS based on the initial El Escorial criteria published in 1994. Pathologic confirmation of disease in those with possible or suspected ALS is limited, however (62,63). The El Escorial criteria were subsequently revised to increase their sensitivity, but the criticism still remains (64,65). There was also significant heterogeneity between the studies likely reflecting differences in equipment, patient populations, imaging protocols, and analysis methods reflecting the need for improved protocol uniformity across research centers. We also used only published studies in this meta-analysis; study trials are more likely to be reported if they are statistically significant creating publication bias (66). We did not include studies that used only voxel-based morphometry analysis, which allows for interrogation of the entire brain but requires appropriate statistical analysis to minimize Type I error. Many of the included studies did not explicitly state whether the protocol met certain quality criteria adding uncertainty to the true study quality. As noted previously, the majority of studies did not incorporate ALS patients with possible ALS; the reported results may therefore be biased by a greater effect size. It would also be interesting to evaluate DTI changes with respect to disease duration as well as other clinical characteristics such as revised Amyotrophic Lateral Sclerosis Rating Scale (ALSFRS-R) score and disease progression rate. However, this analysis would be best accomplished using individual level data rather than the group level analysis implemented here. Consensus guidelines for ALS MRI and clinical information protocols (such as presented at the first Neuroimaging Symposium in ALS) (11) should help reduce statistical variation between studies furthering testing validation efforts. Given these limitations and the modest test accuracy measures of DTI in the setting of ALS, future DTI research efforts should be focused on running larger trials with more rigorous methodological quality standards, using consensus research guidelines and combining other advanced neuroimaging techniques with DTI. These steps will be necessary in order to translate and validate this technology to the clinical setting.

There have been a large number of case-control studies using relatively small numbers of ALS patients investigating changes in DTI measurements. Through this meta-analysis, we present a quantitative review of the literature and provide summary estimates of diagnostic test accuracy. The discriminatory capability of DTI to make a diagnosis of ALS is only modest. There were no significant differences in the diagnostic test accuracy summary estimates with respect to MRI field strength or brain location (corticospinal tract average versus internal capsule). These results indicate that DTI will have to be combined with additional potential diagnostic tests (32) in order to have clinical relevance with respect to ALS.

APPENDIX 1. STATISTICAL APPROACH FOR BINORMAL CURVE ANALYSIS

We assumed a binormal ROC curve data analysis (14) in which the proposed diagnostic marker, A, is normally distributed in the healthy control population (nondiseased) and the ALS population (diseased):

$$M~\{\begin{array}{cc}N\left({\mu }_{\stackrel{‒}{D}}{\sigma }_{\stackrel{‒}{D}}^2\right)\hfill & :\mathrm{nondiseased}\hfill \\ N({\mu }_D,{\sigma }_D^2)\hfill & :\mathrm{diseased}\hfill \end{array}\phantom{\}}$$

For each cutoff value c, specificity and sensitivity are defined as

$$\begin{array}{cc}\hfill sp\left(c\right)& =P(M_{\stackrel{‒}{D}}<c)=\Phi \left(\frac{c-{\mu }_{\stackrel{‒}{D}}}{\sigma }\right)\hfill \\ \hfill se\left(c\right)& =P(M_D\ge c)=1-\Phi \left(\frac{c-{\mu }_{\stackrel{‒}{D}}}{\sigma }\right)\hfill \end{array}$$

Where Φ denotes the cumulative probability function of the standard normal distribution. We derived for each study the optimal cutoff based on the Youden index. We assumed equal variance to calculate the cutoff which results in the midpoint between the means of the diseased and the healthy population to generate proper ROC curves to avoid hooking, an accepted approach (15):

$$c=\frac{{\mu }_{\stackrel{‒}{D}}+{\mu }_D}{2}$$