How Do Statistical Differences in Matrix-sensitive Magnetic Resonance Outcomes Translate Into Clinical Assignment Rules?

There is an inherent difference in perspective between clinical research exploring matrix-sensitive MRI outcome measures and eventual application of these outcomes to individual patients. Basic science studies of group differences resulting from disease or treatment, or from differing demographics, for example, generally focus on the statistical significance of group mean differences. In contrast, the goal of a clinical measurement is to determine whether an individual patient belongs to a given group; this assignment is, in effect, a binary decision between normal and diseased.

Let us take an example from the literature in which enzymatic cartilage degradation is used to model the osteoarthritic process. These experiments are relatively straightforward, and the effects can be quite large; here we focus on T2 measurements as perhaps the most popular matrix-related magnetic resonance outcome. In one study, control cartilage exhibited a mean ± standard deviation (SD) T2_Ctrl value of 55.0 ± 11.1 ms (n = 40), while after 18 hours of degradation with trypsin, values were significantly higher, with T2_Deg = 66.5 ± 10.8 ms (n = 40).¹

How do such results translate into the clinical viewpoint of detection of disease? Clearly, if the T2 values for two groups are very different, and if scatter within each group is small, then the assignment of a sample to control or degraded cartilage will be correspondingly more reliable. That is, the T2 measurement will exhibit a greater sensitivity (SE; accurate detection of degraded cartilage) and specificity (SP; accurate detection of intact control cartilage).

This can be formalized in several ways. First, one must specify how a sample’s T2 measurement will be interpreted in terms of group assignment. One reasonable approach is to assign a new sample with a measured value T2_new to the group whose mean value is closer to T2_new.¹ Thus, if the new sample has T2_new = 61 ms, it would be assigned to the degraded cartilage group because 61 ms is closer to 66.5 ms than it is to 55.0 ms. SE and SP can readily be defined within this framework. Consider two groups, a nondiseased control group, Ctrl, and a group with disease, Dis, with means μ_Ctrl and μ_Dis for parameter p, and a measured value p_new from a patient, Pt. Then:

$$SE=\mathrm{Pr}\{|p_{\mathit{new}}-{\mu }_{\mathit{Dis}}|<|p_{\mathit{new}}-{\mu }_{\mathit{Ctrl}}|\mathrm{Pt}\in \mathit{Dis}\}$$

and

$$SP=\mathrm{Pr}\{|p_{\mathit{new}}-{\mu }_{\mathit{Ctrl}}|<|p_{\mathit{new}}-{\mu }_{\mathit{Dis}}|\mathrm{Pt}\in \mathit{Ctrl}\}$$

In words, the first of these definitions reads: SE is the probability (Pr) that, for a patient with the disease, p_new will be closer to the previously determined mean of the disease group than to the previously determined mean of the control group. That is, a diseased patient will be correctly classified as in fact having disease, based on the measurement of p_new in that patient. Similarly, the second expression may be read as: SP is the probability that, for a patient without the disease, p_new will be closer to the previously determined mean of the control group than to the previously determined mean of the disease group. That is, a nondiseased patient will be correctly classified based on p_new.

To mathematically translate group means and SDs to values for SE and SP, an underlying probability distribution must be assumed. A gaussian distribution of values within each group is a reasonable starting point. Given group means and SDs {μ_Ctrl, σ_Ctrl} and {μ_Dis, σ_Dis}, and μ_Dis > μ_Ctrl, a straightforward calculation yields

$$\begin{array}{l}SE=\mathrm{Pr}\{|p_{\mathit{new}}-{\mu }_{\mathit{Dis}}|<|p_{\mathit{new}}-{\mu }_{\mathit{Ctrl}}|\mathrm{Pt}\in \mathit{Dis}\}\\ =1-\mathrm{\Phi }\left[\frac{({\mu }_{\mathit{Ctrl}}-{\mu }_{\mathit{Dis}})}{(2•{\mathrm{\sigma }}_{\mathit{Dis}})]}\right]\end{array}$$

and

$$\begin{array}{l}SP=\mathrm{Pr}\{|p_{\mathit{new}}-{\mu }_{\mathit{Ctrl}}|<|p_{\mathit{new}}-{\mu }_{\mathit{Dis}}|\mathrm{Pt}\in \mathit{Ctrl}\}\\ =\mathrm{\Phi }\left[\frac{({\mu }_{\mathit{Dis}}-{\mu }_{\mathit{Ctrl}})}{(2•{\mathrm{\sigma }}_{\mathit{Ctrl}})]}\right]\end{array}$$

where, in accordance with standard notation, Φ[z] is the cumulative probability distribution of the standard gaussian distribution, that is, a gaussian with mean μ_Std = 0 and SD σ_Std = 1. The first of these expressions then states that SE equals the integral of the standard gaussian distribution between the integration limits (μ_Ctl − μ_Dis)/(2 • σ_Dis) and ∞; the second indicates that SP equals the integral of the standard gaussian distribution between the limits −∞ and (μ_Dis − μ_Ctrl)/(2 • σ_Ctrl). Similar expressions are obtained for μ_Ctrl > μ_Dis.

With these expressions, the SE and SP of a clinical test, or decision rule, are obtained by simply plugging in the means and SDs for groups Ctrl and Dis. It is generally the case that the population means and SD are unknown, so one must instead use values from previous studies.

For the data presented above, that is, T2_Ctrl = 55.0 ± 11.1 ms and T2_Deg = 66.5 ± 10.8 ms, reflecting a statistically significant increase in T2 with degradation, the accuracy of the T2 measurement to detect whether a new sample belongs to the control group or to a group that underwent enzymatic degradation is rather poor, with SE = 0.70 and SP = 0.70. These values will further decrease if one accounts for random measurement error, which in effect broadens the underlying gaussian distributions and therefore yields even poorer test characteristics.

Let us now look at some of the highest-quality data available on matrix-sensitive magnetic resonance outcome measures for cartilage group differences in human subjects. It was reported that T2 measured at 3 Tesla was larger in subjects with osteoarthritis (T2_Dis = 39.63 ± 2.69 ms; n = 10) compared to controls (T2_Ctrl = 34.74 ± 2.48 ms; n = 10)(P = 0.001).² Using the above equations, these values translate to SE = 0.82 and SP = 0.84 for determining whether a given patient has osteoarthritis based on a T2 measurement. Therefore, these highly statistically significant group differences translate into a clinical decision rule with only modest accuracy.

Further limitations are evident if one incorporates random measurement error; for example, if one (somewhat optimistically) assumes a random error of 2 ms in the T2 measurement, then SE = 0.77 and SP = 0.78. The corresponding values for a 4-ms random error are SE = 0.70 and SP = 0.70. These limitations stem from the narrow dynamic range of magnetic resonance parameters over clinical populations, resulting in parameter value overlap between groups. In contrast, the sensitivity and specificity of magnetic resonance diffusion-weighted imaging for acute ischemic stroke, when performed >12 hours after the event, have been reported to be 92% and 97%, respectively.³

The fact that statistically significant group outcome measures do not necessarily translate into useful clinical outcome measures currently limits the utility of quantitative cartilage matrix–sensitive MRI in clinical decision making. The optimal tactic to create more clinically meaningful tests from basic science studies is not obvious. One promising approach is to apply multivariate statistical techniques, in which changes in several magnetic resonance parameters can be combined to improve SE and SP over values exhibited by any one parameter individually.^4,5 This has the advantage of not requiring new acquisition protocols, or physics and hardware developments, although it does necessitate a change in perspective. The implementation of these somewhat complex analytic approaches will require considerable exploration but may hold a great deal of promise.