Precision and accuracy of diffusion kurtosis estimation and the influence of b-value selection

Abstract

Diffusion kurtosis imaging (DKI) is an extension of diffusion tensor imaging that accounts for leading non-Gaussian diffusion effects. In DKI studies, a wide range of different gradient strengths (b-values) is used, which is known to affect the estimated diffusivity and kurtosis parameters. Hence there is a need to assess the accuracy and precision of the estimated parameters as a function of b-value. This work examines the error in the estimation of mean of the kurtosis tensor (MKT) with respect to the ground truth, using simulations based on a biophysical model for both gray (GM) and white (WM) matter. Model parameters are derived from densely sampled experimental data acquired in ex vivo rat brain and in vivo human brain. Additionally, the variability of MKT is studied using the experimental data. Prevalent fitting protocols are implemented and investigated. The results show strong dependence on the maximum b-value of both net relative error and standard deviation of error for all of the employed fitting protocols. The choice of b-values with minimum MKT estimation error and standard deviation of error was found to depend on the protocol type and the tissue. Protocols that utilize two terms of the cumulant expansion (DKI) were found to achieve minimum error in GM at b-values less than 1 ms/μm², whereas maximal b-values of about 2.5 ms/μm² were found to be optimal in WM. Protocols including additional higher order terms of the cumulant expansion were found to provide higher accuracy for the more commonly used b-value regime in GM, but were associated with higher error in WM. Averaged over multiple voxels, a net average error of around 15% for both WM and GM was observed for the optimal b-value choice. These results suggest caution when using DKI generated metrics for microstructural modeling and when comparing results obtained using different fitting techniques and b-values.

Graphical abstract

1. Introduction

Diffusion weighted MRI (DWI) has unique sensitivity to tissue microstructure. For this reason, DWI has widespread applications in biomedical research and in clinical imaging. In applications of quantitative DWI, the displacement of water molecules underlying the signal attenuation is often approximated by a Gaussian distribution in each dimension. This results in the diffusion tensor signal representation, which yields quantities such as the mean diffusivity (D̄, MD), and fractional anisotropy (FA),¹ and forms the basis of diffusion tensor imaging (DTI). While DTI metrics are very useful, the Gaussian approximation underlying DTI is valid in biological tissue only in a limited regime of low diffusion weighting. At higher b-values, deviations from Gaussianity due to tissue microstructure and compartmentalization impact the signal increasingly. The microstructural information contained in these deviations is valuable and experimentally accessible.

Diffusion kurtosis imaging (DKI)² aims to improve the approximation of the diffusion weighted signal by including the next term in the cumulant expansion (the Taylor expansion of the log signal).^3,4 In this way, DKI accounts for the leading non-Gaussian diffusion effects at higher diffusion weighting. Multiple studies have shown DKI to produce useful biomarkers; for example, it was found to be a valuable tool for early assessment of stroke, with the potential to become a better predictor of tissue outcome.^5-8 Multiple reports also indicate a potential significance of DKI-based metrics in neurological ailments such as gliomas,^9-11 Parkinson's disease,¹² chronic mild stress^13-15 and traumatic brain injury.¹⁶ DKI may also have applications in imaging of the body organs, e.g. kidneys.¹⁷

Despite its promising use as a non-invasive biomarker of disease, there is in general no explicit microstructural interpretation of DKI metrics, although progress has been made recently.^18-22 Specifically, methods such as white matter tract integrity (WMTI),¹⁹ kurtosis analysis of neural diffusion organization (KANDO),¹⁸ LEMONADE (linearly estimated moments provide orientations of neurites and their diffusivities exactly),²⁰ DIVIDE (diffusion variance decomposition),²³ and the neurite diffusion model (NDM)²⁴ all utilize specific microstructural models of diffusion in neural tissue to compute the diffusion and kurtosis tensors as function of tissue model parameters. Subsequently, they use these expressions to estimate model parameters from measured kurtosis and diffusion tensors. Hence, these methods rely on an accurate correspondence between the analytically defined kurtosis and that estimated from the experiments. This is, however, not trivial, given that b-values up to 2.5 ms/μm² (bD ≈ 2.5) are often employed for in vivo studies, while the kurtosis expression originates from a Taylor expansion around b = 0,³ ensuring accuracy only in a sufficiently small neighborhood of 0. Accordingly, the choice of applied gradient strength is known to affect the estimated diffusivity and kurtosis parameters.^3,25 For a model of two Gaussian compartments, for example, underestimation of kurtosis values to a varying degree depending on Db_max has been reported.²⁶ This effect has also been observed experimentally in a phantom study.²⁷

The Cramér-Rao lower bound (CRLB) based optimization has been extensively used for the diffusion weighted experiment design.^28,29 The CRLB provides a lower bound on the variance of almost any estimator. However, when utilized to optimize the DKI experiment, CRLB is used with the assumption that the signal arises from the DKI expression. Thus it does not take into account the complexity of the underlying neural tissue and does not estimate the reliability of the cumulant expansion itself.

Although the performance of various fitting algorithms has been explored in detail²⁵ and the span of b-values for which the cumulant expansion converges (convergence radius) has been calculated for simpler models,³ the dependence of the quality of kurtosis estimation on the range of b-values for microstructural models remains to be addressed. Hence, we here set out to investigate this dependence using simulations and measurements. The simulations use microstructural models, one for gray matter (GM) and another for white matter (WM), to generate noisy diffusion signals. We then fit the DKI expression to these signals and evaluate the DKI metrics to compare with the analytically derived ground truth parameters for each model. We find a relatively high level of discrepancy between the estimates thus obtained, even in the most favorable cases. We then proceed to assess the variability in the evaluation of the kurtosis parameters, which is found to be rather large. These results call for caution when identifying the fourth order cumulant of microstructural models with DKI estimated kurtosis metrics, because the tensor estimate will depend on b-value sampling and analysis method, as well as data quality. This is important for all DKI applications, but especially when attempting to relate kurtosis metrics to specific microstructural tissue parameters.^18-24 In addition to the simulation results, we illustrate the dependence of kurtosis metrics on b-values with DKI data acquired in vivo (human brain) and ex vivo (rat brain).

2. Theory

2.1. Kurtosis from microstructural models

The standard DKI approximation of the log to the normalized DWI signal S(b, n̂)reads

$$log(S(b,\widehat{\mathbf{n}}))=-b\sum _{i,j=1}^3{D}_{i,j}{\widehat{n}}_i{\widehat{n}}_j+\frac{b^2{\overline{D}}^2}{6}\sum _{i,j,k,l=1}^3{W}_{i,j,k,l}{\widehat{n}}_i{\widehat{n}}_j{\widehat{n}}_k{\widehat{n}}_l=-bD(\widehat{\mathbf{n}})+\frac{b^2}{6}{D}^2(\widehat{\mathbf{n}})K(\widehat{\mathbf{n}})$$ (1)

where D_i,j is the i,j element of the rank 2 symmetric diffusion tensor D and W_i,j,k,l is the i, j, k, l element of the symmetric rank 4 kurtosis tensor W, b is the diffusion weighting³⁰ (b-value), and n_i denotes the i th component of measurement direction n̂. The mean of the kurtosis tensor (MKT, W̄) over the sphere S₂ is defined by^26,31,32

$$\overline{W}=\frac{1}{4\pi }\underset{S_2}{\int }\mathrm{d}\widehat{\mathbf{n}}W(\widehat{\mathbf{n}})=\frac{1}{5}(W_{\mathit{\text{xxxx}}}+W_{\mathit{\text{yyyy}}}+W_{\mathit{\text{zzzz}}}+2W_{\mathit{\text{xxyy}}}+2W_{\mathit{\text{xxzz}}}+2w_{\mathit{\text{yyzz}}})=\frac{1}{2}\text{Tr}(\mathbf{W}).$$ (2)

Here, we assess the accuracy of MKT estimation in both GM and WM. This is done using the general “neurites as sticks” compartmentalization³³ with appropriate microstructural model parameters for each tissue type. The general assumption in this framework is that the signal S originates from two non-exchanging compartments:

$$S=(1-f)S_e+f{S}_i.$$ (3)

For GM, the first compartment is assumed to exhibit isotropic Gaussian diffusion. This space in GM accounts for hindered diffusion in extra-neurite space and glial cells (S_e = S_h = S₀ exp(−b·n̂^TdIn̂), where S₀ is the unweighted (b = 0) signal, I is the identity matrix, and d is an effective diffusivity). In neurites, the diffusion signal is approximated by the Gaussian expression, and we employ the spherical harmonic expansion for the neurite orientation distribution. Detailed derivation of the intra-axonal signal S_i = S_c is provided elsewhere.^34,35 In GM, the neurite volume fraction (commonly referred to as v = f) has been found to correlate strongly with myelinated neurite density measured with histology³⁵ and to be sensitive to dendritic remodeling induced by stress.^15,36

For WM, both extracellular (S_e) and intracellular (S_i) compartments were assumed to be non-exchanging and approximated by Gaussian diffusion, so that S_e = exp(−bn̂^TD₁n̂) and S_i = exp(−bn̂^TD₂n̂), as is typically done.³⁷ This modeling approach was chosen for several reasons. First, these representations are known to accurately approximate the measured signal in WM and GM respectively. Second, DKI parameters can be derived analytically for each of the resulting signal expressions, enabling calculation of ground truth values for assessment of estimation fidelity.

Using values obtained from fits to the experimental data, the model was used to generate tissue specific synthetic datasets with different b-value sampling schemes. Noise of variable amplitude was added to evaluate the effect of measurement quality. In this paper we adopted the Gaussian noise model, on account of the least squares estimators having a quantitative bias with non-Gaussian (Rician) data distribution.^38,39 The DKI signal is fit to the noisy signals and DKI parameter estimates from each simulated dataset are obtained subsequently. These values are then compared with their analytical counterparts. Below, we will describe the process of this analytical estimation for both of the cases (GM and WM).

1. As explained above, for GM

   $$S=S_0((1-v)S_h(b,\widehat{\mathbf{n}})+v{S}_c(b,\widehat{\mathbf{n}}))$$ (4)

   where S_c describes the signal originating from within neurites (restricted diffusion), and S_h is the signal from the extra-neurite space. The MKT is then obtained from the Taylor expansion of log(S):²⁴

   $$\overline{W}=\frac{2v}{5{\overline{D}}^2}\text{Tr}\left(T^{\otimes 2}\right)(D_L^2(1-v)-D_L^2)+\frac{3v}{5{\overline{D}}^2}(1-v)(5d^2-\frac{10}{3}{D}_{L}d+\frac{1}{3}{D}_L^2)+\frac{2v}{5{\overline{D}}^2}{D}_L^2$$ (5)

   where D_L is the longitudinal component of the diffusion tensor of the anisotropic intra-neurite compartment, T is the scatter matrix of neurite orientations, T^⊗2 symbolizes the tensor product of T with itself.

2. For WM, MKT can likewise be expressed in terms of the model parameters. The relevant signal equation is

   $$S(b,\widehat{\mathbf{n}})=fexp(-b{\widehat{\mathbf{n}}}^{\mathrm{T}}{\mathbf{D}}_1\widehat{\mathbf{n}})+(1-f)exp(-b{\widehat{\mathbf{n}}}^{\mathrm{T}}{\mathbf{D}}_2\widehat{\mathbf{n}})$$ (6)

   where f is the volume fraction of compartment 1, and D₁, D₂ are the diffusion tensors in each of the compartments. The cumulant expansion of Equation 6 is

   $$ln(S)=-b((f{D}_1)+(1-f)D_2)+b^2(\frac{1}{2}f(1-f){(D_1-D_2)}^2)+O(b^3)$$ (7)

   where D_i = D_i(n̂) = n̂^TD_in̂.

   Using Equation 7, the kurtosis in each direction can be calculated,²⁶

   $$K(\widehat{\mathbf{n}})D{(\widehat{\mathbf{n}})}^2=3f(1-f){({\widehat{\mathbf{n}}}^{\mathrm{T}}{\mathbf{D}}_1\widehat{\mathbf{n}}-{\widehat{\mathbf{n}}}^{\mathrm{T}}{\mathbf{D}}_2\widehat{\mathbf{n}})}^2=W(\widehat{\mathbf{n}}){\overline{D}}^2,$$ (8)

   and the MKT is evaluated using the same approach as in previous works,^19,21 by numerically integrating W(n̂) over the sphere.

2.2. Fitting algorithms

A variety of fitting procedures are currently in use. In this work, we implemented six different routines in MATLAB (MathWorks, Natick, MA, USA) to avoid having our results depend on the choice of a particular algorithm. Specifically, we used weighted linear least squares fitting (LLSQ), non-linear least squares fitting, constrained non-linear least squares, constrained linear least squares,^25,40 and maximum likelihood estimation (MLE).⁴¹ Please refer to the descriptive and comparative studies^25,38 for more details about their implementation and comparison of methods. Additionally, we implemented a direction-wise fit (DWF), where the signal was fit with linear least squares along each gradient direction separately, to obtain apparent diffusion and kurtosis coefficients. Finally, in an attempt to absorb potential contributions from cumulants higher than fourth order, we implemented a variant of DWF, termed DWF+, in which the sixth order term (b³) of the cumulant expansion is included and discarded after the fit. MKT was subsequently estimated from each fit.²⁶ We found the results to depend only marginally on the choice of the fitting algorithm. Since the differences between the algorithms have already been comprehensively explored in the literature, it was decided to simplify the presentation by reporting only two fitting procedures: the weighted linear least square algorithm (LLSQ) and the algorithm that accounts for higher terms of cumulant expansion (DWF+). Both algorithms are briefly described next.

One of the widely practiced techniques for estimating elements of diffusion and kurtosis tensors from the diffusion weighted signal uses linear least squares. In this case, the log signal is fit as a linear function of b using Equation 1. Due to the non-linear nature of the logarithm, the variance of the logarithm of the diffusion signal depends on the signal itself^40-43 (heteroscedasticity), and consequently weighting must be applied to the linear least squares algorithm. Our LLSQ implementation utilizes the linear estimator (θ̂_WLS) suggested in previous works:^44,45

$${\widehat{\theta }}_{\text{WLS}}={({\mathbf{X}}^{\mathrm{T}}\omega \mathbf{X})}^{-1}{\mathbf{X}}^{\mathrm{T}}\omega logS$$ (9)

where X is a design matrix of size N × 22 that combines the b-value and gradient direction and is given in detail elsewhere,²⁵ and logS is the logarithm of the signal. The diagonal weight matrix ω is updated using an iterative dual-step algorithm,^44,45 where the weights are the preliminary estimates of S²(b, n̂), predicted using θ̂ = (X^TX)⁻¹X^T logS so that ω_n = diag(exp(2Xθ̂_n–1)). Complete implementation details are provided in previous works.^25,46

Our direction-wise fit (DWF+) is a two-step algorithm, similar to one implemented in Diffusion Kurtosis Estimator^26,42,47 (available online^*), where in the first step the apparent diffusivity D(n̂) and apparent kurtosis K(n̂) are estimated for each experimental direction. In this study an additional higher order term is included, corresponding to

$$lnS(b)\approx lnS(0)-bD(\widehat{\mathbf{n}})+\frac{1}{6}{b}^2{D}^2(\widehat{\mathbf{n}})K(\widehat{\mathbf{n}})+b^{3}L(\widehat{\mathbf{n}})$$ (10)

where L(n̂) is a coefficient of the b³ term, which depends on direction. In the second step, the apparent diffusion and kurtosis values are used to estimate all elements of the diffusion and kurtosis tensors.

Positivity of the eigenvalues of the diffusion tensor D is required for physically acceptable solutions. This constraint is the only one applied to the fitting procedures. In DWF+, this constraint was achieved using the eigendecomposition of the diffusion tensor, leaving the three eigenvalues and three Euler angles as fitting parameters. In LLSQ, the same constraint was realized instead through Cholesky decomposition of the diffusion tensor.⁴³

3. Methods

3.1. Experimental data protocols

For our ex vivo experiments, a rat brain was fixed,⁴⁸ and the hemispheres were separated and stored in 4% paraformaldehyde. Prior to imaging, one hemisphere was washed in phosphate buffered saline (PBS) for 24 h to increase signal by removal of excess fixative. Data were acquired with a standard diffusion weighted spin echo pulse sequence using a 9.4 T MRI system (Bruker BioSpin, Germany) equipped with a 15 mm quadrature coil. A total of 25 b-values between 0 and 5 ms/μm² were acquired, each one along 33 distinct directions. These directions were a combination of a three-dimensional 24-point spherical seven-design⁵ and the nine directions used for fast kurtosis imaging.^31,32 Imaging parameters were echo time (T_E) = 23.3 ms, repetition time (T_R) = 4 s, δ/Δ = 4/14 ms, resolution of 100 μm × 100 μm × 500 μm; two averages were acquired. Signal to noise ratio (SNR) at b = 0 in GM regions exceeded 65.

Human data were acquired in normal volunteers on a Siemens Trio 3 T scanner using a 32-channel head coil and a double spin echo DW echo planar imaging (EPI) pulse sequence. Image resolution was 3 mm isotropic with an average SNR of 66 in cerebrospinal fluid at b = 0. A total of 19 slices were acquired, with T_R = 4300 ms, T_E = 103 ms. Data consisted of one unweighted image and 15 shells with the same 33 directions as in the ex vivo case and b-values distributed evenly between 0.2 and 8.0 ms/μm². The data were acquired with permission from the local ethics committee, and informed consent was obtained prior to the acquisition.

Prior to analysis, all data were inspected visually to ensure absence of artifacts.

3.2. Simulations

In the simulations, we generate synthetic diffusion signals using the previously described tissue-specific modeling. Segmentation of the two tissue types was performed by thresholding the FA map (FA_threshold = 0.3).

Based on this segmentation, simulation input parameters were obtained from tissue specific model fits to the ex vivo rat brain data. Voxel-wise fitting was performed in MATLAB using a non-linear least squares Levenberg–Marquardt algorithm to yield 14 model parameters for WM and 18 for GM. The distributions of selected fitting parameters are provided in the supplementary material. From each set of parameters, MKT was calculated analytically using Equation 5 for GM or Equation 8 for WM. These values serve as ground truth values. Subsequently, the model parameters were used to generate synthetic signals using the same 33 direction sampling scheme and 14 evenly distributed b-value shells as employed for data acquisition, but varying the maximum b-value (b_max). Gaussian noise was added to yield an SNR of 100. Finally, kurtosis parameters were estimated by fitting the synthetic data to the DKI signal expression (Equation 1), using the algorithms described earlier (LLSQ, DWF+). We report the quality of the parameter estimates using error histograms as well as the mean and standard deviation of the (relative) error (STDE):

$$\begin{array}{c}\text{Net Relative error}=\langle \frac{|\text{Estimated parameter}-\text{Ground truth}|}{\text{Ground truth}}\rangle \\ \text{Stamdard deviation of error}=\text{std}(\text{Relative error})\end{array}$$

where 〈·〉 refers to average over voxels and |·| denotes absolute value.

3.3. Study workflow

The first step in the simulation workflow was to verify the functionality of our algorithm implementation by testing its ability to reproduce the ground truth kurtosis fit parameters from each tissue type (GM and WM). The input diffusion and kurtosis tensor parameters were obtained from a good quality (χ² < 0.1) DKI fit (LLSQ) to a subset of the fixed rat brain data (b ∈ [0; 3] ms/μm²). A pixel in each tissue type was used as basis for the simulation runs, which were repeated over 1000 noise realizations (SNR = 100). The WM and GM voxels were chosen based on the rat brain atlas.⁴⁹

Next, a series of three simulations was performed. These simulations were all based on fitted parameters of the tissue specific model. These parameters were used both to calculate MKT analytically and to generate an input signal for kurtosis tensor estimation. This simulation workflow is illustrated in Figure 1A.

Figure 1.

A, schematic diagrams of simulation work flow. The blue squares depict datasets; orange squares symbolize the processing stages. The asterisk denotes multiple fits with variable set of b-values. B, schematic diagrams of the analysis of highly sampled in vivo and ex vivo datasets. The asterisk denotes multiple fits with variable set of b-values

The first such simulation explored accuracy and precision over 1000 noise realizations in a single pixel of each tissue type.

As the next step, the findings were backed up by a second set of simulations, where fits from 100 random voxels in both GM and WM were used to generate DKI data sets, which were fitted with Equation 1; after this, net relative error and STDE were computed again using the theoretical MKT values as ground truth values.

To conclude this part of the study, in the third simulation the whole slice of the fixed tissue was fit with LLSQ and DWF+ for a single noise realization per voxel. Here, 14 b-values were evenly distributed in the range b = 0 … b_max, with b_max ∈ {1, 2, 2.5, 3} ms/μm². The MKT estimates were compared with the ground truth.

Finally, an analysis was performed to investigate the variation in estimated MKT as a function of maximum b-value. This was performed in both ex vivo and in vivo data by fitting subsets of these densely sampled datasets using LLSQ and DWF+ (Figure 1B). The data subsets were constructed to be equally sized but have increasing b_max values and b-values distributed evenly in the range b = 0 … b_max.

4. Results

Figure 2 illustrates our ability to reproduce the ground truth kurtosis parameters by an LLSQ fit to the DKI signal produced by Equation 1. The figure shows histograms of common DKI parameter estimates from LLSQ fits to 1000 simulation runs each with different noise realizations (SNR = 100) for WM (upper row) and GM (bottom row). Estimates are seen to be narrowly distributed around the ground truth values (red line). Similar performance was seen in 30 voxels for each tissue type (data not shown). The other fitting techniques described in this work were checked as well, and the resulting distributions of the relevant fit parameters were qualitatively the same. A more extensive comparison of the DKI fitting protocols is addressed in detail elsewhere.⁴⁰ To ensure the results are independent of the chosen noise model, tests were performed with MLE on synthetic data with Rician noise added. MLE is unbiased for such data. The MKT estimates obtained with MLE displayed behavior in agreement with the other results presented in this work (data not shown).

Figure 2.

The robustness of the kurtosis (MKT) fitting technique (LLSQ) is verified through estimation of mean diffusivity, MKT, and fractional anisotropy with 1000 noise realizations. The histograms show the estimated values of the parameters for different noise realizations. The upper row shows values from a representative WM voxel; the lower row shows values from one GM voxel. The red line marks the true value

Next, the single voxel behavior was explored using graphs (Figures 3 and 4) based on multiple noise realizations in one WM and one GM voxel.

Figure 3.

The relative error (measuring accuracy, top) and STDE (measuring precision, bottom) in evaluation of MKT for GM and WM. The graphs show results from a single representative voxel and 1000 noise realizations

Figure 4.

The histograms of MKT evaluation in GM (red) and WM (blue): The columns correspond to different fitting techniques (LLSQ, DWF+) and rows correspond to different maximum b-values (0.7; 1.0; 1.5; 2.0; 3.0; 4.0) ms/μm². The red line marks the true MKT value of GM and the blue line marks the true MKT value of WM, both analytically calculated from ground truth parameters

Figure 3 demonstrates the dependence of the net relative error (top) and STDE (bottom) on maximum b-value for GM and WM, respectively.

In GM (red), the minimum relative error for LLSQ occurs close to b_max = 1 ms/μm². This is surprising given the typically employed b_max = 2.5 ms/μm², where the relative error approaches 0.4 for LLSQ. The error of DWF+ is seen to attain a minimum at b_max = 2.5 ms/μm², while showing extremely high relative error for b_max < 1.5 ms/μm². At the same time, comparing DWF+ error (Figure 3, top) and STDE (Figure 3, bottom) reveals that, while DWF+ provides mostly reasonable accuracy at 1.5 ms/μm² < b_max < 3.5 ms/μm², its variability greatly exceeds that of LLSQ over the full b_max range explored. To explore these results further, the same analysis was also performed using an alternative DWF+ type fitting protocol (courtesy of Dmitry Novikov),⁵⁰ which utilizes the full 3D symmetric rank 6 tensor cumulant for the b³ term. This algorithm shows similar behavior to DWF+ (data not shown). The representability of the GM voxel was ensured by visual comparison of its raw signal curve with 30 other voxels of similar tissue. Similar dependence of relative error and STDE on b_max was observed in all GM voxels (data not shown).

The dependence on maximum b-value for a single voxel in WM is also shown in Figure 3 (blue). For both algorithms the minimum error is smaller (∼0.05) than in GM and is attained at higher b_max (Figure 3, top). For LLSQ the minimum relative error occurs at b_max ∼ 2.5 – 3.5 ms/μm², close to the b_max used in standard DKI protocols (2.0 – 2.5 ms/μm²). In the range of typically employed b_max -values, DWF+ has high relative error (>0.2 in the range b_max = 1.8 – 2.5 ms/μm²), from where it further decreases until it attains its minimum relative error at b_max ≈ 4 ms/μm². Turning to the STDE (Figure 3, bottom) we see that STDE for DWF+ is higher (∼0.1) than that of LLSQ (<0.05) at the commonly used b_max = 2.5 ms/μm².

Substantial variation was seen between the WM pixels investigated (data not shown). This was not only observed comparing the raw signal curves but is also evident in the varying error behavior: while WM voxels attain minimum relative error in the interval b_max ∈ [2; 5] ms/μm², the steepness of the relative error curve and the optimal b_max vary a lot between the WM voxels.

Figure 4 shows histograms of MKT estimation in single voxels of WM (blue) and GM (red). The figure shows results from LLSQ in the left-hand column and results from DWF+ in the right-hand column. Within each column, the rows show behavior for six different b_max values in the range that contains the minimum relative error for both LLSQ and DWF+ in WM and GM according to Figure 3.

For GM (red), most LLSQ MKT estimates are lower than the ground truth value (red line), and the most accurate estimate is obtained at the lowest b_max = 0.7ms/μm². For both protocols, the estimate distributions are wide for low b_max and narrow as b_max increases, more so for LLSQ. DWF+ overestimates MKT for small values of b_max, and underestimates it for larger values. The estimated values of MKT become increasingly underestimated for both LLSQ and DWF+ as b_max increases. The best estimates in terms of accuracy of the median and small distribution width are obtained for b_max = 2.0 ms/μm² and b_max = 3.0 ms/μm². The point where the median of the distribution is the closest to the real value is thus seen to be protocol dependent.

For WM (blue), the width of the distribution also varies between the different values of b_max and the estimate median decreases with increasing b_max. MKT in WM is mostly overestimated for both fitting algorithms for b_max ≤ 2 ms/μm², and the point where the median of the distribution coincides best with the ground truth (blue line) occurs between b_max =2ms/μm² and b_max =3ms/μm² for LLSQ and at about b_max =4ms/μm² for DWF+. Estimate distributions are generally wider for WM than for GM and for DWF+ than for LLSQ at a given b_max value.

In Figure 5 we summarize the trade-off between the accuracy and precision for a group of 100 WM and 100 GM voxels. Here each point corresponds to a specific b_max (color encoded; see colorbar), protocol and tissue (marker encoded). Accuracy and precision are given by x and y coordinates respectively: hence, the closer the points are to the origin, the better the performance of the algorithm in terms of having both a low mean error and a low STDE. For convenience, a zoomed-in view of the region near the origin is shown on the right.

Figure 5.

The precision (STDE) and accuracy (net relative error) of MKT evaluation for different fitting techniques as a function of b_max for GM and WM. A zoomed part of the graph is presented on the right. When the curve approaches the origin, it indicates that the performance of the particular set of b-values improves in terms of having low mean error and a low STDE. Maximum b-value is color coded in ms/μm². Dashed black curves are connecting points equidistant from the origin

Focusing first on GM, we note that the LLSQ points closest to the axis origin correspond to b_max in the range of 0.8 – 1.1 ms/μm², again significantly lower than conventional practice. For DWF+ (GM), the best performance is obtained with maximum b-values higher than 3 ms/μm². Algorithms have a comparable performance in terms of mean error and STDE for the best choice of b_max.

For WM, DWF+ performs significantly worse than LLSQ in terms of both the minimal relative error and minimal STDE. The points closest to the origin correspond to b_max = 2.5 – 2.7ms/μm² for the better performing algorithm (LLSQ). All this is in agreement with the results presented in Figure 3. Compared with Figure 3 the graph also shows that overall minimum net relative error of multiple voxels is considerably higher than minimum net relative error for the single voxel and multiple noise realizations.

Scatterplots showing the relationship between the evaluated and the ground truth MKT for multiple voxels in the whole slice of rat brain are provided as supplementary material. These plots support the results in Figures 2-5.

Figure 6 illustrates the b_max -dependence for MKT estimates in ex vivo rat brain (A,B) and in vivo human brain (C,D). The figure demonstrates the average relative change in estimated MKT in GM (A,C) and WM (B,D) relative to the map obtained with b_max = 2.6 ms/μm² using the LLSQ protocol.

Figure 6.

Rat brain ex vivo (A,B) and human brain in vivo (C,D) variability of the estimated values of MKT in the GM (A,C) and WM (B,D) tissue. The change in MKT is depicted relative to the LLSQ value of b_max = 2.6 ms/μm². The values of b_max are slightly shifted for reasons of clarity

In fixed tissue, LLSQ displays lower variance (visualized as errorbars) than DWF+ for the same b_max. This behavior is consistent across both GM (Figure 6A) and WM (Figure 6B). For b_max = 1.6 ms/μm² the DWF+ MKT estimate is more than 50% higher on average in both GM and WM compared with b_max = 2.6 ms/μm².

In human brain, the effect of b_max variation is very substantial as well (Figure 6C,D). In GM (C), the variation of MKT values exceeds 40% for both of the protocols in the range b_max = 1.2 ms/μm² to b_max = 2.5 ms/μm². Also, while the relative change in WM (D) is less significant than in GM, the variance (errorbars) is comparable.

Generally, the two fitting protocols display qualitatively the same dependence of the MKT estimate on the maximum b-value both ex vivo and in vivo: the MKT estimate decreases as a function of b_max. At the same time, a higher degree of variability is seen for DWF+ than for LLSQ in all four panels. Other fitting algorithms (NLSQ, DWF, MLE) were inspected, and they were found to demonstrate behavior similar to that of LLSQ (data not shown).

Figure 7 shows maps of MKT across fitting algorithms and with varying b_max for in vivo (left-hand column) and ex vivo brain (right-hand column). In vivo, the MKT is seen to depend significantly on the fitting protocol (LLSQ on the left and DWF+ on the right in the left-hand column). Moreover, for both algorithms, the contrast changes substantially with changing b_max. The LLSQ protocol appears almost uninfluenced by the variation for b_max = 2 ms/μm² and above. Contrast in small structures also changes substantially during the variation of b_max for both protocols in the ex vivo tissue (right-hand column).

Figure 7.

MKT variability in in vivo human brain (left) and ex vivo rat brain (right) as a result of varying of bmax and fitting procedures (LLSQ, DWF+). Rows are for b_max = (1.2;2;2.5;3) ms/μm²

5. Discussion and Conclusions

In this work we have demonstrated a strong dependence of MKT on the fitting algorithm and maximum b-value, b_max. The results showed that the choice of fitting scheme and maximum b-value influence both the accuracy and the precision of the MKT evaluation in a tissue dependent way. This was observed for both ex vivo and in vivo data and was corroborated by simulations.

We showed that, while the precision increases monotonically when b_max is increased, the accuracy of estimation does not improve with the increase in b-values. With the increase in b_max the estimated MKT decreases on average, sometimes causing a transition from over- to underestimation (Figures 4, 6). Therefore, optimizing the sampling scheme to achieve the best accuracy of MKT estimation is a challenging task, which requires attention at the planning stage of DKI studies.

The comparison of performance of LLSQ and DWF+ in our results shows (Figures 3, 6, 7) that increasing the number of terms in the cumulant expansion does not necessarily improve the accuracy of MKT estimation. Instead, it alters the range of b-values that provide the optimal estimation accuracy. However, as expected, increasing the number of terms does improve the signal approximation accuracy measured as a quality of fit per voxel (χ²) for both WM and GM (data not shown) for the whole range of b_max ∈ [0,7]ms/μm².

For GM, the b_max -value that permits the best estimation of MKT with fitting procedures utilizing the cumulant expansion up to the fourth term (like LLSQ) is significantly lower than what is used in standard DWI protocols (Figure 3, upper panel). Based on our analysis of the behavior of MKT estimates as a function of b_max, we concluded that when estimated with b_max ∼ 2ms/μm² GM MKT may be underestimated by 20–40%, both ex vivo and in vivo (Figures 3, 4, 6). The same results indicate that if GM MKT is estimated with fitting protocols utilizing the sixth term of the cumulant expansion the best accuracy is achieved with 2 ms/μm² < b_max < 3 ms/μm². Using b_max < 2 ms/μm² will result in overestimation and b_max >3ms/μm₂ in underestimation. However, the detailed behavior depends on tissue composition.

In WM, the commonly used b_max ≃ 2.5 – 3 ms/μm² indeed roughly corresponds to the b-values where the highest accuracy is achieved in LLSQ and other protocols that utilize four terms of the cumulant expansion (Figures 3-5). Even so, an average of at least 15% error remains present in WM MKT estimation (Figure 5). Moreover, we find that the minimum is quite narrow (Figure 3) even in the case of WM. From the graph in Figure 5 one might expect the relative error to go higher than 20% on average for multiple WM voxels when b_max > 3 ms/μm² or b_max < 1.3 ms/μm². For DWF+ and similar protocols b_max ≈ 4ms/μm² allows the best accuracy of MKT estimation.

The observed difference in behavior between WM and GM (Figures 3,4) is troubling, especially because our analysis shows that this problem is also strongly dependent on b_max. Consequently, suboptimal choice of b-value may cause estimates to deviate vastly from true values (Figures 4, 6) and in extreme cases may even cause MKT tissue contrast to substantially deteriorate (Figure 7). Furthermore, since WM and GM are typically evaluated based on the same DKI acquisition, disease related alterations may be difficult to detect in both tissues at once. However, the problem persists even when focusing on one tissue type, as our study generally reveals the difficulty of determining the true values of MKT and likely also other kurtosis generated metrics. The complexity of this estimation problem even in a single tissue type was clearly demonstrated when the quality of one voxel estimation was compared to the average accuracy of multiple voxels (Figure 5 versus Figures 3, 4). There, for a single voxel of WM the minimum relative error reached 5% (Figure 3), but when multiple voxels were combined in Figure 5 the minimum relative error did not go below 15% for the best performing algorithm. It is reasonable to suppose that these phenomena stem from the microstructural variability of WM. Thus, microstructural features (e.g. degree of anisotropy, presence of crossing fibers) influence accuracy and precision for each b_max. Accordingly, a given sampling scheme will only be optimal for a minority of the voxels in a heterogeneous slice of neural tissue, leading to a larger combined error overall.

Our results were obtained using a fairly general neural tissue model. Therefore, we expect that, if other and potentially more accurate microstructural models for brain tissue are used, the resulting behavior will be qualitatively similar to our observations, given that the model employed here already closely approximates the observed signals. However, the rat and human data employed in this study (also used in Reference 51) are available⁵² for readers who wish to reproduce our results or test their own microstructural model and analysis scheme. Based on our analysis, we expect such efforts to produce an outcome in accordance with our results (see supplementary material) in that poor correlation between the estimated MKT and the ‘ground truth’ will be found for both GM and WM. However, we have previously observed⁵³ that the correlation between neurite density³⁴ and MKT is high for higher b_max values. Thus, it is conceivable that estimated MKT under some circumstances might approximately reflect one of the model parameters. Hence, in the general case, it is recommended to estimate the kurtosis error for the particular model being assumed, and investigate how this error propagates into the microstructural model parameters. Owing to DKI being increasingly used for evaluation of tissue types other than neural matter,^17,54 the development of appropriate diffusion models for these tissue types can prove to be a crucial step for acquisition optimization and correct parameter interpretation.

Broadly speaking, we find that the optimal experimental design and fitting algorithm varies on a voxel by voxel level. Valid assumptions of tissue properties prior to optimization of DKI sampling scheme and selection of fitting algorithm are thus necessary. In practice, we suggest using anatomical atlases and measures capable of differentiating fiber organization (e.g. kurtosis fractional anisotropy⁵¹) along with publicly available diffusion weighted datasets for preliminary studies that aim to tailor the best experimental paradigm for specific tissue type or structure. Such optimization efforts may also incorporate CRLB optimization. In this way, once b_max optimization has ensured the best cumulant expansion approximation of the microstructural model used, CRLB-based optimization may be carried out as an additional or iterative step that provides a corresponding sampling scheme. This could be done for both ex vivo and in vivo data. Interestingly, we find that estimation behavior is largely similar in fixed tissue and in vivo (Figure 6). This is in agreement with a previous optimization, where similar optimal b-values for fast kurtosis imaging was found in fixed tissue and in vivo brain based on comparison to experimentally determined ground truth values.⁵⁵

While the present study emphasizes the need for careful acquisition optimization in future studies, data that have already been acquired may still serve as an important source of qualitative information. However, the precision and accuracy of the estimation should be investigated and taken into account if one is interested in using the quantitative results of previous studies, e.g. for the sake of building tissue models or for calculation of binary hypothesis power, e.g. for determining a sample size. For tissue types similar to those analyzed in this study, an effort can be made to correct previously acquired kurtosis metrics using the biases and precisions found here. In particular, Figures 5 and 6 show that if the fitting protocols that utilize the fourth term of the cumulant expansion were used in a study with b_max ∼ 2 – 2.5 ms/μm² GM MKT values should be reduced by about 20–40% ex vivo and 40–60% in vivo, while the estimation of WM is approximately unbiased. If fitting protocols that utilize the sixth term of the cumulant expansion were used with b_max∼2 - 2.5 ms/μm², the estimated WM MKT values should be increased 20–40%, while GM MKT do not need correction.

As an alternative and pragmatic clinical approach, it is also possible to view the fitting parameters reported here as biomarkers without considering the relation to ‘true’ kurtosis. Consequently, the objective will be to improve the estimation precision, which can be achieved by increasing b_max, while optimizing sensitivity to a given pathology. In this case, we suggest considering a different terminology for the evaluated biomarkers. A practical limitation of this strategy is an increased echo time and consequent decrease in SNR.

In diagnostics, the usage of non-optimal choice of b-values can cause reduced sensitivity to the subtle pathological effects. Depending on the type of pathology in question, b-values optimal for a specific tissue type (e.g. WM) might be more appropriate than b-values chosen as a compromise to optimize estimation in whole brain. Consequently, the development of clinical recommendations that stipulate an adequate choice of b-values for specific suspected pathologies and tissue types should be prioritized.

Since the DKI field is still relatively new, the extent of meta-analysis is very limited. Even so, there are still a considerable number of clinical studies that report e.g. substantially different values in tissue affected by prostate cancer,^56,57 distinct affected areas in Alzheimer disease,^58,59 or a surprisingly high level of discrepancy in reported values in both thalamus and the internal capsule of normal controls employed in two DKI studies of traumatic brain injury.^16,60 In all of the mentioned studies, different sampling schemes with different b_max -values were adopted, thus the effects discussed in this paper may contribute to the disparity in addition to biological variability. In conclusion, our results demonstrate the difficulty of MKT estimation which, was found to depend strongly on both experimental b-value and tissue type. This reveals a potential vulnerability of meta-studies that compare results obtained using different b-values and fit with different protocols. These results emphasize the need for careful study design and call for studies that could lead to a community consensus on acquisition and analysis details.

Abbreviations

DWF

direction-wise fit

DWF+

direction-wise fit with higher terms

GM

gray matter

LLSQ

weighted linear least squares

MKT

mean of the kurtosis tensor

MLE

maximum likelihood estimation

SNR

signal to noise ratio

STDE

standard deviation of error

WM

white matter