Decoding pulsatile patterns of cerebrospinal fluid dynamics through enhancing interpretability in machine learning

Abstract

Analyses of complex behaviors of Cerebrospinal Fluid (CSF) have become increasingly important in diseases diagnosis. The changes of the phase-contrast magnetic resonance imaging (PC-MRI) signal formed by the velocity of flowing CSF are represented as a set of velocity-encoded images or maps, which can be thought of as signal data in the context of medical imaging, enabling the evaluation of pulsatile patterns throughout a cardiac cycle. However, automatic segmentation of the CSF region in a PC-MRI image is challenging, and implementing an explained ML method using pulsatile data as a feature remains unexplored. This paper presents lightweight machine learning (ML) algorithms to perform CSF lumen segmentation in spinal, utilizing sets of velocity-encoded images or maps as a feature. The Dataset contains 57 PC-MRI slabs by 3T MRI scanner from control and idiopathic scoliosis participants are involved to collect data. The ML models are trained with 2176 time series images. Different cardiac periods image (frame) numbers of PC-MRIs are interpolated in the preprocessing step to align to features of equal size. The fivefold cross-validation procedure is used to estimate the success of the ML models. Additionally, the study focusses on enhancing the interpretability of the highest-accuracy eXtreme gradient boosting (XGB) model by applying the shapley additive explanations (SHAP) technique. The XGB algorithm presented its highest accuracy, with an average fivefold accuracy of 0.99% precision, 0.95% recall, and 0.97% F1 score. We evaluated the significance of each pulsatile feature's contribution to predictions, offering a more profound understanding of the model's behavior in distinguishing CSF lumen pixels with SHAP. Introducing a novel approach in the field, develop ML models offer comprehension into feature extraction and selection from PC-MRI pulsatile data. Moreover, the explained ML model offers novel and valuable insights to domain experts, contributing to an enhanced scholarly understanding of CSF dynamics.

Introduction

Variations in CSF flow dynamics are linked to a broad spectrum of neurological disorders, including Normal pressure hydrocephalus, Chiari malformation, Syringomyelia, Spinal cord tumors, among others¹. Moreover, the variability in CSF dynamics observed in scoliosis cases highlights the profound importance of quantifying^2,3. Quantitative assessment of CSF parameters is pivotal for diagnosing, prognosing, and monitoring CSF-related neurological disorders, pointing up the necessity of enhancing understanding in this domain for effective treatment approaches.

Phase-contrast Magnetic Resonance Imaging (PC-MRI) is a non-invasive imaging technique used to evaluate Cerebrospinal Fluid (CSF) circulation abnormalities within the intracranial and spinal cavities, aiding in the early detection of such conditions and guide treatment decisions⁴. It was initially employed to investigate the direction and velocity of blood flow. Subsequently, Feinberg et al.⁵ utilized PC-MRI sequence to quantitatively measure CSF flow along the aqueduct of sylvius in humans. PC-MRI operates by measuring phase shifts among protons within fluids using a bipolar gradient^4,6. This specialized gradient technique offers both qualitative and quantitative assessments of fluid flow, providing spatial and phase information about proton movement in fluids. By visualizing and measuring CSF motion during a single cardiac cycle, PC-MRI provides valuable insights into flow parameters, including directionality. Numerous studies have highlighted PC-MRI as the preferred non-invasive approach for visualizing the pulsatile movement of CSF and studying both its physiology and pathophysiology^7–11. In contrast to other MRI scans, PC-MRI examinations are less frequently performed in routine medical practice. As a result, obtaining and interpreting PC-MRI scans demands advanced expertise and time; otherwise, there is a notable occurrence of inaccurate outcomes¹². Additionally, the accuracy of manual ROI mapping is heavily influenced by the expertise of the specialist, underscoring the importance of determining the correct ROI by the specialist^13,14. Towards this objective, some semi- or fully automatic computationally lumen segmentation research in PC-MRI images has been proposed. These studies, typically categorized into blood and CSF lumen segmentation tasks, leverage both spatial and pulsatile information for segmentation purposes. While studies focused on blood and cardiac lumen segmentation primarily utilize pulsatile blood data^15–17, those concern to CSF lumen segmentation incorporate pulsatile CSF data alongside spatial information^18–22.

Building upon the need for precise CSF lumen segmentation in PC-MRI data, the Pulsatility-Based Segmentation (PUBS) method, introduced by Alperin et al.¹⁸, utilizes correlations of pulsatile CSF quantities across the cardiac cycle. In PUBS, regions of interest (ROIs) are derived from pixels exhibiting high correlation with a manually determined CSF lumen point by a specialist, surpassing a specified threshold value. Subsequent studies^19,20 also necessitate manual determination of the start point. However, these studies, reliant on manual intervention, are labor-intensive and susceptible to errors. Deep Learning (DL)-based fully automatic studies on PC-MRI data for CFS Lumen segmentation have been performed to overcome these challenges^21,22. While Tsou et al.²¹ did not incorporate pulsatile information, Keles et al.²² utilized pulsatile information in their research. They benefited from both spatial and pulsatile information in the training of the 3D model in their study²². Furthermore, they performed a comparative analysis of both 2D and 3D models in their study, confirming the increase in model accuracy resulting from the integration of pulsatile information.

Contrary to the utilization of spatial image data for model training in the aforementioned DL approaches, our study tries to develop a lightweight and more efficient ML model achieved through training with only pulsatile data. In contrast to¹⁷, this study involved exclusively utilizing phase images containing pulsatile data for training ML models. This approach promises to offer not just an understanding of the pulsatile dynamics of CSF flow, but also to yield ML models that are more easily interpretable, less susceptible to overfitting, characterized by transparent decision-making processes, and cost-effective. Given the traditional ML models' capability to effectively manage smaller datasets, Logistic Regression (LR), Random Forest (RF) and eXtreme Gradient boosting (XGB) algorithms were chosen for implementation in this study, which operates with a relatively limited dataset.

LR model for $y_i$ is linear function of $x_i$, where dataset $\left(x_i,,,y_i\right)iϵN$, $N$ is the sample size. It is particularly well-suited for scenarios where the outcome variable is categorical, and it models the probability of an observation belonging to a particular class. LR estimates the parameters of a logistic function to predict the likelihood of a binary outcome. It is a linear model that can be extended to manage more complex relationships through feature engineering and regularization techniques. RF is an ensemble learning technique that combines the power of multiple Decision Trees (DTs) to improve predictive accuracy and reduce overfitting. A DT is a flowchart of decisions and related outcomes based on certain criteria that give the appearance of a tree. It is the most common algorithm used by a weak learner in ensemble learning. There are two ensemble learning methods: bagging (or Bootstrap Aggregation) and boosting. Based on the Bagging method, the RF algorithm divides the learning process by random feature subsets and then builds a collection of DTs²³. Another ensemble technique, boosting, uses weak learners to have a weighted vote for the final classification, and predictions are generated sequentially rather than independently. The way that DTs weights are calculated and sorted concerning observation difficulty in adaptive (AdaBoost)²⁴ and gradient boosting (GB)²⁵ algorithms is different. The basic concept behind GB is to build a new model in the direction of the gradient of the residual errors to minimize the loss function obtained at each iteration. XGB leverages the strengths of DTs, using a GB framework to optimize predictive accuracy. XGB²⁶ excels in handling structured data and is highly customizable, allowing fine-tuning of hyperparameters to achieve optimal results. XGB have gained widespread popularity and have been applied extensively for its exceptional performance in supervised learning tasks^27–30.

ML algorithms’ interpretability including both traditional models like LR and DTs, as well as DL models has emerged as a critical concern in data-driven applications. Digging deeper into these predictions offers a richer understanding of both the problem at hand and the dataset and allows identification of biases and factors that cause potential model shortcomings. European regulatory frameworks³¹ underline the need for the model to grasp the reasons for automatic predictions as well as extract insights. In this context, recent studies in medical domain focus on the interpretability of their developed ML models. Shapley Additive Explanations (SHAP)³² provides a unified interpretable approach by measuring the contribution of each feature to the output of various ML models including DL models. In line with our research, Xiaozhou et al.³³ implement classical ML models and utilize SHAP to offer a comprehensive interpretation of each feature. However, they have only included CSF α-synuclein level as a feature for the models training. Cranial consortium³⁴ implemented ML model to leverage clinical dataset for CSF rhinorrhea prediction and risk factor analysis and identified the pivotal predictive features with SHAP values. This study represents a pioneering effort in literature as it utilizes CSF flow velocity and acceleration features in a cardiac cycle quantified via PC-MRI for training the classical ML models; LR, RF, and XGB. Through the utilization of the SHAP method, our research attempts to provide novel insights into pulsatile patterns of CSF flow to the prediction of CSF lumen in the spinal region.

The segmentation of the subarachnoid space (SAS) based on pulsatile features was conducted using LR, RF, and XGB in four main sub-steps as depicted in Fig. 1. In the illustrated workflow, the initial step involves standardizing the number of PC-MRI frames using an interpolation technique. Subsequently, the labeled pixels are balanced to improve accuracy and features are extracted from the phase images. Finally, the LR, RF, and XGB algorithms are implemented to classify pixels for segmentation.

Figure 1.

Overview of the pulsatility-based subarachnoid space (SAS) segmentation process.

The subsequent sections of this document are organized as follows: Part 2 focuses on the data utilized and outlines the procedures involved in implementing an ML model to segment the CSF lumen in PC-MRI data. A data-driven approach was taken based on pulsatile data and the ML models were proposed adapted to the frame series. Part 3 contains the results of the predictions produced by these ML models. This chapter includes model accuracy metrics and highlights the SHAP values of the most accurate XGB model. Finally, the results are discussed in Part 4 and conclusions are drawn in the last chapter, Part 5.

Materials and methods

In the first sub-step PC-MRI frames of each slab were stretched to the interpolation length using a spline interpolation technique. The subsequent step involves the manipulation of the class member counts by up-sampling and down-sampling to address the issue of highly imbalanced labeled pixels. The third sub-step involves the extraction of features and their corresponding labels from phase image pixel values of each slab and their related mask images. Finally, in the fourth sub-step, pixels were classified using the LR, RF, and XGB algorithms for segmentation. Implementation of method in this study is coded under the PYTHON 3.12 programming language and 64-bit Windows environment. DICOM (Digital Imaging and Communications in Medicine) format of PC-MRI data was processed using the pyDicom 2 library³⁵.

Dataset

Informed consent was obtained from all participants before conducting PC-MRI scans, and the Ankara Yildirim Beyazit University ethics committee approved all procedures conducted in these studies. In the context of PC-MRI, the term “slab” refers to a three-dimensional volume of PC-MRI scan. A slab consists of a series of frames and each frame captures the spatial and temporal information of the flow of fluids (such as CSF) in the slab. The 1088 phase images are derived from 57 slabs obtained from 39 subjects undergoing PC-MRI. 18 participants were imaged with multiple slabs. In this dataset, the acquisition parameters of PC-MRI scans and demographic information of the participants have been previously detailed in the study conducted by²².

Feature quantities are extracted from phase images. Figure 2 illustrates the sequential presentation of the phase frames of a given slab with the ROI intended for segmentation drawn on frame images to facilitate visualization.

Figure 2.

The phase images of a patient’s slab. The numbers in each image title shows frame sequence number. ROI delineated on the images using dotted line.

The images in the dataset were structured with a resolution of 256 × 256 pixels using nearest neighbor linear interpolation. The expert utilized the LabelMe (http://labelme.csail.mit.edu) tool for the annotation of the ROI. After the labeling process, the ground-truth label set was defined as a binary vector; $Y=\left(y_1,,,y_2,,,\cdots ,,,y_n\right),y_i\in \left\{\text{0,1}\right\},1\le i\le n$, where $n$ is the number of pixels.

During each PC-MRI acquisition, a varying number of rephase and phase images are generated depending on the subject’s heart rate⁶. The establishment of uniform input dimensions within the context of ML methodologies, it is essential to have an equal number of frames. Since determining frame numbers in retrospective scan examinations is not feasible, some studies have opted for prospective acquisition to ensure an equal number of frames from participants. Within our retrospective study dataset, the temporal resolution of each slab varies between 9 and 30 (as illustrated in Fig. 2, the temporal resolution is 13, for instance). On average, each slab in the dataset contains 20 frames. To equalize the number of rephase and phase frames, a spline interpolation method is applied, expanding the frames to the interpolation length (in this study, 32 frames). The interpolation is accomplished using NumPy's package (version 1.26.4) ‘interp’ method, performs one-dimensional linear spline interpolation. The formula for linear interpolation between two points $(x_0,y_0)$ and $(x_1,y_1)$ can be expressed as:

$$interp\left(x,,,x_{\mathit{points}},,,y_{\mathit{points}}\right)=y_0+(x-x_0)\frac{y_1-y_0}{x_1-x_0}$$

where

• ${\text{x}}_0$ is the point at which interpolation is desired.

• ${\text{x}}_{\text{points}}$ and ${\text{y}}_{\text{points}}$ are the arrays of ${\text{x}}_0$ and ${\text{y}}_0$ values, respectively, representing the data points.

• ${\text{x}}_0$ and ${\text{x}}_1$ are the nearest data points to $\text{x}$ such that ${\text{x}}_0\le \text{x}\le {\text{x}}_1$.

• ${\text{y}}_0$ and ${\text{y}}_1$ are the corresponding $\text{y}$ values of ${\text{x}}_0$ and ${\text{x}}_1$, respectively.

Figure 3 illustrates an interpolated example of frames in a slab belongs to 17-years-old female from the dataset. The graph in the figure represents the flow values corresponding to each pixel in the CSF lumen, as labeled by an expert. While the graph on the left of Fig. 3 depicts the acquired cardiac-gated PC-MRI frames, the graph on the right demonstrates its interpolated counterpart. This visual representation highlights the stability and coherence between the original and interpolated frames, emphasizing the effectiveness of the interpolation process in maintaining the integrity of the data.

Figure 3.

Interpolation: The flow values corresponding to each pixel in the CSF lumen (annotated by experts) in each frame. While the left part shows actual frames obtained during acquisition, the right part shows interpolated frames. Actual 9 frames/slab (left) are interpolated to 32 frames/slab (right).

The dataset exhibits a class label imbalance ranging from 1 instance of CSF lumen pixel to 1000 instance of others, up to a maximum of 4 instances to 1000 instances. This situation presents the challenge of imbalanced data. To address this and create a balanced training dataset for use with ML algorithms, we have employed the Synthetic Minority Over-sampling Technique (SMOTE)³⁶ for upsampling/downsampling. Each slab’s variable number of phase frames was stretched into the same length using a spline interpolation technique, as explained in the interpolation section, to produce an equal pulsatile dimension. The algorithm is implemented in a Python environment with the imblearn library.

Feature extraction

PC-MRI data can be seen as a time-varying signal that encodes information about the pulsatile nature of CSF flow. Researchers and clinicians often analyze and interpret PC-MRI data as signal data to gain insights into the dynamics of CSF flow within the central nervous system. The feature extraction strategy involved extracting 64 features for each pixel of the phase images forming the pulsatile data. In addition to velocity values, the dataset's features include the extraction of velocity acceleration. The pixel values of PC-MRI images represent the velocity itself. The computation of flow acceleration between consecutive frames entailed the formula: $acceleration=\mathrm{\Delta }velocity/\mathrm{\Delta }time$, with the time interval ($\mathrm{\Delta }time$) standardized to 1. Notably, the initial and concluding cine frames were considered as sequential instances for this calculation. The associated label Y was added in the final stage of forming the train dataset. After adding target attributes to the training data and labeling the training dataset, a composed input dataset denoted as ${T=\left\{x_i\right)\left({,y}_i\right\}}_{i=1}^n$ was generated. In this representation, $n=64$ and $x_i\in \chi$ $y_i\in \text{Y}$.

Machine learning modelling

We segmented the CSF lumen by classifying each pixel as either belonging to the CSF lumen or not. For segmentation, we utilized LR, RF, and ensemble DTs models, incorporating CSF pulsatile data as features. The algorithms were trained on phase frames spanning the entire cardiac cycle. Unlike previous semi-automatic approaches, our research introduces fully automatic versions of LR, RF, and XGB algorithms, trained on pulsatile flow data. We conducted a comparative analysis of these methods within a fully automated framework, presenting a novel approach to leverage these algorithms for CSF lumen segmentation.

To enhance the ML models’ performance and their ability to generalize effectively, the strategy of “grid search cross validation” was implemented using the GridsearchCV function of scikit-learn library (version 1.4.2). For each model given in Table 1, a systematic exploration of hyperparameters and their various combinations was conducted, subsequently identifying the optimal hyperparameters that yielded the most favorable outcomes. For XGB and RF model; max_depth, controls the depth of trees, balancing model complexity and overfitting and n_estimators parameter sets the number of boosting rounds, influencing model complexity and training time. For XGB model; gamma determines the minimum loss reduction required to make further splits, min_child_weight sets the minimum sum of instance weights needed in a child node, regulating overfitting by controlling node sizes. Additionally, subsample adjusts the fraction of data used per boosting iteration for randomness to mitigate overfitting and reg_alpha and reg_lambda are regularization parameters, controlling the complexity of the model and preventing overfitting. colsample_bytree sets the fraction of features used per tree, adding randomness to prevent overfitting. Lastly, eta (learning rate) governs the step size during optimization, affecting convergence speed and model robustness. In the LR model, solver determines the optimization algorithm used during training, while max_iter sets the maximum number of iterations for the algorithm to converge. Finally, penalty dictates the regularization technique applied to prevent overfitting. The models were trained using the best hyperparameter combinations, listed in third column in Table 1, employing a fivefold cross-validation approach.

Table 1.

Ranges of hyperparameters for XGB, LR, and RF models fine-tuning.

ML model	Ranges of hyperparameters	Selected hyperparameters
XGB	'max_depth': [5, 10, 20, 50] 'gamma': [0.1] 'min_child_weight': [1, 3, 5] 'subsample': [0.5, 0.9] 'n_estimators': [5, 20, 50] 'reg_alpha': [3, 5, 10] 'reg_lambda': [3, 5, 10] 'colsample_bytree': [0.5, 0.9] 'eta': [0.01, 0.1]	'max_depth': 10 'gamma': 0.1 'min_child_weight': 1 'subsample': 0.5 'n_estimators': 5 'reg_alpha': 10 'reg_lambda': 3 'colsample_bytree': 0.5 'eta': 0.01,
LR	'solver': ['sag','saga', 'lbfgs', 'liblinear', 'newton-cg'], 'max_iter': [100,200,500], 'penalty': ['l1', 'l2', 'elasticnet']	'solver': 'saga' 'max_iter': 100 'penalty': 'l1'
RF	‘max_depth’: [10, 50, None] ‘n_estimators’: [5, 20, 50, 100]	'max_depth': 10 'n_estimators': 50

Shapley additive explanatory method

It follows the path of explaining complex models through model agnostic techniques that can be applied in various supervised learning models. These model agnostic methods work by changing the inputs of ML models and measuring the resulting changes in the prediction outputs. Categorically, these methods can be divided according to their focus: either to explain the inclusive behavior of the model or to explain individual predictions. The SHAP value for a specific feature of a particular instance represents the average change in the model's output when that feature is included, compared to when it is excluded, while considering all feature combinations. The general formula for calculating the SHAP value for a feature on a specific instance is as follows:

$${\varphi }_i\left(x\right)={\sum }_{S\subseteq N\{i\}}\frac{\left|S\right|!\left(\left|N\right|,-,\left|S\right|,-,1\right)!}{\left|N\right|!}\left[f,\left(x,\cup ,\{,i,\}\right),-,f,(x_S)\right]$$

Here

• ${\varphi }_i(x)$ is the S.HAP value for feature $i$ on instance $x$

• $N$ is the set of all features

• $S$ represents a subset of features excluding feature $i$

• $x_S$ denotes the instance $x$ with only the features in subset $S$ active

• $f\left(x,\cup ,\{,i,\}\right)$ is model’s prediction for instance $x$ when feature $i$ is included, and $f(x_S)$ is the prediction when feature $i$ is excluded

• The summation iterates over all subsets $S$ of features excluding $i$

The selection of XGB for explainability analysis was based on its highest performance across precision, recall, and F1 score measurements. Additionally, it demonstrated superior correlation performance in mean, median, and stroke volume velocity values. After obtaining SHAP values for explaining the outputs of the XGB model, we opt to focus on a subset comprising only ROI samples (pixels predicted as CSF Lumens) for prediction explanations.

Ethical approval

All procedures conducted in studies involving human participants adhered to the ethical standards established by the institutional and/or national research committee. All research activities were carried out in accordance with the principles outlined in accordance with the Declaration of Helsinki.

Informed consent

Consent was obtained from all individual participants who were part of the study.

Experimental results

After experts’ annotation of the CSF lumen in PC-MRI frames, the velocity and acceleration features from each phase frame were extracted. Following the training of ML models, performance evaluation and interpretability analysis were conducted through the computation of accuracy metrics and SHAP values for the models’ predictions.

The training data consisted of 1824 images after interpolation of frames determined through fivefold cross-validation conducted on the training dataset. Pulsatile features, including velocity and acceleration features, were used to segment CSF lumen and obtain a quantity report of CSF fluid flow. To enhance segmentation accuracy, the obtained image underwent erosion and dilation processes employing a 2 × 2 square filter. This was done to eliminate irrelevant areas that were erroneously predicted as ROI. In Fig. 4, a detailed examination of the segmentation result of a participant is displayed, utilizing XGB, LR, and RF ML algorithms. Panel (a) features an original phase MRI image frame, while panel (b) provides the ground truth mask, labeled by expert clinicians. The subsequent panels, (c), (e), and (g), present the sequentially predicted ROIs denoting the CSF lumen, generated by the ML models. To enhance the precision of the predictions, panels (d), (f), and (h) depict eroded and dilated versions of the predicted images, employing a 2 × 2 square filter tailored to each corresponding model. Notably, the application of morphological operations such as erosion and dilation contribute to refining the model predictions, resulting in improved delineation of CSF structures. Panels (k) and (l) further elucidate the success of our models by presenting waveforms illustrating the mean velocity of CSF flow through the labeled and predicted ROIs, respectively.

Figure 4.

presents an illustrative example depicting the segmentation of the CSF lumen in PC-MRI of a 20-year-old female. Panel (a) displays an original phase MRI image frame, while panel (b) displays the ground truth mask labeled by expert annotators. The subsequent panel (c) and (d), exhibit sequentially predicted ROIs representing the CSF lumen by XGB and eroded and dilated versions of the predicted images achieved using a 2 × 2 square filter for each corresponding model. Moreover, panels (e) and (f) present waveforms illustrating the mean velocity of CSF flow through the labeled and predicted ROIs, respectively. These waveforms offer a comparative analysis of the CSF flow characteristics as captured by the ground truth and predicted segmentation results from the different models.

Accuracy metrics

The overall precision, recall, and F1 score (or Dice similarity coefficient) were calculated for evaluating the segmentation results of ML models using fivefold cross-validation. By applying the harmonic mean precision calculated as TP(TP + FP) and recall(sensitivity) calculated as TP(TP + FN). These metrics were derived from the four components of the confusion matrix: True Positive (TP), representing successful identification of positive cases; True Negative (TN), indicating correct identification of instances in the negative class; False Positive (FP), denoting instances incorrectly predicted as positive; and False Negative (FN), indicating instances incorrectly predicted as negative. The F1 score metric was frequently employed to quantify the ratio of overlap for both classes. F1 score metric is defined with terms precision and recall such as.

$$F1score=\frac{2\ast Precision\ast Recall}{Precision+Recall}$$ 1

The weighted augmented average (WAA), which is an importance ratio of the classes (TP number of class / total instance number), is used to figure out the accuracy of each class individually³⁷. For each of the metrics calculated using WAA demonstrated following Eq. 12 where $N$ is sample size, $V_i$ is calculated evaluation metric for i_th class, and $C_i$ is the number of i_th class members;

$$WAA=\frac{1}{N}\sum _{i=1}^n{V}_{i}x{C}_i$$ 2

Table 2 provides precision, recall, and F1-score metrics for three ML models (LR, RF, XGB) across multiple folds in fivefold cross-validation. The average metrics highlight the robustness of the models, with slight variations in performance across folds. XGB demonstrates the highest average precision, recall, and F1-score among the evaluated models.

Table 2.

Precision, recall, and F1-score metrics of ML models (LR, RF, XGB) in 5-fold cross-validation.

K_fold	LR			RF			XGB
	Precision	Recall	F1-score	Precision	Recall	F1-score	Precision	Recall	F1-score
0	0.9971	0.7231	0.8375	0.9981	0.9327	0.9634	0.9981	0.9460	0.9705
1	0.9958	0.4155	0.5845	0.9973	0.9095	0.9502	0.9973	0.9239	0.9580
2	0.9971	0.6851	0.8114	0.9981	0.9412	0.9680	0.9981	0.9427	0.9688
3	0.9974	0.7108	0.8294	0.9983	0.9497	0.9726	0.9984	0.9614	0.9788
4	0.9976	0.6694	0.8005	0.9984	0.9498	0.9728	0.9984	0.9547	0.9753
Average	0.9970	0.6408	0.7727	0.9980	0.9366	0.9654	0.9981	0.9457	0.9703

Significant values are in bold.

Receiver Operating Characteristic (ROC) curves and area under curve (AUC) of the LR, RF, and XGB models are represented in Fig. 5. After evaluation in Fig. 5, it is evident that the XGB model outperforms LR and RF models in terms of AUC. Specifically, the mean AUC for XGB across the fivefold cross-validation is 0.958, surpassing RF (mean AUC = 0.944) and LR (mean AUC = 0.684) sequentially.

Figure 5.

The ROC curve and the AUC demonstrating the segmentation ability of CSF lumen in the thoracic SAS by with the mean of fivefold AUC XGB = 0.95,8, RF = 0.94,4, and LR = 0.68,4.

Flow metrics

Typically, a PC-MRI report includes statistical metrics such as mean, median, and peak CSF flow velocities and stroke volume which is the volume of CSF displaced during each cardiac cycle. The computation of these flow quantities relies on the regions delineated by radiologists. In Fig. 6, we present a visual representation of the comparative metrics in an analysis of CSF flow measurements. The figure provides insight into the dynamic changes in CSF flow velocity across 32 frames, capturing the variations throughout a single cardiac cycle.

Figure 6.

The comparative metrics in an analysis of CSF flow measurements: The figure illustrates the dynamic changes in CSF flow velocity throughout a single cardiac cycle across 32 frames. Each light gray line represents the flow velocity data for an individual pixel within the CSF lumen. The dashed red line represents the average flow rate, the dashed blue and green lines represent the median and peak values, respectively. Vertical gray dashed lines connect the mean flow velocity values to the x-axis, emphasizing temporal changes during the cardiac beat. The shaded areas under the mean curve correspond to the stroke volume, calculated as the integral of the mean flow velocity curve. Each computed area is texted between two mean values. This graph offers a visual depiction of the pattern of CSF flow over time within a single cardiac beat. The metrics calculated in a comparative analysis of CSF flow measurements are visualized and contribute to our understanding of the flow quantities used to evaluate the accuracy of these ML algorithms.

In this context, evaluating the accuracy of ML algorithms in producing the flow quantities, a comparative analysis of CSF flow measurements between the labeled and predicted ROIs. As part of the evaluation process, we utilize the Interclass Correlation Coefficient (ICC), a metric that quantifies the degree of approximation between two quantities. The computation of ICCs for stroke volume, average, peak, and median flow velocity parameters across both annotated and predicted regions in a single fold is formulated as follows;

$$\frac{{\sum }_{j=1}^N\frac{{\sum }_{i=1}^F\left(L_i,-,\overline{L}\right)\ast \left(P_i,-,\overline{P}\right)}{\sqrt{\sum _{i=1}^F{\left(L_i,-,\overline{L}\right)}^2}\ast \sqrt{\sum _{i=1}^F{\left(P_i,-,\overline{P}\right)}^2}}}{N}$$ 3

where $N$ is the samples count of each fold, $F$ is the frame number of each PC-MRI slab, $L_i=\text{1,2},3,\cdots i$ and $P_i=\text{1,2},3,\cdots i$ are one cardiac cycle frame’s values (mean, peak and median) in the labeled ROI and predicted ROI sequentially.

Table 1 presents the first fold cross-validation results of LR, RF, and XGB models. The table includes the stroke volume, mean, peak, and median ICC velocity values of the flow quantities through the labeled regions by experts and predicted regions by ML models to measure the consistency and agreement between the predicted and actual values. To enhance the interpretation of the results, Fig. 7 offers visual representations in the form of box plots for the stroke volume, mean, peak, and median ICC values. These visualizations aid in better understanding the distribution and variability of the data. XGB demonstrates the highest stroke and mean ICC of 0.7899 and 0.7749 sequentially, followed by RF with 0.7794 and 0.7628 and LR with 0.7448 and 0.7592. Due to their dependence on a single data point, peak values are highly susceptible to noise, potentially leading to less reliable correlation estimates. Consequently, ML models with lower recall exhibit lower peak CSF flow correlations. The LR model with the lowest recall value had the lowest peak correlation. Consequently, ML models with lower recall values tend to show diminished peak correlations in CSF flow. Specifically, the LR model, having the lowest recall value, exhibited the weakest peak correlation. Thus, while the stroke, mean, and median correlations consistently reveal a robust positive correlation, the peak correlation, influenced by its sensitivity to individual data points, exhibits a weaker relation.

Figure 7.

Box plots of ICC quantities by 0-Fold.

Explainability metrics

XGB offers 3 types of feature importance assessments; frequency-based, gain-based, and coverage-based importance which provides a global importance value for each feature. Figure 8 shows the feature importance analysis of XGB model. The features vel_4, vel_6, acc_6, acc_28, and vel_1 in Fig. 8A are the five most consistently utilized features across all stages of DTs in the boosting process. This frequent usage suggests that these features play a significant role in shaping the final predictions. In part Fig. 8B, vel_6, acc_5, acc_7, acc_6 and vel_4 are the five most highlighted as the primary contributors to enhancing the model's accuracy. These features demonstrate substantial gain values, indicating that their inclusion significantly improves the model’s ability to reduce prediction errors. In Fig. 8C, vel_6, acc_7, acc_5, acc_6, and vel_4 are the most five frequently chosen features for constructing splits within DTs suggesting their widespread presence in shaping the internal structure of the model.

Figure 8.

XGB feature importance graphs; part (A) weight feature importance, part (B) gain feature importance, part (C) cover feature importance.

While traditional feature importance methods usually provide an independent global importance value for each feature, SHAP values consider interactions between features and provide more accurate insights into feature importance. It provides localized insights into the probability that a pixel within the PC-MRI image resides within the CSF lumen. The global mean SHAP explanatory diagram of 0-fold XGB model is presented in Fig. 9. The features vel_6, vel_4, acc_7, acc_6, and acc_5 have the highest 5 SHAP values sequentially. In this case, each feature is of continuous nature and is arranged vertically based on its average influence on the prediction outcomes. It becomes evident that during the initial quarter of the cardiac cycle, the greatest influence on predictions for the CSF region is observed.

Figure 9.

The global SHAP explanatory diagram.

In addition to the global values, we also calculated local SHAP values of just one sample of the dataset to assess the significance of the contribution of each pulsatile feature to predictions on a PC-MRI slab. Thus, providing a deeper insight into the behavior of the XGB model in predicting pixels within the CSF lumen using SHAP. Prior to exploring into the local SHAP results, it is illustrative to direct our attention to Fig. 10 presented below, thereby to elaborate on the details of one sample where we calculated SHAP values. In this manner, we visualized velocity and acceleration flows of a sample in which we calculated SHAP values, specifically passing through the CSF lumen (to enhance the clarity of visualization by simplifying the complexity of the flow). On the visualization, we indicated the top 5 values of global SHAP importance using a vertical red line.

Figure 10.

The visualization of flow velocity and acceleration values of a sample’s frames sequence. The top 5 values(vel_6, vel_4, of global SHAP importance are pointed with vertical red lines.

Figure 11 plots provide insights into how each feature affects the model's predictions on a 16-year-old male sample. The left plot in Fig. 11 visually represents the contribution of each feature to the models' predictions on a pixel-by-pixel basis in the PC-MRI phase images. It is observable that features with the highest impact exhibit SHAP values extending to the right, indicating a positive contribution to the prediction of CSF lumen. Notably, the acc_19 feature, displaying a significant spread and high impact, appears to have outliers. This observation suggests the presence of outlier values, which could potentially lead to deviations from the model's prediction norms and should be taken into consideration. The right plot in Fig. 11, the waterfall graph, reveals that all features have positive SHAP values compatible with the left graph. The trend shows that as the value of the top 15 most key features increases, the XGB model's prediction of CSF lumen also increases. It is evident that vel_6, possessing the highest absolute SHAP value of 0.13, holds the greatest influence, just like its average effect on all predictions.

Figure 11.

Left: Local SHAP scores for XGB model displayed as a bee diagram. Scores are shown for each feature for 0-folds. As shown in the ‘feature value’ legend—a high value is indicated in red, and a low value is indicated by blue; for binary variables this means red indicates a value of 1 (i.e., CSF lumen) and blue indicates a value of 0 (i.e., not CSF lumen). Right: A SHAP waterfall plot illustrating the feature-level contributions to a PC-MRI image pixel prediction made by the model. The graph depicts the journey from the baseline prediction (starting point) to the final prediction for the given phase image. Each feature is with upward steps indicating positive contributions.

Discussion

Pulsatile flow data within PC-MRI has played a pivotal role in semi-automatic algorithms for CSF lumen segmentation, as discussed in the introduction. However, our present study introduces a novel approach by employing fully automatic ML algorithms that seamlessly incorporate pulsatile information from phase frames as features. Our motivation for adopting ML techniques stems from two key considerations: firstly, addressing the limitations of previous semi-automatic approaches that exclusively relied on pulsatile data for fully automatic segmentation; and secondly, recognizing the practical advantages of ML over DL methods, particularly when working with limited datasets. Given our modest dataset comprising 57 slabs and 2176 squares, ML solutions were deemed more appropriate for implementation. Our study exposes simpler and more efficient ML models trained only on pulsatile PC-MRI data over traditional DL approaches. This approach promises a deeper understanding of CSF flow dynamics and yields ML models that are interpretable, less prone to overfitting, transparent in decision-making, and cost-effective. We utilized LR, RF, and XGB algorithms due to their effectiveness with smaller datasets. Future research should explore the potential of DL methods, incorporate interpretability techniques specific to DL, and aim for hybrid approaches combining classical ML and DL methods for enhanced performance and transparency in various domains.

The precise quantification of CSF flow is a crucial component of neurological assessment. The evaluation of CSF flow in PC-MRI involves statistical metrics such as mean, median, and peak CSF flow velocities, as well as stroke volume—the volume of CSF displaced during each cardiac cycle. To assess the accuracy of ML algorithms in predicting these flow quantities, we conducted a comparative analysis between labeled and predicted ROIs. Utilizing the Interclass Correlation Coefficient (ICC) as a metric, which quantifies the agreement between two quantities, we present the results in Table 3. XGB is the best performer, exhibiting the highest stroke and mean ICC values of 0.7899 and 0.7749, respectively. Our results demonstrate the effectiveness of XGB in accurately predicting CSF flow metrics, exhibiting better stroke and mean ICC values. These findings underscore the potential of pulsatile data-driven ML algorithms, particularly XGB, in contributing to the automatically CSF flow assessments in PC-MRI, paving the way for enhanced diagnostic capabilities in neuroimaging analysis.

Table 3.

The mean, peak, and median ICC values of the flow quantities through the labeled regions by experts and predicted regions by 0-fold XGB, LR and RF models.

k_fold	Method	Stroke ICC	Mean ICC	Peak ICC	Median ICC
0	LR	0.7448	0.7592	0.1655	0.7273
	RF	0.7794	0.7628	0.5905	0.7258
	XGB	0.7899	0.7749	0.3966	0.7159

This study conducted a feature importance analysis on the XGB model and the SHAP method was applied. Consequently, valuable insights were found out regarding the interpretability of the model and the features affect its predictive performance. In the feature importance analysis of the XGB model, within the presented three types of feature importance assessments (frequency-based, gain-based, and coverage-based), features vel_4, vel_6, and acc_6 have consistently ranked among the top 5 features contributing significantly to improving model accuracy in all three evaluations. The collective mean SHAP explanatory plot (Fig. 9) for the XGB model reveals that vel_6, vel_4, acc_7, acc_6, and acc_5 features possess the topmost SHAP values in descending order. This representation effectively illustrates the specific influence of these features on the estimations of the CSF zone during the initial quarter of the cardiac cycle. Moving beyond global insights, local SHAP values were calculated for a single sample of the dataset, providing a deeper understanding of the XGB model's behavior in predicting pixels within the CSF lumen. Figure 10 offers a detailed visualization of the velocity and acceleration flows for a specific sample, highlighting the top 5 values of global SHAP importance using a vertical red line. The subsequent Fig. 11 plots delve into how each feature affects the model’s predictions on a pixel-by-pixel basis in PC-MRI phase images. The feature extraction strategy employed in this study involved treating PC-MRI data as a time-varying signal encoding information about the pulsatile nature of CSF flow. The extraction of 64 features for each pixel, including velocity and acceleration values, contributed to a comprehensive understanding of CSF dynamics. Notably, the calculation of flow acceleration between consecutive frames provided insights into the temporal changes in CSF flow. Overall, the combined analysis of XGB feature importance and global/local SHAP values offers a comprehensive evaluation of the model's behavior and highlights influential features critical to the accurate prediction of CSF flow in PC-MRI data. These insights significantly contribute to advancing our understanding of the model's internal dynamics and its potential clinical applications.

The thoracic spine’s spinal canal is more narrow and small than the cervical and lumbar spine’s, and there isn’t a significant portion flow in the lumbar cistern’s SAS because of caudal flow³⁸, which highlights the challenges of segmentation work in this area. This study suggested a lightweight ML model with significantly fewer parameters in comparison to the DL study conducted in the same region with the same dataset²². The PC-MRI routinely measures CSF flow rate characteristics at the aqueduct of sylvius location as it is synchronized with the cardiac cycle³⁹. To our knowledge, this study is the first XGB, LR and RF implementation on PC-MRI data to segment CSF lumens in the lower thoracic SAS which is located between the cervical and lumbar spines (T1–12). The fact that the thoracic spine’s spinal canal is smaller and narrower than the cervical and lumbar spine’s, and that due to caudal flow, there is very little flow in the SAS of the lumbar cistern³⁸ demonstrates the difficulties of segmentation work in this region.

The ongoing investigation into the relationship between CSF flow anomalies and idiopathic scoliosis (IS) underscores the importance of further exploration^40,41. In previous study⁴², no significant difference was found between age and luminal biomechanical changes in adolescent IS patients. However, considering that biomechanical changes may be related to differences within the lumen, they could potentially affect our results. With increasing age in adolescent IS patients, it is possible for degenerative changes and other potential accompanying anomalies to alter biomechanical properties. Further comprehensive studies are needed on this topic due to the limited literature in the field of PC-MRI, a burgeoning technique. The forthcoming aim is to automate the classification of patient groups by extracting features from automatically segmented regions of interest. This is crucial for predicting the progression of patients’ curvature using CSF flow data obtained from segmented areas, particularly considering that scoliosis progression may continue beyond the stage of maturity.

The signal-to-noise ratio depends on VENC in PC-MRI. Correct VENC value is therefore critical for high-quality images needed⁴. The proposed model can give better results in images with increased signal strength. The spatial resolution of MRI data is another feature that will improve the model’s success. A reduced FOV approach is feasible for the anatomy of the spinal cord, particularly because of the smaller section sizes⁴³. Better visualization of the cord structure is obtained with a higher spatial resolution by scanning a reduced spinal FOV PC-MRI image. One of the planned investigations will involve developing the model using high-resolution PC-MRI data.

To generate a balanced training dataset for LR, RF, and XGB algorithms, Random downsampling and Synthetic Minority Over-sampling technique (SMOTE)³⁶ are applied for upsampling. In addition to many resampling techniques for classification problems, in the most recent study, Liang and Martel showed the effectiveness of a novel resampling on a tree-based ML algorithm²⁹. This resampling method can be used to compare the ML techniques in subsequent research.

Conclusion

In conclusion, we introduced a novel approach to CSF lumen segmentation in the lower thoracic spinal canal using fully automatic ML algorithms. Unlike previous semi-automatic methods relying solely on pulsatile data, our ML models seamlessly incorporated pulsatile information as features from phase frames. The motivation behind adopting ML techniques was twofold: addressing limitations in DL approaches and recognizing more transparency in decision-making, and cost-effective advantages, especially when working with small datasets. By doing so, we have opened new avenues for more efficient and accurate CSF flow analysis, laying the foundation for future developments in the study of neurological disorders and their diagnosis. Furthermore, this study emphasizes the substantial potential of the interpretability provided by SHAP analysis for the XGB predictions of CSF region, with implications for both research and clinical applications. Future research is expected to make further steps in this area and optimize the model's utilization.

Author contributions

O.A did MR scans P.O and O.A annotate the MRI images and evaluate results A.K. write the codes. A.K. , M.B. and P.O. wrote the main manuscript text. All authors reviewed the manuscript.

Data availability

The data are utilized under MRI image data in DICOM format that support the findings of this study have been deposited in the National Magnetic Resonance Research Center (UMRAM) Archive. The data will not be made publicly available, as the use of such data is restricted solely to research purposes by the Ankara Yildirim Beyazit University ethics committee’s approval. Data are, however, available from the authors upon reasonable request and with the permission of the ethics committee. Those can contact Ayşe KELEŞ for inquiries regarding data access for this study.

Competing interests

The authors declare no competing interests.