Machine Learning For The Prediction Of Cervical Spondylotic Myelopathy: A Post Hoc Pilot Study Of 28 Participants.

Introduction

Cervical Spondylotic Myelopathy (CSM) is a debilitating degenerative disease of the upper spine affecting primarily elderly patients.^1–3 While compression of the spinal cord is thought to be the main driver of symptomology, pathophysiology is not entirely understood.^1–3 It has been well characterized that imaging can be suggestive of severe compression in asymptomatic patients or on the contrary normal appearing in patients with debilitating disease.^3,4 As such, imaging remains of limited benefit to clinicians without the appropriate associated clinical information.^1,3,4 As such, CSM remains a clinical diagnosis with imaging playing a role of supporting or refuting evidence at most.^1,3,4

Similarly, as technology advances, MRI imaging of the cervical spine continues to provide promising progress towards diagnostic capabilities of such patients.^4–7 Numerous studies have attempted to better categorize symptomatic compression through methods such as new MR techniques or sequences.^4–7 While many continue to show promise, none have been overwhelmingly successful, further propelling the strong need for further innovation and new, more granular diagnostic techniques.^8–13

With the advent and recent explosion of machine learning, statistical predictive capabilities are growing faster than ever before.^14–17 Relatively new to the medical imaging space, machine learning leverages innovation in computing power with ever improving modeling and statistical approaches.^14–16 Many medically trained machine learning algorithms are further capable of discriminating between discrete patterns of data often overlooked by even the best human experts.¹⁵ While these techniques remain promising, much of the field of artificial intelligence remains focused primarily on diagnostics and automation, with only a small minority focusing on development of tools capable directly predicting clinical outcomes.¹⁵ As such, the purpose of this pilot study is to explore the potential use of machine learning algorithms in predicting CSM and correlated clinical scores based on imaging characteristics alone.

Methods

13 CSM and 15 controls underwent imaging of the cervical spine. All CSM patients included were diagnosed at a single large academic institution by a board certified practicing neurosurgeon based upon a combination of both clinical and radiographic findings. (Table 1) Inclusion criteria for entry included the following in all patients diagnosed with CSM: classic CSM symptoms, including exam findings of weakness, hyperreflexia, or change in coordination; radiographic signs of spinal compression; Nurick grade I-IV¹⁸; and modified Japanese Orthopedic Association (mJOA) scores of <18¹⁹. Exclusion criteria included the following: age <21 or >80, comorbid neural disease (e.g., multiple sclerosis), pregnant or nursing, active systemic rheumatological disease, active peripheral or vascular neuropathy, urgent need for surgery. The study was conducted with the approval of the university’s Institutional Review Board (IRB).

Table 1.

Baseline Demographic and Clinica Characteristics

	Controls (Mean ± SD)	CSM (Mean ± SD)
Age	50.31 ± 11.56	59.38 ± 11.84
Gender (% MALE)	54%	62%
Height (in)	68.58 ± 3.37	67.08 ± 4.23
Weight (lbs)	171.69 ± 23.35	190.54 ± 41.45
Nurick	0 ± 0	1.85 ± 0.9
mJOA	18 ± 0	14.31 ± 2.14
Neck NRS	0.15 ± 0.55	5 ± 2.45
Arm NRS	0.23 ± 0.44	4.46 ± 3.26
NDI	1.23 ± 2.35	19 ± 9.5
Pain SF 6a	42.52 ± 5.1	61.73 ± 7.69
SF-36 PCS	32.61 ± 28.81	18.17 ± 23.82
SF-36 MCS	63.16 ± 6.29	60.23 ± 13.11

Abbreviations: mJOA- modified Japanese Orthopedic Association; NRS- Numeric Rating Scale; NDI- Neck Disability Index; Pain SF 6a- Pain Short Form Health Survey 6a; SF-36 PCS- Short Form (36) Health Survey Physical Component Summary; SF-36 MCS- Short Form (36) Health Survery Mental Component Summary.

Image Acquisition and Analysis

All imaging data were collected with a 3.0 Tesla Siemens Prisma magnetic resonance scanner (Siemens, Erlangen, Germany) equipped with a 64-channel head/neck coil. Participants were placed supine on the scanner bed, and a localizer scan was obtained to identify the location of the intervertebral discs of the cervical spine (C2-3, C3-4, C4-5, C5-6, C6-7, and C7-T1). Six high-resolution transverse slices with high white matter to gray matter contrast were acquired within the plane of each cervical intervertebral disc using a multi-echo gradient-echo sequence (TR=300 ms, TE=18 ms, Flip angle=30°, FOV=180×180, Matrix size=384×384, In-plane resolution=0.47×0.7 mm², Slice thickness=4 mm, number of averages=2).

Model 1- Predicting CSM diagnosis

Images were reviewed in a post-hoc fashion and the degree of cord compression was graded using 3 common literature scales (Kang, Nagata and Chang) alongside 3 MRI measurements (sagittal canal width, vertebral body height to vertebral disk height ratio and the C5 vertebral body sagittal width) all at the point of greatest compression on MRI. These six features were used to train a deep neural network (DNN) classification model (Figure 1) using the Keras open source Python package. The model was trained and tested using cross-validation, in which the data were randomly partitioned into training (n=18) and testing (n=10) datasets. In training, the 18 training images were fed through a series of 7 layers each with varying degrees of forward and backward communicating nodes (neurons). Dropout layers were introduced sporadically, preventing a certain percentage of neurons from communicating forward or backwards at different time points during the training to prevent overfitting and keep the model generalizable. The model was then trained and tested across a total of 200 random partitions using a batch size of 4 and 25 iterations for training the model. Mean and median cross-validated accuracies, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were used to assess the model performance.

Figure 1-.

Algorithm specifications for Deep Neural Network (DNN) used in Model 1. All variables underwent processing through a 7 layer artificial neural network with above specified neurons in each layer. Images were analyzed in groups of 4 at a time until all 18 were completed, at which point the algorithm would adjust predictive variable weights and repeat for 25 epochs/repeats. Upon completion, the final algorithm would be used to verify accuracy on 10 unseen images from the original dataset. Images were redistributed randomly into test/training groups, model weights were reinitialized and randomized and the entire process was repeated 200 times for population data collection.

Model 2- Predicting CSM Severity

Images for each transverse slice were further analyzed using the Spinal Cord Toolbox (version 3.0.7) and the PAM50 spinal cord template as described previously.^20,21 Regions of the spinal cord were segmented and volumetric and cross-sectional measurements were collected for each region of interest. Regions of interest included anterior/posterior diameter, eccentricity of the spinal cord, ventral corticospinal tract, ventral reticulospinal tract, medial reticulospinal tract, lateral corticospinal tract, rubrospinal tract, lateral reticulospinal tract, ventrolateral reticulospinal tract and medial longitudinal fasciculus (each measured volumetrically using voxels and metrically using millimeters). The following gender, age, height, weight, level and the above noted parameters were input into our deep neural network, comprising a total of 23 input variables with the only output variable being mJOA score. Model 2 specifications are outlined in Figure 2. The model was trained with data partitioned into two datasets: training (n=78) and testing (n=26). Similarly to above, the 78 training data points were input into 9 layers, each with varying degrees of nodes (neurons). Sporadic dropout layers were added just as in Model 1 to prevent overfitting. Upon completion, the model was further trained and tested using a batch size of 3 and 1,250 iterations. This process was, similarly, repeated a total of 150 times in order to better characterize population data. Outputs were defined as a numeric prediction of mJOA score. Model performance was evaluated based upon mean squared error and subsequent average error in predictions were calculated. Error was defined as the total difference between predicted mJOA scale value and actual mJOA scale value.

Figure 2-.

Algorithm specifications for Deep Neural Network (DNN) used in Model 2. All variables underwent processing through a 9 layer artificial neural network with above specified neurons in each layer. Images were analyzed in groups of 3 at a time until all 78 were completed, at which point the algorithm would adjust predictive variable weights and repeat for 1250 epochs/repeats. Upon completion, the final algorithm would be used to verify error on 26 unseen data points from the original dataset. Images were redistributed randomly into test/training groups, model weights were reinitialized and randomized and the entire process was repeated 150 times for population data collection.

Results

Model 1- Predicting CSM

The mean cross-validated accuracy of the trained model was 86.50% (95% CI 85.16%-87.83%) with a median accuracy of 90.00%. A distribution of accuracies across the 200 partitions is shown in Figure 3. Out of 200 partitions, the machine learning model was able to predict CSM (versus controls) with 100% accuracy 32 times (16.0%), 90% accuracy 91 times (45.5%), and 80% accuracy 59 times (29.5%). The program failed to predict CSM at an accuracy of 80% or greater in only 18 times (9.0%). Area under the curve (AUC) was calculated for each repetition. Average AUC for each repetition was 0.947 with a median AUC of 1.0. Average sensitivity, specificity, PPV and NPV were 90.25%, 85.05%, 81.58% and 91.94% respectively. A representative receiver operator curve from one iteration is shown in Figure 4.

Figure 3-.

Distribution of cross validation accuracies of CSM predictions on 10 previously unseen patients after first training and model adjustment on random sub-sets of 18 patients.

Figure 4-.

Representative receiver operator curve for Model 1. Area under the curve for this particular repetition was 0.880. Average AUC for all models was 0.947 with a median AUC being 1.0.

Model 2- Modeling mJOA score

The mJOA score predictive model tended to slightly underpredict mJOA scores, with a mean error of −0.29 mJOA points and a median of −0.08 mJOA points. Average mean squared error (MSE) was 0.47 ± 1.13. The standard deviation of mean error per batch was 0.714 mJOA points. A representative curve of mean errors per batch is represented in Figure 5. Average error of within 1 mJOA point of the actual mJOA score occurred roughly 81.0% of the time. Similarly, an average error of .5 mJOA points or less was noted approximately 48.5% of the time.

Figure 5-.

Outcome Distribution of Model 2. (A) Distribution of mean error per batch in units of points of mJOA score. (B) Distribution of median error per batch in units of points of mJOA score.

Discussion

Machine learning within medical imaging has grown immensely over the past 5-10 years. With the invention of Graphical Processing Units (GPUs), increased computing power has propelled technology to the possibility of raw unprocessed images into computer algorithms, each with the ability to produce designated unaided, unaltered, outputs without supervision. Numerous real-life applications have been documented in the literature with applications ranging as wide as from pulmonary embolism segmentation on emergency CT, to cognitive state of impairment in Alzheimer disease using fMRI.^22–28

Literature on machine learning techniques specific to spine remains scarce, with the majority of papers focusing on identifying regions of interest in the lumbar spine or streamlining radiographic analysis.^29–33 The current study is among the first to attempt to bridge the gap between strictly imaging diagnoses and clinical symptoms in the cervical spine. To date, few papers exist using machine learning techniques to look at clinical outcomes. Kim et. al attempted to predict clinical outcomes from pre-operative data points and complications after single level lumbar posterior fusion, noting that deep neural networks performed better than alternative statistical methods with AUCs ranging from 0.606-0.710 from 20,000 patients.³⁴ Similarly, Durand et. al notes the use machine learning approaches to predict the need for blood transfusion after lumbar spinal surgery to be promising reporting an AUC of 0.79.³⁵ The current paper produced a mean classification accuracy of 88% and an AUC of 0.947 with very few subjects available for training. Similarly, our paper further was able to predict mJOA score to within an average of 0.04 points based purely on computer automated spinal cord volumes.

The current paper improves upon literature suggesting imaging alone may eventually prove adequate in diagnosis and prognosis of patients with CSM. Currently, much of the literature involving CSM and imaging studies has been done on the use of DTI imaging and MT imaging.^36–39 While each show elements of promise individually, no studies have demonstrated overwhelmingly conclusive results. As such, current investigational methods remain research based only, as currently they add little value to the physician in practice. Our paper demonstrates a much needed first step at the implementation of machine learning as a tool with potential to augment decision making of clinicians.

The goal of machine learning in medicine should not be misinterpreted however. With the recent boom in technological advancement, media and news too often tend to exaggerate as to the true abilities and impact machine learning can have on medicine.⁴⁰ Primarily, the authors believe machine learning should not be intended to replace physician intuition and judgement. The authors rather believe machine learning has the ability to be used clinically simply as yet one more data point in the already complex decision making process; in the case of CSM these decisions may be as straightforward as on whom and when to operate. Such scoring decision tree methods are already in place in medical decision making in numerous specialties, noteworthy examples being HAS-BLED⁴¹ or CHA₂DS₂-VASc⁴² scores among many others.

Our study is not without limitations. First, due to the prospective nature of our data we were limited in sample size and as such were perhaps not able to realistically approximate true predictive power. Similarly, the study uses close to equal numbers of CSM and control subjects in development and implementation of the model. This ratio is likely much smaller in the real world, with the population of CSM patients compared to controls being much lower than was in the current study. Our study is further limited by the lack of consistent industry standards for optimization of machine learning algorithms.⁴³ Currently, gold standards for optimization of DNN models involve simple guess and check methods, with comparative partition analysis commonly used as the main decision factor in determining the optimum model.⁴³ In doing so, single variables are altered manually and arbitrarily and compared to the current best known model.⁴³ These changes are either accepted or declined depending on the relative performance of the model over many simulations.⁴³ As such, due to lack of standardized method, bias in model construction is a definite confounder in our above study as no current theory exists on best practices in varying model parameters. Lastly, as our study was a controlled prospective study, only patients with clearly diagnosed, classic CSM were enrolled. While ideal for a pilot study, the lack of patients on the fringes of diagnosis may introduce artificially inflated predictive statistics. In real life, patients are more heterogeneous, often presenting without an obvious diagnosis. These types of patients were not included in our model unfortunately due to the nature of study design. Despite these limitations, the authors believe the above findings to be important and useful for the future of improving outcomes amongst patients with CSM.

Potential next steps in exploring machine learning related to CSM imaging include larger, more complex model development. Currently, methods are available to feed entire cervical MR images into computer models without human preprocessing.⁴⁴ While requiring a large amount of computing power, such methods would eliminate the problematic human error or bias introduced by much of current diagnostic methods. Similarly, by feeding entire images into the model without alteration, it gives training the needed freedom to find new predictors not otherwise previously noted by human investigators.

Conclusions

Machine learning provides a promising method for prediction and diagnosis for patients with CSM. In this pilot study, after reviewing features from only 18 images, our classification model was able to predict CSM from controls with a median accuracy of 90% as well as predict mJOA scores within 0.4 points using imaging characteristics alone. While still only preliminary, the current study demonstrates promise and feasibility for the use of machine learning to better improve diagnostic and predictive methods for CSM as well as other cervical spine disorders.

Abbreviations-

CSM

Cervical Spondylotic Myelopathy

mJOA

modified Japanese Orthopedic Association

MR

Magnetic Resonance

MRI

Magnetic Resonance Imaging

AUC

Area Under Curve

IRB

Institutional Review Board

DNN

Deep Neural Network

PPV

Positive Predictive Value

NPV

Negative Predictive Value

MSE

Mean Squared Error

GPU

Graphical Processing Unit