Predicting early symptomatic osteoarthritis in the human knee using machine learning classification of magnetic resonance images from the Osteoarthritis Initiative

Abstract

The purpose of this study is to evaluate the ability of a machine learning algorithm to classify in vivo magnetic resonance images (MRI) of human articular cartilage for development of osteoarthritis (OA). Sixty-eight subjects were selected from the Osteoarthritis Initiative (OAI) control and incidence cohorts. Progression to clinical OA was defined by the development of symptoms as quantified by the Western Ontario and McMaster Universities Arthritis (WOMAC) questionnaire three years after baseline evaluation. Multi-slice T₂–weighted knee images, obtained through the OAI, of these subjects were registered using a nonlinear image registration algorithm. T₂ maps of cartilage from the central weight bearing slices of the medial femoral condyle were derived from the registered images using the multiple available echo times and were classified for “progression to symptomatic OA” using the machine learning tool, weighted neighbor distance using compound hierarchy of algorithms representing morphology (WND-CHRM). WND-CHRM classified the isolated T₂ maps for the progression to symptomatic OA with 75% accuracy.

Statement of clinical significance

Machine learning algorithms applied to T₂ maps have the potential to provide important prognostic information for the development of OA.

Introduction

Osteoarthritis (OA) is a highly prevalent disease associated with articular cartilage degeneration^{1, 2}. Magnetic resonance imaging (MRI) is a noninvasive and multi-contrast modality that can detect subtle morphologic changes in articular cartilage and is an effective tool for diagnosing established OA^{3, 4}. Computer-aided diagnostic tools have the potential to assess earlier disease status and are currently the focus of significant MRI research^5-8. Specifically, image content descriptors (features) have been shown to be informative in detecting subtle differences between textures and intensities between healthy and pathological cartilage in both the in vitro and in vivo setting^{6, 8}.

Recently, we evaluated the ability for a machine learning algorithm, weighted neighbor distance using compound hierarchy of algorithms representing morphology (WND-CHRM)⁹, to classify MRIs of human osteochondral plugs of normal and pathologic cartilage based on Osteoarthritis Research Society International (OARSI) histological scores^{8, 10}. The features computed from WND-CHRM were used to develop a multivariable least-squares regression model to predict OARSI scores and classify OA cartilage with up to 86% accuracy in various MR contrast modalities. Our previous in vitro results indicate the ability to non-invasively assign individual subjects to a degree of OA pathology⁸. One essential difference between this previous study of explants and analyses performed in the in vivo setting is the requirement for reproducible tissue segmentation in the latter.

Related work has been performed using in vivo data from the Osteoarthritis Initiative (OAI)¹¹. Urish et al. found that texture analysis of cartilage MRI relaxation time (T₂) was effective at differentiating subjects at risk for developing OA symptoms⁶. In that study, they used a rigid registration algorithm to align double echo in the steady state (DESS) images to T₂ parameter maps. Using a global active statistical shape model with a local active contour model, they performed semi-automated segmentation of the DESS images and applied the mask to T₂ parameter maps for feature extraction. They achieved approximately 70% classification accuracy in predicting the symptomatic progression of OA as defined by the Western Ontario and McMaster Universities Arthritis (WOMAC) questionnaire¹². While these results are promising, the analysis was restricted to texture features extracted from the raw image. However, additional families of features computed on multiple image transforms may provide further information about early cartilage degeneration.

It is of additional importance to note that the intra-subject registration method used in Reference 6 does not provide the ability to evaluate localized cartilage segments with spatial correspondence between subjects of different groups. Therefore, the development of 3D nonlinear image registration methods is of particular interest for application to large clinical datasets, such as the OAI, so that highly specific regions of cartilage can be isolated and directly compared between subjects. In addition, improved methods for inter-subject registration would provide the ability to accurately evaluate longitudinal changes to localized regions. Once images are registered to a universal target, automated segmentation of knee structures becomes straightforward. Hence, as we demonstrate in this study, nonlinear registration prior to segmentation has the potential to overcome the various limitations of other previously established automated segmentation algorithms^13-15, which have been reported to overestimate the articular surface, resulting in partial volume effects with subchondral bone, and exhibit substantial variation among methods¹⁶.

Here, we evaluate the ability of a machine learning algorithm to classify in vivo MRI of human articular cartilage for the development of OA. Our approach is based on an automated segmentation of articular cartilage in T₂-weighted (T₂W) images through a nonlinear inter-subject registration scheme applied to data from the OAI. The registered T₂ parameter maps of localized regions of segmented cartilage from the weight-bearing region of the medial femoral condyles were classified by WND-CHRM for symptomatic OA progression, according to a change in three-year WOMAC score.

Materials and Methods

Patient cohort

Data were obtained from the OAI database, available for public access at http://oai.epi-ucsf.org. The specific OAI imaging datasets explored here included 0.E.1 and 0.C.2. The kXR SQ (BU) clinical dataset [versions 0.6 and 5.5] was used for obtaining the Kellgren and Lawrence (KL) grades for each participant¹⁷. The PhysExam dataset [versions 0.2.2 and 5.2.1] was also used to assess the WOMAC score¹² at the time of baseline MR screening and 36 months later. A total of 68 patients were selected from the OAI prospective cohort; 28 were from the non-exposed control sub-cohort and 40 were from the incidence sub-cohort (Fig. 1). In this study, subjects assigned to the “non-progression” group were selected from the non-exposed control sub-cohort (n = 122) and defined at baseline and at 36 months by a WOMAC score ≤10 with a KL score < 2 (Fig. 1). The 20 patients from the incidence sub-cohort were selected based on the initial criteria of a WOMAC score ≤ 10, but with a change in WOMAC score of > 10 within 36 months from baseline, and with minimal baseline radiographic signs of OA defined as a KL < 2 (Fig. 1). Here, we define this group as “symptomatic progression of OA”.

Figure 1.

Outline of study design. The non-progression group was randomly selected from the OAI Control cohort based on criteria of baseline KL grade < 2, baseline WOMAC ≤ 10 and a 36-month change in WOMAC < 10 (n = 28). The symptomatic progression of OA group was randomly selected from the OAI Incidence cohort based on the criteria of baseline KL grade < 2, baseline WOMAC ≤ 10 and a 36-month change in WOMAC > 10 (n = 40). MRIs from these 68 patients were then processed and used for classification analysis.

MRI acquisition

The T₂W images were obtained from the OAI database (http://oai.epi-ucsf.org). Briefly, MRI of the knee joint was performed on whole body MAGNETOM Trio 3 tesla Multisite scanners (Siemens, Erlangen, Germany) using a standard extremity bird-cage coil¹¹. Sagittal 2D T₂W images were acquired using a multi-spin-echo sequence. Seven images with echo time (TE) increasing linearly from 10 to 70 ms were acquired with repetition time = 2700 ms, field of view = 120 × 120 mm², matrix = 384 × 384, and 27 slices of 3 mm thickness. Total acquisition time was ∼ 11 minutes. Full acquisition details are given in Reference 11.

Image registration

All images were first filtered to achieve noise reduction through application of a multi-spectral non-local means filter, an edge preserving filter, which averages pixel intensities across multiple T₂ weightings according to their intensity similarity¹⁸. A nonlinear image registration technique using the Automatic Image Registration (A.I.R) toolbox (http://bishopw.loni.ucla.edu/air5/)^19-24, implemented in Matlab (MathWorks, Natick, MA, USA), was optimized and applied for inter-subject registration of T₂W images obtained at different TE (Fig. 2). A control patient with a BMI of 25.7 and KL grade of zero was selected as the target image to which all subjects' T₂W images were registered.

Figure 2.

Flow chart of processing steps leading from the unfiltered and unregistered moving image to the cartilage segments. The target and unregistered images were cropped to reduce the field of view and to suppress regions of fat and muscle. Then, a multispectral non-local filter was applied to these images. After filtering, both the multi-spectral target and unregistered filtered weighted images were each averaged across TE. The averaged filtered unregistered image was then registered to the averaged filtered target image. The calculated matrix of deformation derived from this registration was directly applied to the original unfiltered and unregistered T₂W image at each TE. A 3D cartilage mask was generated through manual segmentation of each slice from the target image, which was then systematically applied to each of the registered images to isolate cartilage segments from all subjects.

Image registration was performed using a reference image obtained by simple averaging across the images obtained with different TEs in order to maximize signal-to-noise and contrast. The A.I.R. toolbox incorporates variants of nonlinear polynomial spatial transformation models ranging from first (12 parameters) to twelfth (1365 parameters) order. After evaluation of all parameter models (Fig. 3), registration was performed using the tenth order model to optimize accuracy and computational efficiency. The calculated deformation matrix was applied to the original unfiltered T₂W images for each TE in order to create the registered image using linear interpolation in all three directions (Fig. 2).

Figure 3.

Examples of the quality of registration for six different orders of the nonlinear polynomial transformation model. For each model order, performance of the registration was evaluated through superimposed and difference images between target and registered images. The green and purple colors in the superimposed images indicate misregistered regions. Both the overlap and difference between target and registered images decrease as the order of the model increases from second to eighth order, with no improvement seen for higher order corrections.

Registration validation

The quality of registration was evaluated both qualitatively and quantitatively. The qualitative evaluation entailed a visual inspection of the superimposed target and registered images at each TE. The quantitative evaluation was performed by computing the difference image between the target and registered images at each TE.

In order to ensure that the registration process did not introduce bias, we also compared T₂ maps in the cartilage regions of the unregistered images before and after registration. T₂ parameter maps were generated from all seven weighted images taking into account the expectation value of the Rician probability density function, as appropriate for magnitude MR images^{25, 26}. In addition, we used the extended phase graph algorithm^{27, 28} to correct for stimulated echoes caused by radio frequency pulse imperfections^{29, 30}. All numerical calculations were performed using Matlab.

Cartilage segmentation

A cartilage mask was generated through a single manual segmentation of the medial femoral cartilage in each slice of the target T₂ map image. The cartilage mask was systematically applied to the each of the registered images to isolate the desired segments from all subjects. Three central weight-bearing slices from the medial femoral condyle were used for classification analysis.

Classification Analysis

WND-CHRM has previously been described in detail^{9, 31}. Briefly, the WND-CHRM algorithm extracts a generic set of numerical image content descriptors, including textures, which measure spatial variation in intensity for several directions and resolutions, statistical distribution of pixel intensity values from polynomial decomposition of the raw image, as well as several image transforms, such as Fourier, Chebyshev, and wavelet, of the raw image. Further, these features are implemented as numerical decompositions of the image and its transforms according to pre-described procedures. The physical meaning of these descriptors is therefore in terms of image characteristics and features. However, there are no specific correspondences between tissue state and these image features. This is a fact often seen in machine learning applications, in which the physical meaning of classifiers takes a back seat to the structure of the images themselves. This results in a final set of 2,919 features (WND-CHRM version 1.60). The features are then ranked by their Fisher discriminant, defined here as the ratio of variance of class means from the pooled mean to the mean of within-class variances³². Because not all features are equally informative and many represent noise^{9, 32}, only the top 100 were used for classification.

A leave-one-out analysis was performed where the class probabilities were computed using the weighted neighbor distance classifier (WND5)³¹. In the WND5 classifier, the feature vectors from the training images are arranged in a (weighted) feature space, where each feature value is multiplied by its corresponding Fisher discriminant as defined above. The relative probability of a test image belonging to a certain class is calculated as the mean of the inverse fifth power of the Fisher-score weighted distances, r, between the test image and all training images of that class. Each image feature is assigned a Fisher score, W_f :

$$W_f=\frac{1}{N}\sum _{c=1}^N\frac{{({\overline{T}}_f-\overline{T_{f,c}})}^2}{{{\sigma }^2}_{f,c}}$$

where W_f is the Fisher score of feature f, N is the total number of classes, T̄_f is the mean of the values of feature f among the images allocated for training, and $\overline{T_{f,c}}$ and are the mean and variance of the values of feature f among all training images of class c. All variances are computed after the values of feature f are standardized to the interval [0,1].

A leave-one-out cross validation analysis was performed such that training sets were constructed with equal numbers of images from the non-progression and symptomatic progression of OA classes. Each sample was in turn excluded from the training set and designated as a test image for assignment to the class with the highest probability. Classification results are reported as sensitivity (proportion of correctly classified symptomatic progression of OA images), specificity (proportion of correctly assigned non-progression images) and accuracy (proportion of correctly assigned samples).

Results

Patient Cohort

Table 1 shows the mean characteristics of the patients selected from the OAI Control and Incidence cohorts. All subjects in the control and symptomatic progression of OA groups had baseline KL grades of zero. The control group had a 36-month change in WOMAC of -0.1, indicating lack of progression. The symptomatic progression of OA group had a change in WOMAC of 25, indicating clear progression of OA symptoms over the 36-month follow-up period. The mean age and BMI of the symptomatic progression of OA group were slightly higher than in the control group (Table 1).

Table 1. Patient characteristics.

Non-progression group
Mean age (years)	Mean baseline KL	Mean change in WOMAC	Mean BMI
56	0	-0.1	25
Symptomatic progression of OA group
Mean age (years)	Mean baseline KL	Mean change in WOMAC	Mean BMI
59	0	25	29

Image Registration

Performance of the registration as a function of the order of the nonlinear polynomial transformation model is shown in Figure 3. Before registration, the superimposition of the target (green) and the moving (purple) images showed several misaligned regions (Fig. 3). For each model order, the quality of registration was evaluated through superimposition of the target and registered images (Fig. 3, top two rows), and by calculating the difference between the target and registered images (Fig. 3, bottom two rows). The green and purple colors in the superimposed images in Figure 3 indicate mis-registered regions. As shown, both the misregistered regions and the difference between target and registered images decrease as the order of the model increases from second to eighth, with no improvement seen for higher order corrections.

Figure 4 shows a representative example of a T₂ map of cartilage calculated before and after registration. In all cases, calculated T₂ values were similar, within 10% with no systematic trends, before and after registration in all regions of cartilage, indicating that no significant bias was introduced during the proposed registration process.

Figure 4.

Representative T₂ maps calculated before and after registration in knee cartilage regions. T₂ maps were superimposed on their corresponding T₂W images before and after registration. The T₂ values are shown to be minimally changed before and after registration, indicating that the registration methods did not introduce bias to the T₂ maps.

Classification and regression analysis

Table 2 shows the classification accuracy of WND-CHRM applied to T₂ maps of medial femoral condyle cartilage. The sensitivity, specificity and accuracy were 74%, 76% and 75%, respectively. The most informative features for classification, according to their rank ordered Fisher scores, were the Zernike coefficients and Chebyshev statistics computed from the Fourier transform, as well as the Haralick textures computed from the wavelet (Symlet 5, level 1) transform. Other informative features included multi-scale histograms from several of the image transforms, as well as Zernike coefficients and Haralick textures computed from the Fourier transform.

Table 2.

WND-CHRM classification accuracy of isolated T₂ maps of cartilage. Intervals based on 95% confidence using the Normal Approximation method.

Sensitivity	0.74 ± 1.3%
Specificity	0.76 ± 1.3%
Accuracy	0.75 ± 0.9%

Discussion

The weight-bearing regions, specifically the central portion of the medial femoral condyle, of the knee are the most susceptible to changes in cartilage thickness associated with early OA^{33, 34}. Consequently, this region of cartilage may be expected to show subtle pattern differences before the frank transition to OA. T₂ relaxation time as a quantitative metric is not the best marker for OA, as there is no highly-sensitive reference standard with which to compare³⁵. However, since T₂ relaxation is related to the presence of free water molecules, which slow down the loss of transverse magnetization, T₂ maps provide for a direct measurement of water content in cartilage³⁵. In OA, as cartilage matrix begins to break down and become more permeable to water, there is an elevation of T₂ relaxation time. Thus in turn, T₂ provides for an indirect assessment of collagen content and orientation, which are important indicators for early OA, and is used clinically to evaluate disease status^{36, 37}. In this study, we demonstrate the ability for a state-of-the-art machine learning algorithm, WND-CHRM, to classify T₂ maps of the central weight-bearing slices of cartilage within the medial femoral condyle for the progression to symptomatic OA, defined as a change in WOMAC score of >10 three years from baseline evaluation.

WND-CHRM classification is based on inherent image texture and intensity information, rather than measurements such as cartilage volume or thickness. The pattern recognition component of these methods relies on aligned images from which to extract information. Therefore, we developed a new approach for automated articular cartilage segmentation using nonlinear inter-subject registration of T₂W images. The application of nonlinear registration provided the ability to automate cartilage segmentation within a large dataset and to isolate corresponding slices of cartilage from the medial femoral condyles between patients. From this, we were able to apply WND-CHRM to successfully evaluate image features from standardized regions within the knee. We found that image features computed by WND-CHRM were correlated with patient-reported ground truth data, that is, WOMAC score. We demonstrated that WND-CHRM was able to classify T₂ maps of cartilage, obtained prior to clinical OA, to predict the development of clinical OA with 75% accuracy. This represents a substantial improvement as compared to the result obtained using conventional classification analysis based on T₂ according to the Euclidean distance metric (EDM). The EDM showed sensitivity, specificity and accuracy of only 50% using the same dataset, indicating lack of any prognostic ability.

The most informative features for classification provided information about pixel intensities and textures. Specifically, low frequency features including Zernike and Haralick features, Chebyshev statistics and multi-scale histograms measure small variations in pixel values and provide information about texture over large scale areas with smooth intensity transitions³¹. The variation in pixel intensity patterns derived from these groups of texture features have been previously shown to correlate with the biochemical and biomechanical alterations of OA cartilage in the hip joint³⁸. Further, we found that these low frequency features, which measure subtle variations over regions spanning many pixels, were most informative for evaluating cartilage. Since the T₂ maps used in this study included only cartilage, the images had relatively low intensity variation, and in turn, high contrast features, such as edge and object statistics did not provide useful information for classification.

We note that in our previous study, we used WND-CHRM features computed on MRIs of osteochondral plugs to construct a multiple linear regression model for classification based on OARSI score⁸. We found that the accuracy of binary classification based on several MR measurements, including the diffusion weighted image with b = 999 s/mm² and the T₂W image with TE = 50 ms, improved through use of regression, which establishes relationships throughout the entire dataset for classification of single samples. In Reference 8, the predictors, were the descriptive values assigned to particular features, while the outcome variables, were the histologic OARSI score for each sample. Yet, when similar analyses were performed in the present study using change in WOMAC scores as the outcome variables, we found no such improvement. We attribute this difference in findings to the ground truth information associated with the images; in our previous study, each osteochondral plug had a corresponding OARSI score, which is an objective histological measurement for OA and corresponds to degradation status reflected in the images; These plugs [from the previous study] were obtained from patients with severe OA undergoing knee arthroplasty, in contrast to the present study where the patients did not have clinical OA at baseline. Thus the relationships between MR outcomes, symptoms (e.g. WOMAC) and tissue degradation status are expected to be much more subtle in the present case. At this early stage of disease, we do not expect that WOMAC will reflect tissue degradation. The cartilage used in the present study was classified based on WOMAC score, which is a subjective, patient-reported, measurement and does not provide information about cartilage structure.

There are certain limitations to our study. The processing time for the 10th order polynomial registration model was on the order of several hours per subject. The use of lower-order procedures or other more rapid algorithms should be explored in further studies. Since registration was systematically performed using a tenth order polynomial model, there was substantial computational processing time (several hours per patient). Further, we elected to incorporate data from three representative central slices into our analysis in order to efficiently incorporate OA status into the analysis. The question of whether inclusion of all tissue slices, covering the entire knee and perhaps including non-cartilage regions, would improve classification accuracy, may be addressed in further studies. However, in this study we restricted our analysis to the three slices that cover the central weight-bearing region of the femoral condyle and excluded the peripheral slices that are likely subject to partial-volume effects. In addition, our registration method is dependent on the target image. In effect, the accuracy of this approach relies on how well the algorithm can register an image to its corresponding target. We excluded from classification any registered image with interpolation errors, which were occasionally observed in the femur region of patients with substantially smaller bone size than the target. The development of a universal knee atlas has the potential to limit this source of potential bias and would be of particular benefit in the analysis of larger cohorts of patients from the OAI.

Although we analyzed fewer patients than a previous study⁶ that used similar inclusion criteria, the authors of that study note that in order to maximize the size of the dataset (n = 168), they included patients with a baseline KL grade of two in the progression to symptomatic OA group. This KL score indicates early degeneration, so that inclusion of these subjects would enhance classification accuracy. We elected to exclude patients with a KL grade of two in our analysis, resulting in a smaller sample size but presenting a more realistic, though more challenging, classification task. In spite of this, we achieved an overall classification accuracy of 75%, comparable to the results shown in Reference 6.

Important extensions of this study will be to incorporate additional cartilage slices into the analysis, as noted above, and to combine T₂ maps with additional MR imaging contrast modalities. Indeed, it has been shown that incorporation of different MR modalities greatly strengthens classification of early and late cartilage degeneration^39-43, so that this may be a useful approach in the context of WND-CHRM analysis as well. Finally, use of cartilage morphology and incorporation of other joint structures, such as periarticular bone, has the potential to greatly improve classification for progression to symptomatic OA. The promise of this has approach is indicated by the fact that WND-CHRM has already shown success in classifying X-ray images based on KL grade³².

In conclusion, we have demonstrated that image features derived from appropriately pre-processed T₂ maps of cartilage from the weight-bearing region of the medial femoral condyle may be useful for the detection of OA progression.