Automated Segmentation of the Healed Anterior Cruciate Ligament from T₂* Relaxometry MRI Scans

Abstract

Collagen organization of the anterior cruciate ligament (ACL) can be evaluated using T₂* relaxometry. However, T₂* mapping requires manual image segmentation, which is a time-consuming process and prone to inter- and intra- segmenter variability. Automating segmentation would address these challenges. A model previously trained using Constructive Interference in Steady State (CISS) scans was applied to T₂* segmentation via transfer learning. It was hypothesized that there would be no significant differences in the model’s segmentation performance between T₂* and CISS, structural measures versus ground truth manual segmentation, and reliability versus independent and retest manual segmentation. Transfer learning was conducted using 54 T₂* scans of the ACL. Segmentation performance was assessed with Dice coefficient, precision, and sensitivity, and structurally with T₂* value, volume, sub-volume proportions, and cross-sectional area. Model performance relative to independent manual segmentation and repeated segmentation by the ground truth segmenter (retest) were evaluated on a random subset. Segmentation performance was analyzed with Mann-Whitney U tests, structural measures with Wilcoxon signed-rank tests, and performance relative to manual segmentation with RMANOVA/Tukey tests (α=.05). T₂* segmentation performance was not significantly different from CISS on all measures (p>.35). No significant differences were detected in structural measures (p>.50). Automatic segmentation performed as well as retest on all segmentation measures, while independent segmentations were lower than retest and/or automatic segmentation (p<.023). Structural measures were not significantly different between segmenters. The automatic segmentation model performed as well on the T₂* sequence as on CISS, and outperformed independent manual segmentation while performing as well as retest segmentation.

INTRODUCTION

T₂* relaxometry allows for the non-invasive assessment of collagen organization in soft tissues.^1-6 Highly organized collagen structures demonstrate a shorter T₂* relaxation time because the density and anisotropy of the collagen fibers results in increased proton spin interactions that force faster dephasing of free water protons.⁷ Soft tissue injuries negatively impact collagen organization and thus manifest as longer T₂* relaxation times. Due to this relationship, it has been shown that T₂* relaxation time is associated with mechanical properties,^{2; 3} tissue degeneration,^{1; 6} post-surgical risk factors for reinjury,^{2; 3; 8} and osteoarthritis risk.^9-11 This could make T₂*-based measures clinically useful biomarkers to predict surgical outcomes following ACL surgery. Patients at risk of reinjury or adverse long-term outcomes may be identified at early timepoints, enabling risk mitigating interventions. However, a limitation of this approach is the need to manually segment regions of interest from an image volume to obtain T₂* maps of the tissue.

Manual image segmentation is a slow process that is subject to inter- and intra- segmenter variability, even when performed by experienced segmenters. During the segmentation task, the segmenter must proceed slice by slice through the image stack and manually outline the region of interest (e.g., ACL). Automation of image segmentation therefore has the potential to increase research productivity and lower a barrier to the wider adoption of T₂* relaxometry to quantify ligament and tendon integrity in both research and clinical settings.

In two previous studies,^{12; 13} it was demonstrated that a deep learning model could segment intact, reconstructed, and repaired anterior cruciate ligaments (ACL) with greater speed and reliability than manual segmentation. These studies trained, validated, and tested the segmentation algorithm using MR images created using the Constructive Interference in Steady State (CISS) sequence, which has been used to evaluate the size and normalized signal intensity of the ACL.^{8; 14; 15} However, the model has not yet been applied to T₂* relaxometry, which is a more commonly used approach that has two theoretical advantages compared to the CISS sequence. First, T₂* relaxation time is directly related to collagen organization.¹⁶ Second, it is resistant to inter-scanner bias when all acquisition parameters are held constant.¹⁷ However, this sequence poses additional challenges for segmentation due to its lower in-plane resolution and contrast compared to the CISS sequence. This challenge could potentially be mitigated by using a model that was pretrained on a large dataset of CISS MR images, rather than training a model from scratch on the T₂* images.

The objective of the current study was to apply an automated segmentation approach to T₂* relaxometry scans of healing ACLs following a surgical restoration procedure via transfer learning of a segmentation model previously trained on the CISS scans. It was hypothesized that the transferred segmentation model would attain segmentation performance metrics (e.g., Dice coefficient, sensitivity, and precision) on T₂* relaxometry scans of healing ACLs that were not significantly different from their original performance on CISS scans of healing ACLs.¹² In addition, it was hypothesized that structural features extracted from the T₂* automatic segmentations (e.g., median T₂* relaxation time, volume, sub-volume proportions, and cross-sectional area (CSA), which have been used to assess ligament healing,^{2; 3; 9; 18; 19} would not be significantly different from T₂* ground truth manual segmentation. Finally, it was hypothesized that the automatic T₂* segmentation model would perform as well as independent and repeated ground truth (retest) manual segmentation relative to the original T₂* ground truth manual segmentation.

METHODS

Data

De-identified T₂* relaxometry scans from the IRB approved BEAR III Trial for Bridge-Enhanced ACL Restoration (NCT03348995) were made available to the research team for this analysis. Patients in this trial underwent a novel ACL surgical procedure to restore the injured ACL using a suture repair with an extracellular matrix-based implant to stimulate healing.^{15; 20-22} All patients granted their informed consent prior to participating in the IRB approved trial. Only de-identified post-surgical MR images and no other trial data were used in the present study.

All T₂* relaxometry scans were acquired on either a 3T Tim Trio (Siemens, Erlangen, Germany) or a 3T Prisma (Siemens), both with a 15-channel transmit/receive knee coil (Siemens), at 9- and 24- months post-surgery. The Tim Trio T₂* relaxometry scan was acquired using the following specifications: FA=12°; TR=29ms; TE₁=3.4ms; TE₂=6.9ms; TE₃=11.2ms; TE₄=15.6ms; TE₅=20ms; TE₆=24.4ms; FOV=160mm; 384x384 acquisition matrix with voxel size 0.4167mm x 0.4167mm x 0.8mm. The Prisma T₂* relaxometry scan was acquired using the following specifications: FA=12°; TR=29ms; TE₁=2.5ms; TE₂=6.9ms; TE₃=11.2ms; TE₄=15.6ms; TE₅=20ms; TE₆=24.4ms; FOV=160mm; 384x384 acquisition matrix with voxel size 0.4167mm x 0.4167mm x 0.8mm. Note that due to hardware limitations of the particular scanners used, the shortest echo time (TE₁) was not equivalent on both systems. Images were available for 54 knee scans from 45 patients in total (Tim Trio: n=36, Prisma: n=19; 9-month: n=45, 24-month: n=9).

Image Preprocessing

Due to the difference in the shortest echo time parameter (TE₁) between magnets, the signal intensity of the images acquired on the Tim Trio, which had the longer TE₁, were rescaled to be harmonized with the Prisma as previously described.¹⁷ Ground truth manual segmentations of the ACL were performed by one segmenter with 5 years of experience using commercial software (Mimics Research 19.0, Materialise). Prior to model training, each echo was center cropped to a 256x256x20 voxel volume to reduce computation time. The cropping boundaries were sufficiently large to contain the entire ACL. Z-scoring was performed to standardize the signal intensity across all echoes of each image volume.

The imaging data were randomly split into training (70%; 38 scans; 760 slices per echo) and validation (30%; 16 scans; 320 slices per echo) sets. The split was stratified by subject and scanner to prevent data leakage between sets and to ensure proportional representation of data from both scanners in each set.

Model Training

Transfer learning was used to retrain a previous model that performed well for segmentation of the intact and surgically treated ACL from images obtained using the CISS sequence.^{12; 13} The CISS-based segmentation model utilizes a modified 2D U-Net architecture, with the addition of batch normalization and dropout layers, and Dice coefficient loss (Eq. 1; TP=true positive, FP=false positive, FN=false negative). Dice coefficient loss is distinguished from Dice coefficient by the addition of Laplacian smoothing and a negative sign such that it is both continuously differentiable and can be minimized while the Dice coefficient is maximized.

$$Dicecoefficien{t}_{LOSS}=-\frac{2TP+1}{FP+2TP+FN+1}$$ [1]

In the first phase of model transfer, the weights of early model layers were frozen for 5 epochs and the model was trained with a learning rate of 1e-2. Early weight updates can cause large swings in model performance initially; by freezing the early layers, model stability near the target task (ACL segmentation) is maintained. The number of frozen layers was treated as a hyperparameter in the model transfer process to be tuned. In the second phase, the early layers were unfrozen to allow weight updates, and the learning rate was decreased to 1e-3. During phase two, the learning rate was allowed to decrease logarithmically when validation loss plateaued for 5 epochs, to a minimum learning rate of 1e-5. Training was arrested after 10 epochs of no improvement in the validation loss to prevent overfitting.

The model was trained on multiple echoes as a form of natural image augmentation. The number of echoes used was treated as an hyperparameter. Including echoes 1-3 was not found to impact model performance so they were ultimately excluded. Training on images from echoes 4-6 resulted in optimal performance and increased the number of slices used for training and validation to 2280 and 960, respectively.

All training was conducted using a cloud computing service (Google Cloud Platform, Alphabet Inc., Mountain View, CA). A virtual machine was configured with a n1-standard-8 CPU and NVIDIA Tesla K80 GPU (NVIDIA Corporation, Santa Clara, CA). TensorFlow 2.1.0²³ was used for coding the model.

Performance

Segmentation performance of the T₂* model was measured via Dice coefficient (Eq. 2), sensitivity (Eq. 3), and precision (Eq. 4). These metrics were compared to those from the previously reported segmentation performance using the CISS sequence with Mann-Whitney U tests (α=.05).¹² This non-parametric test was selected because segmentation metrics were not normally distributed.

$$Dicecoefficient=\frac{2TP}{FP+2TP+FN}$$ [2]

$$Sensitivity=\frac{TP}{TP+FN}$$ [3]

$$Precision=\frac{TP}{TP+FP}$$ [4]

Structural variables (i.e., ACL median T₂* relaxation time, volume, sub-volume proportion 1 and sub-volume proportion 4, and CSA) for model segmentations and ground truth were also compared. These additional measures were selected because they have been used in downstream structural property prediction models for the ACL.^{2; 3; 9; 18; 19} The T₂* relaxation time of the ACL was calculated by fitting a voxel-wise exponential decay curve to voxels contained in the ACL segmentation mask (Eq.5; M(TE)=magnetization at given echo time, M0=initial magnetization, TE=echo time, T₂*=T₂* relaxation time, DC=direct current offset). ACL volume was calculated as the sum of voxels in the ACL mask multiplied by the voxel volume. Sub-volume proportions were calculated by binning ACL voxels by relaxation time (proportion 1: 0<x≤12.5ms, proportion 4: x>37.5ms) and then dividing the sub-volume by total ligament volume.^{2; 3; 9} CSA was calculated at 1% intervals along the length of the ACL segmentation, and the mean CSA was then extracted for comparison.²⁴ Statistical differences on these measures were analyzed with Wilcoxon signed-rank tests between T₂* ground truth manual segmentation and model predictions on the validation set (α=.05).

$$M(TE)=M_0{e}^{-\frac{TE}{T_2^{\ast }}}+DC$$ [5]

Benchmarking

To benchmark the performance of the model relative to manual segmentation, additional comparisons between repeated segmentation (retest) by the ground truth segmenter (>5 years of experience), independent segmentations by two additional segmenters (A: <1 year experience, B: >4 years of experience), and the model were performed. Each person segmented the ACL from a randomly selected anonymous subset of 8 scans in the validation set. The same segmentation performance (i.e., Dice coefficient, precision, sensitivity) and primary structural metrics (median T₂* relaxation time, volume) were evaluated between manual segmenters and the model versus ground truth. Repeated measures analysis of variance (RANOVA) with Tukey testing was used to assess differences between retest segmentation, the independent manual segmenters, the model, and the ground truth segmentation (α=.05). All statistical analyses were performed in Python with the SciPy Stats and Statsmodels packages.^25;26

RESULTS

Segmentation Performance

The training time for the transferred model was 171.4 minutes. Dice coefficient, precision, and sensitivity of the model relative to ground truth were moderately high when transferring the model to the T₂* relaxometry sequence (Figure 1; median Dice coefficient=.76, precision=.76, sensitivity=.79). This is a small decrease from the previously reported model performance on the CISS healing ACL scans (median Dice coefficient=.80, precision=.79, sensitivity=.82).¹² However, the decrease between the prior performance on the CISS scans and current T₂* performance was not statistically significant (Dice coefficient p=.40, precision p=.35, sensitivity p=.49). Example T₂* segmentations are shown in Figure 2.

Figure 1.

Violin plots of T₂* segmentation performance metrics of the transferred automatic segmentation model on the validation set.

Figure 2.

Example sagittal slice segmentations of the healing ACL from the T₂* sequence validation set (MR Image: 5^th echo sagittal slice, Ground Truth: manual segmentation, Prediction: automated segmentation, Contours Overlay: contours of ground truth [blue] and prediction [red]).

Structural Measures

All structural measures were comparable between ground truth manual segmentation and automatic segmentation on the validation set, with no significant differences detected (Figure 3). Mean pairwise percent differences were for median T₂*=−1.42% (p=.60), volume=1.06% (p=.94), sub-volume proportion 1=.19% (p=.82), sub-volume proportion 4=5.06% (p=.50), and CSA=0.50% (p=.94).

Figure 3.

Violin plots of the structural measures extracted from manual ground truth segmentation and automated segmentation.

Benchmarking

On the randomly selected subset of T₂* images, segmenters A (<1 year of experience) and B (>4 years of experience) showed lower Dice coefficients relative to ground truth than retest segmentation by the ground truth segmenter (>5 years of experience) and automatic segmentation (Figure 4; A=.53, B=.62, retest=.80, automatic=.76). This decrease was significant between segmenter A and automatic segmentation (p=.007) and between segmenter A and retest segmentation (p=.002).

Figure 4.

Segmentation performance of independent manual segmenters (A and B), retest segmentation by the ground truth segmenter (Retest), and the automatic segmentation model (Auto)(p<.05=*, p<.005=**).

Segmenter A showed lower precision than segmenter B, retest, and automatic segmentation (A=.62, B=.84, retest=.79, automatic=.76). This difference was significant between segmenter A and all other segmenters (A vs B p=.001, A vs retest p=.021, A vs automatic p=.005).

Both segmenters A and B showed lower sensitivity than retest and automatic segmentation (A=.60, B=.49, retest=.80, automatic=.71). This difference was only significant between A, B and retest ground truth segmentation (A vs retest p=.023, B vs retest p=.021).

These decreases in segmentation performance did not translate to any significant differences in median T₂* relaxation time or segmentation volume (Figure 5). Relative to ground truth, the mean pairwise percent differences for median T₂* were A=10.6%, B=−2.1%, retest=2.4%, automatic=−2.9%. For volume, the differences were A=−12%, B=−41.2%, retest=.5%, and automatic=−12%.

Figure 5.

Primary structural measures extracted from independent manual segmenters (A and B), retest segmentation by the ground truth segmenter (Retest), the automatic segmentation model (Auto), and the original ground truth manual segmentation (Ground).

DISCUSSION

The automatic segmentation model demonstrated slightly lower segmentation performance on the T₂* relaxometry scans compared to CISS scans of the healing ACL, though the difference was not statistically significant. Furthermore, the small decrease in performance did not affect the fidelity of structural measures (median T₂* relaxation time, volume, sub-volume proportions, and CSA) extracted from the automatic segmentations of the ACL relative to ground truth manual segmentation. This result is crucial to preserve the robustness of mechanical property prediction models that utilize these structural measures to assess the integrity of the healing ACL.^{2; 19}

When assessing the performance of the automatic segmentation model relative to manual segmentation on a randomly selected subset of validation scans, the automatic segmentation model performed as well as repeated segmentation by the ground truth segmenter, and better than the two independent, less experienced segmenters. This finding is consistent with a previous study that determined the segmentation model performed as well as retest segmentation on CISS scans of intact ACLs.¹³ The present analysis also suggests that the aforementioned challenges of T₂* segmentation (e.g., decreased in-plane resolution and contrast) had a greater detrimental effect on independent manual segmentation than on automatic segmentation. It can also be observed that experience has a significant impact on manual segmentation, with segmenter A (<1 year experience) demonstrating lower segmentation performance metrics than segmenter B (>4 years’ experience) or retest segmentation by the ground truth segmenter (>5 years’ experience). This experience curve is a limitation of manual segmentation that does not affect automatic segmentation.

These results fit in with the increasing application of deep learning for MR image segmentation of the knee, where it has been found that deep learning has been able to achieve consistent, accurate segmentation of structures of interest.^{12; 13; 27-32} However, most automated segmentation algorithms developed for the knee have focused on the cartilage, meniscus, and bones.^27-32 This may be due, in part, to some of the unique challenges associated with ACL segmentation. For example, previous semi-automated segmentation approaches faced difficulties segmenting the origin and insertion of the ACL.^{33; 34} Furthermore, for the surgically treated ACL, segmenting only the intra-articular portion of the ligament/graft, and ignoring tissue within the bone tunnels, presents an additional challenge. In the current study and in previous validations of the model for CISS segmentation of the ACL, the proposed model addressed these challenges and the deep learning-based approach yielded greater segmentation performance than other ACL segmentation methods.^{12; 13; 33; 34}

There are several limitations of this study. First, due to the limited sample size, a separate test set was not available. It was found that a smaller split size to withhold an independent test set would not have been sufficiently powered to detect statistically significant differences. The smaller sample size may also have negatively impacted the performance of the final segmentation model. The smaller sample size limitation was mitigated by using a model that was pretrained on a larger corpus of MR images (4920 slices) using the CISS sequence, and by training on multiple echo times as a form of natural image augmentation. Both transfer learning and image augmentation significantly reduce the quantity of data needed to train a deep learning model.^{35; 36} The model may also be retrained and updated as more data become available. In addition, it is possible that the model performed as well as retest segmentation by the ground truth segmenter because it was overfitting that particular segmenter, who was the source of the T₂* training data. However, the pretrained CISS segmentation model had been trained on data from a different ground truth segmenter. Also, given that downstream structural property prediction models that would rely on these segmentations were trained on data from the same segmenter, this scenario could in fact support the robustness of these downstream models at scale. Finally, the results are only applicable to the sequences evaluated, and healing ACLs. Application to other sequences, grafts, or other soft tissues may require further transfer learning.

In conclusion, the automated segmentation model was capable of segmenting healing ACLs from T₂* relaxometry scans with performance metrics that were not significantly different from the CISS scans. Furthermore, structural measures such as median T₂* relaxation time, volume, sub-volume proportions, and CSA were not significantly different from manual segmentation, indicating that automatic segmentation of T₂* scans is compatible with existing mechanical property prediction models of the ACL. Finally, automatic segmentation outperformed manual segmentation by independent segmenters and performed as well as retest segmentation by the ground truth segmenter. By automating segmentation of the ACL, T₂* relaxometry becomes more practical for use in both clinical and research contexts as variability and processing time are reduced, and the need for training clinicians or researchers in manual segmentation is eliminated.