Abstract
Numerous studies have presented fully automated techniques for assessing structural osteoarthritis (OA) progression, with recent work increasingly relying on deep learning (DL)-based methods. The objective of this narrative review was to summarize findings from studies comparing the validity of fully automated methods for assessing progression in (peri-) articular joint tissues with reference measures (e.g., manual segmentation) in clinical OA models. A literature search in PubMed and arXiv.org identified 873 studies. Of these, nine evaluated the clinical validity of fully automated longitudinal measures for assessing progression. Five met the inclusion criteria by comparing sensitivity to differences in change in clinically defined cohorts between fully automated vs. reference assessments, and four reported at least the sensitivity to change for both methods. One of the studies evaluated longitudinal change in radiographic joint space width, five change in MRI-based cartilage thickness, two change in cartilage composition, and one change in thigh muscle and adipose tissue cross-sectional areas. Most of the studies were based on DL methods and relied on data from the Osteoarthritis Initiative (OAI). The included studies reported similar or greater sensitivity to change and similar discriminative power for detecting differences in change between clinically defined groups compared with reference measurements. Therefore, the techniques validated in these studies appear suitable for assessing structural progression provided that key requirements are met, including consistent imaging protocols, scanner settings, and data quality.
Keywords: Osteoarthritis, Progression, Clinical validation, Imaging, Deep learning, Fully automated analysis
1. Introduction
Numerous measurement techniques and scoring systems have been established for imaging-based assessment of joint tissue status and longitudinal OA progression over the last decades [[1], [2], [3]]. Quantitative measurement techniques are predominantly used for assessing joint space width (JSW) from x-ray or for measuring cartilage morphology or relaxation times from MRI [1,2,4,5]. Due to their sensitivity to change [6], quantitative measures are frequently used to assess structural progression in observational studies and to evaluate the efficacy of interventions such as disease modifying OA drug candidates [[7], [8], [9], [10], [11], [12], [13], [14], [15], [16]]. Quantitative measurements have also been used to analyze other (peri-) articular tissues, including the meniscus [[17], [18], [19]], infra-patellar fat-pad [[20], [21], [22]], bone marrow lesions [23,24], thigh muscles [25,26], and adipose tissue [27,28]. In contrast to single tissue-focused segmentation approaches, semi-quantitative scoring systems such as the radiographic OARSI atlas [29] and the MRI-based MOAKS [30] instrument provide a more wholistic assessment with subscales for various joint tissues and pathologies. Semi-quantitative assessments are commonly used to describe baseline disease severity, to study progression and to evaluate treatment efficacy [[31], [32], [33]], but they have been suggested to be less sensitive to change than quantitative assessments [34]. This is likely, because not every worsening leads to a higher score of the respective grading scale. Adding within-grade scoring has therefore been proposed to improve the sensitivity of semi-quantitative assessments to longitudinal change [35].
For decades, most studies investigating longitudinal changes in (peri-) articular tissues have relied on manual expert assessments by radiologists or trained readers, which are costly in both time and resources. Semi-automated image analysis techniques have been used for articular tissue segmentation since 1994 [36] and numerous studies have since proposed semi- or fully automated tissue segmentation methods [37]. Machine learning-based image analysis techniques, particular deep learning (DL), have advanced substantially over the past decade and are now increasingly used in OA research [37]. Most DL-based image segmentation techniques currently rely on convolutional neural networks such as the U-Net architecture [38], a supervised learning method trained on labeled data, which has been adapted and used in several studies for automatically segmenting joint tissues [37,39,40].
Clinical validation aims to show that measures are clinically meaningful, either by demonstrating associations with relevant clinical outcomes (e.g., pain, function, imaging surrogates) or by reproducing known, clinically established patterns of structural change observed with validated reference techniques in clinical models. Such validation is strengthened when the magnitude and nature of the association are compared with established reference standards, providing evidence that the new measure performs comparably to, or better than, existing clinically validated methods.
Although many studies have proposed fully automated methodologies for assessing structural OA progression in recent years, little is known about their clinical validity. The objective of this narrative review was therefore to summarize the findings from studies comparing the validity of fully automated methods for assessing progression in (peri-) articular joint tissues with reference measures in clinical OA models. In addition, this review summarizes the sensitivity to progression observed with both the fully automated and reference methods.
2. Methods
We performed a literature search in Pubmed (query: “((((osteoarthritis) AND ((automated) Odds Ratios [OR] (automatic))) AND ((x-ray) or (MRI) OR (radiographic) OR (JSW) or (image) or (imaging))) NOT (robotic))”) and in arXiv.org (query: “osteoarthritis”) that identified 873 studies (n = 702 in Pubmed, n = 171 in arXiv.org). References of several articles were screened for additional studies.
This review primarily aimed to include studies comparing the clinical validity of fully automated longitudinal measures for assessing progression with reference measures in models that had defined clinical outcomes or clinically characterized cohorts. Because only few articles matched this criterion, and because sensitivity to change is a prerequisite for detecting differences in change between clinically defined cohorts, we additionally included studies that reported the sensitivity to change for both fully automated and reference measures in such cohorts, even if they did not compare the sensitivity to differences in change between techniques.
Based on the identified studies, we also provide a very brief overview over the most relevant fully automated analysis techniques used to study progression and the tissues, for which such techniques have been presented.
3. Results
Of the 873 studies, 794 were excluded during abstract screening because they were either unrelated to the objective of this review or duplicates. After full text review the of the remaining 79 studies, another 70 studies were excluded because they did not use a fully automated analysis technique, did not focus on structural progression, did not compare automatically measured progression to an external reference or between groups, or aimed to predict rather than measure progression. Four of the remaining nine studies compared the clinical validity of fully automated longitudinal measures for assessing progression with reference measures and reported effect sizes reflecting the strength of associations with clinical outcomes for both methods (Table 1). One study compared the clinical validity of a fully automated method for measuring the efficacy of a structural treatment intervention with that of a previously validated reference measure but did not report effect sizes (Table 2). Finally, four articles compared only the sensitivity to change between fully automated and reference measures in clinically defined cohorts (Table 2).
Table 1.
List of included studies that compared the sensitivity to differences in change between fully automated methods vs. reference measures.
| Author | Imaging | Clinical model | Segmentation method | Automated measures and reference method | Sensitivity to change (standardized response mean (SRM) with 95 % confidence intervals (CI)) | Sensitivity to differences in change between clinically defined groups |
|---|---|---|---|---|---|---|
| Panfilov et al. [76] | 3D DESS MRI from the OAI; acquired at 3T | MRI-based cartilage thickness change in 567 knees from the OAI FNIH model (4 groups of knees with and without progression in JSW and pain) | 2D U-Net with an ImageNet-pretrained VGG19 encoder |
Comparison of 12- and 24-month change in various cartilage subregions including the medial femorotibial compartment (MFTC), the medial femur (MF) and the medial tibia (MT) and in medial and lateral meniscus (MM/LM) volume between progressor and non-progressor knees vs. cartilage thickness from manual segmentation and vs. cartilage and MM/LM volume from an automated technique | Sensitivity to change not reported | Odds ratios for 24-month change (U-Net vs. reference): No vs. JSW progression: MFTC: 3.51 vs. 4.37 MF: 2.39 vs. 1.71 MT: 4.00 vs. 0.96 MM: 1.79 vs. 2.22 LM: 1.62 vs. 1.41 No and pain vs. JSW and JSW + pain progression: MFTC: 3.24 vs. 4.59 MF: 1.91 vs. 1.71 MT: 2.98 vs. 0.96 MM: 1.68 vs. 2.22 LM: 1.69 vs. 1.41 |
| Eckstein et al. [77] | 3D DESS MRI from the OAI; acquired at 3T | MRI-based cartilage thickness change in 597 knees from the OAI FNIH model (4 groups of knees with and without progression in JSW and pain) | 2D U-Nets trained on knees with radiographic OA (ROA; U-NetROA), healthy knees, and both ROA and healthy knees | Comparison of 24-month change in MFTC cartilage thickness between progressor and non-progressor knees by progression status vs. cartilage thickness from manual segmentation | SRM for 24-month change in MFTC: JSW and pain progression: U-NetROA: 0.72 (−0.86, −0.57) Manual: 0.74 (−0.85, −0.63) JSW progression only: U-NetROA: 0.60 (−0.55, −0.44) Manual: 0.73 (−0.88, −0.55) Pain progression only: U-NetROA: 0.14 (−0.32, 0.07) Manual: 0.07 (−0.28, 0.13) No progression: U-NetROA: 0.27 (−0.39, −0.12) Manual: 0.21 (−0.36, −0.07) (see Ref. [77] for SRM in other regions) |
Cohen’s d (95 % CIs) for differences in 24-month change in MFTC: No vs. JSW + pain progression: U-NetROA: 0.68 (−0.88, −0.47) Manual: 0.85 (−1.05, −0.64) |
| Eckstein et al. [78] | 3D DESS MRI and 3D fast low angle shot (FLASH) MRI from the OAI; acquired at 3T | MRI-based cartilage thickness change in 322 knees from the OAI FNIH model (4 groups of knees with and without progression in JSW and pain) that had both sagittal DESS and coronal FLASH MRI available | 2D U-Nets trained on DESS (U-NetDESS) or FLASH (U-NetFLASH) MRI from knees with ROA | Comparison of 24-month change in medial femorotibial compartment (MFTC) cartilage thickness between progressor and non-progressor knees stratified by progression status vs. cartilage thickness from manual segmentation | SRM for 24-month change in MFTC: JSW and pain progression: U-NetDESS: 0.74 (−0.95, −0.56) U-NetFLASH: 0.79 (−1.06, −0.55) Manual: 0.66 (−0.82, −0.51) JSW progression only: U-NetDESS: 0.74 (−0.95, −0.54) U-NetFLASH: 0.84 (−1.12, −0.63) Manual: 0.88 (−1.14, −0.66) Pain progression only: U-NetDESS: 0.06 (−0.20, 0.31) U-NetFLASH: 0.01 (−0.33, 0.23) Manual: 0.05 (−0.32, 0.21) No progression: U-NetDESS: 0.30 (−0.47, −0.12) U-NetFLASH: 0.28 (−0.47, −0.09) Manual: 0.08 (−0.26, 0.13) (see Ref. [78] for SRM in other regions) |
Cohen’s d (95 % CIs) for differences in 24-month change in MFTC: No and pain progression vs. JSW and JSW + pain progression: U-NetDESS: 0.70 (−0.93, −0.48) U-NetFLASH: 0.82 (−1.05, −0.59) Manual: 0.87 (−1.10, −0.64) |
| Wirth et al. [80] | 2D multi-echo spin-echo (MESE) MRI from the OAI; acquired at 3T | MRI-based laminar cartilage T2 change in KLG 0 knees from the OAI with joint space narrowing (JSN) in contralateral knees (CL-JSN) vs KLG 0 knees without ROA in contralateral knees (CL-noROA) | 2D U-Nets trained on all echoes of multi-echo spin-echo MRIs | Comparison of 36-month change in laminar T2 in MFTC, LFTC, and TFTJ between CL-JSN and CL-noROA knees vs. laminar T2 from manual segmentation | SRM for 36-month change, U-Net vs. manual reference#: Deep layer, CL-JSN: TFTJ: 0.47 vs. 0.48 MFTC: 0.44 vs. 0.41 LFTC: 0.38 vs. 0.40 Superficial layer, CL-JSN: TFTJ: 0.33 vs. 0.47 MFTC: 0.41 vs. 0.52 LFTC: 0.16 vs. 0.23 Deep layer, CL-noROA: TFTJ: 0.02 vs. 0.06 MFTC: 0.00 vs. 0.17 LFTC: 0.33 vs. −0.04 Superficial layer, CL-noROA: TFTJ: 0.38 vs. 0.60 MFTC: 0.47 vs. 0.59 LFTC: 0.07 vs. 0.26 |
Cohen’s d (95 % CIs) for differences in 36-month change between CL-JSN and CL-noROA knees: Deep layer: TFTJU-Net: 0.41 (0.08, 0.74) TFTJref: 0.28 (−0.04, 0.60) MFTCU-Net: 0.31 (−0.02, 0.63) MFTCref: 0.02 (−0.12, 0.51) LFTCU-Net: 0.41 (0.08, 0.74) LFTCref: 0.27 (−0.06, 0.58) Superficial layer: TFTJU-Net: 0.54 (0.20, 0.87) TFTJref: 0.48 (0.15, 0.81) MFTCU-Net: 0.54 (0.20, 0.87) MFTCref: 0.40 (0.07, 0.72) LFTCU-Net: 0.40 (0.07, 0.72) LFTCref: 0.41 (0.08, 0.73) |
#: 95 % CIs not available because SRM was computed from mean change and standard deviation of change or because 95 % CIs were not reported.
Table 2.
List of included studies that reported the sensitivity to change for both fully automated methods and reference measures in clinical models but did not compare their sensitivity to differences in change.
| Author | Imaging | Clinical model | Segmentation method | Automated measures and reference method | Sensitivity to change (standardized response mean (SRM) with 95 % confidence intervals (CI)) |
|---|---|---|---|---|---|
| Paixao et al. [69] | Fixed-flexion radiographs provided by the osteoarthritis initiative (OAI) | Radiographic joint space narrowing (JSN) and 12-month change in JSN in 3129 OAI participants | AI-based (no details provided) | 12-month change in radiographic joint space width (JSW) at 4 standardized locations (M0 – M3) vs. semi-automated JSW at fixed location (0.25). | SRM for 12-month change: JSN 0: M0: 0.01 (−0.04, 0.02) M1: 0.03 (−0.06, 0.00) M2: 0.04 (−0.07, −0.01) M3: 0.04 (−0.07, −0.01) JSW(0.25): 0.07 (−0.10, −0.04) JSN 1: M0: 0.08 (−0.13, −0.03) M1: 0.10 (−0.14, −0.05) M2: 0.09 (−0.14, −0.05) M3: 0.03 (−0.08, −0.01) JSW(0.25): 0.18 (−0.18 0.09) JSN 2: M0: 0.16 (−0.24, −0.10) M1: 0.19 (−0.24, −0.13) M2: 0.15 (−0.21, 0.10) M3: 0.07 (−0.13, −0.01) JSW(0.25): 0.29 (−0.34, −0.23) JSN 3: M0: 0.05 (−0.18, 0.06) M1: 0.07 (−0.20, 0.04) M2: 0.09 (−0.24, 0.03) M3: 0.07 (−0.20, 0.03) JSW(0.25): 0.26 (−0.40, −0.14) (see Ref. [69] for SRMs stratified by change in JSN) |
| Bowes et al. [75] | 3D double echo at steady state (DESS) MRI from the OAI; acquired at 3T | MRI-based cartilage thickness change in 582 knees from the OAI FNIH model (4 groups of knees with and without progression in JSW and pain) | Active appearance models (AAM) | 12- And 24-month change in central medial femur (cMF) and central medial tibia (cMT) cartilage thickness across all knees and stratified by progression status vs. cartilage thickness from manual segmentation in similar regions of interest | SRM for 24-month change: JSW and pain progression: cMFAAM: 0.96 (−1.09, −0.82) cMFRef: 0.73 (−0.83, −0.62) cMTAAM: 0.72 (−0.82, −0.60) cMTRef: 0.60 (−0.70, −0.47) JSW progression only: cMFAAM: 0.90 (−1.05, −0.73) cMFRef: 0.74 (−0.89, −0.59) cMTAAM: 0.76 (−0.91, −0.58) cMTRef: 0.70 (−0.85, −0.52) Pain progression only: cMFAAM: 0.28 (−0.43, −0.06) cMFRef: 0.12 (−0.24, 0.08) cMTAAM: 0.26 (−0.42, −0.06) cMTRef: 0.16 (−0.26, 0.07) No progression: cMFAAM: 0.52 (−0.62, −0.40) cMFRef: 0.13 (−0.27, 0.01) cMTAAM: 0.42 (−0.52, −0.30) cMTRef: 0.29 (−0.44, −0.15) (see Ref. [75] for 12-month SRM and for SRM across all knees) |
| Eckstein et al. [81] | 2D MESE MRI acquired at 1.5T at a single site | MRI-based laminar cartilage T2 change in 55 knees after anterior cruciate ligament (ACL) injury and 16 knees from healthy controls. | 2D U-Nets trained on all echoes of multi-echo spin-echo MRIs | Comparison of between-group differences in 12-month change in central lateral femur (cLF) | SRM for 12-month change in cLF, U-Net vs. manual reference#: Deep layer: 0.58 vs. 0.52 |
| Brett et al. [9] | 3D spoiled gradient echo (SPGR) MRI acquired at 1.5T and 3T at multiple sites | MRI-based cartilage thickness change in knees treated with Sprifermin 100 μg every 6 months (n = 101) and in knees treated with placebo (n = 96) | Active appearance models (AAM) | Comparison of 24-month change in medial/lateral femorotibial compartment (MFTC/LFTC) and total joint cartilage thickness (TFTJ) vs. cartilage thickness in same regions from manual segmentation. | SRM for 24-month change, AAM vs. manual reference#: Sprifermin 100 μg: TFTJ: 0.45 vs. 0.43 MFTC: 0.20 vs. 0.25 LFTC: 0.58 vs. 0.67 Placebo: TFTJ: 0.50 vs. −0.29 MFTC: 0.29 vs. −0.25 LFTC: 0.40 vs −0.20 |
| Kemnitz et al. [60] | 2D T1-weighted spin-echo MRI from the OAI; acquired at 3T | MRI-based change in thigh MRI cross-sectional areas (CSA) in women and men from OAI with severe weight gain (n = 51) or loss (n = 52) | 2D U-Nets | Comparison of 24-month change in intermuscular fat (IMF), subcutaneous fat (SCF), extensor (EXT) and flexor (FLX) CSAs within groups between methods | SRM for 24-month change, U-Net vs. manual reference: Women, weight gain: IMF: 0.83 vs. 0.51 SCF: 0.92 vs. 0.95 EXT: 0.01 vs. 0.05 FLX: 0.75 vs. 0.69 Men, weight gain: IMF: 1.38 vs. 0.80 SCF: 1.31 vs. 1.39 EXT: 0.35 vs. 0.39 FLX: 0.33 vs. 0.49 Women, weight loss: IMF: 0.82 vs. −0.64 SCF: 1.25 vs. −1.32 EXT: 1.23 vs. −1.13 FLX: 0.94 vs. −0.91 Men, weight loss: IMF: 1.30 vs. −1.34 SCF: 1.04 vs. −1.01 EXT: 1.08 vs. −1.06 FLX: 0.83 vs. −1.06 |
#: 95 % CIs not available because SRM was computed from mean change and standard deviation of change or because 95 % CIs were not reported.
3.1. Fully automated analysis techniques for assessing progression in (peri-) articular tissues
Early studies proposing fully automated techniques for analyzing (peri-)articular tissues relied on techniques such as active shape [41] or appearance models [42], graph-based methods [43], voxel-classification [44], fully automated thresholding [45,46], ray-casting [47], or registration [48]. More recently, many studies have transitioned towards DL-based techniques such as the U-Net. These DL-based segmentation techniques also contributed most data points to recent segmentation challenges, which demonstrated high consistency with reference segmentations [49,50].
Fully automated segmentation techniques have been presented for most of the (peri-)articular tissues. Most studies focused on cartilage [37], but fully automated methods have also been proposed for segmenting bones from x-ray [51] or MRI [52,53], bone-marrow lesions [54], menisci [55], effusion- and Hoffa-synovitis [[56], [57], [58]], ligaments [59], and thigh muscles and adipose tissue [[60], [61], [62]]. Most methods were developed for the automated segmentation of tissues from MRI, including modern MRI protocols such as qDESS MRI, which enables analysis of both cartilage morphology and T2 relaxation time from a single scan [[63], [64], [65], [66]].
3.2. Radiographic progression
Radiographic JSW has traditionally been used to indirectly assess cartilage loss in OA studies. Studies using semi-automated JSW measurements have demonstrated their sensitivity to change [67] and association with clinical outcomes [68]. Automated techniques have also been used to assess other radiography-based measures, such as subchondral bone texture, but none of these studies met the inclusion criteria.
The one included study compared the sensitivity to change of radiographic JSW measured using an AI-based, FDA-approved medical device product (IB Lab GmbH, Vienna, Austria) with publicly available JSW measures form the Osteoarthritis Initiative (OAI) obtained using a semi-automated technique [69]. The comparison was stratified by baseline radiographic joint space narrowing (JSN) scores and by changes in JSN scores. No details were provided about the automated JSW measurement algorithm. The AI-based technique measured JSW standardized to tibial width at four positions in the medial compartment and compared their sensitivity to change with that of semi-automated JSW measured at one fixed position, previously reported as most sensitive to OA progression [24]. The study found a comparable pattern of the magnitude of change and sensitivity to change, measured using the standardized response mean, for both techniques (Table 2). Additionally, AI-based measures were somewhat more sensitive to change and better separated between changes in knees with different JSN grades than semi-automated JSW measures (Table 2) [69]. The same AI-based JSW measurements were also more recently found to be sensitive to structural improvements and to be associated with synovial fluid biomarker level changes and improvements in pain in OA patients treated with surgical knee joint distraction [70].
3.3. MRI-based progression
Only few studies presenting fully automated MRI-based techniques could be included as most assessed only technical validity, did not report longitudinal changes, or focused on predicting outcomes from baseline image assessments. Of the eight included studies, five examined segmentation-based quantification of cartilage morphology, two analyzed cartilage T2 relaxation time, and one investigated thigh muscle and adipose tissue (Table 1, Table 2). One study reporting data on cartilage additionally reported data on meniscal volume [71].
3.3.1. Cartilage and meniscus
Four studies evaluating fully automated, segmentation-based quantification techniques for cartilage morphology used data from the OAI FNIH cohort, for which manual cartilage thickness measurements [72], double echo at steady state (DESS) MRIs [73], and other assessments are publicly available [33,74]. The OAI FNIH cohort comprises four progression groups: a) both radiographic and pain progression, b) radiographic progression only, c) pain progression only, and d) neither radiographic nor pain progression. Bowes et al. [75] used a fully automated segmentation pipeline centered around an active appearance model (AAM) to segment bone and cartilage from MRI and compared 12- and 24-month cartilage thickness change in femorotibial cartilages stratified by progression group with cartilage measures from manual reference segmentations [72]. However, this study did not compare the discriminative power of the two segmentation techniques for detecting differences in change between groups. The authors observed very strong correlations between AAM-based and manual cartilage thickness measures (r = 0.87 for tibial and 0.95 for femoral cartilage) and found AAM-based measures to show a greater sensitivity to change than manual measures (Table 2) [75]. Interestingly, this also applied to the group of knees selected to show neither radiographic nor pain progression (Table 2) [75].
The three studies by Panfilov et al. [76], Eckstein et al. [77], and Eckstein et al. [78] used DL-based cartilage segmentation pipelines and compared the discriminative power of the fully automated segmentation techniques for detecting differences in change between OAI FNIH progression groups with that obtained from reference measures using odds ratios (OR, [76]) or Cohen’s d [77,78] as effect size measures. The segmentation pipelines in these three studies relied on the U-Net architecture, which was modified and extended for the study by Panfilov [76] and combined with automated post-processing steps for the two studies by Eckstein et al. [77,78]. All studies used the cartilage measures derived from manual segmentations as reference and Panfilov et al. also used cartilage and meniscal volume measures obtained with a different automated technique as additional reference [71]. Panfilov et al. reported strong to very strong correlations between the DL-based and the reference measures and found that the pattern of ORs for differences in change between progression groups was largely consistent with that of the reference measures (Table 1, Fig. 1). However, the ORs obtained with the DL-based technique tended to be somewhat lower than those obtained from manual cartilage thickness measures, possibly because the manual cartilage segmentation considered denuded areas of the subchondral bone as having 0 mm thickness, whereas the DL-based technique only computed the cartilage thickness over the non-denuded subchondral bone area (Table 1, Fig. 1). Eckstein et al. reported no statistically significant differences in sensitivity to change within progression groups or discriminative power between progression groups when comparing cartilage thickness measures obtained from a U-Net trained on radiographic OA knees with those from manual reference segmentations (Table 1, Fig. 1) [77]. Additional analyses showed that U-Nets trained on healthy knees, or on a combination of healthy and radiographic OA knees, showed a notably lower sensitivity to change and discriminative power than those trained on radiographic OA knees or manual cartilage thickness measures (Table 1) [77]. More recently, Eckstein et al. repeated the analysis of the OAI FNIH cohort to compare the sensitivity to change of automated segmentations from coronal fast low angle shot MRI with that observed from sagittal DESS MRI in the subset of 322 knees that also had coronal fast low angle shot MRIs acquired by the OAI [78]. This study demonstrated that both imaging protocols provided comparable sensitivity to change and ability to detect differences in change between groups as manual reference segmentations (Table 1, Fig. 1) [78].
Fig. 1.
Effect size measures (OR or Cohen’s d) observed for the sensitivity to differences in change of fully automated (blue) and reference (orange) measures in structural progression models. a) shows the OR for 2-year change in medial femorotibial compartment (MFTC) cartilage thickness (Panfilov et al. [76]), b) and c) show the Cohen’s d for differences in progression over 2 years in MFTC cartilage thickness (Eckstein et al. [77,78]), d) shows the Cohen’s d for differences in progression over 2 years in laminar total femorotibial joint T2 relaxation time between KLG 0 knees with vs. without accelerated risk of progression (JSN vs. no signs of radiographic OA in contralateral knee, Wirth et al. [80]). The OAI FNIH model (studies a) - c)) included 4 progression groups: Radiographic and pain progression, radiographic progression only, pain progression only, and no progression.
Brett et al. applied the AAM-based cartilage segmentation presented by Bowes et al. [75] to coronal spoiled gradient echo MRIs from a large phase II interventional study [9], in which a statistically significant, dose-dependent effect of the treatment with Sprifermin on femorotibial cartilage thickness change had been observed using manual segmentations [7]. Similar to the manual cartilage thickness analysis, Brett et al. reported a statistically significant, dose-dependent structure modification over two years using the AAM-based method [9]. The magnitude of change was again greater than that observed for manual cartilage thickness measurements but it was accompanied by a greater variability of the changes in this study (Table 2) [9]. Because no effect size measures were reported for the two segmentation techniques, it remains unclear whether one technique is more sensitive to differences in change between groups than the other.
3.3.2. Cartilage composition
The cartilage T2 relaxation time has been frequently used to study compositional cartilage alterations in early OA models [5], with elevated T2 indicating OA-related alterations in collagen content or orientation and in cartilage hydration [79]. Two studies compared the sensitivity of DL-based T2 measurements for detecting differences in change between clinically defined groups with that observed using manual cartilage segmentation [80,81]. Both studies used the same image analysis pipeline, adapted from the technique used for DL-based quantification of cartilage morphology [77,82] and analyzed laminar (superficial and deep layer) T2 from multi-echo spin-echo MRIs. Wirth et al. compared laminar T2 between radiographically normal knees with (CL-JSN) and without contralateral JSN or other signs of radiographic OA (CL-noROA) from the OAI and reported a similar sensitivity to differences in laminar T2 change over three years between CL-JSN and CL-noROA knees using the DL-based T2 technique as that observed with the manual T2 analysis method (Table 1, Fig. 1) [80]. This study additionally reported greater accuracy and sensitivity to differences for U-Nets trained on all seven echoes of the multi-echo spin-echo MRIs than for U-Nets trained on the first echo only [80]. Eckstein et al. compared laminar changes in T2 acquired using 1.5T MRI between ACL-injured patients with and without dynamic knee instability, ACL-injured patients after surgical reconstruction, and healthy controls [81]. Over one year, only minimal change in cartilage T2 was observed in this cohort and a statistically significant increase was observed only in the deep layer of the central lateral femoral condyle (across all knees). This increase was, however, detected using both DL-based and manual methods with comparable sensitivity to change (Table 2). The differences in change were not found to differ between groups with either DL-based and manual segmentation methods [81].
3.3.3. Thigh muscle and adipose tissue
Kemnitz et al. compared the sensitivity of a U-Net-based, fully automated thigh MRI segmentation method for evaluating change in knee extensor and flexor muscles as well as adipose tissue cross-sectional areas (CSA) with the sensitivity to change obtained from manual thigh MRI segmentations in a model investigating changes in thigh muscle and adipose tissue in OAI participants with weight loss (≤-10 %) or gain (≥+10 %) over two years [60]. The study by Kemnitz et al. did not compare effect size measures of between-group differences between the DL-based and the reference segmentation method but reported the sensitivity to change stratified by the type of weight change and sex. The sensitivity to change was found to be similar between segmentation techniques except for change in inter-muscular adipose tissue in both women and men with weight gain, where the DL-based technique showed greater sensitivity to change than the reference technique (Table 2) [60].
4. Discussion
This review identified five studies comparing the clinical validity of fully automated methods for assessing progression in (peri-) articular joint tissues with reference measures in clinical OA models. Of these, four studies provided effect size measures characterizing the strength of the associations with clinical outcomes for both fully automated and reference methods. Four additional studies reported only the sensitivity to change for fully automated and reference measures in clinically defined cohorts. Most studies were based on DL techniques, which predominantly used variants of the U-Net architecture. Only two studies relied on the more traditional but still powerful active appearance models. The studies demonstrated that the sensitivity to change and discriminative power of the fully automated techniques were similar to those observed with the respective reference techniques. Interestingly, no study was identified that reported data on the clinical validity of assessing progression using fully automated semi-quantitative scoring methods.
The clinical validation builds on the successful technical validation and needs to demonstrate that a method is suitable (and safe) for its intended clinical purpose. The technical validity of an automated measurement technique can be demonstrated by showing that the accuracy (e.g., segmentation accuracy systematic error of measures vs. reference measures) and precision (e.g., random error from test-retest data) are sufficient for the respective purpose. Additionally, the variability of the observed accuracy and precision (e.g., minimum/maximum observed segmentation accuracy or random/systematic error) should be reported and the technical validation should also examine the robustness of the method, particularly its performance under nonoptimal conditions (e.g. artifacts). Once the technical validity is established, the clinical validity of the method can be investigated by evaluating the sensitivity to cross-sectional differences, to change, and/or to differences in change between clinically defined cohorts or knees with different clinical outcomes, depending on the intended purpose. If possible, clinical validation should include a gold-standard reference technique against which results can be compared. The patients or groups selected for these evaluations should resemble the patients or groups to which the automated technique will ultimately be applied to demonstrate its suitability for the intended purpose. Depending on the technique and the intended purpose, it may also be necessary to validate that safety measures are working correctly (e.g., automated correction of minor errors or refusal to provide output in case of severe errors or unexpected input data).
Most of the studies evaluating the performance of fully automated techniques for assessing progression performed a thorough technical validation, but only a few studies took the additional step of clinically validating the techniques. This may potentially be attributed to a lack of suitable data or clinical models, albeit various clinical models and datasets have become available over recent years. The most widely used clinical model for validating the sensitivity of methods to differences in progression is the OAI FNIH progression model, which comprises 600 knees stratified into four progression groups [74]. For this model, a large variety of clinical data and image assessments are publicly available [33,72,74] and can be used to validate quantitative or semi-quantitative measures of progression in a variety of tissues. Similar measures are, however, also available for other models based on data from the OAI, including knees developing incident radiographic OA [32] or knees progressing to total knee replacement surgery [83,84]. New clinical models can be developed using large databases such as the OAI (https://nda.nih.gov/oai) but also using other cohorts like the multi-center osteoarthritis study (MOST, https://mostonline.ucsf.edu/), the CHECK cohort [85], or the IMI Approach cohort [86,87] (https://approachproject.eu/). The pathway outlined by Hunter et al. [88] and Bauer et al. [89] for the development and validation of biomarkers may be useful for designing own clinical models for the clinical validation of automated techniques.
A clinical validation should ideally be based on diverse datasets and clinical models to demonstrate that the methodology or device is not limited to a specific type of input (e.g., one specific MRI sequence) or clinical model (e.g., sensitive only to cartilage loss but not to treatment-induced cartilage gain). DL-based techniques are often trained on specific imaging protocols such as DESS MRI provided by the OAI, and the translation of models trained on a specific MRI sequence to other sequences has been reported to result in reduced segmentation accuracy [65], which is likely to affect the ability to detect cross-sectional or longitudinal differences between groups. To address this limitation, DL models should incorporate strategies such as training on all relevant image protocols or employ transfer learning to adapt pre-trained models to new protocols and thereby improve generalizability. It should be noted that the interpretation of model performance in studies using the OAI dataset is limited to that dataset only and may not reflect real-world performance, particularly in the context of multi-center studies, which are typically employed for evaluating the efficacy of disease modifying OA drug candidates. Demonstrating robust performance of DL models in such heterogeneous, real-world datasets is therefore essential for the application to clinical studies and future studies should aim to demonstrate that the proposed techniques can be successfully translated to other clinical datasets.
More modern DL techniques than the ones used in the included studies have recently emerged. For instance, unsupervised generative adversarial networks, which learn patterns from unlabeled training images, have already been applied in recent studies [90,91]. Vision transformers, inspired by the attention-layer architecture [92] behind large language models like ChatGPT or DeepSeek [93], are particularly effective for capturing global context in data [94]. These models offer potential advantages over traditional DL approaches, including reduced dependency on specific imaging protocols and the ability to work with multimodal imaging data (e.g., combining CT and MRI). Together with self-supervised learning, which leverages unlabeled data to uncover meaningful representations [95], these techniques represent promising future directions for addressing the limitations of current DL-based segmentation approaches. Preliminary studies have begun to explore their utility in the field. A study by Kim et al. explored the utility of a transformer-based U-Net architecture for musculoskeletal segmentation tasks in CT imaging [96], while Dominic et al. applied self-supervised learning to musculoskeletal MRI tissue segmentation, including cartilage assessment [97]. Despite these advantages, such methods currently require much larger datasets and greater computational resources compared to architectures such as the U-Net. Further technical and clinical validation is necessary to fully harness their potential, particularly for applications such as tracking disease progression and improving the generalizability of automated segmentation techniques across diverse imaging protocols.
Another critical aspect to consider in fully automated techniques is their robustness and reliability. When using manual or semi-automated analysis techniques, the final decision is made by a human observer. While this decision may not always be perfectly accurate, it is unlikely to be completely wrong. Fully automated techniques, in contrast, may fail to provide correct results in some of the analyzed data and, without human supervision, such erroneous results may remain undetected and could potentially affect the study outcome or negatively impact patient diagnosis or treatment. For this reason, fully automated techniques such should implement safety measures such as those described by Zaman et al. [98], which allow to correct (or at least identify) implausible or invalid observations, and they should validate their ability to identify implausible analyses to minimize the risk of providing invalid results. The studies reporting data on the clinical validity of fully automated methods for assessing progression did not provide many (if any) details on their measures for detecting or correcting invalid analysis results. Future studies should provide details on such safety measures and, if possible, systematically evaluate their effectiveness. The latter is of particular importance when applying fully automated techniques to imaging data acquired in multicenter studies that typically comprise images acquired using different scanners and protocols and that also comprise images of lower quality than those available from well-curated studies such as the OAI [99].
In conclusion, only a few studies reported data demonstrating the clinical validity of fully automated techniques for assessing structural progression in (peri-)articular tissues. The fully automated techniques evaluated in the nine included studies showed similar or even greater sensitivity to change and similar discriminative power for detecting differences in change between clinically defined groups compared with the respective reference measurements. The techniques are therefore likely to be suitable for assessing structural progression provided that the image data meet the expectations and are of sufficient quality.
Author contributions
Conception and design of the study, or acquisition of data, or analysis and interpretation of data: Wolfgang Wirth and Jana Kemnitz.
Drafting the article or revising it critically for important intellectual content: Wolfgang Wirth and Jana Kemnitz.
Final approval of the version to be submitted: Wolfgang Wirth and Jana Kemnitz.
Wolfgang Wirth takes the responsibility for the integrity of the work.
Declaration of generative AI and AI-assisted technologies in the writing process
During the preparation of this work the authors used ChatGPT (OpenAI, CA, USA) for proof-reading of the manuscript. After using this service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.
Role of the funding source
No funding was received for this review article.
Conflicts of interests
• WW: Employee and share-holder of Chondrometrics GmbH, Germany.
• JE: Serves on the advisory board of Chondrometrics GmbH, Germany in an honorary role.
Handling Editor: Professor H. Madry
Footnotes
This article is part of a special issue entitled: Artificial intelligence in Osteoarthritis imaging published in Osteoarthritis and Cartilage Open.
References
- 1.Eckstein F., Wluka A.E., Wirth W., Cicuttini F. 30 years of MRI-based cartilage & bone morphometry in knee osteoarthritis: from correlation to clinical trials. Osteoarthr. Cartil. 2024;32(4):439–451. doi: 10.1016/j.joca.2024.02.002. [DOI] [PubMed] [Google Scholar]
- 2.Roemer F.W., Wirth W., Demehri S., Kijowski R., Jarraya M., Hayashi D., et al. Imaging biomarkers of osteoarthritis. Semin. Muscoskel. Radiol. 2024;28(1):14–25. doi: 10.1055/s-0043-1776432. [DOI] [PubMed] [Google Scholar]
- 3.Roemer FW, Jarraya M, Hayashi D, Crema MD, Haugen IK, Hunter DJ, et al. A perspective on the evolution of semi-quantitative MRI assessment of osteoarthritis: past, present and future. Osteoarthr. Cartil. 2024 Apr;32(4):460–472. doi: 10.1016/j.joca.2024.01.001. [DOI] [PubMed] [Google Scholar]
- 4.Wirth W., Ladel C., Maschek S., Wisser A., Eckstein F., Roemer F. Quantitative measurement of cartilage morphology in osteoarthritis: current knowledge and future directions. Skelet. Radiol. 2023;52(11):2107–2122. doi: 10.1007/s00256-022-04228-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Link T.M., Joseph G.B., Li X. MRI-based T1rho and T2 cartilage compositional imaging in osteoarthritis: what have we learned and what is needed to apply it clinically and in a trial setting? Skelet. Radiol. 2023;52(11):2137–2147. doi: 10.1007/s00256-023-04310-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Reichmann W.M., Maillefert J.F., Hunter D.J., Katz J.N., Conaghan P.G., Losina E. Responsiveness to change and reliability of measurement of radiographic joint space width in osteoarthritis of the knee: a systematic review. Osteoarthr. Cartil. 2011;19:550–556. doi: 10.1016/j.joca.2011.01.023. 1522-9653 (Electronic) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hochberg M.C., Guermazi A., Guehring H., Aydemir A., Wax S., Fleuranceau-Morel P., et al. Effect of intra-articular sprifermin vs placebo on femorotibial joint cartilage thickness in patients with osteoarthritis. JAMA. 2019;322(14):1360–1370. doi: 10.1001/jama.2019.14735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Schnitzer T., Pueyo M., Deckx H., van der Aar E., Bernard K., Hatch S., et al. Evaluation of S201086/GLPG1972, an ADAMTS-5 inhibitor, for the treatment of knee osteoarthritis in ROCCELLA: a phase 2 randomized clinical trial. Osteoarthr. Cartil. 2023;31(7):985–994. doi: 10.1016/j.joca.2023.04.001. [DOI] [PubMed] [Google Scholar]
- 9.Brett A., Bowes M.A., Conaghan P.G., Ladel C., Guehring H., Moreau F., et al. Automated MRI assessment confirms cartilage thickness modification in patients with knee osteoarthritis: post-hoc analysis from a phase II sprifermin study. Osteoarthr. Cartil. 2020;28(11):1432–1436. doi: 10.1016/j.joca.2020.08.005. [DOI] [PubMed] [Google Scholar]
- 10.McGuire D., Lane N., Segal N., Metyas S., Barthel H., Miller M., et al. TPX-100 leads to marked, sustained improvements in subjects with knee osteoarthritis: pre-clinical rationale and results of a controlled clinical trial. Osteoarthr. Cartil. 2018;26(2018) [Google Scholar]
- 11.McGuire D., Bowes M., Brett A., Segal N.A., Miller M., Rosen D., et al. Study TPX-100-5: intra-articular TPX-100 significantly delays pathological bone shape change and stabilizes cartilage in moderate to severe bilateral knee OA. Arthritis Res. Ther. 2021;23(1):1–9. doi: 10.1186/s13075-021-02622-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.McAlindon T., LaValley M., Schneider E., Nuite M., Lee J.Y., Price L.L., et al. Effect of vitamin D supplementation on progression of knee pain and cartilage volume loss in patients with symptomatic osteoarthritis: a randomized controlled trial. JAMA. 2013;309(2):155–162. doi: 10.1001/jama.2012.164487. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.McAlindon T.E., LaValley M.P., Harvey W.F., Price L.L., Driban J.B., Zhang M., et al. Effect of intra-articular triamcinolone vs saline on knee cartilage volume and pain in patients with knee osteoarthritis a randomized clinical trial. JAMA, J. Am. Med. Assoc. 2017;317(19):1967–1975. doi: 10.1001/jama.2017.5283. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Pelletier J.-P., Roubille C., Raynauld J.-P., Abram F., Dorais M., Delorme P., et al. Disease-modifying effect of strontium ranelate in a subset of patients from the Phase III knee osteoarthritis study SEKOIA using quantitative MRI: reduction in bone marrow lesions protects against cartilage loss. Ann. Rheum. Dis. 2015;74(2):422–429. doi: 10.1136/annrheumdis-2013-203989. [DOI] [PubMed] [Google Scholar]
- 15.Bennell K.L., Paterson K.L., Metcalf B.R., Duong V., Eyles J., Kasza J., et al. Effect of intra-articular platelet-rich plasma vs placebo injection on pain and medial tibial cartilage volume in patients with knee osteoarthritis: the RESTORE randomized clinical trial. JAMA, J. Am. Med. Assoc. 2021;326(20):2021–2030. doi: 10.1001/jama.2021.19415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Yazici Y., McAlindon T.E., Gibofsky A., Lane N.E., Lattermann C., Skrepnik N., et al. A phase 2b randomized trial of lorecivivint, a novel intra-articular CLK2/DYRK1A inhibitor and Wnt pathway modulator for knee osteoarthritis. Osteoarthr. Cartil. 2021;29(5):654–666. doi: 10.1016/j.joca.2021.02.004. [DOI] [PubMed] [Google Scholar]
- 17.Hunter D.J., Zhang Y.Q., Tu X., Lavalley M., Niu J.B., Amin S., et al. Change in joint space width: hyaline articular cartilage loss or alteration in meniscus? Arthritis Rheum. 2006;54(8):2488–2495. doi: 10.1002/art.22016. [DOI] [PubMed] [Google Scholar]
- 18.Wirth W., Frobell R.B., Souza R.B., Li X., Wyman B.T., Graverand MP Le, et al. A three-dimensional quantitative method to measure meniscus shape, position, and signal intensity using MR images: a pilot study and preliminary results in knee osteoarthritis. Magnes. Res. 2010;63(5):1162–1171. doi: 10.1002/mrm.22380. [DOI] [PubMed] [Google Scholar]
- 19.Roth M., Emmanuel K., Wirth W., Kwoh C.K., Hunter D.J., Hannon M.J., et al. Changes in medial meniscal 3D position and morphology predict knee replacement in rapidly progressing knee Osteoarthritis - data from the Osteoarthritis initiative (OAI) Arthritis Care Res. 2021;73(7):1031–1037. doi: 10.1002/acr.24193. [DOI] [PubMed] [Google Scholar]
- 20.Chuckpaiwong B., Charles H.C., Kraus V.B., Guilak F., Nunley J.A. Age-associated increases in the size of the infrapatellar fat pad in knee osteoarthritis as measured by 3T MRI. J. Orthop. Res. 2010;28(9):1149–1154. doi: 10.1002/jor.21125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Pan F., Han W., Wang X., Liu Z., Jin X., Antony B., et al. A longitudinal study of the association between infrapatellar fat pad maximal area and changes in knee symptoms and structure in older adults. Ann. Rheum. Dis. 2015;74(10):1818–1824. doi: 10.1136/annrheumdis-2013-205108. [DOI] [PubMed] [Google Scholar]
- 22.Ruhdorfer A., Haniel F., Petersohn T., Dörrenberg J., Wirth W., Dannhauer T., et al. Between-group differences in infra-patellar fat pad size and signal in symptomatic and radiographic progression of knee osteoarthritis vs non-progressive controls and healthy knees – data from the FNIH biomarkers consortium study and the osteoarthritis in. Osteoarthr. Cartil. 2017;25(7):1114–1121. doi: 10.1016/j.joca.2017.02.789. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Stirling C.E., Neeteson N.J., Walker R.E.A., Boyd S.K. Deep learning-based automated detection and segmentation of bone and traumatic bone marrow lesions from MRI following an acute ACL tear. Comput. Biol. Med. 2024;178(7) doi: 10.1016/j.compbiomed.2024.108791. [DOI] [PubMed] [Google Scholar]
- 24.Ratzlaff C., Guermazi A., Collins J., Katz J.N., Losina E., Vanwyngaarden C., et al. A rapid, novel method of volumetric assessment of MRI-detected subchondral bone marrow lesions in knee osteoarthritis. Osteoarthr. Cartil. 2013;21(6):806–814. doi: 10.1016/j.joca.2013.03.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Arokoski M.H., Arokoski J.P.A., Haara M., Kankaanpää M., Vesterinen M., Niemitukia L.H., et al. Hip muscle strength and muscle cross sectional area in men with and without hip osteoarthritis. J. Rheumatol. 2002;29(10):2185–2195. [PubMed] [Google Scholar]
- 26.Hudelmaier M., Wirth W., Himmer M., Ring-Dimitriou S., Sänger A., Eckstein F., et al. Effect of exercise intervention on thigh muscle volume and anatomical cross-sectional areas--quantitative assessment using MRI. Magn. Reson. Med. 2010;64(6):1713–1720. doi: 10.1002/mrm.22550. [DOI] [PubMed] [Google Scholar]
- 27.Dannhauer T., Ruhdorfer A., Wirth W., Eckstein F. Quantitative relationship of thigh adipose tissue with pain, radiographic status, and progression of knee osteoarthritis: longitudinal findings from the osteoarthritis initiative. Investig. Radiol. 2015;50(4):268–274. doi: 10.1097/RLI.0000000000000113. [DOI] [PubMed] [Google Scholar]
- 28.Ruhdorfer A., Wirth W., Dannhauer T., Eckstein F. Longitudinal (4 year) change of thigh muscle and adipose tissue distribution in chronically painful vs painless knees - data from the Osteoarthritis Initiative. Osteoarthr. Cartil. 2015;23(8):1348–1356. doi: 10.1016/j.joca.2015.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Altman R.D., Gold G.E. Atlas of individual radiographic features in osteoarthritis, revised. Osteoarthr. Cartil. 2007;15(Suppl A):A1-–56. doi: 10.1016/j.joca.2006.11.009. [DOI] [PubMed] [Google Scholar]
- 30.Hunter D.J., Guermazi A., Lo G.H., Grainger A.J., Conaghan P.G., Boudreau R.M., et al. Evolution of semi-quantitative whole joint assessment of knee OA: MOAKS (MRI Osteoarthritis Knee score) Osteoarthr. Cartil. 2011;19(8):990–1002. doi: 10.1016/j.joca.2011.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Roemer F.W., Aydemir A., Lohmander S., Crema M.D., Marra M.D., Muurahainen N., et al. Structural effects of sprifermin in knee osteoarthritis: a post-hoc analysis on cartilage and non-cartilaginous tissue alterations in a randomized controlled trial. BMC Muscoskelet. Disord. 2016;17:267. doi: 10.1186/s12891-016-1128-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Roemer F.W., Kwoh C.K., Hannon M.J., Hunter D.J., Eckstein F., Fujii T., et al. What comes first? multi-tissue involvement leading to radiographic osteoarthritis: MRI-based trajectory analysis over 4 years in the osteoarthritis initiative. Arthritis Rheumatol. 2015;67(8):2085–2096. doi: 10.1002/art.39176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Roemer F.W., Guermazi A., Collins J.E., Losina E., Nevitt M.C., Lynch J.A., et al. Semi-quantitative MRI biomarkers of knee osteoarthritis progression in the FNIH biomarkers consortium cohort − methodologic aspects and definition of change. BMC Muscoskelet. Disord. 2016;17(1):466. doi: 10.1186/s12891-016-1310-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Cromer M.S., Bourne R.M., Fransen M., Fulton R., Wang S.C. Responsiveness of quantitative cartilage measures over one year in knee osteoarthritis: comparison of radiography and MRI assessments. JMagn Reson. 2014;39:103–109. doi: 10.1002/jmri.24141. 1522-2586 (Electronic) [DOI] [PubMed] [Google Scholar]
- 35.Roemer F.W., Nevitt M.C., Felson D.T., Niu J., Lynch J.A., Crema M.D., et al. Predictive validity of within-grade scoring of longitudinal changes of MRI-based cartilage morphology and bone marrow lesion assessment in the tibio-femoral joint--the MOST study. Osteoarthr. Cartil. 2012;20:1391–1398. doi: 10.1016/j.joca.2012.07.012. 1522-9653 (Electronic) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Peterfy C.G., Dijke CF van, Janzen D.L., Gluer C.C., Namba R., Majumdar S., et al. Quantification of articular cartilage in the knee with pulsed saturation transfer subtraction and fat-suppressed MR imaging: optimization and validation. Radiology. 1994;192(0033–8419):485–491. doi: 10.1148/radiology.192.2.8029420. [DOI] [PubMed] [Google Scholar]
- 37.Ebrahimkhani S., Jaward M.H., Cicuttini F.M., Dharmaratne A., Wang Y., Herrera AGS de. A review on segmentation of knee articular cartilage: from conventional methods towards deep learning. Artif. Intell. Med. 2020;106(February) doi: 10.1016/j.artmed.2020.101851. [DOI] [PubMed] [Google Scholar]
- 38.Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Int. Conf. Med. Image Comput. Comput. Interv. pp 234–241.
- 39.Ambellan F., Tack A., Ehlke M., Zachow S. Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: data from the Osteoarthritis initiative. Med. Image Anal. 2019;52:109–118. doi: 10.1016/j.media.2018.11.009. [DOI] [PubMed] [Google Scholar]
- 40.Norman B., Pedoia B.S.V., Majumdar S. Use of 2D U-Net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology. 2018;288(1):177–185. doi: 10.1148/radiol.2018172322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Fripp J., Crozier S., Warfield S.K., Ourselin S. Automatic segmentation of articular cartilage in magnetic resonance images of the knee. Med. Image Comput. Comput. Assist. Interv. 2007;10(Pt 2):186–194. doi: 10.1007/978-3-540-75759-7_23. [DOI] [PubMed] [Google Scholar]
- 42.Williams T.G., Holmes A.P., Waterton J.C., Maciewicz R.A., Hutchinson C.E., Moots R.J., et al. Anatomically corresponded regional analysis of cartilage in asymptomatic and osteoarthritic knees by statistical shape modelling of the bone. IEEE Trans. Med. Imag. 2010;29(8):1541–1559. doi: 10.1109/TMI.2010.2047653. [DOI] [PubMed] [Google Scholar]
- 43.Yin Y., Zhang X., Williams R., Wu X., Anderson D.D., Sonka M. LOGISMOS--layered optimal graph image segmentation of multiple objects and surfaces: cartilage segmentation in the knee joint. IEEE Trans. Med. Imag. 2010;29:2023–2037. doi: 10.1109/TMI.2010.2058861. 1558–0062 (Electronic) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Folkesson J., Dam E.B., Olsen O.F., Pettersen P.C., Christiansen C. Segmenting articular cartilage automatically using a voxel classification approach. IEEE Trans. Med. Imag. 2007;26:106–115. doi: 10.1109/TMI.2006.886808. 0278–0062 (Print) [DOI] [PubMed] [Google Scholar]
- 45.Prescott J.W., Priddy M., Best T.M., Pennell M., Swanson M.S., Haq F., et al. An automated method to detect interstitial adipose tissue in thigh muscles for patients with osteoarthritis. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2009;2009:6360–6363. doi: 10.1109/IEMBS.2009.5333260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Li W., Abram F., Pelletier J.-P., Raynauld J.-P., Dorais M., D’Anjou M.-A., et al. Fully automated system for the quantification of human osteoarthritic knee joint effusion volume using magnetic resonance imaging. Arthritis Res. Ther. 2010;12(5) doi: 10.1186/ar3133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Dodin P., Martel-Pelletier J., Pelletier J.-P., Abram F. A fully automated human knee 3D MRI bone segmentation using the ray casting technique. Med. Biol. Eng. Comput. 2011;49(12):1413–1424. doi: 10.1007/s11517-011-0838-8. [DOI] [PubMed] [Google Scholar]
- 48.Kauffmann C., Gravel P., Godbout B., Gravel A., Beaudoin G., Raynauld J.P., et al. Computer-aided method for quantification of cartilage thickness and volume changes using MRI: validation study using a synthetic model. IEEE Trans. Biomed. Eng. 2003;50:978–988. doi: 10.1109/TBME.2003.814539. 0018–9294. [DOI] [PubMed] [Google Scholar]
- 49.Dam EB, Desai AD, Deniz CM, Rajamohan HR, Regatte R, Iriondo C, et al. Towards automatic cartilage quantification in clinical trials - continuing from the 2019 IWOAI knee segmentation challenge. Osteoarthr. Imag. 2023 Mar;3(1):100087. doi: 10.1016/j.ostima.2023.100087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Desai A.D., Caliva F., Iriondo C., Mortazi A., Jambawalikar S., Bagci U., et al. The international workshop on osteoarthritis imaging knee mri segmentation challenge: a multi-institute evaluation and analysis framework on a standardized dataset. Radiol. Artif. Intell. 2021;3(3):1–13. doi: 10.1148/ryai.2021200078. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Lindner C., Thiagarajah S., Wilkinson J.M., Wallis G.A., Cootes T.F., arcOGEN Consortium Accurate bone segmentation in 2D radiographs using fully automatic shape model matching based on regression-voting. Med. Image Comput. Comput. Assist. Interv. 2013;16(Pt 2):181–189. doi: 10.1007/978-3-642-40763-5_23. [DOI] [PubMed] [Google Scholar]
- 52.Bowes M.A., Vincent G.R., Wolstenholme C.B., Conaghan P.G. A novel method for bone area measurement provides new insights into osteoarthritis and its progression. Ann. Rheum. Dis. 2015;74(3):519–525. doi: 10.1136/annrheumdis-2013-204052. [DOI] [PubMed] [Google Scholar]
- 53.Woo B., Engstrom C., Baresic W., Fripp J., Crozier S., Chandra S.S. Automated anomaly-aware 3D segmentation of bones and cartilages in knee MR images from the osteoarthritis initiative. Med. Image Anal. 2024;93 doi: 10.1016/j.media.2024.103089. [DOI] [PubMed] [Google Scholar]
- 54.Moradi K, Mohammadi S, Roemer FW, Momtazmanesh S, Hathaway Q, Ibad HA, et al. Progression of bone marrow lesions and the development of knee osteoarthritis: osteoarthritis initiative data. Radiology. 2024 Sep;312(3) doi: 10.1148/radiol.240470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Tack A., Mukhopadhyay A., Zachow S. Knee menisci segmentation using convolutional neural networks: data from the osteoarthritis initiative. Osteoarthr. Cartil. 2018;26(5):680–688. doi: 10.1016/j.joca.2018.02.907. [DOI] [PubMed] [Google Scholar]
- 56.Felfeliyan B., Wichuk S., Hareendranathan A.R., Lambert R.G., Maksymowych W.P., Jaremko J. OMERACT validation of a deep learning algorithm for automated absolute quantification of knee joint effusion versus manual semi-quantitative assessment. Semin. Arthritis Rheum. 2024;66 doi: 10.1016/j.semarthrit.2024.152420. [DOI] [PubMed] [Google Scholar]
- 57.Felfeliyan B., Hareendranathan A., Kuntze G., Jaremko J.L., Ronsky J.L. Improved-mask R-CNN: towards an accurate generic MSK MRI instance segmentation platform (data from the Osteoarthritis initiative) Comput. Med. Imag. Graph. 2022;97 doi: 10.1016/j.compmedimag.2022.102056. [DOI] [PubMed] [Google Scholar]
- 58.Yu K., Ying J., Zhao T., Lei L., Zhong L., Hu J., et al. Prediction model for knee osteoarthritis using magnetic resonance-based radiomic features from the infrapatellar fat pad: data from the osteoarthritis initiative. Quant. Imag. Med. Surg. 2023;13(1):352–369. doi: 10.21037/qims-22-368. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Wang M., Yu C., Li M., Zhang X., Jiang K., Zhang Z., et al. One-stop detection of anterior cruciate ligament injuries on magnetic resonance imaging using deep learning with multicenter validation. Quant. Imag. Med. Surg. 2024;14(5):3405–3416. doi: 10.21037/qims-23-1539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Kemnitz J., Steidle-Kloc E., Wirth W., Fuerst D., Wisser A., Eder S.K., et al. Local MRI-based measures of thigh adipose tissue derived from fully automated deep convolutional neural network-based segmentation show a comparable responsiveness to bidirectional change in body weight as from quality controlled manual segmentation. Ann. Anat. 2021 doi: 10.1016/j.aanat.2021.151866. [DOI] [PubMed] [Google Scholar]
- 61.Mohajer B., Dolatshahi M., Moradi K., Najafzadeh N., Eng J., Zikria B., et al. Role of thigh muscle changes in knee Osteoarthritis outcomes: osteoarthritis initiative data. Radiology. 2022;305(1):169–178. doi: 10.1148/radiol.212771. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Mohajer B., Moradi K., Guermazi A., Dolatshahi M., Roemer F.W., Ibad H.A., et al. Statin use and longitudinal changes in quantitative MRI-based biomarkers of thigh muscle quality: data from Osteoarthritis initiative. Skelet. Radiol. 2024;53(4):683–695. doi: 10.1007/s00256-023-04473-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Williams AA, Asay JL, Asare D, Desai AD, Gold GE, Hargreaves BA, et al. Reproducibility of quantitative double-echo steady-state T2 mapping of knee cartilage. J. Magn. Reson. Imag. 2025 Feb;61(2):784–795. doi: 10.1002/jmri.29431. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Chaudhari A.S., Grissom M.J., Fang Z., Sveinsson B., Lee J.H., Gold G.E., et al. Diagnostic accuracy of quantitative multicontrast 5-Minute knee MRI using prospective artificial intelligence image quality enhancement. AJR Am. J. Roentgenol. 2021;216(6):1614–1625. doi: 10.2214/AJR.20.24172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Schmidt AM, Desai AD, Watkins LE, Crowder HA, Black MS, Mazzoli V, et al. Generalizability of deep learning segmentation algorithms for automated assessment of cartilage morphology and MRI relaxometry. J. Magn. Reson. Imag. 2023 Apr;57(4):1029–1039. doi: 10.1002/jmri.28365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Wirth W., Herger S., Maschek S., Wisser A., Eckstein F., Mündermann A. Validation of a fully automated cartilage SPIN-SPIN (T2) relaxation time analysis workflow from quantitative DESS (QDESS) MRI. Osteoarthr. Imag. 2024;4 [Abstract] [Google Scholar]
- 67.Duryea J., Neumann G., Niu J., Totterman S., Tamez J., Dabrowski C., et al. Comparison of radiographic joint space width with magnetic resonance imaging cartilage morphometry: analysis of longitudinal data from the Osteoarthritis initiative. Arthritis Care Res. 2010;62(7):932–937. doi: 10.1002/acr.20148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Eckstein F., Boudreau R., Wang Z., Hannon M.J., Duryea J., Wirth W., et al. Comparison of radiographic joint space width and magnetic resonance imaging for prediction of knee replacement: a longitudinal case-control study from the osteoarthritis initiative. Eur. Radiol. 2016;26(6):1942–1951. doi: 10.1007/s00330-015-3977-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Paixao T., DiFranco M.D., Ljuhar R., Ljuhar D., Goetz C., Bertalan Z., et al. A novel quantitative metric for joint space width: data from the Osteoarthritis initiative (OAI) Osteoarthr. Cartil. 2020;28(8):1055–1061. doi: 10.1016/j.joca.2020.04.003. [DOI] [PubMed] [Google Scholar]
- 70.Jansen M.P., Salzlechner C., Barnes E., DiFranco M.D., Custers R.J.H., Watt F.E., et al. Artificial intelligence in osteoarthritis: repair by knee joint distraction shows association of pain, radiographic and immunological outcomes. Rheumatology. 2023;62(8):2789–2796. doi: 10.1093/rheumatology/keac723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Hafezi-Nejad N., Guermazi A., Roemer F.W., Hunter D.J., Dam E.B., Zikria B., et al. Prediction of medial tibiofemoral compartment joint space loss progression using volumetric cartilage measurements: data from the FNIH OA biomarkers consortium. Eur. Radiol. 2017;27(2):464–473. doi: 10.1007/s00330-016-4393-4. [DOI] [PubMed] [Google Scholar]
- 72.Eckstein F., Collins J.E., Nevitt M.C., Lynch J.A., Kraus V., Katz J.N., et al. Cartilage thickness change as an imaging biomarker of knee osteoarthritis progression - data from the fnih OA biomarkers consortium. Arthritis Rheumatol. 2015;67(12):3184–3189. doi: 10.1002/art.39324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Peterfy C.G., Schneider E., Nevitt M. The osteoarthritis initiative: report on the design rationale for the magnetic resonance imaging protocol for the knee. Osteoarthr. Cartil. 2008;16(12):1433–1441. doi: 10.1016/j.joca.2008.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Hunter D.J., Deveza L.A., Collins J.E., Losina E., Nevitt M.C., Roemer F.W., et al. Multivariable modeling of biomarker data from the phase 1 foundation for the NIH Osteoarthritis biomarkers consortium. Arthritis Care Res. 2022;74(7):1142–1153. doi: 10.1002/acr.24557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Bowes M.A., Guillard G.A., Vincent G.R., Brett A.D., Wolstenholme C.B.H., Conaghan P.G. Precision, reliability, and responsiveness of a novel automated quantification tool for cartilage thickness: data from the osteoarthritis initiative. J. Rheumatol. 2020;47(2):282–289. doi: 10.3899/jrheum.180541. [DOI] [PubMed] [Google Scholar]
- 76.Panfilov E., Tiulpin A., Nieminen M.T., Saarakkala S., Casula V. Deep learning-based segmentation of knee MRI for fully automatic subregional morphological assessment of cartilage tissues: data from the Osteoarthritis initiative. J. Orthop. Res. 2022;40(5):1113–1124. doi: 10.1002/jor.25150. [DOI] [PubMed] [Google Scholar]
- 77.Eckstein F., Chaudhari A.S., Fuerst D., Gaisberger M., Kemnitz J., Baumgartner C.F., et al. Detection of differences in longitudinal cartilage thickness loss using a deep-learning automated segmentation algorithm: data from the foundation for the national institutes of health biomarkers study of the Osteoarthritis initiative. Arthritis Care Res. 2022;74(6):929–936. doi: 10.1002/acr.24539. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 78.Eckstein F., Chaudhari A., Hunter D.J., Wirth W. Comparison between coronal FLASH and sagittal DESS MR imaging in detecting longitudinal cartilage change by fully automated image segmentation, using convolutional neural networks (CNNs) – data from the FNIH biomarkers study. Osteoarthr. Cartil. Open. 2025;7(3):100657. doi: 10.1016/j.ocarto.2025.100657. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 79.Mosher T.J., Smith H., Dardzinski B.J., Schmithorst V.J., Smith M.B. MR imaging and T2 mapping of femoral cartilage: in vivo determination of the magic angle effect. AJR Am. J. Roentgenol. 2001;177(3):665–669. doi: 10.2214/ajr.177.3.1770665. [DOI] [PubMed] [Google Scholar]
- 80.Wirth W., Maschek S., Wisser A., Eder J., Baumgartner C.F., Chaudhari A., et al. Evaluation of an automated laminar cartilage T2 relaxation time analysis method in an early osteoarthritis model. Skelet. Radiol. 2025;54(3):571–584. doi: 10.1007/s00256-024-04786-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Eckstein F., Brisson N.M., Maschek S., Wisser A., Berenbaum F., Duda G.N., et al. Clinical validation of fully automated laminar knee cartilage transverse relaxation time (T2) analysis in anterior cruciate ligament (ACL)-Injured knees— on behalf of the osteoarthritis (OA)-bio consortium. Quant. Imag. Med. Surg. 2024;14(7):4319–4332. doi: 10.21037/qims-24-194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Wirth W., Eckstein F., Kemnitz J., Baumgartner C.F., Konukoglu E., Fuerst D., et al. Accuracy and longitudinal reproducibility of quantitative femorotibial cartilage measures derived from automated U-Net-based segmentation of two different MRI contrasts: data from the osteoarthritis initiative healthy reference cohort. Magma. 2021;34(3):337–354. doi: 10.1007/s10334-020-00889-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Roemer FW, Kwoh CK, Fujii T, Hannon MJ, Boudreau RM, Hunter DJ, et al. From early radiographic knee osteoarthritis to joint arthroplasty: determinants of structural progression and symptoms. Arthritis Care Res (Hoboken) 2018 Dec;70(12):1778–1786. doi: 10.1002/acr.23545. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Eckstein F., Kwoh C.K., Boudreau R.M., Wang Z., Hannon M.J., Cotofana S., et al. Quantitative MRI measures of cartilage predict knee replacement: a case-control study from the osteoarthritis initiative. Ann. Rheum. Dis. 2013;72(5):707–714. doi: 10.1136/annrheumdis-2011-201164. [DOI] [PubMed] [Google Scholar]
- 85.Wesseling J., Boers M., Viergever M.A., Hilberdink W.K.H.A., Lafeber F.P.J.G., Dekker J., et al. Cohort profile: cohort hip and cohort knee (CHECK) study. Int. J. Epidemiol. 2016;45(1):36–44. doi: 10.1093/ije/dyu177. [DOI] [PubMed] [Google Scholar]
- 86.Helvoort EM van, Spil WE van, Jansen M.P., Welsing P.M.J., Kloppenburg M., Loef M., et al. Cohort profile: the applied public-private research enabling OsteoArthritis clinical headway (IMI-APPROACH) study: a 2-year, European, cohort study to describe, validate and predict phenotypes of osteoarthritis using clinical, imaging and biochemical mark. BMJ Open. 2020;10(7) doi: 10.1136/bmjopen-2019-035101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87.Wirth W., Maschek S., Marijnissen A.C.A., Lalande A., Blanco F.J., Berenbaum F., et al. Test-retest precision and longitudinal cartilage thickness loss in the IMI-APPROACH cohort. Osteoarthr. Cartil. 2023;31(2):238–248. doi: 10.1016/j.joca.2022.10.015. [DOI] [PubMed] [Google Scholar]
- 88.Hunter D.J., Losina E., Guermazi A., Burstein D., Lassere M.N., Kraus V. A pathway and approach to biomarker validation and qualification for osteoarthritis clinical trials. Curr. Drug Targets. 2010;11(5):536–545. doi: 10.2174/138945010791011947. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Bauer D.C., Hunter D.J., Abramson S.B., Attur M., Corr M., Felson D., et al. Classification of osteoarthritis biomarkers: a proposed approach. Osteoarthr. Cartil. 2006;14:723–727. doi: 10.1016/j.joca.2006.04.001. 1063-4584 (Print) [DOI] [PubMed] [Google Scholar]
- 90.Felfeliyan B., Hareendranathan A., Kuntze G., Jaremko J., Ronsky J. MRI knee domain translation for unsupervised segmentation by CycleGAN (data from osteoarthritis initiative (OAI)) Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2021;2021:4052–4055. doi: 10.1109/EMBC46164.2021.9629705. [DOI] [PubMed] [Google Scholar]
- 91.Yu AS, Yang M, Lartey R, Holden W, Ok AH, Khan S, et al. Unsupervised Segmentation of Knee Bone Marrow Edema-like Lesions Using Conditional Generative Models. Bioengineering (Basel) 2024 May 22;11(6):526. doi: 10.3390/bioengineering11060526. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A.N., Kaiser Ł., Polosukhin I. Advances in Neural Information Processing Systems. Vol. 30. Curran Associates, Inc.; 2017. Attention is all you need; pp. 5998–6008. [Google Scholar]
- 93.DeepSeek-Ai, Guo D, Yang D, Zhang H, Song J, Zhang R, et al. Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. Nature. 2025 Sep;645(8081):633–638. doi: 10.1038/s41586-025-09422-z. PMID: 40962978. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 94.Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale (ICLR 2021). arXiv:2010.11929.
- 95.Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning (pp. 1597–1607). Proceedings of Machine Learning Research, Vol. 119. PMLR.
- 96.Kim H.S., Kim H., Kim S., Cha Y., Kim J.-T., Kim J.-W., et al. Precise individual muscle segmentation in whole thigh CT scans for sarcopenia assessment using U-net transformer. Sci. Rep. 2024;14(1):3301. doi: 10.1038/s41598-024-53707-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97.Dominic J, Bhaskhar N, Desai AD, Schmidt A, Rubin E, Gunel B, et al. Improving Data-Efficiency and Robustness of Medical Imaging Segmentation Using Inpainting-based Self-Supervised Learning. Bioengineering (Basel) 2023 Feb 4;10(2):207. doi: 10.3390/bioengineering10020207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 98.Zaman F.A., Zhang L., Zhang H., Sonka M., Wu X. Segmentation quality assessment by automated detection of erroneous surface regions in medical images. Comput. Biol. Med. 2023;164 doi: 10.1016/j.compbiomed.2023.107324. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99.Schneider E., NessAiver M., White D., Purdy D., Martin L., Fanella L., et al. The osteoarthritis initiative (OAI) magnetic resonance imaging quality assurance methods and results. Osteoarthr. Cartil. 2008;16(9):994–1004. doi: 10.1016/j.joca.2008.02.010. [DOI] [PMC free article] [PubMed] [Google Scholar]

