Reliability of Semi-Automated Volumetric Quantification of Knee Effusion Using MRI After Acl Injury

Abstract

Background:

Effusion is a frequent finding after anterior cruciate ligament (ACL) injury and reconstruction, but the reliability of its quantification on routine [non-contrast] magnetic resonance imaging (MRI) is not established. This study sought to evaluate the intra-rater and inter-rater reliability of volumetric quantitative measurements (VQM) of knee effusion volumes (EV) using axial proton density-weighted, fat-suppressed (PD-FS) sequences after ACL injury and reconstruction.

Methods:

Twenty deidentified MRIs (10 ACL-injured knees and 10 uninjured knees) were randomly selected from a prospective observational cohort of participants aged 15-35 years. Two independent raters analyzed EV using a custom MATLAB protocol. Intra- and inter-rater reliability was assessed using intraclass correlation coefficients (ICC_3,1) with minimal detectable changes calculated. Measurement agreement was assessed using Bland-Altman plots.

Results:

Mean EV were higher in the injured compared to uninjured knees (11.88±8.14 mL vs 2.03±0.76 mL). Volumetric effusion measurements demonstrated excellent intra-rater (ICC_3,1=0.993, 95% CI: 0.981-0.997, p<0.001) and inter-rater reliability (ICC_3,1=0.974, 95% CI: 0.907-0.991, p<0.001). The minimal detectable change was 0.145 mL for intra-rater measurements and 0.504 mL for inter-rater measurements. Bland-Altman plots revealed no systematic bias between raters, although variability increased with larger volumes (>8 mL).

Conclusion:

VQM using standard axial PD-FS MRI sequences demonstrate excellent reliability across raters. Restricting segmentation to the anterior compartment aligns with clinical assessment (e.g., sweep tests), though it does not capture whole-joint eflusion. This semi-automated approach may provide a relevant biomarker for monitoring EV in ACL research and rehabilitation trials.

Level of Evidence: III

Introduction

Knee effusion is a biomarker of synovial activation that can be quantified through non-contrast MRI, characterized by increased signal intensity reflecting both intraarticular fluid accumulation and synovial thickening^1–3 Effusion represents a common and significant clinical finding in acute and chronic knee conditions.² Anterior cruciate ligament (ACL) injury is a common cause of knee effusion in acute cases.⁴ ACL injuries are common in sport, with an incidence of 1 in 3500, and approximately 175,000 ACL reconstructions (ACLR) annually in the United States.^5–7 Both ACL and subsequent ACLR are associated with increased effusion which can persist for months after surgery.⁸ The presence of knee effusion after ACL injury is clinically significant because it can negatively impact quadriceps function, knee range of motion and functional performance,^9,10 and if persisting, may contribute to the development of cartilage degradation and osteoarthritis.^1,11–14 This effusion is also a clinical consideration for surgical timing and return to activities.⁸

Healthcare providers evaluate knee effusion volume (EV) through various clinical examination methods including balloon, ballottement, and bulge/stroke tests.^2,15 However, these are subjective and semi-quantitative. Additionally, imaging modalities such as ultrasound, x-ray, computerized tomography (CT) and magnetic resonance imaging (MRI) can be used to assess for presence of an effusion.^3,16–22 Further, MRI is particularly valuable, as it offers fluid-sensitive visualization of the knee and has been incorporated into standardized assessment tools like the Whole-Organ Magnetic Resonance Imaging Score (WORMS) and MRI Osteoarthritis Knee Score (MOAKS).^3,16–22

Advancing on from these semi-quantitative methods, several studies have utilized MRI segmentation techniques to obtain quantitative volumetric effusion measurements in chronic conditions such as osteoar-thritis^3,20,21 and hemophilia.²³ These manual,²¹ semi-automated³ and fully automated²⁰ techniques, provide a more objective approach to effusion assessment in comparison to arthrocentesis, which is invasive and susceptible to sampling errors.

Despite these advancements, volumetric MRI segmentation of EV has not yet to our knowledge been fully validated in the young patient with acute knee trauma, or in post-ACLR populations. Prior studies have focused on dichotomous or semi quantitative effusion analysis within older participants^3,20,21,24 with chronic degenerative conditions (e.g., osteoarthritis) and have relied on specialized 3D or contrast-enhanced MRI protocols.^3,20,21,23 Additionally, anteriorly displaced EV is most relevant to physical examination and early rehabilitation decision-making. However, it is not known whether a volumetric EV protocol using proton density-weighted fat-suppressed (PD-FS) images can reliably capture the full range of EV in both ACL-injured and uninjured knees.

The purpose of this study was to evaluate the intrarater and inter-rater reliability of volumetric quantitative measurements (VQM) of EV on MRI restricted to the clinically relevant anterior compartment on standard axial sequences, in both ACL-injured and uninjured joints. We hypothesized that VQM would demonstrate excellent intra-rater and inter-rater reliability in both ACL injured and contralateral uninjured knees. Injured and uninjured knees were included to identify VQM reliability both in the presence and absence of effusion.

Methods:

Participants

Participants included in this study were part of a parent observational cohort study of 40 participants between the ages of 15-35 years who underwent bilateral knee MRIs at two time points: within one month of ACL injury (before ACL reconstruction) and at 6 months after ACL reconstruction. Exclusion criteria included previous knee injury or surgery of either knee, history of inflammatory disease, immune compromise, body mass index over 30 kg/m², chronic use of non-steroidal anti-inflammatories (NSAIDS), history of cortisone injection during prior 3 months, current pregnancy, contraindications of MRI, concomitant grade III tear to other knee ligaments, meniscus tear requiring meniscectomy, significant chondral lesion or degenerative cartilage changes on post-injury MRI, and open growth plate appearance on from diagnostic radiographs requiring an altered ACLR technique. This study was approved by the Institutional Review Board at the University of Nebraska Medical Center (UNMC) (IRB# 416-17-FB). All adult participants provided written informed consent, while assent with written parental consent was obtained for minors.

A study team member randomly selected and deidentified 20 of these knee MRIs for inclusion in this study. The 20 datasets of MR images were stratified before randomization to represent a) varying limbs (i.e., injured or uninjured), b) clinical presentation of EV using the modified stroke test (i.e., grades 0, trace, 1+, 2+, 3+),¹⁵ and c) time from ACL injury. This resulted in the inclusion of 10 injured and 10 uninjured knee MRIs, which were evenly distributed across modified stroke test grades, and 12 scans within 1 month of ACL injury (before ACL reconstruction) and 8 scans at 6 months after ACL reconstruction.

MRI Acquisition

Participants sat for 30 min before MRI acquisition to unload knee cartilage due to acute effects of loading on T2 relaxation time.²⁵ Bilateral MR image data (injured knee first) were acquired on a 3-Tesla Phillips Ingenia MRI scanner using a 16 channel transmit/receive knee coil (Phillips North America Corporation) in slight knee flexion and neutral rotation.²⁶ For volumetric quantitative measurements, an axial proton density weighted (PD), fat suppressed (FS) MRI sequence in the axial orientation was used (voxel height = 0.1458mm, width = 0.1458, slice thickness = 4.0mm, and spacing between slices = 4.4mm). In addition to axial MR images, a T2 mapping spin echo (SE) sequence, fat suppressed proton density weighted SE sequences in the coronal and sagittal orientations, and a sagittal T1 weighted SE were also included in the MRI protocol.

Volumetric Quantitative Measurement of Knee Effusion

To measure intra-rater reliability, a single assessor completed VQM of EV from de-identified MR images at two separate time points. To measure inter-rater reliability, a second assessor completed VQM of EV using the same set of de-identified MR images.

Analysis of the Digital Imaging and Communications in Medicine (DICOM) images was performed using custom code in MATLAB software (R2022b; MathWorks, Inc., Natick, MA, United States) following methodology established by Jaremko et al.²⁷ First, regions of interest (ROI) were manually drawn using the freehand pen tool for each axial proton density-weighted, fat-suppressed MRI slice.

To match clinical assessments of EV, such as the modified stroke test,¹⁵ ROIs were limited to the anterior joint space. For tibial slices, we adapted the “three column concept” as depicted by Luo et al.,²⁸ limiting the ROI to include the anterior-medial and anterior-lateral regions of the knee whilst excluding the posterior joint space as depicted in figure 1. For femoral slices, the ROI was identified as the joint space anterior to a reference line connecting the anterior-most points of the medial and lateral femoral epicondyles (figure 2). This approach deliberately restricted segmentation to the anterior compartment of the knee joint. This was chosen to align with the modified stroke test, which primarily evaluates fluid displacement toward the suprapatellar pouch and parapatellar gutters, making it most sensitive to anteriorly displaced effusion.¹⁵ Prior imaging studies focused on quantifying effusion volume within the suprapatellar region,^17,29'³¹ supporting an anterior ROI strategy for volumetric measurement. This restriction also reduced variability from inconsistent posterior and suprapatellar coverage in routine axial stacks, and it also enabled reproducible ROI placement across raters. Finally, it excluded non-effusion hyperintense structures (e.g., cysts, vessels) and posterior joint fluid collections (e.g., medial gastrocnemius bursa) not captured by the clinical test. Following manual establishment of a region of interest for each MRI slice, Otsu's thresholding method³² was used to classify each pixel within regions of interest into four intensity categories, with fluid represented by the highest intensity class. Final segmentation of knee Effusion included only the highest-intensity pixels within the manually drawn ROIs. Total fluid volume was calculated as the product of the total segmented voxel count and individual voxel volume (0.08503 mm³).

Figure 1A to 1D.

(1A) Original MRI slice including the tibia. (1B) Three-Column Segmentation Protocol for Tibial Regions of Interest. Manual segmentation of tibial regions of interest used the three-column anatomical approach (as described Luo et al., with boundaries established using three lines that originate from the intercondylar eminence (identified by a white arrow at point i). The first line extends anteriorly toward the tibial tuberosity (indicated by white arrow at point ii), the second projects to the anterior point of the fibular head (marked by white arrow at point iii), and the third connects to the posteromedial ridge of the proximal tibia (shown by white arrow at point iv).²⁸ The segmentation protocol specifically targets the anteromedial to anterolateral tibial regions, ensuring consistent sampling across the anterior aspect of the knee. This systematic approach facilitates reproducible region of interest selection while excluding posterior tibial elements. (1C) Manual region of interest added to the original MRI slice. (1D) Automated and final region of interest computed using Otsu's thresholding method within the manually identified region of interest.

Figure 2A to 2D.

(2A) Original MRI slice including the femur. (2B) Epicondylar-Based Segmentation Protocol for Femoral Regions of Interest. Manual segmentation of femoral regions of interest used the medial and lateral epicondyles (white arrows at point i and ii respectively) to serve as anatomical landmarks to establish the anterior-posterior boundaries of the knee, facilitating the systematic partitioning of analyzed regions. (2C) Manual region of interest added to the original MRI slice. (2D) Automated and final region of interest computed using Otsu's thresholding method within the manually identified region of interest.

Statistical Analysis

Statistical analysis was performed using IBM SPSS Statistics 30 (Armonk, NY). An a priori p-value of 0.05 was used. An independent t-test was used to compare volumetric quantitative measurements of EV between the injured and uninjured knees. Two-way mixed effects intraclass correlation coefficients (ICC_3,1) with 95% confidence intervals were used to assess the absolute agreement of volumetric quantitative measurements of knee effusion for both intra- and inter-rater reliability. ICC_3,1 values range from 0-1, with higher numbers indicating better agreement between variables. ICC_3,1 values were interpreted according to Koo & Li's guidelines: <0.50 indicating poor reliability, 0.50-0.75 indicating moderate reliability, 0.75-0.90 indicating good reliability, and >0.90 indicating excellent reliability.³³

Using methods previously described,³⁴ the minimal detectable change (MDC) was calculated for intra-rater and inter-rater measurements using the sample standard error of the mean (SEM), pooled standard deviation between both testing sessions (SD_pooled), and ICC_3,1:

$$\text{MDC}=1.96\times \text{SEM}\times \sqrt{2}$$

where

$$\text{SEM}=\text{SDpooled }\times (1-\text{ICC})$$

Bland-Altman plots were generated to visualize agreement between repeated measurements and between raters. Limits of agreement were set as ± 1 MDC to visually assess the magnitude of difference between measurements and observe whether most of the measurement differences fell within the range of the MDC (i.e. would be interpreted as a true change or not). We then separated our intra-rater and inter-rater results into two further groups, consisting of the injured and uninjured limbs.

Results:

Participant characteristics are described in Table 1. Volumetric quantitative measurements of knee EV by each rater are presented in Table 2. EV measurements of the involved knees showed large variation (ranges: Rater 1 [Time Point #1] = 2.11-27.42 mL; Rater 1 [Time Point #2] = 1.83-26.31 mL; Rater 2 = 2.07-23.18 mL). Uninvolved knee EV measurements demonstrated minimal variation (ranges: Rater 1 [assessment #1] = 0.92-3.12 mL, Rater 1 [assessment #2] = 1.05-3.47 mL, Rater 2 = 0.94-3.19 mL). Using Rater 1 data from the first time point of measurements, EV were significantly higher in the injured knees (11.88±8.14 mL) compared to the uninjured knees (2.03±0.76 mL).

Table 1.

Demographic and Surgical Characteristics of 17 Included Participants

	Number of Participants (%) or Mean (±SD)	95% Confidence Interval
Sex (female)	10 (58.8)
Age (years)	19.2 (5.1)	16.6-21.8
Height (m)	1.7 (0.1)	1.7-1.8
Weight (kg)	69.6 (11.0)	63.9-75.3
BMI (kg/m2)	24.0 (3.2)	22.4-25.7
Race or Origin
Asian	1 (5.9)
Black or African American	2 (11.8)
Hispanic, Latino or Spanish	1 (5.9)
White	12 (70.6)
Other	1 (5.9)
Concomitant meniscal repair (Yes)	8 (47.1)
Graft type
Hamstring tendon autograft	2 (11.8)
Patellar tendon autograft	10 (58.8)
Quadriceps tendon autograft	5 (29.4)

Abbreviations: %, percentage; SD, standard deviation; m, meters; kg, kilograms; BMI, body mass index.

Table 2.

Volumetric Quantitative Measurements (milliliters [mL]) of Effusion or Each Knee MRI by Each Independent Rater

	Participant	Rater 1 (T1)	Rater 1 (T2)	Rater 2
Injured knee EV (mL)	1	17.21	17.21	15.13
	2	12.48	9.47	9.12
	3	5.26	4.73	5.33
	4	2.11	1.83	2.07
	5	5.56	5.36	5.86
	6	3.35	3.52	3.17
	7	18.11	18.92	16.10
	8	9.90	7.83	7.00
	9	17.42	17.46	15.40
	10	27.43	26.31	23.18
	Mean (SD)	11.88 (8.14)	11.26 (8.17)	10.24(6.86)
Uninjured knee EV (mL)	11	0.92	1.05	0.94
	12	1.76	1.75	2.10
	13	1.52	1.29	1.78
	14	1.43	1.22	1.12
	15	2.91	3.04	2.42
	16	1.43	1.77	1.82
	17	2.97	2.90	2.54
	18	3.12	3.47	3.19
	19	2.16	1.43	1.56
	20	2.05	2.06	1.77
	Mean	2.03 (0.76)	2.0 (0.85)	1.92 (0.67)

Minimal detectable changes (MDC) are present at the bottom of the table. Abbreviations: EV, effusion volume; T1, time point 1; T2, time point 2; MST, Modified Stroke Test; mL, milliliters.

The overall mean value of EV from the first assessment of the primary rater was 6.95±7.56 mL (range: 0.92-27.43), with similar results in the second assessment (6.63±7.39 mL, range: 1.05-26.31) (Table 2). The second rater's measurements of EV showed comparable values to the first rater (6.08±6.38 mL, range: 0.94-23.18). Volumetric quantitative measurements of EV demonstrated excellent intra-rater reliability (ICC_3,1=0.993, 95% CI: 0.981-0.997, p<0.001) and inter-rater reliability (ICC_3,1=0.974, 95% CI: 0.907-0.991, p<0.001). Additionally, the minimal detectable change of the intra-rater and then inter-rater reliability was calculated as 0.145 and 0.504 respectively (Table 3). Bland-Altman plots revealed the absence of bias between raters, as plotted values were consistently between the limits of agreement (figure 3).

Table 3.

Descriptive and Reliability Statistics of Volumetric Quantitative Measurements of Effusion. Intraclass Coefficient Correlations Demonstrated Excellent Intra-Rater and Inter-Rater Reliability

	Rater 1 - Time Point 1 (Mean)	Rater 1 - Time Point 2 (Mean)	SDpooled	ICC3,1 (95% CI)	SEM	MDC (95%)
EV (mL)	6.95	6.63	7.563	0.993 (0.9810.997)	0.052	0.145
	Rater 1 - Time Point 1 (Mean)	Rater 2 (Mean)	SDpooled	ICC3,1 (95% CI)	SEM	MDC (95%)
EV (mL)	6.95	6.08	6.380	0.974 (0.9070.991)	0.182	0.504

Abbreviations: EV, effusion volume; mL, milliliters; SD, standard deviations; ICC, intraclass correlation coefficients; SEM, standard error of mean; MDC, minimal detectable change.

Figure 3A to 3B.

Bland-Altman plots for intra-rater and inter-rater volumetric quantitative measurements of knee effusion. The dashed red lines indicate the limits of agreement, which were calculated as below:

Upper Limit = MeanDifference + (1.96 x SD) Lower limit = MeanDifference - (1.96 x SD)

The dashed black line indicates the mean difference for all intra-rater measurements (between 2 separate time points) or during inter-rater measurements (between 2 independent raters). The solid black line is included as a reference and indicates no difference between measurements. The assessment difference in volumes for the injured and uninjured limbs are documented in blue and red respectively. Abbreviations: mL, milliliters.

Discussion

This study demonstrated excellent intra-rater (ICC_3,1=0.993) and inter-rater (ICC_3,1=0.974) reliability of our semi-automated VQM protocol within young ACL-injured and uninjured knees using standard axial proton density-weighted, fat-suppressed MRI sequences, routinely available in clinical practice. These findings strengthen the feasibility of using non-contrast MRI sequences to provide reproducible VQM across a wide range of EV within research and clinical settings.

Semi-quantitative clinical tests, while valuable, remain limited by potential inter-examiner variability affecting the precision needed for many research applications and detailed clinical monitoring. Unlike aspiration, which is invasive and subject to sampling error. MRI segmentation offers a non-invasive and repeatable assessment of joint fluid, providing potential opportunities for further research and clinical applications. Previous studies reported good to excellent results for VQM. Gilles et al. reported high intra- and inter-rater reliability for their semi-automated VQM derived from a 3D DESS sequence in established OA.³ Our study therefore enriches the literature by applying this within a clinically based context involving the very commonly met, young, ACL-injured patient undergoing rehabilitation. Our clinical context is first supported by our rationale for restricting the VQM of EV to only the antero-medial and antero-lateral aspects of the knee, wherein much of the clinically relevant EV is identified during physical examinations. Anteriorly displaced fluid in the suprapatellar pouch and parapatellar gutters is most relevant to common physical examination maneuvers and to early rehabilitation decision-making.^2,8,15 Secondly, our axial PD-FS images used 4.0-mm slice thickness with a 0.4mm interslice gap, a through-plane resolution similar to that employed in quantitative and semi-quantitative knee MRI studies of meniscal damage, bone marrow lesions, and periarticular lesions.^{20,24,27,35,36} ACL patients routinely undergo 2D fluid-sensitive sequences (T2-FS or PD-FS), rather than 3D DESS or contrast-enhanced acquisitions. Multiple MRI studies have used slice thickness around 3-4 mm and an interslice gap of no more than 0.4 mm for PD-FS or T2-FS sequences in routine protocols.^21,35–40

The exceptional reliability demonstrated with standard PD-FS sequences challenges the need for specialized fluid-sensitive imaging sequences like FLAIR or contrast-enhanced T1 sequences for fluid assessment, even at smaller volumes.⁴¹ While not intended for immediate clinical decision-making, this finding provides value for longitudinal monitoring in prospective cohorts and clinical trials, where objective, consistent biomarkers are needed.

Bland-Altman plots highlight increased variability in VQM during the assessment of larger EV, particularly above 8 mL, which may be explained by increased fluctuation of effusion across joint recesses. Interestingly, this aligns with the ultrasound detection thresholds established by Delaunoy et al. of 7.4 mL for synovial fluid and 9.7-10.1 mL for blood/saline in cadaveric specimens.⁴² Further research examining measurement variability across different EV would help validate this pattern and whether this variability could be mitigated through automation.

We sought to perform a comparison of injured vs uninjured knees to provide an internal control as there is some natural, albeit small volume of residual fluid in the knee.² The distribution of volumes in our sample highlights a broad range of EV within injured knees, while contralateral knees exhibited low but non-zero volumes consistent with physiologic joint fluid. Reliability remained excellent across this spectrum, indicating that segmentation decisions were reproducible both when volumes were substantial and when only a few high-intensity voxels were present. Bland-Altman plots showed no systematic bias between repeated measurements or between raters with most differences lying within narrow limits of agreement. We observed slightly greater dispersion of differences at higher volumes (approximately >8 mL). This seems plausible given that large effusions may shift more across recesses and may be more sensitive to small differences in boundary placement. Similar volume-dependent behavior has been reported in other modalities; for example, ultrasound studies of injected effusions have noted that measurement error and detection thresholds become more pronounced at higher injected volumes.⁴² Future work could examine volume-stratified reliability and explore whether fully automated segmentation algorithms can further reduce variability, particularly in knees with large effusions.

This study has several limitations to consider. First, we assessed reliability of our segmentation technique only and not validity. We did not compare VQM with aspirated joint fluid. Although, we cannot confirm accuracy, this approach reflects our intention to develop a noninvasive assessment method that eliminates aspiration-related risks. Future validation studies comparing VQM measurements with aspiration volumes could strengthen the protocol's validity while maintaining its non-invasive advantages.

Second, our sample size was relatively modest and limited to the younger ACL-injured participants. Larger cohort studies across diverse patient populations and pathologies would help establish normative data and validate the protocol's broader clinical utility.

Third, our protocol deliberately restricted VQM to the anterior compartment of the knee to match the clinical utility of physical effusion examinations.^43,44 This anterior ROI approach improves reproducibility and is supported by prior research,^17,29–31 but it may not capture the totality of EV within the joint since fluid located within the posterior recesses was excluded.^38–40

Fourth, segmentation was semi-automated and required manual placement of regions of interest by trained, but non-expert raters. Manual or semi-automated segmentation is time-consuming and introduces potential for user-dependent variability. The excellent intra- and inter-rater ICCs observed here suggest that, with clear anatomical rules and standardized training, non-expert readers can apply this protocol consistently. However, the dependency on manual input limits scalability to larger cohorts and reduces the likelihood of adoption in routine clinical reporting. Future development of automated measurement tools could address this limitation while potentially further reducing measurement variability.²⁷

Conclusion:

In conclusion, our semi-automated VQM using standard axial PD-FS MRI sequences provide highly reliable measurements of EV in ACL-injured and uninjured knees. Although not a substitute for clinical examination in isolation, this non-invasive measurement of EV offers potential for objective clinical research and longitudinal monitoring without requiring specialized imaging sequences.