Identifying Anterior Cruciate Ligament Injuries Through Automated Video Analysis of In-Game Motion Patterns

Abstract

Background:

Failure to diagnose anterior cruciate ligament (ACL) injury during a game can delay adequate treatment and increase the risk of further injuries. Artificial intelligence (AI) has the potential to be an accurate, cost-efficient, and readily available diagnostic tool for ACL injury in in-game situations.

Purpose:

To develop an automated video analysis system that uses AI to identify biomechanical patterns associated with ACL injury and to evaluate whether the system can enhance the ability of orthopaedic and sports medicine specialists to identify ACL injuries on video.

Study Design:

Descriptive laboratory study.

Methods:

A total of 91 ACL injury and 38 control movement scenes from online available match recordings were analyzed. The videos were processed to identify and track athletes and to estimate their 3-dimensional (3D) poses. Geometric features, including knee flexion, knee and hip abduction, and foot and hip rotation, were extracted from the athletes’ 3D poses. A recurrent neural network algorithm was trained to classify ACL injury, using these engineered features as its input. Analysis by 2 orthopaedic surgeons examined whether providing clinical experts with the reconstructed 3D poses and their derived signals could increase their diagnostic accuracy.

Results:

All AI models performed significantly better than chance. The best model, which used the long short-term memory network with engineered features, demonstrated decision interpretability and good performance (F1 score = 0.63 ± 0.01, area under the receiver operating characteristic curve = 0.88 ± 0.01). The analysis by the 2 orthopaedic surgeons demonstrated improved diagnostic accuracy for ACL injury recognition when provided with system data, resulting in a 0.08 increase in combined F1 scores.

Conclusion:

Our approach successfully reconstructed the 3D motion of athletes from a single-camera view and derived geometry-based biomechanical features from pose sequences. Our trained AI model was able to automatically detect ACL injuries with relatively good performance and prelabel and highlight regions of interest in video footage.

Clinical Relevance:

This study demonstrated the feasibility of using AI to automatically evaluate in-game video footage and identify dangerous motion patterns. Further research can explore the full potential of the biomechanical markers and use of the system by nonspecialists, potentially diminishing the rate of missed diagnosis and the detrimental outcomes that follow.

Injury to the anterior cruciate ligament (ACL) is a common and serious event that occurs in a wide variety of sports at all levels of play. ⁸ Patients who sustain ACL rupture experience the greatest time away from play per incident of any athletic knee injury, with a recommended postoperative recovery period of at least 6 to 12 months.^4,8 However, even after this extensive period of rehabilitation, reinjury of the ACL remains a significant risk after reconstruction, potentially prolonging recovery time even further.^8,24,34 It is estimated that only one-third of patients with ACL injuries return to competitive sports within 12 months, ⁴ posing additional challenges to this condition.

Despite its typical injury pattern, achieving an accurate diagnosis of ACL rupture remains a challenge. Previous studies have reported that the correct diagnosis rate at initial medical evaluation is less than 30%,^1,3 particularly in cases where patients are initially seen by physicians in hospital emergency departments, by primary care physicians, or by physiotherapists.^1,3 There are specific and reliable movements associated with ACL injuries, such as high knee abduction moment and dynamic valgus.^9,14,15,22 Laboratory investigations, mathematical modeling, and video analyses have been used to evaluate those specific movements,^5,13,19 but only video analysis, which uses recordings from a sports event, allows for the evaluation of kinematics in a real game setting. ¹⁹

Failure to quickly and accurately diagnose ACL ruptures leads to a delay in treatment and increased risk of further injuries to the knee. ² Therefore, there is a need for quick, accurate, and readily available technologies to diagnose ACL injuries. Moreover, identification of pathologic movement patterns during an athlete’s performance may indicate an increased risk of injury. Video analysis has the potential to fulfill both demands. The main challenge in using video data is its sheer volume. Manual analysis can be a complex and laborious process. To address this challenge, artificial intelligence (AI)-driven analytics should be considered to automatically analyze motion patterns in vast amounts of data.^25,26

In the field of orthopaedics, marker-based motion capture systems are the gold standard for 3-dimensional (3D) movement data acquisition, producing highly accurate measurements. ²¹ However, in situations like competitive play, injuries can occur as athletes push their bodies beyond limits, which cannot be accurately measured in a laboratory using motion capture markers due to environmental limitations. ²¹ Advancements in deep learning (DL) have improved the accuracy of video-based markerless 3D motion capture systems with single-view or multiview camera setups.^11,28,29,33 Recent data show that markerless solutions can provide satisfactory quality to be used in practice.^21,23

Our primary goal in this study was to develop an automated video analysis system that can function in such a complex and fast-paced environment. We propose a solution that can track athletes and reconstruct their motion in 3D, extract biomechanical patterns, and provide visualizations. Our trained model can automatically detect ACL injuries based only on abnormal movements, and it may also be able, in the next step, to identify acute movement patterns predictive of ACL injury. A secondary goal was to determine whether our system can assist sports medicine specialists in identifying ACL injuries via video footage alone with improved accuracy. Our hypothesis was that, by leveraging AI technologies, we can reconstruct the movement of athletes from broadcast videos with sufficient accuracy to create decision support systems.

Methods

Video Extraction and Evaluation

Video data of ACL injuries were collected from the YouTube video-sharing platform, using phrases related to ACL injuries, such as “ACL tear,”“ACL rupture,”“ACL injury basketball,” etc. Collection was performed between April 2021 and January 2022. We specifically sought to incorporate videos for both female and male athletes across a variety of sports broadcasts, including American football, Australian football, basketball, handball, rugby, soccer, tennis, and volleyball. Athletes with knee injuries were identified and included in the dataset only if their diagnosis was confirmed with an official injury report. Videos that captured injuries other than ACL ruptures or with no diagnosis were excluded. Control videos were also collected showing players falling or executing high-risk movements, with careful follow-up to confirm that no injuries were sustained as a result. These videos were selected by 3 orthopaedic sports medicine experts (M.F.Z., L.L, G.M.), then downloaded by computer engineering experts (A.S., M.C.) for further analysis.

To isolate specific events, the videos were trimmed to remove scenes from multiple games. In addition, injury events and highlights are usually replayed from different camera views and had to be divided into separate input streams. In total, 364 video parts were rendered, each capturing the movement of a target athlete from a unique angle. Most replay scenes were shown in slow motion, making it difficult to determine the true frame rate and apply temporal synchronization to the different camera views, so this factor was not controlled in this study.

Video Annotation

To diagnose ACL injuries automatically, we employed supervised machine learning techniques that rely on informative labels. Unlike tasks where labels are independent, the subsequent labels for our data reflect the dynamics (temporal evolution) of the action in each video frame. Binary labels (on/off) were used for 3 categories: injury, left foot ground contact, and right foot ground contact.

Upon the onset of a traumatic event, the relevant joint is known to pivot and flex resulting in an excessive strain on the ACL, thereby leading to rupture.^5,9,16 This moment of rupture can be considered the apex of the traumatic event. Determining these moments accurately is difficult since evaluators can rely only on abnormal movement patterns. Therefore, we used a custom annotation system (Argus Cognitive Hungary Kft) and instructed evaluators to watch the entire clip frame by frame and mark the interval (onset and offset) in which the moment of injury most likely occurred. To expedite the annotation process, the initial labels were created by evaluators from the computer engineering team (A.S., M.C.) and reviewed by an orthopaedic surgeon (G.M.) to refine the labels as needed. In cases where the judgment of the exact moment of injury was unclear, the video was marked with a label of “ambiguity” and was excluded from further analysis (n = 23). In addition, videos in which the athlete’s movement was not well visualized due to poor image quality (low resolution, far distance footage) or disrupted line-of-sight to the relevant knee, often due to obstruction by other players, were also marked with a “bad visibility” label and were excluded from the analysis (n = 131).

Foot-ground contact is another important piece of information in investigating athletes’ interaction with their environment. Using the same video annotation system, we marked the moments when each athlete’s feet made contact with the ground. To efficiently screen through the ground-contact annotation process, a hybrid annotation approach was used in which models were trained on a subset of the data and then used to automatically preannotate new videos once a sufficient number of labels were collected. These preannotations were then reviewed and corrected by a human annotator (A.S.). Figure 1 shows the overall data processing pipeline.

Figure 1.

Flowchart of data processing pipeline. Blue rectangles, data types; yellow rectangles, multistage processes. 3D, 3-dimensional.

The overall clean-labeled dataset consisted of 210 video parts from 129 individual athletes (~32,000 frames). A total of 17 athletes had 3 camera views, 43 athletes had 2 views, and 69 athletes had 1 view, with a mean ± SD clip duration of 5.3 ± 3.5 seconds (range, 1-20 seconds). In general, injuries had more replays from different camera angles, while control movements were captured from a maximum of 2 angles and typically showed a single view. We used this dataset to train our machine learning algorithms to automatically differentiate between ACL injuries and control, normal physiological movements. Figure 2 shows a representative example of the interval annotation tool used.

Figure 2.

Example of the interval annotation tool. A human annotator validates and corrects the preannotated foot-ground contact labels (bottom panel) belonging to the player marked with the rectangle (top panel).

Automatic Injury Recognition

ACL injury recognition was formulated as a binary (yes/no) classification problem. First, athletes were detected and tracked with a semiautomated solution. Second, the 3D poses of the athletes were estimated and features were extracted. Finally, our proposed solution leveraged simple geometric features and foot-ground contact information to train a recurrent neural network for classification.

Video Frame Preprocessing

We used a combination of automatic detection algorithms and manual review to ensure that each athlete was tracked accurately and consistently over time. Starting with an off-the-shelf person detector (YOLO; Version 3), ²⁷ the bounding boxes of athletes were detected on each frame of all video clips. After successfully identifying the athletes, the ByteTrack algorithm connected detections belonging to the same person. ³⁷ Due to the high complexity of the scenes, manual identification of the target athlete was necessary. Two human examiners (L.L., M.F.Z.) reviewed the preannotated video frames to ensure the precise localization and consistent tracking of the athlete over time, even after temporary occlusions. Detection and tracking were followed by 3D body pose estimation on the cropped image of the athlete. ²⁸

We used 3D pose estimation to generate a skeleton with 28 joints for each frame. Each joint is a 3D position vector with respect to the global coordinate frame. Joint coordinates were further preprocessed following a scheme that transformed them into the body coordinate system. ³⁶ The skeleton was centered around the root (pelvis) joint and rotated to the frontal view based on the horizontal and vertical axes defined in the skeleton as the vector from left hip to right hip and the vector from pelvis to center of spine, respectively.

Learning Directly From Pose

The 3D skeleton with foot-ground contact presence is a compact representation with high-level information about body motion. In our baseline solution a fully connected neural network (FCNN) was used to learn features directly from the frontal view skeleton and ground-contact labels in a single frame.

The baseline was compared with the frequently used long short-term memory (LSTM) network to model the spatiotemporal dependencies of consecutive frames (Figure 3).^18,36 Fixed length sequences were generated from the skeletons for training the LSTM. The sequence length was set to 21 timesteps, which corresponds to a video segment of approximately 1 second. In each sequence, the target frame was located at the center.

Figure 3.

Main steps of the ACL injury prediction algorithm from video input to predicted ACL injury timeseries. The example demonstrates a jump and subsequent landing resulting in ACL rupture. 3D, 3-dimensional; ACL, anterior cruciate ligament; LSTM, long short-term memory.

Both FCNN and LSTM architectures were used for foot-ground contact estimation, with the difference that contact labels were learned directly from root relative poses without reorientation to frontal view.

Feature Engineering

Applying domain knowledge and deriving features for machine learning is an effective strategy for moderately sized datasets such as ours. Reducing input dimensionality can help mitigate overfitting and improve the generalization performance and interpretability of the model. Our final set of descriptors consisted of angle-based measures between body segments (eg, knee flexion, knee abduction, foot rotation, hip abduction, hip rotation, and torso flexion). These features were calculated for all 3D poses and used during training to explore unique patterns in combination with foot-ground contact information for ACL injury classification. Both FCNN and LSTM neural network architectures were tested with these derived features.

Evaluation by Orthopaedic Surgeons

To assess whether our system could aid sports medicine specialists in identifying ACL injuries, 2 sports medicine fellowship-trained orthopaedic surgeons (C.B.G.L. and E.M.F., with 1 and 2 years of practice and expertise in ACL reconstruction, respectively) acted as reviewers. They independently watched video clips of both injury and control movements and categorized them as either having an ACL injury or not.

The video clips were prepared carefully to remove any additional contextual information regarding the injury besides the distinct mechanism of injury itself. For example, players’ faces, numbers, and other potentially confounding identifiers were blurred to avoid recognizing facial expressions, or an athlete’s identity. Videos were also trimmed to brief clips and were cut before athletes began to demonstrate a fall to the ground, as interpretation of whether athletes were able to maintain their stance versus fall to the ground with injury would confound the isolated interpretation of knee kinematics. A total of 54 videos (27 ACL injuries and 27 control) were initially shown to the surgeons. The reviewers reported being familiar with the outcomes of 5 videos, which were then excluded from the analysis, resulting in a total of 49 videos being analyzed.

Each reviewer first watched a raw video clip and made an initial diagnosis of ACL rupture or no ACL rupture based on the raw footage alone. They then rewatched the video using an augmented view of the clip that included the reconstructed 3D poses from the frontal and sagittal planes with the timeseries graphical data also showing torso flexion, rotation of the hip and foot, knee and hip abduction angle, and ground-contact information. The information presented in this augmented view (Figure 4) also formed the input data for the AI model. After interpreting the second augmented clip, they provided a second diagnosis.

Figure 4.

Augmented view for review by orthopaedic surgeons. Upper left, extracted feature timeseries (foot-ground contact and angles between body segments). Upper right, input video with blue rectangle around the target. Lower right, reconstructed 3D pose from camera point of view, side view, and frontal view. 3D, 3-dimensional.

Training Details

All AI models were trained and evaluated with repeated 10-fold cross-validation. Athletes with ≥2 camera views were assigned exclusively to either the training or test fold, but not both. In the training phase, all available views were utilized as separate records. However, during the testing phase, we selected the optimal view for each athlete based on visibility. This approach ensured that the results were not biased by the presence of the same injury multiple times. To balance the dataset, we used data augmentation by duplicating the poses that belonged to the minority class (injury) with horizontal mirroring. The ACL injury class ratio was 23% for the training set and 13% for the test set, on average. The FCNN and LSTM models were trained without resampling to maintain the original distribution of the samples. To avoid overfitting, we applied k-fold cross-validation, with k = 10 folds and repeated the procedure 10 times to counter bias due to random selection.

Statistical Analysis

Both injury and foot-ground contact classifiers predict binary, positive, or negative labels on individual frames. The decision made by the classifier can fall into 4 categories: true positives (TPs) are examples labeled correctly as positives; false positives (FPs) refer to negative examples labeled incorrectly as positive; true negatives (TNs) correspond to negatives labeled correctly as negative; and false negatives (FNs) refer to positive examples labeled incorrectly as negative. Using these categories, we can derive performance metrics that are affected differently by class imbalance, so we reported several. Precision ( $\mathrm{P}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$ ) measures the proportion of recognized instances that are relevant, while recall or sensitivity ( $\mathrm{R}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$ ) measures the fraction of relevant instances that are retrieved by the classifier. The F1 score is the harmonic mean of precision and recall ( $\mathrm{F}1=2\left[\frac{\mathrm{P}·\mathrm{R}}{\mathrm{P}+\mathrm{R}}\right]$ ). Balanced accuracy is calculated as the arithmetic mean of recall and specificity ( $\frac{\mathrm{TN}}{\mathrm{TN}+\mathrm{FP}}$ ) and provides a balanced evaluation of a model’s performance on an imbalanced dataset. The area under the receiver operating characteristic curve (ROC AUC) measures the ability of a classifier to distinguish between positive and negative samples across different decision probability thresholds. The area under the precision-recall curve (PR AUC) measures the ability of a classifier to correctly identify positive samples while minimizing FP.

In addition to frame-level metrics, we also calculated supplementary metrics that considered interval (event-to-event) comparison. Similarly, for object detection tasks, we matched ground truth to predicted regions and defined TP, FP, and FN outcomes based on the intersection-over-union (IoU) threshold. A detection was considered TP when the IoU threshold was ≥0.2 (Version 0.22.2; Scikit-learn library).

Results

Athlete Population

Our dataset included 210 videos in total (142 injury, 68 control), involving 129 professional athletes in competitive play. The dataset exhibited a 2:1 bias favoring male athletes, as the data collection process focused primarily on volleyball recordings for female athletes, most of which were discarded due to poor video quality or challenging camera angles. The majority of videos were from soccer and Australian football, followed by American football and basketball (Table 1).

Table 1.

Sample Counts per Sports Category ^a

Sport	Videos	Athletes	Female Athletes	Male Athletes
Soccer	58	37	12	25
Australian football	57	35	6	29
American football	31	16	0	16
Basketball	29	16	2	14
Volleyball	18	13	13	0
Handball	5	5	4	1
Tennis	5	3	2	1
Badminton	3	1	1	0
Padel	2	1	1	0
Field hockey	1	1	1	0
Rugby	1	1	0	1
Total, n (%)	210 (100)	129 (100)	42 (33)	87 (67)

^aData are shown as number of athletes unless otherwise indicated.

Model Performance

Four different models were compared: a static (FCNN) and a dynamic (LSTM) architecture, each with 2 versions depending on the input data, either normalized 3D poses or derived features. Dynamic models outperformed static models in both input versions (Tables 2 and 3). Both LSTM versions had similar performance, but we preferred the one trained on derived features due to its better interpretability. Grouping frames into events improved model performance, highlighting that the task is better defined as classifying events of interest instead of individual frames. The high degree of skew of the datasets (13% positive frames) attenuated performance metrics considering each frame as an individual sample. The model achieved a precision score of 0.62, correctly classifying 63% of predicted injury events, and a recall score of 0.71, correctly identifying 71% of actual injuries in the dataset. Despite the dataset imbalance, the balanced accuracy score of 0.80 suggests that the model can classify positive and negative samples reasonably well. The ROC AUC score of 0.88 indicates high accuracy in distinguishing between injury and noninjury frames, while the PR AUC score of 0.57 suggests moderate precision and recall in identifying the positive class.

Table 2.

Repeated 10-Fold Cross-Validation Test Results on all ACL Injury Samples ^a

	Frame Level						Event Level
Model	F1 Score	Precision	Recall	BA	ROC AUC	PR AUC	F1 Score	Precision	Recall
FCNN	0.60 ± 0.01	0.56 ± 0.02	0.65 ± 0.01	0.79 ± 0.00	0.87 ± 0.00	0.54 ± 0.03	0.65 ± 0.01	0.58 ± 0.02	0.73 ± 0.02
FCNN: derived	0.57 ± 0.01	0.58 ± 0.01	0.56 ± 0.02	0.75 ± 0.01	0.88 ± 0.01	0.54 ± 0.01	0.63 ± 0.01	0.57 ± 0.02	0.69 ± 0.03
LSTM	0.62 ± 0.01	0.56 ± 0.02	0.70 ± 0.02	0.81 ± 0.01	0.88 ± 0.01	0.56 ± 0.02	0.68 ± 0.02	0.63 ± 0.03	0.74 ± 0.03
LSTM: derived	0.63 ± 0.01	0.58 ± 0.01	0.68 ± 0.03	0.80 ± 0.01	0.88 ± 0.01	0.57 ± 0.02	0.66 ± 0.02	0.62 ± 0.03	0.71 ± 0.02

^aData are reported as mean ± SD. Boldface results are for the final model. ACL, anterior cruciate ligament; BA, balanced accuracy; FCNN, fully connected neural network; LSTM, long short-term memory; PR AUC: area under the precision-recall curve; ROC AUC, area under the receiver operating characteristic curve.

Table 3.

Repeated 10-Fold Cross-Validation Test Results on All Foot-Ground Contact Samples ^a

	Frame Level						Event Level
Model	F1 Score	Precision	Recall	BA	ROC AUC	PR AUC	F1 Score	Precision	Recall
FCNN	0.62 ± 0.01	0.61 ± 0.01	0.64 ± 0.01	0.72 ± 0.01	0.81 ± 0.01	0.64 ± 0.02	0.69 ± 0.01	0.77 ± 0.02	0.62 ± 0.02
LSTM	0.67 ± 0.02	0.65 ± 0.01	0.70 ± 0.01	0.75 ± 0.01	0.84 ± 0.01	0.71 ± 0.02	0.74 ± 0.01	0.79 ± 0.01	0.69 ± 0.02

^aData are reported as mean ± SD. Boldface results are for the final model. BA, balanced accuracy; FCNN, fully connected neural network; LSTM, long short-term memory; PR AUC, area under the precision-recall curve; ROC AUC, area under the receiver operating characteristic curve.

Performance was similar across most sports categories, but a significant reduction was observed in categories with only 1 video sample. In addition, performance was lower for female injuries, likely due to the higher representation of male athletes (67%) in the dataset (Table 4 and Supplemental Table S1, available online).

Table 4.

Performance Differences Within Subgroups ^a

Subgroup	F1 Score (Frame Level)
Sex
Female	0.57 ± 0.03
Male	0.64 ± 0.01
Sport
Australian football	0.64 ± 0.03
Soccer	0.63 ± 0.04
Basketball	0.63 ± 0.04
American football	0.68 ± 0.04
Volleyball	0.64 ± 0.08
Tennis	0.67 ± 0.10
Other	0.47 ± 0.06

^aData are reported as mean ± SD.

Evaluation by Orthopaedic Surgeons

There was a significant difference in the baseline performance of the 2 reviewers, and the use of the system had a disparate impact (Table 5). Reviewer A was able to identify 3 new injuries but had 2 FP detections. Reviewer B performed better compared with reviewer A, with no new cases identified but with a significant increase in precision. We prioritized recall improvement, as the cost of FNs is high. The F1 score, which represents a balance between precision and recall, increased for both reviewers.

Table 5.

Classification Performance of the 2 Reviewers With and Without AI Analysis System ^a

Scenario	Precision	Recall	F1 Score	TN	FP	FN	TP
Reviewer A	0.615	0.364	0.457	22	5	14	8
Reviewer A + AI	0.611	0.500	0.550	20	7	11	11
Reviewer B	0.680	0.708	0.694	17	8	7	17
Reviewer B + AI	0.895	0.708	0.791	23	2	7	17

^aBoldface scores indicate significantly increased overall performance when AI system provided additional information to the reviewer. AI, artificial intelligence; FN, false negative; FP, false positive; TN, true negative; TP, true positive.

Discussion

Our study demonstrated the feasibility of utilizing an AI-based video analysis system for analyzing biomechanical pathological patterns linked to ACL injuries in sports video footage. All AI models performed significantly better than chance, and our best model (LSTM with engineered features) exhibited good performance (F1 score = 0.63 ± 0.01, balanced accuracy = 0.80 ± 0.01, ROC AUC = 0.88 ± 0.01) and also provided interpretable decision-making.

In terms of mechanism of injury, it is known that ACL ruptures usually occur after sudden twists or turns while the foot is planted on the ground or when landing inappropriately after a jump, mostly in noncontact situations. ⁵ More precisely, previous studies have identified 4 specific situational patterns that increase the likelihood of ACL injury: (1) pressing and tackling, (2) regaining balance after kicking, (3) landing from a jump, and (4) being tackled.^9,19 In addition, the dynamic knee valgus that occurs during those movements has been commonly pointed out as the main biomechanical pattern responsible for ACL ruptures.^9,19,34 Della Villa et al ⁹ evaluated professional male soccer players using video analysis to describe the mechanisms, situational patterns, and biomechanics of ACL injuries and found that knee valgus loading was the dominant injury pattern, present in 81% of the study population. Similarly, Lucarno et al, ¹⁹ evaluating video analysis of professional female soccer players, demonstrated that knee valgus loading was present in 88% of ACL injuries, indicating significant similarities to professional male soccer players. Krosshaug et al, ¹⁶ on the other hand, evaluating videos of basketball players from different game levels (high school, college, and professional), agreed that knee valgus is commonly observed in the onset of ACL injury. However, they found that knee valgus collapse occurred more frequently in female than in male players (relative risk, 5.3; P = .002), which was also the case for knee flexion angle (15° for female vs 9° for male, P = .034). This evidence suggests that video analysis can be a valuable tool to analyze specific movements during real game play, although those studies were designed to evaluate mechanisms and movement patterns of ACL-injured players and did not focus on diagnosing an ACL rupture itself from other non-ACL rupture knee injuries or normal physiologic motions. In this study, knee valgus was represented as knee abduction angle. While it was not our intention to measure the actual degree or identify a threshold for ACL rupture, the presence of knee abduction identified in the reconstructed 3D pose as well as in the extracted feature timeseries was crucial for the system to recognize the ACL injury (Supplemental Table S2).

Two-dimensional (2D) video analysis is a commonly used technique to identify pathologic movement patterns associated with certain injuries. Unfortunately, it shows poor correlation with the more accurate and superior 3D analysis such as gait analysis.^10,31 However, 3D analysis does not provide real-time evaluation of athletes’ movement patterns during practice or competition or over the course of a season. This type of analysis would be essential to detect changes in an athlete’s movement that can lead to injuries like an ACL rupture, especially as players fatigue or sustain minor injuries during play. Consequently, improvement of the current 2D video analysis methods is essential to analyze athletes’ motion and changes to motion patterns in real time. This marks the overarching goal of our research group—to monitor and identify trends and changes in athletes. The ability to identify such characteristic changes leading to an ACL rupture would be highly valuable toward efforts to predict ACL rupture, and represents a potential avenue for injury prevention.

In this study, we demonstrated that our AI-based technology that converts single-view video footage of athletes into 3D digital representations may accurately identify ACL injury by considering pathologic movement patterns such as dynamic valgus of the knee. Although our AI model does not surpass the capabilities of human experts, it was trained using a relatively small amount of data—approximately 20 minutes of ACL injury video content—and limited in terms of quality, since it relied on videos collected from social media. Our system implements all the steps of video-based human motion analysis, namely human detection, tracking, 3D pose estimation, feature extraction and high-level processing, in a modular way that allows improving and switching the components easily toward enhancing the system. Our pilot study involving 2 orthopaedic surgeons showed improved performance in recognizing ACL injuries when using the system. This improvement resulted in a 0.08 increase in combined F1 scores. While further research with larger sample sizes is necessary to establish statistical significance, we consider these findings as preliminary evidence supporting the potential effectiveness of the system in assisting field decisions and identifying potential risk patterns leading to ACL injuries. Ultimately, this system may help us identify the signs of an impending tear before the actual event occurs.

Patients who rupture their ACL do not always have the injury correctly diagnosed and attempt to return to play prematurely. ¹ Failure to identify ACL tears may lead to severe consequences in terms of further injuries, delayed treatment, poor recovery, and slow return to sports.^{3,12,30,32,35} Because performing a clinical examination in the acute stage is frequently hampered by the substantial knee pain and swelling that follows the injury, a magnetic resonance imaging (MRI) evaluation may bring valuable information about the status of the ACL.^7,38 However, MRI entails access to heavily specialized and expensive equipment at medical facilities that may not be available in remote settings or on field during a game situation. In this sense, a video camera-based digital tool that can support injury detection “on field” with lower costs and without the need for specialized hardware could bring significant advantages to athletes, trainers, clinicians, and team franchises across all geographic settings. By leveraging AI technology, this demand can be met with a digital diagnostic tool that can be applied in most settings. Currently, for nonprofessional players, the availability of video coverage is very limited, but adapting this technology could provide significant benefits. AI has already gained popularity in medical applications, revolutionizing the way high-volume medical data are processed and interpreted, further supporting the analysis of medical data submitted by health care providers, patients, and other caregivers. ⁶ It has also been applied for image identification and multivariate risk analysis in a wide field of research in orthopaedics. ¹⁷ Image-based analysis software is being developed and has the potential to expedite interpretation of injured knee MRIs, such as ACL and meniscal injuries, and with diagnostic accuracy similar to experienced radiologists and orthopaedic surgeons. ²⁰ In addition, such a system may be used as an aid for physicians who less frequently encounter ACL injuries early in their career or as a diagnostic adjunct in remote medicine situations.

Here, we aimed to evaluate whether ACL rupture could be reliably detected by video analyses using DL algorithms at the time of injury. For that, an AI-based system was developed to recognize and compare specific motion patterns associated with ACL rupture in athletes known to have sustained this injury against athletes who sustained trauma during the course of play but did not sustain an ACL rupture. The overarching goal of our research group was to diagnose ACL injuries accurately and in real time and to prevent such devastating injuries by identifying athletes with pathologic movement patterns. We believe that this automatic biomechanical pattern recognition opens the door for injury prevention in the future by identifying athletes with pathologic movement patterns that might put them at risk for ACL rupture.

In the future, we wish to further investigate the utilization of AI-based injury recognition across longer, whole-match recordings, with the ultimate goal of developing a tool not only for accurate diagnosis but potentially for early detection of at-risk motion patterns to develop a tool of injury prevention. The potential ability to identify an athlete at risk and intervene before a devastating injury, such as an ACL rupture, could provide profound value to athletes, trainers, clinicians, and sports franchises alike.

Limitations

This study has some important limitations. We specifically chose to focus on ACL injuries to validate our methodology and did not explore the classifier’s potential confusion between ACL injuries and other types of injuries. However, we believe the same approach can be used to extend the capabilities of the system to recognize different injuries (posterior cruciate ligament, meniscus, Achilles tendon, etc). Currently, the tracking process requires human supervision. This was necessary because the system occasionally misidentified the target subject when another player blocked the view. Approximately one-third of our initial dataset had to be excluded due to poor visibility. In the future, having access to recordings from a synchronized multicamera system could potentially solve this issue. Training the system to also track distinct athletes based on their jersey patterns and player numbers could also enhance the accurate tracking individual player motions despite complex encounters, tackles, and contact among athletes common to various evolutions of real play. Incorporating an athlete’s direction of movement and velocity would also provide beneficial information in injury prediction, but the intrinsic and extrinsic parameters of the cameras in our dataset were unknown. As dynamic camera movements and zooming can cause significant differences in the estimated global position between consecutive time frames, such unreliable measurements were excluded from the set of input features. Our system also demonstrated improved accuracy for male athletes over female athletes—an outcome we attribute to the larger sample size of male athletes in this study compared with female athletes. This disparity can be addressed by incorporating balanced sample sizes across sexes. Furthermore, athletes usually rotate their bodies during the game activities, which may impact the frontal plane evaluation, impairing the true valgus angle measurement. We are exploring ways to improve our current 3D motion reconstruction capabilities, involving the synchronization of multiple camera views.

Conclusion

This study demonstrated that our video analysis system is effective in reconstructing the 3D motion of athletes from a single camera view. It was able to analyze biomechanical patterns associated with ACL injuries. In addition, our system demonstrated an early promise to enhance orthopaedic surgeons’ ability to accurately diagnose ACL ruptures when reviewing the video footage of athletes in real-game situations. This application in the field of sports medicine may be useful in diagnosing sports-related injuries and also decreasing or preventing future adverse outcomes.