A Machine Learning Model for Post-Concussion Musculoskeletal Injury Risk in Collegiate Athletes

Abstract

Background

Emerging evidence indicates an elevated risk of post-concussion musculoskeletal injuries in collegiate athletes; however, identifying athletes at highest risk remains to be elucidated.

Objective

The purpose of this study was to model post-concussion musculoskeletal injury risk in collegiate athletes by integrating a comprehensive set of variables by machine learning.

Methods

A risk model was developed and tested on a dataset of 194 athletes (155 in the training set and 39 in the test set) with 135 variables entered into the analysis, which included participant’s heath and athletic history, concussion injury and recovery-specific criteria, and outcomes from a diverse array of concussion assessments. The machine learning approach involved transforming variables by the weight of evidence method, variable selection using L1-penalized logistic regression, model selection via the Akaike Information Criterion, and a final L2-regularized logistic regression fit.

Results

A model with 48 predictive variables yielded significant predictive performance of subsequent musculoskeletal injury with an area under the curve of 0.82. Top predictors included cognitive, balance, and reaction at baseline and acute timepoints. At a specified false-positive rate of 6.67%, the model achieves a true-positive rate (sensitivity) of 79% and a precision (positive predictive value) of 95% for identifying at-risk athletes via a well-calibrated composite risk score.

Conclusions

These results support the development of a sensitive and specific injury risk model using standard data combined with a novel methodological approach that may allow clinicians to target high injury risk student athletes. The development and refinement of predictive models, incorporating machine learning and utilizing comprehensive datasets, could lead to improved identification of high-risk athletes and allow for the implementation of targeted injury risk reduction strategies by identifying student athletes most at risk for post-concussion musculoskeletal injury.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40279-025-02196-4.

Key Points

There is a well-established elevated risk of post-concussion subsequent musculoskeletal injury; however, prior efforts have failed to identify risk factors.

This study developed a composite risk score model with an area under the curve of 0.82 from common concussion clinical measures and participant demographics.

By identifying athletes at elevated risk, clinicians may be able to reduce injury risk through targeted injury risk reduction programs.

Introduction

The acute diagnosis of sports-related concussions has improved dramatically over the last two decades [1–3]; however, the determination of recovery remains an ongoing challenge [4]. Concussions affect cognitive function, motor control, and vestibulo-ocular and cardiovascular function and are associated with increased somatic and psychological symptoms [1, 5–8]. Current clinical assessments and neurological screening tools, while effective in identifying acute concussions, may not fully capture the persistent neurophysiological deficits that can linger beyond an athlete’s clinical recovery and return to participation (RTP) [4, 9, 10]. The primary concern for post-concussion RTP decisions was the elevated risk of a subsequent concussion, but current concussion management protocols have significantly reduced this risk [11]. However, concerning evidence has emerged over the last decade of a two times elevated rate of musculoskeletal (MSK) injuries in the year following a concussion [12–20]. This relationship has been identified across diverse populations, including high school athletes [16], collegiate athletes [15, 17], military personnel [18], and the general public [19]. These lower extremity MSK injuries pose substantial challenges to athletes acutely and may increase the rate of chronic conditions across the lifespan. Acutely, MSK injuries result in increased healthcare costs, lost school/work time, and elevated rates of mental health challenges [21–24]. Chronic conditions such as osteoarthritis, difficulty working, decreased quality of life, and elevated healthcare costs are more prevalent in former athletes with prior MSK injuries [24–26]. Thus, identifying athletes at risk for MSK injuries is a critical healthcare priority to improve an athlete’s quality of life and reduce healthcare costs.

Identifying athletes who are at an elevated risk for MSK injuries is possible but challenging owing to the extensive set of potential modifiable and non-modifiable risk factors; indeed, a recent military MSK review identified 950 potential variables [27–29]. A systematic review of extensive previous efforts to develop MSK injury prediction models identifies some consistent risk factors, but results indicate generally limited success in overall prediction [29, 30]. The combination of relatively large samples and the number of potential variables has motivated machine learning/artificial intelligence models to improve outcome prediction [30–33]. A key limitation of these sports medicine injury prediction approaches is the lack of standard data sets [34]. Concussion studies and data sets offer an opportunity to overcome this limitation as assessment techniques generally follow international consensus guidelines while injury characteristics and athlete demographics are commonly recorded by healthcare providers [4]. As in the general MSK injury models, single or small groups of variables do not enable successful prediction models for post-concussion MSK [17, 35]. Thus, we propose a machine learning approach, consistent with military MSK injury risk approaches [28], which incorporates an extensive set of risk factors, including (1) common clinical concussion assessments; [9] (2) demographic and anthropometric measures; (3) concussion injury characteristics; and (4) assessments during the post-concussion recovery process, as variables for a statistical model to predict subsequent MSK injury.

While using concussion assessments consistent with international guidelines will enable general adoption, challenges are still inherent with the use of a large set of heterogeneous variables measures collected at baseline and post-concussion. Three primary challenges are (1) missing data, (2) non-linear relationships between the variables and the outcome, and (3) difficulties in encoding and combining categorical and continuous variables together without increases in dimensionality. To address these issues, we propose to use the weight of evidence (WoE) transformation to unify the set of variables before applying variable selection and logistic regression to model the risk of MSK [36–40]. Weight of evidence is a computationally efficient variable transformation method that enhances the predictive power of variables, simplifies comparison across diverse data types, and curtails dimensionality growth. Weight of evidence has seen applications in credit scoring and risk assessment domains and may be well suited to address current limitations and challenges in identifying post-concussion MSK risk [41, 42].

The elevated risk of MSK injury following a concussion is now well established; however, identifying accurate predictors of elevated risk has been unsuccessful. Therefore, the purpose of this study was to develop a machine learning-based, post-concussion MSK risk model in collegiate athletes, with the composite risk score computed from commonly used clinical assessments as well as participant and injury characteristics. The successful development of this model may allow clinicians to implement targeted injury risk reduction approaches and reduce the post-concussion MSK incidence.

Methods

Participants

We recruited 211 National Collegiate Athletic Association (NCAA) student athletes from a single institution who were diagnosed with a sports-related concussion. The inclusion criteria were any student athlete who experienced a sports-related concussion and had at least 1 year of sports participation both before and after the concussion. Additionally, all participants completed a pre-career baseline assessment and completed the institution-specific RTP protocol, which was consistent with the contemporary consensus recommendations [43, 44]. The exclusion criteria were an athlete who did not RTP following their concussion because of their athletic eligibility being completed, a decision to no longer participate following recovery (i.e., “quit”), or medical disqualification by the team physician [45]. Additional exclusion criteria included a concurrent injury with the concussion, which required additional time loss beyond the concussion recovery (e.g., fracture). Missing timepoint(s) was not an exclusion criterion, but a participant had to have at least one timepoint post-concussion to be included in this study. All participants provided written and oral informed consent as approved by the institution’s institutional review board (Approval Numbers: 740790 and 804454, initial approval 2015). Out of the 211 athletes, 194 athletes met the inclusion criteria (Table 1).

Table 1.

Participant demographics and anthropometrics by sex and sport. No participant was engaged in more than one sport

Sex	Age at baseline (years)	Age at concussion (years)	Height (cm)	Weight (kg)
Female (F)	18.8 ± 0.8	19.6 ± 1.1	168.6 ± 10.9	62.8 ± 8.5
Male (M)	19.0 ± 1.2	20.0 ± 1.4	185.6 ± 8.1	97.1 ± 21.7
Sports Baseball: 7 Basketball (F): 10 Basketball (M): 7 Cheerleading (F): 6 Cheerleading (M): 2 Field Hockey: 9 Football: 28 Lacrosse (F): 19 Lacrosse (M): 15 Rowing (F): 11 Soccer (F): 16 Soccer (M): 9 Softball: 12 Swim/Dive (F): 10 Swim/Dive (M): 2 Tennis (F): 1 Tennis (M): 3 Track and Field (F): 11 Volleyball (F): 16

Instrumentation

Variables for the model included participant demographics and anthropometrics, concussion injury-specific information, and a comprehensive concussion assessment at four timepoints: (1) pre-career ‘baseline’ timepoint, and then at three additional post-injury timepoints; (2) acute (< 48 h of injury); (3) asymptomatic, and (4) RTP [9].

Participant demographics included sex, prior concussion history (yes/no), total number of prior concussions (continuous variable), sport type (collision, contact, noncontact), and sport [9]. Additional variables included a self-reported prior history of anxiety, attention-deficit disorder/attention-deficit hyperactivity disorder, depression, learning disability, and other psychiatric disorder were recorded as binary variables (yes/no). Both the self-reported history of MSK injury (yes/no) and number of injuries (continuous) were recorded from the participants baseline health history for injuries prior to college following an institutional review board-approved review of the athletic training electronic health record for MSK injuries during their collegiate careers.

The participants’ concussion-specific information included loss of consciousness (yes/no) and post-traumatic amnesia (yes/no) as binary variables. Additionally, both the days until asymptomatic and the days until RTP were included as continuous variables.

The concussion assessments included commonly used measures from both the SCAT as well as the NCAA/DoD CARE study, and the assessments were completed according to standard and well-established protocols [9, 43, 44]. All measures were performed at baseline and each of the three subsequent timepoints (acute, asymptomatic, RTP) with the exception of the Hospital Anxiety and Depression Scale (HADS) and Satisfaction with Life Scale (SWLS), which were only performed at baseline.

The participant’s self-reported symptoms were recorded for both the number of symptoms endorsed (0–22) and the graded symptom (0–6) checklist (0–132), with a lower score reflecting fewer symptoms and severity [43, 44]. The participants completed the balance error scoring system, which consists of three stances (double limb, single limb, and tandem) on two surfaces (firm, foam) [46–48]. Deviations from the test position are considered errors with a maximum of ten per condition for a total score range of 0–60 with a higher score reflecting worse balance. Participants also completed both a single- and dual-task tandem gait task [49]. Briefly, the tandem gait task requires the participants to walk heel-to-toe down a 3-m line, turn, and return to the starting point. During dual-task trials, participants responded to working memory cognitive challenges (e.g., subtraction by seven, spelling a five-letter word backwards) [50]. The outcome measure was the total time to complete each task with a higher time reflecting worse performance [51, 52].

Cognition was assessed using the standard assessment of concussion [43, 44], the ImPACT computerized cognitive assessment [53, 54], and the Trail Making Test (TMT) [55]. The SCAT-3 version of the standard assessment of concussion was used (i.e., five-word memory list) with a possible 30 points and a higher score reflected better cognitive performance [43, 44]. The ImPACT neurocognitive assessment has four composite score outcome measures: verbal memory, visual memory, visual motor speed, and reaction time. For reaction time, a lower number reflects better performance as opposed to verbal memory, visual memory, and visual motor speed wherein a higher score reflects better performance. The TMT outcomes are the time taken to complete the task with a lower time reflecting better performance.

To assess vestibular and visual function, the participants completed the Vestibular Ocular Motor Screening (VOMS) and the King-Devick test [56–59]. The VOMS was scored as pass/fail based on an increase of more than two total symptom scores during the examination (smooth pursuits, horizontal and vertical saccades, horizontal and vertical ocular reflex, and visual motor sensitivity). The near point of convergence was recorded as the mean of three trials and a lower score reflected a better near point of convergence [56, 59]. The King–Devick test, performed either with spiral bound cards or a tablet, requires the participant to read three cards of random letters and the outcome measure is the total time to complete the task with a lower score reflecting better performance [57, 58].

The participants completed eight trials of the clinical reaction time (CRT) test and the mean time of the trials was the outcome measure. A lower reaction time reflected better performance [60, 61].

Clinical mental health screenings included the Brief Symptom Inventory-18 (BSI-18), the Hospital Anxiety and Depression Scale (HADS), and the Satisfaction with Life Scale (SWLS) [9, 62–64]. The BSI-18 is an 18-item self-reported questionnaire with a score range of 0–24, with higher scores being worse. The BSI has subsections including depression, anxiety, and somatization with specific thresholds for each: depression and anxiety greater than seven and somatization greater than six. The HADS score range is 0–21 with subsections for both anxiety and depression. The section-specific threshold for each section is greater than eight. The SWLS has a 0–35 range with higher scores reflecting better life satisfaction and a score lower than 20 indicating low satisfaction.

Procedures

A concussion was initially identified by an athletic trainer and the diagnosis was confirmed by a team physician consistent with the current concussion consensus in sport group diagnostic criteria [43, 44]. The student athletes were assessed for symptoms by an athletic trainer on a daily basis until asymptomatic. Many, but not all, participants were enrolled as part of the NCAA-DoD CARE Consortium, which influenced the post-injury testing timeline [9]. The concussion protocol required the student athlete to be asymptomatic, achieve baseline or better values on the clinical exams, and complete a 6-day progressive return to activity protocol. Final clearance for unrestricted RTP was based on successful completion of the progressive exercise protocol and a normal physical examination performed by the team physician.

All participants were tracked for 1-year post-RTP (e.g., a concussion RTP on 1 October would be tracked until 30 September of the following year) for sports-related MSK injuries through the athletic training electronic medical records. An injury was defined as requiring treatment from the athletic training staff or team physicians and resulting in at least 1 day of limited activity [17, 35].

Statistical Analysis

Our approach to form a risk model for MSK injury uses the WoE transformation, which is related to the naive Bayes classifier, combined with a logistic regression analysis [65, 66]. Specifically, continuous and discrete variables are transformed by a binning approach, where the value assigned to each bin is the log-ratio of the frequencies of the variable falling into that bin under the two outcomes (MSK injury or not) and the bin edges are optimized in terms of their information value to enhance their predictive power (see A.2 of the Electronic Supplementary Material [ESM] for more technical details). This value assignment quantifies the evidence in favor of the outcome to evidence against the outcome. Hence, the ‘weight of evidence’ denomination. It is worth noting that missing values can also follow this logic as they are still associated with an outcome, i.e., data imputation is handled automatically. Together, WoE’s optimal binning and transformation of values can model non-linear relationships between the predictive variables and MSK injury risk, even in the presence of missing data. It should be noted that as a supervised data-driven technique, the WoE transformation is based solely on the training set.

In this study, each participant is represented by $P=135$ predictive variables, which consider the (1) variables described in the Instrumentation section and (2) the difference between measures at baseline and all the other timepoints. For example, the difference in CRT between baseline and acute timepoints is calculated as Baselines minus Acute, and in the figures is labeled “CRT Difference Baseline Acute”. Participants are randomly divided, with random numbers generation, into training and test sets (via stratified sampling to ensure a matched injury rate in both sets): the training set contains $N=155$ athletes and the test set contains $N_{\text{test}}=39$ athletes. The training set comprises 96 athletes who experienced an MSK injury after a concussion and 59 who did not. The test set contains 24 athletes who experienced a post-concussion MSK injury and 15 who did not. The test set is a held-out set that is only used for evaluation purposes and has no influence on the model training process.

The model formulation begins with denoting the MSK injury outcome variable as $Y\in \left\{0,,,1\right\}$, with $Y=1$ for an athlete that has a subsequent MSK injury and $Y=0$ for one without an injury. Let $\mathit{x}=\left[x_1,,,\cdots ,,,x_P\right]$ denote a vector of $P$ predictive variables. The log-odds of an athlete having an MSK injury is defined as $\text{log}\frac{\mathbb{P}\left(Y,=,1,|,x\right)}{\mathbb{P}\left(Y,=,0,|,x\right)}$, where $\mathbb{P}\left(Y,=,1,|,x\right)$ is the probability of an athlete having an MSK injury given the observed variables $x$.

Assuming the variables are independently contributing evidence, the log-odds can be expressed in terms of the WoE transformation applied to each variable (see ESM). Combining all the variables, we fit a logistic regression model that weights each of the WoE-transformed variables as a refined estimate of the log-odds so that:

$$\text{log}\frac{\mathbb{P}\left(Y,=,1,|,\mathit{x}\right)}{\mathbb{P}\left(Y,=,0,|,\mathit{x}\right)}\approx w_0+\sum _{i=1}^P{w}_i{\mathit{WoE}}_i\left(x_i\right),$$ 1

$$\mathbb{P}\left(Y,=,1,|,\mathit{x},;,,\mathit{w}\right)=\sigma \left(w_0,+,\sum _{i=1}^P,w_i,{\mathit{WoE}}_i,\left(x_i\right)\right),$$ 2

where $\mathbb{P}\left(Y,=,1,|,\mathit{x},;,,\mathit{w}\right)$ denotes the model’s probability estimate of MSK injury, $\sigma \left(z\right)=\frac{1}{1+e^{-z}}$ is the logistic function, $\mathit{w}=\left[w_0,,,\cdots ,,,w_P\right]$ is the vector of parameters (bias and coefficients), and ${\mathit{WoE}}_i\left(x_i\right)$ denotes the transformation for the $i$th variable.

To remove any unnecessary variables from the model and enhance model performance by mitigating the issue of overfitting, we employ an L1-penalty when fitting the logistic regression model. The L1 penalty has the effect of forcing some coefficient estimates to be exactly zero, effectively removing those variables from the model. The regularization parameter of the L1 penalty is systematically varied, creating a family of models with different numbers of selected variables and predictive performance. For model selection, we use the corrected Akaike Information Criterion, which balances model fit and complexity [67, 68] and provides asymptotically loss-efficient variable selection when the true model is not contained within the candidate variable sets (a realistic assumption in many applications) [69]. Finally, an L2-regularized logistic regression model is fit using the set of variables with non-zero coefficients, with the regularization ameliorating any collinearity among the selected WoE-transformed variables. The performance of this model is evaluated on the test set by computing the area under the receiver operating characteristic (ROC) curve and area under the precision-recall curve, which is known as the average precision and shows the behavior of the accuracy of positive predictions made by a classifier (precision) against the classifier’s ability to identify all relevant instances within the positive class (recall).

As a benchmark, we also fit a logistic regression model, which uses the original variables instead of the WoE-transformed variables. Mean imputation is applied when data are missing for this approach. Additionally, categorical variables are one-hot encoded and the remaining variables are standardized to have zero mean and unit variance so that their magnitudes can be comparable (see ESM).

After model fitting and selection, we apply model explanation techniques to identify the variables that contribute the most to the predictions. Specifically, we adopt Shapley Additive Explanations (SHAP) [61], which quantify the contribution of each variable to the prediction for each subject. In linear models, SHAP values are a product of the model’s coefficients and the deviation of the variable from the mean. More specifically, for a given $x$, the contribution ${\varphi }_i$ of the $i$ th variable on the log-odds ratio estimate $\text{log}\frac{\mathbb{P}\left(Y,=,1,|,\mathit{x}\right)}{\mathbb{P}\left(Y,=,0,|,\mathit{x}\right)}$ is

$${\varphi }_i\left(\text{log},\frac{\mathbb{P}\left(Y,=,1,|,\mathit{x}\right)}{\mathbb{P}\left(Y,=,0,|,\mathit{x}\right)}\right)=w_i\left({\mathit{WoE}}_i,\left(x_i\right),-,\overline{{\mathit{WoE}}_i}\right),$$ 3

where $\overline{{\mathit{WoE}}_i}=\frac{1}{N}{\sum }_{n=1}^N{\mathit{WoE}}_i\left(x_{\mathit{ni}}\right)$ is the mean of the $i$ th variable after the WoE transformation across the sample ${\left\{{\mathit{x}}_n\right\}}_{n=1}^N$ of size $N$. To summarize the impact of each variable, the absolute contributions are averaged by

$$\frac{1}{N}\sum _{n=1}^N\left|{\varphi }_i,\left(\text{log},\frac{\mathbb{P}\left(Y=1|,{\mathit{x}}_n\right)}{\mathbb{P}\left(Y=0|,{\mathit{x}}_n\right)}\right)\right|.$$ 4

Results

Starting with $P=135$ variables that could be used to compose a risk, the corrected Akaike Information Criterion determined that the simplest model that can explain the variance in the outcome variable only needs 48 variables (see Appendix B of the ESM for more details on the model selection and Table B1 of the ESM for a detailed list of the selected variables). The model trained with the selected variables was tested with data from athletes in the held-out test set. The distributions of log-odds ratios for the two groups of athletes (MSK injury and non-MSK injury groups) were significantly different ($p=0.003$) (Fig. 1).

Fig. 1.

Distribution of logit scores for musculoskeletal (MSK) injury and non-MSK injury groups in the test set. The box plot illustrates the spread and central tendency of the logit risk scores computed for our proposed approach. The model trained for weight of evidence (WoE)-transformed variables shows a statistical difference between the two groups. Q quartile

The ROC curve for the WoE-based model has an area under the curve of 0.82, which indicates a strong and clinically meaningful discriminatory ability of the model between the MSK and non-MSK groups (Fig. 2A). At a false positive of 6.67%, a true-positive rate (recall/sensitivity) of 79% is achieved; this corresponds to a precision (positive predictive value) of 95%.

Fig. 2.

Receiver operating characteristic (ROC) and precision recall curves for the logistic regression models post-variable selection. The ROC curve A showcases the true-positive rate against the false-positive rate at various threshold levels with an overall area under the curve (AUC) of 0.82. The precision-recall curve B highlights the precision (positive predictive value) at different levels of recall (sensitivity) with an average precision of 0.85. The best threshold would yield an F1 (harmonic mean of precision and recall) of 0.86. MSK musculoskeletal, WOE weight of evidence, AP average precision

The distribution of the probabilistic outputs for both the training and tests are provided in Fig. 3. In the training set (Fig. 3A),the predictions indicate an overly optimistic performance, resulting from the model’s optimization to this specific dataset. The predictions for the test set exhibit a similar distribution to the training set, despite the model not having been exposed to these data previously (Fig. 3B) This similarity suggests that our model can generalize well to new data.

Fig. 3.

Probabilistic predictions of a logistic regression model for injury classification on a training and b test sets. These plots illustrate the model’s predictions in probability space, differentiating post-concussion injury cases from non-injury cases. Notably, the behavior of the predictions in the test set suggest that our trained model is able to make correct predictions for unseen data. MSK musculoskeletal, Q quartile, WoE weight of evidence

Figure 4 presents the top ten most influential variables in our trained logistic regression model. This ranking is based on the mean absolute SHAP values computed using the training set, which quantify each variable’s impact on the model’s predictions. No single variable or small group of variables dominates the predictions; rather, each variable contributes a small amount to the final outcome.

Fig. 4.

Top ten influential variables in a logistic regression model based on mean absolute Shapley Additive Explanations (SHAP) values. This bar chart shows the mean absolute SHAP values, computed from the training set, of the variables most influential to the model’s predictions. BESS Balance Error Scoring System, CRT clinical reaction time test, ImPACT Immediate Post-Concussion Assessment and Cognitive Test, K–D King–Devick Test, SWLS Satisfaction with Life Scale, TMT-A Trail Making Test-A. Timepoints: Baseline = prior to athletic participation; Acute = within 48 h of injury; Asymptomatic = when the patient reports no concussion-related symptoms

To assess the consistency of the proposed machine learning approach in producing a well-calibrated risk model with a subset of features, the methodology was applied to 20 random training-test splits of the dataset created using the stratified sampling scheme with the additional criterion that splits that resulted in a unique category for a categorical variable only appearing in the test set were rejected. The results (Appendix C, Figs. C5 and C6 of the ESM) from running the entire WoE transformation, feature selection, and model fitting selection showed a high degree of consistency with the model described above. Across the runs, the mean and standard deviation of performance were area under the ROC: 0.79 ± 0.05 (Table C6 of the ESM). The model selection criterion produced models with a similar number of features (average of 47) with a range of 41–52. Because of the correlation between many of the 135 variables (Fig. C8 of the ESM), across the 20 runs, a total of 112 distinct variables were selected and 31 of the 48 variables were in the top 50 variables most frequently selected. “Time Lost,” “Sport,” “ImPACT Motor Speed at RTA,” and “TMT-A at Acute” were included in all 20 runs (Fig. C7 of the ESM). While the exact same variables were not chosen in each split, similar correlated variables were selected. The distribution of SHAP values shows that variables are consistent in their importance (Fig. C9 of the ESM).

To assess whether the set of variables selected in the original split are as informative as other sets, we fit new models using these same variables across the 20 random train-test splits. The performance was nearly identical with area under the ROC: 0.80 ± 0.03 (Table C6 of the ESM). This indicates the 48 variables are consistently good predictors of post-concussion MSK injury risk.

Additionally, we assessed the performance broken down by sport. While predictive performance in terms of ROC curve is consistent for most sports (Figs. C5 and C6 of the ESM), two sports: Cheer along with Swimming and Diving had notably lower performance, and two sports: Tennis, which had only four student athletes in the study, and Track and Field had notably higher performance. Finally, we fit models without variables that have information on sport (Appendix C.5 of the ESM). Without these variables, performance is lower, but still fair, with an area under the curve of 0.77 ± 0.05 (Table C7 of the ESM), which is not unexpected because of the varying prevalence of MSK injury across sports.

Discussion

In the year following a concussion, there is a well-established approximately two-fold elevated risk of a subsequent MSK injury [12–19]. In an ideal scenario, every athlete would receive personalized injury risk reduction treatments when returning to sports post-injury; however, this is neither logistically plausible nor cost effective in amateur athletics. Because of the economic and personal costs associated with these injuries, a predictive risk algorithm that can identify athletes with higher risk can be used to identify those who may benefit from an injury risk reduction intervention. Previous attempts to develop predictive models using traditional analysis approaches considered individual predictors at single timepoints, but these attempts were largely unsuccessful [17, 27, 35]. Our study addresses this gap. We utilized a comprehensive set of variables based on commonly available injury information in a machine learning statistical modeling approach, which led to a well-calibrated composite risk score for post-concussion MSK. The primary finding was the successful development of a clinically feasible model with high predictive accuracy (area under the curve: 0.82) to identify post-concussion MSK injury risk. This model may allow sports medicine clinicians to specifically target the highest-risk patients with established injury risk reduction programs [70, 71].

An innovative component of our approach is the combination of WoE transformation and regularized logistic regression models with an intrinsic feature selection to handle the extensive set of variables. Predictive models that use the WoE transformation are traditionally employed in the financial industry, particularly in credit scoring, risk assessment, and fraud detection. However, these methods have not been applied to model sports medicine injury risk challenges. Our approach combines WoE transformation with optimized binning of continuous and categorical variables as a data-driven approach to identify the best representation of the predictor variable [36–40]. By finding the binning scheme with the highest information value, the variable’s discriminatory power is maximized, leading to more accurate and robust predictive models. By integrating the WoE transformation with the feature selection capabilities of L1-regularized logistic regression, our approach removes redundant predictors and mitigates model overfitting and ameliorates collinearity with subsequent L2-regularized logistic regression on the selected variables to achieve desirable risk segmentation even with a moderate sample size. Our approach also further addresses prior criticisms of MSK injury modeling by maintaining interpretable continuous variables, avoiding linearity assumptions, and using modeling techniques applicable to high dimensional data with various data types. Though the data set had 135 variables, the model developed herein uses only 48 variables (Table B2 of the ESM), which can be reasonably included in concussion assessments and were all components of the NCAA-DoD CARE consortium assessment battery [9]. Furthermore, this is considerably less than the 950 potential MSK risk factors previously identified in military populations [28, 29].

The variables contributing most to the model were a surprising mix of assessments and timepoints (Fig. 4). The strongest predictors (in terms of the SHAP values) came from clinical assessments at both the baseline (pre-participation) and acute (< 48 h) timepoints. A cornerstone of concussion management over the last decade is the use of a multi-faceted assessment battery, and these results support this concept as the top predictor, which includes an array of cognitive, balance, and reaction time measures [72–74]. Each of these domains (i.e., cognition, oculomotor, and balance) has been associated with elevated injury risk, either conceptually or with limited empirical evidence, along with extrinsic factors such as sport and time loss. While each assessment technique has its limitations, especially concerning practice effects and test–retest reliability [75], taken together, they can create a composite score. Interestingly, SWLS was a highly influential variable in the predictive model. This was consistent with a prior report that found a 1-point increase in SWLS was associated with a 36% decrease in risk of MSK injury [35]. Lower satisfaction with life was associated with a higher risk of injury in Korean workers but has received limited application in sports medicine settings. Two of the top ten variables were not assessments but rather sport and time loss. Specific to sport, each sport had a different weight (Table B2 of the ESM), which accounts for the inherent differences in the risk of MSK injury; future studies need to include a wider range of sports. Notably, athletes with 9 or fewer days of time loss were at elevated risk of experiencing a post-concussion MSK, which is consistent with earlier studies showing elevated rates of subsequent concussion with short RTP durations [11]. A recent large CARE consortium finding (1751 participants) reported the mean time to RTP was about 13 days, with only about 15% having full RTP within 7 days, suggesting that most athletes would not be within this risk category [2]. Taken together, this set of clinically feasible data points can be used to identify athletes at elevated risk of post-concussion MSK.

One inherent limitation of this model was the reliance on the influential effect of baseline data, which is a time and resource-intensive process [76]. Current consensus statements indicate that baseline data are not required to interpret assessments following a suspected concussion [4]; however, legal considerations (i.e., state laws) and compliance (e.g., Arrington settlement for NCAA member institutions) often dictate which baseline assessments will be performed [77]. As a large-scale project with numerous data collection timepoints and assessments, missing data were a clear limitation of the study (on average, roughly 35% was missing); nonetheless, a good predictive model was still developed. This model was developed from nearly 200 concussions at a single university and reproducibility of this model at other institutions, which may have a varying distribution of athletes among sports and levels of competition, and other ages (e.g., high school, military) are critical steps forward. All data were collected by clinical athletic trainers or research staff well trained in the assessments, but inconsistency in data collection likely occurred. Another limitation of the model is its complexity. Future work could consider further simplifying the modeling by creating well-defined risk categories, similar to the military traffic light system, to allow clinicians to apply risk reduction interventions to the highest-risk athletes [28, 78]. While the machine learning approach produced a particular model for the training set, similarly performing models with slightly different subsets of variables are produced on different splits (see Appendix C in the ESM). This is owing to the relatively high correlation among some variables, as shown in a correlation heat map (Fig. C8 of the ESM). Relatedly, one drawback of the optimized binning of the WoE transformation is that bins created by the optimization algorithm on the training set may seem overly complicated. For example, the optimized bins for the variable “Time Lost” (the number of days between concussion and RTP dates) generates a binning scheme with three segments: less than 9 days, between 9 and 12 days, and greater than 12 days. How best to simplify the model in a clinically meaningful manner, while retaining its predictive performance, requires further study. One possibility is to impose pre-determined bins when necessary for interpretation.

The results of this dataset analysis indicate that this approach holds the promise of providing robust and accurate risk categorization for post-concussion MSK injury. Incorporating more data from various institutions into the same methodology has the potential to produce an even stronger model. Moving forward, important next steps include a validation study to determine reproducibility and, if this is successful, developing a user-friendly and freely available online tool (e.g., website, app) to allow clinicians to input data to calculate the patient’s injury risk profile. By refining the model with additional data from multiple institutions and/or settings, we expect to enhance further its precision and applicability for clinicians to target high-risk post-concussion athletes with informed decisions, ultimately leading to more effective risk reduction strategies and programs to prevent post-concussion injury occurrences for high-risk athletes.

Conclusions

We proposed and developed an integrative analysis approach that combines numerous clinical measurement variables to assess the risk of subsequent MSK injury in a population of concussed athletes. The approach transforms variables by the WoE transformation before fitting a logistic regression model with variable selection to create a robust statistical model. The results from the risk score distribution and ROC curve analyses affirm the resulting model is highly predictive of subsequent MSK injury, with sensitivity of 79% at a false-positive rate of 6.67%, thus indicating the potential utility in practice.

Supplementary Information

Below is the link to the electronic supplementary material.

Declarations

Funding

All authors disclose funding from the National Institute of Health: National Institute of Neurological Disorders and Stroke (1R21NS122033). Thomas Buckley received additional funding from the NCAA/DoD CARE Consortium, U.S. Department of Defense (W81XWH1420151). Dr. Qian has received additional funding from the NSF (DMS-2413833).

Conflicts of interest/competing interests

Claudio C. Claros, Melissa N. Anderson, Wei Qian, Austin J. Brockmeier, and Thomas Buckley have no conflicts of interest that are directly relevant to the content of this article. Thomas Buckley has received past industry funding within the last 3 years from StateSpace, Inc., as well as Highmark BluePrints, but neither funding was related to this article.

Ethics approval

The study was performed in accordance with the standards of ethics outlined in the Declaration of Helsinki. All study procedures were reviewed and approved by the University of Michigan Institutional Review Board and the US Army Medical Research and Materiel Command Human Research Protection Office for all data related to the NCAA-DoD CARE Consortium. The University of Delaware Institutional Review Board approved all aspects of this study (Approval Numbers: 740790 and 804454, initial approval 2015).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent for publication

Not applicable.

Availability of data and material

The CARE Consortium datasets generated and analyzed during the current study are available in the FITBIR repository (https://fitbir.nih.gov/).

Code availability

The custom data analysis code written in Python with standard libraries is available on Github at https://github.com/cesar-claros/R21_risk_model/.

Authors’ contributions

TB, AB, and WQ were involved in the study concept and design, interpretation of the data, statistical interpretation, manuscript drafting, and final approval. TB and MA were involved in the collection of the data. MA and CC were involved in the data analysis, interpretation of the data, statistical interpretation, manuscript drafting, and final approval. All authors read and approved the final version.