Skip to main content
NCBI home page
International Journal of Sports Physical Therapy logoLink to International Journal of Sports Physical Therapy
. 2022 Dec 1;17(7):1340–1350. doi: 10.26603/001c.55544

Psychological Patient-reported Outcomes Cannot Predict a Second Anterior Cruciate Ligament Injury in Patients who Return to Sports after an Anterior Cruciate Ligament Reconstruction

Ramana Piussi 1,2,3,✉, Rebecca Simonson 1,3, Johan Högberg 1,2,3, Roland Thomeé 1,2, Kristian Samuelsson 3,4, Eric Hamrin Senorski 1,2,3,5
PMCID: PMC9718686  PMID: 36518826

Abstract

Background

Psychological patient-reported outcomes (PROs) are recommended for use in test batteries to aid in decision-making, regarding whether patients are well prepared to return to sports (RTS) after anterior cruciate ligament (ACL) reconstruction. However, the values that should be regarded as “pass” or “fail” are still unclear.

Purpose

This study aimed to identify cut-off values for three commonly used psychological PROs that could differentiate patients who suffer a second ACL injury from patients who do not within two years of RTS in patients after ACL reconstruction with respect to recovery of symmetrical quadriceps strength.

Study design

Diagnostic/prognostic study

Methods

Demographic data, isokinetic strength test data for quadriceps, as well as results for the ACL-Return to Sport after Injury scale (ACL-RSI), Knee Injury and Osteoarthritis Outcome Score (KOOS) Quality of Life, and Function in Sport and Recreation sub-scales, and the 18-item version of the Knee Self-Efficacy Scale (K-SES18) were extracted from a registry. Receiver operating characteristic (ROC) curves were calculated for each PRO. Accuracy of the cut-offs was presented with two summary measures for the ROC: the area under the curve (AUC) and Youden index.

Results

In total, 641 (355 men, 61%) patients (24.8 [SD 7.6] year old at ACL reconstruction) were included. The cut-off values were not able to differentiate patients who suffered a second ACL injury up to 24 months after RTS and ACL reconstruction from patients who did not. Additionally, achieving symmetrical quadriceps strength did not improve the cut-off psychometric properties.

Conclusion

Since cut-off values could not differentiate between patients who suffered a second ACL injury and those who did not, clinicians should not rely only on cut-off values or a single PRO of those analyzed in this study when making decisions on which patients are at risk of experiencing a second ACL injury when returning to sports after ACL reconstruction.

Level of Evidence

Level 3

Keywords: anterior cruciate ligament, knee self efficacy scale, pateient reported outcomes, psychological outcomes, return to sport, return to sport after injury scale

INTRODUCTION

Anterior cruciate ligament (ACL) injury is most commonly treated with rehabilitation, and about 50% of patients undergo surgical reconstruction of the ruptured ligament.1 To sustain a second ACL injury after ACL reconstruction can be devastating; and this further increases the risk for long-term consequences, such as osteoarthritis and lower levels of physical activity and perceived quality of life.2–5 A common goal for patients treated with rehabilitation and ACL reconstruction is to return to sport (RTS).6,7 As participating in sports is associated with a risk of sustaining a second ACL injury,8 a primary concern is to evaluate the patients readiness for RTS, i.e. returning to sports with minimal risk for suffering a second ACL injury.

Responses to psychological patient-reported outcomes (PROs) can differ between patients who suffer a second ACL injury and those who do not, as patients who suffer a second ACL injury have reported greater scores, that is, better responses, on validated PROs.9,10 Collected evidence, including the results from systematic reviews on psychological outcomes after ACL reconstruction and clinical practice guidelines,11–13 as well as patients’ experiences,14 suggest that psychological PROs should be used in test batteries to help identify whether patients are well prepared for RTS after ACL reconstruction.

An issue related to the use of PROs in RTS test batteries is, which cut-off values best identify whether patients “pass” (patients can be cleared to RTS) or “fail” (patients should be advised against RTS)? Different cut-offs for different PROs and outcomes have been proposed.15,16 Notably, some cut-offs commonly used in the decision-making if the patient can RTS have not been anchored against a second ACL injury,15 but are instead based on responses from PROs that were collected several months prior to patients suffering a second ACL injury.16 It is important to acknowledge that the reasons for a second ACL injury are multifactorial and include genetic, physical, psychological, trauma, and surgical factors.17–20 Recovery of symmetrical quadriceps strength has been proposed as a possible protective factor that can reduce the risk of a second ACL injury in high-level athletes with RTS.21 However, the results are inconclusive.22,23

There is a need to better understand which cut-offs for PROs best predict a second ACL injury in patients treated with ACL reconstruction, especially regarding protective factors for a second ACL injury, such as the recovery of symmetrical quadriceps strength. This study aimed to identify cut-off values for three commonly used psychological PROs that could differentiate patients who suffer a second ACL injury from those who do not within two years of RTS after ACL reconstruction with respect to recovery of symmetrical quadriceps strength.

METHODS

Following the guidelines from the Enhancing the Quality and Transparency of Health Research (EQUATOR) network, the Standards for Reporting Diagnostic Accuracy (STARD) statement was used as a checklist to report this study.24

Data for the present study were prospectively collected from a rehabilitation outcome registry, the Project ACL. The registry was established in 2014, which aims to improve the care of patients with ACL injuries. Data in the Project ACL consists of results from muscle function tests and PROs collected prospectively before the surgery (in case of ACL reconstruction), at 10 weeks, 4, 8, 12, 18, and 24 months, and every five years with ACL injury/reconstruction as a baseline.25 Prior to participation in the Project ACL, written consent was collected. Ethical approval was obtained from the Swedish Ethical Review Authority (registration number:2020-02501).

Inclusion and exclusion criteria

All consecutive patients registered in the Project ACL with one ACL injury treated with reconstruction with or without sustaining a subsequent second ACL injury within two years from RTS [Tegner Activity Scale (Tegner)26 ≥6] were eligible for inclusion. The patients included in this study were followed up for two years after RTS. Patients were excluded if one of the following criteria were fulfilled: age <16 or >50 years; not participating in knee-strenuous sports prior to index ACL injury, i.e. reporting a Tegner Activity Scale26 of <6; sustaining a second ACL injury before surgical reconstruction; or not participating in any of the follow-ups of project ACLs. Second ACL injuries are reported by patients themselves, responsible physical therapists, or the test leader at follow-up in Project ACL.

Patient Reported Outcomes

The PROs used in this study were the ACL-Return to Surgery after Injury scale (ACL-RSI),27 Knee Injury and Osteoarthritis Outcome Score subscale Quality of Life (KOOS QoL),28 and Function in sport and recreation (KOOS Sports), the 18-item version of the Knee Self-Efficacy Scale (K-SES18),29 and Tegner Activity Scale.26

The ACL-RSI aims to measure patients’ emotions, confidence, and risk appraisal of RTS after an ACL injury. Herein, the validated 12-item short version was used.27 Each item was graded from 0 to 10, where 10 is the highest response, representing the best possible psychological response to RTS (highest confidence and emotion, and lowest risk appraisal). The final score is calculated by summing the total score of all items (highest score 120), and then normalizing the score to a 0-100 scale as proposed in the original paper.27

The K-SES18 aims to evaluate knee-related self-efficacy,29 that is, the belief in one’s ability to perform a physical task, such as running or jumping. The K-SES18 comprises 18 items divided into two subscales: present (14 items) and future (four items) knee self-efficacies. Each item was graded from 0 to 10, with 10 being the most positive response, representing the greatest belief in carrying out a given physical task. The results from each item were added and divided by the number of items to generate a mean value for the subscales.

The KOOS QoL and KOOS Sports were used in the assessment of patients with ACL injury.28 They comprise four (QoL) and five (Sports) items, respectively. The KOOS QoL assess how often patients are reminded of their knee problems, whether patients can trust their knee function, have to make life changes due to knee function, and whether patients experience problems related to the knee. The KOOS Sports comprises five items assessing a patient’s perceived difficulty in performing different tasks, such as running and jumping during the last week. Each item is rated from 0 to 4 on a 5-point Likert scale, and a normalized score from 0 to 100 is calculated for each subscale, where 0 indicates the most severe symptoms, and 100 indicates no symptoms.

The Tegner Activity Scale (Tegner)26 aims to measure strenuous knee activity. Patients grade their activity based on work and sports activities on a scale from 1 to 10, where one represents disability because of knee problems and 10 represents, for example, national or international level soccer. From level 6 on the Tegner, only sports activities are registered, and therefore when patients rate Tegner ≥6, it is assumable patients are active in a knee strenuous sport.

Strength testing

Strength tests for unilateral concentric knee extension and flexion were performed according to a standardized protocol,30 with an isokinetic dynamometer [Biodex System 4 (Biodex Medical Systems, Shirley, New York, USA)]. The testing procedure started with a standardized warm-up of 10 minutes on a stationary bike and sub-maximum trials on each test. The injured leg was tested first, followed by the uninjured leg. Isokinetic testing was performed at an angular velocity of 90°/s with the patients in a seated position. Three maximum repetitions with approximately 40 s of rest between each repetition were performed, and the greatest peak torque was recorded in the Project ACL database. For this study, the results from the unilateral knee extension test were extracted for analysis.

Study execution

In this study, demographic data, results from PROs, and strength tests for the quadriceps were extracted for analysis from the Project ACL in November 2021.

Time frame for returning to knee-strenuous sports typically varies between 6-13 months31 after primary ACL reconstruction. Results of the PROs and strength tests from the follow-up closest in time to RTS (Tegner Activity Scale, level ≥6) were selected for the included patients as the “index test.”

Primary outcomes of this study were the cut-off values with sensitivity and specificity for predicting a second ACL injury for each of the included PROs. Therefore, patients who did not go on to suffer a second ACL injury were treated as a reference, and patients who suffered a second ACL injury were treated as having the outcome of interest. Clinical reference standard was not applicable, as the analysis aimed to predict the outcome of interest. In order to account for the recovery of symmetrical quadriceps strength as a protective factor,21,32 sensitivity analyses were performed with regard to patients who had and had not recovered ≥90% of their quadriceps strength in the injured limb compared to the uninjured limb.33 Symmetrical quadriceps strength was presented with the limb symmetry index (LSI), where result from the injured limb was divided with result from the uninjured limb and multiplied by 100.

Statistics

Receiver operating characteristics (ROC) were calculated for each PRO at available follow-ups within two years of RTS (Tegner ≥6). The ROC is a graphical method of displaying the discriminatory accuracy of a marker (in this case, responses to PROs) for separating two populations, or distinguishing between patients affected by an outcome, that is, suffering a second ACL injury within two years after RTS (return to Tegner ≥6), and individuals not affected by the outcome of interest, that is, patients who did not suffer a second ACL injury within two years after RTS. A patient was assessed as “positive” if the tested PRO value was greater than a given threshold value; otherwise, the patient was assessed as negative.34 The accuracy of any given threshold value was measured by the probability of a true positive (sensitivity) and true negative (specificity), and presented with two summary measures for ROC: the area under the curve (AUC) and Youden index. The area under the curve (AUC) is a measure of the ability of a classifier to distinguish between outcomes and is used as a summary of the ROC curve. The higher the AUC, the better the performance of the model in distinguishing between positive and negative outcomes. When AUC is comprised between 0.5 and 1, there is a chance that the classifier can be able to distinguish the positive class values from the negative class values, depending on the AUC value, according to the following rule of thumb: 0.5 = no discrimination, with the same value of a coin flip; 0.5-0.7 = poor discrimination, not much better than a coin flip; 0.7-0.8 = acceptable discrimination; 0.8-0.9 = excellent discrimination; and >0.9 = outstanding discrimination.35 Therefore, when interpreting the AUC; values between 0.7 to 0.8 were considered to reflect acceptable accuracy.36 The Youden Index is a frequently used summary measure of the ROC and it measures the effectiveness of a diagnostic marker and enables the selection of an optimal threshold value (cut-off point) for the marker.34 The Youden index value ranges from 0 to 1,37 where 0 means that a diagnostic test gives the same proportion of positive results for groups with and without the disease, i.e. the test is useless, while a value of 1 indicates that there are no false positives or false negatives, i.e. the test is perfect.

Statistical analyses were performed using Statistical Analysis System (SAS) software version 9 (SAS Institute Inc., Cary, North Carolina, USA). Mean values with standard deviations (SD) or medians with minimum and maximum are presented for the demographic data. Significance level was set at 0.05.

RESULTS

In total, 641 patients (355 men, 61%) were included in the present study. Figure 1 shows the inclusion and exclusion process. Table 1 presents the demographic characteristics of the patients.

Figure 1. Flowchart on inclusion and exclusion.

Figure 1.

Open in a new tab

ACL = Anterior Cruciate Ligament; RTS = Return to Sport (return to Tegner Activity level ≥6); n = number

Table 1. Demographic data, mean values, standard deviations (SD), count (n) and proportions (%).

All patients; n=641 Men; n=355 Women; n=286
Age at reconstruction, years 24.8 (7.6) 25.8 (7.4) 23.5 (7.6)
Height, cm 175.7 (9.3) 181.7 (6.4) 168.2 (6.5)
Weight, kg 73.5 (12.7) 80.9 (10.2) 64.2 (8.9)
BMI 23.7 (2.8) 24.5 (2.6) 22.7 (2.7)
Time to RTS, months 11.8 (15.1) 10.3 (13.8) 13.6 (16.4)
Tegner pre-injury level, n (%)
6 44 (6.9%) 20 (5.6%) 24 (8.4%)
7 108 (16.8%) 66 (18.6%) 42 (14.7%)
8 158 (24.6%) 69 (19.4%) 89 (31.1%)
9 213 (33.2%) 129 (36.3%) 84 (29.4%)
10 118 (18.4%) 71 (20.0%) 47 (16.4%)
Returned to pre-injury Tegner
Yes, n (%) 442 (69.0%) 248 (69.9%) 194 (67.8%)
Returned to pre-injury Tegner
Months from index reconstruction
4 108 (24.4%) 79 (31.8%) 29 (15%)
5-8 75 (17%) 45 (18.1%) 30 (15.5%)
9-12 128 (29%) 61 (24.6%) 67 (34.5%)
12-24 70 (15.8%) 35 (14.1%) 35 (18.0%)
>24 61 (13.8%) 28 (11.3%) 33 (17%)
Graft choice
Hamstring graft, n (%) 465 (72.5%) 260 (73.2%) 205 (71.7%)
Patellar graft, n (%) 77 (12.0%) 43 (12.1%) 34 (11.9%)
Other graft, n (%) 6 (0.9%) 2 (0.6%) 4 (1.4%)
Unknown n (%) 93 (14.5%) 50 (14.1%) 43 (15.0%)
Second ACL injury within 24 months from RTS
Yes n (%) 64 (10.0%) 31 (8.7%) 33 (11.5%)
Ipsilateral second ACL injury
n (%)		 43 (67.2%) 22 (71.0%) 21 (63.6%)
Contralateral second ACL Injury n (%) 21 (32.8%) 9 (29.0%) 12 (36.4%)
Open in a new tab

ACL = Anterior Cruciate Ligament; BMI = Body Mass Index; cm = centimeters; kg = kilograms; n = number; RTS= Return to Sport; Tegner = Tegner Activity Scale

Figure 2 presents the frequency of a second ACL injury stratified by months from return to knee-demanding activity (Tegner ≥6).

Figure 2. Frequency of a second ACL injury for every month after RTS.

Figure 2.

Open in a new tab

ACL = Anterior Cruciate Ligament; RTS = Return to Sport (defined as return to Tegner Activity level ≥6); n = number

Table 2 presents the Youden index with specificity and sensitivity, as well as the model AUC for all assessed PROs.

Table 2. Cut-off values, Youden index, and specificity and sensitivity for the included PROs in the analysis.

n Cut-off Youden Index 1 - Specificity Sensitivity Model AUC
ACL-RSI 382 71.7 0.13 0.37 0.50 0.553
KOOS QoL 627 56.0 0.12 0.47 0.59 0.531
KOOS Sports 627 96.0 0.12 0.07 0.19 0.557
K-SES18 present 630 9.4 0.09 0.26 0.34 0.520
K-SES18 future 630 7.0 0.08 0.70 0.78 0.511
Open in a new tab

PROs = Patient Reported Outcomes; n = number i.e., patients responding (please note ACL-RSI is not administered at all follow-ups); ACL-RSI = Anterior Cruciate Ligament Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale; AUC = Area Under the Curve

The cut-offs that best differentiated patients who suffered a second ACL injury after ACL reconstruction from patients who had not were: ≥71.7 for the ACL-RSI, <56 for the KOOS QoL, ≥96.0 for the KOOS Sports, ≥9.4 for the K-SES18 present and ≥7.0 for the K-SES18 future. The AUC and Youden index for each cut-off were below acceptable values, indicating the inability of the cut-offs to properly differentiate between patients who suffered a second ACL injury after ACL reconstruction and those who did not. Table 3 presents the number of patients who suffered a second ACL injury in relation to the achievement of the cut-off values for the different PROs.

Table 3. Proportion and number of patients above or below cut-offs who suffered a second ACL injury.

PROs Cut off Second ACL injury, % (n) 95% CI
ACL RSI ≥71.7 14% (19) 8.4% 20.5%
<71.7 9% (21) 5.4% 12.9%
KOOS QoL ≥56 10% (40) 7.2% 13.2%
<56 11% (24) 7.0% 15.6%
KOOS Sports ≥96 23% (12) 12.5% 36.8%
<96 9% (52) 6.8% 11.7%
K-SES present ≥9.4 23% (12) 12.5% 36.8%
<9.4 9% (52) 6.8% 11.7%
K-SES future ≥7 23% (12) 12.5% 36.8%
<7 9% (52) 6.8% 11.7%
Open in a new tab

PROs = Patient Reported Outcomes; ACL = Anterior Cruciate Ligament; n = number i.e., patients responding; CI = Confidence Interval; ACL-RSI = Anterior Cruciate Ligament Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale

Effect of recovery of symmetrical quadriceps strength

Table 4 presents the cut-off analysis stratified according to patients who had and had not achieved an LSI of ≥90% in the quadriceps strength test at the time of RTS.

Table 4. Cut-off values, Youden index, and specificity and sensitivity for the included PROs in the analysis, stratified by whether patients achieved symmetrical quadriceps strength or not.

Patients with LSI of ≥90% (n=308)
n Cut-off Youden Index 1 - Specificity Sensitivity Model AUC
ACL-RSI 220 71.7 0.22 0.35 0.57 0.591
KOOS QoL 307 56.0 0.24 0.35 0.59 0.580
KOOS Sports 307 96.0 0.18 0.09 0.27 0.600
K-SES18 present 308 9.4 0.17 0.34 0.51 0.569
K-SES18 future 308 8.3 0.05 0.62 0.68 0.495
Patients with LSI of ≤90% (n=181)
ACL-RSI 87 46.7 0.23 0.77 1.00 0.561
KOOS QoL 179 38.0 0.13 0.81 0.94 0.537
KOOS Sports 179 55.0 0.24 0.61 0.84 0.613
K-SES18 present 180 6.1 0.21 0.74 0.95 0.537
K-SES18 future 180 8.0 0.22 0.57 0.79 0.523
Open in a new tab

PROs = Patient Reported Outcomes; n = number, i.e., patients responding (please note that ACL-RSI is not administered at all follow-ups); LSI = Limb Symmetry Index; ACL-RSI = Anterior Cruciate Ligament Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale; AUC = Area Under the Curve

Interpretation of results

Stratifying patients based on achieved symmetrical quadriceps strength (≥90% LSI) or not at the time of RTS did not lead to cut-off values with better psychometric properties, i.e., the Youden index or AUC compared with the cut-offs determined for the entire cohort. In the current results, the best Youden index was 0.24, and the best AUC was 0.613, which is well below acceptable levels.36,37 Therefore, the calculated cut-offs appear unable to separate the two outcomes, with or without a second ACL injury.

Table 5 presents the number of patients who suffered a second ACL injury in relation to the achievement of the cut-offs for the different PROs in patients who achieved symmetrical quadriceps strength at the time of RTS.

Table 5. Proportion and number of patients above or below the determined cut-offs who suffer a second ACL injury were stratified, depending on whether the patients achieved symmetrical quadriceps strength or not at the time of RTS.

PROs Cut off Second ACL injury,
% (n)		 95% CI
Patients with LSI of ≥90% (n=308) ACL-RSI ≥71.7 18.7% (15) 10.9% 29.0%
<71.7 9.3% (13) 5.0% 15.4%
KOOS QoL ≥56 11.6% (27) 7.8% 16.5%
<56 13.3% (10) 6.6% 23.2%
KOOS Sports ≥96 29.4% (10) 15.1% 47.5%
<96 9.9% (27) 6.6% 14.1%
K-SES18 present ≥9.4 29.4% (10) 15.1% 47.5%
<9.4 9.9% (27) 6.6% 14.1%
K-SES18 future ≥8.3 29.4% (10) 15.1% 47.5%
<8.3 9.9% (27) 6.6% 14.1%
Patients with LSI of ≤90% (n=181) ACL RSI ≥46.7 7.2% (9) 3.3% 13.2%
<46.7 8.1% (3) 1.7% 21.9%
KOOS QoL ≥38 9.1% (24) 5.9% 13.3%
<38 5.3% (3) 1.1% 14.6%
KOOS Sports ≥55 8.9% (19) 5.5% 13.6%
<55 7.4% (8) 3.3% 14.1%
K-SES18 present ≥6.1 8.9% (19) 5.5% 13.6%
< 6.1 7.4% (8) 3.3% 14.1%
K-SES18 future ≥8.0 8.9% (19) 5.5% 13.6%
<8.0 7.4% (8) 3.3% 14.1%
Open in a new tab

PROs = Patient Reported Outcomes; ACL = Anterior Cruciate Ligament; n = number i.e., patients responding; CI = Confidence Interval; ACL-RSI = Anterior Cruciate Ligament-Return to Sport after Injury Scale; KOOS = Knee injury and Osteoarthritis Outcome Score; QoL = subscale Quality of Life; K-SES = Knee Self-Efficacy Scale; RTS = Return To Sport

DISCUSSION

The main finding from this registry study was that the determined cut-off values for different PROs were not able to differentiate patients who suffered a second ACL injury up to 24 months after RTS following ACL reconstruction from patients who did not. Additionally, achieving symmetrical quadriceps strength, i.e., ≥90% LSI did not improve the cut-offs’ psychometric properties. Regardless of recovering symmetrical quadriceps strength, patients with either a higher or lower knee perceived quality of life, knee-related self-efficacy, confidence, emotions, and risk appraisal appear as likely to suffer a second ACL injury. Therefore, it can be assumed that passing the calculated cut-off for PROs used in the present study is not sufficient for safe RTS. Caution must thus be taken if patients are cleared to RTS based on the cut-off values used in the present study, regardless, if they achieve ≥90% LSI in quadriceps strength or not.

The used cut-off values for PROs in this study could not generate an acceptable AUC value (0.7-0.8),36 indicating that the cut-off values are no better than ‘flipping a coin’ when used to assess the risk of suffering a second ACL injury. The poor discriminative ability of the cut-offs can be partly explained by the specificity and sensitivity values. The sensitivity ranged between 0.19-0.59 for all the cut-offs except for the K-SES18 future (0.78). Using the ACL-RSI as an example, a sensitivity of 0.50 means that half of the patients who suffered a second ACL injury had a score above 71.7. Further, when a patient scores above 71.7 on the ACL-RSI, there is approximately a 50% chance to correctly guess if the patient will go on to suffer a second ACL injury (AUC = 0.553); therefore, the psychometric value for the cut-off is very low (Youden index = 0.13). Since the cut-offs that best differentiated patients who suffered a second ACL injury from patients who did not have low AUC values,36 there is a possibility that the PROs used in this study are not suitable for identifying patients who are at increased risk of a second ACL injury at the time of RTS.

One issue related to the inability of PROs to identify patients who will continue to suffer a second ACL injury can be related to the development of the PROs used in this study. The ACL-RSI was developed to assess confidence, emotion, and risk appraisal of RTS after ACL injury,27 which are three different psychological domains (constructs). However, the ACL-RSI is summarized into a single score, ranging between to 0-100; this may be questionable because the scale comprises three domains,38 which can lead to limitations as it induces indirectness towards the outcome when interpreting the PRO as a single score.

When assessing the KOOS, using a stringent psychometric method, i.e. the Rasch analysis, criteria for one-dimensionality are respected only in two out of the five subscales, i.e. KOOS Sports and KOOS Quality of life.39 Furthermore, the KOOS´s development quality has been rated as “inadequate,” and the KOOS´s psychometric properties with regard to patients with an ACL injury have been reported to be “poor,” “insufficient,” and “inconsistent”, but the authors opinions are based on that studies evaluating the psychometric properties are lacking.38,40 The inconsistency reported in the psychometric properties of the KOOS with regard to patients with an ACL injury can be due to issues during scale development, since no patients who suffered an ACL injury participated in the development and were asked whether the items were relevant.

As for the K-SES18, a recent publication reported that it has acceptable reliability and validity for measuring self-efficacy in patients with ACL injury.41 However, no Rasch analysis has yet been performed on the K-SES18, and good psychometric values can be obtained by correlating the K-SES18 with other scales of debatable psychometric properties, such as the KOOS, leading to possible misinterpretation of results. Finally, the inability of PROs to identify patients who would suffer a second ACL injury is likely related to the fact that patients with a second ACL injury were not taken into account when developing PROs.

Limitations

One limitation of this study was the relatively small number of patients in the second ACL injury group (n=64), which could have influenced the statistical power. A post-hoc power calculation was performed, which showed that 33 patients in each group were needed to detect an AUC of 0.7 with a 90% power and an alpha of 0.05. Due to the heterogeneity in the population of patients who suffered an ACL injury, 33 patients per group might still not be sufficient for the statistical power; therefore, the results should be appreciated with caution. Another limitation of the present study could be its conceptual character. A second ACL injury is multifactorial, and both high and low values on PROs can be risk factors for a second ACL injury.20 Assuming that both too high and too low knee related self-efficacy could induce a greater risk for a second ACL injury, using only one cut-off value (reflecting a greater risk for a second ACL injury) for one single scale could be conceptually wrong. If a PRO is sensitive in predicting a second ACL injury, future studies might need to use two different cut-off values: taking K-SES as an example, one cut-off reflecting an extremely high knee-related self-efficacy, and one cut-off reflecting an extremely low knee-related self-efficacy. Notably, some patients returned to the pre-injury Tegner level as early as four months after the ACL reconstruction. It is unclear how this might have affected the results. A further possible limitation of this study concerns age as the ACL-RSI has been reported to be more sensitive for change in younger patients (≤ 20 years) who go on to suffer a second ACL injury.16 Patients in this cohort were on average 24.8 years, and it is not known whether the calculated cut-offs are influenced by age, and if age-specific cut-offs may be better to predict second ACL injuries.

Conclusion

The determined cut-off values for three commonly used psychological PROs could not differentiate patients who suffered a second ACL injury from those who did not within two years from RTS, regardless of whether the patients achieved symmetrical quadriceps strength or not, in a cohort of patients who had undergone with ACL reconstruction. Therefore, clinicians are recommended to not only rely on cut-off values or a single PRO of those analyzed in this study when making decisions on which patients are at risk of experiencing a second ACL injury when returning to sports after ACL reconstruction.

Conflict of interest

Authors declare no conflict of interest.

Acknowledgments

Acknowledgments

We would like to thank Editage (www.editage.com) for English language editing, and Bengt Bengtsson at Statistiska Konsultgruppen for the help with statistical analysis.

References

  1. Operative and nonoperative treatment options for ACL tears in the adult patient: a conceptual review. Bogunovic Ljiljana, Matava Matthew J. Nov;2013 The Physician and Sportsmedicine. 41(4):33–40. doi: 10.3810/psm.2013.11.2034. doi: 10.3810/psm.2013.11.2034. [DOI] [PubMed] [Google Scholar]
  2. Wiggins Amelia J., Grandhi Ravi K., Schneider Daniel K., Stanfield Denver, Webster Kate E., Myer Gregory D. The American Journal of Sports Medicine. 7. Vol. 44. SAGE Publications; Risk of secondary injury in younger athletes after anterior cruciate ligament reconstruction: A systematic review and meta-analysis; pp. 1861–1876. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Comparison of concomitant injuries and patient-reported outcome in patients that have undergone both primary and revision ACL reconstruction—a national registry study. Svantesson Eleonor, Hamrin Senorski Eric, Kristiansson Frida, Alentorn-Geli Eduard, Westin Olof, Samuelsson Kristian. Jan 10;2020 Journal of Orthopaedic Surgery and Research. 15(1):9. doi: 10.1186/s13018-019-1532-z. doi: 10.1186/s13018-019-1532-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Histopathological changes in the human posterior cruciate ligament during aging and osteoarthritis: correlations with anterior cruciate ligament and cartilage changes. Levy Yadin D, Hasegawa Akihiko, Patil Shantanu, Koziol James A, Lotz Martin K, D'Lima Darryl D. 2013Annals of the Rheumatic Diseases. 72(2):271–277. doi: 10.1136/annrheumdis-2012-201730. doi: 10.1136/annrheumdis-2012-201730. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Fear of re-injury: a hindrance for returning to sports after anterior cruciate ligament reconstruction. Kvist Joanna, Ek Anna, Sporrstedt Katja, Good Lars. Feb 10;2005 Knee Surgery, Sports Traumatology, Arthroscopy. 13(5):393–397. doi: 10.1007/s00167-004-0591-8. doi: 10.1007/s00167-004-0591-8. [DOI] [PubMed] [Google Scholar]
  6. Sports participation 2 years after anterior cruciate ligament reconstruction in athletes who had not returned to sport at 1 year: a prospective follow-up of physical function and psychological factors in 122 athletes. Ardern Clare L., Taylor Nicholas F., Feller Julian A., Whitehead Timothy S., Webster Kate E. Jan 12;2015 The American Journal of Sports Medicine. 43(4):848–856. doi: 10.1177/0363546514563282. doi: 10.1177/0363546514563282. [DOI] [PubMed] [Google Scholar]
  7. Return to sport following anterior cruciate ligament reconstruction surgery: a systematic review and meta-analysis of the state of play. Ardern C. L., Webster K. E., Taylor N. F., Feller J. A. Mar 11;2011 British Journal of Sports Medicine. 45(7):596–606. doi: 10.1136/bjsm.2010.076364. doi: 10.1136/bjsm.2010.076364. [DOI] [PubMed] [Google Scholar]
  8. Paterno Mark V., Rauh Mitchell J., Schmitt Laura C., Ford Kevin R., Hewett Timothy E. The American Journal of Sports Medicine. 7. Vol. 42. SAGE Publications; Incidence of second ACL Injuries 2 years after primary ACL reconstruction and return to sport; pp. 1567–1573. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Confidence, ability to meet return to sport criteria, and second ACL injury risk associations after ACL-reconstruction. Paterno M.V., Thomas S., VanEtten K.T., Schmitt L.C. 2021J Orthop Res. doi: 10.1002/jor.25071. [DOI] [PubMed]
  10. Greater psychological readiness to return to sport, as well as greater present and future knee-related self-efficacy can increase the risk for an ACL re-rupture: a matched cohort study. Piussi Ramana, Beischer Susanne, Thomeé Roland, Thomeé Christoffer, Sansone Mikael, Samuelsson Kristian, Hamrin Senorski Eric. Apr;2022 Arthroscopy: The Journal of Arthroscopic & Related Surgery. 38(4):1267–1276.e1. doi: 10.1016/j.arthro.2021.08.040. doi: 10.1016/j.arthro.2021.08.040. [DOI] [PubMed] [Google Scholar]
  11. Psychological predictors of anterior cruciate ligament reconstruction outcomes: a systematic review. Everhart Joshua S., Best Thomas M., Flanigan David C. 2015Knee Surgery, Sports Traumatology, Arthroscopy. 23(3):752–762. doi: 10.1007/s00167-013-2699-1. doi: 10.1007/s00167-013-2699-1. [DOI] [PubMed] [Google Scholar]
  12. Psychosocial factors influencing the recovery of athletes with anterior cruciate ligament injury: a systematic review. te Wierike S.C., van der Sluis A., van den Akker-Scheek I., Elferink-Gemser M.T., Visscher C. 2013Scand J Med Sci Sports. 23(5):527–540. doi: 10.1111/sms.12010. [DOI] [PubMed] [Google Scholar]
  13. How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II) Andrade Renato, Pereira Rogério, van Cingel Robert, Staal J Bart, Espregueira-Mendes João. 2020British Journal of Sports Medicine. 54(9):512–519. doi: 10.1136/bjsports-2018-100310. doi: 10.1136/bjsports-2018-100310. [DOI] [PubMed] [Google Scholar]
  14. DiSanti Justin, Lisee Caroline, Erickson Karl, Bell David, Shingles Michael, Kuenze Christopher. Journal of Orthopaedic & Sports Physical Therapy. 12. Vol. 48. Journal of Orthopaedic & Sports Physical Therapy (JOSPT); Perceptions of rehabilitation and return to sport among high school athletes with anterior cruciate ligament reconstruction: A qualitative research study; pp. 951–959. [DOI] [PubMed] [Google Scholar]
  15. Muller Bart, Yabroudi Mohammad A., Lynch Andrew, Lai Chung-Liang, van Dijk C. Niek, Fu Freddie H., Irrgang James J. The American Journal of Sports Medicine. 11. Vol. 44. SAGE Publications; Defining thresholds for the patient acceptable symptom state for the IKDC subjective knee form and KOOS for patients who underwent ACL reconstruction; pp. 2820–2826. [DOI] [PubMed] [Google Scholar]
  16. McPherson April L., Feller Julian A., Hewett Timothy E., Webster Kate E. The American Journal of Sports Medicine. 4. Vol. 47. SAGE Publications; Psychological readiness to return to sport is associated with second anterior cruciate ligament injuries; pp. 857–862. [DOI] [PubMed] [Google Scholar]
  17. Risk factors associated with revision and contralateral anterior cruciate ligament reconstructions in the Kaiser Permanente ACLR registry. Maletis Gregory B., Inacio Maria C.S., Funahashi Tadashi T. 2015The American Journal of Sports Medicine. 43(3):641–647. doi: 10.1177/0363546514561745. doi: 10.1177/0363546514561745. [DOI] [PubMed] [Google Scholar]
  18. Risk Factors and predictors of subsequent ACL injury in either knee after ACL reconstruction: prospective analysis of 2488 primary ACL reconstructions from the MOON cohort. Kaeding Christopher C., Pedroza Angela D., Reinke Emily K., Huston Laura J., Spindler Kurt P., Amendola Annunziato, Andrish Jack T., Brophy Robert H., Dunn Warren R., Flanigan David, Hewett Timothy E., Jones Morgan H., Marx Robert G., Matava Matthew J., McCarty Eric C., Parker Richard D., Wolcott Michelle, Wolf Brian R., Wright Rick W., MOON Consortium Apr 21;2015 The American Journal of Sports Medicine. 43(7):1583–1590. doi: 10.1177/0363546515578836. doi: 10.1177/0363546515578836. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Incidence of second anterior cruciate ligament tears and identification of associated risk factors from 2001 to 2010 using a geographic database. Schilaty Nathan D., Nagelli Christopher, Bates Nathaniel A., Sanders Thomas L., Krych Aaron J., Stuart Michael J., Hewett Timothy E. Aug 1;2017 Orthopaedic Journal of Sports Medicine. 5(8):2325967117724196. doi: 10.1177/2325967117724196. doi: 10.1177/2325967117724196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Fältström Anne, Kvist Joanna, Bittencourt Natalia F.N., Mendonça Luciana D., Hägglund Martin. The American Journal of Sports Medicine. 6. Vol. 49. SAGE Publications; Clinical risk profile for a second anterior cruciate ligament injury in female soccer players after anterior cruciate ligament reconstruction; pp. 1421–1430. [DOI] [PubMed] [Google Scholar]
  21. Simple decision rules can reduce reinjury risk by 84% after ACL reconstruction: the Delaware-Oslo ACL cohort study. Grindem Hege, Snyder-Mackler Lynn, Moksnes Håvard, Engebretsen Lars, Risberg May Arna. May 9;2016 British Journal of Sports Medicine. 50(13):804–808. doi: 10.1136/bjsports-2016-096031. doi: 10.1136/bjsports-2016-096031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Webster Kate E., Hewett Timothy E. Sports Medicine. 6. Vol. 49. Springer Science and Business Media LLC; What is the evidence for and validity of return-to-sport testing after anterior cruciate ligament reconstruction surgery? A systematic review and meta-analysis; pp. 917–929. [DOI] [PubMed] [Google Scholar]
  23. Losciale Justin M., Zdeb Rachael M., Ledbetter Leila, Reiman Michael P., Sell Timothy C. Journal of Orthopaedic & Sports Physical Therapy. 2. Vol. 49. Journal of Orthopaedic & Sports Physical Therapy (JOSPT); The association between passing return-to-sport criteria and second anterior cruciate ligament injury risk: A systematic review with meta-analysis; pp. 43–54. [DOI] [PubMed] [Google Scholar]
  24. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Bossuyt P.M., Reitsma J.B., Bruns D.E.., et al. 2015Br Med J. 351:h5527. doi: 10.1136/bmj.h5527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Return to knee-strenuous sport after anterior cruciate ligament reconstruction: a report from a rehabilitation outcome registry of patient characteristics. Hamrin Senorski Eric, Samuelsson Kristian, Thomeé Christoffer, Beischer Susanne, Karlsson Jón, Thomeé Roland. 2017Knee Surgery, Sports Traumatology, Arthroscopy. 25(5):1364–1374. doi: 10.1007/s00167-016-4280-1. doi: 10.1007/s00167-016-4280-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Rating systems in the evaluation of knee ligament injuries. Tegner Y., Lysholm J. Clin Orthop Relat Res. 1985(198):43–49. [PubMed] [Google Scholar]
  27. Development and validation of a short version of the anterior cruciate ligament return to sport after injury (ACL-RSI) scale. Webster Kate E., Feller Julian A. Apr 1;2018 Orthopaedic Journal of Sports Medicine. 6(4):2325967118763763. doi: 10.1177/2325967118763763. doi: 10.1177/2325967118763763. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. The Knee injury and Osteoarthritis Outcome Score (KOOS): from joint injury to osteoarthritis. Roos Ewa M, Lohmander L Stefan. 2003Health and Quality of Life Outcomes. 1:64. doi: 10.1186/1477-7525-1-64. doi: 10.1186/1477-7525-1-64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Validation of an 18-item version of the Swedish Knee Self-Efficacy Scale for patients after ACL injury and ACL reconstruction. Beischer S., Hamrin Senorski E., Thomeé P., Thomeé R. Oct 25;2021 Journal of Experimental Orthopaedics. 8(1):96. doi: 10.1186/s40634-021-00414-2. doi: 10.1186/s40634-021-00414-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Development of a strength test battery for evaluating leg muscle power after anterior cruciate ligament injury and reconstruction. Neeter Camille, Gustavsson Alexander, Thomeé Pia, Augustsson Jesper, Thomeé Roland, Karlsson Jon. Feb 14;2006 Knee Surgery, Sports Traumatology, Arthroscopy. 14(6):571–580. doi: 10.1007/s00167-006-0040-y. doi: 10.1007/s00167-006-0040-y. [DOI] [PubMed] [Google Scholar]
  31. Eighty-three per cent of elite athletes return to preinjury sport after anterior cruciate ligament reconstruction: a systematic review with meta-analysis of return to sport rates, graft rupture rates and performance outcomes. Lai Courtney C H, Ardern Clare L, Feller Julian A, Webster Kate E. 2018British Journal of Sports Medicine. 52(2):128–138. doi: 10.1136/bjsports-2016-096836. doi: 10.1136/bjsports-2016-096836. [DOI] [PubMed] [Google Scholar]
  32. King Enda, Richter Chris, Daniels Katherine A.J., Franklyn-Miller Andy, Falvey Eanna, Myer Gregory D., Jackson Mark, Moran Ray, Strike Siobhan. The American Journal of Sports Medicine. 3. Vol. 49. SAGE Publications; Can biomechanical testing after anterior cruciate ligament reconstruction identify athletes at risk for subsequent ACL injury to the contralateral uninjured limb? pp. 609–619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Consensus criteria for defining ‘successful outcome’ after ACL injury and reconstruction: a Delaware-Oslo ACL cohort investigation. Lynch Andrew D, Logerstedt David S, Grindem Hege, Eitzen Ingrid, Hicks Gregory E, Axe Michael J, Engebretsen Lars, Risberg May Arna, Snyder-Mackler Lynn. 2015British Journal of Sports Medicine. 49(5):335–342. doi: 10.1136/bjsports-2013-092299. doi: 10.1136/bjsports-2013-092299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Fluss Ronen, Faraggi David, Reiser Benjamin. Biometrical Journal. 4. Vol. 47. Wiley; Estimation of the Youden Index and its associated cutoff point; pp. 458–472. [DOI] [PubMed] [Google Scholar]
  35. Assessing the Fit of the Model. Lemeshow H. 2013Applied Logistic Regression. :153–225.
  36. Receiver operating characteristic curve in diagnostic test assessment. Mandrekar Jayawant N. Sep;2010 Journal of Thoracic Oncology. 5(9):1315–1316. doi: 10.1097/jto.0b013e3181ec173d. doi: 10.1097/jto.0b013e3181ec173d. [DOI] [PubMed] [Google Scholar]
  37. Youden W. J. Cancer. 1. Vol. 3. Wiley; Index for rating diagnostic tests; pp. 32–35. [DOI] [PubMed] [Google Scholar]
  38. Psychometric properties of patient-reported outcome measures for use in patients with anterior cruciate ligament injuries: A Systematic Review. Gagnier J.J., Shen Y., Huang H. 2018J Bone Joint Surg Rev. 6(4):e5. doi: 10.2106/JBJS.RVW.17.00114. [DOI] [PubMed] [Google Scholar]
  39. Rasch analysis of the Knee injury and Osteoarthritis Outcome Score (KOOS): a statistical re-evaluation. Comins J., Brodersen J., Krogsgaard M., Beyer N. Jun;2008 Scandinavian Journal of Medicine & Science in Sports. 18(3):336–345. doi: 10.1111/j.1600-0838.2007.00724.x. doi: 10.1111/j.1600-0838.2007.00724.x. [DOI] [PubMed] [Google Scholar]
  40. Four of five frequently used orthopedic PROMs possess inadequate content validity: a COSMIN evaluation of the mHHS, HAGOS, IKDC-SKF, KOOS and KNEES-ACL. Hansen C.F., Jensen J., Odgaard A.., et al. 2021Knee Surg Sports Traumatol Arthrosc. doi: 10.1007/s00167-021-06761-y. [DOI] [PubMed]
  41. Validation of an 18-item version of the Swedish Knee Self-Efficacy Scale for patients after ACL injury and ACL reconstruction. Beischer S., Hamrin Senorski E., Thomeé P., Thomeé R. Oct 25;2021 Journal of Experimental Orthopaedics. 8(1):96. doi: 10.1186/s40634-021-00414-2. doi: 10.1186/s40634-021-00414-2. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from International Journal of Sports Physical Therapy are provided here courtesy of North American Sports Medicine Institute

RESOURCES