Efficacy and Degree of Bias in Knee Injury Prevention Studies: A Systematic Review of RCTs

Abstract

Background

Knee injury prevention programs have been developed to address the epidemic of knee injuries in young athletes. These programs include exercises that focus on balance, proprioception, and neuromuscular control. Some studies have suggested such specialized exercise programs may reduce the risk of knee injury.

Questions/purposes

We conducted a systematic review of randomized controlled trials (RCTs) of knee injury prevention programs and evaluated the risks of bias in these studies.

Methods

We performed a systematic search using MEDLINE (from 1966), CINAHL (from 1982), Cochrane (CENTRAL), and EMBASE (from 1974) in April 2011. The searches were limited to RCTs. Two reviewers independently assessed the internal validity of the included studies using the van Tulder critical appraisal tool for RCTs. Authors were contacted when internal validity was unclear in the methodology. Ten Level I studies (RCTs) met the inclusion criteria. The average risk of bias score for these studies was 7 of 11 (range, 5–10).

Results

Two of the 10 studies reported a reduction in knee injuries. Of the three studies that provided ACL injuries as an outcome measure, none showed a reduction in overall ACL injury.

Conclusions

Current evidence suggests most (eight of 10) well-designed RCTs show no difference of treatment benefit. Perhaps refinements of interventions may lead to a reduction in knee and ACL injuries in future trials. Limitations in the number and heterogeneity of currently published RCTs of injury prevention programs place restraints on quantifying intervention efficacy with a meta-analytic approach. Future research articles should more clearly describe the different elements of their methodology, consistent with the standards set forth by the CONSORT statement.

Introduction

ACL and acute knee injuries are increasingly recognized as serious problems in young athletes. In particular, female athletes have a three to five times higher risk of serious knee injury, compared with male athletes competing in similar sports [1–3, 9, 35]. The sports medicine community has focused on research and training programs to reduce these injuries.

Four categories of risk factors for noncontact ACL injuries were identified by a group of experts in several fields: physicians, physical therapists, epidemiologists, athletic trainers, and biomechanists [17]. This group divided risk factors into several categories: environmental, anatomic, hormonal, and neuromuscular, each with its own subcategories. To date, most preventative research has been aimed at targeting strength issues and neuromuscular development and control.

During the last 30 years, numerous centers [7, 8, 14, 18, 20, 25, 27, 33] have developed intervention programs designed to reduce the risk of ACL and other serious knee injuries in young athletes. Many programs document trends toward reducing knee injuries in treatment groups compared with control groups [18, 20, 23, 27, 32]; however, only two [8, 25] reported a statistically significant trend.

The purposes of our review therefore are to: (1) coalesce the primary literature of randomized controlled trials (RCTs) on knee injury prevention programs and critically evaluate the methodologic elements of the individual studies for internal validity; (2) analyze the overall injury prevention effectiveness of the identified literature, and (3) determine the degree of publication bias of RCTs of knee injury prevention studies.

Search Strategy and Criteria

We searched medical literature databases using the search terms “knee,” “injur$”, “ACL”, “prevent$”, and “anterior cruciate ligament” in an attempt to identify the primary literature on injury prevention programs for knee and ACL injuries. The search range was from inception to April 2011 for the following databases: MEDLINE via PubMed, EMBASE, CENTRAL-Cochrane Database, and CINAHL using the following search criteria: ((“knee injuries”[MeSH Terms] OR (“knee”[All Fields] AND “injuries”[All Fields]) OR “knee injuries”[All Fields] OR (“knee”[All Fields] AND “injury”[All Fields]) OR “knee injury”[All Fields]) AND (“prevention and control”[Subheading] OR (“prevention”[All Fields] AND “control”[All Fields]) OR “prevention and control”[All Fields] OR “prevention”[All Fields])) AND Clinical Trial[ptyp] and (ACL[All Fields] AND (“Inj Prev”[Journal] OR (“injury”[All Fields] AND “prevention”[All Fields]) OR “injury prevention”[All Fields])) AND Clinical Trial[ptyp]. The search included supplemental bibliographic reference searches of articles to identify potentially missed, relevant studies. The search strategy applied to the literature databases (PubMed, CINAHL, EMBASE) yielded a total of 374 articles.

Studies eligible for inclusion had to be RCTs, as defined by the Cochrane Handbook [21]; include ACL and/or knee injuries as an outcome measure; and involve interventions focusing on injury prevention. We excluded studies if they did not include either ACL or knee injuries as an outcome measure. Additionally, studies were excluded if they were nonrandomized, used only an exogenous modality (eg, bracing), and did not involve a jumping and/or cutting sport/activity. The rationale for the latter was, typically, noncutting sports have a low incidence of ligamentous injury about the knee [36, 43], and injury prevention would be too heterogeneous to compare. Our study design was not exclusive to one sex, age group, sport, or playing level. Our definition of knee injury for study inclusion was any trauma, whether contact or noncontact, whether acute or overuse, whether ligamentous or nonligamentous, occurring to the knee. Acute injuries were those with sudden onset with obvious trauma and overuse injuries were those occurring with an insidious onset.

Two reviewers (NLG, RWL) independently conducted the article selection and review process. The identified articles were reviewed to determine whether they met the inclusion and exclusion criteria. When disagreements were noted, a third reviewer (KGS) facilitated group consensus agreement. After application of inclusion and exclusion criteria to titles, abstracts, and full text, 11 of the 374 articles [6, 11–13, 16, 18, 28, 45–47, 50] were eligible for more detailed review (Fig. 1). Of the 11 articles initially included, one study [18] was excluded because the randomization scheme was inappropriate, according to direct contact with the authors. Therefore, 10 studies [6, 11–13, 16, 28, 45–47, 50] met the inclusion criteria for systematic review (Table 1). Of the studies included, seven were designed for soccer player injury prevention [11–13, 16, 45–47], two for handball injury prevention [28, 50], and one for active army recruits [6]. All but one study [13] used a prospective, cluster-randomized controlled study design scheme (Table 1).

Fig. 1.

A PRISMA flow diagram shows the selection process for studies included in our review.

Table 1.

Descriptive trial information

Study	Level of evidence	Study design	Mean followup (months)	Number of subjects at followup	Activity	Sex	Mean age (years)	Program exercises
Ekstrand et al. [11] (1983)	I	PCRCT	6	180	Soccer	Male	24.0 (17–37)	Multifaceted approach: protective gear, taping, warmup, and flexibility program
Wedderkopp et al. [50] (1999)	I	PCRCT	10	237	Team handball	Female	NR (16–18)	Balance training with functional warmup
Soderman et al. [45] (2000)	I	PCRCT	6	140	Soccer	Female	20.4 (NR)	Balance training
Olsen et al. [28] (2005)	I	PCRCT	8	1837	Team handball	Mixed	16.0 (15–17)	Multifaceted approach: warmup, balance, strength, and technique training
Steffen et al. [47] (2008)	I	PCRCT	6	2020	Soccer	Female	15.4 (13–17)	Multifaceted approach: core stabilization, balance, plyometrics, and strength training
Brushoj et al. [6] (2008)	I	PCRCT	3	977	Active army soldiers	Male	20.9 (19–26)	Strength training and stretching
Soligard et al. [46] (2008)	I	PCRCT	8	1892	Soccer	Female	15.4 (13–17)	Multifaceted approach: warmup, strength, plyometrics, and balance training
Engebretsen et al. [13] (2008)*	I	PRCT	8	131	Soccer	Male	NR	Balance training
Gilchrist et al. [16] (2008)	I	PCRCT	3	1435	Soccer	Female	19.9 (NR)	Multifaceted approach: warmup, stretching, strength, plyometric, and agility training
Emery and Meeuwisse [12] (2010)	I	PCRCT	5	744	Soccer	Mixed	NR (13–18)	Multifaceted approach: warmup, stretching, strength, agility, and balance training

* Only subjects in the high-risk group completing the knee injury prevention program were considered; PCRCT = prospective cluster-randomized controlled trail; PRCT = prospective randomized controlled trial; NR = not reported.

The primary outcomes of interest for this study were specific to reporting of knee injuries and/or ACL injuries. Although several studies reported secondary outcomes (ankle, hamstring, upper extremity, etc), our review focused on the studies including data relative to the knee and, more specifically, ACL injuries. However, when methodologic (Table 2) or outcome data (Table 3) were not explicitly identifiable in the analysis, we contacted the authors to clarify or verify the unclear or missing data. Included studies were evaluated for degree of clinical, methodologic, and statistical heterogeneity to determine the appropriateness of pooling data for meta-analysis.

Table 2.

Methodologic risk of bias results according to the van Tulder scale [48]

Methodologic design element	Ekstrand et al. [11]	Wedderkopp et al. [50]**	Soderman et al. [45]	Olsen et al. [28]	Steffen et al. [47]	Brushoj et al. [6]**	Soligard et al. [46]	Engebretsen et al. [13]	Gilchrist et al. [16]	Emery and Meeuwisse [12]
Adequate randomization	Yes*	Yes*	Yes*	Yes*	Yes	Yes	Yes*	Yes*	Yes*	Yes*
Concealment of allocations	Yes*	Yes*	Yes*	Yes	Yes*	Yes	Yes	Yes*	No	Yes*
Homogeneity of cohort	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
Subject blinding	No	No	No	No	No	Yes	No	No	No	No
Care provider blinding	No	No	No	No	No	Yes*	No	No	No	No
Outcome assessor blinding	No	Yes	No	Yes	Yes	Yes	Yes	No	No	Yes
Cointerventions avoided or similar	No	Yes*	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Adequate compliance	Yes*	Yes*	Yes	No	No	Yes	Yes	No	No*	No
Acceptable dropout rate with description	No*	No*	No	No	Yes	Yes	Yes	No	Yes	Yes
Timing of outcome assessment	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Intention-to-treat analysis	Yes*	Yes*	No	Yes	Yes	No*	Yes	Yes	No	Yes
Total score	6	8	6	7	8	10	9	5	5	8

* Not found in text of manuscript but verified through direct communication with the author; **studies requiring consensus protocol for inclusion.

Table 3.

Quantitative outcome data

Study	Knee injuries			ACL injuries			Power analysis
	Prevention	Control	p value*	Prevention	Control	p value*
Ekstrand et al. [11]	1.1% (1 of 90)	18.9% (17 of 90)	< 0.05	NR	NR	NR	NR
Wedderkopp et al. [50]	1.8% (2 of 111)	6.3% (8 of 126)	> 0.05	NR	NR	NR	NR
Soderman et al. [45]	12.9%‡ (8 of 62)	7.7%‡ (6 of 78)	NR	6.5%‡ (4 of 62)	1.3%‡ (1 of 78)	NR	Yes (a priori)
Olsen et al. [28]	2.0% (19 of 958)	4.3% (38 of 879)	≤ 0.05	0.3% (3 of 958)	1.1% (10 of 879)	> 0.05‡	Yes (a priori)
Steffen et al. [47]	3.4% (37 of 1073)	3.2% (30 of 947)	> 0.05	NR	NR	NR	Yes (a priori)
Brushoj et al. [6]	13.6% (66 of 487)	11.6% (57 of 490)	> 0.05	NR	NR	NR	Yes (a priori)
Soligard et al. [46]	3.1% (33 of 1055)	5.6% (47 of 837)	0.08	NR	NR	NR	Yes (a priori)
Engebretson et al. [13]†	10.8% (7 of 65)	12.1% (8 of 66)	0.93	NR	NR	NR	NR
Gilchrist et al. [16]	6.9% (40 of 583)	6.8% (58 of 852)	0.86	1.2% (7 of 583)	2.1% (18 of 852)	0.20	Yes (a priori)
Emery and Meeuwisse [12]	0.8% (3 of 380)	2.2% (8 of 364)	0.23	NR	NR	NR	Yes (a priori)

* As reported in individual studies using complex statistical calculations appropriate for cluster randomization; ^†only subjects from the high-risk group were analyzed, as knee injuries were compared for only this group; ^‡data verified by direct communication with author; NR = not reported.

A qualitative assessment of the heterogeneity of the included studies showed substantial differences among the interventions used in the studies. For example, a subset of the studies used an intervention program focusing on balance and proprioception [13, 45, 50]. Another subset used a multifaceted intervention approach consisting of strengthening, balance, stretching, or warmup [11, 12, 16, 28, 46, 47] (Table 1), whereas the remaining study focused solely on strengthening as its intervention [6].

Regarding the three studies that included ACL injury as an outcome measure, statistical testing showed a degree of statistical heterogeneity, which would not be appropriate for combining the separate trial estimates. Moreover, several studies had missing or incomplete data pertaining to the outcomes we were interested in evaluating. Therefore, quantitative pooling of data to perform a meta-analysis was deemed inappropriate.

We also reviewed the studies for several design elements using the scale of van Tulder et al. [48], including (1) randomization method, (2) concealment of allocation, (3) group similarities, (4) blinding of subject, provider, and outcome assessor, (5) cointerventions avoided or similar, (6) intention-to-treat analysis (ITT), (7) high dropout rates, (8) compliance, and (9) timing of outcome assessment. The literature suggests that Elements 1, 2, 4, 6, and 7 can lead to an exaggeration of treatment effects [34, 37, 39–42, 51].

Two reviewers (NLG, RWL) independently appraised the content of the included articles using a pretested and standardized data abstraction instrument [48]. Again, discrepancies and disagreements were resolved by bringing in a third reviewer (KGS). When the internal validity elements were either missing or not clear, we contacted the authors of the studies for verification or clarification. The analysis of publication bias and funnel plot derivation was performed with STATA^® statistical software (Release 11; StataCorp LP, College Station, TX, USA) using Egger’s test and Begg’s test, which are general approaches to test publication bias in the meta-analysis. Egger’s test has been widely used and has become a standard procedure [21, 31]. This test is based on a linear regression approach, which is parametric. However, Begg’s test is based on a rank correlation method, which is nonparametric. However, both tests have the power to detect publication bias even with a small study sample.

Results

Using a priori inclusion and exclusion criteria our search yielded 10 RCTs of knee injury prevention programs from major medical literature databases. The methodologic design elements for each of these studies were reviewed using the van Tulder scale [48], and the average risk of bias score for these studies was 7 of 11 (range, 5–10) (Table 2). Blinding of subjects and/or care providers was absent in the majority of the studies; however, given the nature of the study designs, this was difficult or impossible to accomplish. All randomized studies had adequate randomization methods. Appropriate concealment of allocations was found in all studies but one [16]. Of the studies reviewed, seven of 10 used an ITT analysis as part of their statistical methods. Although all internal validity measures were obtained from each study, we contacted the authors of all studies to clarify at least one element described by the van Tulder scale (Table 2).

Of the studies reviewed that reported knee injury as an outcome measure, two of 10 [11, 28] showed a reduction in knee injuries whereas seven of 10 [6, 12, 13, 16, 46, 47, 50] showed no reduction in overall knee injuries and one of 10 [45] did not provide the p value for the knee injury comparison we analyzed (Table 3). In comparison, of the studies that reported ACL injuries as an outcome measure, none of the three [16, 28, 45] showed a reduction in overall ACL injuries (Table 3). Four of the studies reported a nonsignificant increase in the proportion of knee injuries in the intervention group [6, 16, 45, 47], one of which reported a higher proportion of ACL injuries in the intervention group [45].

Regarding publication bias, after comparing the calculated relative risks of the studies published thus far on knee and/or ACL injury prevention programs, it appeared there was no gross publication bias as can be seen by the inverted shape of the funnel plot [10] (Fig. 2). This was confirmed by Egger’s test (p = 0.106) and Begg’s test (p = 0.107).

Fig. 2.

A funnel plot with 95% confidence limits shows no gross publication bias. SE (logRR) = standard error of the natural logarithm of the relative risk.

Discussion

Knee injuries create one of the highest clinical and public health, injury-related burdens given their ubiquity across the age continuum and their need for surgical intervention [15]. Furthermore, the knee is the most commonly injured joint in adolescent athletes [44] and more specifically the ACL is one of the most commonly injured knee structures in this age group [43]. The public, athletes, coaches, and researchers are becoming increasingly aware of this fact and injury prevention programs have been developed in an attempt to address this issue. However, only a handful of Level I studies have been designed to evaluate if specialized injury prevention programs can reduce knee injuries. The current study was a systematic review, which evaluated the internal validity, or risk of bias, of RCTs of knee injury prevention studies. This study also compared effectiveness reported in each trial and quantified the degree of publication bias on reporting of knee injury prevention.

Although we adhered to a strict protocol for collection, interpretation, and analysis, our study is not without limitations. First, we did not blind the reviewers to study author, institution, or journal in which the study was published. Although the extra step of blinding the reviewed manuscripts has been done in other reviews, it is an onerous extra step with little evidence to support its ability to protect against bias in systematic reviews and no difference in evaluation results have been shown [5]. Second, our definition of knee injury was broad and did not narrow down injuries to a single structure (eg, ACL). Our a priori definition was designed to capture relevant trials of knee injuries. In retrospect, had we operationalized knee injury to include only ligamentous injuries it would have yielded too few studies (three), which would have yielded a review of questionable value. Third, one of the authors (KGS) has published on knee injury prevention programs [33], which showed no treatment benefit, and an argument could be made that this could introduce selection bias. However, we attempted to control for this by removing this author as a reviewer of selected studies and involving him only when there were discrepancies needing consensus resolution. For example, if randomization was unclear to one reviewer but clear and appropriate to another, our third reviewer (KGS) was asked to review and discuss the study for a consensus agreement. Additionally, use of consensus agreement only occurred with two studies. First was inclusion of the study by Soderman et al. [45] owing to a missing p value; on consensus the three reviewers agreed that this study still yielded valuable information regarding study design, quality, and it was understood at that point pooling of data would not be possible owing to the degree of heterogeneity. Second was the inclusion of the study by Brushoj et al. [6].

Although the study by Brushoj et al. [6] looked exclusively at overuse injuries, this study fit our inclusion criteria for including knee injuries. Additionally, as Bahr [4] stated, overuse injuries can overlap with acute injuries and in some cases present as an acute injury. Furthermore, previous studies have suggested that injury prevention for acute injuries may be a way to prevent overuse injuries [24, 29, 30]. We found it appropriate to include this well-designed RCT on injury prevention in our review.

It is well understood that nonrandomization has the potential to lead to systematic bias, which can overestimate or underestimate the effects of a treatment. Other design flaws can lead to bias and potential exaggeration of treatment and intervention effects. For example, the importance of ITT analysis is its ability to preserve the randomization scheme used to allocate the subjects to either a treatment or control arm reducing the degree of statistical bias. Herman et al. [19] showed there is poor use of ITT analysis in orthopaedic studies. In the series we reviewed, seven of 10 used an ITT analysis; however, only five of 10 clearly stated its use in the methods section. Additionally, studies have shown that a lack of or inadequate concealment of allocation leads to exaggerated effects of interventions between 30% and 40% [26, 40]. In our study, we found nine of 10 studies performed adequate concealment of allocations; however, this was clearly described in the manuscript in only three of 10 studies [6, 28, 46]. Clearly, our review has elucidated the fact that many of these RCTs are well designed; however, collectively the methodologic elements of the studies are ill-described in the body of the manuscript. This same trend was noted by Hill et al. [22] in their 2002 study of RCTs in rheumatology. As they pointed out, and we echo these concerns, the discrepancies between published reports and the actual conduct of RCTs make it difficult to conduct a best-evidence synthesis of the literature.

In our systematic review of RCTs for knee injury and ACL injury prevention, two of 10 studies [11, 28] showed a reduction in overall knee injury, and none of the three studies reported a reduction in overall ACL injuries. However, although there was no reduction in overall ACL injuries, a subgroup analysis performed by Gilchrist et al. [16] showed a reduction in ACL injuries for individuals who had a prior ACL injury, ACL injuries occurring during practice, and ACL injuries occurring late in the season. Of the studies that failed to show a reduction, some had fewer knee injuries [12, 13, 46, 49] and ACL injuries [16, 28] in the treatment group and some of the studies had fewer knee injuries [6, 16, 45, 47] and ACL injuries [45] in the control group. Unfortunately, owing to the degree of heterogeneity among studies and missing or incomplete data, we were unable to resolve the overall treatment effect by pooling the data in a meta-analysis. Furthermore, in instances where the p values were not reported, we were unable to make these calculations because of incomplete or missing data and more complex randomization statistics.

After our analysis of research on the topic of injury prevention, we concur with the conclusions of Griffin et al. [17] that “presently there is insufficient although encouraging evidence supporting the premise that ACL prevention programs improve performance and/or alter what are thought to be key risk factors for injury”. At this time, some of the clinical trials suggest a treatment benefit for injury reduction, some show a harmful effect, and some show no difference. Based on this limited and conflicting information, we believe counseling patients about participation in these programs is warranted, but patients should be informed about the limited evidence of treatment efficacy. Although RCTs of this type are difficult to conduct and are logistically complex, further well-designed research trials will be necessary to determine whether these injuries can be prevented with specialized training programs and these trials should more clearly describe the different elements of their methodology, consistent with the standards set forth by the CONSORT statement [38]. In terms of future injury prevention design, the study by Olsen et al. [28] may serve as an example for future directed research on knee injury prevention, which consisted of boys and girls performing a multifaceted 15- to 20-minute warmup program (Table 1).