Clinical Outcome Reporting in Youth ACL Literature Is Widely Variable

Abstract

Background:

Advances in anterior cruciate ligament (ACL) reconstruction procedures in pediatric and adolescent patients have resulted in an increase in recent clinical studies on this topic. However, the consistency with which outcome measures are reported in this demographic is unknown.

Purpose:

To document outcome reporting patterns of youth ACL reconstruction studies in high-impact journals.

Study Design:

Systematic review; Level of evidence, 4.

Methods:

All articles published in 5 high-impact orthopaedic journals from 2010 to 2016 were reviewed to identify those reporting clinical outcomes of young patients who underwent ACL reconstruction. Studies that were nonclinical, reported on patients older than 18 years, or included fewer than 10 patients were excluded. Outcome measures used in all included studies were recorded.

Results:

Seventeen studies encompassing 772 subjects (mean age, 14.3 years; range, 6.3-18.0 years) were analyzed. Eight studies (47%) reported on Tanner stage of subjects, while 1 study reported skeletal age. Ten studies (59%) clearly documented the presence or absence of surgical complications. Range of motion was reported in 65% of studies. Leg-length discrepancy and angular deformity were each reported in 76% of studies, with 12% quantifying results through radiographic measurements. Ligament testing was variably defined by inclusion of instrumented testing (65%), Lachman test (53%), and pivot-shift test (53%). Fourteen studies (82%) explicitly reported on the rate of ACL rerupture, while 71% reported on the rate of revision surgery. Rate of return to preinjury activity was reported in 8 studies (47%), of which 2 defined criteria for return to sport and 3 defined the level of competitive sport. Patient-reported outcome measures (PROMs) were used variably. For the 3 most commonly reported PROMs (Lysholm, International Knee Documentation Committee, and Tegner), 24% of studies reported all 3 PROMs, 35% of studies reported 2 PROMs, and 6% of studies reported 1 PROM in isolation. A pediatric-specific PROM was reported in 1 of the 17 studies.

Conclusion:

Studies on pediatric ACL reconstruction published in high-impact journals unreliably defined subjects’ skeletal maturity, inconsistently reported on objective outcome measures, and used disparate adult-validated PROMs to assess subjective outcomes. These findings highlight the need for standardized, pediatric-specific outcome measures to be applied in future studies evaluating ACL reconstruction in children and adolescents.

Injuries to the anterior cruciate ligament (ACL) are increasingly prevalent among children and adolescents. An estimated 60 million Americans between 6 and 18 years of age participate in organized athletics.^23,36 Participation in youth sports promotes physical and psychosocial well-being, reduces the risk of developing chronic medical conditions, and is believed to promote healthy behaviors.^2,8,13,47 Increased participation in youth sports has led to a concomitant increase in musculoskeletal injuries in this population.^11,37 The initiation of athletics at a younger age, single-sport specialization, and increased year-round play all contribute to an increased incidence of ACL injuries, and improvements in diagnostic modalities have increased the diagnosis of such injuries.^3,18

Management of ACL injuries in skeletally immature patients is distinct from that in adults. Some surgical techniques traditionally used for adults are not performed in children and adolescents in order to prevent physeal violation and subsequent growth arrest. Alternative, physeal-respecting ACL reconstruction techniques have been developed to minimize the risk of this complication.^19,25 Newer, milestone-based postoperative rehabilitation programs also account for the variable neuromuscular development in young athletes.⁴⁹ Considerable debate regarding the optimal treatment strategies in this population persists and has resulted in heightened study of clinical outcomes among children and adolescents undergoing ACL reconstruction procedures. Ongoing clinical outcomes research is needed to delineate the indications for and outcomes associated with the treatment options currently in practice.

The extent to which these recent studies have used age-appropriate methods for evaluating clinical outcomes is unknown. This is important because standardized methods for evaluating and reporting surgical outcomes enable meaningful data pooling and the potential to draw powerful conclusions to guide clinical practice. The purpose of this study was to document outcome reporting patterns in high-impact studies of youth ACL reconstruction. In doing so, we aimed to provide a consistent framework for evaluation and/or highlight the need for future studies to employ the most commonly used and age-appropriate metrics, thereby enabling meaningful comparison of youth ACL outcomes studies.

Methods

Five high-impact orthopaedic journals were selected for identification of source articles: The American Journal of Sports Medicine (AJSM), The Journal of Bone and Joint Surgery (American volume) (JBJS), Clinical Orthopaedics and Related Research (CORR), Arthroscopy, and The Journal of Pediatric Orthopaedics (JPO). Prior studies have similarly used these journals to identify high-quality clinical studies in sports medicine.^21,32,34 All articles published in each of these journals from January 2010 to May 2016 were reviewed to identify those reporting clinical outcomes of young patients who underwent ACL reconstruction in recent years. Studies that were nonclinical, reported on patients older than 18 years, or entailed fewer than 10 patients were excluded.

Two computerized databases (PubMed, EMBASE) with a comprehensive index of each of the 5 journals were searched to identify articles. “ACL” or “anterior cruciate ligament” was searched with the phrases [“pediatric” or “adolescent” or “immature”] and [“outcomes” or “complications” or “adverse events”] as well as all relevant iterations. Two researchers independently retrieved articles and reviewed studies to determine eligibility for inclusion. All articles were screened initially by title and abstract to exclude irrelevant studies. Remaining articles underwent full-text review. Discrepancies between reviewers were resolved through consensus with the senior author (P.D.F.). After an initial search yielded 258 unique articles, 17 articles met eligibility criteria (Figure 1).^¶

Figure 1.

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram for pediatric anterior cruciate ligament clinical outcomes literature.

Characteristics for each study article were recorded, including journal title, authors’ country of origin, publication year, level of evidence, and study type (retrospective vs prospective). All clinical outcomes identified through full-text review of included studies were recorded.

Results

General Characteristics

Seventeen articles that reported on ACL reconstruction in pediatric populations were analyzed (Table 1). Fifteen studies (88%) were retrospective in design and reported level 4 evidence (Table 2). No level 1 or 2 studies were identified. The median number of participants per study was 29, and the mean patient age was 14.3 years (range, 6.3-18.0 years). The study population consisted of 42% females. Mean duration of follow-up was 3.8 years. Five studies (29%) reported patient-level data. Eight studies (47%) reported on Tanner stage of subjects, while 1 study reported the skeletal age of subjects using the Greulich-Pyle method.

Table 1.

Overview of Studies on Youth ACL Injuries

	% (n)
Journal
American Journal of Sports Medicine	52.9 (9)
Arthroscopy	23.5 (4)
Journal of Pediatric Orthopaedics	11.8 (2)
Journal of Bone and Joint Surgery (American volume)	5.9 (1)
Clinical Orthopaedics and Related Research	5.9 (1)
Level of evidence
1 or 2	0 (0)
3	12 (2)
4	88 (15)
Study type
Prospective randomized controlled trial	0 (0)
Prospective, nonrandomized	12 (2)
Retrospective	88 (15)

Table 2.

Characteristics of Included Studies (Chronological Order)^a

Study	Year	Journal	Level of Evidence	No. of Patients	Mean Patient Age at Surgery, yb
Krych et al27	2010	AJSM	4	99	16 (13-18)
Ellis et al14	2012	Arthroscopy	3	79	16 (14-18)
Hui et al22	2012	AJSM	4	16	12 (8-14)
Redler et al41	2012	Arthroscopy	4	18	14 (±0.9)
Goddard et al20	2013	AJSM	4	31	13 (8-16)
Kumar et al28	2013	JBJS	4	32	11 (9-14)
Cassard et al6	2014	JPO	4	28	13 (9-15)
Demange and Camanho10	2014	AJSM	4	12	11 (8-12)
Engelman et al15	2014	AJSM	3	73	15 (11-18)
Schmale et al42	2014	CORR	4	29	14 (±1)
Calvo et al5	2015	AJSM	4	27	13 (12-16)
Luo et al31	2015	AJSM	4	124	16 (±1.5)
Mansson et al33	2015	AJSM	4	29	15 (±1.1)
Willimon et al48	2015	AJSM	4	21	12 (9-14)
Domzalski et al12	2016	Arthroscopy	4	22	12 (10-13)
Larson et al30	2016	Arthroscopy	4	29	14 (9-16)
Cruz et al9	2017	JPO	4	103	12 (±1.8)

^aAJSM, American Journal of Sports Medicine; CORR, Clinical Orthopaedics and Related Research; JBJS, Journal of Bone and Joint Surgery (American volume); JPO, Journal of Pediatric Orthopaedics.

^bStandard deviation is reported in parentheses when available. Otherwise, age range is reported.

Objective Outcome Measures

Ten studies (59%) clearly documented the presence or absence of surgical complications. Rate of infection was reported in 29% of studies, rate of nerve injury was reported in 24%, and rate of deep vein thrombosis was reported in 12%.

Twelve studies (71%) reported on postoperative radiographic imaging. Two studies reported radiographic measurements to assess for lower limb deformity, while 1 study obtained postoperative magnetic resonance imaging (MRI) in all subjects. One study reported on MRI imaging for a subset of patients.

Range of motion was reported in 65% of studies. Leg length discrepancy and angular deformity were each reported in 76% of studies, with 12% providing radiographic measurements. Fifteen studies (88%) documented at least 1 form of laxity testing (Figure 2). Ligament testing was variably defined by inclusion of instrumented testing (65%), Lachman test (53%), and pivot-shift test (53%). Of 11 studies reporting instrumented laxity testing, 73% used KT arthrometer instruments (Medmetric), 18% used Aircast (Rolimeter), and 1 study reported measurements from Genourob (GNRB). All studies reporting KT-instrumented laxity testing used the KT-1000 arthrometer. In 82% of studies reporting instrumented laxity testing, no precise force settings were reported.

Figure 2.

(A) Type of anterior cruciate ligament (ACL) physical examination reported. Anterior drawer test was not reported in any included study. (B) Number of different ACL physical examination tests reported per study.

Four studies (24%) reported hop-testing results for assessment of knee function. Each study reported single-hop testing results, while 1 study additionally reported triple-hop testing results.

Fourteen studies (82%) explicitly reported on the rate of ACL reruptures, while 12 studies (71%) reported on the rate of revision surgery. Three studies (18%) reported the rate of contralateral ACL rupture.

Subjective Outcome Measures

Rate of return to preinjury activity was reported in 8 studies (47%). Of these studies, 2 defined criteria for determining eligible return to sport, and 3 defined the level of competitive sport.

Fifteen (88%) studies reported a total of 8 patient-reported outcome measures (PROMs). The remaining 2 studies reported on general patient satisfaction. Specific PROMs were used variably (Figure 3). Of the 3 most commonly reported PROMs (Lysholm, International Knee Documentation Committee [IKDC], and Tegner), 24% studies reported all 3 PROMs, 35% reported 2 PROMs, and 6% reported 1 PROM in isolation. Although the Tegner score was designed to complement the Lysholm score,^4,46 the Lysholm score was used independently in 4 studies (24%). A pediatric-specific PROM, the Pediatric International Knee Documentation Committee (Pedi-IKDC) subjective knee form,²⁶ was reported in 1 of the 17 studies.⁴⁸

Figure 3.

(A) A variety of patient-reported outcome measures (PROMs) were reported. One study used a PROM that is tailored to a young population. (B) Studies varied in the number of PROMs used. EQ-5D, EuroQol Group-5 Dimensions; IKDC, International Knee Documentation Committee questionnaire; KOOS, Knee injury and Osteoarthritis Outcome Score; Pedi-IKDC, Pediatric IKDC; SF, short form.

Discussion

The findings of the present research demonstrate that the level of evidence of published studies examining treatment of ACL injuries in pediatric and adolescent populations remains low. In a systematic review of all articles published through 2011, Moksnes et al³⁵ identified no randomized controlled trials and a dearth of prospective studies in the youth ACL literature. Our review of data from 2010 to 2016 reveals no measurable improvements in the quality of studies published in the field. Makhni et al³² recently performed a similar study specifically of the adult ACL reconstruction literature. Whereas 50% of the articles identified by Makhni et al were level 1 or 2 studies, all articles identified in the present analysis of youth ACL literature were level 3 or 4 studies.

The low quality and retrospective nature of available studies may contribute to the inconsistencies observed among study parameters and outcome measures. Widespread variability was noted in defining pediatric patients, including different chronologic ages and, in 2 instances, radiographic estimates of skeletal age. Particularly relevant to pediatric patient groups, the presence of limb length discrepancy or angular deformity was widely reported (76%); however, just 12% of studies reported quantitative radiographic measurements to objectively determine these outcomes. The widespread variability noted in clinical outcome measures may in part reflect the retrospective cohort study design most commonly used. Researchers relying on retrospective studies are limited to data that are readily available at their institutions. In clinical settings where surgeons care jointly for adult and pediatric ACL injuries, limited sample sizes among youth ACL injuries may preclude dedicated analysis of this subgroup.

Implementation of PROMs as a means to assess therapeutic interventions has been a transformative development in medical research.^7,43 The current study confirms the widespread use of PROMs in the youth ACL literature. However, no consensus is available regarding which measures are to be used uniformly. This issue, which has been identified in the adult ACL reconstruction literature as well,³² precludes efficient translation of outcomes data across studies.

A unique consideration for pediatric-focused research is that many PROMs developed and validated for adult patients may lack specificity for children and adolescents. To our knowledge, for this demographic the IKDC is the only known adult PROM that has been secondarily shown to be translatable to the pediatric version in adolescent athletes.⁴⁵ In a study by Oak et al,³⁸ the adult and pediatric forms of the IKDC were shown to be significantly different by 1.5 points, although this difference was not clinically significant. The authors concluded that the adult version of the IKDC could be used in adolescents aged 13 to 17 years. Conversely, a study of the IKDC by Iversen et al²⁴ established that children experienced difficulty in comprehending several aspects of the adult form of the IKDC. While an argument may exist in favor of continuing to use the adult form longitudinally in this demographic for investigations that are already underway, it stands to reason that new research in children and adolescents should preferentially use the pediatric version, which was specifically validated for that age group.³⁹ That said, only 1 study from our sample incorporated a pediatric-specific, validated instrument.⁴⁸ Recently developed pediatric-focused knee and activity PROMs may have a widespread impact if appropriately used for future clinical outcomes assessments in this demographic.^16,17,26

This study must be considered in the context of its limitations. Only articles published in the 5 aforementioned peer-reviewed journals were analyzed. Moreover, the study focused on the most recent 6-year period. Together, these inclusion criteria intentionally emphasize studies with the greatest potential to influence future research. However, they do not offer a comprehensive review of the entire youth ACL literature, nor do they offer sufficient historical context to identify trends in outcome measures used. It is also possible that focusing on articles from high-impact journals may not reveal the true variability in reporting of clinical outcome measures, which may be even more widespread than we have identified.

The present study reveals a need for more research on standardized, validated outcome measures tailored to pediatric patients with ACL injuries. Furthermore, ongoing clinical research should be performed prospectively or, at a minimum, should entail the retrospective analysis of prospectively collected registry data, as such research is less subject to selection bias than retrospective case series.⁴⁰ This recommendation is consistent with large-scale initiatives to promote both multicenter collaborative studies and more widespread use of patient registries.^1,44 Establishing procedure-specific patient registries is a best-practice approach to maximizing the informational yield in clinical outcomes assessment. Although patient registries require resources and infrastructure, new technologies are already allowing the effective implementation of such registries.²⁹ These recommendations should guide future directions of clinical outcomes research, and research should be undertaken to identify the most relevant outcome metrics for ACL reconstruction in children and adolescents.

Conclusion

Studies on pediatric ACL reconstruction published in high-impact journals unreliably defined subjects’ skeletal maturity, inconsistently reported on objective outcome measures, and used disparate adult-validated PROMs to assess subjective outcomes. These findings highlight the need for standardized, pediatric-specific outcome measures to be applied in future studies evaluating ACL reconstruction in children and adolescents; further, the development of patient registries and multicenter collaboration should be considered.