Which Metrics Are Being Used to Evaluate Children and Adolescents After ACL Reconstruction? A Systematic Review

Abstract

Purpose

To identify a comprehensive list of outcome measures previously used in the literature to evaluate clinical outcomes after reconstruction of the anterior cruciate ligament (ACL) in patients 18 years of age or younger.

Methods

A literature search was performed by querying MEDLINE, Embase and Cochrane computerized databases for relevant articles that reported clinical outcomes in pediatric patients undergoing ACL reconstruction. Studies that were nonclinical, that reported on patients older than 19 years, that were not available in English, or that included fewer than 10 patients were excluded. Outcome measures of all eligible studies were recorded.

Results

We identified 77 studies published between 1986 and 2018 in 20 peer-reviewed journals. The mean age of the patients was 13.9 years. The ACL rerupture rate was reported in 60% of studies; 32 studies (42%) reported a rate of return to preinjury activity or sports. The use of adult-validated patient-reported outcome measures were reported in 63 (82%) articles. The Lysholm (64%), International Knee Documentation Committee (IKDC) (56%) and Tegner (37%) scores were the most commonly reported. Two patient-reported outcome measures designed for pediatric patients (the Pedi-IKDC and Hospital for Special Surgery Pediatric Functional Activity Brief Scale (Pedi-FABS) were employed in 5 (6%) recent studies.

Conclusions

There is variability across studies in the metrics used to assess clinical outcomes following ACL reconstruction in children and adolescents. Validated pediatric-specific instruments were used infrequently.

Clinical Relevance

A large body of existing pediatric ACL-reconstruction literature relies on a variable set of outcome measures that have not been developed or validated for children and adolescents. More recently, contemporary studies have begun to employ pediatric- and adolescent-specific validated measures, yet their use remains uncommon.

Over the past 3 decades, injury to the anterior cruciate ligament (ACL) among children and adolescents has become increasingly recognized as a significant contributor to morbidity in pediatric sports medicine.^1,2 An increasingly diverse population of young athletes has resulted in growing participation in an array of organized youth sporting activities.³ A concomitant rise in ACL injuries has challenged sports-medicine providers to develop treatment strategies tailored to young athletes. Earlier entry into organized play, documented patterns of single-sport specialization and year-round competition have magnified the frequency and severity of such injuries.4, 5, 6, 7, 8

In concert with advancements in the surgical management of ACL injuries in youth, researchers have devised new tools for measuring the success of surgical interventions. Across health care, widespread implementation of patient-reported outcome measures (PROMs) has revolutionized the manner in which providers understand how patients respond to therapies.9, 10, 11 Several recent studies have focused on the utility of such outcome measures in pediatric sports medicine.12, 13, 14 To date, the plethora of reported outcome measures raises concern that data heterogeneity across the literature may impede meaningful data pooling that allows researchers to draw conclusions and to guide clinical practice.¹³ Moreover, outcome measures must be validated in the patient populations of interest in order to ensure the accuracy and validity of information being collected, a challenge when studying children and adolescents.15, 16, 17

The purpose of this study was to identify a comprehensive list of outcome measures previously used in the literature to evaluate clinical outcomes after ACL reconstruction in patients aged 18 years or younger. We hypothesized that the patient-reported outcomes used would vary greatly among studies and would not be validated for pediatric patients in some cases.

Methods

This systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.^18,19 Two authors performed independent queries of the scientific literature using 3 computerized literature databases (MEDLINE, Embase and Cochrane) indexed from the earliest possible date through January 17, 2018. The electronic search algorithm included the following Medical Subject Heading terms: “Adolescent” OR “child” OR “pediatrics” AND “anterior cruciate ligament” OR “anterior cruciate ligament reconstruction” AND “treatment outcome” OR “outcome assessment.”. In conjunction with university librarians, a comprehensive list of relevant iterations to each Medical Subject Heading term (e.g., “pediatric” OR “paediatric”) was incorporated into the final search strategy (Appendix 1). Articles were included if they reported outcome measures in young patients undergoing ACL reconstruction. Outcomes measures were defined as either patient-reported outcome instrument scores or significant events in the clinical course that influence outcome following ACL surgery, such as a complication, return to preinjury activities or sports, ACL rerupture, or secondary surgery. All levels of evidence were included. Studies that were nonclinical or comprised fewer than 10 patients, conference abstracts, studies unavailable in English, and those that reported on patients older than 19 years of age were excluded. This chronologic age was selected as a cut-off because it is associated with physeal closure at the knee at a population level and is the definition currently used by the World Health Organization.20, 21, 22, 23

Two researchers (C.M.B. and J.M.S.) independently screened the titles and abstracts of all articles using Covidence software (Covidence, Melbourne, Victoria, Australia) to determine study eligibility by applying the aforementioned criteria.²⁴ Studies were excluded if either the title or the abstract clearly refuted eligibility (N = 1,860). The remaining articles underwent full-text review. Discrepancies between reviewers were resolved by reaching consensus with 2 senior authors (M.J.M. and P.D.F.). Bibliographies from all included articles were reviewed manually to identify additional pertinent studies. To ensure that duplicate patient populations were not included, articles with cross-matching authors were reassessed to examine them for overlapping dates of enrollment. If 1 or more authors conducted studies with overlapping patient populations, the study with the greater number of patients, longer follow-up and/or more comprehensive outcome reporting was included.²⁵

The methodological quality of included studies was formally assessed using the Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies.²⁶ Adapting a system utilized by Montalvo et al.,²⁷ studies were scored based on the number of affirmative responses to the instrument’s checklist of 12 relevant questions. The percentage of studies that scored ≥ 75% and ≥ 50% for relevant questions was quantified.

The study characteristics and parameters of each article were extracted and recorded using a custom spreadsheet in Microsoft Excel (Microsoft, Redmond, WA, USA). Data consisted of information regarding study design and participants’ demographic information. All documented clinical outcomes identified by full-text review of included studies were recorded. Descriptive summary statistics were employed to report the frequency of the various outcome measures in the group of all studies, including the frequency and the years in which outcome measures validated for pediatric populations were used.

Results

From 2,404 unique articles initially identified, 80 articles met the eligibility criteria. Three studies were subsequently removed to avoid including duplicate patient populations,28, 29, 30 leaving 77 studies in the final qualitative review (Fig 1). Agreement between the 2 primary reviewers had a free marginal kappa of 0.67, indicative of substantial agreement.³¹

Fig 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram for pediatric anterior cruciate ligament clinical outcomes literature.

General Characteristics

Seventy-seven articles reporting the clinical outcomes of pediatric and adolescent patients undergoing ACL reconstruction were analyzed.31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107 Studies were performed by primary authors from 17 different countries and published in 20 different peer-reviewed journals, dated between 1986 and 2018 (Fig 2). Of the articles, 71 (92%) were retrospective in design, and 66 studies (86%) were composed of level 4 evidence. Level 3 studies used comparison groups to evaluate variables such as graft type^91,101 or performed a cost-effectiveness analysis.¹⁰⁰ No level 1 or level 2 studies were identified. The mean number of patients per study was 78 ± 141.3 (range 10 to 902 participants), consisting of 44% female patients. The mean chronological age at the time of surgery was 13.9 ± 1.5 years (range 10.7 to 17.2 years). Twenty-seven studies (35%) reported on patients’ Tanner stage, and 9 studies (12%) reported patients’ skeletal age through 1 of a variety of methods. The mean duration of follow-up was 4.2 ± 2.9 years (range 0.4 to 18.3 years). Individual patient data were reported in 17 studies (21%); 6 studies fulfilled at least 75% of Quality Assessment Tool criteria, and 67 studies fulfilled at least 50% of the criteria. Ten studies fulfilled fewer than 50% of criteria. All studies reported on the graft type used to perform the ACL reconstruction.

Fig 2.

The number of pediatric ACL reconstruction publications has increased over the past 20 years. Pediatric-specific patient-reported outcome measures (PROMs) have been implemented in a minority of studies in recent years.

Clinician-Reported Outcome Measures

The rates at which several postoperative clinical outcomes are reported in the existing literature are summarized in Table 1. Additionally, postoperative leg-length discrepancy and angular deformity were described in 60% of studies, whereas postoperative radiographic measurements were reported in 31% of studies. Nine studies (12%) reported on overall longitudinal growth following surgery. Two studies used preoperative full-length lower-extremity alignment radiographs as a comparison for postoperative radiographic data.^35,95 General assessments of range of motion were described in 39% of the articles. Instrumented ligament testing was performed in 56% of the articles. The KT arthrometer (Medmetric, San Diego, CA, USA) was the most commonly used instrument, and specific force settings were reported in 16% of the studies (6 of 37) that used the device.

Table 1.

Reporting of Objective Outcome Measures

Clinical Outcome Metric	% of Studies (N = 77) Reporting Metric
ACL rerupture rate	60% of studies
Revision ACL reconstruction rate	55% of studies
Rate of subsequent knee surgery	44% of studies
Postoperative complication rate	33% of studies
Infection rate	22% of studies
Arthrofibrosis rate	14% of studies
Nerve injury rate	12% of studies
Rate of contralateral ACL rupture	13% of studies

Thirty-two studies (42%) reported rates of return to preinjury activities or sports. Of these studies, “return to sport” was defined as patients’ preinjury sports level in 17 (53%) studies and “competitive sport” in 6 (19%) studies. Of the studies, 6 (19%) did not define the return-to-sport outcome explicitly. One study reported on rate of return to “cutting or pivoting” sports.⁵¹ One study defined “return to sport” as International Knee Documentation Committee (IKDC) level 5 activity.⁹⁰ Finally, 1 study categorized return to sports participation as “very strenuous, strenuous, or moderate.”⁷¹

Patient-Reported Outcome Measures

PROMs were reported in 63 (82%) articles; 14 total PROMs were used variably across studies (Fig 3A, B). The number of PROMs used per study ranged from 0 to 5. The Lysholm (64%),¹⁰⁸ IKDC¹⁰⁹ (56%) and Tegner Activity Scale¹¹⁰ (37%) scores were the most commonly reported. Ten studies reported all 3 PROMs, and 2 PROMs, which were specifically created and validated for use with pediatric patients, the Pedi-IKDC¹¹¹ and the Hospital for Special Surgery Pediatric Functional Activity Brief Scale (HSS Pedi-FABS),¹¹² were employed in 5 (6%) studies. Considering that the Pedi-IKDC was first published in May 2011, only 15% (6/34) of studies published in May 2013 or later used a pediatric-specific PROM. Focusing on this latter subgroup provides a 2-year lag time during which new studies were performed when a pediatric-designed and validated outcome measure was available in the literature. Four PROMs were originally developed and validated in adult patients and later validated in pediatric patients (Fig 3A). Fourteen studies (18%) reported 0 PROMs in their patient cohorts. Patients’ self-assessments were reported in a minority of studies, including rates of general satisfaction in 9% of studies, pain scores in 8% of studies and Likert-scale assessments of knee function in 5% of studies.

Fig 3.

(A) Patient-reported outcome measures (PROMs) were used inconsistently across studies. Pediatric-designed instruments (Pediatric International Knee Documentation Committee Subjective Knee Form [Pedi-IKDC] and Pediatric Functional Activity Brief Scale [Pedi-FABS]) were employed in a small minority of studies. (B) The number of PROMs used per study was variable. One asterisk (∗) indicates a PROM that was originally designed and validated in adults and later validated in a pediatric population; 2 asterisks (∗∗) indicate a PROM that was originally designed and validated in a pediatric population. The OAK scale has not been validated in English in an American population. Included PROMs: EQ-5D, EuroQol Group-5 Dimensions; KOS-ALDS, Knee Outcome Survey-Activities of Daily Living Scale; IKDC, International Knee Documentation Committee questionnaire; KOOS, Knee injury and Osteoarthritis Outcome Score; Marx, Hospital for Special Surgery Modified Marx Activity Rating Scale; Pedi-FABS, HSS Pediatric Functional Activity Brief Scale; Pedi-IKDC, Pediatric IKDC; SF, short form. OAK, Orthopädische Arbeitsgruppe Knie.

Discussion

The findings of the present study demonstrate that use of PROMs to evaluate pediatric ACL reconstruction is neither standardized nor typically validated for the patient population being studied. Patients’ subjective perceptions and attitudes toward their medical care have become increasingly relevant in outcomes assessment across health care. Patient-centered data for ACL reconstruction have shown greater correlation with overall patient satisfaction than physician-centered assessment or objective clinical data.¹¹³ In turn, use of patient-reported outcome measures has gained widespread popularity. However, 19% of the 77 studies reported no PROMs, which is clearly a weakness of the available literature in terms of allowing for an understanding of the true implications of surgical intervention for ACL rupture in children and adolescents. In assessing trends in the use of different metrics over time, we believe that these findings should encourage future studies to apply more consistently a standardized set of age-appropriate outcome measures and, ultimately, improve the study and treatment of ACL injuries in children and adolescents.

The lack of high-quality evidence in studies identified in the present systematic review is consistent with prior investigations into the literature concerning pediatric and adult ACL reconstruction.^13,114 The vast majority of studies were conducted retrospectively, which constrains the data available for analysis and precludes use of newer PROMs, which have greater applicability to pediatric populations. It is important to consider that many studies were conducted prior to the development of survey instruments validated in pediatric populations. Gebhard et al.⁴⁶ explicitly cited the lack of a pediatric-focused survey as a limitation to their work. The present analysis demonstrates that although many of the prior studies appropriately used the best methodological tools available at the time, investigators of future studies should be aware of newer means of improving the quality of data being gathered. A validated instrument must demonstrate test-retest reliability, construct validity and internal consistency when applied to a sample population representative of the group under study (i.e., children and adolescents).¹¹² Future work is needed to determine whether further delineation of age groups among patients younger than 18 years old is warranted. Moreover, clinical research focused on pediatric patients requires customization beyond PROMs. For instance, Gebhard et al.⁴⁶ illustrated how the KT-1000, the instrument used for testing ligamentous laxity in 48% of included studies, may not be appropriately sized for children and could, therefore, give erroneous results (Fig 4).

Fig 4.

Photograph of a standard KT-1000 arthrometer on a young athlete’s left leg. Even on the smallest setting, the device is unable to fit a pediatric patient properly, as illustrated by the joint line indicator (white label with black arrow meant to align with joint), which aligns above the superior pole of the patella.

This study identified 14 unique survey instruments (Fig 3A) employed to address the clinical efficacy of ACL reconstruction in children and adolescents. Such variability in methodology impairs efforts to aggregate data or draw more powerful conclusions. Standardized tools would also allow for the use of fewer overall survey instruments, which translates to less time spent by participants in completing questionnaires. In the present analysis, 35% of studies utilizing PROMs required participants to complete 3 or more separate survey instruments. Unfortunately, there is no universally accepted gold standard outcome measure or minimum number or PROMs that will provide a comprehensive assessment of patients’ function following pediatric ACL reconstruction. However, using fewer, more age-appropriate survey instruments reduces the risk of survey fatigue, decreases participant attrition and procures better data from respondents.115, 116, 117

Return to sports participation is often a highly prioritized goal for patients following ACL reconstruction. Despite 42% of studies’ quantifying the rate of return to sports, there was minimal description of how this outcome was determined. Nonetheless, several studies provided compelling ancillary data that may become useful if reported more commonly. Holwein et al.⁸⁸ reported the mean time that participants spent playing sports pre- and postoperatively. Calvo et al.⁷⁹ reported on athletes’ subjective knee instability upon return to sports. Dekker et al.⁹⁸ reported the number of sports played after surgery, as well as whether athletes returned to their primary sport. Consensus on defining return to preinjury sports participation, as well as identifying the most useful ancillary outcomes related to sports participation, will benefit future studies. To that end, the PLUTO study group¹¹⁹ is prospectively collecting outcome measures after pediatric and adolescent ACL reconstruction (Table 2).

Table 2.

Clinical Outcomes Used by the PLUTO Study Group

Clinical Outcome	Measurement Instrument
Knee function at 2 years post-ACL reconstruction	➢Pedi-IKDC
Health-related quality of life	➢Pediatric Quality of Life Inventory
Graft failure	➢Physical examination (Lachman test, pivot-shift test) ➢Magnetic resonance imaging ➢Reoperation
Activity level	➢HSS Pedi-FABS ➢Physical Activity Questionnaire
Growth disturbance	➢Physical examination ➢Hip-to-ankle radiograph until skeletal maturity
Subsequent meniscal or chondral injury	➢Physical examination ➢Magnetic resonance imaging ➢Reoperation

ACL, anterior cruciate ligament; HSS Pedi-FABS, Hospital for Special Surgery Pediatric Functional Activity Brief Scale; Pedi-IKDC, Pediatric International Knee Documentation Committee Subjective Knee Form; PLUTO, Pediatric ACL: Understanding Treatment Outcomes.

Limitations

This systematic review has several limitations that must be considered. The low overall rigor of constituent studies highlights a continued need for prospective studies that employ appropriate comparative cohort and/or control groups. Furthermore, studies were excluded if they contained adult patients. This was done in order to increase the overall rigor of this systematic review so it could focus exclusively on data reporting in studies of pediatric and adolescent patients, rather than analyze any study that contained patients younger than 19 years of age. Additionally, articles not available in English translation were excluded. Such a restriction in our literature review may have excluded additional PROMs and created a selection bias that underestimates the true variability of study design in the pediatric literature concerning ACLs. Nonetheless, 17 countries were represented in the final dataset, including many in which English is not the native language. Interestingly, 1 study also suggested that restricting systematic reviews to the English language does not introduce bias into systematic reviews.¹¹⁸ Finally, the current study aimed to provide a comprehensive account of all clinical outcomes pertaining to pediatric ACL reconstruction by not setting date limits for study inclusion. Advancements in surgical management, methodological design and data collection likely all contribute to differences among studies from 1988 to 2018. However, a recent study evaluating the pediatric ACL literature from 2010 to 2016 in a limited number of journals demonstrated significant heterogeneity in clinical outcomes reporting, consistent with the findings of the current review.¹³

Conclusions

There is significant variability across studies in the metrics used to assess clinical outcomes following ACL reconstruction in children and adolescents. Validated pediatric-specific instruments are used infrequently.

Appendix I. Systematic Review Comprehensive Search Terms

PUBMED

"adolescent"[MeSH] OR "adolescent"[TW] OR "adolescents"[TW] OR "youth"[TW] OR "child"[MeSH] OR "child"[TW] OR "children"[TW] OR "teenager"[TW] OR "teenagers"[TW]

OR "infant"[MeSH] OR "infant"[TW] OR "infants"[TW] OR "pediatrics"[MeSH] OR "pediatrics"[TW] OR "pediatric"[TW] OR "paediatric"[TW] OR "paediatrics"[TW] OR “juvenile”[TW] AND "Anterior Cruciate Ligament/surgery"[Mesh] OR "Anterior Cruciate Ligament Reconstruction"[Mesh] OR ("anterior"[Tw] AND "cruciate"[Tw] AND "ligament"[Tw]) OR "anterior cruciate ligament"[Tw] OR "acl"[Tw] AND "treatment outcome"[MeSH] OR ("treatment"[Tw] AND "outcome"[Tw]) OR "treatment outcome"[Tw] OR "outcome assessment (health care)"[MeSH] OR ("outcome"[Tw] AND "assessment"[Tw] AND "(health"[Tw] AND "care)"[Tw]) OR "outcome assessment (health care)"[Tw] OR ("outcome"[Tw] AND "assessment"[Tw]) OR "outcome assessment"[Tw] OR "clinical outcomes"[Tw]

OR "outcome instruments"[Tw] OR "functional outcomes"[Tw] OR "risk factors"[MeSH] OR risk factors[Tw] OR ("patient"[Tw] AND "reported"[Tw] AND "outcome"[Tw] AND "measures"[Tw])

OR "patient reported outcome measures"[Tw]

EMBASE

('adolescent'/exp) OR (adolescent:ti,ab,de,tn) OR (adolescents:ti,ab,de,tn) OR (youth:ti,ab,de,tn) OR ('child'/exp) OR (child:ti,ab,de,tn) OR (children:ti,ab,de,tn)

OR (teenager:ti,ab,de,tn) OR (teenagers:ti,ab,de,tn) OR ('infant'/exp)

OR (infant:ti,ab,de,tn) OR (infants:ti,ab,de,tn) OR ('pediatrics'/exp)

OR (pediatrics:ti,ab,de,tn) OR (pediatric:ti,ab,de,tn) OR (paediatric:ti,ab,de,tn)

OR (paediatrics:ti,ab,de,tn) OR (juvenile:ti,ab,de,tn) AND ('Anterior Cruciate Ligament Reconstruction'/exp) OR (anterior:ti,ab,de,tn AND cruciate:ti,ab,de,tn AND ligament:ti,ab,de,tn) OR ("anterior cruciate ligament":ti,ab,de,tn) OR (acl:ti,ab,de,tn) AND ('treatment outcome'/exp)

OR (treatment:ti,ab,de,tn AND outcome:ti,ab,de,tn) OR ("treatment outcome":ti,ab,de,tn)

OR ('outcome assessment'/exp) OR (outcome:ti,ab,de,tn AND assessment:ti,ab,de,tn AND (health:ti,ab,de,tn AND care):ti,ab,de,tn) OR ("outcome assessment (health care)":ti,ab,de,tn)

OR (outcome:ti,ab,de,tn AND assessment:ti,ab,de,tn) OR ("outcome assessment":ti,ab,de,tn)

OR ("clinical outcomes":ti,ab,de,tn) OR ("outcome instruments":ti,ab,de,tn) OR ("functional outcomes":ti,ab,de,tn) OR ('risk factor'/exp OR "risk factors":ti,ab,de,tn)

OR (patient:ti,ab,de,tn AND reported:ti,ab,de,tn AND outcome:ti,ab,de,tn AND measures:ti,ab,de,tn) OR ("patient reported outcome measures":ti,ab,de,tn)

COCHRANE

([mh adolescent]) OR (adolescent:ti,ab,kw) OR (adolescents:ti,ab,kw) OR (youth:ti,ab,kw)

OR ([mh child]) OR (child:ti,ab,kw) OR (children:ti,ab,kw) OR (teenager:ti,ab,kw) OR (teenagers:ti,ab,kw) OR ([mh infant]) OR (infant:ti,ab,kw) OR (infants:ti,ab,kw)

OR ([mh pediatrics]) OR (pediatrics:ti,ab,kw) OR (pediatric:ti,ab,kw) OR (paediatric:ti,ab,kw)

OR (paediatrics:ti,ab,kw) OR (juvenile:ti,ab,kw) And ([mh "Anterior Cruciate Ligament/surgery"])

OR ([mh "Anterior Cruciate Ligament Reconstruction"]) OR (anterior:ti,ab,kw AND cruciate:ti,ab,kw AND ligament:ti,ab,kw) OR ("anterior cruciate ligament":ti,ab,kw) OR (acl:ti,ab,kw) AND ([mh "treatment outcome"]) OR (treatment:ti,ab,kw AND outcome:ti,ab,kw)

OR ("treatment outcome":ti,ab,kw) OR ([mh "outcome assessment (health care)"])

OR ((outcome:ti,ab,kw) AND (assessment:ti,ab,kw) AND (health:ti,ab,kw) AND (care:ti,ab,kw))

OR ("outcome assessment (health care)":ti,ab,kw) OR (outcome:ti,ab,kw AND assessment:ti,ab,kw) OR ("outcome assessment":ti,ab,kw) OR ("clinical outcomes":ti,ab,kw)

OR ("outcome instruments":ti,ab,kw) OR ("functional outcomes":ti,ab,kw) OR ([mh "risk factors"]) OR ("risk factors":ti,ab,kw) OR ((patient:ti,ab,kw) AND (reported:ti,ab,kw) AND (outcome:ti,ab,kw) AND (measures:ti,ab,kw)) OR ("patient reported outcome measures":ti,ab,kw).