DEVELOPMENT OF A PHYSICAL PERFORMANCE ASSESSMENT CHECKLIST FOR ATHLETES WHO SUSTAINED A LOWER EXTREMITY INJURY IN PREPARATION FOR RETURN TO SPORT: A DELPHI STUDY

Abstract

Purpose/Background:

To develop a consensus on the critical constructs necessary to be included in a physical performance assessment checklist (PPAC) to assess an athlete's ability for return to sport following a lower extremity injury.

Methods:

The study used a 3‐round Delphi method to finalize the PPAI originally developed by a panel of experts. Fourteen Delphi representative sample participants were randomly derived from the authors of peer‐reviewed publications of lower extremity injuries. Nine participants completed all 3 rounds.

Results:

Throughout the 3 rounds, the 10 initial constructs were modified and revised to produce the finalized PPAC consisting of 12 constructs necessary to consider for an athlete's return to sport after a lower extremity injury.

Conclusions:

This instrument can be used as a checklist to advocate for prospective batteries of physical performance tests to incorporate the elements identified by this study.

Level of Evidence:

5

INTRODUCTION

Approximately 33% athletes that participate in sports will sustain an injury severe enough to seek treatment.¹ The most commonly injured areas are the ankle and the knee.^1,2 Moreover, previous injury^2,3 and premature return to sport have been shown to be strong risk factors for sustaining a subsequent lower extremity injury.² Although the rate of lower extremity re‐injury is variable among varied individual sports, some athletes' sport specific activities place them at a re‐injury risk rate between 35%‐80%.^4‐9

It is crucial to be able to determine when it is appropriate for an athlete to return to sport in order to prevent further injuries. Rehabilitation of the athlete is complex and requires the clinician to evaluate the specific demands of the individual sport, as well general rehabilitation constructs (pain, motion, strength) necessary to return to play.¹⁰ Currently, some common methods utilized to prepare athletes for return to sport include: pain subsidization, restoration of strength, restoration of joint range of motion,¹ and length of time post‐injury.^11‐13 Clinically the use of physical performance tests (PPTs) have been used to evaluate the ability of an athlete in preparation for return to sport. Combinations of PPTs may be used to evaluate the ability of an individual in a variety of sport‐specific skills, however; many PPTs are used in isolation. Some examples of PPT's cited in the literature include: single‐leg hop, single leg squat, active straight leg raise, in‐line lunge, and deep squat.¹⁴ Recently, Narducci et al¹⁵ examined the utility of PPTs in a systematic review, and found no studies that investigated the predictive validity of PPTs for return to sport for post‐anterior cruciate ligament (ACL) injured athletes. Furthermore, none of the included studies utilized PPTs as a measure to determine readiness to return to sport; they were used only as an outcome measure or a measure of function.¹⁵

Another study by van Grinsven et al¹⁶ suggests that criteria for return to sport after a lower extremity injury should include hop test and strength of the hamstrings and quadriceps at least of 85% compared to the contralateral side and when the patient tolerates sport‐specific activities. However, this may not be detailed enough for athletes attempting to return to competitive sports, based on how different individuals may define tolerance.¹⁰

There are different injuries that can be sustained in the lower extremity that prevent an athlete from returning to sporting activities. The injury to the ACL is predominant in the literature. There are several studies suggest specific return to sport recommendations following an ACL injury in specific sports including: soccer, basketball, alpine skiing, and American football.^10,17‐19 While these authors^10,17‐19 investigated rehabilitation options, and utilized PPTs as part of the rehabilitation for returning an athlete to a specific sport, many of these PPTs still lack predictive validity as to the determinate of success of the recovered athlete. Unfortunately, no general consensus exists on which PPTs and constructs should be utilized for evaluation if an athlete suffers a lower extremity injury and is seeking to return to sport.

There is current lack literature identifying the predictive validity for the PPTs for the return to sport. Specifically this variation in general recommendations amongst professionals rehabilitating these athletes for return to sport prompted the idea of the current study. The identification of the critical elements within the return to sport construct may allow lower extremity performance tests to be developed, tested in isolation, or in test batteries in order to incorporate elements that are deemed critical for the ability to return to sport. The purpose of this study was to develop and enhance a useful assessment or checklist type tool following, after completing the first 3 steps in a Delphi study outlined by Streiner and Norman.²⁰ The complete process consists of: (1) preliminary conceptual decisions; (2) item generation; (3) assessment of face validity; (4) field trials; and 5) generation of a refined instrument.²⁰ The expert panel served as a pilot to run the first three steps before sampling to a larger sample group. The expert panel included experts in sports and/or orthopedics were to develop critical constructs deemed necessary when assessing an athlete's ability for return to sport following a lower extremity injury.

METHODS

The Delphi Technique

The Delphi technique is a series of survey or questions, interspersed by controlled feedback, that seek to gain the most reliable consensus of opinion from a group of experts in a timely and orderly fashion.²¹ It is a technique that is useful for situations where individual judgments must be tapped and combined in order to address a lack of agreement or incomplete state of knowledge. Normally, three rounds of survey/questionnaires are sent to a pre‐selected panel, often is the expert group and then a sample representative group, although the decision over the number of rounds is largely pragmatic based on agreement of constructs and ideas. The role of the first round is to identify issues that will be addressed in later rounds. Open‐ended questions are utilized in the first round because they increase the richness of the data collected.²¹ This technique was chosen because there is a lack of literature supporting decisions regarding rehabilitation testing and retuning an athlete to sport. Contacting clinical experts is a reliable way to collect initial information.

Study Design

This study was a Delphi study design that consisted of three rounds of questionnaires. (Rounds I‐III) By the end of the Round III, a consensus decision was made by the panel of experts in sports and orthopaedic physical therapy,²² a process explained in Figure 1.

Figure 1.

Flowchart of Delphi Process.

Study Participants

Work Groups

There were three separate groups who took part in this study. The first group was the expert panel that initially gathered to develop the preliminary conceptual decisions for a checklist for an athlete who suffered a lower extremity injury. Nine clinical experts were contacted based on publication record in the area of lower extremity rehabilitation of the athlete, years of experience, and professional contribution to the area of sports/orthopaedics physical therapy. Two panelists did not reply to the initial invitation. Therefore, 7 expert clinicians made up the expert panel and served as a pilot group for this checklist development. All seven of these participants qualified as content experts, possessing a minimum of a board certification of Orthopaedics Certified Specialist (OCS) and/or Sports Certified Specialist (SCS) from the American Physical Therapy Association (APTA). The discussion within this panel was mediated by the primary investigator, who assembled the work team for the subsequent Delphi study.

The second work group (the work team) for the Delphi process consisted of 4 investigators, who were responsible for interacting with the expert panel and distributing the questionnaires to the representative sample. The primary investigator (MD), an orthopaedic manual physical therapist with 11 years of clinical experience, is experienced in qualitative research, and has contributed to the literature including a recently published systematic review¹⁵ on this topic. The second investigator (CC) is also an expert in orthopedics, manual therapy, and research, including being an author 5 Delphi studies.^22‐26 The other work team members were student researchers who worked with the primary investigator during their final year of entry‐level doctor of physical therapy education.

Delphi Representative Sample Group

The third group, Delphi representative sample group (the Delphi sample group) was systematically selected by use of PubMed, Cinahl, SPORTDiscus, and Health Source to assess their publications. The initial search strategy was limited to humans and the English language and included: “Leg Injuries”[Mesh] OR “Knee Injuries”[Mesh] OR “Hip Injuries”[Mesh] OR “Ankle Injuries”[Mesh] OR “Lower Extremity Injuries” OR “Lower Extremity Injury”) AND (“return to sport” OR “Athletic Performance”[Mesh] OR “physical performance” OR “functional performance,”) and other combinations or variations of the above. This search yielded a total of 339 results. The work group select every third corresponding author was taken to ensure a systematic selection based on suggestions of previous Delphi studies. After duplicates were eliminated, authors (n=101) were contacted with an email to test accuracy of the contact information.

The number of viable emails (n=72) resulted in the conduction of another search with different associated key search terms related to this study: “lower extremity injuries” OR “lower extremity injury” AND “return to sport” AND “rehabilitation” OR “physical therapy.” This yielded another 334 results and every third corresponding author was recorded. After duplicates were removed, emails (n = 133) were again tested for accuracy. The viable emails from this search were combined with the initial search for a total of 175 email invitees.

The Delphi sample group was asked to participate in all 3 rounds of the Delphi process when contacted. After the initial invitation, 2 did not wish to be included, 14 (8.0%) agreed to participate in the study, whereas the rest of the authors did not respond to the invitation email. Although the response rate was low, traditionally Delphi studies do not identify a required a set number of participants in their representative sample. All 14 of these experts were invited to participate in each of the remaining 2 rounds. Nine experts completed the entire Delphi process and made up the Delphi sample group who worked with the checklist that the expert panel created.

Of the 14 selected Delphi sample group participants that responded during the initial round of this study all possessed a post‐baccalaureate degree; 93% had clinical or academic doctorate degrees, and 50% reported specialty certification in sports and/or orthopaedics. The participants ranged from new ‐clinicians (less than 5 years) to experienced clinicians (greater than 25 years), with the majority (63%) having over 15 years of clinical experience. All participants have peer reviewed publications relating to lower extremity injuries and/or rehabilitation of sports injuries, 36% reported greater than 25 publications in this area. The Delphi sample group of 9 participants who completed the entire Delphi process represented a total of 6 countries, including: USA, England, Norway, Netherlands, Israel, and Ireland. None of the Delphi sample group participants knew of one another's participation during the process.

Initial Constructs

The initial constructs for this topic, comprised of critical constructs in the assessment for return to sport for athletes who suffered a lower extremity injury were generated through a discussion between the primary investigator and the expert panel during the pilot phase of the Delphi. Through open discussion over a 2‐month period, these clinical experts revised, eliminated, and added constructs that would outline the critical elements used in the next phase of the Delphi process. After this two‐month period, the expert panel generated, and through consensus agreed upon, ten critical constructs of an athlete's physical performance that they felt should be assessed prior to return to sport after sustaining a lower extremity injury (Table 1). Following the development and agreement of the initial physical performance assessment checklist (PPAC) from the expert group (pilot), the next phase required the participants systematically selected for the Delphi sample group to be responsible for adaptation and final approval of the constructs included in the assessment checklist.²²

TABLE 1.

Original constructs of the Physical Performance Assessment Checklist (PPAC).

1.)	The test or battery (of tests) evaluates multiple planes of lower extremity movement.
2.)	The test or battery (of tests) is specific to the movement demands of the targeted sporting event.
3.)	The test or battery (of tests) evaluates the athlete landing after explosive movements.
4.)	The test or battery (of tests) evaluates appropriate neuro‐muscular control during deceleration of movement.
5.)	The test or battery (of tests) identifies compensatory and/or dysfunctional movement patterns.
6.)	The test or battery (of tests) identifies if pain modifies the movement during assessment.
v.)	The test or battery (of tests) has a validated and reliable threshold which determines return to sport.
8.)	The test or battery (of tests) evaluates single leg stability.
9.)	The test or battery (of tests) evaluates change of direction.
10.)	The test or battery (of tests) should evaluate for symmetrical motor control (stability) of active lower extremity movements bilaterally.

Delphi Procedure

The three‐round Delphi process was managed by the primary investigator through the online survey website, Zoomerang®. This website allowed the Delphi sample group participants to have access to the appropriate survey questions while blinding investigators to the identities of the individual experts. After the initial contact information was obtained by 2 of the investigators the primary investigator managed any further confidential information during each round of the study. Follow‐up reminders were sent at 10 and 20 days after the initial email, utilizing the survey mailing/emailing guidelines set by Dillman²⁷ and Lopopolo.²⁸

The Delphi sample group was made aware that the rationale for developing the PPAC was to provide an adequate way to identify when an athlete is ready to return to sport. Round I allowed experts to respond to the initial constructs in an open‐ended fashion. This gave the participants the opportunity to modify, add, or suggest to delete any sections of the PPAC accordingly, and to suggest new items to be evaluated throughout the process. The data were obtained and modified by the research work team to fit the preferences of the Delphi sample group.

Round II provided the Delphi sample group participants with the newly modified constructs, along with a Likert scale to offer input on the importance of each construct for determining an athlete's readiness for return to sport. The scoring choices offered were: strongly agree, agree, disagree, and strongly disagree. All participants had the ability to choose a score as well as provide comments on the constructs. Upon completion of round II, results were converted into bar graph format and sent out as round III (Figure 2).

Figure 2.

Sample Question Round III Delphi Study.

Continuing the identified Delphi procedure, the final round allowed the Delphi sample group participants to view how other experts rated the constructs of the PPAC. They were then asked to re‐score on the importance of each construct using the same Likert scale provided in round II. Participants were also given the opportunity to provide a general comment upon completion of the round of the Delphi process.

Data Analysis

For each construct, scores from round III were divided into positive and negative responses, with strongly agree and agree representing a positive response, and disagree and strongly disagree representing a negative response. Based on apriori consensus of the research work team, a construct was designated as part of the final PPAC if there was a positive response of 75% or greater from the experts.²² Upon completion of round III, a finalized list of constructs for the PPAC was confirmed.

RESULTS

Round I

As stated, the first round of the Delphi study allowed the Delphi sample group participants to give additional item suggestions, remove items, or modify the 10 initial constructs initially designed by the expert panel during the pilot. After compiling data from the 14 participants, the initial 10‐item assessment instrument was revised and expanded with a total of 6 new concepts added. These new concepts included evaluating: active range of motion (AROM) and passive range of motion (PROM) of the involved joint; trunk control/strength; specific movement demands of the involved joint; full range of motion (ROM) against 50% of 1 repetition max; the long latency reflex, silent period and H‐reflex from lower extremity by electrophysiological study; and fear‐avoidance behavior.

Rounds II and III

Round II provided all 14 Delphi sample group participants with the revised list of 16 constructs from Round I. Round II in the Delphi methodology allowed the participants the opportunity to score the constructs on a 4‐point Likert scale. Eleven of the round I participants viewed the survey, and 9 responses were received. The results were then compiled into a bar graph format and sent out as round III.

During round III, all 9 Delphi sample group participants were provided with the list of the 16 revised constructs, accompanied by a bar graph depicting the responses from round II (Figure 2). The participants were asked to re‐score the constructs on the same Likert scale utilizing the visual representation of other experts' ratings. There was 100% return rate from round II, with the same 9 participants submitting responses.

Based on the apriori consensus requirement of a 75% positive response rate, a total of 4 constructs were excluded. These eliminated constructs included evaluation of: specific deficits while performing single leg stability during static and dynamic perturbations on multiple surfaces; AROM and PROM of the involved joint; full ROM against 50% of 1 repetition max; and the long latency reflex, silent period and H‐reflex from lower extremity by electrophysiological study.

The final PPAC consists of 12 critical constructs that were deemed necessary to determine an athlete's readiness for return to sport following a lower extremity injury (Table 2). Nine of the initial 10 constructs from the pilot study were accepted as well as 3 newly developed constructs from the Delphi sample group. The general concepts in the final PPAC include: stability, power, neuromuscular control, coordination, pain, movement, balance and psychological assessment.

TABLE 2.

Revised constructs of the Physical Performance Assessment Checklist (PPAC).

	Construct	Construct present? Yes or no
1.)	Does the test or battery (of tests) evaluate multiple planes of lower extremity movement in isolation and/or in combination?
2.)	Is the test or battery (of tests) specific to the movement demands of the targeted sporting event?
3.)	Does the test or battery (of tests) evaluate the athlete landing after explosive movements, unilaterally and bilaterally?
4.)	Does the test or battery (of tests) evaluate appropriate neuromuscular control of bilateral lower extremities during deceleration (negative acceleration) of sports specific movement?
5.)	Does the test or battery (of tests) identify compensatory and/or acquired dysfunctional movement patterns?
6.)	Does the test or battery (of tests) identify if pain during or after the movement modifies the movement during assessment?
7.)	Does the test or battery (of tests) have a validated and reliable threshold which determines a sport specific return to sport?
8.)	Does the test or battery (of tests) evaluate efficiency in change in direction during sport specific movements?
9.)	Does the test or battery (of tests) evaluate for symmetrical motor control (stability) of active lower extremity movement bilaterally and asymmetrical motor control (stability) of active one leg movement during sport specific movements?
10.)	Does the test or battery (of tests) evaluate trunk control/strength required for sport specific activity?
11.)	Is the test or battery (of tests) specific to the movement demands of the involved joint for a particular sport?
12.)	Does the test or battery (of tests) include the fear‐avoidance questionnaire?

DISCUSSION

The purpose of this study was to utilize a 3‐round Delphi process in order to develop an assessment checklist for use when designing a battery of tests to evaluate an athlete's ability to return to sport following a lower extremity injury. This method has been used in several studies to validate the use of assessment tools.^29‐33 To the authors knowledge, this is the first study to develop a generalized consensus on return to sport after a lower extremity injury. Both the expert panel and the Delphi sample group demonstrated expertise in this topic area both by clinical experience and history of publications. Additionally, the Delphi sample group was more diverse than that of the initial expert panel, providing an international viewpoint on this topic. Throughout the three rounds, there were numerous modifications made to the PPAC, including the addition of three constructs and removal of one of the original constructs. The final PPAC consists of 12 critical constructs, determined by the initial pilot and international Delphi representative samples, which should be considered when assessing the patient's ability to return to sport. It may be used as a checklist for clinicians when determining the readiness of an athlete for return to sport‐based activities, including choosing and utilizing assessments of: stability, power, neuromuscular control, coordination, pain, movement, balance, and psychological factors.

The results of this study are similar to previous research and the suggestions to assess power,^10,18,19,30 neuromuscular control,^10,17‐19 coordination,^{10,17‐19,30} pain,^17,19 movement,^{10,17‐19,30} and balance^10,17,18,30 prior to determining the readiness of an athlete for return to sport‐based activities are consistent between this study and previous studies. Additionally, the concept of focusing rehabilitation and functional testing towards sports specific movements was evident throughout the literature.^{10,17‐19,30,35} Each of these elements were viewed as important by the Delphi representative sample during the Delphi process and therefore added to the PPAC. Finally, previous research supports the use of a battery of tests, rather than a single PPT, in order to address all of the critical concepts necessary to identify when an athlete can return to sport.^17,18,30

Other performance measures or concepts, not addressed in the PPAC, but outlined in current literature included: cardiovascular conditioning,^10,18,19,30 muscle flexibility,^10,18,19,30 and muscular strength/endurance.^{10,17‐19,30} The concept of ROM/flexibility failed to meet consensus of the Delphi group. Additionally, it could be argued that muscle flexibility and strength/endurance could be addressed by some of the other concepts identified in the PPAC, such as power, movement, and control. The concept of cardiovascular conditioning may be a valid consideration for further evaluation. Because this study only conducted steps one through three of the Delphi process, subsequent researchers need to examine the currently identified concepts as they relate to the final 2 steps of the Delphi process (field trials and generation of a refined instrument) as outlined by Streiner and Norman.²⁰ A benefit of the PPAC produced in the current study is that it may serve as a general basis for the development of a field tests or a test battery.

A novel finding in the current study was the recommendation by the Delphi sample group that a fear avoidance behavior questionnaire be included in the PPAC. This was not an initial concept identified by the expert panel; however there is prior research that supports the use of such a questionnaire in athletes with regard to their readiness for return to sport. Several authors have reported that fear avoidance behaviors can limit an athlete's readiness to return to sport.^17,36,37 Kvist et al³⁸ listed psychological factors as one of the seven main constructs that influence an athlete's ability to safely return to sport. Additionally, Ardern and colleagues³⁷ found that fear of re‐injury was inversely related with return to sport at pre‐injury level. Interestingly, specific core or spinal assessment and/or function was not suggested to be a critical assessment by either the expert panel or the representative sample during the Delphi assessment. These constructs were only addressed relating to the constructs of power, control, endurance, and strength.

Limitations

The primary limitation to this study was the overall low response rate (8%), and the poor retention rate from round I to round II (64%) despite multiple reminders having been provided to promote increased or continued participation. A larger sample size of experts may have generated different results for the PPAC due to a broader perspective; however, an optimal response rate for a Delphi study has not been previously established.³⁹ Additionally, as this was an online survey, e‐mail was used as the only form of communication limiting the random selection of participants to those with e‐mail access at the time of solicitation. Furthermore, not all e‐mail addresses provided a viable source of communication.

CONCLUSION

The PPAC was created using Delphi methodology in order to describe the critical constructs necessary to assess when testing an athlete and determining his/her readiness for return to sport after sustaining a lower extremity injury. This instrument can be used as a checklist to advocate for or assemble batteries of PPTs that incorporate the elements identified by this study. Future studies should address the final 2 steps of the Delphi methodology outlined by Streiner and Norman²⁰ including field trials and refinement for validation of this checklist in clinical use.