Abstract
Introduction
Upper limb research is currently lacking detailed clinical guidance on the provision of unilateral transradial prostheses. Clinical practice guidelines are meant to serve as assistance for the decision-making process, and Delphi surveys have been used with increasing frequency within orthotics and prosthetics to create these guidelines for clinical practice.
Methods
A three round Delphi survey was used to gain consensus on clinical statements regarding unilateral transradial prostheses.
Results
We achieved consensus (> 80% agreement) on a total of 40 statements by surveying 22 experts on upper limb prosthetics over three rounds of surveys. Response rate ranged from 81.8–86.4% with a total of 55 total statements under consideration throughout the duration of the survey. The 40 passing statements were arranged into nine guidelines for provision of prosthetic care in this population.
Conclusions
The Delphi technique allowed for the creation of a set of clinical practice guidelines for the unilateral transradial patient in the absence of conclusive empirical evidence.
Keywords: upper limb prosthetics, prosthetic control, amputation, amputees, tasks
Introduction
Of the estimated 2.2 million Americans projected to be living with amputation in 2020, approximately 16% have had an amputation of the upper extremity. 1 For patients with a major amputation (at the wrist or above), this amounted to a projected 41,000 people in the United States, as of 2005. 1 Though there are limited studies on the effectiveness of upper limb prostheses, the rate of published research in this area lags markedly behind that of lower limb prostheses. 2 To the extent that such efforts have been published, the existing body of research does not yet present strong, detailed clinical guidance for the practicing clinician.2,3 For example, while a recent literature review by Carey et al. suggested eleven empirical evidence statements supported by 31 reviewed papers, these were generally supported by low levels of evidence and failed to define many clear indications, contraindications, and clinical considerations associated with broad classes of prosthesis (e.g., external vs body power, hand vs hook), concluding simply that “prosthetic selection should be based on a patient’s individual needs and include personal preferences, prosthetic experience, and functional needs.” 2 A second example is seen in the 27 recommendations that comprise the VA/DoD Clinical Practice Guideline for the Management of Upper Extremity Amputation Rehabilitation, of which only one is empirically based, with the remaining 26 statements supported by expert opinion. 4
Because of the lack of published evidence on the effectiveness of one type of upper limb prosthesis over another and the relative absence of clear indications for use of certain upper limb prosthetic components, clinical decision-making has been largely left up to the discretion of the treating prosthetist. However, the number of prosthetists who have a high level of experience treating the upper limb amputee is small relative to those treating lower limb patients. 5 Notably, a recent State of the Science Conference on design options for upper limb prostheses concluded that those rehabilitation professionals, including physicians, therapists, and prosthetists, who have amassed considerable experience in working with this patient population should be recognized as the most informed source of currently available evidence.3,6
In recent years, the field has begun to develop and publish clinical practice guidelines (CPGs) across a range of care episodes including evidence-based guidelines of amputation and prosthetics of the lower extremity,7,8 postoperative care following transtibial amputation, 9 prosthetic foot selection for individuals with lower limb amputation, 10 transtibial socket design, 11 and prosthetic knee selection for individuals with transfemoral amputation.12, 13 The scope and depth of these CPGs has been variable, with direct implications on their resultant clinical relevance and ultimate incorporation into practice.
When objective data from clinical trials for a given episode of care or intervention is limited, the highest available level of evidence for the creation of CPGs is through a collaborative consensus process conducted among subject matter experts.14, 15 Clinical practice guidelines created through consensus process are likely to become more and more prevalent within orthotics and prosthetics as additional recommendations are sought out by the field.
One of the most common ways to reach consensus within allied health is through the Delphi process.16, 17 The Delphi consensus model utilizes a group of experts in a specific field anonymously completing multiple rounds of surveys to evaluate the accuracy of clinical postulates. The aim of this process is to establish consensus among the expert participants by eventually coming to agreement on the statements that are most clinically appropriate.14, 16, 17 After each round, the surveys are reviewed for comments and feedback on the individual statements by a group of moderators.14, 16, 17 Each subsequent round of surveys includes new or re-stated postulates which move the process forward toward consensus.
Delphi consensus techniques have already been used to generate many prosthetic and orthotic clinical practice guidelines on a variety of topics including prescription of lower limb prostheses,18, 19 prescription of microprocessor-controlled prosthetic knees, 12 orthotic management of children with spastic diplegic cerebral palsy, 20 orthotic management of plagiocephaly, 21 and factors influencing transtibial prosthesis fit. 22 The use of Delphi consensus techniques in prosthetic and orthotic rehabilitation was recently summarized in a systematic literature review culminating in a number of recommendations of best practices within this development process. 14
The purpose of this publication is to report upon the use of a Delphi consensus effort to establish CPGs for prosthetic design and terminal device options for individuals with unilateral transradial amputation or limb deficiency. This patient population was chosen due to the relative volume of patients with this level of amputation and the ability to develop straightforward guidelines that would be immediately applicable to patient care. The target audience for this guideline includes prosthetists, surgeons, physicians, therapists, case managers, and policy makers. The target patient population comprises individuals with unilateral transradial amputation or limb deficiency.
Methods
All activities in this study were performed by project directors from a national provider of upper limb prostheses in close coordination with a group of subject matter experts (SME) in the field of upper limb prosthetics. Initial item generation was completed in a face-to-face focus group of experienced upper limb clinicians following a review of available evidence facilitated by two recent systematic reviews in this area.2,23 The focus of these postulates was on the indications, contraindications, and considerations associated with prosthesis type (e.g., body powered vs externally powered) and terminal device type (e.g., hand vs hook vs activity specific device) with regard to individuals with unilateral transradial amputations.
Once established, these postulates were entered into a secured, web-based survey platform and sent to a larger review panel of 20 certified prosthetists (CPs) and two occupational therapists (OTs.). Inclusion within this panel required that the clinical prosthetists provided an average of greater than 50 upper limb prostheses per year for the last 3 years, and the participating occupational therapists treated at least 50 upper limb patients per year. The clinicians were asked to consider the 40 postulates and rate each statement on a five-point Likert scale (strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree.) In addition, for each statement, there was an opportunity for the respondent to provide comments. This was specifically requested if they marked “disagree” or “strongly disagree” to any postulate to explain their position. Consistent with Delphi processes, the surveys were completed anonymously using an online survey platform.
Results from first round of Delphi surveys were collected and reviewed. The consensus threshold of 80% agreement was established for inclusion within the practice guidelines. Those postulates that did not score 80% or greater were reviewed for responder comments and revised in an effort to gain consensus during a second stage of Delphi surveys. In some instances, this required separating a single postulate into two or more distinct postulates to better isolate areas of consensus and areas of concern. This smaller subset of postulates was sent to the same panel of CPs and OTs and responses were collected and analyzed. Those postulates from the second round of surveys that exceeded the 80% consensus threshold were added to the guideline. Those that failed to reach this standard were reviewed and revised for a final round of Delphi surveys. A third round of the surveys was sent out to finish the process of gaining consensus on the postulates with those statements exceeding 80% consensus added to the guidelines, and those falling short of that standard being discarded from further consideration.
Results
The initial SME panel was comprised of eight prosthetists with expertise in upper limb prosthetics. The larger group of 20 CPs and two OTs represented a diverse geographic area and had an average of 21 years of experience in the treatment of patients with upper limb amputation or limb deficiency. Each prosthetist surveyed was responsible for the care of at least 85 new upper limb cases per year. The occupational therapists surveyed treat at least 75 patients annually and educate hundreds of occupational therapists through direct mentorships and courses throughout the year.
The SME panel originally generated 40 postulates for use during round one of the Delphi survey. These were grouped into larger categories describing general considerations of prosthesis provision (n = 4), body powered device considerations, (n = 11) externally powered device considerations, (n = 11) oppositional device considerations, (n = 5) and other considerations for device type and design (n = 9.)
Following round 1, 31 of the 40 postulates reached 80% consensus (77.5%) with a response rate of 19/22 respondents (86.4%) The statements that reached consensus averaged 95.8% agreement. The postulates that failed had an average score of 72.5%. Of the nine statements that did not reach consensus, two were removed completely from further consideration based on comments from the experts. The remaining seven items were edited for clarity. Three postulates that reached consensus in round one were revised to reflect a small wording change requested by several of the expert panelists. An additional four postulates were added by the survey moderators after considering the comments collected during stage 1, for a total of 14.
Round two of the survey was completed by 18 out of the 22 specialists surveyed (81.8%) and 11/14 surpassed the 80% consensus benchmark (78.6%.) The statements that reached consensus scored an average agreeance of 94.4%. The statements that did not reach consensus scored an average agreeance of 75.9%. The moderators reviewed the postulates that did not reach consensus along with the aggregated feedback from the survey participants, removed two of them, and revised the remaining postulate.
The third, and final, Delphi survey had just one statement. Nineteen out of 22 specialists responded (86.4%) and the single statement passed consensus with a score of 100%. (Figure 1.)
Figure 1.
Results of the Delphi process
After all three rounds were completed, the statements were grouped according to subject to allow a clinical practice guideline to be written based on the consensus of the group. The 40 individual postulates were combined into nine guidelines that cover a variety of considerations for the unilateral transradial patient. (Appendix A)
Discussion
The aim of this effort was to establish clinical practice guidelines for the development of prosthetic treatment plans for individuals with unilateral transradial amputation or limb deficiency and thereby advance both clinical practice and the body of scientific research in this area. By reviewing the available literature, developing clinical postulates within a small focus group of SMEs and establishing consensus among a broader cohort of expert respondents, we utilized a Delphi process to develop a set of statements that were clinically pertinent in today’s prosthetic practice. We grouped this set of statements into nine clinical practice guidelines for the unilateral transradial patient that cover: when a unilateral transradial prosthesis is appropriate, clinical indications for a body powered transradial prosthesis, clinical indications for an externally powered prosthesis, clinical indications for an oppositional prosthesis, clinical indications for an activity specific prosthesis, selection of terminal device type, determination of voluntary opening vs. voluntary closing control strategies, heavy duty prosthesis use, and indications for use of multiple prostheses by the same patient. The entirety of the CPG is available in Appendix A.
Although a degree of subjectivity is innate to Delphi consensus efforts, our standards were consistent with those used in prior Delphi consensus efforts within the field. 14 Modified Delphi processes have suggested that literature reviews be incorporated into the initial development of clinical postulates. Our item generation began with an open discussion of the systematic review of Carey et al., published in 2015 and updated in 2017.2, 23 Recognizing the scarcity of actionable recommendations from this review, our panel set out to define more clinically relevant considerations related to the determination of device type and control.
The selection of the expert panel has been described as the most important step in the entire process because it directly relates to the quality of the results generated. 24 Expert qualification criteria are unique to every Delphi study. The American Board for Certification recently reported that the average certified prosthetist spent 10% of their clinical time in the performance of upper limb prosthetic rehabilitation. 25 By contrast, the clinicians that made up our expert panel have largely confined their clinical practice to upper limb prosthetic rehabilitation as evidenced by their reported annual caseloads.
The sample size of the expert panel is another area of variability within Delphi efforts. Although a panel that is too small may fail to represent the entire target population, larger samples may lead to lower response rates and may have little impact on the ultimate reliability and validity of the end results.14, 26 Sample sizes of approximately 20 individuals have been suggested as reliable 27 and samples of 10–30 individuals have been predominant in Delphi efforts performed within the field. 14 Our panel of 22 initial participants was reflective of these observations. Falbo et al. recommended a response rate in excess of 70% to ensure broad consensus throughout the Delphi process. 14 Our response rates exceeded this standard through all three rounds of survey administration.
An additional consideration in recruiting expert survey respondents is their relative heterogeneity. Although arguments for heterogeneous panels have been made, Baker et al. suggested that the conclusions of such diverse panels may be confined to more trivial, less relevant points as these are the only areas when consensus can be reached. 27 In contrast, more homogenous panels allow researchers to use specific criteria for participation and the identification of true experts in the targeted decisions. 27 In consideration of these principles, we elected to use a fairly homogenous panel of high-volume prosthetists and occupational therapists focusing on upper limb prosthetic rehabilitation. Precedent for such homogeneity in orthotic and prosthetic Delphi efforts is well established. 14
Consistent with the broad trends observed by Falbo et al., we encouraged comments from survey participants, especially dissenting comments, to allow us to clarify and refine postulates to achieve consensus acceptance. 14 There is no consistent clear standard for consensus within the Delphi approach. 14 The most commonly employed method in orthotic and prosthetic applications has been to simply define a percentage of agreement, with Delphi surveys endemic to the field ranging from 67% to 80% consensus. 14 To ensure the relative reliability of the accepted statements, we utilized a very high consensus standard of 80%.
Falbo et al. suggest that two to three rounds of Delphi surveys may be ideal and reported three rounds to be the most commonly utilized. 14 Though, not determined a priori, our process also required three rounds to adequately establish areas of broad consensus.
Following rounds one and two of the surveys, the moderators reviewed the statements that did not reach consensus and attempted to revise or re-state them in the following round to gain approval from the experts. However, four total statements were removed during the process without being sent on to the next round. The first two statements that were removed following round 1 both had to do with funding for a prosthesis. Although funding must be considered for each patient, the expert panel was adamant that funding limitations should not be viewed as equivalent to clinically based considerations and indicators. The two postulates that were removed after round two were statements that had already been revised after round one and were still not supported by the expert panel. The moderators did not feel that rewriting these concepts again for a third round would improve the agreement on these topics to be able to reach consensus.
There are a number of limitations associated with this effort. Although some have advocated for the participation of end-users in the development of CPGs, it is unclear whether their input adds validity to the study.27, 28 In this effort, our concern was that the biases a limited number of end-users may associate with their personal prosthetic design and components, and their likely lack of awareness and experience with the range of prosthetic alternatives, might serve to undermine the level of detail sought out in these guidelines. Notably, the majority of Delphi efforts in the field have also excluded end-users. 14 However, in deference to the critical importance of patient-focused care, we recognize thoughtful end-user perspectives available in recent publication, 29 and observe that a number of our final postulates stressed the importance of patient education and engagement throughout the development of a treatment plan.
Each statement generated and reaching consensus through the Delphi process provides opportunity for further clinical research. The field of upper limb prosthetics has been studied with increasing frequency recently, but there remain many areas that require further examination to allow a conclusive body of evidence for clinical practice. Using similar consensus techniques, it is probable that clinical practice guidelines will be developed for other upper limb levels that may have even less empirical evidence and smaller population numbers, such as patients with bilateral upper limb loss or patients with more proximal amputation levels.
Conclusion
The Delphi process allowed for clinical practice guidelines to be generated for the unilateral transradial amputee in the absence of strong evidence from existing clinical research and literature reviews. By surveying a small group of experts, the process moved quickly and efficiently from item generation through three rounds of surveys to establish a strong set of guidelines for the practicing prosthetist. Many prosthetists do not have enough expertise in the area of upper limb prosthetics to allow a high level of confidence in treating this patient population independently. However, with the addition of clinical practice guidelines presented here, the decision-making process can be streamlined, while still allowing for individual judgment of the clinician and for the opinion of the patient in determining their care.
Acknowledgments
The authors would like to acknowledge the following colleagues who were a vital part of the process for this study: Jack Uellendahl, Robert Picken, Bill Limehouse, AJ Westford, Matthew Mikosz, Mike Duston, Bambi Lombardi, Bryan Lott, Beth Button, Patrick McGahey, Fred Schaumburg, Jim Caywood, Kyle Sherk, Ryan Sheridan, Brian Monroe, Branden Petersen, Tony Berland, Chris Lake, Ryan Blanck, and Ken Hung.
Appendix A.
1) For patients with a unilateral transradial limb deficiency or amputation, a prosthesis should be considered when:
a. The person is unable to accomplish self-care activities or ADLs independently, or
b. The person has functional, vocational, or avocational needs that cannot be met without a prosthesis, or
c. The person’s psychosocial acceptance of their amputation/limb deficiency would be improved by the use of a prosthesis, or
d. The person is at risk of overuse syndrome on the sound side.
2) For patients with a unilateral transradial prosthesis, a body powered prosthesis should be considered when:
a. The restriction, associated pressures, and donning and doffing requirements of a control harness are fully understood and will be tolerated by the patient, and
b. The patient possesses adequate soft tissue coverage and integrity, or can be managed with appropriate interface materials or socket design, to allow cyclical loading of the limb within the prosthesis caused by cable activation of the terminal device, and
c. The patient possesses adequate strength and range of motion to generate the necessary cable force and excursion to actuate their terminal device, and
d. The patient fully accepts and understands that activities requiring dynamic prehension will be predominantly performed with a hook, rather than a hand, and
e. The patient possesses adequate soft tissue coverage and integrity over those body segments underlying the necessary harness.
3) For patients with a unilateral transradial prosthesis, an externally powered prosthesis should be considered when:
a. The patient possesses adequate control input to control external power (through EMG, FSR, electronic switch, or linear transducer), and
b. The noise, weight, and charging requirements associated with an externally powered device are fully understood and accepted by the patient, and at least one of the following:
c. The patient lacks the strength or range of motion required to generate the necessary cable force or excursion for a body powered prosthesis, or
d. The patient’s functional work envelope cannot be confined primarily to the area immediately in front of them, or
e. The patient does not possess adequate soft tissue coverage and integrity, or cannot be managed with appropriate interface materials or socket design, to allow cyclical loading of the limb within the prosthesis caused by cable activation of a body powered terminal device during active use, or
f. The need for sustained high grip strength through movement is anticipated, or
g. There is a compromise to gross body movements of the shoulders or back and/or an existing neurological compromise to the sound side upper limb (such as pain, numbness, or tingling), or
h. The patient has been previously fit with either an oppositional or body powered prosthesis and could not integrate it fully into their desired ADLs or vocational responsibilities, either because of mechanical constraints or psychosocial rejection.
4) For patients with a unilateral transradial prosthesis, an oppositional silicone restoration prosthesis should be considered when:
a. The user’s primary priority for their prosthesis is an aesthetic restoration of their forearm and hand, and
b. The absence of active prehension is fully understood and accepted by the patient, and
c. The cosmetic limitations of an oppositional prosthesis are fully appreciated by the patient.
5) Terminal Device Selection:
- a. Hook-type terminal devices: For patients with a unilateral transradial prosthesis, a hook-type terminal device should be considered when:
- i. Enhanced visibility and fine motor dexterity during object manipulation are desired, or
- ii. The user of a body powered prosthesis requires a durable terminal device.
- b. Hand-type terminal devices: For patients with a unilateral transradial prosthesis, a hand-type terminal device should be considered when:
- i. The psychosocial acceptance of a hand-like appearance is indicated for the patient, and
- ii. The cosmetic limitations of a hand-type terminal device are fully understood by the patient, and
- iii. The fine motor dexterity limitations of a hand-type terminal device are fully understood by the patient.
6) VO/VC body powered control strategies:
a. A voluntary opening body powered terminal device should be considered when the relationship between available grip strength and the strain experienced through the harness during the operation of the terminal device is fully understood and accepted by the patient.
b. A voluntary opening body powered terminal device should be considered when the patient presents with adequate strength to overcome the mechanical resistance mandated by the necessary grip strength.
c. A voluntary closing body powered terminal device should be considered when the potential energy expenditure and cognitive load associated with sustaining grip strength through ROM are fully understood and accepted by the patient.
7) For users of a unilateral transradial prosthesis, an appropriately designed body powered prosthesis or an appropriately designed externally powered prosthesis may be considered when exposure to moisture, debris, or heavy duty use is anticipated.
8) For users of a unilateral transradial prosthesis, an activity specific prosthesis should be considered when the user’s needs during a given activity exceed the capabilities of alternate prosthetic designs and/or terminal devices.
9) For users of a unilateral transradial prosthesis, multiple prostheses or terminal devices may be indicated when the user’s needs exceed the capabilities of a single prosthesis type or terminal device.
Footnotes
Declaration of conflicting interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
Guarantor: EO Contributorship: PS, EO, and SM researched literature and conceived the study. All authors served as moderators for the Delphi survey rounds. EO wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.
ORCID iDs
Erin O’Brien https://orcid.org/0000-0001-7757-1715
Phillip M Stevens https://orcid.org/0000-0003-3488-0707
References
- 1.Ziegler-Graham K, MacKenzie EJ, Ephraim PL, et al. Estimating the prevalence of limb loss in the United States: 2005 to 2050. Arch Phys Med Rehabil 2008; 89(3): 422–429. [DOI] [PubMed] [Google Scholar]
- 2.Carey SL., Lura DJ, Highsmith MJ. Differences in myoelectric and body-powered upper-limb prostheses: systematic literature review. J Rehabil Res Development 2015; 52: 247–262. [DOI] [PubMed] [Google Scholar]
- 3.Stevens PM, Highsmith MJ. Myoelectric and body power, design options for upper-limb prostheses: introduction to the state of the science conference proceedings. J Prosthet Orthot 2017; 29(4s): 1–3. [Google Scholar]
- 4.VA/DoD Clinical Practice Guideline for the Management of Upper Extremity Amputation Rehabilitation, Version 1.0. US Department of Veterans Affairs and US Department of Defense. 2014 https://www.healthquality.va.gov/guidelines/Rehab/UEAR/VADoDCPGManagementofUEAR121614Corrected508.pdf
- 5.Uellendahl J. Myoelectric versus body-powered upper-limb prostheses: a clinical perspective. J Prosthet Orthot 2017; 29(4s): 25–29. [Google Scholar]
- 6.Melton DH. Physiatrist perspective on upper-limb prosthetic options: using practice guidelines to promote patient education in the selection and the prescription process. J Prosthet Orthot 2017; 29(4s): 40–44. [Google Scholar]
- 7.Geertzen J., van der Linde H., Rosenbrand K., et al. Dutch evidence-based guidelines for amputation and prosthetics of the lower extremity: amputation surgery and postoperative management. Part 1. Prosthetics Orthotics Int 2015; 39(5): 351–360. [DOI] [PubMed] [Google Scholar]
- 8.Geertzen J., van der Linde H., Rosenbrand K., et al. Dutch evidence-based guidelines for amputation and prosthetics of the lower extremity: rehabilitation process and prosthetics. Part 2. Prosthetics Orthotics Int 2015; 39(5): 361–371. [DOI] [PubMed] [Google Scholar]
- 9.Stevens P, Rheinstein J, Campbell J. Acute postoperative care of the residual limb following transtibial amputation: a clinical practice guideline. Arch Phys Med Rehabil 2016; 97(10): e21. [Google Scholar]
- 10.Stevens P, Rheinstein J, Wurdeman S. Prosthetic foot selection for individuals with lower limb amputation: a clinical practice guideline. Arch Phys Med Rehabil 2016; 97(10): e21–e22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stevens PM, DePalma RR, Wurdeman SR. Transtibial socket design, interface, and suspension: a clinical practice guideline. JPO J Prosthetics Orthotics 2019; 31(3): 172–178. [Google Scholar]
- 12.Sedki I, Fisher K. Developing prescribing guidelines for microprocessor-controlled prosthetic knees in the South East England. Prosthetics Orthotics Int 2015; 39(3): 250–254. [DOI] [PubMed] [Google Scholar]
- 13.Stevens PM, Wurdeman SR. Prosthetic knee selection for individuals with unilateral transfemoral amputation: a clinical practice guideline. JPO J Prosthetics Orthotics 2019; 31(1): 2–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Falbo KJ, Brinkmann J. Characteristics of delphi processes in orthotics and prosthetics research. JPO J Prosthetics Orthotics 2020; 32: 161–174. DOI: 10.1097/JPO.0000000000000287. [DOI] [Google Scholar]
- 15.Linde Hvd, Hofstad CJ, Geurts ACH, et al. A systematic literature review of the effect of different prosthetic components on human functioning with a lower limb prosthesis. J Rehabil Res Development 2004; 41(4): 555–570. [DOI] [PubMed] [Google Scholar]
- 16.Keeney S, Hasson F, McKenna HP. The Delphi Technique in Nursing and Health Research. Chichester, UK: Wiley-Blackwell, 2011. [Google Scholar]
- 17.Keeney S, Hasson F, McKenna HP. A critical review of the delphi technique as a research methodology for nursing. Int J Nurs Stud 2001; 38(2): 195–200. [DOI] [PubMed] [Google Scholar]
- 18.Linde Hvd, Hofstad CJ, Limbeek Jv, et al. Use of the delphi technique for developing national clinical guidelines for prescription of lower limb prostheses. J Rehabil Res Development 2005; 42(5): 693–704. [DOI] [PubMed] [Google Scholar]
- 19.Schaffalitzky E, Gallagher P, Maclachlan M, et al. Developing consensus on important factors associated with lower limb prosthetic prescription and use. Disabil Rehabil 2012; 34(24): 2085–2094. [DOI] [PubMed] [Google Scholar]
- 20.Weber D. The development of an orthotic management protocol for preambulatory children with spastic diplegic cerebral palsy. JPO J Prosthetics Orthotics 1994; 6(3): 67–70. [Google Scholar]
- 21.Lin RS, Stevens PM, Wininger M, et al. Orthotic management of deformational plagiocephaly: consensus clinical standards of care. Cleft Palate-Craniofacial J 2016; 53(4): 394–403. [DOI] [PubMed] [Google Scholar]
- 22.Baars EC, Schrier E, Geertzen JH, et al. Biomedical and psychosocial factors influencing transtibial prosthesis fit: a delphi survey among health care professionals. Disabil Rehabil 2015; 37(21): 1946–1954. [DOI] [PubMed] [Google Scholar]
- 23.Carey SL, Stevens PM, Highsmith MJ. Differences in myoelectric and body-powered upper-limb prostheses: systematic literature review update 2013–2016. J Prosthet Orthot 2017; 29(4s): 17–20. [DOI] [PubMed] [Google Scholar]
- 24.Hsu CC, Sandford BA. The delphi technique: making sense of consensus. Pract Assess Res Eval 2007; 12(10): 1–8. [Google Scholar]
- 25.American Board for Certification . Practice Analysis of Certified Practitioners in the Disciplines of Orthotics and Prosthetics, 2019, https://www.abcop.org/individual-certification/Documents/Practitioner%20Practice%20Analysis.pdf (accessed 10 Jun 2020). [Google Scholar]
- 26.Powell C. The delphi technique: myths and realities. J Adv Nurs 2003; 41(4): 376–382. [DOI] [PubMed] [Google Scholar]
- 27.Baker J, Lovell K, Harris N. How expert are the experts? An exploration of the concept of ‘expert’ within delphi panel techniques. Nurse Res 2006; 14(1): 59–70. [DOI] [PubMed] [Google Scholar]
- 28.Fink A, Kosecoff J, Chassin M, et al. Consensus methods: characteristics and guidelines for use. Am J Public Health 1984; 74(9): 979–983. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Davis C, Onge StM. Myoelectric and body-powered upper-limb prostheses: the users’ perspective. J Prosthet Orthot 2017; 29(4S): 30–34. [Google Scholar]