Prosthesis preferences for those with upper limb loss: Discrete choice study of PULLTY® for use in regulatory decisions

Abstract

Introduction

The patient’s voice in shared decision-making has progressed from physician’s office to regulatory decision-making for medical devices with FDA’s Patient Preference Initiative. A discrete-choice preference measure for upper limb prosthetic devices was developed to investigate patient’s risk/benefit preference choices for regulatory decision making.

Methods

Rapid ethnographic procedures were used to design a discrete-choice measure describing risk and benefits of osseointegration with myoelectric control and test in a pilot preference study in adults with upper limb loss. Primary outcome is utility of each choice based conjoint (CBC) attribute using mixed-effects regression. Utilities with and without video, and between genders were compared.

Results

Strongest negative preference was for avoiding infection risk (B = −1.77, p < 0.001) and chance of daily pain (B = −1.22, p, 0.001). Strongest positive preference was for attaining complete independence when cooking dinner (B = 1.62, p < 0.001) and smooth grip patterns at all levels (B = 1.62, B = 1.28, B = 1.26, p < 0.001). Trade-offs showed a 1% increase in risk of serious/treatable infection resulted in a 1.77 decrease in relative preference. There were gender differences, and where video was used, preferences were stronger.

Conclusions

Strongest preferences were for attributes of functionality and independence versus connectedness and sensation but showed willingness to make risk-benefit trade-offs. Findings provide valuable information for regulatory benefit-risk decisions for prosthetic device innovations.

Trial Registration

This study is not a clinical trial reporting results of a health care intervention so is not registered.

Introduction

Patient preference information is a key factor in this era of patient-centered health care and shared decision-making. Including patients in health care decisions can increase their adherence to treatments and increase patient-physician communication, improving outcomes, and aiding their treatment decisions.^1–12

The inclusion of the patient’s voice in shared health care decision-making has progressed from the physician’s office to regulatory decision-making for medical devices. The Food and Drug Administration (FDA) has made consideration of patient preference a priority through their Patient Preference Initiative (PPI) whose goal is to develop and refine patient preference methods for use in weighing risks and benefits for regulatory decision making.^12,13 FDA’s Center for Devices and Radiological Health (CDRH) is pursuing a quantitative approach which permits decision makers to evaluate information about how patients trade-off the risks and benefits when making decisions about adopting a new treatment.^14,15 Patient preference is part of the field of behavioral economics which models rational behavior around health decisions to ask if expected benefits of an intervention outweigh the perceived risks. A better understanding of patient preferences could inform regulatory decisions but the FDA has little experience around.¹⁶

Incorporating patient preference of risk and benefit trade-offs into regulatory decisions.¹⁷ New prosthetic devices which we study here, are on the CDRH list of preference sensitive conditions needing preference studies to act as examples for use in regulatory approvals.^18–21 Technological advances in prosthesis design for upper limb loss include a wide range of approaches from surgical to virtual reality prosthetic training programs, to achieve multi-functional, durable, easy to use, and intuitive motor-controlled prostheses with touch sensation and proprioception.²² These advances, include moving from body-powered prostheses to myoelectric prostheses that can translate electromyographic (EMG) signals from muscles to a preprogrammed function (either sequential one-dimensional functions or a more complex pattern recognition programmed actions). These EMG signals can be non-invasive surface electrodes, which often suffer from patient factors that reduce communication between the patient and the prosthetic, and also surgically implanted electrodes which can improve the signal and thus prosthetic function. Recent developments include attaching electrode devices directly to nerves that can detect and generate both sensory and motor feedback. Other sensory implants under development range from those placed on peripheral nerves to those that directly stimulate the somatosensory cortex. Different electrode types and implantation techniques as well as haptic communication methods of communication are also being investigated. Osseointegration implants into bone address many of the complications of socket-based prostheses, while providing some proprioceptive advantages. All of these developments, have the goal of restoring normal arm/hand function, but they also all have risks, require training, lack durability, and can be slow, visible, and unpredictable.²²

Our objective is to conduct a preference study from the regulatory device perspective to provide patient preference evidence for use in regulatory approval across a broad range of prosthetic devices in development.

One early FDA promoted proof-of-concept preference study was used by CDRH as critical evidence for approval of Enteromedic’s Maestro system for obesity despite not meeting main outcomes.^17,22–26 This and other FDA proof-of-concept studies provide examples for how patient preference can frame benefits and risks for device decisions for highly preference-sensitive prosthetic devices which are being developed at a rapid rate.

Projections suggest that more than 2.2 million people are living with limb loss in the U.S.²⁷ National Health Interview Survey estimates 50,000 new hand/arm amputations every year due to congenital, tumor, disease, and trauma.^28,29 The rapid innovation in new prosthetic devices is astonishing in their ability to better mimic a human arm, wrist and hand and their motions. However,³⁰ 44–73% of those with upper limb loss do not use their prosthesis, making their adoption highly preference-sensitive, and ideal for including the patient voice.^27,28,31,32

We studied two integrated innovations: 1) Osseointegration which is the direct connection of a load- carrying rod implanted into the bone of a residual limb affording an easy, rapid snap on connection for a prosthesis resulting in increased intentional movement, and 2) myoelectric control, which uses electric signals (surface or implanted) to move the prosthesis. These device innovations can significantly improve a person’s ability to function in the world but also have significant risks of serious infection or other failures. It is imperative that we understand what those with upper limb loss want, and how they weigh their risks and benefits, if and when adopting a device. We present the discrete choice ULL patient preference measurement tool (PULLTY) and the results of its use in a pilot study designed to investigate the risks and benefits of prosthetic choices that can be used by the FDA for regulatory decisions.

This study is the first to create a choice based conjoint discrete choice measurement tool and to demonstrate patient preferences for risk and benefit tradeoffs in upper limb prosthetic devices, providing crucial preference information regarding acceptance of osseointegration and myoelectric controlled prosthetic devices. Other studies have examined preferences using other methods than discrete choice, often using a survey approach,^33–36 focus group interviews,³⁷ or literature searches.^34,38 For example,³⁹ Kelley et al., (2019) conducted an environmental scan using key informant interviews showing preference for greater device control versus invasiveness of implants, and Engdahl, et al., (2017) conducted an online survey asking how likely those with ULL would be to try noninvasive and invasive interfaces for prosthesis control.^34,38 Additionally, Rekant et al., (2022) compared priorities of different stakeholder perspectives through a survey approach, showing subjects generally prioritized ADL performance improvements and safety for future device design.³⁵ Another study used semi-structured focus groups or interviews to learn about ideal prosthesis qualities, finding most interest in improving dexterity and durability of prosthetic options in their willingness to consider invasive prosthetic interfaces.³⁷ One study used a best worst scaling prioritization survey of 47 stakeholders, finding reliability of the device most important and outdating of device least important.⁴⁰ Finally, in a large study in Veterans with a diagnosis of major upper limb amputation who were surveyed by telephone indicated that about 41% of amputees would consider surgery to restore a sense of touch, while 49% indicated they would consider surgery to gain more movement control.⁴¹

Device innovation

We developed the two new preference instruments PULLTY-T and PULLTY-V to measure risks and benefits of two innovations which have not yet received transhumoral device approval but are expected to require class III level decisions: osseointegration and myoelectric control. Osseointegration is a new technology which surgically implants a titanium post into the bone and is anchored by growth of bone and tissue around it. The prosthesis snaps onto this osseointegration device, eliminating the heavy shoulder harness and socket and problems with fit and skin abrasion. Osseointegration also allows a better range of motion and feeling of device integration not currently obtainable by current socket technology.^42,43 However, it also can require up to two surgical procedures and an uncertain and continual risk of infection and possible bone loss.^44–47 This device represents a big risk, big reward decision for the FDA, patients, and providers and it will be useful to know if and which patients are willing to take that risk.¹⁵

Prostheses with myoelectric control have either surface electrodes or implanted electrodes (some still in research phase) which detect minute muscle electromyographic (EMG) signals to control prosthetic limb movements. They offer more natural and accurate motions than without myoelectric control, but also add weight, may require surgical implantation of sensors, and require training for successful use.⁴⁸ We chose to assess these two innovations together as well as both invasive and noninvasive myoelectric control, so we best reflected the risks and benefits of across all these choice options in our survey. Current technology advances a combined approach where the myoelectric controls are an integral part of the osseointegration device.⁴⁷

Methods

Study design

This is a descriptive study of patient preference for use in regulatory approval of new prosthetic devices. The study included tool development using discrete choice, choice based conjoint (CBC) for two prosthetic device innovations and its testing and analysis; including comparisons of the value of presentation approaches, with video versus without, for use in regulatory decision making.

Sample

Adult consenting individuals with above wrist limb loss, including bilateral, were included. Eligible candidates were identified by physicians, prosthetists, or patient support group leaders at major clinical sites in the San Francisco Bay Area. Individuals nationally were also engaged at our 2018 Annual Amputee Coalition (AC) meeting recruitment booth where they completed the survey in-person. Advertisements were also posted on the AC website and on targeted social-media website, Reddit. Those interested contacted us, completed the consent form, training, and the computerized survey with a member of the research team either on the telephone, or a secure video chatting platform with shared screens.

Measure design and data collection

Sawtooth Software was used to select our CBC experimental-design algorithm based on a random, full profile, balanced overlap design to maximize the information that can be obtained for each response and using their web-based administration method.^32,49,50 The survey consisted of 18 pairs of prosthesis choices displaying different levels of the same attributes per choice and asked to choose their preferred option each time. A payment of $25 was provided after survey completion.

Two forms of the CBC PULLTY measure were developed: one using text only (PULLTY-T), and the other using video (PULLTY-V) to demonstrate characteristics of selected attributes describing prosthetic motion, but otherwise identical (Figure 1, Figure1 Supplement). Demographic information and details of cause and time with limb loss were collected, as well as quality of life information, including the standardized Disabilities of the Arm, Shoulder and Hand (DASH) outcome survey of satisfaction and current adjustment to their prosthesis.^51–54

Figure 1.

PULLTY-V Example of Discrete Choice Measure with Video pictures.

Statistical methods

The primary outcome is the utility value of each attribute level. Mixed effects logistic regression was used for analysis, with individual’s hypothetical prosthesis choice (of each pair) the dependent variable and attribute levels the independent variables. Preference scores (beta coefficients) are the part-worth utility of each attribute level and relative importance of that risk or benefit to the individual. For continuous variables, the stated preference reflected a 1% increase or decrease in preference for the given attribute, representing how variations in the levels of an attribute affect patient outlook. Mixed effects regressions were also run to determine any potential differences in preference amongst men and women as well as between the tool with video compared to the tool without video. The study was exempted by the UCSF Institutional Review Board (IRB) and followed all IRB guidelines.

Results

Instrument development

Measure development involved a selection and description of the attributes important to both the function of the device innovations, and also to the patient using rapid ethnographic methods.⁵⁵ A rapid ethnographic approach allows integration of factors into higher and higher conceptual level constructs.⁵⁵ First selection of attributes was from a broad review of the relevant literature, and ten one-on-one interviews with clinicians, key CDRH engineers, product developers, prosthetists and persons with limb loss. They evaluated more than 100 initial attributes, eliminating the least important attributes and combining others into higher constructs; also simplifying the attribute descriptions. This process resulted in 14 attributes which were then reviewed by the same group again to remove or combine some attributes, resulting in nine attributes each with 3–4 levels. We were particularly interested in finding a single complex functional attribute that represented a real-life manifestation of the usefulness of the prosthetic motion; and one that couldn’t be completed independently without the prosthesis. Single functions, often used in prosthesis testing can often also be accomplished without a prosthesis. Cooking dinner independently seemed to both be a universal task, and one that afforded enough complexity to represent the everyday usefulness of a prosthesis and its motions. Our initial concern that cooking dinner might be gender specific, were not shared by the several males that we asked, who indicated strong relevance to them. Repeated interviews then focused on understanding the attributes which are first seen in the measurement tool and also the longer descriptions underpinning these attributes that are read by the respondents as preparation to take the survey. Finally, in an effort to best represent the concepts of prosthetic motion (grip, strength, and smoothness) and independence in cooking dinner, we constructed and repeated pilot testing in the 3 subjects with upper limb loss and one prosthetist, by having them complete a version of our survey which conveyed the two movement-related attributes (grip strength and smoothness, and independence in cooking dinner) with a short video showing each attribute level, in addition to the wording describing the motion. These subjects gave informal verbal feedback on the clarity of the video compared to the word-based approach alone.

Although video may provide a clearer measure of the concepts of motion and independent functioning, other aspects of a video may affect responses (more prominence in noticing that attribute, cosmesis factors, and choice of function shown) in unpredictable ways.⁵⁶ Therefore, subjects took both measures in the same order (PULLTY-V and PULLTY-T) (Figure 1, Figure 1 Supplement).

Instrument description

The final measures had nine attributes with 3–4 levels each (Table 1). Attributes of risk included: 1) Chance of daily pain/discomfort 2) a chance of serious but treatable infection, defined as having a serious infection near the prosthesis which can be treated with three days in the hospital and two weeks of taking an infused drug, 3) Complete device failure, which resulted in not being able to use a prosthesis for five years, daily chance of pain or discomfort due to prosthesis use, and 4) number of surgeries or procedures needed (which differed in complexity and time between surgeries/procedures). Attributes of benefit included: 1) independence when cooking dinner, regarded as a crucial element for prosthetic users, 2) ease of prosthesis use with categorical levels (e.g. if it is heavy or not, or if it causes skin problems), 3) feeling of connectedness of prosthesis to your body, which can affect how motion may feel and preference of use, 4) number and strength of grip patterns and 5) ability to feel tactile sensations, which relates to what types of objects/surfaces can provide sensation to the user. The levels for each of the attributes allowed for a wide range of possibilities to allow for broad but conceivable outcomes (Table 1, Figures 1).

Table 1.

Osseointegration and myoelectric control attributes and levels.

Attribute	Levels	Description
Chance of daily pain/discomfort is ____.	6 in 10 2 in 10 0 in 10	Each presented option that you might choose may have a different chance for people to feel pain or discomfort
Independence in cooking dinner is ____.	Not independent at all Need some help Completely independent	Each presented option that you might choose may allow you a different level of independence in cooking dinner
Ease of prosthetic use is ____.	Heavy socket and harness Easily clips onto screw in bone No skin problems No heavy socket and harness	Each presented option that you might choose may have a different feature that affects how easy it is to use your prosthetic
Chance of serious but treatable infection is ____.	0 in 100 1 in 100 10 in 100 0 in 100	Each presented option that you might choose may have a different chance of having a serious infection near your prosthetic. This infection can be treated with 3 days in the hospital and 2 weeks taking an infused drug. You will not be able to use your prosthetic for 1 month. But then the infection will be completely cured and you can use your prosthetic again.
Prosthetic feels connected to your body ____.	Does not feel connected Feels somewhat connected Feels connected like normal arm	Each presented option that you might choose may have a different feeling of connectedness to your body. This may affect how your motion feels and how much you prefer to use your prosthetic arm
Able to feel touch sensations ____.	No sensations Barely feel object Feel object is rough or smooth	Each presented option that you might choose may have a different feeling of touch sensation when you bump your prosthetic hand into an object or run your prosthetic hand across a surface or object.
A failure results in no prosthetic for 5 years in ____.	0 in 100 1 in 100 10 in 100 40 in 100	Each presented option that you might choose may have a different chance of failure. This failure can be due to many things including a broken bone, loosening of the prosthetic over time, skin dying around the prosthetic, a deep bone infection, or your inability to adapt to the prosthetic. If your prosthetic fails you will not be able to use any other prosthetic for 5 years.
Surgery or procedures needed ____.	None One clinic procedure Two hospital procedures Four hospital procedures	Each presented option that you might choose may require a different number and complexity of surgeries. If more than one surgery is required, you may also have to wait for several months between surgeries.
Grip patterns, strength and motion	Two grip patterns, little strength and choppy motion Four grip patterns, moderate strength and motion Eight grip patterns, full strength and fluid motion	Each presented option that you might choose may allow you to have a different number of grip patterns with different levels of strength and motion

Sample characteristics

Thirty patients were contacted for participation, with 27 of them responding. Of those, only two did not complete all portions of the survey. Subjects were not told which technologies were being studied, so they would primarily focus on the attributes, instead of a particular technology. Of the 25 participants completers, the mean age was 47 years, (range 19–79), while 68% were male. A majority were married or partnered (64%), and all had obtained a high school diploma or higher, however most were unemployed or homemakers (64%). Unilateral limb loss was predominant (72%) with mean age at time of limb loss at 34 years. Most limb loss was below the elbow (trans-radial) (72%) with traumatic accident the primary reason for amputation (60%). Participants owned six prostheses over their lifetime, with only 2.48 prostheses being used currently. All patients had insurance covering their prostheses, and most (56%) were government plans. Many wore their prosthesis <1 hour per day (32%), but 16% wore it 4–8 hours/day and most (40%) wore it for more than 8 hours/day. Because osseointegration and myoelectric control procedures can require a period without use of their prosthesis and also a renewed period of rehabilitation, we asked how long they would be willing to forgo use of their prosthesis for “the perfect” prosthesis and how long they would be willing to undergo 2 h daily of rehabilitation for “the perfect” prosthesis, to mimic the typical utility questions. The majority (52%) of respondents were willing to go without their prosthesis for 0–3 months only, but there were also 40% willing to forgo prosthesis use for more than 12 months. Rehabilitation time showed a similar split, with 44% willing to have 2 hours/day for only 0–3 months, but 20% willing to rehabilitate for more than 12 months. These responses demonstrate patients are willing to undergo these inconveniences required for osseointegration and myoelectric control adoption. (Tables 2 and 3)

Table 2.

Patient demographics.

Characteristic (n = 25)	Value
Age, mean (range)	47.0 (19–79)
Male sex, No. (%)	17 (68.0)
Marital status, No. (%)
Divorced	2 (8.0)
Married or partnered	16 (64.0)
Single	6 (24.0)
Widowed	1 (4.0)
Education level, No. (%)
Graduate degree	9 (36.0)
Undergraduate or college degree	6 (24.0)
High school diploma or equivalent	10 (40.0)
Employment status; employed, No. (%)	9 (36.0)
Right arm dominant, No. (%)	21 (84.0)
Bilateral limb loss, No. (%)	7 (28.0)
Location of limb loss, No. (%)*
Above the elbow (transhumeral)	8 (32.0)
At the elbow (elbow disarticulation)	3 (12.0)
Below the elbow (transradial)	18 (72.0)
At the wrist (wrist disarticulation)	3 (12.0)
Age at limb loss, mean (range)	34.1 (0–77)
Reason for limb loss, No (%)*
Congenital anomaly (present at birth)	3 (12.0)
Infection-related	10 (40.0)
Traumatic accident	15 (60.0)
Tumor-related	4 (16.0)

^*May exceed total of 25, patients may have more than one limb loss.

Table 3.

Prosthetic Characteristics

Characteristic (n = 25)	Value
Insurance type, No. (%)
Government	14 (56.0)
Self-insured private	2 (8.0)
Employer-insured	9 (36.0)
Uninsured	0 (0.0)
Insurance covers prosthetic, No. (%)*	21 (91.3)
Prosthetics owned in lifetime, mean (range)	6.08 (0–20)
Prosthetics owned now, mean (range)	2.48 (1–10)
Hours a day of prosthetic use, No. (%)
Less than 1 h per day	8 (32.0)
>1–4 h per day	3 (12.0)
>4–8 h per day	4 (16.0)
>8–12 h per day	3 (12.0)
>12–16 h per day	3 (12.0)
>16–20 h per day	3 (12.0)
>20 h per day	1 (4.0)
Months willing to go without prosthetic for perfect prosthetic, No. (%)
0–3 months	13 (52.0)
>3–6 months	1 (4.0)
>6–9 months	1 (4.0)
>12 months	10 (40.0)
Months willing to undergo 2 h of rehab for perfect prosthetic, No. (%)
0–3 months	11 (44.0)
>3–6 months	3 (12.0)
>6–9 months	4 (16.0)
>9–12 months	2 (8.0)
>12 months	5 (20.0)

^*Only 23 respondents.

Preference utilities

Relative preferences for the nine attributes of risk and benefit are represented as beta-coefficients from a mixed effects regression (Table 4, Figure 2, and Table1 Supplement, Figure 2 Supplement) using PULLTY-V (with video) as the standard. Attributes of risk all showed significance and a negative preference in relation to other attributes, as expected. Attributes of benefit were mostly of high preference, however, feelings of connectedness to the prosthesis as well as feelings of touch and sensation did not show significance at any level.

Table 4.

Attributes and relative preferences for upper limb prosthetic devices (with picture/video) n = 25.

Attribute	β - coefficient	SE	95% CI	P-value
Pain	−1.22	0.31	−1.83 to −0.60	<0.001
Dinner
Not independent at all	Reference	—	—	—
Need some help	1.07	0.19	0.69 to 1.44	<0.001
Completely independent	1.62	0.20	1.23 to 2.01	<0.001
Ease
Heavy socket and harness	Reference	—	—	—
Easily clips onto screw in bone	0.49	0.22	0.06 to 0.93	0.03
No skin problems	0.56	0.22	0.12 to 0.99	0.01
No heavy socket and harness	0.37	0.22	−0.06 to 0.80	0.09
Infections	−1.77	0.39	−2.53 to −1.01	<0.001
Connected
Does not feel connected	Reference	—	—	—
Feels somewhat connected	0.15	0.19	−0.22 to 0.53	0.42
Feels connected like normal arm	0.23	0.19	−0.15 to 0.60	0.23
Touch
No sensations	Reference	—	—	—
Barely feel object	−0.30	0.19	−0.67 to 0.07	0.12
Feel object is rough or smooth	0.05	0.19	−0.32 to 0.43	0.79
Failure	−1.04	0.48	−1.98 to −0.09	0.03
Procedures
None	Reference	—	—	—
One clinic procedure	−0.23	0.22	−0.66 to 0.19	0.29
Two hospital surgeries	−0.70	0.22	−1.13 to −0.27	0.001
Four hospital surgeries	−1.01	0.22	−1.45 to −0.58	<0.001
Grip
Two pattern grip	Reference	—	—	—
Four pattern grip	1.28	0.19	0.90 to 1.66	<0.001
Eight pattern grip	1.26	0.19	0.88 to 1.64	<0.001
Constant	−0.96	0.33	−1.60 to −0.32	0.003

Abbreviations: CI, confidence interval; SE, standard error.

Figure 2.

Relative preferences for attributes of risk and benefit in CBC with pictures/videos versus without pictures/videos.

Avoiding a serious but treatable infection was the most important attribute of risk for respondents with a 1.77 decrease in preference shown for every 1% increase in infection rate (p < 0.001). Preference for avoiding a chance of daily pain was next most important, with every 1% increase in pain resulting in a 1.22 decrease in relative preference (p < 0.001). The risk of device failure resulting in no prosthesis for 5 years followed closely behind, with 1.04 decrease in relative preference for every 1% increase in failure rate (p = 0.03). All levels of required procedures or surgeries were associated with negative preferences, with four hospital procedures holding the least preference of all (β = −1.01, p < 0.001) and two hospital procedures slightly less important (β = −0.70, p = 0.001).

Patients showed the strongest positive preferences for levels of ‘independence in cooking dinner’. Complete independence when cooking dinner showed the most positive preference (β = 1.62, p < 0.001), similar to avoidance of 1% serious infection risk; while ‘needing some help while cooking dinner’ was slightly less favored (β = 1.07, p < 0.001). Specific levels describing the ‘Ease of prosthesis use’ attribute, were all held with similar preference, with “easily clips into screw in bone” (β = 0.49, p = 0.03) and ‘having no skin problems’ (β = 0.56, p = 0.01). However, the ‘ability to avoid having a heavy socket and harness’ was surprisingly not shown to be significant (p = 0.09). The attributes describing ‘ability to feel sensations of touch’ and ‘feelings of prosthesis connection to the body’, strong benefits of osseointegration; were both insignificant. On the other hand, grip patterns enabled by myoelectric control were significant for both four-pattern (β = 1.28, p < 0.001) and eight-pattern grips (β = 1.26, p < 0.001), and notably more grip patterns were only slightly more preferred than fewer (Table 4).

Preferences Comparing Video versus No Video

Our standard preference measures was reported with video, which were provided for only two of the attributes 1) all levels of the ‘ability to cook dinner independently’, and 2) for all levels of ‘number and strength of grip patterns ‘. The preference to avoid infection risk was similar for the measures with video (β = −1.77, p < 0.001), and without video (β = −1.73, p < 0.001). and remained the strongest negative preference. There were important differences in preferences when the video was used to describe those attributes, however. Those responding to video, showed a much stronger preference for ‘complete independence’ (β = 1.62, p < 0.001), and’ some help (β = −-1.07, p < 0.001). in cooking dinner’, than when responding with no video (β = 1.06, p < 0.001 and β = 0.78, p < 0.001). The preferences when video was shown for grip pattern number and strength were also more than double (4 grip:β = 1.28,p < 0.001, 8 grip: β = 1.26, p < 0.001) those when the video for this attribute were not shown. (4 grip: β = 0.52, p = 0.006, 8 grip:β = 0.69, p < 0.001). Perhaps as a result, other preferences in PULLTY-T were correspondingly lower than were the same attributes in the PULLTY-V, even though these attributes had no video on either measure; for example avoidance of pain, ease of use, and avoidance of procedures. Surprisingly though, the desire to avoid device failure was 3 times stronger in the no video measure (β = 3.15, p < 0.001) than when video was part of the measure β = −1.04, p < 0.03). Additionally, in the measure without video, the preference for feelings of connectedness with the prosthesis were significant, and one level of ability to feel touch sensation, was also significant, though neither were significant when taking the measure with video. (Figure 2, Figures 2 and 3 Supplement, and Table1 Supplement).

Gender differences in preference

There were important variations in preference between men and women. (Figure 3 and Tables2, 3 Supplement) Infection risk was consistently the attribute with the strongest negative preference for both men and women, but males showed stronger risk aversion to a serious infection (β = −2.09, p < 0.001), than did women ((β = 1.55, p0.03). Preference to avoid a ‘daily chance of pain’ was not significant for women (β = −0.82, p = 0.16), nor was avoidance of device failure important to women (β = −0.05, p = 0.95). Men on the other hand, considered pain avoidance to be the most important attribute (β = −1.56, p < 0.001), with device failure following closely behind (β = −1.49, p = 0.02). Another difference among gender groups was with ‘ease of prosthesis use’. Women held ‘ease of use’ in high regard as all three levels were shown to be significant ((β− = 1.11, p = 0.01), (β = 1.56, p < 0.001), (β = 1.46, p = 0.001)), with a preference for ‘no skin problems’ having the highest preference of any attribute for women. Conversely, men showed no significant preference for ‘ease of use’ with the prosthesis at any level. Men also showed slightly higher preference for ‘complete independence in cooking dinner’ (β = 1.89, p < 0.001), and slightly lower for ‘some independence’ (β = 1.06, p < 0.001), than did women (β = 1.44, p = 0.001), (β = 1.15, p < 0.001).

Figure 3.

Relative preferences for attributes of risk and benefit in mens versus women in choice-based conjoint with pictures/videos.

Discussion

This study is the first to create a choice based conjoint discrete choice measurement tool and to demonstrate patient preferences for risk and benefit tradeoffs in upper limb prosthetic devices, providing crucial preference information regarding acceptance of osseointegration and myoelectric controlled prosthetic devices. Other studies have examined preferences using other methods than discrete choice, often using a survey approach,^33–36 focus group interviews,³⁷ or literature searches.^34,38 For example, Kelley et al., (2019) conducted an environmental scan using key informant interviews showing preference for greater device control versus invasiveness of implants, and Engdahl, et al., (2017) conducted an online survey asking how likely those with ULL would be to try noninvasive and invasive interfaces for prosthesis control.^34,38 Additionally, Rekant et al., (2022) compared priorities of different stakeholder perspectives through a survey approach, showing subjects generally prioritized ADL performance improvements and safety for future device design.³⁵ Another study used semi-structured focus groups or interviews to learn about ideal prosthesis qualities, finding most interest in improving dexterity and durability of prosthetic options in their willingness to consider invasive prosthetic interfaces.³⁷ One study used a best worst scaling prioritization survey of 47 stakeholders, finding reliability of the device most important and outdating of device least important.⁴⁰ Finally, in a large study in Veterans with a diagnosis of major upper limb amputation who were surveyed by telephone indicated that about 41% of amputees would consider surgery to restore a sense of touch, while 49% indicated they would consider surgery to gain more movement control.⁴¹

Our study is informed by these studies, but unique in using choice based conjoint analysis to ask those with ULL to directly weigh the risks and benefits of osseointegration and myoelectric control choices. We found that patients preferred complete independence in cooking dinner above all other positive attributes which likely reflects their desire for independence in their daily life tasks. Four pattern grips and above were heavily favored by patients, especially in those who saw their movement with video; further demonstrating the desire for enhanced motion and independence that comes along with prosthetic device improvements. Interestingly, respondents slightly preferred the four grip patterns to the 8 grip patterns when shown the video of their motions, which may indicate they may not prefer the most complex device features. Ease of prosthesis use, determined as a major advantage of osseointegration, was shown to be significant but only slightly preferred by patients. Another advantage for these prosthetic innovations (feelings of connectedness) also were generally not found to be of significant importance relative to the other attributes. Easily being able to clip onto screw in bone and having no skin problems both had similar levels of preference, but much lower than any level of independence in cooking dinner. Infections, thought to be one of the major hurdles for patient’s considering osseointegration, was shown to be the most important for respondents to avoid. Next, was avoidance of the chance of daily pain or discomfort followed by chance of device failure resulting in 5 years without a prosthetic device. Levels of pain or discomfort are often reduced by osseointegration, however these innovations do carry the risk of potential device failure.

These findings highlight that individuals are willing to make trade-offs between the benefits of osseointegration/myoelectrics compared to the potential risks associated with the innovations. The preference to avoid a 1% risk of infection, is only slightly higher than the preference for independence in cooking dinner, which shows a willingness to make this trade-off. Individuals also showed a willingness to trade 1% chance of pain increase, for the benefits of independence in cooking dinner and for attaining four and 8 new grip patterns. The differences shown between genders, are also important, as men showed more risk aversion, but also more benefit seeking than did women.

This study was subject to certain limitations. First, sample size calculations indicate a preferred sample of 75 given the number of attributes and number of choice tasks shown. However, since ULL is not common, we were unable to obtain the desired sample over the allowed study period. Despite this small sample however, a majority of the chosen attributes showed significance, indicating post-hoc, an adequate sample size. Our 25 patient sample is still small and thus may not represent preferences of all those with upper limb loss.³² The smaller sample size also did not allow for many subgroup comparisons which could further demonstrate respondent heterogeneity as we did across gender We did not present the video and picture only CBC’s in a random order, so our results may be affected by this order of presentation. However, since the seven attributes in the video version that did not have video, had similar responses to the picture only version, this effect is likely small. Most respondents were current prosthetic device users and therefore may not represent those not choosing to use a prosthetic device. Many patients who have undergone osteointegration may have experienced physical issues limiting their use of a conventional socket prosthesis such as a short residual limb, recurring skin breakdown, or inability to tolerate socket materials. Despite the strong significance across attributes found in this study, a larger study sample is needed to further the understanding of preferences to these device innovations. Despite these limitations, this study provides insightful data on the preferences of upper limb loss patients in relation to novel prosthetic devices.

There are several unique features of this study that will progress the use of patient preference in regulatory decision-making. First, we tested two different discrete choice measure approaches (with and without video) which provides evidence for the FDA that the use of video to show more clearly the function of these devices can increase the strength of individuals preferences. There are few validity studies in this field. Some studies show little agreement across measurement methods, and variation around other factors such as tool presentation, but also that CBC methods can match real-world behavior.^57,58 More validity evidence such as for our study, is clearly needed to support the use of preference measures for healthcare decisions.

Secondly, we made sure that the attributes selected included both evidence-based features of the devices that are a requirement for use by the FDA, but also factors that were important to the individuals with limb loss. Additionally, attributes of cost were not used, despite affecting prosthesis choice, because these are not allowed in regulatory decisions. During a pre-submission application process for the PULLTY measures and a preliminary discussion meeting at the FDA, emphasis was put on including primarily only proven clinical trial attributes of the devices, with less acceptance of patient-specific outcomes identified as important by those with limb loss.^59,60 Further collaboration of discrete choice researchers with FDA decision makers is needed to validate patient-centered attributes the way that other quality of life and outcome measures are validated. Our study project team included members of CDRH, an individual with upper limb loss, prosthetists and research experts, who provided the full range of perspectives necessary for providing new information that is valid and useful for decision making. Our study demonstrates the importance of including both proven device features and associated patient-chosen attributes.

Given the lack of consistent use of newly developing and expensive prostheses, it is essential to learn about the viewpoint of affected individuals for weighing the risks and benefits of these innovations, especially for the FDA who will be evaluating these innovations for the first time as they are reviewed for approval and marketing. It is anticipated that the osseointegration procedure will require either a Class III premarketing approval (PMA) or go through the humanitarian device exemption (HDE) process, given its potential infection risk to weigh against its benefits. The myoelectric device itself is expected to require a Class II approval process where patient preference information will be used more to inform performance standards.^61–63

Our overall objective is uniquely to provide an example for measuring patient preference as a means to enable the incorporation of patient preference into the benefit-risk assessments made by FDA. This study builds on this small but existing evidence base where PPI was used in regulatory decisions.²³ This study provides another example of patient preference measurement with anticipated usefulness to the FDA in assessing the fast-moving prosthetic device innovations and will further the science and acceptance of quantitative patient preference measurement methods by the FDA.

Appendix.

List of abbreviations

FDA

Federal drug administration

CBC

Choice based conjoint

PPI

Patient preference initiative

CDRH

FDA’s center for devices and radiological health

EMG

Electromyography

DARPA

Defense advanced research projects agency

UCSF

University of california san francisco

CERSI

Center of excellence in regulatory science and innovation

AAOP

American academy of orthotists and prosthetists

ULL

Upper limb loss

DASH

Disabilities of the arm, shoulder, and hand

CHR

Committee on human subjects

BTO

Biological technologies officer

PULLTY

Preferences upper limb loss tool

PULLTY -T

Preferences upper limb loss tool- no video

PULLTY-V

Preferences upper limb loss tool - video

ORCID iD

Leslie Wilson https://orcid.org/0000-0002-3305-8730