Development of a Patient Preference Survey for Wearable Kidney Replacement Therapy Devices

Key Points

• We included the risks of serious bleeding and serious infection based on patient concerns and regulator input about future trial end points.

• The survey will estimate maximal acceptable risks for serious bleeding and infection and willingness to wait for devices with lower risk.

Visual Abstract

Abstract

Background

Recent innovations have the potential to disrupt the current paradigm for kidney failure treatment. The US Food and Drug Administration is committed to incorporating valid scientific evidence about how patients weigh the benefits and risks of new devices into their decision making, but to date, premarket submission of patient preference information (PPI) has been limited for kidney devices. With input from stakeholders, we developed a survey intended to yield valid PPI, capturing how patients trade off the potential benefits and risks of wearable dialysis devices and in-center hemodialysis.

Methods

We conducted concept elicitation interviews with individuals receiving dialysis to inform instrument content. After instrument drafting, we conducted two rounds of pretest interviews to evaluate survey face validity, comprehensibility, and perceived relevance. We pilot tested the survey with in-center hemodialysis patients to assess comprehensibility and usability further. Throughout, we used participant input to guide survey refinements.

Results

Thirty-six individuals receiving in-center or home dialysis participated in concept elicitation (N=20) and pretest (N=16) interviews. Participants identified reduced fatigue, lower treatment burden, and enhanced freedom as important benefits of a wearable device, and many expressed concerns about risks related to device disconnection—specifically bleeding and infection. We drafted a survey that included descriptions of the risks of serious bleeding and serious infection and an assessment of respondent willingness to wait for a safer device. Input from pretest interviewees led to various instrument modifications, including treatment descriptions, item wording, and risk-level explanations. Pilot testing of the updated survey among 24 in-center hemodialysis patients demonstrated acceptable survey comprehensibility and usability, although 50% of patients required some assistance.

Conclusions

The final survey is a 54-item web-based instrument that will yield estimates of the maximal acceptable risk for the described wearable device and willingness to wait for wearable devices with lower risk.

Introduction

RRT technology has been stagnant for decades. The majority of people with kidney failure are treated with in-center hemodialysis, a therapy with debilitating side effects and burdensome thrice-weekly clinic visits (1). However, catalyzed by US regulatory reform and the Kidney Innovation Accelerator (KidneyX) prize competition (2,3), there has been unprecedented RRT research and development in recent years (4). It is anticipated that this focus will yield innovations in wearable and implantable RRT technologies—products that could disrupt the current RRT paradigm and improve the lives of people affected by kidney failure.

In making regulatory approval decisions for medical devices, the US Food and Drug Administration (FDA) Center for Devices and Radiologic Health (CDRH) considers whether submitted evidence provides reasonable assurance that a device is safe and effective for its intended use (5). In addition, and when the supporting information meets FDA specifications for valid scientific evidence, the FDA may consider patient perspectives of risk tolerance and perceived benefits in their assessment of the device’s risk/benefit profile (6,7). For example, patient preference information (PPI), defined by CDRH as “qualitative or quantitative assessments of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions” (6), was used to support the labeling expansion of a home hemodialysis system to permit solo use during waking hours (8). Valid PPI is generated from well-designed and -conducted studies that use “fit-for-purpose” data-collection strategies and can be used by regulators to prioritize outcomes for clinical trials, establish patients’ perspectives on minimum acceptable performance thresholds (i.e., minimal acceptable benefit and maximal acceptable risk), and inform acceptable levels of uncertainty for outcomes (6,9). Incorporating stakeholder perspectives, particularly patient voices, into the process of designing PPI studies is critical to ensuring the relevance and quality of the resultant data.

With an overall objective of supporting the incorporation of patient perspectives into regulatory decision making regarding RRT technologies, we partnered with patients, regulators, innovators, and clinicians to develop a survey intended to yield valid, regulatory-grade PPI, capturing how patients trade off the potential benefits and risks of RRT devices.

Materials and Methods

Overview

A steering committee provided overall project guidance and was supported by a survey development workgroup. Steering committee and workgroup members included patients (n=4), preference experts (n=4), regulators (n=7), and academic nephrologists (n=4; Supplemental Table 1). After conducting an environmental scan of RRT research and development (Supplemental Table 2), the committee selected wearable RRT devices (both hemodialysis and peritoneal dialysis) as the alternative treatment and in-center hemodialysis as the reference treatment for the PPI survey. Wearable devices were selected due to their potential for near-term market readiness, and because patient preferences play an important role in the adoption of such innovations (6).

Following recommendations for formative qualitative research to inform the development of quantitative preference instruments, we used a two-staged approach for survey development that included concept elicitation and subsequent content refinement based on target population input (6,9–11). We considered the FDA-recommended qualities of patient preference studies (6) (Table 1) and followed a five-step process: (1) conducting concept elicitation interviews, (2) constructing a draft survey, (3) pretesting and responsively updating the survey, (4) pilot testing the survey in the in-center hemodialysis setting, and (5) planning survey fielding (Figure 1). The RTI Institutional Review Board deemed this research exempt from further review (Study #00021084).

Table 1.

FDA-recommended qualities of patient preference studies (6)

Recommended Quality (definition)	Consideration in Survey Development
Patient centeredness (Ensure that the patient, not the health care professional is focus of the study)	• Patient input • Concept elicitation interviews • Pretesting
Representativeness of the sample and generalizability of results (Measure the preferences of a representative sample of adequate size)	• Pilot testing • Fielding planning • Purposive sampling on specific respondent characteristics
Capturing heterogeneity of patients’ preferences (Reflect preferences of patients from full spectrum of disease for which the device is intended to be used)	• Concept elicitation interviews • Pretesting • Purposive sampling on specific respondent characteristics
Established good research practices (Follow guidelines established by a recognized professional organization)	• All development steps aligned with ISPOR and Medical Device Innovation Consortium best practices (31,32) • Involvement of PPI experts in study design
Effective communication of benefit, harm, risk, and uncertainty (Communicate the quantitative aspects of the health information in ways that the patient can understand and cognitively process this information)	• Patient input • Survey construction using best practices • Pretesting • Pilot testing
Minimal cognitive bias (Minimize potential cognitive biases such as framing, anchoring, simplifying heuristics, or ordering effect)	• Survey construction • Pretesting • Pilot testing
Logical soundness (Test logic and consistency of presented data)	• Patient input • Pretesting • Pilot testing
Relevance (Include critical aspects of harm, risk, benefit, and uncertainty, ensuring some consistency with end points from clinical studies of the device)	• Concept elicitation interviews • Patient, regulator, and expert input
Robustness of analysis of results (Ensure appropriate interpretation of the collected evidence with attention to understanding the potential sources for uncertainty)	• Involvement of PPI experts in study design
Study conduct (Administer by trained research staff or, when self-administered, use a tutorial and quiz before answering questions to ensure adequate comprehension)	• Survey construction • Pilot testing • Fielding planning
Comprehension by study participants (Ensure that participants fully understand the harm, risk, benefit, and uncertainty and other medical information being communicated to them)	• Patient input • Pretesting • Pilot testing

FDA, Food and Drug Administration; PPI, patient preference information.

Figure 1.

Survey development process. Survey development involved a five-step process: (1) conducting concept elicitation interviews, (2) constructing a draft survey, (3) pretesting and responsively updating the survey, (4) pilot testing the survey in the in-center hemodialysis setting, and (5) planning final survey fielding.

Concept Elicitation Interviews

Concept elicitation interviews captured patient perspectives on potential benefits and risks of wearable devices and characterized patient knowledge and questions about such devices. Experienced interviewers used a semi-structured interview guide (Supplemental Table 3) to conduct telephone-based interviews in May and June 2020. Most interviews (n=18) were conducted as dyads (i.e., two participants at a time). Dyadic interviewing falls between individual interviews and focus groups on the spectrum of interactive qualitative data collection and allows interviewees to react to and interact with each other, enhancing the depth of data collected (12). We selected a virtual approach due to COVID-19 safety concerns. We stopped interviews when no new benefits or risks emerged after three consecutive interviews (data saturation).

We recruited participants via telephone and email, using a research firm’s national database that includes individuals with kidney disease as the recruitment source. Individuals were eligible to participate if they were ≥22 years old (FDA definition of adulthood) (13), English speaking, and currently receiving in-center hemodialysis, home hemodialysis, or peritoneal dialysis. We used purposive sampling to ensure representation of individuals of varying sociodemographic characteristics, education levels, dialysis modalities, and levels of patient activation as measured by the Consumer Health Activation Index (14). Participants received $75 remuneration.

Survey Instrument Construction

We selected the threshold technique as the analytic approach to quantifying patient preferences, given its simpler design and smaller sample size requirement (compared with discrete choice experiments), capacity to produce individual respondent-level (versus sample-level) estimates of maximal acceptable risk, and precedent for use in regulatory PPI studies (9,15–17). We then constructed a draft instrument using concept elicitation interview findings to inform treatment descriptions and risks/benefits (attributes). We relied on published literature to quantify risk estimates for each treatment (Supplemental Table 4). In addition, we interviewed individuals from teams (industry and academic) actively engaged in developing wearable dialysis devices to inform our device descriptions and graphics. We revised the instrument based on iterative input from steering committee members, patient advisors, industry representatives, and content experts.

Survey Pretest Interviews

After survey revision, we performed two rounds of pretest interviews with patients to assess survey face and content validities and comprehensibility and relevance. Results from each round informed subsequent survey modifications. Experienced interviewers used a semi-structured interview guide (Supplemental Table 3) to conduct telephone-based interviews in October and November 2020. Before the interview, we mailed participants a paper copy of the survey instrument so they could view the device descriptions and graphics during the interview. The interviewers then used the think-aloud technique—a process by which participants verbalize their thoughts as they complete a task to obtain feedback on instructions, wording, response options, and graphics (18). Interviews were approximately 90 minutes, and participants received $100 remuneration.

We used the same participant selection criteria and recruitment source for pretest interviews as for concept elicitation interviews. We used purposive sampling to include individuals of varying sociodemographic characteristics, dialysis treatment modalities, and comfort with technology. Consistent with expert recommendations, our target sample size was five to ten participants per round (19), and we stopped recruitment upon reaching data saturation.

Survey Pilot Test

After responsive survey revisions, a professional, native Spanish-speaker translated the survey into Spanish, refining in response to pretesting with five Spanish-speaking dialysis patients. The goal of the translation process was conceptual equivalence (20). We then converted the survey from paper to a web-based format and conducted a pilot test to assess the feasibility of survey administration in the in-center hemodialysis setting. Pilot test participants completed the surveys on tablet computers during hemodialysis treatments while research coordinators recorded observations in structured field notes on assistance required, questions and/or comments, observed difficulties (comprehension or technical), and survey completion time.

We used fliers and in-person approaches to recruit participants from four US Renal Care–operated clinics in Alaska, Georgia, and Texas. Individuals were eligible to participate if they were ≥22 years old, English or Spanish speaking, and had received in-center hemodialysis for ≥3 months. We used purposive sampling to identify individuals with varying technology comfort, oversampling for those self-reporting less tablet computer experience. Participants received $50 remuneration.

Analytic Approach

We used descriptive statistics (count [%], mean [±SD]) to report participant characteristics and pre- and pilot testing quantitative findings.

Interviews were audio recorded and transcribed. We organized concept elicitation interview data by question and used directed content analysis to identify potential risks and benefits of wearable devices important to patients (21,22). Through iterative discussion, researchers resolved discrepancies and reached consensus. To evaluate question performance, we organized pretest interview data by survey section (e.g., treatment and risk descriptions, risk/benefit trade-off questions, waiting time question). We also examined pilot test field notes to understand sources of participant challenges. We created overall summaries, which the study team collectively reviewed, along with accompanying notes, to confirm accurate data summation, and then made responsive survey updates.

Results

Participant Characteristics

Table 2 displays participant characteristics. We conducted 20 concept elicitation and 16 pretest interviews with 17 (47%) in-center hemodialysis, 11 (31%) home hemodialysis, and eight (22%) peritoneal dialysis patients. Participants ranged from 26 to 77 years of age, 16 (44%) were women, nine (25%) had a high school or equivalent education or less, and ten (28%) were of “low” patient activation (14). Pilot test participants included 24 in-center hemodialysis patients who ranged from 34 to 72 years of age. Eleven (46%) were women, 14 (58%) had no more than a high school or equivalent education, and five (21%) had received in-center hemodialysis for <1 year. Notably, 12 (50%) reported using a computer or tablet computer never or rarely, and five (21%) reported being uncomfortable using a computer or tablet.

Table 2.

Characteristics of patient participants in survey development

	Survey Development Participants (N=60)
Characteristic	Concept Elicitation Interviews (N=20)	Survey Pretest Interviews (N=16)	Survey Pilot Test (N=24)c
Age, yr
22–40	5 (25)	5 (31)	1 (4)
41–52	6 (30)	2 (13)	4 (17)
53–65	6 (30)	6 (38)	10 (44)
66–75	0	1 (6)	8 (35)
≥76	3 (15)	2 (13)	0
Sex
Men	10 (50)	10 (63)	13 (54)
Women	10 (50)	6 (38)	11 (46)
Race and ethnicity
Black, non-Hispanic	8 (40)	7 (44)	4 (17)
Asian, non-Hispanic	1 (5)	0	0
White, non-Hispanic	9 (45)	5 (31)	7 (29)
Hispanic or Latino	2 (10)	4 (25)	13 (54)d
Education level
Less than high school	0	0	6 (25)
High school or equivalenta	6 (30)	3 (19)	8 (33)
Associates degree/trade school	1 (5)	3 (19)	0
Some college	8 (40)	0	7 (29)
College graduate	2 (10)	1 (6)	3 (13)
Postgraduate	3 (15)	2 (13)	0
Census region
Northeast	2 (10)	0	0
South	8 (40)	10 (63)	20 (83)
Midwest	6 (30)	5 (31)	0
West	4 (20)	1 (6)	4 (17)
Current dialysis modality
In-center hemodialysis	9 (45)	8 (50)	24 (100)
Home hemodialysis	6 (30)	5 (31)	0
Peritoneal dialysis	5 (25)	3 (19)	0
Time on dialysis, yr (any modality)
3 mo–1	—	3 (19)	5 (21)
>1	—	13 (81)	19 (79)
Time on current dialysis modality, yr
<1	6 (30)	3 (19)	—
1–2	3 (15)	4 (25)	—
≥3	11 (55)	9 (56)	—
Patient activationb
Low	6 (30)	4 (25)	—
Medium	9 (45)	10 (63)	—
High	5 (25)	2 (12)	—

Values are presented as n (%). All characteristics were patient reported. —, data element was not collected. We recruited different individuals for each stage of development. There is no overlap in participants across concept elicitation interviews, survey pretest interviews, and survey pilot test.

^aCompletion of high school degree or tests of General Educational Development (GED).

^bAssessed using the Consumer Health Activation Index (14).

^cAge value was missing for one pilot test participant (age n=23).

^d8 (33%) completed the survey in Spanish.

Concept Elicitation Interview Findings

Of the 20 concept elicitation interviewees, seven (35%) had heard of wearable RRT devices, and after reviewing a description of such devices, six (30%) expressed strong interest in use, 13 (65%) expressed moderate interest, and one (5%) had no interest. Participants were asked to rank the potential benefits and downsides (risks) of a wearable device that were most important to them. For potential benefits, participants were most likely to rank “feel better and have more energy” (nine interviewees, 45%), “needing fewer medications” (six interviewees, 30%), and “ability to drink more fluids” (six interviewees, 30%) as first or second in importance. For potential downsides, participants were most likely to rank “catheter may become accidentally removed or disconnected “(18 interviewees, 90%) and “device may stop working” (12 interviewees, 60%) as first or second in importance. Specific participant concerns regarding catheter disconnection included pain, bleeding, and/or infection.

Table 3 displays illustrative quotations. Although participants were intrigued by wearable devices and enthusiastic about potential benefits—especially enhanced freedom—they wanted more information about device safeguards (e.g., disconnection alarms, remote monitoring), characteristics (e.g., size/weight, visibility), function (e.g., battery life, fluid storage), and effectiveness (e.g., clearance, fluid removal). Finally, participant interest in using a wearable device appeared to differ by treatment modality, with patients receiving in-center hemodialysis expressing more definitive interest in wearables, and people receiving home dialysis expressing more moderate interest. All three peritoneal dialysis users who responded noted that their current modality offers benefits similar to those of a wearable device (e.g., flexible treatment schedule).

Table 3.

 Illustrative quotations about wearable RRT devices from concept elicitation interview participants

Domain/Responses	Participant Quotation (Current Dialysis Modality)
Knowledge
Heard of wearables	“I heard that it is pretty much just like a PD machine, and it dialyzes as you go, and its battery operated … It allows you to be not worried about going into a center.” (ICHD)
Not heard of wearables	“For dialysis, no. But I am a diabetic, and I do have a wearable continuous meter.” (PD)
	“Not this type of wearable device. Heard of something like a pacemaker.” (HHD)
Initial reactions
Strong interest	“It sounds 100% great. When will it be available?” (ICHD)
	“I would be very interested … it means I wouldn’t have to get up every other day and go into dialysis. It would be more convenient. I could get out and do what I want to do.” (ICHD)
	“If I could have a device in a backpack that I can make discrete that would allow me to travel, I would definitely do it.” (HHD)
Moderate interest	“I actually think it’s great, but it depends if you are OK carrying the device with you … it can be more tiring carrying the device than you expect.” (ICHD)
	“Theoretically, I’d be very interested in it, but I would have to know a whole lot more about the actual device … But the idea of having more freedom to be able to do stuff and not being tethered to the machine as long as I am almost every day is very appealing.” (HHD)
	“I would be interested to see how it goes. I never want to be the first one to use something. I would wait on the sidelines for a good while.” (PD)
No interest	“I accepted the fact that I’m on dialysis, that I have to do dialysis to stay alive. No, not for me … Plain and simple, I would not be interested in it at all.” (ICHD)
Potential benefitsa
More freedom/ability to be active	“Just a general overall increased quality of life.” (ICHD)
	“You’re not tied down three times a week sitting on a chair and not [using] one of your arms. To me, it sounds like it could be something very positive.” (ICHD)
	“The biggest benefit would be freedom from the machine. You kind of normalize dialysis, but the reality is that you got to do it to stay alive. So, you fit that schedule into your life.” (HHD)
	“Mobility. Being able to move around and get things done without having to stay in one room … It [would] just [be] a blessing … if it’s able to give me a little bit more freedom.” (PD)
More independence	“Being self-sufficient would be the ultimate benefit.” (ICHD)
	“Well, depending on the machine and how it’s actually designed … to be able to be more self-sufficient and independent, to not have to be stuck in my house all day long.” (HHD)
Fewer symptoms	“I experience dizziness, shortness of breath, fainting, so [better symptom control] is important.” (ICHD)
	“…fewer symptoms is really important because, for me, I’ve had episodes where I have a sudden blood pressure drop … and my stomach starts hurting, and I start sweating. And I start getting really short of breath, and I feel like I’m going to faint. My heart starts racing, and it’s a pretty painful and a little scary. It’s almost like a near death experience.” (PD)
Fewer medications/diet restrictions	“Needing fewer medications and better control of blood pressure are close to the top of the list for me, because … I’m on four different blood pressure medications, and it’s a fight to keep my blood pressure down.” (ICHD)
	“Not taking the phosphate binders would be a benefit. They are big pills.” (HHD)
	“Being on dialysis and having a very strict diet on top of being diabetic is stressful. So, if I’m able to eat a little bit more things that I would like and drink a little bit more because it’s hot in the summer…without it actually causing harm to my body, then that’s a benefit.” (PD)
Potential harms or downsidesa
Infection	“I worry about infection. I have an autoimmune condition so I’m more vulnerable.” (ICHD)
	“My biggest concern would be the infection … with catheters. I had infections with [my dialysis catheter] … which was very annoying, very painful, and potentially extremely dangerous. I would be very worried about any kind of wearable device that was attached to a hemodialysis catheter. I’d also wonder, it would probably have to be a chest catheter as well I assume, or neck. That certainly would worry me.” (HHD)
	[Reflecting on experience with peritonitis]: “Yeah, I had to stay in the hospital. They filled me up with antibiotics. I don’t want to go through that no more.” (PD)
Device disconnection	“I’ve had needles pulled out on accident in hemodialysis, and the amount of blood that came out of those little holes was a lot. I mean it was like a murder scene. Now imagine a tube … that you guys are probably going to be using. You could bleed out. And you can catch it on things. And also [get] an infection.” (ICHD)
	“The catheter becoming accidentally removed if it’s connected to the bloodstream. This’ll cause bleeding, pain, and possible infection or even bleeding out. Especially if you’re on heparin or something. I mean forget it.” (ICHD)
	“Catheter may become accidentally removed or disconnected … that is really scary.” (PD)
Lack of supervision	“…if you ever had a problem, you’re not with any kind of professionals that can fix the problem.” (ICHD)
	“So, if it stopped, I’d want to be around somebody. I’d want to be able to get to my dialysis center or if you were out of town then what do you do?” (ICHD)
Feeling self-conscious	“…when I work, I don’t want things getting in my way and, in certain social settings, that’s kind of a downfall. It’s like, I’m sorry, but people do look down on people that are handicapped. Somebody in a wheelchair, or somebody’s got a big old machine or oxygen in their nose. And if you trying to do work, you’re trying to give them business….. You might lose an account because people discriminate, and they don’t say why. I am a freelancer. And if I go to a client and then they see that, this tube and a machine on me, they’re not going to pick me again because they think I’m too sick to do the job. That’s why I don’t mind going to the center, because once I’m out of there, nobody knows I have dialysis.” (ICHD)
	“I’m still young. I don’t necessarily want to walk around with the tube hanging and people asking questions.” (HHD)
Return to clinic when device not working	“Then, [the] device may stop working, and you have to adjust your mind to going back to in-center. That just becomes a disruption, rethinking everything, trying to get the schedule that you need versus the schedule they can give you.” (HHD)
	“[What would be of most concern is] when something breaks with the device and getting it fixed and restoring regular treatment.” (PD)
	“…if [the device stops working], you may need to go to another form of dialysis for a period of time. That one could be tough. Like I said, it’s a big change, a big risk going into the clinic if that’s the one that you have to do.” (PD)
Concerns about device effectiveness	“For me, it all depends on its effectiveness. What’s the point of it if it’s not as effective and requires more maintenance and care than what I have to do [at] my three times a week, three hours a day center?” (ICHD)
	“I would question how the machine knows how much fluid to take off. Because if you take too much, then you end up cramping … Then if you don’t take enough, you end up getting sick and possibly end up in the hospital.” (ICHD)
Concerns about clinician knowledge of the device	“I would just hope that they would have the health care professionals properly trained so that it won’t make us feel like outcasts when we have a problem [with the device].” (PD)
	“Trying to imagine dealing with a hospital situation with a wearable artificial kidney would be very, very scary because they won’t know anything about it. They’ll want to put you onto what they know, which is going to be your standard in-center thing. They won’t know how to deal with it. It would be the scariest thing I could imagine having to deal with.” (HHD)
Comparison to current modalityb
Wearable better	“It would be a huge improvement in quality of life, I wouldn’t have to worry about fatigue half the week, I wouldn’t have to worry about nausea and not being able to eat.” (ICHD)
	“I would try it right away. It would help me to get back to school. I’m keeping a job offer waiting. I mean, this would just help me to get back into those things, versus what I’m doing right now. Let’s say you've got a meeting late in the night, and you’ve got an early morning class the next day … Sometimes your schedule is just that busy, and so this [device] helps to be able to do that.” (PD)
Unsure	“I think it could possibly be better even though it has its downsides.” (ICHD)
	“It sounds like it is less painful. You don’t have to stick yourself with needles. But I think I would stick to what I am doing.” (HHD)
	“Advantages with PD and wearables are similar because you do PD at home. So, it’s similar to the wearable because your life isn’t completely revolving around treatment.” (PD)

HHD, home hemodialysis; ICHD, in-center hemodialysis; PD, peritoneal dialysis.

^aPotential benefit/harm mentioned by at least two participants. Potential benefits mentioned by one participant were better blood cleaning (HHD) and fewer supplies (HHD). Potential downsides mentioned by one participant were cost (ICHD) and always having something attached to you (PD).

^bNo participants identified their current dialysis modality as definitively better than a wearable device. No PD or HHD participants identified a wearable device as definitively better than their current dialysis modality.

Survey Instrument Construction

We selected serious bleeding and serious infection as the risks of interest on the basis of regulator input on the anticipated study end points for trials of wearable RRT devices and patient interviews revealing concern for device disconnection-related risks. After draft instrument construction, input from patient advisors and external industry experts led to instrument updates, including: (1) revision of the treatment graphics (e.g., added masks to in-center patient and health care professional, changed wearable device tubing color); (2) change in the weight of the wearable device from five to between five and ten pounds; (3) addition of information about safeguards for both treatment types; (4) specification that the device would be worn “most of the time, both day and night,” and (5) clarification of the risk denominator (i.e., risk over a year’s time).

Pretest Interview Findings

We then pretested the survey with the target population. Overall, round 1 participants (n=7) displayed good comprehension of survey content, with all responding correctly to questions about the two dialysis treatment types (i.e., wearable devices and in-center hemodialysis). Of the seven participants, six (86%) understood the pictographs depicting the proportion of people who would experience the risks (bleeding or infection) in a year. Participant responses resulted in survey refinements to increase clarity (e.g., modifications to the treatment graphics and descriptions, clarification of terminology). Round 2 participants (n=9) also displayed good comprehension of survey content, but two (22%) needed assistance understanding the comparator populations for the risk trade-off and wait-time questions. In response, we added clarifying text to the relevant instructions. Table 4 and Figure 2 summarize findings and responsive survey revisions.

Table 4.

Representative pretest interview findings and survey updates organized by select FDA-recommended qualities of PPI studies (6)

Round 1 Interview Findings (N=7)	Survey Updates	Round 2 Interview Findings (N=9)	Survey Updates
Patient centeredness
Considered risks (serious bleeding and infection) important to their RRT decision making	N/A	Consistent with round 1	N/A
Found treatment descriptions easy to understand (see minimal cognitive bias)	N/A	Consistent with round 1	N/A
Effective communication of benefit, harm, risk, and uncertainty
Displayed good understanding of the potential risks and benefits of the treatments	N/A	Displayed good understanding of the potential risks and benefits of the treatments	N/A
All but one participant displayed good understanding of the pictorial representation of riska	N/A	Displayed good understanding of the pictorial representation of risk	N/A
		Misunderstood the risk comparator to be the average person (versus other dialysis patients)	Underlined the phrase “other dialysis patients” to emphasize comparator
Minimal cognitive bias
Indicated that the in-center HD description was accurate	N/A	Consistent with round 1	N/A
Indicated that the wearable description was straightforward and understandable	N/A	Raised specific questions about the wearable (e.g., battery life, fluid storage, cleaning)	Added that the descriptions are of wearables in general and that features may vary by device
Found the wearable graphic helpful in showing the different ways could carry the device	N/A	Noted that the wearable graphic made the device look inconvenient and questioned whether they would want to use it	No change as participants understood the graphic
Comprehension by study participants
Desired more information about patient monitoring and caring for the wearable in the comparison table of wearables and in-center HD	Added information about patient monitoring and device care to the comparison table	Indicated that the comparison table was understandable and sufficient	N/A
Expressed unfamiliarity with the term “peritoneum”	Added definition of term	Expressed understanding of all terminology	N/A
		Thought waiting time might be on current dialysis modality (versus in-center HD) in the time trade-off questionb	Clarified the instructions by adding text to emphasize the assumption of in-center HD use while waiting for device B
Exhibited difficulty understanding longer, complex sentences	Shortened and simplified sentence structure.	Exhibited sufficient understanding	N/A

Based on FDA PPI guidance document (6). Some interview findings apply to more than one FDA-recommended quality (e.g., input on treatment graphics and descriptions applicable to minimal cognitive bias and comprehension by study participants). FDA, Food and Drug Administration; PPI, patient preference information; HD, hemodialysis; N/A, not applicable.

^aNo changes were made in response to this finding given understanding by all other participants and use of best practices in risk communication (e.g., use of text and pictures, absolute scales). This is consistent with FDA guidance that PPI studies “should aim to measure preferences and perspectives on benefits and risks of well-informed patients” (6).

^bTime trade-off questions were not included in round 1 interviews because they were under development at the time.

Figure 2.

Evolution of dialysis treatment graphics based on stakeholder input. The survey graphics of in-center hemodialysis, the reference treatment, and wearable RRT devices, the alternative treatment, underwent iterative stakeholder-guided revisions. Changes to the in-center hemodialysis treatment graphic included: addition of a blood line and a health care professional (B), followed by addition of a second blood line, elevation of the patients’ feet, addition of machine detail, and change in color of the blood lines from gold to red to resemble actual in-center bloodlines better (C). Changes to the wearable device graphics included: addition of a blood line, change in color of the people from black to gray, and removal of the vest-based graphic (B), followed by change in color of the people from gray to tan, conversion of the backpack style from bulky to more compact, and change in color of the blood lines from red to gold to signify the potential discreteness of the device (C).

Pilot Test Findings

Of the 24 in-center hemodialysis pilot test participants, 16 (67%) completed the survey in English, and eight (33%) completed the survey in Spanish. Of the 24 participants, 12 (50%) required assistance with survey completion. Types of assistance provided included navigating the tablet computer (e.g., advancing screens, tapping responses, scrolling), holding the tablet, and reading questions aloud for patients without their glasses or with severe visual impairment. The mean±SD time to completion was 40±18 minutes. Field notes suggested that survey completion during in-center hemodialysis was feasible for most patients, but some individuals, especially those with limited computer experience, required assistance.

Final Survey Instrument

The final survey is a 54-item, web-based instrument that includes (1) risk trade-off questions designed to quantify the levels of potential risks of serious bleeding and serious infection that patients are willing to accept in exchange for the benefits of the wearable RRT devices; (2) modified time trade-off questions to determine respondents’ discount rate for time until wearable devices are available; (3) comprehension questions to assess understanding of the presented information; and (4) background questions (Figures 3 and 4; Supplemental Material). The survey will yield estimates of the maximal acceptable risk for the wearable device described and willingness to wait for wearable devices with lower risk in people living with kidney failure.

Figure 3.

Survey content overview. The final survey is a 54-item web-based instrument that includes (1) risk trade-off questions designed to quantify the levels of potential risks of serious bleeding and serious infection that patients are willing to accept in exchange for the benefits of the wearable RRT devices; (2) modified time trade-off questions to determine respondents’ discount rate for time until wearable RRT devices are available; (3) comprehension questions to assess understanding of the presented information; and (4) health and background information questions The full survey is available in the Supplemental Material.

Figure 4.

Example of a treatment choice question in the question series for eliciting maximum risk of serious bleeding patients would accept from a wearable RRT device in exchange for the benefits of the device relative to in-center hemodialysis. To estimate the maximum acceptable risk for different devices, the survey includes two sets of three risk trade-off questions, where respondents must choose between pairs of treatments: fixed reference treatment (in-center hemodialysis) and the alternative wearable RRT device as risk levels are varied. In the first set of risk trade-off questions, the respondent must choose between pairs of treatment that differ in terms of the risk of serious bleeding (as shown in the figure). In the second set of risk trade-off questions, the respondent must choose between pairs of treatment that differ in terms of the risk of serious infection (not shown). For each respondent, the survey generates a range in which the respondent’s maximum acceptable risk for switching from the reference treatment to the alternative treatment. The data from the threshold technique portion of the survey will be analyzed using an interval regression model. The coefficients from this model will allow us to determine the average maximum acceptable risk for the sample and how that maximum acceptable risk varies by patient characteristics.

Future Survey Fielding

The purposes of survey fielding are to assess the risk tolerance of patients for hypothetical wearable RRT devices and to demonstrate the feasibility of administration of a PPI survey to people receiving dialysis. Survey respondents will be adults with dialysis-dependent kidney failure who may be eligible and interested in enrolling in a clinical trial of a wearable RRT device. The survey will be fielded via partnerships with patient and dialysis organizations. On the basis of our pilot test experiences, we will use both self- and research team-assisted administration approaches, permitting survey completion during dialysis treatment if preferred by the patient. Given that patient interviews suggested potential preference heterogeneity based on current dialysis modality, we will aim for a sample size to support subgroup analyses among patients using in-center hemodialysis, home hemodialysis, and peritoneal dialysis. Where sample size is sufficient, we will also examine the influence of other respondent characteristics (e.g., age, time on dialysis) on maximal acceptable risk.

Discussion

We described the development of a PPI survey that captures patient preferences for wearable RRT devices in comparison to in-center hemodialysis based on the potential benefits of the treatments and their potential risks of serious bleeding and serious infection. We engaged diverse stakeholders throughout survey development and followed best practices in preference science to maximize the validity and scientific rigor of our final instrument. We intend for the survey and our development approach to serve as models in future endeavors to capture regulatory-grade PPI for other innovative RRT technologies.

According to FDA guidance, PPI can be useful in evaluating the risk/benefit profiles of medical devices in the setting of “preference sensitive” patient decisions in which (1) there are multiple treatment options with no clearly superior option for all patients, (2) evidence supporting one treatment option over another is uncertain, and (3) patients’ perspectives on the benefits and risks of a device vary within a population or differ from those of health care professionals (6). Our PPI survey addresses a “preference-sensitive” decision because there is no RRT treatment option that is clearly superior for all patients, and patient views on RRT benefits and risks vary (23–26). Although submission of PPI to the FDA is voluntary, such data may strengthen the FDA’s ability to identify important patient-perceived benefits and risks of devices, assess how patients trade off benefits and risks of devices, and also understand heterogeneity in patient preferences (6). For example, we anticipate that findings from our survey may identify patient subpopulations with higher tolerance for wearable device risks (e.g., an individual’s current RRT modality may influence their perceptions of risks). This highlights the importance of targeting a survey sample size large enough to support modality-based subgroup analyses.

Patient preference surveys using the threshold technique typically present two treatment options and two to three potential treatment risks (15,17). Although inclusion of additional risks may be of scientific interest, it greatly increases respondent burden. Our survey thus considers two treatments, a wearable device and in-center hemodialysis, and systematically alters two potential risks, serious bleeding and serious infection, selected based on our interviews with the target population and input from regulators identifying these risks as key safety end points for clinical trials of wearable devices. The risk level at which respondents “switch” to the alternative treatment indicates the respondents’ relative strength of preference and can be used in decision analyses and clinical trial design (27–30).

We acknowledge that the risks of serious bleeding and serious infection do not represent the totality of potential risk related to wearable devices. For example, frequent clotting poses a challenge to hemodialysis-based wearable functionality such that it may require significant amounts of anticoagulation to maintain pump function. Preference surveys capturing patient perspectives on the risks of blood loss from frequent system clotting versus the risks of bleeding from anticoagulation-related complications may be important. Moreover, because detailed information about wearable devices is not yet available, our survey describes “hypothesized” peritoneal dialysis- and hemodialysis-based devices, and assumptions about their features, safeguards, and potential benefits. As such, we used composite risk estimates for bleeding and infection, yielding an average of wearable peritoneal dialysis- and hemodialysis-related risk. We anticipate that future PPI surveys for RRT innovations will be device specific, supporting greater precision in assessing patient risk/benefit trade-offs. Our developed survey will hopefully serve as a model for such future efforts.

Strengths of our study include involvement of diverse stakeholders, use of purposive sampling to capture perspectives from heterogeneous patients, and adherence to best practices in preference science. Limitations relate to the lack of a specific wearable RRT device on which to focus the survey and absence of published data on wearable device risks of serious bleeding and serious infection. In addition, although we sought to represent the population as best as possible, the nature of our survey could preclude its applicability to all people treated with dialysis. We acknowledge that the risk and time trade-off questions are hypothetical and require abstract thought, which could make it difficult for some individuals to respond to the survey. Related, we selected a web-based format to support future computerized adaptive testing for varied risk and wait-time thresholds, potentially limiting survey accessibility to some patients. Similarly, the survey length could be a deterrent to some respondents. However, our pilot test showed that patients were able to complete the full survey and that most patients could complete it electronically when technology-related (not content) assistance was provided. In addition, our approach is consistent with FDA guidance to “measure preferences and perspectives on benefits and risks of well-informed patients.”

In conclusion, we described the stakeholder-engaged process of developing a PPI survey for wearable RRT devices. The next step is to assess the risk tolerance of patients for hypothetical wearable devices and to demonstrate the feasibility of administration of a PPI survey to people receiving dialysis by administering the survey to its target population.

Disclosures

K.L. Cavanaugh reports consultancy for the Kidney Health Initiative, REATA Pharmaceuticals, and Responsum Health; ownership interest in HCA Healthcare; and an advisory or leadership role for the National Kidney Foundation (KDQOI education committee), Clinical Journal of American Society of Nephrology (editorial board), American Journal of Kidney Diseases (editorial board), Kidney360 (associate editor), and Medical Decision Making (editorial board). J.E. Flythe reports consultancy for AstraZeneca and Fresenius Medical Care Medical Advisory Board; research funding from National Institutes of Health/National Heart, Lung, and Blood Institute, National Institutes of Health/National Institute of Diabetes and Digestive and Kidney Diseases, Patient-Centered Outcomes Research Institute, Renal Research Institute (subsidiary of Fresenius Medical Care, North America), and Robert Wood Johnson Foundation; honoraria from the American Society of Nephrology, National Kidney Foundation, and numerous universities; and an advisory or leadership role for the American Journal of Kidney Diseases (editorial board 2017–2021), Clinical Journal of American Society of Nephrology (editorial board 2017–), Hemodialysis (theme editor 2018–), KDIGO Executive Committee (2020–), Kidney Health Initiative (board of directors 2019–), Kidney Health Initiative (Patient Preferences Project chairperson 2019–), Kidney Medicine (editorial board 2019–), Kidney360 (associate editor 2019–), and Nephrology Dialysis and Transplantation (editorial board). D. Forfang reports employment by the American Society of Nephrology Kidney Health Initiative; consultancy for Ardelyx, Inc., Scientific Advisory Board, the American Society of Nephrology and Responsum, CareDX, HSAG, and University of North Carolina Kidney Center; honoraria from the American Society of Nephrology, European Association for Dialysis, HSAG ESRD Network #17, National Kidney Foundation; an advisory or leadership role for Arbor Research, HSAG ESRD Network #17 (board member), Kidney Health Initiative (Patient Advisory Committee), National Forum of ESRD Networks (board member), National Forum of ESRD Networks Kidney Patient Advisory Council (chair); National Kidney Foundation, SONG Group, and Unity Health Toronto OPPUS, UCSF Kidney Project (patient advisor); and other interests or relationships as a volunteer for the Forum of ESRD Networks as Kidney Patient Advisory Council chair and board member, a volunteer for ESRD Network #17 as Patient Advisory Committee Chair and Network board member, a volunteer for the National Kidney Foundation as a member of their Public Policy Committee, and a volunteer for the National Kidney Foundation as a Regional Leader of their Kidney Advocacy Committee, Kidney Health Initiative PFPC member. N. Gedney reports honoraria from the American Society of Nephrology, IDEAs, and the University of Washington. R.C. Harris reports consultancy for Bayer, Fibrocor, and Nicoya; ownership interest in Apple; research funding from Bayer; patents or royalties from eNOS db/db mouse; an advisory or leadership role for the Kidney Health Initiative (paid); and other interests or relationships with the Kidney Health Initiative (co-chair, board of directors). F.P. Hurst reports employment by the Food and Drug Administration/Center for Drug Evaluation and Research, and other interests or relationships with a US government employee. C. Mansfield reports the employment by RTI Health Solutions, which receives funding from for-profit pharmaceutical companies to conduct research. C.Y. Neuland reports being a member of the board of directors for the Kidney Health Initiative, representing the Food and Drug Administration/Center for Devices and Radiological Health during this time period. A. Saha reports employment by the Food and Drug Administration. C.S. Soloe reports employment by RTI International. G. Squillaci reports consultancy for The Griff Group. M.E. Tarver reports employment by the Food and Drug Administration/Center for Devices and Radiological Health. K. Treiman reports employment by RTI International. M.L. Unruh reports consultancy for Cara Therapeutics to chair of Data Monitoring Committee; a consulting agreement between Cara and the University of New Mexico; research funding from DCI; and honoraria related to lectures from the American Society of Nephrology, National Kidney Foundation, and Renal Research Institute. M. West reports employment by the American Society of Nephrology. D.M. White reports employment by Debevoise & Plimpton; consultancy for the Kidney Transplant Collaborative, the National Committee for Quality Assurance, and Responsum Health; ownership interest in Amgen, Inc.; honoraria from AstraZeneca, Hennepin Healthcare, the National Institute of Diabetes and Digestive and Kidney Diseases, the National Institutes of Health, New York Academy of Medicine, and PFCC partners; and other interests or relationships with the American Association of Kidney Patients, American Society of Transplantation, Kidney Health Initiative, National Kidney Foundation, National Patient Advocate Foundation, Patient Advocate Foundation, and Quality Insights. C. Wilkie reports employment by Kuhns Law Firm, PLLC; consultancy for the University of North Carolina, University of Pennsylvania, and University of Pittsburg; ownership interest in Kuhns Law Firm, PLLC; an advisory or leadership role for the Kidney Health Initiative, University of Pennsylvania with the HOPE Consortium, and University of Pittsburgh; and other interests or relationships with the Kidney Health Initiative, National Kidney Foundation, University of Pennsylvania with the HOPE Consortium, University of North Carolina at Chapel Hill, and University of Pittsburgh. D. Wood reports employment by RTI International. The remaining authors have nothing to disclose.

Funding

This work was supported by US Food and Drug Administration Broad Agency Agreement award 75F40119C10124; BAA-HDDISWP#88. This work was also supported by the Kidney Health Initiative (KHI), a public/private partnership between the American Society of Nephrology, the US Food and Drug Administration, and more than 100 member organizations and companies to enhance patient safety and foster innovation in kidney disease. KHI funds were used to defray costs incurred during the conduct of the project, including project management support that was expertly provided by American Society of Nephrology personnel. The authors of this paper had final review authority and are fully responsible for its content. KHI makes every effort to avoid actual, potential, or perceived conflicts of interest that may arise as a result of industry relationships or personal interests among the members of the workgroup. More information on KHI, the workgroup, or the conflict-of-interest policy can be found at www.kidneyhealthinitiative.org. K.L. Cavanaugh reports research support from the National Institutes of Health via P30DK114809.

Author Contributions

K.L. Cavanaugh, J.E. Flythe, D. Forfang, N. Gedney, R.C. Harris, F.P. Hurst, C. Mansfield, C.Y. Neuland, A. Saha, M. Sheldon, C.S. Soloe, G. Squillaci, M.E. Tarver, K. Treiman, M.L. Unruh, M. West, D.M. White, C. Wilkie, and D. Wood reviewed and edited the manuscript; K.L. Cavanaugh, J.E. Flythe, D. Forfang, N. Gedney, R.C. Harris, F.P. Hurst, C. Mansfield, C.Y. Neuland, A. Saha, M. Sheldon, G. Squillaci, M.E. Tarver, K. Treiman, M.L. Unruh, M. West, D.M. White, and C. Wilkie were responsible for conceptualization; J.E. Flythe was responsible for visualization and wrote the original draft of the manuscript; J.E. Flythe, R.C. Harris, C. Mansfield, G. Squillaci, and M. West were responsible for funding acquisition; J.E. Flythe, C. Mansfield, M. Sheldon, and M.E. Tarver were responsible for supervision; J.E. Flythe, C. Mansfield, C.S. Soloe, K. Treiman, and D. Wood were responsible for the methodology; J.E. Flythe, C. Mansfield, and K. Treiman were responsible for the investigation; J.E. Flythe, G. Squillaci, K. Treiman, M. West, and D. Wood were responsible for project administration; C. Mansfield, C.S. Soloe, K. Treiman, D. Wood were responsible for formal analysis; C.S. Soloe, K. Treiman, and D. Wood were responsible for data curation; K. Treiman was responsible for resources; K. Treiman and D. Wood were responsible for software and validation.

Supplemental Material

This article contains the following supplemental material online at http://kidney360.asnjournals.org/lookup/suppl/doi:10.34067/KID.0001862022/-/DCSupplemental

Supplemental Table 1

Steering committee members. Download Supplemental Table 1, PDF file, 229 KB ^{(1.9MB, pdf)}

Supplemental Table 2

Environmental scan of RRT research and development activities. Download Supplemental Table 2, PDF file, 229 KB ^{(1.9MB, pdf)}

Supplemental Table 3

Representative concept elicitation and pretest interview questions. Download Supplemental Table 3, PDF file, 229 KB ^{(1.9MB, pdf)}

Supplemental Table 4

Survey risk levels and supporting evidence. Download Supplemental Table 4, PDF file, 229 KB ^{(1.9MB, pdf)}

Final patient preference survey.