Abstract
Shared decision-making about clinical care options in end-stage kidney disease is limited by inconsistencies in the reporting of outcomes and the omission of patient-important outcomes in trials. Here we generated a consensus-based prioritized list of outcomes to be reported during trials in peritoneal dialysis. In an international, online, three-round Delphi survey, patients/caregivers and health professionals rated the importance of outcomes using a 9-point Likert scale (with 7-9 indicating critical importance) and provided comments. Using a Best-Worst Scale, the relative importance of outcomes was estimated. Comments were analyzed thematically. In total, 873 participants (207 patients/caregivers and 666 health professionals) from 68 countries completed round one, 629 completed round two and 530 completed round three. The top outcomes were peritoneal dialysis-related infection, membrane function, peritoneal dialysis failure, cardiovascular disease, death, catheter complications, and the ability to do usual activities. Compared with health professionals, patients/caregivers gave higher priority to six outcomes: blood pressure (mean difference, 0.4), fatigue (0.3), membrane function (0.3), impact on family/friends (0.1), peritoneal thickening (0.1) and usual activities (0.1). Four themes were identified that underpinned the reasons for ratings: contributing to treatment longevity, preserving quality of life, escalating morbidity, and irrelevant and futile information and treatment. Patients/caregivers and health professionals gave highest priority to clinical outcomes. In contrast to health professionals, patients/caregivers gave higher priority to lifestyle-related outcomes including the impact on family/friends and usual activities. Thus, prioritization will inform a core outcome set to improve the consistency and relevance of outcomes for trials in peritoneal dialysis.
Keywords: Peritoneal dialysis, kidney disease, core outcome sets, outcomes, patient-centered care, trials
INTRODUCTION
Peritoneal dialysis (PD) may be a preferred modality for some patients as it can offer more autonomy and flexibility compared to centre-based hemodialysis (HD).1 However, patients and clinicians face major challenges including technique failure, infection and access problems. Clinical trials which aim to address these challenges have implications for patient survival, comorbidities, psychosocial status and day-to-day physical functioning.2–4 As such, evidence-informed, shared decision-making is highly pertinent in the context of PD, and can only occur if trials consistently report outcomes that are meaningful to patients, their families and health professionals.
The last three decades have seen a number of randomized trials of PD interventions evaluating PD solutions, catheters, systems, solute clearance and anti-infective agents.5–8 However, the outcomes reported across trials are heterogeneous, which limits the ability to compare the effect of interventions across trials, and are often mostly biochemical endpoints with patient-reported outcomes absent.9,10
The problems with reporting of outcomes in trials are well recognized in nephrology and other disciplines, which has driven the development of core outcome sets, defined as an “agreed minimum set of outcomes to be reported in all trials”.11,12 As part of the Standardized Outcomes in Nephrology – Peritoneal Dialysis (SONG-PD) initiative13, this study aimed to generate consensus among patients, caregivers and health professionals on critically important outcomes for trials in PD. This will be used to establish a core outcome set that reflects the shared priorities of these stakeholders, thereby enhancing the usability and uptake of trial evidence in shared decision-making about clinical care options.
METHODS
Study design
The Delphi survey is a validated technique that has been used for developing consensus on core outcomes for clinical trials across a variety of health disciplines.14–17 The survey was conducted online and involved three iterative rounds completed by a panel of participants with experience or expertise in PD. In the second and third round, participants were able to see their previous survey scores, the distribution of the group results and comments provided by participants. The SONG-PD Delphi process is shown in Supplementary Figure 1.
Participant selection and recruitment
Patients, caregivers and health professionals with an interest or experience in PD were eligible. Patients/caregivers included patients currently or previously receiving PD, with chronic kidney disease (CKD) stages 1-5, transplant recipients and family members or friends. Health professionals included physicians, nurses, allied health professionals, researchers, policy makers, regulators and industry.
We used multiple recruitment strategies to capture a diverse range of participants. Patients were recruited from hospitals, patient/consumer organizations, the SONG database and social media (Supplementary Figure 2). Health professionals were recruited through the SONG database, investigator networks and professional organizations. Participants received an email invitation after registering their email on the SONG website (www.songinitiative.org). Ethics approval was provided by the University of Sydney (2015-228) and participating institutions (Supplementary Table 1).
Data collection
Selection of outcome domains:
The outcome domains included in the survey were identified from a systematic review of outcomes reported in PD trials, and a focus group study with nominal group technique, in which 126 patients and caregivers identified and prioritized outcomes in PD on their own terms.18 This ensured that patient-important outcomes (e.g. flexibility with time, impact on family/friends, usual activities) were included in the Delphi survey. Each outcome included a plain language definition (Supplementary Table S2) and the order of outcomes was randomized. The SONG-PD Steering Group and investigators reviewed the list of outcomes, and the survey was piloted among 12 health professionals at the British Renal Society conference in April 2017. The survey was administered online using Qualtrics (Qualtrics software, Provo, UT, United States) from August 2017 to April 2018 to enable wider dissemination and to minimize errors in data transfer.
Round 1:
Participants rated the importance of each of the 39 outcome domains using a 9-point Likert scale. Scores 1-3 indicated “limited importance”, 4-6 indicated “important but not critical” and 7-9 indicated “critical importance”, based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) process.19 An option of “unsure” was provided. Participants could enter comments for each outcome in free-text boxes and could suggest new outcomes. New outcomes suggested by more than 10% of participants were eligible to be included in the next round. Outcomes with a mean score of less than 6.5 or median less than 7 in both groups were excluded from Round two.
Round 2:
Round two had 23 outcomes. Participants were shown their own scores from Round one and reviewed the distribution of scores by patients/caregivers, health professionals and the total sample combined displayed in a column graph. Instructions on how to read the graph were provided. Participants could read de-identified comments provided in Round one, which were grouped by patients/caregivers and health professionals. Participants re-rated the outcomes using the 9-point Likert scale and could again enter comments. Outcomes with a mean or median less than 7 were excluded from Round three.
Round 3:
Round three had 16 outcomes, which participants rated again using the 9-point Likert scale and provided comments after reviewing the scores and comments from Round two. To assess the importance of the outcomes relative to each other, participants also completed a Best-Worst Scale (BWS) Survey. The BWS survey is a preference elicitation method that involves less cognitive burden and provides better discrimination between outcomes, and greater insight into preferences of respondents compared to rating scales.20 Participants were presented with five best-worst choice sets each consisting of six of the 16 outcomes. The outcomes included in each set were determined using a balanced, incomplete block design. For each block of outcomes, participants selected the most important and least important.
Data analysis
Quantitative analysis:
We calculated the mean score, median score and proportion of participants who rated the outcome as critically important (from 7 to 9) for each outcome and for each round. We calculated the scores separately for patients/caregivers and health professionals, and compared the two groups using a Mann-Whitney U test or t-test, depending on distribution of the data. Given the observed distributions, only the t-test was applicable. The relative importance was determined using a multinomial logistic regression model. Utility functions containing all outcomes and interaction terms for participant characteristics were constructed for the Best-Worst choice task. Following this approach, the mean regression coefficients of this function provided the relative importance scores for each outcome.20 As the regression coefficients have the same underlying scale, preference scores were able to be adjusted to any convenient scale. In this survey, a scale of 1 (least important) to 9 (most important) was used. Statistical analyses were undertaken using SPSS (IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY), Excel (Microsoft Corporation, Product version 16.0), and NLOGIT V6 (Econometric Software Inc.) for the BWS. A p-value of less than 0.05 was considered statistically significant.
Definition of consensus for core outcomes:
The criteria for consensus for the core outcome domains could not be defined a priori because the distribution of scores was unknown. We sought to identify the top 3-5 outcome domains indicated as critically important by both stakeholder groups on the Likert scale. “Consensus” for the critical outcome domains was defined based on both patient/caregiver and health professional groups yielding median scores ≥8 and mean scores >8, as well as the proportions of both stakeholder groups rating the outcome as ‘critically important’ being greater than 85%. These thresholds were discussed and approved by the SONG-PD Steering Group. The scores obtained from the BWS were used to examine relative differences in preference scores between patients/caregivers and health professionals.
Qualitative analysis:
The survey comments were imported into HyperRESEARCH (Version 3.7, Randolph, MA, United States) software for data analysis. Using thematic analysis, investigator (KEM) coded the text and inductively identified themes focusing on reasons for ratings, differences between stakeholder groups and changes in ratings across rounds. A second investigator (AT) read the qualitative data and reviewed the preliminary analysis to ensure that the themes captured all the data.
RESULTS
Participant characteristics
In Round one, 873 participants from 68 different countries completed the survey, of whom 666 (76%) were health professionals and 207 (24%) were patients/caregivers. In Round two, 629 (72% overall retention rate) participants from 63 countries completed the survey, of whom 469 (75% [70% retention]) were health professionals and 160 (25% [77% retention]) were patients/caregivers. In Round three, 530 (61% overall retention rate) participants from 59 countries completed the survey, of whom 390 (74% [59% retention]) were health professionals and 140 (26% [68% retention]) were patients/caregivers. Participant characteristics are provided in Tables 1 and 2.
Table 1.
Characteristics of patients/caregivers.
| Characteristic | Round 1, n (%) 207 participants |
Round 2, n (%) 160 participants |
Round 3, n (%) 140 participants |
|---|---|---|---|
| Participant typea | |||
| Patient | 177 (83) | 134 (82) | 117 (81) |
| Caregiver/family member | 36 (17) | 30 (18) | 27 (19) |
| Gender | |||
| Female | 115 (56) | 86 (54) | 75 (54) |
| Male | 92 (44) | 74 (46) | 65 (46) |
| Age group (years) | |||
| 18 – 40 | 40 (19) | 29 (18) | 21 (15) |
| 41 – 50 | 47 (23) | 35 (22) | 27 (19) |
| 51 – 60 | 55 (27) | 43 (27) | 40 (29) |
| 61 – 70 | 45 (22) | 36 (23) | 35 (25) |
| > 70 | 20 (10) | 17 (11) | 17 (12) |
| Marital statusb | |||
| Married | 123 (62) | 101 (66) | 88 (66) |
| Partner/de-facto | 15 (7.5) | 8 (5) | 7 (5) |
| Single | 35 (18) | 23 (15) | 21 (16) |
| Divorced/separated/widowed | 26 (13) | 21 (14) | 18 (13) |
| Employment statusb | |||
| Employed full-time | 65 (33) | 53 (35) | 45 (34) |
| Employed part-time/casual | 37 (19) | 28 (19) | 22 (17) |
| Unemployed | 20 (10) | 10 (7) | 7 (5) |
| Retired | 56 (29) | 44 (29) | 44 (34) |
| Student or other | 18 (9) | 15 (10) | 13 (10) |
| Educationb | |||
| Did not complete high school | 27 (14) | 22 (14) | 22 (17) |
| High school graduate | 33 (17) | 24 (16) | 21 (16) |
| Professional certificate | 44 (22) | 33 (22) | 28 (21) |
| Undergraduate degree | 57 (29) | 42 (28) | 34 (26) |
| Postgraduate degree | 37 (19) | 31 (20) | 28 (21) |
| Current type of treatment (patients only) | |||
| Peritoneal dialysis | 89 (50) | 68 (51) | 57 (49) |
| Kidney transplant | 56 (32) | 46 (34) | 42 (36) |
| Hemodialysis | 20 (11) | 13 (10) | 11 (9) |
| No renal replacement therapy | 12 (7) | 7 (5) | 7 (6) |
| Years on peritoneal dialysis (PD patients only) | |||
| < 1 | 38 (43) | 33 (49) | 27 (47) |
| 1-3 | 28 (31) | 19 (28) | 15 (26) |
| 3-6 | 15 (17) | 11 (16) | 10 (18) |
| > 6 | 8 (9) | 5 (7) | 5 (9) |
| Country* | |||
| United States | 91 (44) | 63 (39) | 55 (39) |
| Australia | 35 (17) | 30 (19) | 24 (17) |
| United Kingdom | 29 (14) | 24 (15) | 24 (17) |
| Hong Kong | 15 (7) | 13 (8) | 11 (8) |
| Canada | 13 (6) | 10 (6) | 9 (6) |
| Other* | 24 (12) | 20 (13) | 17 (12) |
Some have multiple roles;
N ≠ 207, 160 and 140 for rounds 1, 2 and 3, respectively, due to missing data;
Other includes 10 countries (in descending order of number of participants): New Zealand, China, Denmark, Brazil, Italy, Spain, Germany, Ireland, India and Nigeria.
Table 2.
Characteristics of health professionals.
| Characteristic | Round 1, n (%) 666 participants |
Round 2, n (%) 469 participants |
Round 3, n (%) 390 participants |
|---|---|---|---|
| Participant rolea | |||
| Nephrologist | 390 (53) | 293 (55) | 249 (57) |
| Nurse | 160 (22) | 99 (19) | 72 (16) |
| Researcher | 60 (8) | 51 (10) | 48 (11) |
| Industry | 29 (4) | 20 (4) | 16 (4) |
| Dietician | 25 (3) | 24 (5) | 19 (4) |
| Social Worker | 12 (2) | 3 (1) | 3 (1) |
| Surgeon | 10 (1) | 8 (2) | 6 (1) |
| Psychologist | 9 (1) | 6 (1) | 3 (1) |
| Policy maker | 9 (1) | 6 (1) | 6 (1) |
| Pharmacist | 7 (1) | 5 (1) | 4 (1) |
| Psychiatrist | 1 (0) | 0 (0) | 0 (0) |
| Others | 23 (3) | 16 (3) | 14 (3) |
| Gender | |||
| Female | 418 (63) | 282 (60) | 225 (58) |
| Male | 248 (37) | 187 (40) | 165 (42) |
| Age group (years) | |||
| 18 - 40 | 261 (39) | 147 (31) | 106 (27) |
| 41 – 50 | 189 (28) | 140 (30) | 121 (31) |
| 51 – 60 | 165 (25) | 135 (29) | 121 (31) |
| 61 – 70 | 48 (7) | 44 (9) | 39 (10) |
| > 70 | 3 (0) | 3 (1) | 3 (1) |
| Experience in peritoneal dialysis (years)b | |||
| ≤10 | 303 (47) | 183 (40) | 142 (37) |
| 11-20 | 203 (31) | 158 (34) | 132 (35) |
| > 20 | 140 (22) | 117 (26) | 106 (28) |
| No. of peritoneal dialysis trials as investigatorsb | |||
| 0 | 303 (50) | 240 (55) | 200 (55) |
| 1-5 | 223 (37) | 149 (34) | 122 (34) |
| 6-10 | 25 (4) | 18 (4) | 15 (4) |
| 11-15 | 17 (3) | 10 (2) | 10 (3) |
| >15 | 42 (7) | 18 (4) | 16 (4) |
| Specific role in peritoneal dialysis researcha,b | |||
| Clinical practice guidelines | 232 (51) | 143 (49) | 118 (49) |
| Government, policy making | 45 (10) | 29 (10) | 27 (11) |
| Funding (grant review, charity) | 19 (4) | 10 (3) | 9 (4) |
| Other | 158 (35) | 111 (38) | 85 (36) |
| Country* | |||
| China | 148 (22) | 50 (11) | 30 (8) |
| United states | 79 (12) | 60 (13) | 52 (13) |
| Australia | 74 (11) | 67 (14) | 59 (15) |
| Hong Kong | 59 (9) | 37 (8) | 31 (8) |
| United Kingdom | 57 (9) | 50 (11) | 42 (11) |
| Canada | 53 (8) | 42 (9) | 38 (10) |
| New Zealand | 18 (3) | 15 (3) | 14 (4) |
| India | 15 (2) | 10 (2) | 7 (2) |
| Other* | 163 (24) | 138 (29) | 117 (30) |
Some have multiple roles;
N ≠ 666, 469 and 390 for rounds 1, 2 and 3, respectively, due to missing data;
Other includes 59 countries (in descending order of number of participants): Germany, Brazil, Belgium, Portugal, Peru, Argentina, Netherlands, Mexico, South Africa, France, Italy, Greece, Switzerland, Spain, Thailand, Philippines, Malaysia, Romania, Saudi Arabia, Egypt, Finland, Serbia, Singapore, Japan, Denmark, Nigeria, Republic of Korea, Turkey, Austria, Belarus, Bosnia and Herzegovina, Pakistan, Guatemala, Uruguay, Croatia, Czech Republic, Poland, Syrian Arab Republic, Vietnam, Sweden, Slovakia, Bolivia, Iraq, Montenegro, Lithuania, Ecuador, Jordan, Indonesia, United Arab Emirates, Chad, Morocco, Paraguay, Bulgaria, Costa Rica, Mozambique, Estonia, Slovenia, Colombia, and El Salvador.
Of the 390 health professionals who completed all three rounds, 249 (57%) were nephrologists, 72 (16%) were nurses and 48 (11%) were researchers (total N >390 due to multiple roles). Psychologists, social workers, surgeons, dieticians, pharmacists, policy makers and industry representatives also participated. Health professionals were from 58 countries, including Australia (59, 15%), the United States (52, 13%), the United Kingdom (42, 11%) and Canada (38, 10%). Of the 140 patients/caregivers who completed all three rounds, 57 (40%) were patients on PD, 42 (29%) were kidney transplant recipients, 11 (8%) were on hemodialysis, 7 (5%) were patients not receiving renal replacement therapy and 27 (19%) were caregivers/family members (total N >140 due to multiple roles). The patients/caregivers were from 11 countries, including the United States (55, 39%), the United Kingdom (24, 17%), Australia (24, 17%) and Hong Kong (11, 8%).
Delphi scores
Round 1:
The mean and median scores and the proportion of participants scoring the outcomes from 7-9 for each of the 39 outcome domains in Round one are provided in Supplementary Table S3. The top five outcomes with the highest mean score for patients/caregivers were PD-related infection (mean 8.1), death (8.0), membrane function (7.9), PD failure (7.8) and cardiovascular disease (7.7). For health professionals, the top five outcomes were PD-related infection (8.3), death (7.9), PD failure (7.8), cardiovascular disease (7.8) and catheter complications (7.5). Sixteen outcomes had mean scores less than 6.5 or median scores less than 7 among both groups and were excluded from Round two (Supplementary Table S3). None of the new outcomes were suggested by more than 10% of the participants (Supplementary Table S4) and were therefore not included in the next round.
Round 2:
The mean and median scores and the proportion of participants scoring the outcomes from 7-9 for each of the 23 outcome domains in Round two are provided in Supplementary Table S5. The top five outcomes with the highest mean scores for patients/caregivers were PD-related infection (8.3), membrane function (8.1), PD failure (7.9), death (7.8) and cardiovascular disease (7.8). For health professionals, the highest five were PD-related infection (8.6), death (8.2), cardiovascular disease (8.1), PD failure (8.1) and catheter complications (8.0). Seven outcomes were excluded from Round three as they had a mean or median score of less than 7 (Supplementary Table S5).
Round 3:
For each of the 16 outcome domains in Round three, the mean score, median score and proportion of participants scoring the outcome as ‘critically important’ are shown in Supplementary Table S6. The top five outcomes with the highest mean scores for patients/caregivers were membrane function (8.6), PD-related infection (8.6), PD failure (8.3), cardiovascular disease (8.3) and death (8.2). The top five outcomes for health professionals were PD-related infection (8.8), cardiovascular disease (8.5), PD failure (8.5), death (8.4) and catheter complications (8.4).
Changes in scores from Round 1 to 3
The changes in mean scores from Rounds one to three are presented in Figures 1 and 2. Patient/caregiver mean scores increased significantly between Rounds one and three for the following 14 outcomes: usual activities (mean score difference 0.83, p<0.001), impact on family/friends (0.69, p<0.001), membrane function (0.68, p<0.001), fatigue (0.68, p<0.001), cardiovascular disease (0.62, p<0.001), catheter complications (0.60, p<0.001), residual renal function (0.55, p=0.002), peritoneal thickening (0.55, p=0.004), depression (0.55, p=0.004), PD failure (0.54, p<0.001), PD-related infection (0.50, p<0.001), hospitalization (0.45, p=0.011), fluid (0.40, p=0.026) and blood pressure (0.35, p=0.016). For health professionals, mean scores increased significantly for all of the 16 outcomes from Rounds one to three, with the greatest increases in scores being for fatigue (mean score difference 1.04, p<0.001), catheter complications (0.96, p<0.001) and usual activities (0.91, p<0.001).
Figure 1.
Mean scores of patients/caregivers in rounds 1-3.
Figure 2.
Mean scores of health professionals in rounds 1-3.
Differences between stakeholder groups
Differences in mean scores between patients/caregivers and health professionals are shown in Figure 3. In Round three, health professionals rated the following 6 outcomes significantly higher than patients/caregivers on the Likert scale: fluid (absolute mean difference 0.54, p<0.001), residual renal function (0.30, p=0.027), hospitalization (0.28, p=0.037), catheter complications (0.28, p=0.022), death (0.28, p=0.043) and PD-related infection (0.23, p=0.007). In comparison, patients/caregivers rated 3 outcomes significantly higher on the Likert scale: blood pressure (absolute mean difference 0.43, p<0.001), fatigue (0.31, p=0.007) and membrane function (0.29, p<0.001).
Figure 3.
Difference in mean scores between patients/caregivers and health professionals for rounds 1, 2 and 3. Error bars refer to 95% confidence interval.
Best-Worst Scale
Results from the BWS survey are shown in Figure 4 and Supplementary Table 6. Patients and caregivers considered PD-related infection the most important outcome, followed by membrane function, PD failure, cardiovascular disease and death. However, the differences between these top five ranked outcomes were small, ranging from 6.3 (95%CI 5.8, 6.8) for PD-related infection to 6.0 (95%CI 5.4, 6.5) for death. Health professionals considered PD-related infection most important, followed by death, PD failure, cardiovascular disease and catheter complications. In contrast to patients/caregivers, the differences between the top ranked outcomes were comparatively large, ranging from 9.0 (95%CI 8.7, 9.3) for infection to 5.7 (95%CI 5.4, 6.0) for catheter complications. The most important patient-reported outcomes for patients/caregivers were depression (3.4; 95%CI 2.8, 3.9) and usual activities (3.3; 95%CI 2.8, 3.8), and for health professionals were usual activities (5.1; 95%CI 4.8. 5.4) and depression (4.9; 95%CI 4.6, 5.2).
Figure 4.
Mean relative importance scores of patients/caregivers and health professionals based on the Best-Worst Scale. Ordered by the mean importance scores of patients/caregivers (bars with 95% CI).
Themes from comments
We identified four themes, which reflected the reasons, changes and differences in the rating of outcomes (contributing to treatment longevity, preserving quality of life, escalating morbidity, and irrelevant and futile information and treatment). The subthemes are described below and reflect the perspectives of both patients/caregivers and health professionals unless otherwise specified. Illustrative quotations for each theme are provided in Table 3.
Table 3.
Selected illustrative quotations
| Theme | Illustrative Quotations |
|---|---|
| Contributing to treatment longevity | |
| Making or breaking treatment | Paramount significance, as it can make or
break PD.[PD infection, HP] Absolutely important. No access = No dialysis [catheter complications, HP] Vital to the whole process of PD. Like a car without a motor. Got the parts but nothing to drive it. [membrane function, PC] If the catheter doesn’t work, you can’t conduct treatment therefore it is critically important to maintain that lifeline with great care.[catheter complications, PC] I would have like to know more about the function of my membrane and, more importantly, how to keep it functioning well. PD is my preferred dialysis treatment and I would not like to have to switch to hemo. [membrane function, PC] |
| Demanding frequent monitoring | If it is not monitored properly the outcome
could be devastating. [PD infection, PC] Repeat instructions, education and constant monitoring is required to maintain an optimal fluid status. [fluid, HP] Must be diligent in monitoring. [diabetes, PC] The need to chart the patient’s weight each day and report any significant changes to medical staff is time consuming but necessary. [weight change, PC] |
| Preserving quality of life | |
| Quality versus quantity | QOL is about thriving not just surviving.
[usual activities, PC] All of us will one day have to go through this, but it would have been better to improve the quality of life of the patient than to pay attention to the question of death. [death, PC] Death comes to us all. So it isn’t that important, more important is remaining as healthy as possible until you do die.[death, PC] Death is death, I’m more interested in helping people get more out of living. [death, HP] Life is more than not dying. [usual activities, HP] Death is still the hard outcome to be analysed what can be more important? [death, HP] It remains an important outcome for us as health professionals, we hope to save life. [death, HP] |
| Interfering with daily life | This is ongoing each day and can impact on my
work and recreational activities [fatigue, PC] I rated this highly because travel, even for short trips, was very complicated. You have to be very organized and plan ahead. You can’t just take off for the weekend or take advantage of an airline seat sale. Feels very limiting. [usual activities, PC] This is an ongoing daily problem that can sometimes interfere with working and recreational life. [gastrointestinal problems, PC] |
| Escalating morbidity | |
| Debilitating symptoms | The single worst symptom. Determined my
ability to do anything. [fatigue, PC] Most visible and first hand impactful part of PD that affected me as the patient 24/7 [catheter complications, PC] This is one of the most hardest feelings to deal with and it effects most of the day to day things in life having to work around fatigue and tiredness. [fatigue, PC] |
| Calamitous complications | Associated with morbidity, catheter loss,
transfer to hemodialysis, membrane damage, and occasionally death. So,
is crucial. [PD infection, HP] On a day by day basis this causes a lot of problems - and is an important reason for switch to HD [catheter complications, HP] It has direct impact on outcomes (CV disease, Hospitalization rate, Death) [fluid, HP] Infection = PD failure or death [PD infection, HP] |
| Irrelevant and futile information and treatment | |
| Avoiding redundant information | An important outcome, but I believe we already
know enough about it. [residual renal function, HP] Too much studied in the past… more important for HD patients. [blood pressure, HP] Already extensive research so danger duplication. [diabetes, PC] |
| Imperceptible or intangible | I suppose I should give greater importance to
death, but it seems far away to me. It’s hard for me to imagine
when I feel okay. [death, PC] Never had an issue so could not fully determine the consequences. [PD infection, PC] BP does not tell me a how a person feels. [blood pressure, HP] |
| Inability to control | If it’s failed, it’s failed. Not
much can be done about it. [PD failure, HP] I aim for a positive attitude, death is a fact of life I’ll cross deterioration in health as I get to it. [death, PC] Although sleep is very important, having sleep disturbance is part and parcel of being on PD and having end stage disease in general. [sleep, HP] |
| Easy to prevent and treat | One of the things we can actually (usually)
control. [blood pressure, HP] Usually solvable with current treatments. [anemia, HP] I think that if you keep everything clean and your technique is good, you can avoid infections. [PD infection, PC] Peritonitis is a preventable event and therefore modifiable. There is an opportunity to fix it. [PD infection, HP] |
Abbreviations: HP – Healthcare Professional; PC – Patient/Caregiver
Contributing to treatment longevity
Making or breaking treatment:
Both health professionals and patients/caregivers recognized that outcomes (e.g. PD-related infection, membrane function and catheter complications) could “make or break” the success of PD. Patients and caregivers emphasized that a “decent” and “functioning” membrane was required otherwise PD “would not be possible”. This was particularly important for patients who “would not like to have to switch to hemo” and therefore wanted to learn what they could do to prolong PD. Health professionals were more focused on providing the best treatment to patients depending on their clinical status and felt that it was “often hard to get them to accept a move to HD”.
Demanding frequent monitoring:
Some outcomes, including infection, fluid, weight change, blood pressure and diabetes, were given a high rating by participants if regular monitoring was required in order to manage and prevent complications of these outcomes – “If it is not monitored properly the outcome could be devastating” (patient).
Preserving quality of life
Quality versus quantity:
Many patients and caregivers acknowledged that “death comes to all of us” and hence felt that “it isn’t that important”. They believed that life was about “thriving not just surviving”. Some health professionals deemed survival as the “ultimate criterion of success or failure” as it was their perceived role to save lives – “as health professionals, we hope to save life.” Others were “more interested in helping people get more out of living” and felt that “quality of life is more valuable than quantity of years lived”.
Interfering with daily life:
Some patients rated outcomes highly if they disrupted their daily life or made planning activities “very complicated”. Such outcomes included fatigue and gastrointestinal problems that caused “ongoing daily problems” (patient) which interfered with their work and recreational activities.
Escalating morbidity
Debilitating symptoms:
Patients who experienced frequent or severe symptoms as a result of certain outcomes, including catheter complications and fatigue, rated them highly as they contributed to the symptom burden and incapacitating effect it had on their life – “Most visible and first hand impactful part of PD that affected me as the patient 24/7”.
Calamitous complications:
Health professionals considered the potentially detrimental complications or sequelae resulting from certain outcomes, including PD-related infection, fluid and catheter complications, and hence rated them highly – “Associated with morbidity, catheter loss, transfer to hemodialysis, membrane damage, and occasionally death. So, [PD-related infection] is crucial.”
Irrelevant and futile information and treatment
Avoiding redundant information:
Some judged that they “already know enough about” (health professional) specific outcomes, including residual renal function, blood pressure and diabetes. They believed there was “danger [in] duplication” of research for these outcomes and thus gave these outcomes lower priority.
Imperceptible or intangible:
Outcomes which some patients had never experienced, or which felt “far away” (patient), such as PD-related infection and death, were given lower ratings by some patients as these were “hard to imagine when you feel okay” (patient). Health professionals thought some outcomes did not convey how a patient feels, and hence gave lower ratings to some biochemical endpoints, such as blood pressure.
Inability to control:
Some participants gave lower ratings to outcomes they considered inevitable or unable to control, including death and PD failure – “If it’s failed, it’s failed. Not much can be done about it” (health professional). Some health professionals also gave lower ratings to outcomes which they perceived to be “part and parcel of being on PD”, such as sleep disturbances.
Easy to prevent and treat:
Outcomes perceived to be modifiable, preventable, or treatable were given lower ratings by some participants. Health professionals believed anemia and blood pressure were “usually solvable” and hence gave lower ratings. Some patients and health professionals felt that PD-related infections could be avoided if “you keep everything clean and your technique is good”.
DISCUSSION
The highest rated consensus-based outcomes among patients/caregivers and health professionals were PD-related infection, membrane function, PD failure, cardiovascular disease, catheter complications and death. The top outcomes for both stakeholder groups were consistent as rated by the Likert scale and ranked by the BWS scale. These important outcomes reflect a focus on maintaining PD, and preventing or reducing debilitating and disruptive complications, as well as improving survival. Being able to participate in usual activities was the most important patient-reported outcome by both stakeholder groups in all rounds, whereas some patient-reported outcomes, including sexual function, appearance, weight change, dizziness, pain, itch/skin and mood, were regarded as less important. This may reflect that PD is often chosen as a dialysis modality because it permits freedom and flexibility, allowing patients to do their usual activities.1 Biochemical outcomes, including calcium, potassium, parathyroid hormone and lipids were also of lower importance to both stakeholder groups.
This study has shown that patients/caregivers and health professionals give clinical and patient-reported outcomes higher priority than biochemical markers to be reported in trials. Biomarkers have potential benefit as prognostic tools to identify at-risk patients, as surrogates for clinical outcomes to increase statistical power or as early diagnostic tools.21 However, our study has shown that health professionals and patients/caregivers consider outcomes which have a direct and tangible impact on a patient’s ability to do PD to be of greatest priority for trials to inform decision-making.
Health professionals gave higher priority to many clinical outcomes, including fluid, residual renal function, hospitalization, catheter complications, death and PD-related infection, compared to patients/caregivers. This is consistent with prior work in hemodialysis and transplantation, in which health professionals consistently indicate death and hospitalization to be of higher importance compared with patients/caregivers.14,15 These outcomes relate to both long-term patient and technique survival and may reflect a perception of health professions’ primary clinical role to save lives and ensure treatment success (often defined by biochemical parameters).22 However, patients emphasized more the day-to-day symptoms (e.g. fatigue) and lifestyle disruption they experienced, which often remain unreported in studies.23,24 This highlights the need for studies to implement patient-reported outcomes, including fatigue, ability to do usual activities and depression, as these are critically important to patients.
Compared to health professionals, patients and caregivers gave greater priority to fatigue, membrane function and blood pressure. Fatigue has been consistently indicated to be of higher importance to patients/caregivers than health professionals.15 This may be because of the profound debilitation and impact of fatigue on function and psychosocial wellbeing that may be under-recognized by clinicians.18,23,25,26 The discrepancy in the prioritization of membrane function may be due to how patients/caregivers conceptualize this outcome. Our data suggest that patients perceive membrane function to be synonymous with their ability to maintain PD and reflects a fear of having to transfer to hemodialysis. However, studies have shown that poor membrane function accounts for only a small proportion of PD technique failures globally.27–29 A number of studies have also identified an important relationship between membrane function and mortality, such that high membrane solute transport (measured using the peritoneal equilibration test) is a significant, independent predictor of mortality.30–32 While some health professionals acknowledged and commented on this relationship, the majority of comments focused around the potential for technique failure and need for hemodialysis if the membrane function fails. The higher prioritization of blood pressure by patients and caregivers may relate to the need for frequent monitoring of blood pressure and having a visible indicator of their health status.18,33
Death and cardiovascular disease were also identified as top prioritized outcomes in hemodialysis.15 This is perhaps expected given the high risk of mortality and cardiovascular disease in the dialysis population.32,34,35 Vascular access function and dialysis adequacy were also top prioritized outcomes in hemodialysis. In comparison, catheter complications were highly prioritized in our study. Vascular access problems and catheter complications both impact on the success of HD and PD, respectively, and were recognized to have debilitating consequences. Fatigue was ranked lower relative to the other outcomes in the PD survey compared with the HD survey. This may be because patients on hemodialysis experience severe fatigue that impacts on their ability to participate in daily activities, which may in part be due to the hemodialysis prescription, and the higher comorbidities and symptom burden in this population.26,36
Of note, there was a greater degree of concordance in the importance of outcomes between patients/caregivers and health professionals in PD compared to hemodialysis. In the PD survey, differences between stakeholder groups were generally small with mean differences in round three at a maximum of 0.5 for health professionals and 0.4 for patients/caregivers. In comparison, for the HD Delphi survey, differences between stakeholder groups were larger with maximum mean differences of 1.0 for health professionals and 0.9 for patients/caregivers.15 This may be because patients on PD are more engaged with self-care and monitoring, and perceive themselves to be better informed about their dialysis compared to patients on HD.37
The Delphi technique is used to gain consensus on a specific topic.38 Participants in a Delphi Panel are anonymous and do not directly interact with each other. They are less likely to be inhibited by confrontation or domination by outspoken individuals, which can occur in other group settings.39 The online format of the Delphi allows for widespread international participation. This survey included a large sample size with a broad representation of stakeholders from 68 countries and a high retention rate of 61% from round one to round three. Using a systematic approach, we generated consensus among patients, caregivers and health professionals on important outcome domains to be reported in all trials in patients on PD. While outcomes such as mortality and infection may be expected to be of high importance (both rated higher by health professionals than patients/caregivers), we have identified high-priority patient-reported outcomes – fatigue, usual activities and depression. We assessed absolute and relative importance using the Likert and Best-Worst scales, respectively. The qualitative data provide insight into the reasons for participant priorities.
In rounds two and three, participants were provided with graphs of the distribution of scores and comments from the participants. It is possible that the graphs were difficult to interpret, however we provided instructions on how to read the graph and an example for participants to see prior to commencing the survey. We did not edit the comments for readability. For technical reasons and feasibility, we administered the survey online and in English-language, which precluded involvement of non-English speaking participants and those without internet access or with limited computer literacy. We acknowledge that the priorities of non-English speaking people, with lower educational attainment, and those residing in low-income countries may be different. Although definitions were provided for each outcome, we acknowledge that participants may have interpreted the outcomes differently.
Results from this survey were reviewed at the SONG-PD consensus workshop held at the 17th Congress of the International Society for Peritoneal Dialysis in Vancouver, Canada. Participants including patients, caregivers, clinicians, researchers and policy makers, from 13 different countries discussed the feasibility, acceptability and appropriateness of the top prioritized outcomes from the Delphi survey to be included in a core outcome set for PD. The Delphi survey and consensus workshop will be used to establish a core outcome set to be reported in all trials in PD. Once the core outcome domains are established, work to identify core outcome measures will follow.
Our international SONG-PD Delphi survey has identified critically important outcomes that should be reported in all trials involving patients on PD, and can inform other research including observational and qualitative studies and registries. The most important outcomes identified by both patients/caregivers and health professionals reflect major clinical challenges – PD-related infection, death, PD failure and cardiovascular disease. The most important patient-reported outcomes were being able to participate in usual activities, fatigue and depression. By identifying what matters most to stakeholders we can better develop all levels of care – education, training, resources, priorities for advocacy and priorities for funding. Ultimately, this will strengthen the relevance and reliability of evidence to support shared decision-making for people undergoing PD.
Supplementary Material
Figure S1. SONG-PD Delphi process.
Figure S2. SONG-PD Delphi recruitment flier.
Table S1. Collaborating organizations and institutions.
Table S2. Outcomes and the definition provided in the Delphi Survey.
Table S3. Round 1 mean scores, median scores and proportion (rating 7-9) of patients/caregivers and health professionals for 39 outcomes.
Table S4. Outcomes suggested by participants in Round 1.
Table S5. Round 2 means, medians and proportions (rating 7-9) of patients/caregivers and health professionals for 23 outcomes.
Table S6. Round 3 means, medians, proportions (rating 7-9), and BWS scores of patients/caregivers and health professionals for 16 outcomes.
Acknowledgements:
We thank all the patients, caregivers and health professionals who gave their time to participate in the study.
Disclosure:
KEM is supported by a National Health and Medical Research Council (NHMRC) Postgraduate Scholarship (APP1151343). AT is supported by a NHMRC Fellowship (APP1106716). DWJ is supported by a NHMRC Practitioner Fellowship (APP1117534). JS is supported by grants K23DK103972 from the NIH-NIDDK. YC is supported by a NHMRC Early Career Fellowship (APP1126256). RM serves as a consultant to Zytoprotec. This study was supported by the International Society for Peritoneal Dialysis (ISPD). The funding organization had no role in the preparation, review or approval of the manuscript.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Supplementary information is available at Kidney International’s website.
References
- 1.François K, Bargman JM. Evaluating the benefits of home-based peritoneal dialysis. Int J Nephrol Renovasc Dis. 2014;7:447–455. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Li PK-T, Kwong VW-K. Current challenges and opportunities in PD. Semin Nephrol. 2017;37(1):2–9. [DOI] [PubMed] [Google Scholar]
- 3.Kolesnyk I, Dekker FW, Boeschoten EW, Krediet RT. Time-dependent reasons for peritoneal dialysis technique failure and mortality. Perit Dial Int. 2010;30(2):170–177. [DOI] [PubMed] [Google Scholar]
- 4.Lew SQ, Piraino B. Quality of life and psychological issues in peritoneal dialysis patients. Semin Dial. 2005;18(2):119–123. [DOI] [PubMed] [Google Scholar]
- 5.Paniagua R, Amato D, Vonesh E, et al. Effects of increased peritoneal clearances on mortality rates in peritoneal dialysis: ADEMEX, a prospective, randomized, controlled trial. J Am Soc Nephrol. 2002;13(5):1307–1320. [DOI] [PubMed] [Google Scholar]
- 6.Maiorca R, Cancarini GC, Broccoli R, et al. Prospective controlled trial of a y-connector and disinfectant to prevent peritonitis in continuous ambulatory peritoneal dialysis. Lancet. 1983;322(8351):642–644. [DOI] [PubMed] [Google Scholar]
- 7.Davies SJ, Woodrow G, Donovan K, et al. Icodextrin improves the fluid status of peritoneal dialysis patients: Results of a double-blind randomized controlled trial. J Am Soc Nephrol. 2003;14(9):2338–2344. [DOI] [PubMed] [Google Scholar]
- 8.Williams JD, Topley N, Craig KJ, et al. The Euro-Balance Trial: The effect of a new biocompatible peritoneal dialysis fluid (balance) on the peritoneal membrane. Kidney Int. 2004;66(1):408–418. [DOI] [PubMed] [Google Scholar]
- 9.Baigent C, Herrington WG, Coresh J, et al. Challenges in conducting clinical trials in nephrology: conclusions from a Kidney Disease - Improving Global Outcomes (KDIGO) Controversies Conference. Kidney Int. 2017;92(2):297–305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Svensson S, Menkes DB, Lexchin J. Surrogate outcomes in clinical trials: A cautionary tale. JAMA Intern Med. 2013;173(8):611–612. [DOI] [PubMed] [Google Scholar]
- 11.Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13(1):132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Tong A, Manns B, Wang AYM, et al. Implementing core outcomes in kidney disease: report of the Standardized Outcomes in Nephrology (SONG) implementation workshop. Kidney Int. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Manera KE, Tong A, Craig JC, et al. Standardized Outcomes in Nephrology-Peritoneal Dialysis (SONG-PD): Study Protocol for Establishing a Core Outcome Set in PD. Perit Dial Int. 2017;37(6):639–647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sautenet B, Tong A, Manera KE, et al. Developing consensus-based priority outcome domains for trials in kidney transplantation: A multinational delphi survey with patients, caregivers, and health professionals. Transplantation. 2017;101(8):1875–1886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Evangelidis N, Tong A, Manns B, et al. Developing a set of core outcomes for trials in hemodialysis: An international delphi survey. Am J Kidney Dis. 2017;70(4):464–475. [DOI] [PubMed] [Google Scholar]
- 16.Kaufman J, Ryan R, Lewin S, et al. Identification of preliminary core outcome domains for communication about childhood vaccination: An online Delphi survey. Vaccine. 2018;36(44):6520–6528. [DOI] [PubMed] [Google Scholar]
- 17.Kuizenga-Wessel S, Steutel NF, Benninga MA, et al. Development of a core outcome set for clinical trials in childhood constipation: a study using a Delphi technique. BMJ Paediatr Open. 2017;1(1). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Manera K, Johnson D, Craig J, et al. Patient and caregiver priorities for outcomes in peritoneal dialysis: multinational nominal group technique study. Clin J Am Soc Nephrol. 2019;14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Schunemann H, Brozek J, Oxman A. GRADE Handbook for Grading Quality of Evidence and Strength of Recommendation [Internet]. 2009. http://gdtguidelinedevelopmentorg/app/handbook/handbookhtml.
- 20.Flynn TN, Louviere JJ, Peters TJ, Coast J. Best–worst scaling: What it can do for health care research and how to do it. J Health Econ. 2007;26(1):171–189. [DOI] [PubMed] [Google Scholar]
- 21.Aufricht C, Beelen R, Eberl M, et al. Biomarker research to improve clinical outcomes of peritoneal dialysis: consensus of the European Training and Research in Peritoneal Dialysis (EuTRiPD) network. Kidney Int. 2017;92(4):824–835. [DOI] [PubMed] [Google Scholar]
- 22.Tong A, Winkelmayer WC, Wheeler DC, et al. Nephrologists’ Perspectives on Defining and Applying Patient-Centered Outcomes in Hemodialysis. Clin J Am Soc Nephrol. 2017;12(3):454–466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Urquhart-Secord R, Craig JC, Hemmelgarn B, et al. Patient and Caregiver Priorities for Outcomes in Hemodialysis: An International Nominal Group Technique Study. Am J Kidney Dis. 2016;68(3):444–454. [DOI] [PubMed] [Google Scholar]
- 24.Sautenet B, Tong A, Williams G, et al. Scope and Consistency of Outcomes Reported in Randomized Trials Conducted in Adults Receiving Hemodialysis: A Systematic Review. Am J Kidney Dis. 2018;72(1):62–74. [DOI] [PubMed] [Google Scholar]
- 25.Ju A, Unruh M, Davison S, et al. Establishing a Core Outcome Measure for Fatigue in Patients on Hemodialysis: A Standardized Outcomes in Nephrology-Hemodialysis (SONGHD) Consensus Workshop Report. Am J Kidney Dis. 2018;72(1):104–112. [DOI] [PubMed] [Google Scholar]
- 26.Ju A, Unruh ML, Davison SN, et al. Patient-Reported Outcome Measures for Fatigue in Patients on Hemodialysis: A Systematic Review. Am J Kidney Dis. 2018;71(3):327–343. [DOI] [PubMed] [Google Scholar]
- 27.Davies SJ, Phillips L, Griffiths AM, et al. What really happens to people on long-term peritoneal dialysis? Kidney Int. 1998;54(6):2207–2217. [DOI] [PubMed] [Google Scholar]
- 28.Lan PG, Clayton PA, Johnson DW, et al. Duration of Hemodialysis Following Peritoneal Dialysis Cessation in Australia and New Zealand: Proposal for a Standardized Definition of Technique Failure. Perit Dial Int. 2016;36(6):623–630. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Jaar BG, Plantinga LC, Crews DC, et al. Timing, causes, predictors and prognosis of switching from peritoneal dialysis to hemodialysis: a prospective study. BMC Nephrol. 2009;10(1):3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Brimble KS, Walker M, Margetts PJ, et al. Meta-Analysis: Peritoneal Membrane Transport, Mortality, and Technique Failure in Peritoneal Dialysis. J Am Soc Nephrol. 2006;17(9):2591–2598. [DOI] [PubMed] [Google Scholar]
- 31.Rumpsfeld M, McDonald SP, Johnson DW. Higher Peritoneal Transport Status Is Associated with Higher Mortality and Technique Failure in the Australian and New Zealand Peritoneal Dialysis Patient Populations. J Am Soc Nephrol. 2006;17(1):271–278. [DOI] [PubMed] [Google Scholar]
- 32.Kendrick J, Teitelbaum I. Strategies for Improving Long-Term Survival in Peritoneal Dialysis Patients. Clin J Am Soc Nephrol. 2010;5(6):1123–1131. [DOI] [PubMed] [Google Scholar]
- 33.Tong A, Lesmana B, Johnson DW, et al. The perspectives of adults living with peritoneal dialysis: thematic synthesis of qualitative studies. Am J Kidney Dis. 2013;61(6):873–888. [DOI] [PubMed] [Google Scholar]
- 34.Robinson BM, Zhang J, Morgenstern H, et al. Worldwide, mortality risk is high soon after initiation of hemodialysis. Kidney Int. 2014;85(1):158–165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Zoccali C, Mallamaci F, Tripepi G. Traditional and emerging cardiovascular risk factors in end-stage renal disease. Kidney Int. 2003;63:S105–S110. [DOI] [PubMed] [Google Scholar]
- 36.Jhamb M, Weisbord SD, Steel JL, Unruh M. Fatigue in patients receiving maintenance dialysis: a review of definitions, measures, and contributing factors. Am J Kidney Dis. 2008;52(2):353–365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Rubin HR, Fink NE, Plantinga LC, et al. Patient ratings of dialysis care with peritoneal dialysis vs hemodialysis. JAMA. 2004;291(6):697–703. [DOI] [PubMed] [Google Scholar]
- 38.Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–380. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011;8(1):e1000393–e1000393. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Figure S1. SONG-PD Delphi process.
Figure S2. SONG-PD Delphi recruitment flier.
Table S1. Collaborating organizations and institutions.
Table S2. Outcomes and the definition provided in the Delphi Survey.
Table S3. Round 1 mean scores, median scores and proportion (rating 7-9) of patients/caregivers and health professionals for 39 outcomes.
Table S4. Outcomes suggested by participants in Round 1.
Table S5. Round 2 means, medians and proportions (rating 7-9) of patients/caregivers and health professionals for 23 outcomes.
Table S6. Round 3 means, medians, proportions (rating 7-9), and BWS scores of patients/caregivers and health professionals for 16 outcomes.