Establishing Core Outcome Domains in Hemodialysis: Report of the Standardized Outcomes in Nephrology−Hemodialysis (SONG-HD) Consensus Workshop

Abstract

Evidence-informed decision-making in clinical care and policy in nephrology is undermined by trials that selectively report a large number of heterogeneous outcomes, many of which are not patient-centered. The Standardized Outcomes in Nephrology−Hemodialysis (SONG-HD) Initiative convened an international consensus workshop on November 7, 2015, to discuss the identification and implementation of a potential core outcome set for all trials in hemodialysis. The purpose of this article is to report qualitative analyses of the workshop discussions, describing the key aspects to consider when establishing core outcomes in trials involving patients on hemodialysis. Key stakeholders including eight patients/caregivers and 47 health professionals (nephrologists, policy makers, industry, researchers) attended the workshop. Attendees suggested that identifying core outcomes required equitable stakeholder engagement to ensure relevance across patient populations; flexibility to consider evolving priorities over time; deconstruction of language and meaning for conceptual consistency and clarity; understanding of potential overlap and associations between outcomes; and an assessment of applicability to the range of interventions in hemodialysis. For implementation, they proposed that core outcomes must have simple, inexpensive and validated outcome measures that could be used in clinical care (quality ndicators) and trials (including pragmatic trials), and endorsement by regulatory agencies. Integrating these recommendations may foster acceptance and optimize the uptake and translation of core outcomes in hemodialysis, leading to more informative research, for better treatment, and improved patient outcomes.

Hemodialysis is a demanding and resource-intensive regimen that places an immense burden on patients with chronic kidney disease (CKD), their families and the healthcare system (1–3). The failure to ameliorate the devastating and adverse outcomes in patients on hemodialysis may be partly explained by the use of unvalidated surrogate end points, omission of patient-centered outcomes, variability of outcomes across trials, and outcome reporting bias (4).

Biochemical outcomes, such as mineral metabolism, dialysis adequacy (Kt/V), hemoglobin, blood pressure, serum albumin, are frequently measured. Recent meta-analyses have shown that serum parathyroid hormone, calcium, and phosphorus are weakly and inconsistently correlated with all-cause and cardiovascular mortality, and other important outcomes including cardiovascular events (5, 6). Also, studies consistently show that patients with CKD prioritize quality of life, mental health, impact on family, fatigue, and employment and consider these outcomes in making treatment decisions (7–10); yet, trials rarely report these outcomes.

In contrast, the changing research landscape marked by strong momentum towards standardized reporting of high-priority patient-centered outcomes is apparent in many other specialties. The Outcome Measures in Rheumatology (OMERACT) is a longstanding initiative that since 1992 has developed and validated clinical and radiographic core outcome measures in rheumatic disease, which has improved the relevance and reporting of outcomes in the rheumatology trials (11–14). Recently, the Core Outcome Measures in Effectiveness Trials (COMET) network was convened to support the development and implementation of core outcome sets—defined as an agreed minimum set of standardized outcomes to be measured and reported in all trials for a particular clinical area (15). (Figure 1) The outcomes in a specific trial do not have to be restricted to those in the core outcome set and investigators can include additional outcomes (16).

Figure 1.

Core outcomes

In 2014, the international Standardized Outcomes in Nephrology (SONG) initiative was formed to develop core outcomes across the spectrum of CKD based on the common priorities of patients, caregivers, clinicians, policy makers. researchers, and industry. The initial and current focus is on hemodialysis (SONG-HD) (4). Using the validated OMERACT methodological framework (11, 17), SONG-HD investigators have completed a systematic review to identify outcomes reported in hemodialysis trials, conducted a nominal group technique study to elicit outcomes important to patients and caregivers, interviewed nephrologists to ascertain the values, attitudes and beliefs about outcomes currently included in trials and the development of core outcomes, and have completed an international Delphi survey to produce an evidence-informed and consensus-based prioritized list of core outcome domains for hemodialysis (4).

Based on previous core outcome initiatives, approximately 3–5 core outcomes are identified for the core outcome set (i.e. prioritized to be of critical importance by all stakeholder groups to include in all trials in hemodialysis). All other outcomes identified during the SONG-HD process will be classified as “outer core” outcomes or as outcomes to consider in the research agenda. (Figure 1)

As part of the broader SONG-HD Initiative, key stakeholders were invited to participate in an international consensus workshop to review and discuss the proposed core outcomes. The aim of this workshop report is to describe and summarize stakeholder perspectives on the development, establishment, and implementation of a core outcome set for hemodialysis. Such discussions are critical for gaining a better understanding of the potential challenges in establishing and translating core outcome domains in order to foster acceptance and inform strategies to optimize uptake and translation of core outcomes for hemodialysis. Ultimately, this can help to strengthen the quality of research and improve patient outcomes in hemodialysis.

SONG-HD CONSENSUS WORKSHOP

Overview and Context

The international SONG-HD consensus workshop was convened to elicit stakeholder feedback on the identification and implementation of a potential core outcome set for hemodialysis trials and other forms of research. The potential core outcomes are based on preliminary data and interim analysis from an international Delphi survey that was completed by patients, caregivers, healthcare providers, policy makers, and funders. The outcomes with a mean and median score of ≥ 7 (defined as of critical importance) in both stakeholder groups were dialysis adequacy, vascular access problems, fatigue, dialysis-free time, washed out after dialysis, cardiovascular disease, anemia, ability to work, blood pressure, mortality, mobility, impact on family/friends, and infection/immunity. The detailed analysis and final results of the Delphi are beyond the scope of this workshop report and will be published separately.

Participants and Contributors

Patients and caregivers with experience of hemodialysis (n=8), and health professionals including nephrologists, nursing and allied health professionals, researchers, policy makers, and representatives from industry (n=47) attended the workshop (total n=55 attendees). The SONG-HD investigators invited patients and caregivers from the United States (San Diego, n=1;, Los Angeles, n=3; Houston, n=3), and United Kingdom (London, n=1 [member of SONG Executive Committee]). Patients/caregivers received reimbursement for travel (interstate flight, transport, accommodation); thus their numbers were limited due to available resources.

Health professionals were purposively identified to include a range of practice locations, clinical experience, and roles in research, policy and industry. The health professional workshop attendees were from seven countries: the United States (n=20), Australia (n=12), Canada (n=7), United Kingdom (n=4), Germany (n=2), New Zealand (n=1), and The Netherlands (n=1). Workshop contributors (n=17) were health professionals who provided feedback on the pre-workshop materials and preliminary report, but were unable to attend the workshop in person due to conflicting schedules.

The health professionals, including attendees and contributors, had a broad range of experience and expertise in research (epidemiology, clinical trials in hemodialysis, outcomes and outcome measures), and clinical nephrology (hemodialysis). Some held leadership and advisory positions on key global and national professional societies (e.g. International Society of Nephrology, American Society of Nephrology) and research, policy, industry, and consumer organisations, including (but not limited to) the US Centers for Disease Control and Prevention (CDC), Centers for Medicare & Medicaid Services (CMS), Food and Drug and Administration (FDA), and National Institutes of Health (NIH), as well as KDIGO (Kidney Disease: Improving Global Outcomes).

Workshop Program and Materials

The two-hour workshop was held on November 7, 2015 at a hotel function room in San Diego, United States. This coincided with the American Society of Nephrology’s Kidney Week Annual Conference 2015 to maximize attendance. The workshop program and materials were provided to participants one week in advance. During the workshop, we presented an overview of the SONG-HD process and preliminary results, and a list of potential core outcomes. Participants were allocated to four breakout groups with 10–12 members. Each group had at least one patient or caregiver. Mixed-stakeholder breakout groups involving patients/caregivers, physicians, policy makers, and representatives from industry were convened to allow for explanations and clarifications of concepts, richer exchange of ideas and knowledge, and breadth of discussion. The groups were facilitated by B.M. and S.Y. (Group 1); A.T., W.C.W., and J.P. (Group 2); H.T.-T., D.C.W., and N.L. (Group 3); and B.H. and J.S. (Group 4). The facilitators attended a briefing session prior to the workshop and were provided with a question guide (Item S1, available as online supplementary material).

Each facilitator asked participants in the breakout group to reflect and comment on the preliminary SONG-HD process and potential core outcomes, and to discuss strategies for implementing core outcomes. In the final plenary session, all the groups reconvened and a member from each group presented a summary of their discussion. B.M. facilitated a general discussion of the key issues raised across groups. All breakout and plenary discussions were audio-taped and transcribed. The transcripts were entered into HyperRESEARCH (ResearchWare Inc, Version 3.0) to facilitate coding and analysis of the data. From the transcripts, A.T. identified and summarized the key considerations, challenges, and recommendations with regards to developing and implementing a core outcome set for hemodialysis. All participants and contributors received a draft workshop report to provide feedback within a two-week timeframe. Additional comments were integrated into the final report.

Postworkshop Consultation

All participants and contributors received a draft workshop report to provide feedback within a two-week timeframe. Additional comments were integrated into the final report.

SUMMARY OF WORKSHOP DISCUSSION

Overview

The themes arising from the workshop discussion on the identification and implementation of core outcomes in hemodialysis are described in the following section. For some themes, a brief explanation of the SONG-HD principles and process has been included to provide context for the discussion. Illustrative quotations for each theme are provided in Table 1. Key recommendations of the consensus workshop are outlined in Box 1.

Table 1.

Selected quotations from the workshop discussions on the identification and implementation of core outcomes in hemodialysis

Themes	Quotations
Identification
Equitable stakeholder engagement	“It might even be different rankings for different regions. It might be that there’s commonalities between different regions or whatever. I’m not sure one shoe size does fit all broadly.” (health professional)
	“There’s a severe dearth of Asia in this whole thing.” (health professional)
	“Is there symmetry between the stakeholder groups in terms of the impact? Physicians or the health care professionals didn’t change their views [in the Delphi]. I just wondered if there’s a power dynamic going on there? I just wonder if all the stakeholders are influenced more by the health care professionals than the other way around. So is it asymmetrical?” (health professional)
Evolving priorities	“It is best as an ongoing initiative rather than ‘we’re done now, this is the answer.’ Some priorities change over time.” (health professional)
	“This needs to be a breathing initiative…the [priority for outcomes] might be totally different in five years.” (health professional)
Deconstructing language and meaning	“We almost need to deconstruct language. What one person feels when they talk about dialysis adequacy is totally not what the next person feels. We really need to peel off the layers until we understand the core meaning of what it is.” (health professional)
	“Dialysis is the process and adequacy is the overall concept, but there are probably multiple ways to measure that and that’s what we’re hearing.” (health professional)
	“There’s a huge movement, even amongst the health professionals, that dialysis adequacy means much more than the Kt/V. If anything, the patients seem to already know that, but we’re still trying to like figure this out.” (health professional)
	“When your numbers are adequate, your blood is being cleaned, your outcomes are well, then you feel better, you can handle things happening in your life. You can spend more quality time with your family instead of having to take X amount of trips to the emergency room because something has happened, or having those days when you’re so weak you can’t get out of bed. It all comes down to quality of life.” (patient)
	“When I look at dialysis I see in the general that I am at a deficit. My life is not normal. So in essence I’m striving to get that normalcy back into my life so that I can live the quality of life that I see others live and enjoy. So making that difference be understood between what they mean when they say dialysis adequacy and what a patient hears when they hear dialysis adequacy. To me, dialysis adequacy means I have an excellent quality of life even though I am on dialysis.” (patient)
Disentangling interdependency	“The whole point of dialysis is to keep the patient feeling well and feeling better and keeping their normal day-to-day life, it’s all of these other outcomes that contribute to that. So just finding what those four other important things are to measure, that leads to the patient feeling well, right?” (health professional)
	“You start off with fundamentals—whether or not somebody has a fistula or a catheter and whether or not somebody has a measurable lab value or whatever. Then you go to a different group of values like whether someone has an infection or what the quality of life metrics are. Then you end up with the top of the pyramid what really, really matters—getting up in the morning and not feeling wasted, having a good day, knowing that you don’t have to worry about your fistula working. Making sure you’ve got adequate dialysis so you can really live a good life. There’s a pyramid effect and we start with the fundamentals but we end up with what really, really matters to now in patients, any one of those as a human being. What matters and how can we have a good day.” (health professional)
	“Looking at this list of outcomes, it occurs to me some of [these] are like apples and oranges. What you’re really talking about is some things are hard outcomes like the ability to work, or the ability to live, or the ability to not be hospitalized, and some things are surrogate outcomes which may or may not be on the causal pathway to those hard outcomes, and I think health practitioners and patients have a very different perception, just because of knowledge of what things lead to better quality of life. Perhaps a drop in blood pressure is not important if we just asked you, but if you ask us we would say a drop in blood pressure is very important, because we know that that’s going to affect your brain function, your heart function, and ability to function, and so I wonder if some of these things are just a difference in perception, of what leads to what.” (health professional)
	“Let’s say you knew about blood pressure like you know now, because you’re a patient, people have told you that. Let’s say you knew nothing about it, but then once somebody tells you that blood pressure if it’s too high, can lead to a stroke and if it’s too low, it can also lead to a stroke or other harmful things, then would you care about it or would it still not matter if you didn’t have symptoms?” (health professional)
	“So blood pressure is just a number. Are patients always fully informed about the consequences of very high or low blood pressure?” (health professional)
	“Anemia and fatigue may very well be telling you the same thing for example.” (health professional)
	“I’ve got some problems with it really because the outcomes are not mutually exclusive and they may be conflicting. Ability to travel and dialysis free time, they conflict with, say, dialysis adequacy. To categorize them like this is a bit simplistic and may not actually reflect real achievable goals. The other thing is that a lot of the biochemical parameters, which we use as surrogates, may have been used as surrogates for some of these other softer outcomes.” (health professional)
	“A lot of my patients would feel really well if you dialyzed them for eight hours and seven nights and their fatigue would be much lower. But yet that would conflict with the dialysis free time.” (health professional)
Interventional applicability	“In hemodialysis, we have trials that are specifically looking at improving vascular access care. Trials that are looking at reducing cardiovascular morbidity and mortality. Trials that are looking at survival. Trials that are looking at quality of life and symptom control. If the outcomes for one of those are not relevant to some of the other ones, there would be no point in vascular access trial necessarily measuring some of the quality-of-life issues.” (health professional)
	“The end points are going to have to reflect what your drug or device is targeting. We’re going towards cluster randomized trials. The challenge is going to be to create this pre-specified data set or a set of data fields on the dialysis unit that we always collect that potentially in the future will work for a new device for vascular access that maybe comes out. Because depending on your product or your device, your things you’re going to want to collect even in a pragmatic trial are going to be very different.” (health professional)
	“I have no objection whatsoever that we do need to measure patient-centric outcomes and I do it in all my trials. What I worry about is people are going to be very prescriptive about what I have to measure and how I measure it. Rather than allowing me to weigh up what I think is the best mechanism.” (health professional)
Procedural efficiency	“That’s the beauty about the process. We have this smorgasbord of different outcomes but at the end of the day we can’t measure everything. It’s a matter of focus, it’s a matter of efficiency and this process really helps us. On one hand, of course focus on what was ranked extremely highly. Also, there’s a good list of outcomes that people have considered that really are not important and just getting that noise out of the entire system I think is already a good contribution.” (health professional)
Implementation
Feasibility of outcome measures	“If you just distill this patient experience down to no more than five scales, one might be on vascular access, one might be in terms of dialysis adequacy or something. Patients would all opt in and not be opposed to responding to four or five questions.” (health professional)
	“The dialysis adequacy what we measure and how we measure it before you can implement something is probably your crawl before you can walk.” (health professional)
	“You have to think about it from a perspective of designing the trial and that everything that I measure is going to cost time and money and it will detract from my ability to do other things in the trial.” (health professional)
	“Some of us would be concerned that if it was regulated, that we had to have these outcomes, that it might increase the cost of the study beyond what we could do. So we’ve got to be a bit careful about being too prescriptive about what’s collected.” (health professional)
Propagating and patient-centered paradigm	“The other thing in terms of translation was just as a physician seeing the disparity between where certain items fall is eye opening for me. That is informing physicians about these, even just this process could be very useful for individual patients and physician relationships.” (health professional)
	“We should have the end point be patient driven. Death of course is always an end point but before there’s also quality of life that is critical. I would rather be healthy and alive than anything but I’d rather be feeling good while I’m alive.” (patient)
	“It’s a trial that’s being conducted in dialysis units. It’s a large trial. The outcomes that they’re looking at don’t include, I think, patient-centered outcomes. It would be nice if they did, so that you would have a better idea of what longer dialysis means to people, hundreds of thousands of people.” (health professional)
	“This lecture at this table really impressed me and I believe some things that I said made a difference to doctors and patients. Pretty much I feel there is more discussion down the road, more seminars like this and more knowledge to learn from both sides. I would like to add education for the doctors, the clinicians, the people working with the patients. A term may mean one thing to you but to the patient every term basically boils down to quality of life. When I wake up in the morning do I feel good enough to be able to spend time with my family, to be able to travel the way I’d like to, to be able to go to work if I want to, to be able to do the hobbies that I enjoy doing.” (patient)
	“That’s a major point because we get labs every month of course. We get them passed out and there are patients that crumple them up, some of them fold them up put them in a bag whatever because they’re looking at these numbers and they don’t know what they mean. So it’s so important to translate it from the numbers to something that even the newest guy who’s a patient can wrap their head around it and understand.” (health professional)
	“Patient-centered outcomes is more and more relevant to them today than ever. They’re also willing to start thinking about trade-offs from let’s say mortality goes up a little bit, the quality of life improves significantly.” (health professional)
	“There may be trial end points that the FDA tell us that we have to use. We would also like some end points in there that have some key relevance to the patients. Now there may be secondary end points but at least we’re collecting the vital information to assess those end points.” (health professional)
Contextualizing translation of outcomes	“It’s different the context in which you’re asking that question, one is are you planning clinical trials with these outcomes or you’re measuring those as a measure of qualitative care that can pay for performance schemes that have cropped up all over the world. The scope of that is going to vary based upon the context in which you’re asking, and are variable across the health care systems.” (health professional)
	“There are two different issues though. One is clinical studies and how you standardize those. But the other is actually the carrot to get people to practice and treat people in certain ways, the incentives that are out there.” (health professional)
	“Is that really what the goal is though, is to drive the research agenda by choosing which outcomes are important? Or is it to ensure that whatever the research agenda is, is that the outcomes that are in it are reasonably standardized across different trials with similar goals?” (health professional)
	“Doesn’t it stifle innovation and interest in new things? If you concentrate the funders on four things that just means that we’re going to be investigating those four things. It removes any chance of anything novel and new coming into the market and changing the paradigm.” (health professional)

FDA, Food and Drug Administration

Box 1. Key workshop recommendations on identifying and implementing core outcomes in hemodialysis.

A core outcome domain

• Should be intrinsically important to the majority or all patient populations

• Should have a clear, precise, and standardized definition

• Should be conceptualized by all stakeholder groups in a consistent way

• Must be relevant over a longer time-frame (ie, can be a short-term and long-term outcome)

• Should be single-attribute (ie, does not include multiple outcome domains)

• Cannot be in direct conflict with another high-priority outcome

• Must have broad relevance to a range of interventions in hemodialysis

• Should be feasibly applied in different types of trials (including pragmatic or registry trials)

• Should be applicable in the context of assessing quality of care (eg, quality indicator)

• Can be considered to drive the research agenda, as well as to be reported in current trials

A core outcome domain set

• Should be flexible and allow for periodic changes to outcomes as necessary (as they may change over time) and to allow for innovation

• Must include mortality as it is inherently fundamental to all other outcomes

Implementation of core outcomes requires

• Simple and inexpensive outcome measures

• Validated outcomes measures

• Support and endorsement from regulatory agencies

Identification of Core Outcomes

Equitable Stakeholder Engagement

Broad inclusion of patients

Patient involvement is a fundamental principle underpinning the SONG-HD process. One of the phases in SONG-HD included a nominal group technique study with patients on haemodialysis (n=58) and caregivers (n=24) from Australia (Sydney, Melbourne) and Canada (Calgary), who identified and ranked outcomes they considered to be important (18). The outcomes identified in the nominal group technique were augmented with outcomes identified in the systematic review to form the list of outcomes used in the international Delphi survey (September-November 2015) in which patients/caregivers and health professionals rated and ranked the importance of outcomes. Of the 1181 respondents in Round 1, patients/caregivers constituted 202, and by Round 3 (one week prior to closing), 133 patients/caregivers had completed the survey. Compared with other initiatives to develop core outcomes, the SONG-HD Delphi survey includes the highest number of patients/caregiver respondents. However, the workshop participants recognized the potential exclusion of specific patient groups by demographic (e.g. socioeconomic status [low-income countries], low educational attainment, non-English speaking) and clinical characteristics (e.g. depression, cognitive impairment, critically ill). Thus, it was deemed necessary to acknowledge the potential limitations in the generalizability of core outcomes across all patient populations, and to assess strategies for broader engagement particularly of marginalized, minority or vulnerable groups, and to ensure that core outcomes are at least intrinsically important to most, if not all patients requiring hemodialysis.

Maintaining balance of power

Participants observed that the preliminary Delphi results suggested that health professional ratings appeared to remain similar throughout all rounds, whilst patient/caregiver ratings changed. For example, the 2 outcomes cardiovascular disease and vascular access problems were moved up into the top 10 outcomes for patients/caregivers in Round 2. For some participants, this suggested an “asymmetry” of power in the Delphi process, which was thought to cause possible “contamination” that required strategies to address an undue power imbalance (e.g. sensitivity analysis). However, some health professional attendees who had completed the Delphi stated they had changed their score to “reflect” the patient priorities. The Delphi results also indicated that health professionals increased their scores based on patient preferences, although the outcomes did not move into the top 10.

Evolving Priorities

Some participants expected that priorities for outcomes would inevitably change over time and suggested that SONG-HD should be a “breathing initiative” to allow opportunities to review, change, and add new outcomes. This was also regarded as a way of safeguarding against the possibility of core outcomes stifling innovation and interest in developing new and novel interventions “coming into the market and changing the paradigm,” as one health professional stated. Also, changes in dialysis technology may give rise to other new important outcomes. They also considered that the relevance of outcomes could be dependent on specific time-frames (“If you’ve got a study that’s looking at outcomes and prevalent patients in five years, you won’t use the ones that are based on outcomes of five days” observed one health professional), which suggests that core outcomes should be relevant and measurable over time given the variability in the duration of trials.

Deconstructing Language and Meaning

Across all breakout groups, participants remarked on the controversy, ambiguity, and variability in the definition of some outcomes; particularly in reference to dialysis adequacy. Dialysis adequacy (defined in the survey as “how well the dialysis cleans the blood, clearance, Kt/V”) was rated by patients/caregivers to be of highest importance based on the mean and median scores of ratings from 1 (lowest importance) to 9 (highest importance), which health professionals thought was unexpected. Participants discussed the controversies among nephrologists in defining dialysis adequacy based on urea kinetics, Kt/V, and fluid, and how it could be reflected in a broad range of other outcomes including phosphate, quality of life, rehabilitation, vascular access. Also, they reiterated the incongruence between health professionals and patients in how they conceptualized dialysis adequacy: as one patient stated, “adequate means dialysis that’s adequate for patients and caregivers—‘to make me feel as normal as I can’, for doctors, it’s a reduction ratio.” Patients in the workshop conflated dialysis adequacy with a range of other outcomes including mortality, quality of life, fatigue, function, time with the family; one patient put it this way:

When you ask about adequacy, I think, what’s my quality of life once I’m leaving the dialysis session? During my treatments that day, am I going to feel well enough to continue on with my day, as far as the adequacy and my feeling well, or am I going to feel tired?

One patient challenged the term “adequacy” and urged to take “dialysis practice from adequate to rehabilitative.” Based on patients’ interpretations of dialysis adequacy, health professionals agreed that “the patient/caregivers’ number one outcome is not really dialysis adequacy, but wellbeing.” Another interpretation was that dialysis adequacy simply meant that dialysis was working properly and would instinctively be regarded as fundamentally important. The imprecision and “complexity” in defining dialysis adequacy meant that it would be too difficult to gain consensus on the definition, and thus potentially warranted exclusion as a core outcome.

Disentangling Interdependency

Participants noted challenges arising from the apparent “overlap,” “association,” or “conflict” between outcomes. As one health professional expressed it, they questioned whether “by speaking about one item, does that actually reflect another item?” Some perceived outcomes related to quality of life were difficult to disentangle as they clustered together; explained one patient:

I could summarize this as the desire to live as normal a life as possible, but you’ve got snippets, it’s fragmented. So work is part of that, travel, family, friends, are all fragmented, so not one aspect of those can rise to the top. If those were combined, they might be the most important thing.

For clarity of meaning and feasibility, core outcome domains need to be explicitly disaggregated or if this is not possible, to be transparent about which outcomes overlap and their associations. Some suggested that outcome domains should remain as a single attribute outcome, rather than a multi-attribute outcome such as quality of life or well-being. Mortality was believed to be implicit in other outcomes and in principle should be included as an outcome domain in the core outcome set.

Some speculated that the prioritization of outcomes was influenced by people’s level of knowledge and their confidence in the strength of the associations and causal pathways between outcomes, particularly between surrogate markers and clinical/quality of life end points. Health professionals suggested that patients may prioritize outcomes based on the extent they were educated about the “consequences” of certain outcomes, as illustrated in the following statement:

perhaps a drop in blood pressure is not important if we just asked patients, but if you ask us [nephrologists] we would say a drop in blood pressure is very important, because we know that that’s going to affect your brain function, your heart function, and ability to function. I wonder if some of these things are just a difference in perception, of what leads to what.

Some argued the need to consider and prioritize the “fundamental outcome” that triggered a cascading effect on other outcomes, for example one health professional made the case that:

if we prevent cardiovascular and heart disease, we prevent hospitalization for let’s just say myocardial infarction, then by logic, the patient should be feeling better. If you are able to improve one thing, you should be able to improve the other.

Some outcomes were, as pointed out by a health professional, “not mutually exclusive and may be conflicting.” For example, workshop participants noted that ability to travel and dialysis-free time directly conflicted with dialysis adequacy and that fatigue potentially conflicted with dialysis-free time. They suggested that core outcomes should, in the words of one health professional, “reflect real achievable goals.”

Interventional Applicability

As clinical trials usually evaluate a specific intervention, some participants asserted that triallists should not be “forced to measure outcomes that are irrelevant to the trial,” as argued by a health professional. They perceived that it might be challenging to establish core outcomes that would be applicable to the range of interventions for patients on hemodialysis including those targeting cardiovascular morbidity and mortality, vascular access, and quality of life and symptom control.

Procedural Efficiency

Participants agreed that it would be impossible to measure the plethora of outcomes within a trial in hemodialysis, and some believed that the SONG-HD consensus based process was effective and valuable for distilling the range of outcomes to a set based on the shared priorities of stakeholders, as expressed by one health professional:

that’s the beauty about the process—we have this smorgasbord of different outcomes but at the end of the day we can’t measure everything. It’s a matter of focus, it’s a matter of efficiency and this process really helps us. On one hand, of course, focus on what was ranked extremely highly. Also there’s a good list of outcomes that people have considered that really are not important and just getting that noise out of the entire system is already a good contribution.

Some suggested outcomes that had not been included or identified as a priority in the Delphi survey as less than 10% of participants on the Delphi panel had submitted it as a new outcome (e.g. residual kidney function as mentioned as an important quality indicator). However, all suggestions for outcomes would be explicitly included in the research agenda depicted in Figure 1.

Implementation

Feasibility and Validity of Outcome Measures

Feasibility

Participants in all breakout groups emphasized that core outcomes would require simple and inexpensive outcome measures in order to be effectively implemented in trials. “Data costs money,” stated one health professional, “and so simple outcomes like mortality are cheap and affordable, but other outcomes may actually involve some cost and complexity of measurement.” As an example, they regarded lengthy patient-reported outcome measures as—in the words of a health professional–“obtrusive and burdensome.” Some discussed the increasing focus on pragmatic clinical trials and maintained that it would only be feasible to include outcome measures that, as described by one health professional, “can be done day-to-day, that doesn’t really require a lot of data collection by research coordinators, things that are already available in the unit.” Moreover, some suggested that the outcomes should first be measured in routine clinical practice prior to implementing them in the trial setting to ensure feasibility.

Validity

The participants mentioned that core outcomes and outcome measures would also need to be validated. Speaking on this theme, one health professional explained:

My own bias is not to leap in there with clinical trials of that particular outcome but to include it in an observation data set so at least you can generate some hypotheses around it, look at them in a meaningful way. I noticed that these are just a list of things that were in clinical trials but it doesn’t discriminate about whether they were successful in clinical trials or whether they were like clinical trialed and then it’s gone.

Some considered that patient-reported outcome measures must be carefully selected or developed in view of the “possibility that the patient is not being accurate with you and the questions you ask—that’s when you have a flawed set of data,” a perspective offered by one health professional.

Propagating a Patient-Centered Paradigm

The SONG-HD Initiative was viewed as contributing to a paradigm shift in hemodialysis research by identifying outcomes important to patients, and regarded as an effective mechanism for “bringing the patient voice to the table,” in the words of one patient. The major mismatch between health professional and patient priorities for outcomes in the Delphi survey was “eye-opening,” which for patients demonstrated the importance of “learning from both sides” and underscored a need to “reconcile the people with the numbers.” Some advocated that patient-important outcomes should be regarded and used as a starting point; in the words of one health professional:

we don’t have the trial [to give people the best possible life]. We’re going to talk for another 10 years about how we’re going to do this when maybe we should start publishing data of what patients really want to have and bring this out. Then these so-called soft outcomes will be the hard outcomes and we influence it from that angle.

Participants expected that SONG-HD would garner interest and support from policy and regulatory agencies and industry with a shared interest and goal to identify and integrate patient preferences in research and clinical care; noted one health professional, “what everyone is looking for is making patients a much bigger part of all of this.” They suggested active dissemination of this information to KDIGO, which, as explained by one health professional:

is a worldwide authority and does this clinical trial design workshop. At the moment, they are going in a totally different direction. They are more on the drug side, make trials bigger and KDIGO has to look at other outcomes and bring this to the front, that other outcomes are more important than this. You mentioned outcomes which are on the patient side, and I think that politicians and funding bodies listen to patients and they will provide money and then these trials and new information can become available.

Some suggested that it would be important for regulatory agencies to approve drugs or devices based on the core outcomes as part of the SONG-HD implementation strategy.

Contextualizing Translation of Outcomes

The different contexts for implementing core outcomes were identified by some participants; in the view of one health professional, “one is clinical studies and how you standardize those; but the other is actually the carrot to get people to practice and treat people in certain ways, the incentives that are out there.” Also, some questioned whether there might be different implications for establishing a core outcome set “to drive the research agenda by choosing which outcomes are important” as one health professional explained it, or “to ensure that whatever the research agenda is, that the outcomes that are in it are reasonably standardized across different trials with similar goals,” Also, some suggested that core outcomes could potentially be used to support the shift to outcome based commissioning in the health sector.

DISCUSSION

Overall, the workshop participants and contributors supported the principle of establishing a set of core outcomes to optimize the relevance and value of research to guide decision-making in hemodialysis. The SONG-HD process was deemed valuable in focusing attention on outcomes that were regarded as important across all stakeholder groups, and in revealing important discrepancies in how outcomes were prioritized and conceptualized between patient/caregivers and health professionals. The workshop participants and contributors also identified several challenges and recommended that establishing core outcomes in hemodialysis required: equitable stakeholder engagement to ensure relevance across different patient populations; flexibility to consider evolving priorities over time; deconstruction of language and meaning for conceptual consistency and clarity; separation of attributes (i.e. single attribute outcome) to avoid overlap; acknowledgement of potential associations between outcomes; and an assessment of applicability to the range of interventions in hemodialysis. For implementation, they proposed that core outcomes must have simple, inexpensive and validated outcome measures, which can be used in the context of clinical care (quality indicators) and trials (including pragmatic trials), and be endorsed by regulatory agencies.

Some of these recommendations are similar to those put forward by OMERACT and the COMET initiative on developing core outcome sets for clinical trials, which also highlighted the need for broad and diverse stakeholder involvement, periodic review and update of core outcomes, and development of valid and feasible outcome measures (11, 16). However, this workshop identified additional challenges and recommendations in the context of establishing and implementing core outcomes in hemodialysis; particularly with regards to ensuring clarity and consistency in defining specific outcomes (e.g. dialysis adequacy, vascular access complications), and application to different trial and clinical contexts.

The workshop recommendations (Box 1) will be integrated into the finalization of the core outcome domains for hemodialysis, the subsequent development of core outcome measures, and implementation strategies for translating the core outcomes into hemodialysis trials and other forms of research. We believe that this will foster acceptance and optimize the uptake and translation of core outcomes in hemodialysis research, for better treatment and patient outcomes in hemodialysis.

Supplementary Material

Supplementary Item S1 (PDF). Facilitator question guide for breakout discussion.