Moving the Science of Patient-Reported Outcome Measures Forward

Although almost everyone knows the feeling of fatigue, few people experience it as often as patients with ESKD on maintenance hemodialysis (HD). A majority of HD patients report feeling fatigued on a regular basis (1), affecting their ability to accomplish the tasks they want and need to do, including work, caring for family, and pursuing personal aspirations (2). Although fatigue is associated with mortality in maintenance HD patients, many patients and caregivers say that relieving fatigue matters more to them than avoiding death (3).

The article by Ju et al. (4) in this issue of CJASN is grounded in recognition of the importance of fatigue relative to many other outcomes studied in nephrology. Ju et al.’s work stems from a larger effort known as the Standardized Outcomes in Nephrology (SONG) initiative, which aims to establish core outcomes and measures that reflect the priorities of key stakeholders in nephrology, including patients and caregivers. In this article, the authors report the findings of the first psychometric evaluation of a patient-reported outcome measure (PROM) for fatigue in HD patients. Put simply, a patient-reported outcome (PRO) is information about a patient’s health, quality of life, or functional status that is obtained directly from the patient; a PROM is an instrument for reporting a PRO (5). Chart review alone could reveal the “hard” outcomes that have dominated clinical trials in nephrology, such as survival, cardiovascular events, and hospitalizations, as well as surrogate outcomes, such as laboratory values. However, as the SONG-HD initiative has demonstrated, outcomes such as these are not the only ones that matter to patients (3). A Kidney Disease Improving Global Outcomes controversies conference acknowledged this fact when it recently recommended that future trials include PROMs as end points (6). In 2009, the US Food and Drug Administration issued guidance to the pharmaceutical and medical device industries describing how it will review and evaluate PROMs used to support claims in product labeling (7). This guidance, along with the Consensus-Based Standards for the Selection of Health Status Measurement Instruments (8), specifies that the key measurement properties of a PROM are reliability, validity, and responsiveness (Table 1).

Table 1.

Key measurement properties of patient-reported outcome measures

Measurement Property	Property Subtype	Description	Evaluation
Reliability	∙ Test-retest/intrainterviewer reliability (for interviewer-administered PROMs only)	∙ Stability of scores over time when no change is expected in the concept of interest	∙ Intraclass correlation coefficient
			∙ Time period of assessment
	∙ Internal consistency	∙ Extent to which items comprising a scale measure the same concept	∙ Cronbach’s α for summary scores
		∙ Intercorrelation of items that contribute to a score	∙ Item-total correlations
	∙ Interinterviewer reliability (for interviewer-administered PROMs only)	∙ Agreement among responses when the PROM is administered by ≥2 different interviewers	∙ Interclass correlation coefficient
Validity	∙ Content validity	∙ Evidence that the instrument measures the concept of interest. Includes evidence from qualitative studies that the items and domains of an instrument are appropriate and comprehensive for its intended measurement concept, population, and use	∙ Derivation of all items
			∙ Qualitative interview schedule
			∙ Interview or focus group transcripts
			∙ Items derived from the transcripts
			∙ Composition of patients used to develop content
			∙ Cognitive interview transcripts to evaluate patient understanding
	∙ Construct validity	∙ Evidence that relationships among items, domains, and concepts conform to a priori hypotheses of logical relationships that should exist with measures of related concepts or scores produced in similar or diverse patient groups	∙ Strength of correlation testing a priori hypotheses (discriminant and convergent validity)
			∙ Degree to which the PROM can distinguish among groups hypothesized a priori to be different (known-groups validity)
Responsiveness		∙ Evidence that a PROM can identify differences in scores over time in individuals or groups who have changed with respect to the measurement concept	∙ Within-person change over time
			∙ Effect size statistic

Adapted from ref. 7. PROM, patient-reported outcome measure.

Ju et al. present evidence that their PROM for fatigue in HD patients is strong in two out of these three properties. Named the SONG-HD Fatigue measure, this PROM consists of three simple questions about the patient’s experience of fatigue in the past week, including tiredness, energy, and limits on usual activities. Notably, these questions inquire about the aspects of fatigue that patients themselves identified as important. Respondents answer “not at all,” “a little,” “quite a bit,” or “severely” on a Likert scale of 0–3 to constitute a total score ranging between 0 and 9. In a sample of English-speaking adults from the United Kingdom, Australia, and Romania, the authors found that the SONG-HD Fatigue measure is reliable, meaning that with repeated completion by the same person in the same condition, it is likely to yield similar results (9). The intraclass correlation coefficients of 0.68–0.74, depending on country, demonstrate test-retest reliability, which indicates that patients’ responses did not significantly change between the first administration of the scale and repeat administration 1 week later. The scale also displayed adequate internal consistency, as denoted by a Cronbach’s α of 0.81–0.86, showing that the items all measure the same variable: fatigue.

In addition to its reliability, the authors found that the SONG-HD Fatigue measure is valid, meaning that it measures what it intends to measure (9). The SONG-HD initiative’s comprehensive, inclusive process for establishing the questions for this scale, summarized in the introduction of the article, ensured its content validity; the items included are appropriate for measuring fatigue. The scale’s construct validity, or the extent to which it behaves like other measures of fatigue (10), is evident in its significant correlations with other instruments that patients in the study also completed, namely, the 12-item Short Form Survey version 2, the Functional Assessment of Chronic Illness Therapy–Fatigue, and a visual analog scale for fatigue.

The most notable limitation to this study is that while it examined two key measurement properties of the SONG-HD Fatigue measure, reliability and validity, it did not examine the third, responsiveness, which is the scale’s ability to detect clinically meaningful change. The authors explain that they could not assess responsiveness because they did not know between which time points to anticipate a change in an individual’s fatigue level in the absence of intervention, and they did not have an intervention to offer. Another limitation relevant for American researchers is that, due to where this research was conducted, the racial and ethnic composition of the study sample—very few Black patients and no Latin-American patients—does not reflect the racial and ethnic composition of the maintenance HD population in the United States. Given that mean scores on the scale significantly differed by country, it is possible that mean scores could significantly differ by racial or ethnic background of the respondents within a country. However, we emphasize that even if one group rates their fatigue as worse than another group, the scale can still be reliable and valid in both groups. This study proves that point; although patients in the United Kingdom scored their fatigue higher than patients in Australia or Romania, the scale’s reliability and validity parameters were similar in all countries. The authors do state that one of their future goals is testing the SONG-HD Fatigue instrument’s crosscultural validity.

The brief, reliable, valid PROM for fatigue in maintenance HD patients that Ju et al. unveil has its roots in research that began in 2014 (3). The extensive effort spread over several years that is required to develop a sound PROM, along with the ready availability of non-PROs, perhaps explains why there are still so few PROMs included in nephrology trials. The question is, now that Ju et al. have delivered a well-tested instrument in the SONG-HD Fatigue measure, will there be trials to use it? Although this measure will no doubt have a role in observational studies, the stated goal of the authors is to “facilitate the uptake of [fatigue as a] core outcome in trials.” At this time, trials including fatigue as a core outcome are still scarce. A search of ClinicalTrials.gov in September 2020 using “ESKD on dialysis” as the condition of interest and “fatigue” as another term turned up two active trials evaluating fatigue as an outcome, in both cases secondary. Only one of these trials, which aims to recruit a total of ten patients, is located in the United States. Given the multifactorial complexity of fatigue in maintenance HD patients, it is possible that only a multicomponent intervention will adequately address it. Although many barriers to developing and testing such an intervention still exist, because of the work of Ju et al., the lack of a validated PROM is no longer one of them.

While we await an intervention, clinicians can inquire whether their patients on maintenance HD are experiencing fatigue and, if so, how much of a priority is it for them to attend to the fatigue versus other issues. In some cases, treating fatigue might entail trade-offs, such as spending more time on dialysis. Presented with these trade-offs, people will make different decisions. The work of the SONG-HD initiative reminds us that we cannot assume we know what matters to our patients; we must ask them and then listen closely to their answers. Just as the SONG-HD initiative has done this with individuals and groups for research purposes, nephrologists can do it with each and every patient in the clinical setting. Even when interventions are available, we cannot intervene upon that which we do not recognize as a problem. Patients on maintenance HD have been very clear that fatigue is an important problem for them. Investigators such as Ju et al. have heard them. Now it is time for all us, clinicians and researchers alike, to do the same.

Disclosures

J.S. Scherer reports receiving honoraria from Cara Therapeutics and serving as scientific advisor or holds membership with Cara Pharmaceuticals. She received fees for attending an advisory group to develop a drug for uremic pruritis. The remaining author has nothing to disclose.

Funding

S.J. Ramer is supported by National Institutes of Health award T32 AG049666. J.S. Scherer is supported by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under award number K23DK125840 and a National Kidney Foundation Young Investigator Award.