Health Outcome Prioritization to Elicit Preferences of Older Persons with Multiple Health Conditions

Abstract

Objective

To develop and test a simple tool to elicit the preferences of older persons based on prioritization of universal health outcomes.

Methods

Persons age ≥ 65 participating in a larger study were asked to rank 4 outcomes on a visual analogue scale: 1) maintaining independence, 2) staying alive, 3) reducing/eliminating pain, 4) reducing/eliminating other symptoms.

Results

Interviewers rated 73% of the 81 participants as having good to excellent understanding, and cognitive interviews demonstrated the tool captured how participants thought about trade-offs. Test-retest reliability was fair to poor for ranking most of the outcomes as either most or least important (kappa .28-1.0). Patient characteristics associated with ranking “keeping you alive” as most important have been shown to be associated with a preference for life-sustaining treatment, a related construct. There was substantial variability in the outcome ranked as most important.

Conclusions

The task of ranking 4 universal health outcomes was well understood, captured what was important when considering trade-offs, and demonstrated content validity. However, test-retest reliability was fair to poor.

1. Introduction

Over 40% of older Medicare beneficiaries have at least 3 chronic conditions, and 24% have four or more (1). There is growing recognition that persons with multiple morbidities may be at increased risk of adverse consequences of medical therapy (2, 3). These consequences have traditionally been thought of as “side effects.” However, studies demonstrate that these adverse effects are outcomes that are as important to older persons as the beneficial effects of medications (4, 5). The choice of appropriate therapy depends upon understanding older persons’ attitudes toward the trade-offs among these competing outcomes.

Decision aids have proven very useful in helping patients to clarify their preferences for specific decisions involving trade-offs among alternative treatment options (6). However, for persons with multiple conditions, it may not be feasible to utilize an aid for each decision involving a trade-off. In addition, decision aids do not address the problem that different disease-specific outcomes are difficult to compare, such as the risk of stroke versus the risk of bleed in warfarin therapy for atrial fibrillation.

One approach to facilitate the comparison of disease-specific outcomes is to map their effects onto a universal set of outcomes, such as survival and function. Although there are a number of existing approaches to elicit values for these outcomes, including decision analytic approaches (7) and multiattribute utility theory (8), these methods are complex and have not been widely adopted in clinical practice. The purpose of this study was to determine the feasibility of using a simple tool to elicit the preferences of older persons based on their prioritization of universal outcomes.

2. Methods

2.1. Development of outcome prioritization instrument

Instrument development was based on the results of qualitative research elucidating the universal outcomes considered by older persons when faced with competing outcomes: length of life, physical, cognitive, and social functioning, and improving symptoms, such as pain, shortness of breath, and depression (5): The instrument asked participants to rank a set of outcomes. Prioritization between pairs of outcomes is a well accepted technique used to elicit preferences in decision support tools (9, 10). We elected to have participants prioritize a complete set of outcomes rather than doing pairwise comparisons so that they could think about all of the outcomes relative to one another and potentially avoid the internal inconsistencies that arise with the latter (10). Pilot testing revealed that participants understood the concept of prioritization but became overwhelmed when asked to consider more than four outcomes. The outcomes in the tool were: 1) maintaining independence, 2) reducing/ eliminating pain, 3) keeping you alive, 4) reducing/ eliminating symptoms: dizziness, fatigue, shortness of breath. Participants were asked to place these outcomes, each presented in a moveable box on a computer screen, onto a visual analogue scale. A picture of the computer screen is provided in Figure 1. This exercise was preceded by an introductory script explaining the concept of competing outcomes (Appendix).

Figure 1.

Screen Shot of Prioritization Tool

2.2. Participants

Participants were recruited from members of a study examining the priority older adults with hypertension and fall risk give to optimizing cardiovascular versus fall- and medication symptom-related outcomes (11). Of the 123 participants in the parent study, 81 consented to completion of the outcome prioritization tool. Participants who did not complete the tool were significantly more likely to be non-white (40% versus 18%, P < .001) and have no more than a high school education (59% versus 37%, P = .003) than participants completing the tool. The two groups did not differ significantly according to age, gender, or number of chronic conditions.

2.3. Data collection

Participants were interviewed in their homes. The instrument was administered either as part of the parent study or by itself by trained research assistants. The script was read to participants, and the research assistants could help participants move the boxes on the screen. Participants were also asked questions regarding sociodemographic status (age, gender, education, ethnicity, marital status, household income), health status (chronic conditions, medications, depressive symptoms defined as a score of ≥ 2 on the Physician Health Questionnaire-2 (12), functional status measured using the instrumental activities of daily living scale (13), and healthcare utilization).

The interviewers were asked to rate their global perception of participants’ understanding on a 5-point scale, ranging from “did not understand at all” to “understood completely.” Although this was a single subjective rating, it was based on a number of criteria applied across interviews, including need to repeat directions, number of questions asked by participants, and ability of participants to explain their rankings to the interviewer.

2.4. Analysis

We utilized univariate statistics, including proportions and means, to describe the participants and their outcome rankings.

Test-retest reliability was assessed by re-interviewing the 21 participants residing at a single housing site in their homes one week after their initial interview. We examined responses categorized as the proportions of respondents who selected a given outcome as most important and as least important versus all other outcomes using the kappa statistic.

Because of the absence of a “gold standard” against which criterion validity could be measured, we used cognitive interviewing to examine face validity. The first 10 participants were asked to talk aloud as they completed the task. These interviews were audiotaped, transcribed, and analyzed using content analysis (14) for evidence that the rankings chosen by the participants reflected which outcomes were most and least important to them.

Construct validity was assessed by examining the association between ranking “keeping you alive” as most important and patient characteristics that have been shown to be associated with a preference for receiving life-sustaining treatment, a closely related concept. These include male gender (15), non-white race (15-17), younger age (18), lower socioeconomic status (15), being unmarried (16), depression (15, 19), and poorer functional status (19). These relationships were examined in bivariate analysis, utilizing chi-square, Fisher exact, and Wilcoxon sign-rank tests as appropriate.

3. Results

3.1. Description of participants and their outcome priority rankings

A description of the 81 participants is provided in Table 1. Maintaining independence was the outcome ranked as most important with greatest frequency, but there was considerable variability in the outcomes ranked as both most and least important.

TABLE 1.

Description of 81 Participants and Their Outcome Priority Rankings

Characteristic	Value
Female, %	69
White, %	82
Hispanic, %	3
Age, %:
65-74	16
75-84	54
85+	30
Married, %	31
Education (years) , %
<= 8	6
9-12	31
>=13	63
Income ($), %
<10,000	9
10,000-19,999	25
20,000-39,999	33
>40,000	33
Hospital admission in past year, %	27
Nursing home admission in past year, %	10
Depressed, %	43
Requires assistance with ≥ 1 IADL, %	41
Chronic conditions, mean (SD)	2.9 (1.1)
Medications, mean (SD)	8.6 (3.8)
Most important outcome, %
Keeping you alive	27
Maintaining independence	42
Relief of pain	21
Relief of other symptoms	10
Least important outcome, %
Keeping you alive	36
Maintaining independence	5
Relief of pain	27
Relief of other symptoms	32

IADL= instrumental activity of daily living

3.2. Psychometric properties

The cognitive interviews performed with the first 10 participants demonstrated that their prioritization of outcomes reflected the ways in which they thought about the different outcomes. One woman who ultimately ranked “maintaining independence” as most important reasoned:

In other words, what I am trying to say is maintaining my independence is a lock for me, but I can’t do it if I am not alive, so I mean I don’t know.... If I can’t keep my independence, then, I am not good to myself or anyone else, and I wouldn’t want to live, let’s put it that way.

Understanding of the instrument was rated as a 4 or a 5 for 73% of participants. However, test-retest reliability was fair to poor (Table 2). Of the changes in rank, 50% were by one rank order and 50% were by more than one.

TABLE 2.

Test-Retest Agreement and Interviewers’ Assessment of Participant Understanding

Scale rank	% agreement	Kappa
Keeping you alive
Most important vs other	64	.42
Least important vs other	48	.36
Maintaining independence
Most important vs other	45	.32
Least important vs other	100	1.0
Reducing pain
Most important vs other	52	.38
Least important vs other	61	.44
Reducing other symptoms
Most important vs other	52	.30
Least important vs other	48	.28

Level of understanding (1= did not understand at all – 5=understood completely	%
5	30
4	44
3	14
2-1	13

The relationships between participant characteristics and ranking “keeping you alive” as most important were, with the exception of gender, in the hypothesized direction to support construct validity (Table 3).

TABLE 3.

Participants Ranking Keeping You Alive as Most Important Outcome according to Sociodemographic and Health Characteristics

Characteristic	Keeping you alive most important outcome (N=22)	P-value
	%
Gender
Male (n=25)	32
Female (n=56)	25	.51
Race
White (n=66)	24
Non-white (n=15)	40	.33
Marital status
Married (n=25)	12
Other (n=56)	34	.04
Education
≤ 8 years (n=5)	40
9-12 years (n=25)	44
≥ 13 years (n=51)	18	.04
Income
< $10,000 (n=13)	46
10,000-19,999 (n=21)	38
20,000-49,999 (n=24)	13
≥ $50,000 (n=15)	13	.05
Depression
Present (n=34)	38
Absent (n=46)	20	.06
Functional status
Requires assistance with ≥ 1 IADL (n=33)	36	.12
Independent in all IADLs (n=48)	21

IADL = instrumental activity of daily living

4. Discussion and Conclusion

4.1. Discussion

The results of this study suggest that older persons can complete a tool asking them to rank the relative importance of 4 universal health outcomes: length of life, maintaining function, relief of pain, and relief of other symptoms when considering trade-offs among competing outcomes. There was substantial inter-individual variability among the rankings.

Although participants’ understanding of the tool appeared good, test-retest reliability was fair to poor. It is difficult to know whether this finding represents limitations in the instrument or true changes in participants’ priorities. Few preference assessment tools have reported test-retest reliability; therefore, there is no established evidence base for the reliability generally achieved by such tools. It is possible that participants had not previously thought about these outcomes, such that some of the change represents a process of reflecting upon attitudes and values. It is also possible that further changes in the tool may improve participants’ understanding of trade-offs. For example, the interviewers observed that participants frequently conceptualized the outcome of “keeping you alive” as being alive in their current state of health. It may be helpful to present each of the outcomes in a way that helps participants consider the potential trade-offs, such as “keeping you alive, regardless of your level of function or symptoms.” The absence of good test-retest reliability is an important limitation of the tool, and the reason(s) underlying this finding require additional exploration.

Understanding older persons’ priorities regarding universal health outcomes may help to inform the many decisions involving trade-offs among competing outcomes faced by persons with multiple conditions, as interventions designed to increase life expectancy may have adverse effects on function and symptoms (20).

4.2. Conclusion

In preliminary testing, the outcome priority assessment tool is understood and captures what it important to older persons when faced with trade-offs. However, its test-retest reliability was fair to poor.

Appendix: Script accompanying computer tool

Often we are faced with different situations where we have to make a choice and 1 choice can be better in 1 way and worse in another. This is called a trade off. Many people have had to make a decision that involves a tradeoff for their health. The exercise we are going to do has to do with your healthcare.

Lots of medical decisions have trade-offs. Think about the decision whether or not to take a pill for high cholesterol which could lower your risk of future heart disease.. For some people, these pills can cause lots of side effects, like aches and pains in the legs. People who have side effects have two choices, each has different advantages and disadvantages. They can continue to take the medicine. This would mean they would have to live with the side effects, but they would be doing something to lessen their risk of a heart attack or stroke in the future. They can stop the medicine. This would mean that they would feel better right away, but they would not be lessening their risk of a heart attack or stroke in the future.

People facing this decision need to think about whether it is more important to feel as best as they can today or to lessen the chances of something happening to them in the future.

Can you think of an example when you had to make a decision that involved a trade-off? In other words, you had two choices, one of which was better in some ways and the other in other ways? This could be a decision about your health, but it could also be a decision about some other area of your life, for example a job, living situation etc.

A pill to treat a disease that may help you to live longer may also cause you symptoms that make you feel bad. Or, a pill that helps you keep your ability to do your daily activities might increase the chance of your getting a disease down the road that could shorten your life.

Like medications with side effects a lot of medical decisions need to be thoroughly considered and while doing this you need to think about what is most important to you.

When you think about what doctors are trying to do when they treat you, regardless of your specific conditions, the doctor is probably trying to provide you with one or more of the following outcomes:

1. Keeping you alive for as long as possible, regardless of what kind of health you are in.

2. Maintaining your current ability to do your daily activities.

3. Reducing or eliminating symptoms that can make you feel bad, such as dizziness, shortness of breath, and/or fatigue.

4. Reducing or eliminating pain.

I would like you to do an exercise to think about which of these 4 outcomes are most important to you. On the computer screen there are 4 boxes that have these outcomes written on them. I’d like you to tell me where you would place each box on the scale. “100” means that the outcome is the absolutely most important to you and you would be willing not to have any of the other outcomes in order to achieve this most important one. “0” means that the outcome has absolutely no importance to you. The only rule is that you can’t put two boxes on the same number. Where you put the box should show how willing you are to give up one thing in order to get another thing. For the outcomes of reducing or eliminating pain and other symptoms, I want you to think about what would be most important to you if you were having pain or other symptoms such as dizziness, fatigue, or shortness of breath.

Take your time with this. You can move the outcome boxes after you have placed them. Do you have any questions? Let’s begin.

Conversation during exercise

Remember – by putting one box on a higher number than a second, it means that you would be willing to give up some of the second to get the first. For example, if you put “Keeping you alive no matter what” higher than “Maintaining your current ability to do daily activities,” that means you would be willing to be more dependent in your daily activities to stay alive longer. If, on the other hand, you put “Maintaining your current ability to do daily activities,” higher than “Keeping you alive no matter what,” it means that you would be willing not to live as long if you could continue to do your daily activities the way that you do them now.