Comparing Pretransplant and Posttransplant Outcomes When Choosing a Transplant Center: Focus Groups and a Randomized Survey

Abstract

Background.

In response to calls for an increased focus on pretransplant outcomes and other patient-centered metrics in public reports of center outcomes, a mixed methods study evaluated how the content and presentation style of new information influences decision-making. The mixed methods design utilized qualitative and quantitative phases where the strengths of one method help address limitations of the other, and multiple methods facilitate comparing results.

Methods.

First, a series of organ-specific focus groups of kidney, liver, heart, and lung patients helped to develop and refine potential displays of center outcomes and understand patient perceptions. A subsequent randomized survey included adult internet users who viewed a single, randomly-selected variation of 6 potential online information displays. Multinomial regression evaluated the effects of graphical presentations of information on decision-making.

Results.

One hundred twenty-seven candidates and recipients joined 23 focus groups. Survey responses were analyzed from 975 adults. Qualitative feedback identified patient perceptions of uncertainty in outcome metrics, in particular pretransplant metrics, and suggested a need for clear guidance to interpret the most important metric for organ-specific patient mortality. In the randomized survey, only respondents who viewed a note indicating that transplant rate had the largest impact on survival chose the hospital with the best transplant rate over the hospital with the best posttransplant outcomes (marginal relative risk and 95% confidence interval, _1.161.50_1.95).

Conclusions.

The presentation of public reports influenced decision-making behavior. The combination of qualitative and quantitative research helped to guide and enhance understanding of the impacts of proposed changes in reported metrics.

INTRODUCTION

Recently, there have been calls for an increased focus on pretransplant outcomes and other patient-centered metrics in public reports of transplant center outcomes.^1-5 Best practice guidelines for healthcare reporting reinforce the need to not only provide information but help users interpret information effectively in decision-making.^6,7 For a transplant candidate (hereafter referred to as candidate), the feasible number of choices for transplant centers (hereafter referred to as centers) vary by region, organ type, and individual circumstances; however, many patients report having multiple choices available,⁸ chose centers far from their homes^9,10 or list at multiple centers.¹¹ Currently, public reports can be used to compare pretransplant and posttransplant outcomes, for example waitlist mortality, transplant rate,¹² and posttransplant organ survival.¹³ The relevance of outcome metrics is organ-specific, and some center outcomes are associated with future patient mortality.^14-16 There is a potential need for patients to consider trade-offs between pretransplant and posttransplant outcomes, or any future quality metric, when deciding on a center. Existing studies on patient priorities have used only abstracted center information such as text-based surveys^8,17 and do not test decisions when viewing data. There is little understanding of how the content and presentation style of current and new information will influence decision-making.

A mixed methods approach combines qualitative (eg, focus groups) and quantitative (eg, randomized trial) methods and has been utilized in related fields to improve the usability of public healthcare reports,^18-20 such as reports to aid in choosing a hospital.²¹ This approach can be an advantage over the use of a single method,^22,23 for example, reporting if multiple methods produce consistent findings. These prior studies in related fields demonstrate significant effects of presentation style and content on decision-making, suggesting a need to evaluate reports specifically relevant to organ transplantation.

A better understanding of decision-making can inform how information is presented to patients and will be an important consideration as new quality metrics are evaluated. This study first used focus groups of transplant patients to develop and refine potential reports of center outcomes and understand patient perceptions when viewing information. A subsequent randomized survey of nonpatients was used to evaluate the effects of graphical presentations of information on decision-making. The 2 combined study phases provided a feasible approach to understand how information about centers can change patient perceptions and also statistical differences in decision-making.

MATERIALS AND METHODS

First, in-depth patient feedback informed iterative development of new graphics, content, and survey instruments. Second, evaluating multiple graphical presentations to identify potential impacts on decisions required a large sample that is not feasible when recruiting patients. Both study phases were approved by the human subjects research review board at each recruiting site. Additional detail about software and statistical methods is provided in Supplemental Materials, http://links.lww.com/TP/B755.

Qualitative Phase: Focus Groups

The qualitative phase included focus groups of local and national adults (age 18 or older) stratified into candidates and recipients. Local participants were transplant candidates at Hennepin Healthcare (HHS) and the University of Minnesota-Fairview (UMN). National participants were recipients who were healthy enough to travel. All groups were organ-specific and included kidney, liver, heart, or lung transplant patients. Non-English speaking candidates were excluded. Each participant gave written, informed consent, and participants received a $40 stipend.

For each organ, candidate focus groups were conducted first and located at HHS and the UMN clinics in the Upper Midwestern United States. Inclusion criteria for a convenience sample were English speaking, on the transplant waiting list, or proceeding with an evaluation. Recruiting was conducted by research coordinators during or after a regularly scheduled transplant appointment or by mail.

National transplant recipient focus groups were conducted in Chicago, IL. Participant expenses were paid, including airfare and lodging. Participants were recruited through email and social media via national patient groups (eg, National Kidney Foundation). Inclusion criteria for national groups were English speaking and a previous transplant (or retransplant). National participants were purposively recruited to ensure there was at least 1 participant for each organ type from most Organ Procurement and Transplantation Network regions.

Focus groups were moderated by the same individual (C.R.S.), trained in qualitative research, and attended by a social sciences researcher (M.J.B.) and transplant physician (A.K.I). The focus groups were described as an opportunity to learn about patient experiences when choosing a center and discuss feedback about new graphics and information displays. A discussion guide included mockups and questions (Table S1, SDC, http://links.lww.com/TP/B755) based on a phenomenological approach to understand how patients choose a center.²⁴ The scope of the current analysis includes only discussions in which participants viewed mockups of public reports and described decisions based on the information provided. Discussions outside this scope, such as past experiences choosing a center, have been reported elsewhere.²⁵ Mockups were developed iteratively in response to feedback; therefore, some concepts with consistent negative feedback were replaced with new iterations during the qualitative phase. All mockups shown during discussions of trade-offs included realistic pretransplant and posttransplant metrics.

Focus groups were 90–120 minutes. All participants completed a demographics and comorbidity questionnaire. Discussions were audio recorded and transcribed verbatim. All transcripts were read and coded by one analyst (C.R.S.), and 4 focus groups were independently coded by a second analyst (W.T.M.). Each researcher (C.R.S. and W.T.M.) reviewed each other’s coding schemes and discussed discrepancies in applying codes. Discrepancies that were not resolved were reviewed by A.K.I. and M.J.B. to reach consensus. Transcripts were open coded and axial coded, and themes emerged through an inductive, thematic analysis.^24,26

Qualitative patient feedback about mockups informed the selection of several final design elements to test in a randomized survey. Each final design element was selected based on evidence of effectively helping patients interpret the content of reports (compared to other alternatives discussed in focus groups).

Quantitative Phase: Randomized Survey

Internet users from the United States were recruited online using Amazon Mechanical Turk (AMT) (www.mturk.com, Amazon.com, Inc. Seattle, WA) to evaluate potential concepts for new online reports. AMT is a platform to post short tasks to an online workforce. Any member of the AMT platform who met the following criteria was eligible to participate: US internet (IP) address, a 95% rate of accepted work on the AMT platform, and at least 50 completed tasks that did not include similar prior transplant surveys. Each participant consented to the survey electronically; written consent was waived because participants were anonymous. Participants received nominal payments (eg, $0.50) for the completion of the survey.

Surveys were provided in English. Survey participants answered 5 demographic questions and a series of questions related to organ transplantation, specifically to understand perceptions of risk (questions and response choices are shown in Table S2, SDC, http://links.lww.com/TP/B755). Survey questions were piloted in a previous randomized survey (before the development of a 5 tier transplant rate measure).²⁷ Each participant viewed a randomly assigned image depicting a mockup of transplant center search results using graphical elements similar to those used on recent Scientific Registry of Transplant Recipients search tools (www.srtr.org). Mockups depicted hypothetical centers and data using distance and transplant volume as well as 5 tier assessments of waitlist mortality, transplant rate, and 1-year organ survival. The hypothetical center choices were designed to minimize a bias towards posttransplant outcomes. In particular, the center with the highest 1-year organ survival was also the greatest distance, and choices were sorted with the highest transplant rate at the top. The survey tested the hypothesis that adding context to these mockups would reduce the bias towards 1-year organ survival described in focus groups and seen in previous pilot survey tests.

Three graphical elements were evaluated for effects on decision-making, and combinations of multiple elements were used based on a factorial study design with 2 levels for each factor (described in more detail in Table S3, SDC, http://links.lww.com/TP/B755) with equal allocation in each arm. A control was included in which all new elements were omitted. The graphical elements included (1) a data table to provide numerical values corresponding to each tier (eg, a typical rate for 1-y organ survival at a tier 3 center was 91%); (2) an impact statement where an icon was shown next to the transplant rate heading and accompanied by a statement: “For transplant candidates, this measure has the largest impact on survival after listing. 1-year organ survival includes only candidates who received a transplant”; and (3) an uncertainty statement below metric headings: “Note: Future outcomes may be different than past outcomes. Groups (number of bars) do not represent the outcome for each individual patient.” Figures 1-3 include representative examples of the 6 mockups. All mockups included 3 identical hypothetical center options, where each of the 3 centers was shown as having the best rating in only 1 of the 3 included metrics (eg, Lake Hospital had the best rating for transplant rate).

FIGURE 1.

Mockup #1 randomly shown to survey participants, representing a control image with new graphical elements omitted. Color survey image is shown in grayscale.

FIGURE 3.

Mockup #5 randomly shown to survey participants, including the new graphical elements of the data table and uncertainty statement. Color survey image is shown in grayscale.

Images were presented using a Qualtrics survey (www.qualtrics.com), which includes a function to randomize questions. After viewing a randomly assigned image, survey participants indicated which center they would choose based on the information shown. Participants also indicated which factor was most important in the decision (or “no single factor”). The target survey duration was 5–10 minutes. The survey for image mockups was available online between 27 February 2018 and 1 March 2018.

The primary outcome was the choice of center (see question #7, Table S2, SDC, http://links.lww.com/TP/B755). A multinomial regression evaluated the effect of the different presentations on the choice of center, because the outcome variable (ie, the selected hospital) had >2 unordered levels. Specifically, the analysis estimated the difference in relative risks (RRs) and marginal RR and 95% confidence intervals for selecting Alpine Hospital and Lake Hospital compared to Meadow Hospital (Meadow Hospital was depicted as the highest 1-y organ survival). The regression only included the main effects of the design elements because the likelihood ratio test for the interaction was not significant (P = 0.976) and adjusted for the perception of risk (response to question #6). See the Supplement for additional details on the methodology.

RESULTS

Qualitative Phase: Focus Groups

A total of 127 candidates and recipients participated in organ-specific focus groups: 62 kidney, 23 liver, 17 heart, and 20 lung (demographics shown in Table 1). Table 2 describes the number of local and national participants and time period for each organ type.

TABLE 1.

Characteristics of kidney, liver, heart, and lung transplant candidate and recipient focus group participants

	Local candidates			National recipients
	Kidney	Liver	Heart	Kidney	Liver	Heart	Lung
Transplant centers within 100 milesa, median (min, max)	5 (0, 5)	3 (3, 3)	3 (2, 3)	3 (1, 24)	2 (0, 7)	4 (1, 11)	1 (0, 5)
Age, mean (SD)	55.4 (11.0)	56.8 (12.4)	53.6 (13.9)	52.3 (13.3)	56.2 (15.5)	58.2 (13.0)	61.6 (8.7)
Sex, n (%)
Male	27 (60)	11 (85)	4 (80)	7 (41)	3 (30)	10 (83)	12 (60)
Race, n (%)
Black	13 (29)	1 (8)	0 (0)	3 (18)	0 (0)	3 (25)	1 (5)
White	29 (64)	10 (77)	5 (100)	13 (76)	8 (80)	9 (75)	16 (80)
Hispanic	1 (2)	1 (8)	0 (0)	1 (6)	0 (0)	0 (0)	3 (15)
Other	2 (3)	1 (8)	0 (0)	0 (0)	1 (10)	0 (0)	0 (0)
Education, n (%)
Less than high school	3 (7)	1 (8)	0 (0)	0 (0)	0 (0)	0 (0)	1 (5)
High school	7 (16)	2 (15)	1 (20)	1 (6)	1 (10)	0 (0)	2 (10)
At least some college	24 (53)	8 (62)	3 (60)	7 (41)	3 (30)	2 (17)	4 (20)
Graduate education	11 (24)	2 (15)	1 (20)	9 (53)	6 (60)	10 (83)	13 (65)
Annual household income, n (%)
<$15 000	4 (9)	4 (31)	1 (20)	1 (6)	0 (0)	0 (0)	0 (0)
$15 000–$30 000	13 (29)	3 (23)	1 (20)	3 (18)	1 (10)	0 (0)	2 (10)
$30 001–$45 000	3 (7)	2 (15)	0 (0)	3 (18)	0 (0)	0 (0)	1 (5)
$45 001–$60 000	5 (11)	0 (0)	2 (40)	1 (6)	1 (10)	1 (7)	2 (10)
$60 001 –$75 000	4 (9)	2 (15)	0 (0)	3 (18)	4 (40)	2 (17)	2 (10)
>$75 000	14 (31)	2 (15)	1 (20)	2 (12)	3 (30)	9 (75)	10 (50)
Declined to answer	2 (4)	0 (0)	0 (0)	4 (24)	1 (10)	0 (0)	3 (15)
No. of household members, mean (SD)	2.1 (1.2)	1.5 (0.5)	1.4 (0.5)	1.9 (1.0)	1.7 (0.5)	2.7 (1.0)	1.9 (0.54)
Form(s) of insurance, n (%)b
Private	21 (47)	7 (54)	4 (80)	12 (71)	6 (60)	8 (67)	13 (65)
Medicare	22 (49)	2 (15)	4 (80)	10 (59)	5 (50)	5 (42)	14 (70)
Medicaid	3 (7)	4 (31)	1 (20)	2 (12)	0 (0)	0 (0)	1 (5)
Not insured	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Other	4 (9)	3 (23)	0 (0)	2 (12)	1 (10)	2 (17)	3 (15)
Self-reported health status, n (%)
Excellent	1 (2)	0 (0)	0 (0)	3 (18)	2 (20)	3 (25)	1 (5)
Very good	10 (22)	3 (23)	2 (40)	8 (44)	5 (50)	3 (25)	11 (55)
Good	18 (40)	5 (38)	0 (0)	5 (29)	3 (30)	2 (17)	7 (35)
Fair	12 (27)	4 (31)	3 (60)	1 (6)	0 (0)	2 (17)	1 (5)
Poor	4 (9)	1 (8)	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Has had previous transplant, n (%)	7 (16)	1 (8)	0 (0)	17 (100)	10 (100)	12 (100)	20 (100)
Currently on the waiting list, n (%)
Yes	25 (56)	13 (100)	5 (100)	0 (0)	0 (0)	0 (0)	0 (0)
No	2 (4)	0 (0)	0 (0)	15 (88)	10 (100)	12 (80)	20 (100)
Not sure	4 (9)	0 (0)	0 (0)	0 (0)	0 (0)	2 (13)	0 (0)
No answer	14 (31)	0 (0)	0 (0)	2 (12)	0 (0)	0 (0)	0 (0)

Percentages may not equal 100 due to incomplete responses.

^aPostal codes provided were used to determine how many transplant centers were located within 100 miles (excluding “Children’s” hospitals and any center with 0 adult transplants in the previous y).

^bSome participants marked multiple options.

SD, standard deviation.

TABLE 2.

Summary of local and national focus group times and participants (number of focus groups)

	Time period	HHS candidates	UMN candidates	National recipients	Total participants
Kidney	6/2017–9/2017	22 (5 groups)	23 (4 groups)	17 (3 groups)	62 (12 groups)
Liver	7/2017–9/2017	N/A	13 (2 groups)	10 (2 groups)	23 (4 groups)
Heart	2/2018–5/2018	N/A	5 (1 group)	12 (2 groups)	17 (3 groups)
Lung	5/2018–5/2018	N/A	N/A	20 (3 groups)	20 (3 groups)
Total participants		22 (5 groups)	41 (7 groups)	64 (11 groups)	127 (23 groups)

HHS, Hennepin Healthcare; UMN, University of Minnesota-Fairview.

The development of mockups was informed by feedback on concepts with less explicit guidance about relevant risks and also recent research indicating organ-specific metrics were correlated to survival.^14,15 For example, transplant rate has the largest impact on survival from waitlisting for kidney, liver, and heart; 1-year organ survival has the largest impact for lung. Therefore, direct statements, such as the “impact statement” (Figure 2), were used in later focus groups and well received. Mockups not selected for the randomized survey included patient narratives, general descriptions of the importance of both pretransplant and posttransplant outcomes (without explicitly highlighting a single outcome metric), and alternate headings and definitions of outcome metrics.

FIGURE 2.

Mockup #3 randomly shown to survey participants, including the new graphical element of the impact statement. Color survey image is shown in grayscale.

Three themes are described, including supporting quotations. Themes reflect general patient perceptions rather than responses to individual mockups: (1) outcome metrics have uncertainty relative to individual experiences; (2) patients, in particular candidates, describe a focus on posttransplant outcomes; and (3) individual circumstances factor into decisions. Table S4, SDC, http://links.lww.com/TP/B755 includes additional quotations illustrating the main themes.

Theme 1: Outcome Metrics Have Uncertainty Relative to Individual Experiences

Some participants described the pretransplant outcome of transplant rate as having limited utility due to perceived high levels of uncertainty, often resulting from random chance and high variability for the wait time to receive a deceased donor organ. Waitlist mortality was described as not consistently within the control of a center. Although not as frequently discussed, uncertainty in posttransplant outcomes was similarly described as reflecting deaths that were unrelated to transplant or outside the control of a center.

“Because you can have the most phenomenal medical center and there can still be issues. It’s going to be each individual experience.” [Local kidney candidate]

“I always qualify to the patients that an average is just an average… In our area, the average wait time for a liver transplant is 1 year. I waited two and a half years. A colleague of mine waited 8 days.” [Liver national recipient]

“So the statistic is kind of blurred if it’s including someone that died of other causes like a bike accident, a car accident.” [National heart recipient]

“Surviving on a waitlist, I mean, that’s just the roll of the dice.” [Lung national recipient].

Theme 2: Patients, in Particular Candidates, Describe a Focus on Posttransplant Outcomes

Participant points of view differed when making trade-off decisions; however, candidates appeared to more commonly prioritize a center with high posttransplant outcomes, and recipient decisions appeared more mixed. Table S4, SDC, http://links.lww.com/TP/B755 includes additional quotes and also supporting quotes with a focus on pretransplant outcomes.

“The survival afterwards is pretty much more important if it takes 2 years or 10.” [Local kidney candidate]

“I was thinking about survival. Mine was all survival.” [National liver recipient]

“The only reason I’m getting the transplant is to last longer after the transplant.” [Local liver candidate]

“But I’m looking at the survival rate… If I go through the surgery and everything else, where am I going to be in a year?” [National heart recipient]

Theme 3: Individual Circumstances Factor Into Decisions

Participants commonly described how information viewed or used in a decision would be considered within the context of other constraints or experiences. Examples included how medical urgency, number of available options, potential for a living donor, and personal values would potentially be considered along with center outcome data to make a trade-off decision.

“It’s based on situation. I had a live donor, and I wouldn’t even look at the [transplant rate] column. For me, it took away my decision.” [National kidney recipient]

“It just depends how sick you are.” [National liver recipient]

“I think that’s for the individual to decide, and I think this is what helps them decide.” [Local heart candidate]

“It depends on your value system.” [National heart recipient]

Quantitative Phase: Randomized Survey

Two thousand thirty-seven participants were randomized, and the analysis set included 975 responses (Table 3). Exclusions included a programming error (n = 630), missed attention question (n = 68), incomplete responses (n = 66), and data for a pilot concept (n = 293). Randomization and details of these exclusions are shown in Figure S1, SDC, http://links.lww.com/TP/B755.

TABLE 3.

Characteristics of participants who completed the survey (questions #1–#5) and responses to perception of risk (question #6)^a

	All	Analysis set
Completed surveys, n	1673	975
Age, mean (min/max)	37.7 (18/84)	38.3 (18/82)
Sex, % (n)
Male	43 (836)	39 (480)
Education, % (n)
Less than high school	<1 (9)	<1 (5)
High school	10 (188)	9 (113)
At least some college	70 (1365)	69 (848)
Graduate education	20 (389)	22 (266)
Has had previous transplant, % (n)
Yes	3 (58)	1 (18)
Family members needed/received transplant, % (n)
Yes	28 (551)	27 (332)
Perception of risk
The risk of death is greater due to complications after transplant than due to waiting too long.	6% (112)	4% (55)
The risk of death is greater due to waiting too long than due to complications after transplant.	61% (1183)	60% (744)
The risks are about the same.	34% (658)	35% (434)

Figure S1, SDC, http://links.lww.com/TP/B755 includes a consort diagram with exclusions and randomization details.

^aSome participants left blank responses; therefore totals may not add up to 100%.

Viewing different mockups impacted decision-making, specifically the proportion of responses selecting each center (Table 4). The control image (mockup #1, Figure 1) resulted in 50% choosing Meadow (highest 1-y organ survival) and 37% choosing Lake (highest transplant rate). The Impact statement (mockup #3, Figure 2) resulted in the largest increase in the proportion that selected Lake, relative to the control; 33% selected Meadow and 51% selected Lake. Individual design elements used in the factorial design impacted decision-making, shown as the relative probability of selecting Alpine Hospital and Lake Hospital compared to Meadow Hospital (Table 5). The impact statement graphical element resulted in a 50% higher probability of selecting Lake Hospital, depicted as the highest transplant rate, compared to Meadow Hospital, depicted as the highest 1-year organ survival (RR, _1.161.50_1.95). Alpine Hospital (best waitlist mortality) was selected least frequently when viewing the control; however, viewing the data table increased the probability of choosing Alpine Hospital from 0.31 (Control) to 0.55 (Data table). In summary, using an impact statement to indicate that the transplant rate had the greatest impact on survival resulted in the greatest proportion of participants selecting the center with the highest transplant rate. Similarly, respondents in this group were more likely to answer that the most important factor in a decision was Getting a Transplant Faster. The results for factors rated as most important are summarized in Table S7, SDC, http://links.lww.com/TP/B755.

TABLE 4.

Proportion of participants in the analysis set selecting each program after viewing a randomly assigned search tool mockup

Hypothetical program	Dominant metric	Responses to question #7 for each search tool mockup (What hospital would you choose for a transplant based on the information shown?)
		#1: Control (%)	#2: Uncertainty statement (%)	#3: Impact statement (%)	#4: Data table (%)	#5: Data table and uncertainty statement (%)	#6: Data table and impact statement (%)
Alpine Hospital	Waitlist survival	13	13	16	18	20	24
Lake Hospital	Transplant rate	37	42	51	37	40	45
Meadow Hospital	1 y organ survival	50	44	33	44	39	31
Count		169	165	159	163	161	158

TABLE 5.

The effect of the graphical elements for all participants in the analysis set on selecting Alpine or Lake Hospital compared to Meadow Hospital, which had the best tier rating (dominant metric) for 1 y Organ Survival

Metric	Selected program	Dominant metric	Control	Graphical elements
				Uncertainty statement	Impact statement	Data table
Difference in relative risks	Alpine Hospital	Waitlist survival	–	1.20 (0.77-1.88)	1.92 (1.23-3.00)	1.66 (1.15-2.39)
	Lake Hospital	Transplant rate	–	1.30 (0.93-1.82)	1.93 (1.36-2.72)	1.05 (0.79-1.39)
Marginal relative risks	Alpine Hospital	Waitlist survival	0.31 (0.19-0.45)	0.40 (0.29-0.55)	0.63 (0.46-0.86)	0.55 (0.43-0.69)
	Lake Hospital	Transplant rate	0.59 (0.42-0.81)	0.99 (0.78-1.25)	1.50 (1.16-1.95)	1.07 (0.87-1.30)

The difference in relative risks metric was the relative change in the probability selecting, eg, Alpine Hospital vs Meadow Hospital. In contrast, the marginal relative risk metric was the ratio of the probabilities for selecting, eg, Alpine Hospital compared to Meadow Hospital with the different graphical element. Parentheses contain the 95% confidence interval.

DISCUSSION

The focus groups informed the development of concepts used in the randomized survey in a number of important ways. Local focus groups were conducted first, and these candidates often described a priority for posttransplant outcomes (see Theme 2 and Table S4, SDC, http://links.lww.com/TP/B755). The impact statement was included in survey options to quantity effects on a bias towards posttransplant outcomes.

Many participants described potential sources of uncertainty in the outcomes displayed (see Theme 1 and Table S4, SDC, http://links.lww.com/TP/B755). Guidelines for public healthcare reports suggest omitting confidence intervals on public reports, because users are unlikely to be familiar with these statistical concepts.⁶ A statement describing uncertainty in general terms was appropriate to help users interpret information (Figure 3) and included in the randomized survey to assess the impact on decisions.

The wide variability in waiting times was acknowledged by some participants (see Theme 1. These comments often reflected variability within a center for individuals, rather than geographic variation or variation across centers. A table of numerical values (Figure 3) allowed users to identify the potential range of outcomes reflected across the 5 tiers, for example the magnitude of differences in posttransplant outcomes compared to the magnitude of differences in transplant rates. However, presenting this data in patient-friendly displays was hindered by the complexity of underlying calculations and units for pretransplant measures, and additional research is warranted to best provide this context. This table was assessed in the randomized survey to determine if viewing the relative magnitude of differences would influence decisions. Current Scientific Registry of Transplant Recipients search tools (www.srtr.org) use less technical versions of units shown on data tables, for example “years of waiting” rather than “person-years.”

Although the wide variation in waiting times across regions is one reason a transplant rate metric would be relevant to patients, the wide variation in waiting times within a center may make the information harder to apply in a decision. Helping patients narrow this potentially wide range of waiting times based on individual characteristics may increase perceived relevance.^28,29

The results of the randomized survey reinforce findings from related studies^18-21,27 that content and presentation style can affect decisions. Although hypothetical choices were designed to reduce bias toward posttransplant outcomes, a preference for choosing a center with the best 1-year organ survival metrics (Table 4) was consistent with prior transplant surveys,³⁰ and prior studies indicating patients may underestimate waitlist mortality risks.^31,32 In this regard, the behavior of candidates and nonpatient survey participants was consistent. The qualitative responses confirmed that when faced with trade-off decisions, “fast and frugal” cognitive processes can be a default,¹⁹ and the short survey was intended to evaluate this type of decision.

The most significant change in decision behavior is attributed to the impact statement (Table 5), where this information resulted in a 50% higher probability of selecting Lake Hospital (highest transplant rate) compared to Meadow Hospital (highest 1-y organ survival). Waitlist mortality was consistently the lowest priority based on center decisions; however, the probability of this choice increased when viewing the data table. One potential explanation for this change is the presence of the word “Deaths” in the waitlist mortality units, which may draw attention to this measure.

The randomized survey results suggest that the impact statement would be effective in reducing a bias to choose centers based on posttransplant outcomes when multiple measures are provided. The use of this statement is not intended to communicate that a particular patient should choose based on the metric that is noted as most important to survival (or any other metric). Patients make decisions based on individual circumstances and values (see Theme 3 and Table S4, SDC, http://links.lww.com/TP/B755), and these values may not align with predicted risks. In other words, the graphical elements evaluated were not intended to convince users to choose a particular option because of predicted risks. Rather, if the information is available and a patient is aware of relevant risks, the reported data can support informed decision-making.

Additional mixed methods research is warranted to understand if new data or context to explain data would further support individual or shared decisions. New data may include a single survival measure combining pretransplant and posttransplant outcomes, patient-specific estimates of waitlist outcomes,^33,34 center measures reflecting waitlist access,³ or inferred criteria for listing candidates with higher-risk characteristics.²⁸ The number of potential future changes is high, and a combination of qualitative patient perspectives and decision-making data from nonpatients is a feasible approach to compare many alternatives, in particular for reporting tools targeting patients before a transplant evaluation.

A discrepancy was evident, where 4% of survey respondents had a perception that risks after transplant were greater than risks due to waiting (Table 3); however, a higher range of 31%–50% (Table 4) chose a center with the best 1-year survival. This discrepancy may be explained by the possibility that pretransplant metrics are unfamiliar and therefore prioritized less when viewed on a report. However, this issue warrants further study.

Additional themes related to making a choice about a center were discussed and have been previously described as common factors in decisions.²⁵ These included the importance of perceived reputation, experience of staff or surgeons, proximity, convenience, provider referrals, etc.

The study included a number of important limitations. Focus groups included small samples, with the exception of kidney, which may not have been representative of patients nationally and were not sufficient to draw conclusions about differences in decision-making across organ types. Audio recordings did not allow transcriptions to identify individual speakers, limiting the ability to analyze multiple comments by the same individual. The randomized survey included participants who were not necessarily transplant patients; however, additional research is needed to verify that internet users from the general public perceive information similarly to early-stage patients seeking a referral. The AMT platform used for survey recruiting allows screening for workers of high rated quality and has previously been validated in academic research,^35,36 including applications related to organ transplantation, for example, perceptions of left ventricular assist devices³⁷ and liver organ donation.³⁸ Participants were biased towards higher education, and survey participants were younger on average than transplant candidate populations (see Table S5, SDC, http://links.lww.com/TP/B755), which may limit generalizability. Focus group and survey protocols did not include assessments of health literacy and numeracy to analyze decisions across a literacy spectrum. Table S6, SDC, http://links.lww.com/TP/B755 includes a summary of a post hoc subgroup analysis. Groups stratified by age, sex, and education (survey questions #1–#3) did not correlate to changes in decisions. The survey did not assess how the respondents’ values may or may not align with data on predicted risks, although this was a theme of qualitative discussions.

In conclusion, qualitative feedback identified patient perceptions of uncertainty in outcome metrics, in particular pretransplant metrics, and suggested a need to explicitly provide guidance to interpret the most important metric for organ-specific patient mortality. Randomized survey results demonstrated that a note to indicate which metric has the largest impact on survival, for example, transplant rate, significantly increased the probability respondents choose hospitals with the best outcomes in this metric. The presentation of public reports influenced decision-making behavior. As new metrics are proposed and validated, understanding patient perspectives as well as decision-making behavior will better inform how to support patients.