Treatment Preferences for Cardiac Procedures of Patients With Chronic Kidney Disease in Acute Coronary Syndrome: Design and Pilot Testing of a Discrete Choice Experiment

Abstract

Background:

Chronic kidney disease is associated with a high incidence of acute coronary syndrome and related morbidity and mortality. Treatment choices for patients with chronic kidney disease involve trade-offs in the potential benefits and harms of invasive management options.

Objective:

The objective was to quantify preferences of patients with chronic kidney disease toward invasive heart procedures.

Design:

Design and pilot a discrete choice experiment.

Setting:

We piloted the discrete choice experiment in 2 multidisciplinary chronic kidney disease clinics in Calgary, Alberta, using an 8-question survey.

Patients:

Eligible patients included those aged 18 years and older, an estimated glomerular filtration rate < 45 mL/min/1.73 m², not currently receiving dialysis, and able to communicate in English.

Measurements:

Quantification of the average importances of key attributes of invasive heart procedures.

Methods:

We identified attributes most important to patients and physicians concerning invasive versus conservative management for acute coronary syndrome, using semi-structured qualitative interviews. Levels for each attribute were derived from analysis of early invasive versus conservative acute coronary syndrome management clinical trials and cohort studies, where subgroups of patients with chronic kidney disease were reported. We designed the pilot study with patient partners with relevant lived experience and considered statistical efficiency to estimate main effects and interactions, as well as response efficiency. Hierarchical Bayesian estimation was used to quantify average importances of attributes.

Results:

We recruited 43 patients with chronic kidney disease, mean (SD) age 67 (14) years, 67% male, and 35% with a history of cardiovascular disease, of whom 39 completed the survey within 2 weeks of enrollment. The results of the pilot revealed acute kidney injury requiring dialysis and permanent kidney replacement therapy, as well as death within 1 year were the most important attributes. Measures of internal validity for the pilot discrete choice experiment were comparable to those for other published discrete choice experiments.

Limitations:

Discrete choice experiments are complex instruments and often cognitively demanding for patients. This survey included multiple risk attributes which may have been challenging for some patients to understand.

Conclusions:

This pilot study demonstrates the feasibility of a discrete choice experiment to quantify preferences of patients with chronic kidney disease toward the benefits and trade-offs related to invasive versus conservative management for acute coronary syndrome. These preliminary findings suggest that patients with chronic kidney disease may be on average similarly risk averse toward kidney replacement therapy and death. This pilot information will be used to inform a larger discrete choice experiment that will refine these estimates of patient preferences and characterize subgroups with distinct treatment preferences, which should provide new knowledge that can facilitate shared decision-making between patients with chronic kidney disease and their care providers in the setting of acute coronary syndrome.

Introduction

Current guidelines recommend early invasive management of acute coronary syndrome (ACS) for high-risk individuals,¹ although observational studies suggest many eligible individuals with chronic kidney disease (CKD) do not receive these interventions.^2-4 Whether this apparent risk treatment paradox is driven by physician recommendations or patients’ treatment preferences remains unclear.⁵ Understanding patients’ values and preferences is increasingly appreciated as fundamental to supporting patients in treatment decision-making, and is being incorporated into clinical guidelines and health policy worldwide.⁶ The Grading of Recommendations Assessment, Development and Evaluation working group includes information about values and preferences as key components of their criteria recommended for inclusion in health care guidelines.⁷ Eliciting patient preferences is also a key element of shared decision-making, an approach where care providers and patients share best evidence, and where patients are supported to consider treatment options that best achieve their informed preferences and treatment goals.⁸ In an ACS setting, there is often more than 1 treatment option and no clear “best” option for all patients. As the invasive and conservative management options have associated benefits (longer survival, reduced risk of recurrent myocardial infarction) and potential harms (acute kidney injury [AKI] and acceleration of progression to kidney replacement therapy), which individual patients may value differently, their informed preferences should guide the decision.⁹

Patient preferences toward the attributes of treatment decisions can be quantified using discrete choice experiments (DCEs), a stated preference method that involves having people state their preferences for different hypothetical treatment options. By varying options presented over a series of choice questions, the relative importance of each attribute and level can be quantified.¹⁰ DCE survey questions are designed such that respondents are asked to choose a treatment option by considering its individual attributes. This requires respondents to trade off the benefits and harms of each option and allows for measurement of the part-worth utility of each attribute.^10-13 Although DCEs have been used widely in patient-oriented research and can inform guideline recommendations, there have been no prior DCEs conducted to understand the preferences of patients with ACS, including those with CKD, facing an invasive treatment versus conservative management decision.

Here, we report the design and pilot testing of a DCE to elicit and quantify the preferences of patients with CKD for the key attributes (the characteristics of the treatment alternative) to consider when faced with an invasive versus conservative management treatment decision for ACS.

Methods

The methodology for this DCE was guided by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist for good research practices when conducting a conjoint analysis, and the approach to conducting DCEs by Lancsar and Louviere.^13,14 We followed the recommendations to define our research objective, identify appropriate attributes and levels, and ensure efficient experimental design with regard to choice question format and statistical analysis.

Patient Engagement

The development of this DCE was supported by 2 patient partners with lived experience with CKD and heart disease (C.C. and W.P.), who helped inform the design of the study, the study information for patients, the survey design, participated in preliminary testing of the DCE during its development, and assisted with coauthoring this article.

Identification of Attributes and Levels for the DCE

To identify the attributes relevant to this DCE, we first conducted semi-structured interviews with 20 patients with CKD who had recently experienced an ACS to identify the attributes relevant to ACS treatment decisions.¹⁵ We also conducted individual interviews with 10 cardiologists with experience guiding patients through the ACS decision-making process. We identified 5 attributes that were consistently important to both patients and physicians (Table 1), which were selected for inclusion in the DCE.

Table 1.

Identified Attributes and Levels.

Attribute	Levels
Treatment approach	- An angiogram is performed immediately upon admission - Conservative management results in heart stability and no angiogram is required
Risk of death within one year	3 of 100 9 of 100 15 of 100
Risk of acute kidney injury requiring dialysis in hospital	1 of 100 3 of 100 10 of 100
Risk of kidney damage resulting in the need for permanent dialysis or kidney transplant	1 of 100 5 of 100 10 of 100
Risk of another heart attack within one year	6 of 100 9 of 100 12 of 100

We selected clinically plausible levels of risk for each attribute based on literature review of randomized control trials of early invasive versus conservative management strategies for non-ST elevation ACS as well as cohort studies that reported the incidence of kidney outcomes following ACS.^16-18 Table 1 provides attributes as presented in the pilot DCE and the levels of risk for each attribute included in the DCE.

Experimental Design

With 5 attributes, including 4 with 3 levels and 1 with 2 levels, the total number of combinations in a full factorial design is given by L^A (A = no. of attributes, L = no. of levels; 34 × 21), resulting in a total of 162 possible choice profiles. Because this would be too many choice tasks for each patient to complete, we chose a fractional factorial design with a balanced overlap approach consisting of 100 versions. With sufficient sample size, a fractional factorial design still allows for estimating main effects and interactions; however, higher order interactions may be unobtainable.¹³ We selected a design with a forced choice approach with no opt-out option because we deemed no treatment would not be an option for hospitalized patients following an ACS event. We designed the choice process to consist of 2 options per questions. The order of attributes presented in the choice task may affect the participants’ responses, with more focus being placed on attributes at the top of the screen.¹² As such, the order of attributes varied randomly across versions of the survey with the exception of treatment approach. This attribute was placed first in each choice task to orient respondents to the information being presented.

We determined the sample size to ensure statistical efficiency for estimating main effects part-worths and preidentified interactions (treatment approach × risk of kidney damage resulting in the need for permanent dialysis or kidney transplant). That is, to ensure standard errors for estimating attribute/level combinations are less than 0.05 for main effects and interactions, Table 2 shows the required sample sizes, at varying number of choice tasks, to meet these criteria and maintain similar design efficiency, as measured using D-efficiency.¹⁰ Our first design consisted of 13 questions, and a sample size of 150 questionnaires was identified as the point beyond which the incremental gain in precision of estimates leveled off.^10,19 This sample size is within the common range for conjoint analysis experiments, with 1 systematic review reporting sample sizes typically in the range of 150 to 300 respondents.¹¹ The survey design was developed using Lighthouse Studio, Sawtooth Software, Orem, Utah.

Table 2.

Design Efficiency Statistics.

No. of questions	Sample size	Main effects standard error	Interactions standard error	D-efficiency
6	320	0.042	0.050	695
7	280	0.041	0.050	703
8	240	0.041	0.050	695
9	210	0.041	0.049	689
12	160	0.042	0.048	699
13	150	0.040	0.048	705

Development and Preliminary Testing

We performed preliminary testing and revision of the DCE design with 2 patient research partners (C.C. and W.P.) to ensure readability and understanding of the introductory information surrounding the purpose and context of the study (asking patients to consider their decisions if they were admitted to hospital with a heart attack), description of the treatment decision, and nature of the attributes and levels.

The first iteration of the DCE consisted of 13 choice tasks; however, our patient partners found this format to be too long and repetitive, requiring more than 30 minutes to complete, and thus, response efficiency was anticipated to be compromised. To obtain balance between response efficiency and statistical efficiency, we reduced the number of questions to 8, which led to an adjustment of the required sample size to 240 for a full DCE.

Patient Recruitment

Our target patient population for this DCE was patients with CKD. Eligible patients included those aged 18 years and older, an estimated glomerular filtration rate (eGFR) < 45 mL/min/1.73 m², not currently receiving dialysis, and able to communicate in English. Eligibility criteria did not require that patients had a history of cardiac disease or a prior ACS. Patient recruitment occurred at 2 CKD clinics in Calgary, Alberta. Patients with CKD were asked to complete the survey through Web site access, either in the clinic or at a later time. In addition, patient demographic information was collected as part of the survey questions including age, sex, race/ethnicity, recent eGFR measurement, cardiovascular disease history, and whether they had received education about dialysis modality options.

Nursing staff from clinics determined which patients were potentially eligible for the study from daily lists of clinic appointments. During their waiting time in clinic, the nature of the study was explained by the nursing staff and patients were asked whether they were willing to participate in the study and receive more information. Research coordinators explained the details of the study to patients, and those who provided informed consent to participate were provided a tablet computer to complete the survey while waiting for their appointment, and/or an e-mail was sent to the patient with a survey link allowing patients to complete the survey at home. Research coordinators also collected feedback from participants on areas where respondents required more information or clarification on the DCE and also where there may have been extraneous information that could be reduced.

Data Analysis

Characteristics of the patients who completed the DCE were described using descriptive statistics. DCE results were estimated to determine which attribute/level combination was statistically significant at a 5% level of significance, and the order of attribute importance to patients. In addition, importance weights were normalized and displayed. Internal validity of responses to the DCE was analyzed by comparing importance weights with a priori expectations of preferences (death and need for permanent kidney replacement therapy were expected to be the attributes with highest importance weightings). In addition, we assessed for the following measures of DCE validity: attribute dominance, where the better level of a single attribute is chosen in almost all scenarios; within-set dominated pairs, where participants choose the alternative that is worse for all attributes; across-set dominated pairs, where the alternative chosen in 2 different choice sets is logically inconsistent; and straight-lining, where participants choose the alternative in the same position for each question.²⁰

Data generated from the DCE was analyzed using a hierarchical Bayes model. With this model, the probability of choosing a specific profile, in a choice task, is a function of the attribute levels in that profile and the attribute levels of the other profiles in the choice task.²¹ The coefficients from the model are preference weights of the attribute levels, but the values of these coefficients only have meaning relative to each other. Analysis of the pilot study data was limited to determining the average importances of the attributes, relative to each other. Average importances were determined from the range of each attribute’s utility and scaling such that the total importance value across all 5 attributes summed to 100, along with a 2.5% and 97.5% credibility interval.

Ethical Considerations

Ethical approval was obtained from the University of Calgary, Conjoint Health Research Ethics Board. Informed consent was obtained from participants prior to completing the survey.

Results

The 8-question pilot DCE was administered to 43 patients between September and December 2018. Figure 1 shows an example of a choice task presented to patients. The majority of patients completed the survey in clinic, with the assistance of a research coordinator. Many patients found it challenging to complete the survey using a tablet computer as it did not allow patients to move back and forth between questions, and therefore, they did not always appreciate that the risks were changing with each question. Research coordinators guided many of the patients through to completion of the survey to assist them in understanding the varying nature of the scenarios and to encourage patients to consider each question carefully. When a patient did not complete the survey in clinic, research coordinators followed up by telephone to guide participants through to completion of the remainder of the survey. The pilot DCE survey, including one version of the eight choice tasks, is included in supplementary material.

Figure 1.

Example choice task for the discrete choice experiment.

Analysis of Pilot Data

The characteristics of the participants of the pilot study are shown in Table 3. Thirty-nine of 43 participants completed the DCE. These participants were able to complete the survey within a reasonable timeframe of 15 to 20 minutes.

Table 3.

Characteristics of Patients Completing Pilot DCE.

Completion rate, no. starting DCE/no. completing DCE (%)	39/43 (91%)
Mean (SD) age of respondents	67 (14) y
Percent male	62%
Percent history of cardiovascular disease	35%
Mean (SD) eGFR	24 (14) mL/min/1.73 m2
Mean (SD)/Median (IQR) years since CKD diagnosis	10 (14)/4 (7) y

Note. DCE = discrete choice experiment; eGFR = estimated glomerular filtration rate; IQR = interquartile range; CKD = chronic kidney disease.

Figure 2 illustrates the importance weights of the 5 attributes and their levels included in the pilot analysis. The highest average importance was observed with the attribute of AKI requiring dialysis, although the credibility intervals for the pilot results overlapped considerably (Table 4). Feedback from participants identified that the wording of the attribute for AKI requiring dialysis in hospital did not distinguish possible need for temporary versus permanent dialysis. Figure 3 shows an example of a choice task after modifications to the survey based on feedback from this pilot DCE.

Figure 2.

Average importance of attributes (scaled 0-100).

Table 4.

Average Importance of Attributes (Scaled 0-100).

Attribute	Average importance (95% confidence interval)
Treatment approach	11 (7-15)
Risk of death within 1 y (3%-15% risk)	22 (18-26)
Risk of acute kidney injury requiring dialysis in hospital (1%-10% risk)	28 (23-32)
Risk of kidney damage resulting in the need for permanent dialysis or kidney transplant (1%-10% risk)	22 (18-26)
Risk of another heart attack within 1 y (6%-12% risk)	17 (13-20)

Figure 3.

Illustration of an example of a choice task following the modifications incorporated after the pilot phase.

Internal validity tests showed 4 (10%) respondents chose the dominated choice profile in 1 choice task and 1 (3%) respondent chose 2 dominated choice profiles (within-set dominated pairs). No respondents failed the across-set dominated pairs test. With regard to dominated preferences, 10 (31%) respondents focused on a single attribute in at least 7 choice tasks, with 3 (8%) focusing on a single attribute in all 8 tasks (AKI requiring dialysis, treatment, or death). These were primarily death or dialysis (3 and 6 respondents, respectively); however, 1 respondent chose conservative treatment in all 8 tasks. Two (5%) respondents chose alternative 2 in 7 choice tasks and 1 (3%) respondent chose alternative 1 in all 8 choice tasks, indicating potential straight-lining by these participants. Table 5 compares these interval validity results from our pilot study with those reported from other published DCEs.²⁰

Table 5.

Interval Validity Failure Comparison to Published DCEs.

Test type	DCE pilot study % failures	Published DCE % failures (mean [SD])
Within-set dominated pairs	13	18 (20)
Across-set dominated pairs	0	6 (9)
Dominated preferences	31	22 (14)
Straight-lining	8	7 (11)

Note. DCE = discrete choice experiment.

Discussion

We fielded a pilot DCE to quantify preferences of patients with CKD toward invasive heart procedures for ACS, consisting of 8 questions, with 43 patients in 2 clinics in Calgary, Alberta, Canada. The analysis of patient responses from the pilot study, based on average importances for the 5 attributes, was in line with a priori expectations, with the exception that AKI requiring dialysis while in hospital received the highest weighting. In addition, measures of internal validity of the DCE results were consistent with those of other published DCE studies.²⁰ This suggests patients were able to comprehend the choice tasks presented in the DCE and is evidence of the feasibility for the full DCE. By conducting a pilot DCE, we were able to identify issues in the design that will be modified to improve the validity for the future full DCE.

DCEs have become popular tools for measuring patient preferences across clinical specialties.^10,11,21 There are numerous strengths to DCEs for eliciting patient preferences toward treatment options and health outcomes. When assessing patient preferences by conducting a DCE, the survey can be designed to capture all relevant aspects of a health intervention.²² A DCE breaks down the choices into attributes and levels of attributes. Rather than having to value each attribute separately, the relative importance of attributes can be valued together.¹² This provides useful information on which attributes of a treatment option are important to patients, and the incremental benefits patients derive from the individual attributes can be characterized. Subgroup analysis is another advantage of using DCEs to estimate patient preferences.¹² With appropriately collected data on patient characteristics (eg, education, age, sex), differences in preferences across subgroups can be analyzed and this information potentially used for targeting treatment regimens, or tailoring information to specific patients.

A properly designed DCE must go through a rigorous development process often involving qualitative research for attribute identification, appropriate selection of levels, number of choice tasks, and consideration of the optimal design to answer the research question.¹³ In addition, the design needs to be pilot tested and revised through an iterative process. Furthermore, the part-worth of attributes can only be considered relative to each other and within the context of the model; therefore, they are not generalizable to all decision-making situations.²²

DCEs are complex instruments and often cognitively demanding for patients. A DCE that appears to be well designed may be challenging for patients if they do not comprehend what is being asked of them. Patients need to understand the choice tasks and the meaning of attributes and levels. A lack of comprehension by patients can lead to misspecification bias where patients are not interpreting the choice tasks in the way they were intended by the researchers.¹² In addition, patients may find the cognitive burden increased because some trade-offs may be especially challenging to consider. Alternatively, instead of considering each attribute and choosing an overall preferred option, patients may consider only a subset of the attributes and choose based on this subset to reduce cognitive burden.¹² A further limitation of DCEs related to patient responses is that the responses provided may not be consistent with their true preferences. Patient responses may tend toward the status quo or reflect fads or trends.¹⁰ In addition, because DCE choice tasks are hypothetical, responses may not reflect true preferences. This could be because patients are not deliberating carefully over the attributes or patients select the choice they think they are supposed to prefer.¹²

Strengths of this DCE include the assistance of patient partners in the study design, who helped ensure the DCE was patient centered. We used research coordinators, who were able to collect information from survey participants while they completed the DCE. This allowed for valuable feedback on areas where respondents required more information or clarification on the DCE and also where there may have been extraneous information that could be reduced for greater efficiency. In addition, this DCE was designed by following the ISPOR guidelines, and our finding that most results of the DCE were in line with expectations and patient responsiveness is evidence of a robust experiment.

Based on feedback from participants and analysis of the pilot DCE data, we identified 4 areas where modifications were needed for the full DCE study:

• Modified introductory information describing the treatment decision. This included reducing the amount of text required to be read and removing warm-up questions.

• Some patients found it challenging to appreciate and compare differences in the absolute risks of the various outcomes. Therefore, we added visual graphics to aid communication of risk (shaded boxes out of 100) in addition to written information on the number of events out of 100 people.

• In response to participant requests, we also expanded the choice of media for patients by adding the option of a pencil-and-paper format for the DCE for those who may prefer it over the computer format. Ten versions of the choice profiles will be created. Use of a paper-based survey will mean fewer total available choice profiles will be presented in the survey for the final DCE. However, 10 versions remained appropriate for statistical efficiency.

• The description of AKI requiring acute dialysis was revised to distinguish acute dialysis that may be temporary in the setting of AKI versus dialysis that is required permanently for kidney replacement therapy.

Limitations to this DCE include the multiple risk attributes which may have been challenging for some patients to understand. Internal validity tests showed that some patients chose the dominated pair or focused solely on 1 attribute. However, internal validity test failures for the pilot DCE were within the range of those reported from other DCEs.²⁰ In addition, because patients are not actually facing the decision, there may be a hypothetical bias and not truly reflect how one would behave when faced with this treatment decision. Many patients commented that they were unaware of their increased risk of heart disease as a result of their CKD. As such, patients may have already formulated preferences around dialysis but may have not considered myocardial infarction as a potential adverse event. Risk of stroke was an attribute identified in physician interviews as a consideration when making treatment decisions for ACS, but not included in this DCE. In the general population, the risk of stroke was not found to be significantly different between an early invasive and select invasive management strategy for ACS; however, the recent International Study of Comparative Health Effectiveness with Medical and Invasive Approaches (ISCHEMIA)-CKD trial identified small but statistically significant differences in patients with CKD between the invasive and conservative strategies, with those in the invasive arm experiencing a higher cumulative event rate.²³ Although the ISCHEMIA-CKD trial was based on a different patient population, patients with stable coronary disease, risk of stroke is nevertheless an important consideration for treatment decisions for ACS that was not addressed by this DCE.

The results of this pilot DCE have informed the development of a full DCE, and its results should have important implications for clinical care and future research. Despite the fact that 1 in 4 patients with ACS has CKD and randomized trials have demonstrated efficacy of invasive management in high-risk patients, studies have shown that patients with CKD are less likely to receive coronary angiography or revascularization following a coronary event than patients without CKD.²⁴ Furthermore, patients with lower levels of eGFR are even less likely to receive these procedures than patients with otherwise similar clinical characteristics.²⁵ Although there have been temporal increases in the use of invasive management for ACS in the general population, patients with CKD continue to disproportionately receive conservative treatment rather than invasive management.²⁶ This DCE will help us understand whether these contemporary observed treatment patterns align with the stated preferences of patients with CKD.^26,27 This pilot has suggested kidney replacement therapy is a key concern for patients with CKD, and some patients may be unwilling to accept any increased risk for living with dialysis. However, because of the lower number of participants in the pilot study, we were underpowered to detect statistically significant differences. Information from the full DCE will also help inform shared decision-making between patients with CKD and their care providers in an ACS setting, where there is more than 1 treatment option and no clear “best” option for all patients. The DCE is based on a theoretically rigorous method to characterize patient preferences and is supported by guidelines and recommendations for use in supporting decision-making in health care.²⁸ The results of this work could contribute important information for the future development of decision aids to translate this knowledge into patient care. There is evidence that the way information is presented to a patient has strong influence on how they construct preferences.²⁹ Such findings can be beneficial to understand these considerations from patient and provider perspectives. In addition, the results of this work can inform guideline development incorporating patient preferences.³⁰

In summary, we have designed and pilot tested a DCE to address a research priority identified by patients and health care providers to characterize and support those with kidney disease at highest risk of adverse kidney and cardiovascular outcomes.^15,31 Improved management of cardiovascular disease in people with CKD has the potential to significantly improve outcomes. This DCE has been designed to identify the preferences of patients with CKD and cardiovascular disease, and should provide new knowledge that can facilitate shared decision-making between patients with CKD and their care providers in the setting of ACS to allow for greater patient involvement and recognition of their treatment values and goals.