Skip to main content
PMC home page
Springer logoLink to Springer
. 2022 Apr 18;15(5):551–564. doi: 10.1007/s40271-022-00576-w

The Patient Voice: Exploring Treatment Preferences in Participants with Mild Cognitive Concerns to Inform Regulatory Decision Making

Carol Mansfield 1,✉, Kristin Bullok 2, Jillian Venci Fuhs 2, Antje Tockhorn-Heidenreich 2, J Scott Andrews 2,4, Dana DiBenedetti 3, Brandy R Matthews 2, Joshua C Darling 2,5, Jessie Sutphin 1,6, Brett Hauber 1,7
PMCID: PMC9365745  PMID: 35435572

Abstract

Objective

We aimed to assess the feasibility of developing a discrete-choice experiment survey to elicit preferences for a treatment to delay cognitive decline among people with a clinical syndrome consistent with early Alzheimer’s disease, including the development of self-reported screening criteria to recruit the sample.

Methods

Using input from qualitative interviews, we developed a discrete-choice experiment survey containing a multifaceted beneficial treatment attribute related to slowing cognitive decline for respondents with self-reported cognitive concerns. In two rounds of in-person pretest interviews, we tested and revised the survey text and discrete-choice experiment questions, including examples, language, and levels associated with the Alzheimer’s Disease Assessment Scale-Cognitive Subscale, along with a set of de novo self-reported questions for identifying respondents who had neither too mild nor too advanced cognitive decline. Self-reported memory and thinking problems were compared with symptoms from studies of patients with early Alzheimer’s disease (e.g., mild cognitive impairment, mild Alzheimer’s disease) to determine whether those studies’ recruited patients were similar to our anticipated target population. Round 1 pretest interviews resulted in significant simplifications in the survey instrument, revisions to the inclusion and exclusion criteria, and revisions to the screening process. In round 2 of the pretest interviews, the ability of participants to provide consistent responses to the self-reported screening questions was further assessed. In addition, to evaluate participants’ ability to understand and independently complete the discrete-choice experiment survey, two interviewers independently evaluated each participant’s ability to make trade-offs in the discrete-choice experiment questions and to understand the content of the survey.

Results

Round 1 (15 pretest interviews) identified challenges with the survey instrument related to the complexity of the choice questions. The screening process did not screen out seven respondents with more advanced cognitive decline, as determined qualitatively by the interviewers and by these participants’ inability to complete the survey. The survey instrument and screening criteria were revised, and an initial online screener was added to the screening process before round 2 pretests. In round 2 pretests, 12 participants reported cognitive problems similar to the target population for the survey but were judged able to understand and independently complete the discrete-choice experiment survey.

Conclusions

We developed self-reported screening criteria that identified a sample of individuals with memory and thinking concerns who were similar to individuals with clinical symptoms of early Alzheimer’s disease and who were able to independently complete a simplified discrete-choice experiment survey. Quantitative patient preference studies provide important information on patients’ willingness to trade off treatment benefits/risks. Adapting the technique for patients with cognitive decline requires careful testing and adjustments to survey instruments. This work suggests it is the severity of cognitive impairment, rather than its presence, that determines the ability to complete a simplified discrete-choice experiment survey.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40271-022-00576-w.

Key Points for Decision Makers

With careful screening, persons with self-reported memory and thinking concerns can independently complete a simplified discrete-choice experiment survey.
The simplified discrete-choice experiment tool can be used to collect quantitative patient preference data for use in regulatory decision making and by other stakeholders interested in patient preferences.
Open in a new tab

Introduction

Understanding patients’ perspectives on current or potential treatment options is increasingly important to regulators and other stakeholders [1–7]; however, the impact of cognitive decline on a person’s ability to participate in stated-preference research is largely unknown. Alzheimer’s disease (AD) is a progressive neurodegenerative disease characterized by neuropathological hallmarks that can now be qualitatively and quantitatively assessed [8–10]. Clinical manifestations include gradual declines in memory, executive function, and language. Early cognitive symptoms emerge many years after AD neuropathological changes begin but may be subtle and involve no functional impairment.

Guidance from the US Food and Drug Administration on trials in early AD (an umbrella term for the clinical syndromes of mild cognitive impairment [MCI] and mild dementia with evidence of AD neuropathology) recommends that a treatment show a clinically meaningful effect but notes that clinical meaningfulness of changes on neuropsychological performance measures is unclear [10]. Previous clinical research with amyloid-antibody therapeutics in those with early AD investigated whether such treatments slowed the rate of cognitive decline with unconvincing results [11–13]. Recent trials of donanemab, lecanemab, and aducanumab have been more supportive of the potential for amyloid-antibody treatments to slow cognitive decline [14–16]. Clinically meaningful effects of such treatments in early AD may be difficult to measure using existing instruments, when only subtle and variable cognitive and functional deficits are detectable and when existing measures may not capture the impact that patients find important. [17, 18]. In this situation, the meaningfulness of changes in cognition irrespective of improvement in functional impairment may be assessed by the people affected, highlighting the importance of eliciting perspectives and preferences in early AD.

Preference studies can quantify risks patients are willing to accept to achieve specific treatment benefits and may help elucidate the clinical meaningfulness of those benefits [19, 20]. Discrete-choice experiments (DCEs) are a statistically rigorous method to collect preferences, allowing the consideration of multiple attributes at a time, and have informed regulatory decisions [21, 22]. For AD, some studies have reported on trade-offs people were willing to make among features of potential AD treatments; however, except for Oremus and colleagues [23], these studies were conducted with healthy adults [24–26] or caregivers [27, 28] rather than patients.

Because preference surveys can be cognitively challenging, the first objective of this study was determining whether a DCE survey of people with memory and thinking problems (i.e., subjective cognitive complaints and/or MCI) was feasible to reliably elicit preferences for delaying cognitive decline. The second objective was determining whether self-reported screening criteria could reliably identify a population with cognitive impairment consistent with early stages of an AD clinical syndrome who could understand and independently complete the DCE survey. To achieve these objectives, a preliminary DCE survey was developed on the basis of qualitative interviews [29] and assessed through two rounds of individual face-to-face pretest interviews. This article describes these pretest interview results and the interplay between the survey design and screening criteria, including the implication that a point along the AD cognitive impairment continuum may exist beyond which one cannot independently complete certain preference surveys.

Initial Survey Development

Survey Instrument

Previous qualitative interviews with participants self-reporting cognition problems consistent with early stages of an AD clinical syndrome (target sample) were conducted to identify meaningful benefits of a treatment that could slow the progression of memory and thinking problems and to assess participants’ reactions to several adverse event (AE) risks [29]. These interviews informed and tested potential formats and graphics for the current DCE survey.

The DCE survey offered target sample participants three options: two hypothetical medicines and no new medicine (opt out) (Fig. 1). The development of benefit and risk attributes is described below. The survey instrument also collected information about current memory and thinking problems, current/previous treatments for memory and thinking problems, participants’ ability to comprehend attribute descriptions, and demographic questions.

Fig. 1.

Fig. 1

Open in a new tab

Example of a discrete-choice experiment question from the first day of round 1 pretest interviews

Treatment Benefit Attribute

The benefit attribute was based on then-published clinical studies of treatments in symptomatic AD that suggested a potential slowing of cognitive decline [13, 30]. These experimental treatments resulted in smaller mean changes on the Alzheimer’s Disease Assessment Scale–Cognitive Subscale (ADAS-Cog) [average: 2 points] than changes observed with placebo at 18 months [12]. Therefore, the benefit attribute in this study was described as delaying or slowing decline in memory and thinking problems.

Cognitive decline, a multifaceted concept, was described in the survey using five daily-life examples of impacts to memory and thinking ability: financial management, medication management, finding the right words in conversation, remembering what you have read in a new book or magazine, and understanding verbal instructions. The choice of these five impacts was based on clinical data (natural history/clinical studies) demonstrating the ADAS-Cog items that were most likely impacted early in the AD clinical syndrome (e.g., item scores that decreased earliest within an 18-month clinical trial) [31–33]. The language used for the impacts was informed by previous research [34], input from practicing physicians, and previous qualitative interviews [29].

The survey described a person who had “no problems,” “some problems,” and “more problems” for each of the five impacts. “Some problems” and “more problems” reflected a specific change in the ease with which a person could complete the described impact (see Electronic Supplementary Material [ESM]). Cognitive decline was described as the time it would take to progress from having “some problems” with memory and thinking to having “more problems.” Although a minimal clinically important difference for ADAS-Cog has been published [35], no studies translate specific changes in ADAS-Cog scores to relevant practical changes in a patient’s life. Thus, the change from “some” to “more” problems was developed to represent a modest change in the ADAS-Cog and was based on medical judgment of what could reasonably occur over the natural course of AD and on observations from some clinical trials of investigational disease-modifying treatments for early AD [13, 32, 33, 36].

The opt out option was defined as a decline from “some problems” to “more problems” in 1 year. The two hypothetical medicines offered an additional 6 months, 12 months, or 18 months to the time until memory and thinking problems worsened. Time to progression was depicted graphically in the initial survey instrument (Fig. 1).

Treatment Risk Attributes

To capture the range of possible AEs, one mild AE and one severe AE were included. A common AE of antibody treatment is mild/moderate hypersensitivity reactions, which can manifest as flu-like symptoms (i.e., chills, fatigue, and muscle weakness). In previous qualitative interviews [29], participants perceived “flu-like symptoms” as worse than intended, thus the attribute description of “feeling tired and weak” was used as a mild AE associated with treatment. For a severe AE, recurrent noncardiac chest pain that felt like a heart attack, requiring a doctor visit, was selected based on its rare occurrence in solanezumab clinical trials [12]. A plausible duration or frequency range of the selected AEs, based on Siemers and colleagues [12] and the authors’ medical knowledge of solanezumab, was 1–4 days of feeling tired and weak and 1–6% frequency for recurrent noncardiac chest pain. These risk attribute levels were depicted using an icon array (Fig. 1).

Although many amyloid-targeting therapeutics in development are associated with amyloid-related imaging abnormalities [37–39], the clinical impact of amyloid-related imaging abnormalities was not well understood at the time this study was conducted [40]. Thus, the presence of amyloid-related imaging abnormalities was not included as a risk attribute in this study.

Human Subjects Protection

Before study initiation, the RTI International Institutional Review Board reviewed and approved the study materials (No. 14025, dated 11 July, 2017, amended 13 November, 2017). Each participant provided written informed consent, which included teach-back questions to verify participants understood key elements of the consent form.

Round 1 Pretest Interviews

Procedure

Fifteen in-person, 1-hour pretest interviews were conducted over 3 days. During each interview, participants were asked to read the survey aloud and discuss their responses to survey questions. Respondents were asked to assume they currently had “some problems” when they answered the DCE treatment questions.

Interviewers asked follow-up questions to assess participants’ understanding of survey questions and text. Survey text and content were revised between interviews to address identified issues. Additional revisions were made to the survey after completing all round 1 interviews.

Participant Screening

Inclusion and exclusion criteria were developed to recruit a sample of participants for an online survey. The goal of the criteria was to identify a group of participants who have a clinical syndrome consistent with early AD and who could be recruited from existing online panels or other databases of research participants, without requiring a clinician assessment. The inclusion and exclusion criteria were designed to identify people who had self-reported noticeable changes in memory and thinking that were significant enough to visit their doctor but not so severe that the changes would preclude the participant from completing a DCE survey.

Figure 2 depicts the flow of screening questions for recruiting round 1 pretest interview participants. The ESM provides round 1 screening questions. Although the research framework for the clinical and neuropathological diagnosis of AD has since evolved [9], the screening criteria included core clinical criteria for the diagnosis of MCI from the National Institute on Aging and the Alzheimer’s Association [40–42]. The participants’ level of cognitive decline was ascertained through participant-reported symptom and treatment history. Screening criteria were enhanced by requiring participants to recently have visited their doctor about their memory and thinking concerns and discussed using medications for treatment of these symptoms. Including those who discussed concerns with their doctor helped identify those more likely to have early signs of cognitive impairment. As physicians use a variety of terms with patients when discussing the cause or severity of cognitive complaints, and may avoid saying “Alzheimer’s disease” or “dementia” [43], screening on whether a discussion of AD symptomatic medication took place enabled the identification of those who may have early signs of cognitive impairment as judged by their physician but with whom the physician did not use “Alzheimer’s” or “dementia” terminology.

Fig. 2.

Fig. 2

Open in a new tab

Inclusion and exclusion criteria for round 1 interviews. AD Alzheimer’s disease, ADLs activities of daily living, meds medications, MCI mild cognitive impairment, Rx prescription. Note: The screening criteria were revised for round 2 interviews. The revisions included the following: time since a person first spoke with their doctor about memory concerns was decreased from 4 years to 2 years, participants who reported taking memantine (Namenda) or the combination of memantine and donepezil (Namzaric) were excluded, and those self-reporting that they were not comfortable with a computer were ineligible

For round 1 pretest interviews, potential participants were screened for eligibility via telephone. Re-administration of screening questions during the in-person interview provided a consistency assessment of self-reported responses. Screening question responses also were compared with the interviewers’ subjective assessment of the participant’s ability to understand the survey. Interviewers assessed whether differences in responses to screening questions during the telephone compared with in-person interviews correlated with the participant’s ability to understand the survey. Evaluation of whether screening questions could identify people able to independently complete the survey was also performed.

Results

The 15 participants in round 1 pretests averaged 64 years of age, with approximately half being women (Table 1). Most were married and had a 4-year college or graduate degree. Table 2 provides participants’ responses to some questions about their memory and thinking problems. The participant characteristics and types of problems reported in these interviews were consistent with a population diagnosed with MCI or mild AD (using Petersen or NINCDS-ADRDA criteria, respectively), with the possible exception of orientation [31, 44–46].

Table 1.

Participant characteristics

Question Round 1 pretest interviews (n = 15) Round 2 pretest interviews (n = 12)
Age (in years)
Mean (SD) 63.8 (6.5) 59.2 (4.4)
Are you …?
Male 7 4
Female 8 8
Do you live …?
In your own home or apartment alone or with spouse/partner 12 10
With another relative (not your spouse/partner) 3 2
What is your marital status?
Single/never married 2 1
Married/living as married/civil partnership 10 7
Divorced or separated 2 3
Did not answer 1 1
What is the highest level of education you have completed?
High school or equivalent (e.g., GED) 1 1
Some college but no degree 2 2
Technical school 1 1
4-year college degree (e.g., BA, BS) 2 5
Some graduate school but no degree 0 1
Graduate or professional degree (e.g., MBA, MS, MD, PhD) 8 2
Did not answer 2 0
Open in a new tab

GED general education development, SD standard deviation

Table 2.

Responses to selected questions related to memory and thinking problems

Question in Round 1 pretest Round 1 pretest participants (n = 15) Question in Round 2 pretest, if question wording changed Round 2 pretest participants (n = 12) ADAS-Cog domain
Have any of your close blood relatives had problems with their memory or thinking? This could include diagnosed problems, like dementia or Alzheimer’s disease, or notable undiagnosed memory or thinking problems
Yes 11 9
No 2 3
I don’t know 0 0
Did not answer 2 0
Have any of your blood relatives been diagnosed with Alzheimer’s disease?
Yes 7 4
No 3 4
I don’t know 1 2
Did not answer 4 2
We would like to ask you about some common memory and thinking problems people may have. For each issue in the table below, please indicate whether the issue has gotten worse in the last year or stayed the same by checking the appropriate box beside each problem. We would like to ask you about some common memory and thinking problems people may have. For each memory or thinking problem in the table below, please indicate whether you have not had a problem with this in the last year, the problem has gotten worse in the last year, or the problem has stayed the same in the last year by checking the appropriate box beside each problem.
Gotten worse in the last year Gotten worse in the last year
Difficulty remembering new information (for example, new people’s names, something you read, or details about upcoming events) 13 Trouble remembering new information (for example, new people’s names, something you read, or details about upcoming events) 8 Memory
Difficulty recalling familiar information (names of people I’ve known a long time, words to songs) 9 Trouble recalling familiar information (names of people I’ve known a long time, words to songs that I used to remember) 8 Memory
Difficulty remembering how to do things I used to know how to do (using the TV remote, forgetting recipes, using tools) 3 Trouble remembering how to do things I used to know how to do (using the TV remote, forgetting recipes, using tools) 1 Memory
Forgetting appointments or showing up on the wrong day 7 Trouble remembering appointments or showing up on the wrong day 7 Memory
Relying more on lists or reminders or my phone to help me remember things 10 Relying more on lists or reminders or my phone to help me remember things 6 Memory
Walking into a room and forgetting why I am there 7 Walking into a room and forgetting why I am there 6 Memory
Trouble remembering to take my medicines on time 6 Memory
Difficulty managing finances or forgetting to pay bills on time 5 Trouble paying bills or managing money 7 Planning and executing
Forgetting where I have placed items 13 Trouble remembering where I have placed items 9 Memory
Trouble finding the right words in conversations 8 Trouble finding the right word in a conversation 6 Language
Difficulty understanding complicated instructions 9 Trouble understanding and following verbal instructions for tasks that involve 4 or 5 steps 3 Language
Having a hard time following and remembering the story on TV or in a book or article 8 Trouble remembering what I have read in a new book, magazine, or newspaper 6 Language
Trouble remembering what has happened on the show I am watching on TV 4 Language
Difficulty with knowing what steps I should take next when performing a task 4 Trouble with knowing what steps I should take next when performing a task 5 Planning and executing
Forgetting where I am or the direction I need to go 5 Trouble remembering where I am or the direction I need to go 3 Orientation
Leaving the water running or leaving the stove or oven on 3 Leaving the water running or leaving the stove or oven on 5 Memory
Open in a new tab

ADAS-Cog Alzheimer’s Disease Assessment Scale-Cognitive Subscale.

On the basis of the judgment of experienced interviewers, five participants were unable to understand and complete the survey, even with assistance from the interviewer, and two needed substantial help. Primarily, these participants struggled to understand text describing the five impacts on memory and thinking problems, DCE attributes, and choice questions. Results from the remaining eight participants who independently completed the survey and changes made to the survey are summarized in Sects. 3.3.2 and 3.3.3. Out of the eight participants who were able to complete the survey independently, three met the eligibility criteria based on both the telephone and in-person screening. Another five reported answers during the in-person pretests that were different than the answers they reported during the telephone screening and that would have made them ineligible because their cognitive impacts reported during the in-person pretests were too mild.

Screening Questions

All participants provided at least one inconsistent response when completing screening questions in person compared with completing screening question during the initial telephone screening; however, not all of these inconsistencies would have precluded participants from the pretest interviews (Table 3). Participants most frequently provided inconsistent responses to when the participant first discussed memory and thinking concerns with their doctor, what the doctor told them was the cause of memory and thinking problems, and with which daily activities they need help.

Table 3.

Consistency of responses to screening questions comparing telephone screening with responses at in-person interview in pretest interviewsa

Question Round 1 Round 2
Number of times answer changed in the in-person interview for round 1 pretest interviews Number of times the changed answer would have made the person ineligible for round 1 pretest interviews Number of times answer changed in the in-person interview for the round 2 pretest interviews Number of times the changed answer would have made the person ineligible for round 2 pretest interviews
S1. How old are you? 1 0 1 0
S2. Are you …? (gender) 0 0 0 0
S3. Do you live …? (living arrangements) 3 0 1 0
S4. Have you noticed that your memory is not what it used to be? 0 0 0 0
S5. Do you find that changes in your memory interfere with your daily activities? 4 4 2 2
S6. Over the last 6 months, has your memory: (Stayed the same or gotten worse) 6 6 2 2
S7. Did you mention your memory concerns to your physician? 1 1 0 0
S8. When did you first discuss your memory concerns with your doctor? 10 2 5 1
S9. What did your doctor say you had? (Memory related problems) 9 2 7 0
S10. Did the doctor offer a prescription medicine for your memory concerns? This could be a conversation that occurred even if you didn’t take the medication. 3 2 2 0
S11: Do you currently take a prescription medicine for memory concerns? Not asked Not asked 0 0
S12: What medicines do you currently take for your memory or thinking concerns? Not asked Not asked 0 0
S13. Have you been diagnosed with any of the following? (mental diseases) 1 1 1 1
S14. Which of the following do you need help with? (daily activities) 11 0 7 0
S15. Are you comfortable using a computer to type e-mails or look at websites for news or shopping? 0 0 0 0
Open in a new tab

aFor round 2 pretest interviews, respondents were screened online first, then by telephone. The discrepancies reported in this table are only for discrepancies between the telephone interview and the in-person interview

On the basis of responses to screening questions during in-person interviews, 11 of the 15 respondents would have been ineligible for the study, with eight reporting impacts that were too mild, two reporting impacts that were too advanced, and one reporting a stroke. Six of the seven participants who could not independently complete the survey would have been ineligible based on their responses to screening questions in person compared with responses via telephone screening. Specifically, two participants changed their answer about when they had first talked to their doctor to “more than 4 years ago,” which was an indicator of more severe disease and would have made them ineligible. Five of the seven reported currently taking a medicine for memory problems (not an exclusion criterion in round 1 interviews). Finally, three participants who could not complete the survey were the only participants self-reporting a formal AD diagnosis: “Alzheimer’s,” “early Alzheimer’s,” or “pre-Alzheimer’s.” No other notable patterns in types of reported memory and thinking concerns or other characteristics were identified to inform the differential ability to complete the survey.

Survey Instrument

For the eight participants who could independently complete the survey, all could answer questions about attribute descriptions, understand and remember the information presented, respond logically to the choice questions, and provide rationale for the choices they made. Specifically, these eight participants appeared to understand the distinctions among the different severities of memory and thinking problems (i.e., difference between “no problems,” “some problems,” and “more problems”) presented for each of the five impacts.

Despite understanding the text about “some problems” and “more problems,” it appeared participants did not follow instructions to answer the DCE questions as if they currently had “some problems.” During interviews, when asked what level of benefit they were thinking about, seven of these eight participants reported they were thinking about a medicine that would slow the decline from their current level of memory or thinking problems to a worse level, rather than imagining a decline from “some problems” to “more problems” as instructed. Nevertheless, participants understood and applied the concept of a modest incremental decline in cognitive problems when answering the DCE questions.

The eight participants who independently completed the survey accurately described trading different levels of AEs with a slowing of the decline in memory and thinking problems, indicating an understanding of choices offered in the DCE questions. However, some participants described having difficulty understanding the graphics used to present time until worsening of memory and thinking problems, as well as risk grids used to describe the risks of recurrent heart-related chest pain.

Changes to the Screening Procedure and Survey Instrument

The results from the round 1 interviews provided guidance on the changes to the screening criteria and screening process that might improve the ability of the screening questions to identify the target sample, as well as the changes to the survey instrument that would result in a survey that the target population could complete independently. Because seven participants in round 1 pretests could not independently complete the survey, and considering results from round 1 interviews, it seems a level of cognitive impairment exists beyond which participants cannot independently complete a DCE survey, and that cut-off may be distinguishable from those who can independently complete the survey. Thus, two adjustments were made to the screening approach for round 2 pretests: use of an online screener before the telephone screening, as the final goal was developing an online survey, and revision of two screening criteria to exclude those with more severe cognitive impairment. The revised screening criteria were the time since a person first spoke with their doctor about memory concerns (decreased from 4 years to 2 years) and exclusion of participants taking memantine (Namenda) or combination memantine/donepezil (Namzaric), given memantine is indicated for moderate-to-severe dementia (ESM). Finally, a question about self-reported ease of computer use for e-mail and websites was added; those uncomfortable with computer use were ineligible.

Three major changes were made to the DCE survey during and after round 1 pretest interviews. After the first six interviews, four with participants who were able to independently complete the survey, graphics illustrating “time until symptoms worsened” and risk grids depicting levels of noncardiac chest pain were replaced with text descriptions. Second, the question format was changed from offering a choice among two hypothetical medicines and no new medicine to offering a choice between no treatment and one hypothetical treatment to reduce the question complexity. Third, in the DCE question, the beneficial attribute was revised to describe a change from the participant’s current cognitive status and as the time until the participant’s memory and thinking problems got “noticeably worse.” Participants were observed to ignore instructions that they were to assume a hypothetical baseline and level of change using the terms “no problems,” “some problems,” and “more problems.” Therefore, these terms were removed from the DCE questions. Figure 3 presents an example of the revised DCE question used on the first day of round 2 pretest interviews.

Fig. 3.

Fig. 3

Open in a new tab

Example of a discrete-choice experiment question from round 2 pretest interviews

Round 2 Pretest Interviews

Procedure

In round 2 pretest interviews, a three-part sequential screening process was used: an online screener, telephone screening, and in-person rescreening to start the in-person interview. For the round 2 interviews, prespecified criteria were developed to assess the participants’ ability to independently complete the survey. Specifically, after each of the two practice DCE questions, one of the two interviewers asked the participant to explain the difference between two options presented in the DCE question. Each interviewer independently rated whether the participant was (1) able to explain the two options correctly without assistance, (2) able to explain the two options correctly after minor prompting, (3) able to explain the two options correctly after detailed prompting, or (4) not able to explain the two options correctly. The two interviewers also independently rated the participant’s ability to complete the full survey using a scale from 1, “the participant would not be able to complete the survey on their own or would need significant help,” to 5, “the participant would be able to complete the survey on their own with no more difficulty than what the interviewers would expect with a typical respondent to a DCE survey.”

Results

Twelve participants were recruited and interviewed. The participants had an average age of 59 years; most were female, married, and had a 4-year college or graduate degree (Table 1). Table 2 reports participants’ responses to some of the questions about memory and thinking problems, which were consistent with a population diagnosed with MCI or mild AD (using Petersen or NINCDS-ADRDA criteria, respectively) [31, 44–46].

All 12 participants in round 2 pretest interviews could understand and complete the revised survey instrument with little to no difficulty, based on the interviewers’ assessments and ratings (ESM). When comparing responses to screening questions across the three modes of administration (online, telephone, and in person), some inconsistencies in responses were noted. Four respondents would have been ineligible based on their responses during the in-person interviews (Table 3): three were too mildly affected and one reported brain trauma during the in-person interview.

Discussion

As the investigation of potentially disease-modifying AD treatments continues, understanding patient preferences for such treatments and willingness of patients to accept risks to attain treatment benefit will be a consideration in drug development and regulatory/health authority decision making. Designing a patient preference survey for disease-modifying AD treatments presents additional challenges compared with constructing a patient preference survey for populations that are not cognitively impaired.

On the basis of previous qualitative interviews [29], a preliminary DCE survey was developed to collect preferences from participants with cognitive concerns consistent with an early AD clinical syndrome. However, in round 1 pretests, seven of the 15 participants could not independently complete the survey. Screening question responses associated with difficulty completing the survey included questions about how long ago participants visited their doctor because of memory concerns and the use of medicines for memory and thinking concerns. Both suggest a higher degree of cognitive impairment for those unable to independently complete the survey compared with the intended target population.

Among those participants independently completing the survey, some struggled to understand the graphics used to communicate how long the treatment slowed cognitive decline and the chance a risk would be experienced. Anecdotally, we have encountered problems with time-to-event graphics in other surveys, although the graphics we have used to convey risk generally have been well understood [47]. The literature on the use of graphics to communicate risk and other attributes is mixed [48, 49]. It may be that typical graphics used in DCE surveys, such as risk grids, are less helpful in cognitively impaired populations who struggle with abstract thinking. More research on the best approach to communicate attributes would inform future survey design in this population.

While the eight participants in round 1 who completed the survey met eligibility criteria based on their responses to the telephone screening prior to in-person interviews, five others changed their answers to a few screening questions during the in-person pretest in a manner that would have made them ineligible because their cognitive impacts were too mild. However, these participants still provided useful feedback on the survey during the pretest interviews. Specifically, the input from these five participants helped clarify survey text and graphics that were confusing for many participants when compared with survey text and graphics that seemed confusing only for people with greater cognitive challenges. Researchers designing surveys for patients with cognitive decline might consider including respondents with milder problems during initial pretests of the survey instrument, thus identifying issues with the survey, before presenting the survey to patients with more advanced problems. The results highlighted the importance of carefully pretesting graphics used in any preference survey, to confirm that the graphics are easy to understand and support the interpretation of the attributes. The variability among individuals with cognitive decline may make designing and testing graphics difficult.

In round 2 pretest interviews, eligibility criteria regarding the time since speaking with a doctor and the use of symptomatic AD medicines were revised with the aim of targeting earlier stages of disease. In addition, this round of screening was conducted through an online screener, with a future goal to launch an online survey. Round 2 participants still discussed problems associated with cognitive impairment, consistent with an early AD clinical syndrome, but they were judged capable of independently completing the survey.

The main limitation of this work is the abstract beneficial attribute. There is no validated crosswalk between an instrument that has been consistently used in clinical trials such as the ADAS-Cog and a set of activities that are relevant and relatable to respondents. For this survey, categorization of ADAS-Cog-11 items from Appels and Scherder [50] and the scoring guide for the ADAS-Cog [34] combined were used to integrate the clinical and practical aspects related to incremental decline in memory and thinking. Specific levels for the five impacts were included to establish a reference level of memory and thinking problems and achieve a uniform baseline: hypothetically progressing from “some problems” (i.e., forget appointments once or twice a month) to “more problems” (i.e., forget appointments two to three times a week). When participants struggled to follow this instruction, a less uniform baseline was collected for the round 2 pretest, whereby participants were asked to use their current cognitive status as their baseline for the scenario rather than assuming a hypothetical baseline. Asking respondents to use their current cognitive status as the baseline makes it more difficult for the researchers to know how large a benefit each respondent assumed they would receive in exchange for the risks. This limitation may be present for any stated-preference survey designed to collect patient preferences on slowing cognitive decline given the complexity of cognition, the spectrum of AD clinical severity, and the diversity of patients with AD. The challenge participants faced by assuming a hypothetical baseline may be important for other health preference studies, as many studies ask respondents to place themselves in a hypothetical baseline. Again, careful testing of the survey instrument is important for the interpretation of the preference data gathered in a study.

Another limitation was the lack of formal assessment within the examples of whether moving from one level to the next would be rated as a “noticeably worse decline in memory and thinking.” Challenges of recall-based screening for those with cognitive impairment, while still present, are relatively minimized for the target population of an early AD clinical syndrome. On the basis of round 2 pretesting results, participants seemed capable of appropriate recall for study screening. Careful testing of screening questions is as important as testing other survey questions, to assure that the questions identify the sample of interest for the study. The survey has not been administered, thus results from the survey are not available.

Conclusions

This work is intended to provide guidance in patient preference research within the field of dementia. Taken together, our results suggest that the severity, not presence, of cognitive impairment determines whether a participant can independently complete a stated-preference survey. Thus, when recruiting those with cognitive impairment, care should be taken to characterize where participants may be along the clinical continuum of the disease impacting cognition. On the basis of round 2 pretest results, the refined screening questions and mode of screening successfully identified those with self-reported cognitive concerns consistent with an early AD clinical syndrome who could independently complete the refined DCE survey. The survey included careful drafting and revision of the attribute description that was meaningful and understandable to participants. Results also suggest that a simplified DCE survey for use in the target population is possible. Careful pretesting and inclusion of comprehension questions to assess respondent understanding are important for any study but are crucial for studies in populations with cognitive decline.

Patient preference research for disease-modifying AD treatments would be greatly enhanced by developing a clinical instrument suitable for both clinical measurement of cognitive and functional decline and patient preference studies that used relevant examples from a patient’s life. This study, along with ongoing work by Alzheimer’s Disease Patient and Caregiver Engagement [17], may inform regulators and other relevant decision makers on disease-modifying AD treatment assessments. Further, these results, possibly in combination with results forthcoming from IMI-PREFER (e.g., in those with neuromuscular disease [51]), may inform researchers and other stakeholders on preference research in persons with cognitive impairment.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 43 KB) (42.7KB, docx)
Supplementary file2 (DOCX 19 KB) (18.3KB, docx)
Supplementary file3 (DOCX 19 KB) (18.2KB, docx)
Supplementary file4 (DOCX 16 KB) (16.3KB, docx)

Acknowledgements

The authors acknowledge the contributions of Kelley Myers who helped conduct round 2 pretest interviews, Kate Klein and Josh Coulter for their aid in conducting independent quality control of the data, and Kimberly Moon for overall project management. Additionally, the authors thank Eric Siemers and Kristin Wrobleski for substantial contributions to the study design and interpretation of initial data provided while employed at Eli Lilly and Company.

Author Contributions

All authors contributed to the design of the study and development of the study materials, including the survey instruments. Patient interviews were conducted by CM, DD, JS, and BH. CM, JS, and BH were responsible for conducting the research, analysis, and interpretation of the results. All authors were responsible for reviewing and revising the manuscript and gave their approval for publication of this version.

Declarations

Funding

The study was funded by Eli Lilly and Company.

Conflicts of interest

Carol Mansfield and Dana DiBenedetti are employees of RTI Health Solutions, which was contracted by Eli Lilly and Company to conduct the study. Kristin Bullok, Jillian Vinci Fuhs, Brandy R Matthews, and Antje Tockhorn-Heidenreich are employed by and stakeholders in Eli Lilly and Company. J. Scott Andrews was employed by Eli Lilly at the time this study was conducted and is currently an employee of Takeda Pharmaceuticals. Joshua Darling was employed at Eli Lilly and Company at the time this study was conducted and is currently an employee of Seagen Inc. Jessie Sutphin was an employee of RTI Health Solutions at the time this study was conducted and is currently an employee with Duke University. Brett Hauber was an employee of RTI Health Solutions at the time this study was conducted and is currently an Affiliate Associate Professor in the School of Pharmacy at the University of Washington and an employee of Pfizer, Inc.

Ethics approval

Approval for the study was obtained from the RTI International Institutional Review Board (No. 14025).

Consent to participate

Written informed consent was obtained from all study participants.

Consent for publication

Written informed consent was obtained from all study participants.

Availability of data and material

All relevant data have been provided in the publication.

Code availability

Not applicable.

References

  • 1.Medical Device Innovation Consortium. Patient centered benefit-risk project report: a framework for incorporating information on patient preferences regarding benefit and risk into regulatory assessments of new medical technology. 2015. https://mdic.org/wp-content/uploads/2018/05/MDIC_PCBR_Framework_Web.pdf. Accessed 24 Feb 2021.
  • 2.US Food and Drug Administration. 21st Century Cures Act. https://www.fda.gov/regulatory-information/selected-amendments-fdc-act/21st-century-cures-act. Accessed 13 Apr 2021.
  • 3.IMI-PREFER. Innovation medicine initiative: patient preferences in benefit-risk assessments during the drug life cycle. 2016. https://www.imi-prefer.eu/about/. Accessed 24 Feb 2021.
  • 4.European Medicines Agency. EMA regulatory science to 2025: strategic reflection. 2020. https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/ema-regulatory-science-2025-strategic-reflection_en.pdf. Accessed 17 Nov 2020.
  • 5.Bouvy JC, Cowie L, Lovett R, Morrison D, Livingstone H, Crabb N. Use of patient preference studies in HTA decision making: a NICE perspective. Patient. 2020;13(2):145–149. doi: 10.1007/s40271-019-00408-4. [DOI] [PubMed] [Google Scholar]
  • 6.Hauber AB, Mange B, Zhou M, Chaudhuri S, Benz HL, Caldwell B, et al. Parkinson’s patients’ tolerance for risk and willingness to wait for potential benefits of novel neurostimulation devices: a patient-centered threshold technique study. MDM Policy Pract. 2021;6(1):2381468320978407. doi: 10.1177/2381468320978407. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Benz HL, Caldwell B, Ruiz JP, Saha A, Ho M, Christopher S, et al. Patient-centered identification of meaningful regulatory endpoints for medical devices to treat Parkinson’s disease. MDM Policy Pract. 2021;6(1):1–13. doi: 10.1177/23814683211021380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Dubois B, Feldman HH, Jacova C, Cummings JL, Dekosky ST, Barberger-Gateau P, et al. Revising the definition of Alzheimer’s disease: a new lexicon. Lancet Neurol. 2010;9:1118–1127. doi: 10.1016/S1474-4422(10)70223-4. [DOI] [PubMed] [Google Scholar]
  • 9.Jack CR, Jr, Bennett DA, Blennow K, Carrillo MC, Dunn B, Haeberlein SB, et al. NIA-AA research framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dement. 2018;14(4):535–562. doi: 10.1016/j.jalz.2018.02.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.US FDA. Early Alzheimer’s disease: developing drugs for treatment; draft guidance for industry. February 2018. https://www.fda.gov/media/110903/download. Accessed 24 Feb 2021.
  • 11.Klein G, Delmar P, Voyle N, Rehal S, Hofmann C, Abi-Saab D, et al. Gantenerumab reduces amyloid-β plaques in patients with prodromal to moderate Alzheimer’s disease: a PET substudy interim analysis. Alzheimers Res Ther. 2019;11:101. doi: 10.1186/s13195-019-0559-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Siemers ER, Sundell KL, Carlson C, Case M, Sethuraman G, Liu-Seifert H, et al. Phase 3 solanezumab trials: secondary outcomes in mild Alzheimer’s disease patients. Alzheimers Dement. 2016;12(2):110–120. doi: 10.1016/j.jalz.2015.06.1893. [DOI] [PubMed] [Google Scholar]
  • 13.Honig LS, Vellas B, Woodward M, Boada M, Bullock R, Borrie M, et al. Trial of solanezumab for mild dementia due to Alzheimer’s disease. N Engl J Med. 2018;378:321–330. doi: 10.1056/NEJMoa1705971. [DOI] [PubMed] [Google Scholar]
  • 14.Mintun MA, Lo AC, Duggan Evans C, Wessels AM, Ardayfio PA, Andersen SW, et al. Donanemab in early Alzheimer's disease. N Engl J Med. 2021;384(18):1691–1704. doi: 10.1056/NEJMoa2100708. [DOI] [PubMed] [Google Scholar]
  • 15.Swanson CJ, Zhang Y, Dhadda S, Wang J, Kaplow J, Lai RYK, et al. A randomized, double-blind, phase 2b proof-of-concept clinical trial in early Alzheimer’s disease with lecanemab, an anti-Aβ protofibril antibody. Alzheimers Res Ther. 2021;13(1):80. doi: 10.1186/s13195-021-00813-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Aduhelm package insert, 2021. https://www.biogencdn.com/us/aduhelm-pi.pdf. Accessed 20 Jun 2021.
  • 17.DiBenedetti D, Slota C, Wronski SL, Vradenburg G, Comer M, Callahan LF, et al. Assessing what matters most to patients with or at risk for Alzheimer’s and care partners: a qualitative study evaluating symptoms, impacts, and outcomes. Alzheimers Res Ther. 2020 doi: 10.1186/s13195-020-00659-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Hartry A, Aldhouse NVJ, Al-Zubeidi T, Sanon M, Stefanacci RG, Knight SL. The conceptual relevance of assessment measures in patients with mild/mild-moderate Alzheimer’s disease. Alzheimers Dement (Amst). 2018;10:498–508. doi: 10.1016/j.dadm.2018.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Ho M, Saha A, McCleary KK, Levitan B, Christopher S, Zandlo K, et al.; Medical Device Innovation Consortium’s Patient Centered Benefit-Risk Steering Committee. A framework for incorporating patient preferences regarding benefits and risks into regulatory assessment of medical technologies. Value Health. 2016;19(6):746–50. [DOI] [PubMed]
  • 20.González JM, Johnson FR, Levitan B, Noel R, Peay H. Symposium title: preference evidence for regulatory decisions. Patient. 2018;11(5):467–473. doi: 10.1007/s40271-018-0311-y. [DOI] [PubMed] [Google Scholar]
  • 21.Hauber AB, Fairchild AO, Johnson FR. Quantifying benefit-risk preferences for medical interventions: an overview of a growing empirical literature. Appl Health Econ Health Policy. 2013;11(4):319–329. doi: 10.1007/s40258-013-0028-y. [DOI] [PubMed] [Google Scholar]
  • 22.Ho MP, González JM, Lerner HP, Neuland CY, Whang JM, McMurry-Heath M, et al. Incorporating patient-preference evidence into regulatory decision making. Surg Endosc. 2015;29(10):2984–2993. doi: 10.1007/s00464-014-4044-2. [DOI] [PubMed] [Google Scholar]
  • 23.Oremus M, Tarride JE, Pullenayegum E, Clayton N; Canadian Willingness-to-Pay Study Group, Raina P. Patients’ willingness-to-pay for an Alzheimer’s disease medication in Canada. Patient. 2013;6(3):161–8. [DOI] [PubMed]
  • 24.Johnson FR, DiSantostefano RL, Yang J-C, Reed SD, Streffer J, Levitan B. Something is better than nothing: the value of active intervention in stated preferences for treatments to delay onset of Alzheimer’s disease symptoms. Value Health. 2019;22(9):1063–1069. doi: 10.1016/j.jval.2019.03.022. [DOI] [PubMed] [Google Scholar]
  • 25.Oremus M, Tarride JE, Raina P, Thabane L, Foster G, Goldsmith CH, et al. The general public’s willingness to pay for tax increases to support unrestricted access to an Alzheimer’s disease medication. Pharmacoeconomics. 2012;30(11):1085–1095. doi: 10.2165/11594180-000000000-00000. [DOI] [PubMed] [Google Scholar]
  • 26.Hauber AB, Johnson FR, Fillit H, Mohamed AF, Leibman C, Arrighi HM, et al. Older Americans’ risk-benefit preferences for modifying the course of Alzheimer’s disease. Alzheimer Dis Assoc Disord. 2009;23(1):23–32. doi: 10.1097/WAD.0b013e318181e4c7. [DOI] [PubMed] [Google Scholar]
  • 27.Oremus M, Tarride JE, Pullenayegum E, Clayton N, Mugford G, Godwin M, et al.; Canadian Willingness-To-Pay Study Group, Raina P. Caregivers’ willingness-to-pay for Alzheimer’s disease medications in Canada. Dementia (London). 2015;14(1):63–79. [DOI] [PubMed]
  • 28.Hauber AB, Mohamed AF, Johnson FR, Cook M, Arrighi HM, Zhang J, et al. Understanding the relative importance of preserving functional abilities in Alzheimer’s disease in the United States and Germany. Qual Life Res. 2014;23(6):1813–1821. doi: 10.1007/s11136-013-0620-5. [DOI] [PubMed] [Google Scholar]
  • 29.Mansfield C, Bullok K, Fuhs JV, Wrobleski KK, Tockhorn-Heidenreich A, Andrews JS, et al. The patient voice: exploring treatment preferences in participants with mild cognitive concerns to inform regulatory decision making. Poster presented at the Alzheimer’s Association International Conference; 16–20 July 2017; London.
  • 30.Salloway S, Sperling R, Fox NC, Blennow K, Klunk W, Raskind M, et al. Two phase 3 trials of bapineuzumab in mild-to-moderate Alzheimer’s disease. N Engl J Med. 2014;370(4):322–333. doi: 10.1056/NEJMoa1304839. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Henneges C, Reed C, Chen YF, Dell’Agnello G, Lebrec J. Describing the sequence of cognitive decline in Alzheimer’s disease patients: results from an observational study. J Alzheimers Dis. 2016;52(3):1065–1080. doi: 10.3233/JAD-150852. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Honig LS, Mayeux R. Natural history of Alzheimer’s disease. Aging (Milano). 2001;13(3):171–182. doi: 10.1007/BF03351476. [DOI] [PubMed] [Google Scholar]
  • 33.Doody RS, Thomas RG, Farlow M, Iwatsubo T, Vellas B, Joffe S, et al.; Alzheimer’s Disease Cooperative Study Steering Committee; Solanezumab Study Group. Phase 3 trials of solanezumab for mild-to-moderate Alzheimer’s disease. N Engl J Med. 2014;370(4):311–21. [DOI] [PubMed]
  • 34.Mohs RC. Administration manual for the Alzheimer’s Disease Assessment Scale. Adapted from the administration and scoring manual for the Alzheimer’s Disease Assessment Scale. New York (NY): The Mount Sinai School of Medicine; 1994.
  • 35.Schrag A, Schott JM. What is the clinically relevant change on the ADAS-Cog? J Neurol Neurosurg Psychiatry. 2012;83:171–173. doi: 10.1136/jnnp-2011-300881. [DOI] [PubMed] [Google Scholar]
  • 36.Petersen RC, Aisen PS, Beckett LA, Donohue MC, Gamst AC, Harvey DJ, et al. Alzheimer’s Disease Neuroimaging Initiative (ADNI): clinical characterization. Neurology. 2010;74(3):201–209. doi: 10.1212/WNL.0b013e3181cb3e25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Budd Haeberlein S, O’Gorman J, Chiao P, Bussière T, von Rosenstiel P, Tian Y, et al. Clinical development of aducanumab, an anti-Aβ human monoclonal antibody being investigated for the treatment of early Alzheimer’s disease. J Prev Alzheimers Dis. 2017;4(4):255–263. doi: 10.14283/jpad.2017.39. [DOI] [PubMed] [Google Scholar]
  • 38.Carlson C, Siemers E, Hake A, Case M, Hayduk R, Suhy J, et al. Amyloid-related imaging abnormalities from trials of solanezumab for Alzheimer’s disease. Alzheimers Dement (Amst). 2016;2:75–85. doi: 10.1016/j.dadm.2016.02.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Sperling R, Salloway S, Brooks DJ, Tampieri D, Barakos J, Fox NC, et al. Amyloid-related imaging abnormalities in patients with Alzheimer’s disease treated with bapineuzumab: a retrospective analysis. Lancet Neurol. 2012;11(3):241–249. doi: 10.1016/S1474-4422(12)70015-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Sperling RA, Jack CR, Black SE, Frosch MP, Greenberg SM, Hyman BT, et al. Amyloid-related imaging abnormalities in amyloid-modifying therapeutic trials: recommendations from the Alzheimer’s Association Research Roundtable Workgroup. Alzheimers Dement. 2011;7:367–385. doi: 10.1016/j.jalz.2011.05.2351. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.McKhann GM, Chertkow H, Hyman BT, Jack CR, Kawas CH, Klunk WE, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):263–269. doi: 10.1016/j.jalz.2011.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Albert MS, Dickson D, Dubois B, Feldman HH, Fox NC, Gamst A. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):270–279. doi: 10.1016/j.jalz.2011.03.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Alzheimer’s Association. 2015 Alzheimer’s disease facts and figures. Alzheimers Dement. 2015;11(3):332–84. [DOI] [PubMed]
  • 44.Brown PJ, Devanand DP, Liu X, Caccappolo E. Functional impairment in elderly patients with mild cognitive impairment and mild Alzheimer disease. Arch Gen Psychiatry. 2011;68(6):617–626. doi: 10.1001/archgenpsychiatry.2011.57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999;56(3):303–308. doi: 10.1001/archneur.56.3.303. [DOI] [PubMed] [Google Scholar]
  • 46.McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34:939–944. doi: 10.1212/wnl.34.7.939. [DOI] [PubMed] [Google Scholar]
  • 47.Mansfield C, Poulos C, Boeri M, Hauber B. Performance of a comprehension question in discrete-choice experiment surveys (DCE). Poster presented at the ISPOR 2019 European Conference; Copenhagen, Denmark. 5 November 2019. [abstract] Value Health. 2019;22(S3).
  • 48.Fagerlin A, Zikmund-Fisher BJ, Ubel PA. Helping patients decide: ten steps to better risk communication. J Natl Cancer Inst. 2011;103(19):1436–1443. doi: 10.1093/jnci/djr318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Veldwijk J, Lambooij MS, van Til JA, Groothuis-Oudshoorn CG, Smit HA, de Wit GA. Words or graphics to present a discrete choice experiment: does it matter? Patient Educ Couns. 2015;98(11):1376–1384. doi: 10.1016/j.pec.2015.06.002. [DOI] [PubMed] [Google Scholar]
  • 50.Appels BA, Scherder E. Review: the diagnostic accuracy of dementia-screening instruments with an administration time of 10 to 45 minutes for use in secondary care: a systematic review. Am J Alzheimers Dis Other Demen. 2010;25(4):301–316. doi: 10.1177/1533317510367485. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Jimenez-Moreno AC, van Overbeeke E, Pinto CA, Smith I, Sharpe J, Omrod J, et al. Patient preferences in rare diseases: a qualitative study in neuromuscular disorders to inform a quantitative preference study. Patient. 2021;14:601–612. doi: 10.1007/s40271-020-00482-z. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary file1 (DOCX 43 KB) (42.7KB, docx)
Supplementary file2 (DOCX 19 KB) (18.3KB, docx)
Supplementary file3 (DOCX 19 KB) (18.2KB, docx)
Supplementary file4 (DOCX 16 KB) (16.3KB, docx)

Articles from The Patient are provided here courtesy of Springer

RESOURCES