Abstract
Background
Recovery-promoting drugs (RPDs) offer a pharmacological approach to improve motor recovery in stroke survivors. Investigating stroke survivors’ preferences for taking RPDs would allow preferences to be considered during development of intervention protocols and optimise participant adherence. The aim of this study was to develop and pilot a robust discrete choice experiment (DCE) that examines stroke survivors’ preferences for RPDs in combination with physical training to improve motor recovery. Pilot objectives were to assess the: (1) effectiveness of the data collection strategy; (2) effectiveness of the survey instrument design; and (3) robustness of the discrete choice experiment design.
Methods
An online DCE with two-alternative choice tasks consisting of six attributes (efficacy, out-of-pocket cost, side effects, treatment duration, dosage, and risk) depicting a stroke rehabilitation program involving RPDs. Respondents completed eleven choice tasks each: nine choice tasks divided into four blocks, plus a dominant choice and duplicate task. Participants were community-dwelling Australian stroke survivors. Outcomes included measures of data completeness, choice validity, and content validity. Conditional logit model analysis was used to estimate the marginal willingness-to-pay, and β coefficients for attributes and levels.
Results
Seventy-three stroke survivors completed the online DCE, with a survey completion rate of 89%. The survey instrument design was effective, with 94.5% and 84.9% respondents passing the dominant and duplicate choice tasks respectively. The DCE design was robust with clear and appropriate attributes and levels, accurate choice tasks and experimental design.
Conclusions
Systematic development and piloting of this DCE has confirmed its robustness and feasibility. This study provides an example of robust procedures and reporting requirements for developing and piloting robust discrete choice experiments.
Supplementary Information
The online version contains supplementary material available at 10.1186/s40814-025-01760-z.
Keywords: Stroke recovery, Stroke rehabilitation, Drugs, Medication, Therapeutics, Patient preferences, Discrete choice experiment, Pilot study
Key messages regarding feasibility
- What uncertainties existed regarding the feasibility?
- The feasibility of the DCE to quantitatively examine stroke survivors’ preferences for RPDs in combination with rehabilitation to improve motor recovery was unknown, and it was therefore necessary to assess the 1) effectiveness of the data collection strategy; 2) effectiveness of the survey instrument design; and 3) robustness of the DCE design prior to full implementation.
- What are the key feasibility findings?
- Systematic development and piloting of this DCE has confirmed its robustness and feasibility, and its suitability for full implementation of the main study. The data collection strategy and survey instrument design were found to be effective, and the experimental design was found to be robust.
- What are the implications of the feasibility findings for the design of the main study?
- It is reasonable to expect that the fully deployed DCE should return valid and reliable results that will provide usable insights into stroke survivors’ preferences for RPDs.
Background
Recovery-promoting drugs (RPDs) are a pharmacological intervention that may promote motor recovery in stroke survivors. RPDs are envisaged to increase outcomes gained during rehabilitation or expedite recovery to achieve the same outcomes in a shorter period [1–3]. Unfortunately, recent large international trials have cast doubt on one of the most promising RPD candidates investigated, concluding that functional outcomes after acute stroke did not improve after 6 months of taking the trial intervention, on a background of usual care dose and content of rehabilitation [4–6]. The main limitation to these trials was the moderate adherence to the drug intervention/placebo, which was measured through respondent self-report via monthly phone calls during the trial and a count of unused capsules at the end of the trial period [4, 5]. Respondent self-report has been previously shown to overestimate adherence by as much as 200% [7]. Furthermore, medication adherence and persistence are low in individuals after stroke [8]. The question that remains is whether the results indicate that RPDs were not effective or rather that there were issues with the trial protocol or participants’ adherence to the RPD intervention that influenced these trial outcomes. If poor adherence to the RPD intervention did affect the outcomes of the RPD trials, identifying the factors which influenced decision-making about whether to take each dose of RPD is important.
With multiple drug candidates that could plausibly promote recovery after stroke, understanding stroke survivors’ preferences is essential for designing person-centred trials to improve adherence and ultimately optimise treatment protocols for better recovery outcomes. Accommodating stroke survivors’ preferences for interventions combining RPD with rehabilitation to support adherence to trial protocols could lead to a scientifically robust intervention and more reliable trial results. It is well established that compliance is greater when interventions are consistent with patient preferences [9]. Given the importance of stroke survivor adherence to determine effectiveness of RPDs, understanding preferences for how they can be incorporated into rehabilitation protocols remains an important area of study. Given the variety of influences on stroke survivors preferences and choices, a discrete choice experiment (DCE) is an ideal method to understand their preferences in this area. DCEs are designed to capture trade-offs and quantify preferences, reflect real-world decision-making, engage patients in research and support value-based healthcare [10–13].
A DCE must be well-designed, easy to understand, comprehensive enough to collect sufficient data for analysis, and effectively distributed to provide reliable insight into the preferences of the target population [13, 14]. Otherwise there is a risk that DCE respondents may adopt simplified decision-making approaches, experience cognitive fatigue due to a high number of choice tasks, be confused by poorly defined attributes and levels, or be inattentive to the tasks due to the hypothetical nature of DCEs thereby impacting the quality of their response to each choice task [15, 16]. Conversely, apparent violations might instead be deliberate choices based on the respondent’s preferences, or due to expected randomness in choices that is inherent with DCEs [17]. Unexpected responses may also be due to (1) research design, where the questions are not asked appropriately; (2) data analysis, where the responses weren’t interpreted accurately; or (3) researchers making assumptions about what respondent preferences should be. These points must all be considered when assessing the overall validity and reliability of DCE results [13].
The aim of this study was to develop and pilot a robust DCE that quantitatively examines stroke survivors’ preferences for RPDs in combination with rehabilitation to improve motor recovery. To confirm the feasibility of the DCE prior to commitment to full implementation, the objectives of this study were to develop the DCE experimental design and survey tool, and then pilot the DCE to assess the (1) effectiveness of the data collection strategy; (2) effectiveness of the survey instrument design; and (3) robustness of the DCE design.
Methods
This study was conducted in two parts. Part one was devoted to the development of the DCE prototype and part two focused on piloting the DCE prototype. The study is described here according to the relevant sections of the good research practices for conjoint analysis application in health from the International Society For Pharmacoeconomics and Outcomes Research (ISPOR) [13]. Ethics approval for this study was granted by James Cook University Human Research Ethics Committee (H7620).
Part 1—development of DCE prototype
This DCE was developed in five stages: (1) research question and study perspective development; (2) attribute and level selection; (3) choice task construction and experimental design; (4) survey instrument design; and (5) analysis strategy development.
Stage 1: research question and study perspective development
The research question ‘What are stroke survivors’ preferences for RPDs in combination with rehabilitation to improve motor recovery?’ was borne from the conflict between the evidence from preliminary investigations into the potential benefits of RPDs and the unfavourable results of recent RPD clinical trials. This, coupled with evidence of poor adherence to pharmacological and rehabilitation interventions, impacts both stroke survivor outcomes and trial results. The research question was designed to capture the perspective of stroke survivors as the intended end-users of RPD interventions, recognising that their treatment preferences would be critical factors in RPD trial success. As RPDs are still experimental and therefore, a hypothetical concept, stroke survivors with experience of pharmacological interventions for the purpose of improving motor outcomes were not available.
Stage 2: attribute and level selection
Attributes were identified following good research practices [13, 18] through focus groups with stroke survivors exploring rehabilitation experiences and medication trade-offs [10]; literature on stroke survivor preferences for rehabilitation, stroke survivor medication adherence, and RPD trials [4–6, 8, 11, 12, 19, 20]; and reflective sessions with senior rehabilitation clinicians, with attributes emerging from recurrent themes across all sources. Attribute levels were determined from: clinical trial results [4–6, 20]; adverse drug event warnings listed in Product Information documents for fluoxetine and citalopram (two medications from the selective serotonin reuptake inhibitor antidepressant drug class that have been investigated for their recovery-promoting benefits) [21, 22]; current out-of-pocket costs for medications through the Australian Pharmaceutical Benefits Scheme (PBS) [23]; and experimental treatment protocols for emerging RPD interventions [2]. While there was no limit placed on the number of attributes or levels, reducing the cognitive burden for survey respondents whilst still allowing for reliable valuation data to be obtained was an important consideration [13, 18]. Five attributes with four levels and one attribute with two levels selected are presented in Table 1.
Table 1.
Attributes and levels
| Attribute | Description (provided to respondents) | Levels • Reference level in italics • A priori expectation (most likely respondent preference) in bold |
|---|---|---|
|
Clinical efficacy (continuous) |
How well it (the medication) works. | Mostly recovered |
| A lot of recovery | ||
| Quite a bit of recovery | ||
| Some recovery | ||
|
Out of pocket cost (continuous) |
Monthly cost (of medication) in Australian dollars. | No cost to you |
| $25 | ||
| $50 | ||
| $75 | ||
|
Side effects (categorical) |
Possible side effects (of medication). | No side effects |
| Severe nausea only | ||
| Mild nausea and mild sexual dysfunction that occurs some of the time | ||
| Severe sexual dysfunction that occurs the entire time you take the medication | ||
|
Treatment duration (continuous) |
How long you have to take the medication. | Two weeks |
| Three months | ||
| One year | ||
| Two years | ||
|
Dosage (categorical) |
How you will to take the medication. | 1 tablet every morning at home |
| 1 tablet three times a day at home | ||
| 1 subcutaneous injection (into skin folds on your stomach or thigh) once a week at home, administered by yourself | ||
| 1 tablet 30 minutes prior to rehab session at hospital outpatient clinic | ||
|
Risk (continuous) |
The risks of taking the medication. | 1% fracture risk |
| 3% fracture risk |
Stage 3: experimental design
Choice tasks were constructed using Ngene software (version 1.3.0) [24]. Each choice task contained two options with a full profile of all six attributes. No opt-out or status-quo option was provided to maximise the information gained about the trade-offs made and to reduce bias due to how participants evaluate opt-out options [25, 26]. An experimental design with optimal D-efficiency was generated to produce 36 choice tasks, which were divided into four blocks so each respondent would answer nine choice tasks from the generated design [24]. The preference parameters were set at zero during the initial DCE design in the absence of published information about stroke survivors’ preferences for RPDs. An example choice task is presented in Fig. 1.
Fig. 1.
Example choice task, which is also a dominant choice task
Stage 4: survey instrument design
The DCE was presented as an online survey via Qualtrics (Qualtrics, Provo, UT, USA) for self-administration by participants [27, 28]. The survey contained five sections: (1) Introduction to the aims of the study, details of the research team, and consent to participate; (2) respondent demographics and their stroke rehabilitation experience, through closed questions with pre-defined response options; (3) a video explaining how to complete the DCE and describing the attributes and levels in detail (with PDF alternative for download); (4) DCE study questions; and (5) post-DCE evaluation (see Additional file 1). Respondents were randomly allocated to one of four blocks of choice tasks. The survey was pre-tested by two stroke survivors using the think-aloud method [29, 30]. Pre-testing conducted by the primary investigator focused on content validity, cognitive burden, credibility, interpretation, usability, and time taken to complete. Based on pre-testing feedback, some wording of introductory information and questions were revised to improve clarity.
Stage 5: DCE analysis strategy development
The analysis strategy was developed using a synthesis of best practice guidelines and published DCE papers [13, 18, 31]. Descriptive statistics were selected to analyse respondent characteristics. Conditional logit modelling (CLM) and mixed logit modelling (MXL) was selected to estimate the strength of preferences for the attributes and attribute levels, and heterogeneity of preferences within the sample [31, 32]. To ensure validity and reliability of the DCE, quality of responses were assessed using a range of common methods [16, 32, 33].
Part 2—pilot of DCE prototype
Data collection and data analysis for the DCE prototype was piloted to determine the (1) effectiveness of the data collection strategy; (2) effectiveness of the survey instrument design; and (3) robustness of the DCE design (Table 2).
Table 2.
Analysis strategy
| Study objective | Outcome measure | Analysis strategy |
|---|---|---|
| 1. Effectiveness of the data collection strategy | ||
| (i) Respondent characteristics | Data sample: target population (Australian stroke survivors) | Descriptive statistics |
| (ii) Survey administration methods | Respondent dropout rate | Descriptive statistics |
| Survey completion times | Descriptive statistics | |
| 2. Effectiveness of the survey instrument design | ||
| Monotonicity | Proportion of respondents that selected the dominant choice | |
| Choice validity | Proportion of respondents that answered the repeated choice tasks differently | |
| Straight-lining (or flat-lining) | Number of respondents that answer the choice tasks in a straight line were counted | |
| 3. Robustness of the DCE design | ||
| (i) Clarity and appropriateness of the attributes and levels | Content validity | Magnitude and direction of the preference weights (β coefficient) calculated for each attribute ± levels using CLM, compared with the a priori estimates |
| Marginal willingness-to-pay (mWTP) | Cost attribute β coefficient: comparison attribute β coefficient | |
| (ii) Accuracy of the choice tasks and experimental design | Content validity | Magnitude and direction of the preference weights (β coefficient) calculated for each attribute ± levels using CLM, compared with the a priori estimates |
| Marginal willingness-to-pay (mWTP) | Cost attribute β coefficient: comparison attribute β coefficient | |
Stage 1: data collection
Data were collected using the DCE prototype from eligible respondents who were aged 18 years and over and who self-identified as having a previous clinical diagnosis of stroke, had undertaken rehabilitation in some form for their stroke and were able to provide informed consent. No minimum time period post-rehabilitation was required for participation. Respondents were recruited via social media and through the Stroke Foundation of Australia website, via a hyperlink or QR-code.
The target sample size of 60 represented 20% of the full DCE sample target (n = 300), the latter value based on Orme’s recommendations for detecting main effects [13, 18, 34]. This proportion aligns with guidance for pilot studies where the objective is to estimate parameters that will inform the sample size calculation and design of the definitive study [35]. For this pilot DCE, the sample of 60 was considered sufficient to: (1) assess the effectiveness of recruitment methods and completion rates; (2) evaluate the accuracy of choice task construction and comprehension; and (3) generate preliminary parameter estimates to serve as robust priors for developing an updated D-efficient experimental design with informative priors for the full-scale study.
Stage 2: data analysis
To assess the effectiveness of the data collection methods, the representativeness of respondents compared to the target population of Australian stroke survivors was assessed, along with the respondent dropout rate and survey completion times.
Validity and reliability tests were employed to assess the effectiveness of the survey instrument design [15]. Two extra choice tasks were incorporated into the survey: a dominant choice task to measure within-set monotonicity, and a duplicate choice task to measure choice validity (or stability) [15, 16]. These tests were used to provide a general impression of the design based on the collective monotonicity rate [33], rather than to assess the reliability of individual results. Straight-lining (or flat-lining) describes the response pattern where the respondent always or nearly always chooses the same alternative in the same position. The likelihood that the more preferred alternative will always appear in the same position within a choice task is small enough to suggest that respondents who straight-line are not evaluating the alternatives carefully [16].
To assess the robustness of the DCE design: (i) clarity and appropriateness of the attributes and levels; and (ii) accuracy of the choice tasks and experimental design were examined. Clarity and appropriateness of attributes and levels, which refers to how well respondents appear to comprehend and relate to the attributes and levels, were measured using tests for content validity and marginal willingness-to-pay (mWTP). Content validity is assured when the direction and magnitude of the calculated preference β coefficients for attribute levels align with a priori expectations. A priori expectations (or hypotheses) for the expected respondent preferences in relation to attribute levels were derived from expert opinion, literature review [19, 36–38], and qualitative research [10] and are detailed in Table 1. Alignment suggests that the attributes and levels capture the salient features of the decision context and are understood by respondents. Marginal WTP is a measure of the maximum amount of money that an individual is willing to give up in choosing one alternative over another. This measure depicts the strength of preference for attributes through monetary units, providing an easy-to-understand comparison, and a ratio of the cost attribute coefficient to the comparison attribute coefficient. In piloting a DCE prototype, mWTP magnitude and direction are compared to the a priori expectations (Table 1), while the actual values are less important.
Accuracy of choice task construction and experimental design is a measure of how well the choice tasks have been constructed using Ngene with optimal D-efficiency. Again, content validity and mWTP tests were used to provide an indication of whether the respondents were engaging with the DCE in a way that seemed reasonable compared with a priori expectations [39, 40]. The β coefficients for each attribute and/or level were estimated using a CML.
Results
Effectiveness of the data collection strategy
The data collection strategy was deemed effective with 73 stroke survivors completing the online survey: 105 people consented, of which 23 were ineligible and nine did not finish the survey. Hence, 89% of eligible respondents completed the survey (Fig. 2). A higher proportion of stroke survivors identifying as female and a lower proportion of Aboriginal and Torres Strait Island Australians responded to the DCE than proportions reflected in the national stroke datasets [41, 42]. Respondent characteristics are presented in Table 3. The median duration for survey completion was 18.6 min (IQR 12.8 min).
Fig. 2.
Survey responses
Table 3.
DCE pilot study – respondent characteristics (n = 73)
| Characteristic | n | % |
|---|---|---|
| Gender (M:F:Other) | 28:45:0 | 38:62:0 |
| Mean age (SD) | 58 +/- 14.3 years | - |
| Age range | 23 – 90 years | - |
| Country of birth | ||
| Australia | 58 | 79.5 |
| England | 4 | 5.5 |
| New Zealand | 2 | 2.7 |
| Philippines | 2 | 2.7 |
| Scotland | 2 | 2.7 |
| South Africa | 1 | 1.4 |
| Tonga | 1 | 1.4 |
| UK | 2 | 2.7 |
| USA | 1 | 1.4 |
| Indigenous status | ||
| Aboriginal Australian | 3 | 4.1 |
| Torres Strait Islander | 2 | 2.7 |
| Aboriginal Australian and Torres Strait Islander | 0 | 0 |
| Neither Aboriginal Australian nor Torres Strait Islander | 68 | 93 |
| Type of strokea | ||
| Ischaemic stroke | 29 | - |
| Haemorrhagic stroke | 13 | - |
| Transient ischaemic attack | 7 | - |
| Can’t remember | 9 | - |
| Did not answer | 20 | - |
| Number of strokes | ||
| 1 | 55 | 75.3 |
| 2 | 11 | 15.1 |
| 3 | 2 | 2.7 |
| More than 3 | 5 | 6.8 |
| Mean number of years since first stroke (+/- SD) | 6.94 +/- 7.9 years | - |
| Number of years since first stroke (range) | 0 – 40 years | - |
| Mean number of years since last stroke (+/- SD) | 6 +/- 7.35 years | - |
| Number of years since last stroke (range) | 0 – 40 years | - |
aTotal is more than n as some participants reported experiencing more than one type of stroke
Effectiveness of the survey instrument design
The survey instrument design was deemed effective. Respondents were found to be attentive to the task and to understand the premise of the DCE. Sixty-nine (94.5%) respondents chose the dominant alternative for the dominant choice task and no respondents provided straight-line responses, suggesting that all respondents considered each choice task with attention (Table 4). Sixty-two (84.9%) respondents answered the duplicate choice task identically, indicating a high level of task attendance (Table 4). Only one respondent failed both the dominant choice task and duplicate choice task.
Table 4.
DCE pilot study—reliability and validity test results
Robustness of the experimental design
The experimental design was found to be robust. The attributes of clinical efficacy, side effects and dosage were deemed to be clear and appropriate, returning negative and statistically significant β coefficients (p < 0.05), aligning with a priori predictions (Table 4). The attribute for risk returned a negative but non-significant β coefficient that aligned with the a priori prediction. The attributes for out-of-pocket cost and treatment duration returned negative, non-significant β coefficients that did not align with a priori predictions (Table 4). With the small sample size of this pilot study, it is difficult to ascertain whether the lack of statistical significance indicates that these attributes were not important enough to respondents to influence their decision-making, were not well understood, or whether the sample is heterogeneous, and some attributes are important to subgroups and not others.
The choice tasks and the experimental design of the DCE were accurately devised with robust results generated. The CLM analysis results are presented in Table 5, and include the preference β coefficients for each attribute level, as well as the odds ratio and mWTP.
Table 5.
Results of conditional logit model analysis of pilot study (n = 73)
| Attributes | β coefficient (preference weight) | OR (95% CI) | P-value | mWTP ($) |
|---|---|---|---|---|
| Clinical efficacya | −0.613832 | 0.54 (0.42 to 0.69) | <0.001 *** | −134.6954 |
| Mostly recovered (level 1) | ||||
| A lot of recovery (level 2) | ||||
| Quite a bit of recovery (level 3) | ||||
| Some recovery (level 4) | ||||
| Out of pocket costa | −0.004557 | 1.00 (0.99 to 1.01) | 0.39 | |
| No cost to you (level 1) | ||||
| $25 (level 2) | ||||
| $50 (level3) | ||||
| $75 (level 4) | ||||
| Side effects | ||||
| No side effects | Reference level (omitted) | |||
| Severe nausea only | −1.398645 | 0.25 (0.09 to 0.64) | 0.004 ** | −306.9098 |
| Mild nausea and mild sexual dysfunction that occurs some of the time | −0.593554 | 0.55 (0.23 to 1.34) | 0.19 | −130.2457 |
| Severe sexual dysfunction that occurs the entire time you take the medication | −0.911099 | 0.40 (0.23 to 0.70) | 0.001 ** | −199.9257 |
| Duration of treatmenta | 0.000349 | 1.00 (0.99 to 1.01) | 0.94 | 0.07657793 |
| Two weeks (level 1) | ||||
| Three months (level 2) | ||||
| One year (level 3) | ||||
| Two years (level 4) | ||||
| Dosage | ||||
| 1 tablet every morning at home | Reference level (omitted) | |||
| 1 tablet three times a day at home | −0.547949 | 0.58 (0.34 to 0.98) | 0.043 * | −120.2384 |
| 1 Injection (subcutaneously, into skin folds on your stomach or thigh) once a week at home, administered by yourself | −0.276474 | 0.76 (0.33 to 1.76) | 0.52 | −60.66779 |
| 1 tablet 30 minutes prior to rehab session at hospital outpatient clinic | −0.793234 | 0.45 (0.27 to 0.77) | 0.004 ** | −174.0621 |
| Riska | −0.071148 | 0.93 (0.71 to 1.23) | 0.61 | −15.61223 |
| 1% fracture risk (level 1) | ||||
| 3% fracture risk (level 2) | ||||
| Concordance | 0.717 (SE = 0.025) | |||
| Likelihood ratio test | 176.7 on 10 df, p < 0.001*** | |||
| Wald test | 125.7 on 10 df, p < 0.001*** | |||
| Score (logrank) test | 159 on 10 df, p < 0.001*** | |||
SE standard error
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
A negative mWTP value indicates the respondents would have to be paid to accept the particular attribute/level. A positive mWTP indicates the respondents would be willing to pay for the particular attribute/level
aOrdinal variables
Discussion
The DCE prototype piloted in this study is the first DCE developed to explore stroke survivors’ preferences for RPDs to improve motor recovery after stroke. This DCE was developed, piloted and found to be robust and feasible for implementation on a larger scale. This study contributes to the literature on DCE development and piloting, most notably in the area of stroke rehabilitation and RPDs. Few DCEs have been conducted to investigate the preferences of stroke survivors regarding any recovery intervention after stroke [19, 36–38], and there appear to be none that investigate their preferences for pharmacological interventions after stroke. The insights generated from full-scale DCE implementation could guide patient-centred trial design, dosing protocols, and rehabilitation integration, improving adherence and accelerating RPD translation into clinical practice.
The data collection strategy employed in this pilot was effective. However, there was an overrepresentation of female-identifying respondents and an underrepresentation of Aboriginal and Torres Strait Island Australians in comparison with the national stroke datasets. These findings will guide future recruitment strategies. To ensure that results reflect the preferences of the majority of Australian stroke survivors for RPDs when it is fully deployed, future recruitment strategies will be aimed at achieving a more reflective sample of the target population. Respectful and culturally appropriate engagement of Aboriginal and Torres Strait Island Australians will be required to increase representation. This may include connection to their communities through respectful engagement with local service providers could facilitate access, and enlisting Aboriginal and Torres Strait Island Australian interviewers to support respondents as they undertake the questionnaire which could be provided in paper form where preferred [43, 44]. Strategies shown to be effective at recruiting males over the age of 50, such as referral through an affiliated health service provider, media coverage and mass mailings, will be incorporated into the full DCE recruitment plan to rectify the gender imbalance [45]. It is important that the full DCE study population is representative of the Australian stroke survivor population, to ensure that generalizability of the findings is possible [13].
The survey instrument was effectively designed, as indicated by the high levels of attentiveness to the task and apparent understanding of the premise of the DCE seen in monotonicity and choice validity results. Quality of the responses is a major contributor to the success of a DCE, and the strength of the results, and is contingent on respondents’ understanding of the task. Within the literature, the concept of understanding often refers to the respondents’ capacity to comprehend the choice task and the terminology contained within it to describe the attributes and level, but also the concept of making a choice [46–48]. Developing an effective survey instrument to deliver the DCE and succinctly communicate the hypothetical nature of the choice scenarios is critical to ensure the reliability and validity of the respondents’ preference results [15].
This pilot DCE was deemed robust based on the combination of clear, appropriate attributes and levels, and an effective experimental design generated using Ngene. When embedded within an effective survey instrument and disseminated using an effective data collection strategy, such as those piloted in this study, it is reasonable to expect that the fully deployed DCE should return valid and reliable results that will provide usable insights stroke survivors’ preferences for RPDs.
Learnings from piloting the DCE prototype
There is an abundance of literature that reinforces the importance of ensuring a DCE is robust, accurate and of high quality so that valid and reliable results can be obtained [15, 16, 32, 33, 49]. Piloting the DCE prototype was a critical step before large-scale implementation. However, the process of piloting a DCE prototype differs significantly from the process of conducting a DCE. Both study types have very different objectives and therefore have different outcome measures. Additional analysis strategies are required to calculate the different outcomes. The well-established format for reporting results of DCE studies is not an easily translatable guide for reporting DCE pilot outcomes if the objectives of a DCE pilot are to be clearly and effectively addressed. This highlights an important gap in the literature for guidance on a standardised approach to conducting and reporting DCE pilot studies. This study demonstrates one possible approach to analysing and reporting DCE pilot studies, where the pilot objectives are to assess the (1) effectiveness of the data collection strategy; (2) effectiveness of the survey instrument design; and (3) robustness of the DCE design.
Strength and limitations
While results of this pilot DCE indicate that the feasibility objectives have been met, the full-scale DCE will provide more precise parameter estimates and confirm the effectiveness of recruitment and data collection methods at scale. Although formal progression criteria were not pre-specified, recruitment and completion rates, response quality, and preliminary parameter estimates collectively suggest the DCE design and data collection methods are feasible for a larger study. Future pilot DCE work would benefit from establishing explicit progression criteria a priori [50]. It should be noted that the calculated β coefficients are preliminary estimates based on the pilot sample; these will be used as informative priors for the experimental design for the full study, but robust interpretation of attribute preferences will require the full target sample size. The systematic approach to the development, piloting and reporting of this DCE prototype is a strength of this study.
Conclusion
In this study, a robust DCE that quantitatively examines stroke survivors’ preferences for RPDs in combination with rehabilitation to improve motor recovery, was developed and piloted. Procedures and reporting requirements differ for DCE pilots to established DCE study guidelines. This study provides an example framework for developing and reporting a robust DCE pilot to ensure reliable and valid results will be generated when implemented on a larger scale to reach the full target sample size. This in turn will have important implications for development of scientifically robust interventions and clinical trial protocols.
Supplementary Information
Additional file 2. Conjoint Analysis Checklist.
Acknowledgements
The authors acknowledge the participants of this study for sharing their knowledge, opinions and experience through their feedback during the pre-testing of the DCE prototype and through responses to the DCE questions during the pilot. We would also like to thank Emeritus Professor Rhondda Jones for her guidance on statistical analysis, and Associate Professor Michelle Redman-MacLaren for her guidance and support for PhD candidate Nerida Firth.
Authors' contributions
NF, RB, KH, JB, and RR were responsible for the study conception/design and the development of the study protocol. NF was responsible for all major study tasks, which included setting up the study, designing the DCE and all other study documentation, applying for ethical approval, collecting the data, analysing/interpreting the results and writing the manuscript. AP provided feedback, expertise and guidance on study design, documentation and the DCE design. OA and AP provided expertise and guidance on analysing the data and interpreting the results. RB provided guidance on major study tasks, particularly drafting the manuscript. All authors were involved in reading, revising and approving the manuscript.
Funding
KH receives support from a NHMRC of Australia Investigator Fellowship (#2016420) and Heart Foundation of Australia Future Leader Fellowship (#106607).
Data availability
The datasets generated during and/or analysed during the current study are not publicly available. However, investigators who would like access to specific parts of the data should contact the corresponding author.
Declarations
Ethics approval and consent to participate
Ethics approval for this study was granted by James Cook University Human Research Ethics Committee (H7620).
Consent for publication
Informed consent was obtained from all individual participants included in the study.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Beghi E, Binder H, Birle C, Bornstein N, Diserens K, Groppa S, et al. European Academy of Neurology and European Federation of Neurorehabilitation Societies guideline on pharmacological support in early motor rehabilitation after acute ischaemic stroke. Eur J Neurol. 2021;28(9):2831–45. [DOI] [PubMed] [Google Scholar]
- 2.Firth N, Barker RN, Hayward KS, Bernhardt J, Bellingan M, Gunnarsson R. Safety and efficacy of recovery-promoting drugs for motor function after stroke: a systematic review of randomized controlled trials. J Rehabil Med. 2019;51(5):319–30. [DOI] [PubMed] [Google Scholar]
- 3.Stinear CM, Lang CE, Zeiler S, Byblow WD. Advances and challenges in stroke rehabilitation. Lancet Neurol. 2020;19(4):348–60. [DOI] [PubMed] [Google Scholar]
- 4.Dennis M, Mead G, Forbes J, Graham C, Hackett M, Hankey GJ, et al. Effects of fluoxetine on functional outcomes after acute stroke (FOCUS): a pragmatic, double-blind, randomised, controlled trial. Lancet. 2019;393(10168):265–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hankey GJ, Hackett ML, Almeida OP, Flicker L, Mead GE, Dennis MS, et al. Safety and efficacy of fluoxetine on functional outcome after acute stroke (AFFINITY): a randomised, double-blind, placebo-controlled trial. Lancet Neurol. 2020;19(8):651–60. [DOI] [PubMed] [Google Scholar]
- 6.Lundström E, Isaksson E, Näsman P, Wester P, Mårtensson B, Norrving B, et al. Safety and efficacy of fluoxetine on functional recovery after acute stroke (EFFECTS): a randomised, double-blind, placebo-controlled trial. Lancet Neurol. 2020;19(8):661–9. [DOI] [PubMed] [Google Scholar]
- 7.Krueger KP, Berger BA, Felkey B. Medication adherence and persistence: a comprehensive review. Adv Ther. 2005;22(4):313–56. [DOI] [PubMed] [Google Scholar]
- 8.Zhang J, Gong Y, Zhao Y, Jiang N, Wang J, Yin X. Post-stroke medication adherence and persistence rates: a meta-analysis of observational studies. J Neurol. 2021;268(6):2090–8. [DOI] [PubMed] [Google Scholar]
- 9.Losi S, Berra CCF, Fornengo R, Pitocco D, Biricolti G, Federici MO. The role of patient preferences in adherence to treatment in chronic disease: a narrative review. Drug Target Insights. 2021;15:13–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Firth N, Hayward KS, Bernhardt J, Ray R, Barker RN. Stroke survivors’ perspectives on decision-making about rehabilitation and the prospect of taking recovery-promoting drugs: a qualitative study. Explor Res Clin Soc Pharm. 2023;11:100297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Laver K, Ratcliffe J, George S, Lester L, Crotty M. Preferences for rehabilitation service delivery: a comparison of the views of patients, occupational therapists and other rehabilitation clinicians using a discrete choice experiment. Aust Occup Ther J. 2013;60(2):93–100. [DOI] [PubMed] [Google Scholar]
- 12.Laver K, Ratcliffe J, George S, Lester L, Walker R, Burgess L, et al. Early rehabilitation management after stroke: what do stroke patients prefer? J Rehabil Med. 2011;43(4):354–8. [DOI] [PubMed] [Google Scholar]
- 13.Bridges JFP, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, et al. Conjoint analysis applications in health—a checklist: a report of the ISPOR good research practices for conjoint analysis task force. Value Health. 2011;14(4):403–13. [DOI] [PubMed] [Google Scholar]
- 14.Soekhai V, de Bekker-Grob EW, Ellis AR, Vass CM. Discrete choice experiments in health economics: past, present and future. Pharmacoeconomics. 2019;37(2):201–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Janssen EM, Marshall DA, Hauber AB, Bridges JFP. Improving the quality of discrete-choice experiments in health: how can we assess validity and reliability? Expert Rev Pharmacoecon Outcomes Res. 2017;17(6):531–42. [DOI] [PubMed] [Google Scholar]
- 16.Johnson FR, Yang J-C, Reed SD. The internal validity of discrete choice experiment data: a testing tool for quantitative assessments. Value Health. 2019;22(2):157–60. [DOI] [PubMed] [Google Scholar]
- 17.Lancsar E, Louviere J. Deleting ‘irrational’ responses from discrete choice experiments: a case of investigating or imposing preferences? Health Econ. 2006;15(8):797–811. [DOI] [PubMed] [Google Scholar]
- 18.Reed Johnson F, Lancsar E, Marshall D, Kilambi V, Mühlbacher A, Regier DA, et al. Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force. Value Health. 2013;16(1):3–13. [DOI] [PubMed] [Google Scholar]
- 19.Geidl W, Knocke K, Schupp W, Pfeifer K. Measuring stroke patients’ exercise preferences using a discrete choice experiment. Neurol Int. 2018;10(1):6993. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Lundstrom EMDP, Isaksson ERNP, Greilert Norin NRN, Nasman PP, Wester PMDP, Martensson BMDP, et al. Effects of fluoxetine on outcomes at 12 months after acute stroke: results from EFFECTS, a randomized controlled trial. Stroke. 2021;52(10):3082–7. [DOI] [PubMed] [Google Scholar]
- 21.Eli Lilly Australia Pty. Limited. PROZAC® Fluoxetine Hydrochloride [Product Information] 2016. Available from: https://www.tga.gov.au/resources/artg/14653.
- 22.Lundbeck Australia Pty Ltd. Cipramil® film coated tablets [Product information] 2013. Available from: https://www.tga.gov.au/resources/artg/61164.
- 23.Department of Health and Aged Care (Australian Government). Pharmaceutical benefits - fees, patient contributions and safety net thresholds Canberra, ACT Australia: Commonwealth of Australia. 2023. Available from: https://www.pbs.gov.au/info/healthpro/explanatory-notes/front/fee.
- 24.ChoiceMetrics. Ngene 1.4 user manual & reference guide. Australia: ChoiceMetrics; 2024. [Google Scholar]
- 25.Lancsar E, Louviere J. Conducting discrete choice experiments to inform healthcare decision making. Pharmacoeconomics. 2008;26(8):661–77. [DOI] [PubMed] [Google Scholar]
- 26.Mühlbacher A, Johnson FR. Choice experiments to quantify preferences for health and healthcare: state of the practice. Appl Health Econ Health Policy. 2016;14(3):253–66. [DOI] [PubMed] [Google Scholar]
- 27.Determann D, Lambooij MS, Steyerberg EW, de Bekker-Grob EW, de Wit GA. Impact of survey administration mode on the results of a health-related discrete choice experiment: online and paper comparison. Value Health. 2017;20(7):953–60. [DOI] [PubMed] [Google Scholar]
- 28.Qualtrics. Qualtrics. Provo, Utah, USA: Qualtrics. 2020.
- 29.Eccles DW, Arsal G. The think aloud method: what is it and how do i use it? Qual Res Sport Exerc Health. 2017;9(4):514–31. [Google Scholar]
- 30.Wolcott MD, Lobczowski NG. Using cognitive interviews and think-aloud protocols to understand thought processes. Curr Pharm Teach Learn. 2021;13(2):181–8. [DOI] [PubMed] [Google Scholar]
- 31.Pérez-Troncoso D. A step-by-step guide to design, implement, and analyze a discrete choice experiment. arXiv:200911235 [econEM]. 2020.
- 32.Hauber AB, González JM, Groothuis-Oudshoorn CGM, Prior T, Marshall DA, Cunningham C, et al. Statistical methods for the analysis of discrete choice experiments: a report of the ISPOR conjoint analysis good research practices task force. Value Health. 2016;19(4):300–15. [DOI] [PubMed] [Google Scholar]
- 33.Tervonen T, Schmidt-Ott T, Marsh K, Bridges JFP, Quaife M, Janssen E. Assessing rationality in discrete choice experiments in health: an investigation into the use of dominance tests. Value Health. 2018;21(10):1192–7. [DOI] [PubMed] [Google Scholar]
- 34.Orme BK. Chapter 7. Sample size issues for conjoint analysis. Getting started with conjoint analysis: strategies for product design and pricing research. 4th ed. Madison, WI: Research Publishers. 2019.
- 35.Ying X, Freedland KE, Powell LH, Stuart EA, Ehrhardt S, Mayo-Wilson E. Determining sample size for pilot trials: a tutorial. BMJ. 2025;390:e083405. [DOI] [PubMed] [Google Scholar]
- 36.Burton CR, Fargher E, Plumpton C, Roberts GW, Owen H, Roberts E. Investigating preferences for support with life after stroke: a discrete choice experiment. BMC Health Serv Res. 2014;14(1):63. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Noto S, Murata T, Saito S, Watanabe T, Kobayashi M. Preferences for rehabilitation in persons with a history of stroke: a discrete choice experiment. Patient Prefer Adherence. 2023;17:1611–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Doshi K, Henderson SL, Seah DJL, De Silva DA, Lee JJ, Huynh VA, et al. Stroke survivors’ preferences for post-stroke self-management programs: a discrete choice experiment. J Stroke Cerebrovasc Dis. 2023;32(3):106993. [DOI] [PubMed]
- 39.de Bekker-Grob EW, Ryan M, Gerard K. Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012;21(2):145–72. [DOI] [PubMed] [Google Scholar]
- 40.Campoamor NB, Guerrini CJ, Brooks WB, Bridges JFP, Crossnohere NL. Pretesting discrete-choice experiments: a guide for researchers. Patient. 2024;17:109–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Australian Institute of Health Welfare. Heart, stroke and vascular disease: Australian facts. Canberra: AIHW; 2023. [Google Scholar]
- 42.Australian Bureau of Statistics. 2018 survey of disability, ageing and carers, customised data report. 2023.
- 43.Lee KSK, Fitts MS, Conigrave JH, Zheng C, Perry J, Wilson S, et al. Recruiting a representative sample of urban South Australian Aboriginal adults for a survey on alcohol consumption. BMC Med Res Methodol. 2020;20(1):183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Marin T, Copley IT, Taylor A, editors. Culturally appropriate engagement strategies for collecting quality data in an Aboriginal population. 12th National Rural Health Conference. 2013.
- 45.Bracken K, Askie L, Keech AC, Hague W, Wittert G. Recruitment strategies in randomised controlled trials of men aged 50 years and older: a systematic review. BMJ Open. 2019;9(4):e025580. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Pearce A, Harrison M, Watson V, Street DJ, Howard K, Bansback N, et al. Respondent understanding in discrete choice experiments: a scoping review. Patient. 2021;14(1):17–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Ryan M, Watson V, Entwistle V. Rationalising the ‘irrational’: a think aloud study of discrete choice experiment responses. Health Econ. 2009;18(3):321–36. [DOI] [PubMed] [Google Scholar]
- 48.Ali S, Ronaldson S. Ordinal preference elicitation methods in health economics and health services research: using discrete choice experiments and ranking methods. Br Med Bull. 2012;103(1):21–44. [DOI] [PubMed] [Google Scholar]
- 49.Mühlbacher AC, de Bekker-Grob EW, Rivero-Arias O, Levitan B, Vass C. How to present a decision object in health preference research: attributes and levels, the decision model, and the descriptive framework. Patient. 2024;18:291–302. [DOI] [PMC free article] [PubMed]
- 50.Mbuagbaw L, Chen LH, Aluko E, Stevens-Uninsky M, Eze-Onuorah ACJ, Garcia MC, et al. Empirical progression criteria thresholds for feasibility outcomes in HIV clinical trials: a methodological study. Pilot Feasibility Stud. 2023;9(1):96. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 2. Conjoint Analysis Checklist.
Data Availability Statement
The datasets generated during and/or analysed during the current study are not publicly available. However, investigators who would like access to specific parts of the data should contact the corresponding author.