Preferences for rapid implementation science: a real-time discrete choice experiment

Abstract

Background

A persistent challenge in implementation science is the gap between the pace and priorities of academic research and the needs of decision-makers. While traditional research emphasizes methodological rigor, policymakers often require timely, relevant, and stakeholder-engaged evidence to inform decisions.

Methods

We conducted a real-time discrete choice experiment (DCE) with participants at an implementation science conference in Washington, D.C. during a 75-minute session on rapid implementation science. At the midpoint of the panel, the audience were invited to participate in a 10-minute DCE based on a vignette describing a hypothetical study on implementing a new vaccine facing public hesitancy. Choice sets included six attributes: rapidity of study, study design, primary outcome, level of community engagement, leadership, and costs. We used a mixed logit model to analyze preferences in real time and presented results back to the audience at the end of the session.

Results

Ninety-four participants completed the DCE; all reported working in research and 83% reported working in program implementation. Respondents placed high value on timing of results, with strong preferences for receiving results in 6 months compared 12 months (β=–1.1, 95% CI: − 1.6- − 0.6; p < 0.001) and 18 months (β=–1.7,–2.3- − 1.0; p < 0.001). Participants preferred the primary outcome to be vaccine uptake rather than vaccine acceptability (β = 1.7, 1.1–2.2; p < 0.001) and the study to be community-engaged rather than expert-led (β = 1.6, 1.0-2.2; p < 0.001). We found no preference for randomization compared to before and after (p = 0.88), nor leadership by the ministry of health versus academic institutions (p = 0.12). Respondents were willing to wait 11.3 (95% CI 7.0-15.5) additional months for a program developed with engaged stakeholders compared to expert-driven (p < 0.001) and 11.9 months (95% CI 8.1–15.8) for results on vaccine uptake rather than acceptability (p < 0.001).

Conclusions

This real-time DCE demonstrated that with adequate preparation, it is feasible-to generate robust, interpretable, and actionable results within an hour. Findings challenge the assumption that randomization is the highest priority in research design, highlighting instead the importance of outcome and community engagement. By aligning more closely with the values of decision-makers, rapid science approaches such as real-time DCEs may help bridge the research-policy gap.

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s43058-026-00986-z.

Text box 1. Contributions to the literature
• Research timelines often fail to match the urgent needs of policymakers. This study demonstrates a practical approach for producing timely evidence that is effective for decision-making and expands the methodological tools for rapid implementation science.
• We conducted a rapid, novel application of a real-time discrete choice experiment (DCE) that captured stakeholder preferences on rapid research during a live conference session.
• These findings highlight that stakeholders valued quick results, community engagement, and outcomes of uptake, more than traditional experimental rigor such as randomization. Community engagement and measurement of uptake were worth nearly a year of additional research time.

Background

There has been growing recognition that traditional research timelines are misaligned with the urgent and often evolving needs of policymakers and practitioners [1–3]. Research findings, even when methodologically sound, often arrive too late to inform critical decisions because they are out of sync with practice and policy-making cycles [4–7]. Decision-makers often need to act within weeks or months, yet traditional research approaches can require years to produce results. As a result, policymakers may sometimes need to decisions based on information that is out-of-date, undermining the credibility of evidence-based policies [8]. On the other hand, research must follow certain scientific principles with transparency to claim a scientific basis — also an important principle in a time when health related information is increasingly unscientific or politically driven [9, 10].

Understanding how to effectively develop rapid research that meets both the timelines of policy and practice as well as the standards of science requires information on the preferences of the community who commission, conduct, and use research. For instance, while academic researchers might prioritize methodological rigor measured through traditional markers like randomization, policymakers and practitioners might place greater weight on factors such as stakeholder engagement, practical relevance, or speed of delivery [6]. While existing qualitative research has noted that policy makers operate on a policy-cycle that often occurs rapidly, and also that research occurs at a suboptimal speed, what is lacking in the literature is a quantification of what matters most, how these imperatives are weighted against each other, and how that differs within and across stakeholder groups, The COVID-19 pandemic provides a clear illustration of this temporal mismatch where policymakers were required to introduce and repeatedly revise interventions (e.g., vaccination strategies, non-pharmaceutical interventions) on accelerated timelines, often before rigorous evidence or quantified stakeholder preferences were available. As a result, research frequently lagged behind decision-making needs, contributing to rapid policy iteration, inconsistent implementation, and communication challenges, such as those observed during vaccine rollout amid shifting eligibility and dosing guidance, highlighting a mismatch between research outputs and policy needs [11–13].

This study examines preferences for characteristics of rapid implementation research, defined as a systematic and agile approach that “provides the best possible evidence-based practice of a program or intervention to those who need it, with speed and efficiency, by redefining rigor, and adapting both methods and trial design, to fit research aims and objectives”, [14] using a real-time discrete choice experiment (DCE) [15–17]. We sought to understand how an implementation science audience values different aspects of an implementation research study (including the rapidity, design, outcomes, and leadership). We also used this opportunity to also analyzes how people weigh trade-offs between these attributes, different subgroups and identify characteristics associated subgroup membership. We conducted this experiment during a conference session focused on rapid research methods. In doing so we both aimed to both generate insights about research needs, challenges, and preferences as well as demonstrate an instance of rapid science in action.

Methods

Study design and setting

DCEs are a stated-preference method grounded in random utility theory which assumes that behavior is driven by internal units of happiness or satisfaction (i.e., utilities), and that individual behavior is based on utility maximization: making choices that obtain the most utilities under constraints [18–20]. By presenting participants with sets of hypothetical scenarios composed of varying combinations of attributes and levels, DCEs allow researchers to estimate the relative importance of each attribute, and to understand the trade-offs individuals are willing to make between their preferences for each attribute level. We adhered to the DIRECT reporting checklist for DCEs in health [21].

We conducted the DCE during a 75-minute-long session on “Rapid Research Methods in Implementation Science: Lessons from the Global Context” at Academy Health’s 17th Annual Conference on the Science of Dissemination and Implementation in Arlington, VA, USA on December 10th, 2024. The session was described in the conference program in this way: “This session will be presented in a podium format and categorized as a discussion forum, encouraging interactive dialogue around the timely application of rapid research methodologies in global implementation science. This session aims to foster a discussion on the potential of rapid research methodologies to significantly impact policymaking and program development in a timely manner, addressing the critical needs of fast-evolving health challenges in global health. Each presentation aims to highlight both methods and governance arrangements for rapid research, emphasizing the importance of aligning research questions with the operational planning cycles of country programs. The session will explore the integration of both qualitative and quantitative insights to enhance program outcomes efficiently.” The session (Table 1) included both talks as well as space for collection of data and sharing the results with the audience in real time.

Table 1.

Schedule for session

Segment	Presenter(s)	Topic	Time
Introduction	Susan Vorkoper (Fogarty International Center, NIH)	Overview of the session and need for rapid research; unique opportunities in global settings	5 min
Presentation	Alex Ramsey (Washington University in St. Louis)	FAST: A Framework to Assess Speed of Translation of Health Innovations to Practice and Policy	5–7 min
Presentation	Noelle Le Tourneau (Washington University in St. Louis)	DCE Administration: Engaging the audience (5 min); 10 min to complete (via QR code)	15 min
Presentation	Andrew Kanyike (University of Exeter)	Findings from a Survey of Rapid Methods; recent results on rapid research methodologies in practice	5–7 min
Presentation	Ejemai Eboreime (Dalhousie University)	Leveraging Existing Programmatic Structures‚ strategies and barriers in using qualitative data rapidly	5–7 min
Panel Discussion & Q&A	Noelle Le Tourneau (DCE Results); Moderator: Rohit Ramaswamy (University of North Carolina at Chapel Hill)	Framing questions, presentation of DCE results, and open audience discussion on rapid research approaches	35 min

Choice experiment and measures

The DCE survey was designed prior to the session by the core organizing team of the session (the authors on this paper). We reviewed literature and discussed the relevant attributes and levels in three hour-long meetings as researchers, funders and people who work closely with government and implementation agencies. Prior to deployment of the DCE, we conducted a small round of review and refinement with approximately seven individuals closely involved in organizing the session. These individuals completed the survey and provided feedback on clarity, flow, and feasibility, which informed minor revisions. While we did not conduct formal cognitive interviewing, our aim was to ensure basic usability and coherence while preserving the ability to implement the survey in a live, discussion-based setting. The resulting choice experiment itself involved a stem (Box 1) that asked respondents to consider a hypothetical scenario in which they are a director of health for a country of 20 million people, and tasked them with considering a research protocol for implementation of a new highly efficacious vaccine with low actual uptake. Considering a range of barriers, participants are asked to fund a piece of implementation research that will help inform their decision as director of health, considering that this needs to be addressed quickly and done well (Box 1).

Box 1.

Prompt for discrete choice experiment on rapid implementation research

• You are the director of health for a country of 20 million people.

• A new highly efficacious vaccine became available but has low uptake (only 10% of eligible people have accessed it) since it was made publicly available (and free) one year ago.

• The condition it prevents is a major public health problem in the country, ranking #3 in adult morbidity and mortality.

• An assessment found a range of barriers. On the supply side, only large hospitals received the vaccine and were knowledgeable about the indications. On the demand side, some rumors about side effects limited public enthusiasm.

• Of note, you want to do this well, but you also need to do if fast. The President ran on a platform of expanding health and is eager to see tangible results from her Health Ministry.

• After discussion with an academic partner, NGO’s and the government unit, you decide to fund a piece of implementation research that will inform your decision.

• The study develops an implementation strategy and test the formulated strategy.

• You have resources to implement the strategies immediately once the study is done.

• Which one of the following proposals for a study are you more likely to choose? Assume the methods for each are sound.

Based on the literature and discussion, the organizing team developed a set of attributes that reflected realistic choices in research design, including stakeholder engagement, study design, outcome measures, timeliness, study leadership, and cost [22, 23]. The DCE was structured around six key attributes (Fig. 1), each designed to reflect realistic trade-offs that decision-makers might face when commissioning or conducting rapid implementation research, specifically in the context of evaluating strategies to improve vaccine uptake: how the strategy was developed (community-engaged vs. more expert led and centralized); study design (randomized design vs. before-and-after rollout design); main study outcome (actual vaccine uptake versus only acceptability through a survey); timing of results (delivering findings in 6, 12, or 18 months); leadership of the study (academic partners vs. ministry led) and costs, which were expressed as a percentage of an imaginary program research and development (R&D) budget (5%, 10%, or 15%).

Fig. 1.

DCE attribute and levels. Participants were asked to complete a series of eight comparisons or choice tasks, selecting the option they preferred in each set of two scenarios (Study Proposal A or Study Proposal B) with varying levels across six attributes above

Each participant was asked to complete a series of eight comparisons or choice tasks, selecting the option they preferred in each set of two scenarios (Study Proposal A or Study Proposal B) with varying levels across six attributes. Each choice task is designed to present elements that reveal relative preferences and their trade-offs (e.g., lower rigor but faster results vs. higher rigor with delayed impact) and together, these attributes and levels allowed us to understand how participants prioritized different elements of rapid research design when faced with realistic constraints. Images were created and displayed alongside each level to increase clarity and ease of responding to each choice task (Fig. 1).

After a brief 5-minute introduction to DCE methods and sharing the prompt for the DCE using a PowerPoint presentation during a break between speakers, a QR code with a direct link to the survey was presented on the podium screen for participants to scan on their cellphones or mobile devices. Participants were given 10 min to complete the survey. All attendees were eligible to participate and participation in the survey was optional and anonymous, with no names or identifying information collected. The survey is available: https://RapidScienceDCEPublic.sawtoothsoftware.com.

In addition to the DCE, we asked participants non-identifying characteristics such as how many years of experience they had in program implementation, in research, and global regions that they work (Appendix 1). After DCE results were shared at the end of the session, participants had about 25 min to reflect on findings and other panel discussion. Notes on the DCE reflections by participants were taken by one author and discussed with co-authors after the session to confirm general audience views and consensus.

We used rule of thumb sample size calculations proposed by Johnson and Orme [24–26] where it is suggested that the sample size required for the main effects depends on the number of choice tasks (t =8), the number of alternatives (a=2), and the number of analysis cells as the largest number of levels for any of the attributes (c =3) according to the following equation: N>500c/(t × a) where for our DCE N >500(3)/(8 × 2) = 93.75.

We used Sawtooth Software version 9.15.6 (Lighthouse Studio Sawtooth Software, Provo, Utah, USA) [27] to design and generate 300 versions of the DCE, randomly allocated across participants to ensure that participants received different random combinations of attribute levels in each question. We used an orthogonal main effects design to construct a balanced and statistically efficient set of choice tasks [28]. Prior to administering the survey, we tested the survey design in Sawtooth with the estimated sample size of 94 to ensure standard errors were below 0.05. The strength of design for our model, reflecting the D-Efficiency of one design relative to the other, is 376.48. Due to the time constraint, we minimized the number of choice tasks and did not include fold-over pairs to test for dominance.

Data analysis

We pre-programmed the analysis in Sawtooth so that incoming responses could be automatically analyzed and generated graphs using pre-designed Excel templates during the conference session (Appendix 2). After the rapid session, the data were further analyzed in Stata 16.1 using a multinomial logit model to estimate the influence of each attribute level on the probability of choice, allowing us to quantify preference weights and infer the relative value placed on different dimensions of rapid research. We used mixed logit models to generate mean utilities (mean preferences) and 95% confidence intervals (CIs) of the random coefficients, using 500 Halton draws, dummy coded variables as random parameters for the estimations, and assuming normal distribution [18, 28]. In the real-time analysis using Sawtooth, as standard practice, 20,000 iterations were run for convergence [29]. Mixed logit coefficients can be interpreted as the strength of the relative preference for the particular attribute comparison, with positive coefficients representing positive preferences (more desirable) and negative coefficients representing negative preferences (less desirable). We performed a willingness to wait analysis to explore tradeoffs participants are willing to make after establishing linearity of attribute levels for wait time value using Lowess plots [30]. To better understand preference heterogeneity, we conducted exploratory analysis across subgroups of research and implementation experience, however due to small group sizes, our subgroup analysis were hypothesis generating only and consisted of only two groups for analysis for model stability. We planned latent class analysis if power allowed. For the analyses, we used surveys where participants completed the DCE.

We ultimately re-ran our utility estimates using Stata 16.1 [31] because we preferred to report utilities using a reference group for each attribute, rather than the zero-sum approach that Sawtooth Software uses by default. In the zero-sum approach, utility estimates within each attribute are constrained to sum to zero; in the reference-category approach, one level is designated as the reference and all other levels are interpreted relative to it. Importantly, the interpretations of the estimates did not differ between the Sawtooth and Stata output. The choice between parameterizations is primarily one of presentation convention: the reference-category approach is more familiar to clinical and health services research audiences, while the zero-sum approach is native to Sawtooth and standard in marketing research.

Results

In a room with an estimated 150 attendees, a total of 110 participants started the survey, and 94 completed. Respondents represented a diverse range of experience levels with research and program implementation (Table 2). While all participants reported working in research for at least some period of time, only 83.0% had experience working in program implementation. Participants reported working across multiple global regions, with most working in North America (70.9%) and Africa (43.4%). Other regions included Central and South America (20.0%), Asia (20.0%), Europe (7.3%), and smaller proportions from the Eastern Mediterranean and Western Pacific (each at 3.6%).

Table 2.

Participant characteristics

	Completed DCE (n = 94)
	N (%)
Years in program implementation
1–2 years	22 (23.4%)
3–5 years	16 (17.0%)
6–10 years	21 (22.3%)
11–14 years	11 (11.7%)
15 + years	8 (8.5%)
I do not work in program implementation	16 (17.0%)
Years in research
1–2 years	3 (3.2%)
3–5 years	20 (21.3%)
6–10 years	29 (30.9%)
11–14 years	17 (18.1%)
15 + years	25 (26.6%)
I do not work in research	0 (0.0%)
Regions of work*
Africa	41 (43.6%)
North America	65 (69.1%)
Central and South America	17 (18.1%)
Asia	19 (20.2%)
Europe	8 (8.5%)
Eastern Mediterranean	4 (4.3%)
Western Pacific	3 (3.2%)

*Regions of work are not mutually exclusive

Main preferences

The DCE found notable differences in the relative importance participants assigned to various attributes of rapid research. The most highly prioritized attribute was the timing of results, which accounted for 25.3% of the overall importance weight. This was closely followed by outcome type (23.9%) and developing the strategy (22.9%). In contrast, study design for testing the strategy accounted for only 7.5% and lead of the study received only 8.2% of total weight. Cost of the study contributed 12.3% to participants’ decision-making (Fig. 2).

Fig. 2.

Importance of attributes (n = 94)

The DCE demonstrated (Fig. 3) that while there were negative preferences for higher study costs, these were not significant as the research and development (R&D) budget approached statistical significance (β = − 0.4, 95% CI: − 0.9 to 0.0, p = 0.05) only for 15% versus 5% of the R&D budget comparison with no significant between 10% and 5% (β = − 0.2, 95% CI: − 0.6 to 0.2, p = 0.29) indicating that moderate increases in the R&D budget were not important. Respondents showed strong positive preferences for community engagement over expert-led strategy development (β = 1.6, 95% CI: 1.0 to 2.2, p < 0.001), and for uptake as the primary outcome compared to acceptability (β = 1.7, 95% CI: 1.1 to 2.2, p < 0.001). No significant differences in preferences were observed between studies led by the ministry of health compared to academic institutions (β = − 0.3, 95% CI: − 0.6 to 0.1, p = 0.12) or between randomized versus rollout testing strategies (β = 0.03, 95% CI: − 0.3 to 0.4, p = 0.88). Timing of results strongly influenced preferences: both 12 months (β = − 1.1, 95% CI: − 1.6 to − 0.6, p < 0.001) and 18 months (β = − 1.7, 95% CI: − 2.3 to − 1.0, p < 0.001) compared to 6 months.

Fig. 3.

Relative preferences for rapid research (n = 94). The relative preference value represents the strength of preference relative to the baseline attribute level. Mixed logit model. CI, confidence interval

Subgroup analysis

Due to small sample, subgroup analysis to better understand preference heterogeneity across implementation and research experience were exploratory (Supplementary Figs. 1 & 2). There were generally consistent preference patterns across those with 1–10 years of experience compared those with > 10 years of experience, with some differences in magnitude of preferences, though none of these differences were statistically significant (Supplementary Fig. 1). Respondents with > 10 years of research experience had stronger preferences for community engaged over expert-led strategies (β = 3.4, 95% CI: 1.3 to 5.4, p < 0.001) compared to those with 1–10 years of research experience (β = 1.5, 95% CI: 0.9 to 2.2, p < 0.001), though again these differences were not significant. Both 12-month and 18-month timelines (vs. 6 months) were associated with negative utilities for both groups, with substantially larger negative preferences among respondents with > 10 years of experience (β = -2.6, 95% CI: -4.4 to -0.9, p < 0.001 and β = -3.8, 95% CI:-6.0 to -1.7, p < 0.001 for 12 and 18 months vs. 6 months, respectively). Preferences did not differ across research experience meaningfully for studies led by the ministry versus academic institutions, study cost of 15 or 10% versus 5% of the R&D budget, nor for randomized versus rollout testing strategies.

In more exploratory analyses, implementers with more than 10 years of experience (only n = 19) had very strong preferences for community engaged versus expert led strategy development (β = 39.0, 95% CI: -18.9 to 97.0, p = 0.187) and measuring the outcome of uptake versus acceptability (β = 17.9, 95% CI: -6.7 to 42.4, p = 0.154) (Supplementary Fig. 2). Implementers with more than 10 years of experience also had potentially stronger negative preferences for both 12-month and 18-month timelines (vs. 6 months) (β = -17.2, 95% CI: -42.6 to 8.2, p = 0.184 and β = -39,5, 95% CI:-97.5 to 18.4, p = 0.181 for 12 and 18 months vs. 6 months, respectively), compared to those with 1–10 years of experience. There were also negative preferences for randomization compared to rollout (β = -6.5, 95% CI: -16.9 to 3.8, p = 0.216), and cost (10% and 15% vs. 5%: β = -7.8, 95% CI: -19.3 to 3.8, p = 0.216 and β = -11.6, 95% CI: -31.1 to 8.0, p = 0.246, respectively) among implementers with more experience. However, due to the small subgroup size, confidence intervals were very wide and none of these differences were statistically significant.

Due to the multi-select option for regions of work, we did not compare preferences across region subgroups as our sample size was too small.

Willingness to wait

Respondents were willing to wait on average 11.3 additional months (95% CI 7.0 to 15.5) for a program developed with engaged stakeholders compared to expert-driven (p < 0.001) (Fig. 4). Respondents were also willing to wait almost a year (11.9 additional months) for results (95% CI 8.1 to 15.8) in order to measure the outcome of uptake compared to acceptability (p < 0.001). Conversely, respondents would wait less than 1 week extra for randomized testing over rollout (essentially no real willingness-to-wait (0.2, 95% CI -2.9 to 2.4) (p = 0.869)). Respondents only accepted ministry leadership if it shortened timing to results by about 2.2 months (95% CI 0.0 to 4.4) compared to academic leadership (p = 0.046). An increase in the R&D budget by 10% (from 5 to 15%) was valued at delaying the program by about 3.3 extra months (95% -0.1 to 6.7) (p = 0.057). Respondents were willing to pay a significant proportion of the R&D budget to shorten the study.

Fig. 4.

Relative trade off in months for rapid research (n = 94). Positive months represent the additional time in months worth trading for the feature. Mixed logit model. CI, confidence interval

Participant feedback

During the discussion of the DCE results at the end of the session, audience members generally agreed that the results were accurate and reflected their preferences for rapid research, but some expressed surprise that there was a strong preference for measuring uptake over acceptability. Many audience members were interested and noted the ability to generate and review results in real time enhanced engagement and made them excited about this rapid approach. Several members noted the potential to employ real-time DCEs in other settings, specific to their research, where rapid, participatory approaches could strengthen the relevance and uptake of findings for stakeholders.

Discussion

In a sample of individuals interested in a global implementation science and research rapidity, we were able to rapidly quantify preferences for a number of aspects of design. One of the most striking findings from this exercise is that respondents placed essentially no value on randomization. This result challenges a deeply held tenet in much of clinical and traditional research—that randomization is the gold standard and the only credible way to make strong inferences [32]. This indifference may reflect implementation science’s focus on real-world effectiveness over efficacy, where before-and-after rollout designs better capture scale-up dynamics. Alternatively, it could indicate our sample’s sophistication in understanding internal validity tradeoffs—a hypothesis supported by the 83% implementation experience rate (Table 2). If these findings are generalizable, perhaps the intended users of evidence care little about randomization. Preferences do not obviate the potential value of randomization but do provide data about where randomization might fit with regard to emphasize on timeliness, relevance, and stakeholder engagement [33, 34]. While there was some indication of disutility of higher costs, overall there was cost-insensitivity within the 5–15% range of the R&D budget, which may have been due to framing cost around R&D or the hypothetical nature of cost. These results suggest that while methodological purity is prized in academic science, decision-makers and practitioners may prioritize other attributes when deciding what research is valuable.

The strong preference for vaccine uptake rather than mere acceptability is noteworthy. Although acceptability is widely used in the field and considered an important implementation outcome, it is also true that acceptability does not always predict behavior [23, 35].Acceptability of a new health care intervention does not always predict use, as well as acceptability by end users also does not predict use [36–38]. The discomfort with acceptability in this audience might reflect a wider skepticism about the value of acceptability itself.

This audience strongly favored research that was community engaged. Much implementation research is highly invested in community engagement, reflected in a strong preference for community engaged strategies by researchers and implementers with more than 10 years of experience [39]. The premise is that end users can and should be involved in design of programs and practices that affect them has become widely accepted and championed [40–43]. Community engagement has involved from merely consultation to co-creation and now involved many methods such as group model building or crowdsourcing [44–46]. These data suggest that community engagement is among the elements of research design that this audience most prizes. In some ways, this study suggests that doing it right is worth waiting for and doing it right means community engagement.

These findings acknowledge the importance of both the source and quality of evidence to drive sustained change, even if a more deliberate research timeline is required. This may also add insight to broader efforts aiming to “reveal the optimal pace for implementation for different innovations, with different stakeholder groups, at different implementation phases, and under different resource contexts” [47]. Though analyses were exploratory, researchers and implementers with more experience may have a stronger aversion to longer research timelines, indicating possible preference heterogeneity in stakeholder groups. Follow-up preference-based utility studies may delve further into this tension to better understand the relative utility of implementation outcomes in a more nuanced context, for instance by comparing research scenarios involving 1) no implementation outcomes, 2) implementation outcomes as secondary to health-related outcomes, 3) health-related outcomes as secondary to implementation outcomes, and 4) only implementation outcomes.

The findings from this study speak directly to the perceived tension between strict research methodology and timely insights that surround rapid research. Rather than seeing researchers as dogmatically adhering to the gold standard of evidence generation on one side of the divide and the users evidence waiting for data to inform real-time decision-making on the other, we found that there is sincere recognition in the research community that methods should be adapted to practical urgency, particularly when immediate evidence is needed to guide policy or practice. This raises the question: what else are we willing to compromise in the name of speed? As the field continues to weigh the trade-offs inherent in rapid research, we must consider what are the compromises (e.g., generalizability, long-term validity, or sustainability) we accept when acting quickly.

This study had several limitations and strengths. Convenience sampling from an audience at a scientific conference conducted in the United States is inherently limited to those who attend conferences and have an interest in rapid methods in a global context, which does not represent all people working in public health and healthcare. DCEs rely on the sampling methodology which effects how broadly we can generalize our conclusions, and we note that this DCE was designed as a practice to inform decisions in a defined policy and implementation context for stakeholders, rather than the general public. We did have a small sample size, however, we met our target sample size from initial power calculations [25, 26] and we did have a large response rate for participants with both self-reported research and program implementation backgrounds in all global regions, reflecting the stakeholders that ultimately implement evidence-based interventions. However, we note that this sampling approach at a conference may have excluded certain perspectives, particularly those of community members and frontline staff, which could shift the estimated preference weights if included. This study did focus on global health and a fictional country, so these findings may not translate to rapid research preferences within programs in the United States. We also note that the group setting of this exercise may have produced a social desirability bias toward selecting ‘community engagement’ as the method testing the strategy, though we tried to reduce this potential bias by completely anonymizing survey responses and using the DCE method which requires respondents to consider all components of the strategy. The hypothetical nature of a DCE may differ from real-world choices made based on lived experience, for example, the DCE may overstate patience for delays compared to real-world decisions. Again, this type of exercise may resonate with some stakeholders (e.g., policymakers and analyst) but feel less relevant to others (e.g., community groups). We also note that DCEs may be most appropriate when decisions require explicit trade-offs among clearly defined attributes and can be strengthened, as in our case, by integrating qualitative work (e.g., during attribute development or for interpretation of findings), thereby bridging experimental structure with lived experience. We encourage researchers to align sampling and use of a DCE with their audience of interest. Additionally the rapid nature of the survey did not allow for a test-retest reliability assessments [48] and the 10-minute survey time constraint may have caused attribute non-attendance (i.e., respondents ignore an attribute completely to simplify the decision-making process) [49–51]. From a procedural standpoint, we demonstrated how certain methodologies, particularly DCEs, can be effectively used to generate rapid, meaningful insights. In short, we demonstrated an instance of rapid science. When background work is done in advance and data collection tools are deployed in settings where participants are co-located and motivated, such as professional meetings or training sessions, it is possible to obtain high-quality data quickly [52, 53]. In our case, conducting the choice experiment in a live session led to immediate engagement, with 94 participants completing the survey in about 10 min and results available and discussed in under an hour. The results were statistically robust and interpretable, and they spurred active interest and discussion among attendees. DCEs have broad applicability in implementation science as tools for prospective, decision-oriented policy support, by quantification of stakeholder preferences and trade-offs [54, 55]. Beyond our case use, rapid DCEs can be applied to prioritize implementation strategies (e.g., feasibility, cost, and effectiveness), inform adaptations of interventions across contexts by capturing preference heterogeneity, optimize service delivery models (e.g., access, location, provider type), and support policy design [56–58]. We have previously used similar rapid approaches, such as sending SMS-based surveys during in-person training sessions to conduct real-time social network analysis again achieving high response rates and immediate analytic output [59]. These experiences suggest that leveraging convening opportunities, such as conferences or workshops (in-person or remote), alongside structured scientific methods like surveys and choice experiments can be a powerful strategy for gathering rapid, rigorous data on time-sensitive issues [60].

Conclusions

We highlighted the use of real-time DCEs as not only a helpful tool for conducting participatory rapid research itself, but also as a method to better understand the priorities and preferences among researchers and program implementers when conducting rapid research. Our findings suggest that researchers place little value on randomization compared to community-engaged research and prefer outcomes that are meaningful and timely. Implementation science may benefit from broadening its methodological portfolio to emphasize timely, rigorous, and relevant rapid methods, such as real-time DCEs, for bridging the research-policy gap.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Author contributions

EHG led the conceptualization, interpretation, and original draft and made additional substantive revisions to the manuscript. NL led the analysis, interpretation and figure generation, and substantively revised the manuscript. RR contributed to the study design and facilitation of results discussion with participants during the conference panel. RS and SV contributed to the study design, facilitation of results discussion with participants during the conference panel, and substantive revision of the manuscript. ATR and AK contributed to the study design and manuscript revision. EE contributed to the manuscript revision. All authors participated in the conference panel, and read and approved the final manuscript.

Funding

This work is supported by the National Institute of Health (K24 AI134413 to EHG, K02 DA065836 to ATR).

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Ethics approval and consent to participate

This study did not involve the collection of identifiable or private information about individuals and therefore determined to be non-human subjects research.

Consent for publication

Not applicable.

Competing interests

EHG is a co-Editor-in-Chief of Implementation Science Communications. RR is an Associate Editor of Implementation Science Communications. All other authors declare that they have no competing interests.