A Randomized Controlled Trial of Behavioral Nudges to Improve Enrollment in Critical Care Trials

Abstract

Rationale: Low and slow patient enrollment remains a barrier to critical care randomized controlled trials (RCTs). Behavioral economic insights suggest that nudges may address some enrollment challenges.

Objectives: To evaluate the efficacy of a novel preconsent survey consisting of nudges on critical care RCT enrollment.

Methods: We conducted an RCT in 10 intensive care units (ICUs) among surrogate decision-makers (SDMs). The novel multicomponent behavioral nudge survey was administered immediately before soliciting SDMs’ informed consent for their patients’ participation in a sham trial of two mechanical ventilation weaning approaches in acute respiratory failure. The primary outcome was the enrollment rate for the sham trial. Secondary outcomes included undue and unjust inducements. We also explored SDM and patient predictors of enrollment using multivariate regression.

Results: Among 182 SDMs, 93 were randomized to receive the intervention survey and 89 to receive standard informed consent. There was no statistically significant difference in enrollment rates between the intervention (29%) and standard consent (34%) groups (percentage difference, 5%; 95% confidence interval [CI], −9% to 18%; P = 0.50). There was no evidence of undue or unjust inducement. White SDMs were more likely to enroll the patient compared with non-white SDMs (odds ratio, 3.7; 95% CI, 1.1 to 12.2; P = 0.03). SDMs who perceived a higher risk of participation were less likely to enroll the patient (odds ratio, 0.57; 95% CI, 0.46 to 0.71; P < 0.001).

Conclusions: A preconsent behavioral nudge survey among SDMs of patients with acute respiratory failure in the ICU did not increase enrollment rates for a sham RCT compared with standard informed consent procedures.

Clinical trial registered with ClinicalTrials.gov (NCT03284359).

Participant recruitment is one of the most challenging aspects of and common barriers to the successful completion of randomized controlled trials (RCTs) (1). According to an Institute of Medicine report, over 40% of National Cancer Institute–sponsored clinical trials failed to achieve their minimal accrual goals at the time of enrollment closure, often due to slow or underenrollment (2, 3). Indeed, difficulties with participant recruitment frequently result in missed benchmarks and require study extensions (4, 5). Recruitment also represents one of the largest costs of conducting clinical trials (6). Trials requiring longer-than-expected recruitment may lead to increased costs, and failure to complete trials results in wasted financial resources. Furthermore, studies that are completed, but underpowered or terminated early due to low enrollment, expose participants to the trial’s risks, but reduce the likelihood of benefit, thus raising ethical concerns (7, 8).

Trial recruitment and enrollment challenges are often magnified in the intensive care unit (ICU) setting. One-third of the ICU RCTs published between 2007 and 2013 failed to recruit >95% of the target sample size (9), and multiple recent ICU RCTs have stopped early due to slow enrollment (10–16). Refusal to participate among critically ill patients and, more commonly, their surrogate decision-makers (SDMs), represents an important factor in duration of trial enrollment and generalizability of the results. Although an accurate estimate of real-world ICU RCT refusal rates is unknown, due, in part, to differences in recruitment approach and inconsistencies in reporting across trials, studies of hypothetical ICU RCT enrollment scenarios suggests refusal rates as high as 54% for patients and 59% for SDMs (17).

In an effort to address issues of slow and underenrollment, trialists are encouraged to evaluate different recruitment strategies by embedding trials of interventions within their parent trials (18, 19). Behavioral economics offers insights into such interventions, or “nudges,” and how they may address common enrollment barriers for patients and SDMs (20). Figure 1 depicts a conceptual model for surrogate decision-making regarding trial enrollment in the ICU. Factors, such as uncertainty surrounding patient outcomes, status quo bias, decision fatigue, and anticipated regret, are prevalent and may hinder SDMs’ ability to perform substitutive decision-making (21–24). The goal of nudges in the trial recruitment context is to rebalance these nonnormative influences acting against enrollment without causing undue or unjust inducement. Undue inducement occurs when payment blinds participants to risk or causes them to evaluate risks less clearly (25, 26). Unjust inducement occurs when an intervention selectively encourages enrollment among persons of lower socioeconomic status (27). Thus, we conducted an RCT to test the efficacy of a novel, preconsent behavioral nudge survey on enrollment decisions among SDMs of patients with acute respiratory failure (ARF) using a sham trial of different mechanical ventilation weaning approaches. We hypothesized that this simple, low-cost intervention would increase the enrollment rate for the sham ICU trial without undue or unjust inducement.

Figure 1.

Conceptual model of heuristics, cognitive biases, and nudges within surrogate trial enrollment decisions. (A) As patients rarely express wishes regarding research participation to surrogates, surrogates ideally rely on substitutive judgment, weighing the risks and benefits of participation. (B) Barriers, heuristics, and cognitive biases systematically bias against enrollment. (C) Nudges can be used to facilitate patient-centered decisions by rebalancing the scales toward a more normative decision-making process. (D) However, nudges may diminish risk perception, resulting in a bias toward enrollment, known as undue inducement.

Methods

Study Design and Setting

We conducted a two-arm, single-blind RCT to test the efficacy of a preconsent nudge survey compared with standard informed consent procedures on enrollment into a sham ventilation weaning trial among patients with ARF in the ICU. The trial was conducted in 10 ICUs across two hospitals at an urban academic medical center. ICU types included three medical, two surgical, two neurologic/neurosurgical, and three heart and vascular ICUs, with a total of 185 beds. The study protocol was approved by the University of Pennsylvania Institutional Review Board with a waiver of informed consent, and was registered before enrollment commencement on clinicaltrials.gov (NCT03284359).

Study Population

Participants included English-proficient adults (>18 yr of age) who self-identified as the SDM of a patient eligible for the sham RCT. Out of an abundance of caution to not exacerbate existing challenges in the patient–family–clinician relationship, SDMs were excluded if the treating clinician expressed concern regarding a strained clinician–family relationship or perceived distrust in the healthcare system. Patient eligibility criteria included an ICU stay >24 hours and receipt of invasive mechanical ventilation. Exclusion criteria included the presence of a tracheostomy, or anticipated extubation or transition to comfort-focused care within the subsequent 24 hours. These criteria were selected to reflect an ARF population likely to be eligible for an actual ventilation weaning trial.

Randomization

SDMs were approached in the patient’s ICU room. After affirmation that they were willing to hear about a research study, group allocation was determined by the Research Electronic Data Capture (28) randomization module, with assignment probabilities of 50% to each group stratified by research coordinator.

Intervention

We developed the preconsent behavioral nudge survey based on insights from colleagues at Penn’s Center for Health Incentives and Behavioral Economics (Table 1). The survey consisted of six questions incorporating five behavioral economic principles. First, a foot-in-the-door nudge (29, 30) involves asking a participant to perform a small request that has a high participation rate, followed by a larger request. The preconsent survey served as the foot-in-the-door for this trial. Second, the framing effect (31–33) describes peoples’ tendency to make decisions differently depending on how the information is delivered, with a focus on altruism in this study. Third, descriptive social norms (34, 35) highlight the common behaviors of one’s peers, such as participating in research studies. Fourth, a duty of reciprocity (36, 37) is the sense that one should repeat prosocial behavior from which they have benefited; for example, their loved one may be benefitting from treatments discovered from prior human subjects research. Fifth, the mere-measurement effect (38, 39) describes a tendency to actually perform an action after being asked about it in a hypothetical setting. Our overarching hypothesis was predicated upon a priming effect, whereby SDMs are more likely to enroll patients in the sham trial after receiving these simple nudges without restricting their choice to decline.

Table 1.

Pre-enrollment survey

Nudge Survey	Behavioral Economic Principle
Introduction: This is a survey to help us understand the relationship between the patient’s behaviors and their family’s feelings about participation in medical research.	Foot in the door
Please answer the following to the best of your ability about your loved one: (yes/no).	Framing: altruism
• My loved one would give directions to someone they didn’t know.
• My loved one would delay an elevator and hold the door for someone they didn’t know.
• My loved one would give clothes or goods to a charity.
• My loved one would give money to a charity.
• My loved one would offer to help a handicapped or elderly person across the street.
Statement: Many treatments used today that improve survival for patients in the intensive care unit were discovered through many patients’ participation in research studies.	Descriptive social norm
	Duty of reciprocity
Hypothetical: My loved one would participate in a research study if it was low risk to them and the information learned may help other patients in the future. (yes/no)	Mere-measurement effect

Sham ICU Trial

A waiver of informed consent was used to preserve participant blinding to the intent of the behavioral nudge, thus avoiding contamination of any potential intervention effect. We opted for a sham RCT for this study due to slow and low enrollment of ongoing ICU trials at the study institution. The sham nature of the study was not revealed to SDMs to promote the likelihood of genuine responses. Instead, it was described as a real, two-arm RCT comparing ventilation weaning approaches that differed by whether they prioritized reducing the fraction of inspired oxygen or the positive end-expiratory pressure first. The potential risks to participation were described as lung or other organ damage from exposure to elevated fraction of inspired oxygen and/or positive end-expiratory pressure, delayed recovery, and breach of data confidentiality. After data collection, research coordinators engaged all SDMs in an institutional review board–approved debriefing process in which the sham nature of the trial and the intent of the parent trial were revealed. Surrogates were approached in person by a physician who was also co–principal investogator of the sham ventilation weaning trial (D.C.K.) or by a research coordinator. Recruiters underwent initial observed training and periodic on-site checks to ensure quality control and adherence to the recruitment script (see the online supplement). SDMs were provided an opportunity to ask questions about the study. If the SDM deferred an enrollment decision, they were reapproached on subsequent days until a decision was made or the patient was no longer eligible for enrollment.

Data Collection, Measures, and Definitions

We collected patient demographic and clinical data from the electronic health record. An SDM demographic survey and all study instruments were completed in person using the Research Electronic Data Capture platform on electronic tablets (28). Only SDMs randomized to the intervention group completed the preconsent nudge survey. All SDMs were engaged in informed consent for the sham RCT, and we collected their enrollment decision, followed by two participation-risk assessments. First, SDMs rated their perception of the overall riskiness of trial participation using a Likert scale ranging from 1 (not risky at all) to 7 (very risky). We also used the nine-item comparative riskiness scale (CRS) (40) to provide a standardized context to the level of risk perceived across SDMs. Specifically, CRS asks whether participation in the sham RCT is riskier than other salient, more commonly encountered risks (e.g., talking on a cell phone while driving) using a dichotomous (yes or no) response. The CRS has been used in previous trials testing behavioral nudges to assess the risk perception of research participation (40, 41).

Outcomes

The primary outcome was enrollment of the patient in the sham RCT. The secondary outcome was the perceived risk of trial participation.

Statistical Analysis

Based on previous studies of enrollment rates for hypothetical critical care trials (17) and our pilot studies described in the online supplement, we assumed a 30% enrollment rate in the standard consent group. We calculated that enrolling 182 SDMs (91 per study arm) would yield at least 80% power to detect an absolute increase in enrollment to 50% at a two-sided α of 0.05 based on a chi-square test (42).

Characteristics of patients and SDMs are presented using number (%) and mean (SD), as appropriate. We included all randomized, eligible SDMs for the primary and main secondary analyses. Participants who declined further participation after being randomized and exposed to the intervention, and those who failed to make an enrollment decision for the sham trial before becoming ineligible, were considered to have declined sham trial enrollment. We used a chi-square test in primary unadjusted analyses to evaluate the primary outcome of the proportion of patients enrolled in the sham RCT. We used a Wilcoxon rank-sum test to compare the perceived risk of trial participation among each study group. Evidence of undue and unjust inducement was assessed by determining the interaction between the study group and 1) risk perception (1–7 count on Likert risk scale or 0–9 count on CRS) or 2) highest level of education (college or less than college) on enrollment rate, respectively.

Multivariate logistic regression was used to explore the relationship between baseline patient and SDM characteristics and enrollment in the sham RCT. Variables were included in the exploratory analysis based on their mechanistic plausibility to act as an independent predictor of enrollment, as determined by an experienced critical care trialist not involved in the study and blinded to the data. The total number of variables included in the final model was determined by the number of events (enrollments into the sham trial) to not exceed a 1:10 covariate-to-event ratio. We did not adjust for multiple comparisons for this study. All analyses were performed using Stata v14.2 (StataCorp). A two-sided P value of less than 0.05 was considered statistically significant.

Results

Participant Characteristics

Between February 2018 and August 2019, we screened 6,458 ICU admissions and identified 724 eligible patients (Figure 2). For 515 (71.1%) patients, their SDM was not at the bedside or they became ineligible before the research team could approach their SDM. Of the 209 SDMs who were approached, 18 (8.6%) declined to hear about the study, and the remaining 191 (91.4%) were randomized. Nine (4.7%) of these were identified as ineligible after randomization and excluded from the analytic sample. Of the 93 SDMs randomized to the intervention arm, 15 (16%) declined to complete the preconsent survey, 6 (6%) declined to engage in informed consent procedures after survey completion, 6 (6%) deferred their sham trial enrollment decision, and 66 (71%) made an enrollment decision. Of the 89 SDMs randomized to the standard consent arm, 13 (15%) deferred their enrollment decision and 76 (85%) made an enrollment decision.

Figure 2.

Consort diagram. EMR = electronic medical record; ICU = intensive care unit; LOS = length of stay.

Patient (Table 2) and SDM characteristics (Table 3) were similar between the two study groups, except primary diagnosis. Patients had a mean (SD) age of 60.0 (±15.5) years and a mean pre-enrollment length of stay of 106 (±82.4) hours or 4.4 days. The mean age among SDMs was 55.9 years (±14.1) and the most common relationship to the patient was spousal (53%). The 32 (34.4%) SDMs in the intervention group and 29 (32.6%) SDMs in the standard consent group opted not to complete either risk assessment or provide their demographic information.

Table 2.

Characteristics of the enrolled patients

Characteristic	Intervention	Standard Consent
Total N	93	89
Age, yr, mean (SD)	61.3 (13.6)	58.6 (17.2)
Female sex, n (%)	35 (38%)	36 (40%)
Race, n (%)
White	74 (80%)	59 (66%)
Black or African American	15 (16%)	23 (26%)
Other	4 (4%)	7 (8%)
Hispanic, n (%)	3 (3%)	4 (4%)
Length of ICU stay at enrollment, h, mean (SD)	97.8 (73.1)	114.5 (90.6)
Prior ICU admission, n (%)
Yes	20 (22%)	21 (24%)
No	41 (44%)	39 (44%)
Unknown	32 (34%)	29 (33%)
Prior research participation, n (%)
Yes	20 (22%)	20 (22%)
No	35 (38%)	38 (43%)
Unknown	38 (41%)	31 (35%)
Admission diagnosis, n (%)
Neurologic disease	8 (9%)	16 (18%)
Pulmonary infection	8 (9%)	13 (15%)
Cardiac surgery	15 (16%)	6 (7%)
Other medical	12 (13%)	7 (8%)
Abdominal surgery	8 (9%)	10 (11%)
Cardiac disease	12 (13%)	6 (7%)
Septic shock	6 (6%)	10 (11%)
Other surgery	8 (9%)	5 (6%)
Respiratory failure	5 (5%)	7 (8%)
Trauma	6 (6%)	2 (2%)
Malignancy	1 (1%)	6 (7%)
Transplant	4 (4%)	1 (1%)
APACHE IV score, mean (SD)	97.8 (33.4)	95.9 (32.3)

Definition of abbreviations: APACHE = Acute Physiology and Chronic Health Evaluation; ICU = intensive care unit; SD = standard deviation.

Table 3.

Characteristics of the enrolled surrogates

Characteristic	Intervention	Standard Consent
Total no. providing data	61	60
Age, mean (SD)	58.4 (13.3)	53.5 (14.5)
Female sex, n (%)	47 (77)	46 (77)
Race, n (%)*
White	48 (83)	42 (72)
Black or African American	7 (12)	10 (17)
Other	3 (5)	6 (10)
Hispanic, n (%)*	2 (3)	1 (2)
Education, n (%)*
Grade school or some high school	0 (0)	1 (2)
High school graduate	19 (32)	9 (15)
Some college	14 (24)	16 (27)
College graduate	16 (27)	19 (32)
Advanced degree	10 (17)	15 (25)
Relationship to patient, n (%)*
Spouse or partner	29 (48)	35 (58)
Child	12 (20)	10 (17)
Parent	10 (17)	10 (17)
Other	9 (15)	5 (8)
Prior ICU admission, n (%)	11 (18)	6 (10)
Prior research participation, n (%)*	13 (22)	11 (18)
Prior research exposure, n (%)*†	30 (50)	30 (51)

Definition of abbreviations: ICU = intensive care uni; SD = standard deviation.

^*Missing data: race (n = 5); ethnicity (n = 7); education (n = 2); relationship to patient (n = 1); prior research participation (n = 1); and prior research exposure (n = 2).

^†“Has anyone you have known ever participated in a medical research trial?”

Enrollment Rate

Overall, 31% of surrogates consented to enroll the patient into the sham trial. There was no statistically significant difference in enrollment rates between the intervention (29%) and standard consent (34%) groups (percentage difference, 5%; 95% confidence interval [CI], −9% to 18%; P = 0.50).

In the multivariate model (Table 4), surrogate race and perception of trial risk were independently associated with sham trial enrollment. White SDMs were more likely to enroll the patient compared with non-white SDMs (odds ratio [OR], 3.7; 95% CI, 1.1–12.2; P = 0.03). SDMs who reported a higher perceived risk of participation on the CRS were less likely to enroll the patient (OR, 0.57; 95% CI, 0.46–0.71; P < 0.001).

Table 4.

Final multivariable model for patient and surrogate characteristics associated with enrollment into sham trial

Characteristic	OR (95% CI)	P Value
Patient previously enrolled in research	1.76 (0.57–5.37)	0.32
Surrogate previously enrolled in research	1.21 (0.34–4.35)	0.77
CRS	0.57 (0.45–0.71)	<0.001
Surrogate perception of patient trajectory as improving	0.89 (0.31–2.58)	0.83
Surrogate white race	3.69 (1.11–12.21)	0.03
APACHE IV score	1.01 (0.99–1.03)	0.11

Definition of abbreviations: APACHE = Acute Physiology and Chronic Health Evaluation; CI = confidence interval; CRS = comparative riskiness scale; OR = odds ratio.

Analytic sample included patients with complete data (n = 105).

Risk Perception and Undue and Unjust Inducement

There was no significant difference in CRS scores between the intervention (median = 2; interquartile range [IQR], 0–5) and standard consent arm (median = 2; IQR, 0–5.5) (Z = −0.10; P = 0.92). Likewise, there was no significant difference in the Likert risk perception scores between the intervention (median = 4; IQR, 2–5) and standard consent arm (median = 4; IQR, 2–5) (Z = 0.43; P = 0.66). There was no evidence of undue inducement as measured by either the CRS (OR for interaction with study arm, 1.16; 95% CI, 0.77–1.73; P = 0.48) or the Likert risk perception score (OR for interaction with study arm, 1.01; 95% CI, 0.70–1.47; P = 0.95). There was also no evidence of unjust inducement on SDMs with the highest completed education of less than college (OR for interaction with study arm, 3.1; 95% CI, 0.49–19.2; P = 0.23).

Discussion

RCTs are widely considered the highest standard for evidence-based medicine; however, slow and underenrollment lead to increased costs and threaten RCTs’ scientific integrity. This RCT found that a collection of behavioral nudges, delivered as a preconsent survey, did not increase enrollment into a sham mechanical ventilation weaning trial when compared with a standard recruitment approach.

There are several potential explanations for the study’s null findings. First, the nonadherence rate among SDMs randomized to the intervention arm was nontrivial, with one in five SDMs either deciding not to complete the survey or declining to consider the sham trial after completing the survey. The preconsent survey was introduced to SDMs in a standard manner as “a survey to help us understand the relationship between the patient’s behaviors and their family’s feelings about participation in medical research.” Perhaps SDMs were anticipating a therapeutic rather than a sociobehavioral research study, which was not perceived as relevant to their loved one’s current clinical status. Although we explained that the research study was entirely separate from the patient’s ICU care, SDMs may have also been concerned that their survey answers may result in value judgements by the ICU clinicians and potentially impact care. Whatever the explanation, our choice to code all of these SDMs as not being willing to provide consent for the patient to enroll was conservative, potentially biasing the results toward the null.

In addition, despite the choice to compare two standard approaches to ventilation weaning in the sham trial, SDMs were very concerned about trial-related risks. For example, many surrogates perceived the risk of trial participation as greater than taking three times the recommended amount of a pain killer, riding a motorcycle without a helmet, and bungee jumping on the relative risk scale. Furthermore, risk perception was an independent predictor of SDMs’ trial enrollment decision, with higher risk perception portending significantly lower odds of enrollment. Although it is not surprising that SDMs consider risk in their enrollment decision-making, the high clinical stakes in the ICU setting may lead to an inflated perception of risk. Nonetheless, it is important that nudges for research recruitment do not cause undue inducement by reducing the perceived risk of participation, which the preconsent nudge did not do. This finding also suggests that future interventions that detect and realign perceived versus actual risk among eligible SDMs may prove more effective.

We found that SDM race was associated with enrollment, with white SDMs more likely to enroll their loved ones in the sham trial. This is contrary to a study examining the screening logs of acute lung injury clinical trials, which showed no difference in the likelihood of enrollment across racial groups (43). However, racial minorities have been underrepresented in cancer trials (44), human immunodeficiency virus trials (45), and therapeutic trials leading to U.S. Food and Drug Administration approvals (46). Differences in the willingness of racial minorities to participate in clinical research likely stems from long-standing patterns of health inequities, historical research exploitation, dishonest risk disclosures, and skepticism about researchers’ motivations (44). Future interventions aimed at increasing critical care trial enrollment should specifically consider these additional barriers that racial minorities face, as such underrepresentation limits the generalizability of the knowledge generated.

This study has several important strengths. To our knowledge, it is the first randomized trial of a hypothesis-driven intervention to mitigate the enduring problem of low enrollment in critical care randomized trials. Second, by conducting this study among actual SDMs in the ICU setting using a waiver of consent, more than 90% of approached SDMs participated, and their enrollment choices were not artificially altered by knowledge of the sham nature of the trial. Our incorporation of a robust debriefing process, without engendering any negative responses from participants or reports of untoward impacts on clinician–family relationships, suggests that the use of a waiver of consent and a sham trial among SDMs of critically ill patients is a feasible approach for evaluating future behavioral interventions to increase enrollment. Finally, because undue and unjust inducements are still possible, even when the intervention does not produce a main effect on enrollment, the absence of evidence of either form of adverse inducement provides support for future efforts to nudge enrollment decisions.

This study also has several important limitations. First, the use of a sham ventilation weaning trial may limit generalizability to real-world trials testing other interventions. Although it would be optimal to embed enrollment intervention trials within ongoing critical care trials, the problem of slow and low enrollment that such interventions are designed to address is exactly why this approach is often infeasible. Second, we limited our recruitment efforts to business hours due to resource constraints, thus potentially missing SDMs who are only present in the ICU during nights and weekends. SDMs that visit during nonbusiness hours may differ from weekday visitors in ways that could affect their willingness to consent to participation in RCTs, which limits the study’s generalizability. As recruitment is often one of the costliest components of conducting a trial, creative approaches to vary recruitment timing in future ICU studies are needed to capture a more broadly representative SDM population. Third, the SDM predictors of enrollment should be interpreted with caution, given the high rate of missing risk and demographic data among SDMs who declined enrollment into the sham trial. Finally, we are unable to examine the independent effects of each of the five nudges in the survey to determine if one was more or less effective than another.

In summary, we found that a collection of nudges, incorporated into a novel preconsent survey, was not effective in increasing enrollment into a sham mechanical ventilation weaning trial when compared with a standard enrollment approach. Future research is needed to better understand factors that influence the perception of risk of trial participation in the ICU. Furthermore, our experience suggests that the use of a sham trial with waiver of informed consent among active ICU SDMs is a feasible method, thereby enabling the efficient evaluation of future enrollment interventions.