Recall of clinical trial participation and attrition rates in survivors of acute respiratory distress syndrome

Abstract

Purpose

To measure the rate of recall of study participation and study attrition in survivors of acute respiratory distress syndrome(ARDS).

Materials/Methods

In this ancillary study of the Re-evaluation of Systemic Early neuromuscular blockade(ROSE) trial, we measured the rate of study participation recall 3 months following discharge and subsequent study attrition at 6 months. We compared patient and hospital characteristics, and long-term outcomes by recall. As surrogate decision-makers provided initial consent, we measured the rate of patient reconsent and its association with study recall.

Results

Of 487 patients evaluated, recall status was determined in 386(82.7%). Among these, 287(74.4%) patients recalled participation in the ROSE trial, while 99(25.6%) did not. There was no significant difference in 6-month attrition among patients who recalled study participation (9.1%) and those who did not (12.1%) (p = 0.38). Patient characteristics were similar between groups, except SOFA scores, ventilator-free days, and length of stay. 330(68%) were reconsented. Compared to those not reconsented, significantly more patients who were reconsented recalled study participation(78% vs. 66%;p = 0.01).

Conclusions

One in 4 ARDS survivors do not recall their participation in a clinical trial during hospitalization 3 months following hospital discharge, which did not influence 6-month attrition. However, more patients recall study participation if reconsent is obtained.

1. Introduction

Clinical research, particularly randomized clinical trials, relies on the participation of volunteers. Altruism—a desire to help others and further medical knowledge—often underlies this decision. Some perceive a benefit for themselves, including gaining more information about their own condition, access to additional care, and careful monitoring [1]. Additionally, those who have participated in a prior research study are more likely to do so in the future [2,3]. However, for many of the altruistic benefits of research study participation to accrue, for both the patient and future research, participants must be able to recall their participation in a trial.

Patients who experience critical illness often report memory deficits of their illness and hospitalization [4,5]. This lack of recall for hospitalization events can persist well beyond discharge [5]. Anecdotal reports obtained during longitudinal follow-up suggest patients do not always remember having been enrolled in a trial. However, no prior systematic assessment of the incidence or risk factors associated with lack of participation recall, nor its association with future attrition during long-term follow up has been performed.

Therefore, in a cohort of 577 adult survivors of moderate to severe acute respiratory distress syndrome (ARDS) enrolled in the Re-evaluation of Systemic Early neuromuscular blockade trial [6] we measured 1) the rate of recall of study participation at 3 month follow-up 2) the rate of attrition between those who remembered study participation and those who did not and 3) identified risk factors associated with lack of recall for study participation.

2. Methods

This is an ancillary study of the Re-evaluation of Systemic Early neuromuscular blockade (ROSE) trial [6], which sought to determine the efficacy and safety of early neuromuscular blockade in patients with moderate to severe ARDS. In total 1006 patients were enrolled, 501 in the intervention group and 505 in the control group. The primary outcome of the RCT was 90-day mortality (for which there was no difference), with secondary endpoints that evaluated long-term outcomes following the intervention. At 3, 6, and 12 months following hospital discharge, participants were contacted via telephone for the survey interview. Specifically, seven measures were assessed: Impact Event Scale-Revised (IES-R), EuroQol (EQ5D) health questionnaire, Katz Activities of Daily Living (ADL)/Lawton Instrumental Activities of Daily Living Scale (IADL), Montreal Cognitive Assessment (MoCA), Medical Outcomes Study Short Form-36 Health Survey (SF-36) Self-Reported Health, paralysis recall (using a modified Brice questionnaire), and return to work status using items from the Improving Care of ALI Patients (ICAP) study questionnaire. Interviews were completed by the patient or their proxy if the patient was unable.

At each telephone follow-up, research assistants were asked to determine if the respondent remembered being enrolled in the survey or not (we refer to this as enrollment recollection). The research assistants did not directly ask if the respondent recalled being enrolled in the study. Rather, the research assistants were asked to use comments or phrases stated by the respondent during the conversations (e.g. if the patient directly stated they remembered, if they said they were reminded about being in the study, if they remembered reading materials about the study). If the research assistant was unsure, they could so indicate.

2.1. Outcomes

We measured the rate of recall ascertained by research assistants at 3 months and study attrition by recall status (remembered vs. did not remember) at the 6-month follow-up. We compared baseline hospital and patient characteristics between those who remembered study participation and those who did not. Patient characteristic included age, sex, race (white or non-white), and education (high school degree or not). Hospitalization characteristics measured were baseline sequential organ failure assessment (SOFA) score, day 7 SOFA score, organ-failure free days in the hospital, ventilator free days in the hospital, day 7 Richmond Agitation-Sedation Score (RASS), corticosteroid use, duration of hypoxemia (days oxygen saturation <90%), and duration of shock (days in which the cardiovascular SOFA score is greater than or equal to 2). In addition, we measured the difference in the 12-month study outcomes by recall status. We also present study attrition and baseline characteristics among patients for whom research assistants were unsure of ability to recall study participation.

We also evaluated the rate of reconsent among study participants. Reconsent of study participants was attempted only prior to hospital discharge. Secondly, we measured the association between reconsent and study recall.

In a sensitivity analysis, we grouped those who remembered study participation and those for whom the research assistant was uncertain about recall status. We then compared this combined cohort (remember and unsure) to those who recalled being enrolled in the clinical trial during hospitalization.

2.2. Statistical analysis

Patient characteristics are presented as means (SD) or numbers (percentage). We compared in-hospital characteristics between respondents who remembered and did not remember being enrolled in the study. The difference in attrition rates and long-term outcomes between those who recalled study participation and those who did not was determined by two-tailed t-tests and tests of the equality of proportions. Additionally, we report the effect size between those who recall study participation and those who did not. Using a multinomial logistic regression model, we tested whether pre-defined patient or hospital characteristics were associated with enrollment recollection (yes vs. no vs. unsure). We report the average marginal effects (AME), confidence interval, and P-value for all coefficients in the model. All analyses were performed according to the intention-to-treat principle, without adjustment for multiple comparisons. Two-sided P-values of less than 0.05 were considered to indicate statistical significance. Analyses were performed with Stata software, Version 15 (StataCorp), and R software (R Foundation for Statistical Computing).

3. Results

Of the 1006 patients enrolled in the ROSE trial, 556 survived 3 months post-randomization. Of these, 487 (87.6%) received follow up at 3, 6, or 12 months following randomization and were included in our analysis (Fig. 1 ).

Fig. 1.

Flow diagram of patient enrollment, reconsent, and 3-month survey participation.

Of the 487 patients, recall of study participation was determined in 386 patients at 3 months post-randomization. Of these, 99 (25.6%) did not recall study participation, while 74.4% (n = 287) did. The 6-month attrition rate was 9.1% in patients who remembered compared to 12.1% in those who did not recall study participation (p = 0.38). Among those who did not recall participation at 3 months, an additional 60 patients reported recalling at 6 months. Less than 2% of patients who remembered study participation at 3 months, did not at 6 months. Seven patients did not remember study participation across all time points (3, 6 and 12 months) (Fig. 2 ).

Fig. 2.

Frequency of study participation recall.

Represents the number of patients by recall status at 3, 6, and 12 month follow up evaluation.

Among the 287 patients who recalled trial participation, 42% were female, 74%, were White, with a median age of 51.7 years and mean ICU length of stay of 11.5 days (Table 1 ). Of those who did not remember participation in the clinical trial (n = 99), 53% were female, 62%, were White, with a median age of 53.8 years and mean 13.9 day ICU length of stay. There were a significantly more ventilator-free days among patients who did recall participation compared to those who did not (16.2 vs 14.6, p < 0.01) and somewhat shorter ICU length of stay (11.5 vs 13.9 days, p < 0.05). Additionally, patients who recalled study participation at 3 months had a lower baseline SOFA score compared to those who did not (4.25 vs. 4.95; p = 0.04). A higher proportion of patients who recalled study participation were White compared to those who did not (74% vs. 62%; p = 0.02).

Table 1.

In-hospital characteristics of ARDS survivors, by recollection of enrollment at 3 months following hospital discharge.

	Recalls participation	Does not recall	P-value	Effect Size	95% CI
	n = 287	n = 99
Age (Median)	51.7	53.8		0.14	(−0.09, 0.37)
Age n(%)
<45 years	94 (33%)	25(25%)	0.16	−0.16	(−0.39, 0.07)
45–65 years	141 (49%)	54 (55%)	0.32	0.12	(−0.11, 0.34)
65+ years	52 (18%)	20 (20%)	0.70	0.04	(−0.18, 0.27)
Female n (%)	121 (42%)	52 (53%)	0.07	0.21	(−0.02, 0.44)
White n (%)	211 (74%)	61 (62%)	0.03	−0.26	(−0.49, −0.03)
Baseline SOFA, mean (SD)	4.3 (2.9)	4.9 (2.9)	0.04	0.24	(0.01, 0.47)
Day 7 SOFA	3.1 (2.9)	3.2 (2.9)	0.96	0.006	(−0.26, 0.27)
Organ Failure Free Days, mean (SD)	21.2 (7.5)	19.8 (8.4)	0.14	−0.18	(−0.41, 0.06)
Ventilator Free Days, mean (SD)	18.2 (8.1)	14.6 (9.3)	<0.01	−0.43	(−0.66, −0.19)
ICU Length of Stay, mean (SD)	11.5 (7.8)	13.9(8.0)	0.16	0.30	(0.07, 0.53)

The effect size is the difference in the mean or prevalence between those who recall participation and those who do not divided by an estimate of the standard deviation of the variable.

Research assistants were unsure of the respondents recall for 81 patients. For patients for whom the research assistant was unsure of their recall status, 6-month attrition was 22.2%. The cohort was 46% female and 58% White, with a mean age of 56 years, ICU stay of 15 days, and baseline SOFA score of 4.9. Patients had a mean 16.5 organ failure free days and 13.7 ventilator free days.

Patients who did not recall study participation had a lower MoCA blind score compared to those who did (21.1 Vs 22.9; difference − 1.86; p = 0.01). However, the EQ-5D-5L score was significantly higher in those who recalled participation compared to those who did not (0.68 vs 0.60; difference 0.08; p = 0.01). Finally, patients who recalled study participation were more likely to reside at home (0.92 vs 0.78; difference 0.14; p < 0.01) and return to work (0.21 vs 0.11; difference 0.10; p = 0.03). (Table 2 ).

Table 2.

Long-term outcomes of ARDS survivors, by recollection of enrollment.

	Recalls participation	Does not recall	P-Value
	n = 287	n = 99
	Mean (SD)	Mean (sSD)
EQ-5D-5L	0.68 (0.24)	0.60 (0.29)	0.01
Disability score	2.8 (2.3)	3.3 (2.7)	0.08
Self-rated health	3.3 (1.1)	3.4 (1.0)	0.34
Pain interference	2.6 (1.4)	2.6 (1.4)	0.84
MoCA blind	23.0 (4.7)	22.9 (4.8)	0.01
AD8 score	1.2 (2.3)	2.7 (2.7)	0.09
Other outcomes, n(%)
Residence at home	262 (92%)	77 (78%)	<0.01
Hospital readmission	82 (29%)	30 (30%)	0.77
ER visit	58 (20%)	14 (14%)	0.17
Return to work	59 (21%)	11 (11%)	0.03
Significant change in work duties	107 (63%)	34 (67%)	0.63

Having a high school diploma was associated with 17% increased probability of study recall at 3 months (Odds Ratio 2.91; Confidence Interval 1.6–5.3; p < 0.04). Additionally, for each day of corticosteroid use, probability of study recall increased by 3.8% (AME 0.038; 95% CI 0.01–0.07; p = 0.01). Alternatively, for each day of hypoxemia, probability of study recall decreased by 9.8% (AME -0.098; 95% CI -0.19 - - 0.001; p = 0.05). No other baseline characteristics were predictive of increased probability of recall, including age, days of shock, and ICU length of stay (Table 3 ). There were no statistically significant differences between the patients for whom research assistants could not holistically determine recall status and other patients.

Table 3.

Multinomial logistic regression model predicting recollection of study participation.

	Remember			Don't Remember			Unsure
	AME	95% CI	P-Value	AME	95% CI	P-Value	AME	95% CI	P-Value
Age
45–65	−0.07	[−0.17,0.025]	0.15	0.06	[−0.02,0.14]	0.15	0.01	[−0.07,0.09]	0.75
>65	−0.12	[−0.24,0.011]	0.07	0.06	[−0.047,0.17]	0.27	0.05	[−0.05,0.16]	0.31
High school diploma	0.17	[0.056,0.29]	0.04	−0.17	[−0.28,-0.056]	0.003	−0.08	[−0.10,0.09]	0.87
Days of hypoxemia	−0.098	[−0.19,-0.001]	0.05	0.05	[−0.029,0.12]	0.23	0.051	[−0.02,0.12]	0.14
Days of steroid use	0.038	[0.0078,0.068]	0.01	−0.02	[−0.048,0.01]	0.13	−0.017	[−0.04,0.09]	0.19
Days of shock	−0.004	[−0.030,0.022]	0.78	−0.01	[−0.03,0.01]	0.33	0.015	[−0.06,0.04]	0.16
>2 weeks in ICU	−0.083	[−0.19,0.022]	0.12	0.01	[−0.08,0.10]	0.8	0.072	[−0.01,0.16]	0.1

Average marginal effects (AME) reported and interpreted as: For one unit increase in X the probability of recalling study participation increases by AME.

Interpretive Example: For each day of hypoxemia, the probability of recall of study participation decreased by 9.8%. Alternatively, having a high school diploma increased the probability of recall of study participation by 17%.

3.1. Reconsent status

Of the 487 patients evaluated at the 3 month survey, 330 (68%) had been reconsented. Among those who were not reconsented, 81 (52%) were not because they had not regained decision-making capacity. 78% of patients who were reconsented recalled study participation at the 3 month survey compared to 66% of patients who had not been reconsented (p = 0.01).

3.2. Sensitivity analysis

In total, 368 (78.8%) patients recalled their participation in the ROSE clinical trial or the research assistant was unsure about recall status, while 99 (21.2%) did not recall (Online Supplement, eTable 1) participation. The 6-month attrition rate was 11.9% among patients who recalled study participation at 3-months and 12.1% among those who did not (p = 0.96). Unlike the primary cohort, there was no difference in EQ-5D-5L scores between groups. However, patients who did not recall study participation at 3 months had a modestly lower MoCA blind score compared to those who did (Online Supplement, eTable 2). Similar to our primary analysis, having a high school degree was associated with recall of study participation at the 3-month survey. However, no other baseline characteristics were associated with increased odds of recall (Online Supplement, eTable 3).

4. Discussion

In this secondary analysis of the adults with moderate to severe ARDS, nearly 1 in 4 patients did not recall participation in the ROSE clinical trial three months following hospital discharge. However, there was no significant difference in 6-month attrition between patients who recalled study participation and those who did not. There was no difference in outcome measures by recall status, except for cognition and functional status, which were modestly lower in those patients who did not recall participation. Finally, patients who were reconsented for study participation had a higher rate of recall at 3 months compared to those who were not reconsented but still more than 1 in 5 of reconsented patients did not recall participation.

In our data, patients who recalled study participation at 3 months had more ventilator free-days and a lower SOFA score. These differences suggest that even among a critically ill population, those who are “less sick” may have less difficulty with subsequent recall. Understanding risk factors for participant attrition has been a key component to improve overall study retention rates. Prior work has examined both study characteristics such as sample size, study duration and retention strategies, as well as individual patient factors including age, gender, and socioeconomic status [7,8]. A recent meta-analysis of cohort studies demonstrates retention rates are not moderated by these study or patient characteristics, with the exception of gender where cohorts with more female participants report higher retention rates [7].

Overall, patients for whom research assistants could not determine recall status were similar to those whose status was determined (remember or not remember). Additionally, demographic and hospitalization characteristics were not predictive of indeterminate recollection status. However, the attrition rate among this group was higher (22%) compared to patients who remembered study participation (9%) as well as those who did not recall study participation (12%).

Given the illness severity of patients with ARDS and resulting decisional incapacity, the ROSE clinical trial frequently relied on surrogate consent for study participation. Surrogate consent is a widely utilized method to obtain consent for research participation when a patient is unable to provide it themselves [9]. Indeed, prior studies suggest patients and families prefer this method of consent [[10], [11], [12]]. However, it is considered best practice for patients to be frequently reassessed for decision making capacity and once this is regained, consent from the patient should be obtained [10]. Similar to a prior study of ARDS patients [13], over 30% of patients were not reconsented for study participation. In our study population, half were not reconsented prior to discharge due to continued lack of decisional capacity.

Maintaining long-term follow-up is necessary to ensure the validity and reliability of a research study. As such, retention strategies are vital to limiting study attrition. Among patients with ARDS, prior work has identified respect for patients, tracking, and study personnel to be key areas for retention of study participants [14]. Similarly, Teague et al. grouped strategies into tracking, as well as barrier reduction, and follow-up/reminder approaches [7]. These strategies may be more or less feasible and influential depending on whether or not a patient recalls study participation. For example, extra effort may be allocated in training staff to explain the study to those who do not remember study participation.

In our data, any process that excluded patients who did not recall being a part of the study would systematically exclude less educated patients from follow-up. Our data also suggest that, having been reminded of their participation in the study, those who did not recall participation were equally likely to continue to participate—study attrition at 6 months was not significantly different between groups. Similarly, in our sensitivity analysis including patients for whom the research assistant was unsure if the patient was able to recall or not, there was no significant difference in the attrition rate at 6 months.

Knowledge surrounding study recall has also been examined in the retention of information disclosed during the informed consent process. Participants often display inadequate knowledge surrounding not only the general purpose of the study, but also the study risk and procedures [15]. Additionally, some indicate uncertainty surrounding whether or not they were even enrolled in a research study [16]. Recall of informed consent has been influenced by clinical factors, including severity of illness where the most severely ill patients retained the least information about risk and side effects [17], also similar in our findings.

This study has several limitations. First, as a longitudinal follow up study, it may be influenced by a survivorship bias, as we are unable to ascertain the recall status of those patients who died prior to 3 months. Second, because our study was a secondary analysis of the ROSE clinical trial, the recall status of each patient was determined by the research assistant, rather than a direct question of recall. Thus, the results may be subject to differences in inter-rater reliability. Finally, we relied on the data elements collected as part of the primary study and therefore may have been unable to identify pertinent risk factors of study recall.

In conclusion, nearly 1 in 4 ARDS survivors do not recall their participation in a clinical trial in the 3 months following hospital discharge, which did not influence 6-month attrition rates. However, the rate of recall is over 10% greater if re-consent is obtained, suggesting additional strategies to ensure patient knowledge surrounding clinical trial participation are warranted.

Disclaimer

This work does not represent the views of the US Government or Department of Veterans Affairs.

Support

This work was supported by U01 HL123031.

Declaration of Competing Interest

None.

Appendix A. Supplementary data

Supplementary material

Supplemental Tables 1, 2, 3