Impact of Past Research Experience on Subsequent Trials: A Caution

To the Editor:

One of the primary goals for recruitment into a clinical trial is to maximize enrollment while minimizing expenses (1). For inner-city minority populations, recruitment of participants can be especially challenging, and one of the strategies used by the National Institute of Allergy and Infection Diseases (NIAID) Inner-City Asthma Consortium research program has been to recruit from a pool of current participants in natural history or epidemiologic protocols. These pools of participants typically provide researchers with a well-characterized population within which eligible study participants can be more easily identified and recruited at a considerably lower cost.

This report assesses the impact that previous participation in a research study has on the response to an intervention. The NIAID Inner-City Asthma Consortium portfolio provides a unique opportunity to assess this question. The initial National Cooperative Inner-City Asthma Study (NCICAS) was a large-scale, descriptive epidemiologic study followed by an asthma intervention study (NCICAS II). Because nearly one-third of the approximately 1,500 original participants proceeded to enroll in the subsequent intervention, this question of prior research experience can be directly tested.

NCICAS phase II, a 2-year randomized controlled trial of a tailored intervention based on the family’s asthma risk (2), enrolled 1,033 participants between 5 and 11 years of age who had a physician diagnosis of asthma and persistent asthma symptoms and/or evidence of emergency department visits or hospitalizations for asthma in the prior 6 months. The study was designed as a follow-up to NCICAS I, a 12-month epidemiologic observational study that enrolled 1,529 children, ages 4 through 9 years, with the goal of identifying risk factors for asthma morbidity (3); 457 (44%) of the children enrolled in NCICAS II were recruited from NCICAS I.

The primary clinical outcome measure in NCICAS II was self-reported maximum symptom days (MSDs), a validated asthma outcome (4) that has been used in numerous NIAID Inner-City Asthma Consortium studies (5–7). MSDs were defined as the maximum number of days in the prior 2 weeks during which participants experienced any of the following: wheezing, loss of sleep due to asthma, or reduction in play activity caused by asthma (2). Secondary clinical outcome measures included a 2-month recall of asthma-related hospitalizations and unscheduled visits to the doctor or emergency department. These three measurements were collected every 2 months over the course of 2 years. The MSDs were averaged over each of the 2 years of follow-up, and hospitalization data were defined as “ever” over the course of each year separately, and number of unscheduled visits was summed within each year.

We reanalyzed the NCICAS II data, using analysis of variance as reported previously (2), but stratified on prior research involvement (research naive vs. research experienced). Analyses of the primary and secondary endpoints were adjusted for site, MSDs (MSDs <4 vs. MSDs ≥4), and baseline variables, including those that differed significantly between the research-naive and research-experienced groups. Baseline data were compared with either analysis of variance or the Mantel-Haenszel chi-square test. Statistical analyses were performed with SAS version 9.4 (SAS Institute) and R version 3.2.2 software.

All participants in NCICAS phase II, regardless of recruitment source, met the same protocol entry criteria. Research-naive participants—those who have not been involved in prior NIAID-funded inner-city asthma trials—were significantly older, were less likely to be black, and had more MSDs than research-experienced participants (Table 1).

Table 1.

Baseline Characteristics of Study Populations for NCICAS II

	Enrolled in NCICAS I		P Value
	Yes (n = 457)	No. (n = 576)
Sex, M	63% (287)	65% (347)	0.48
Age, yr, mean ± SD	7.5 ± 1.7	7.8 ± 1.9	0.003
Race
Hispanic	17% (76)	18% (102)	0.02
Black	78% (353)	72% (415)
Other	6% (26)	10% (59)
Income <$15,000/yr	71% (320)	64% (362)	0.02
Maximum symptom days (randomization)	4.6 ± 4.2	5.5 ± 4.8	0.001
Any hospitalizations?*	3% (15)	5% (31)	0.10
Number of asthma medications, mean ± SD	2.3 ± 1.4	2.9 ± 1.5	<0.001

Definition of abbreviation: NCICAS = National Cooperative Inner-City Asthma Study.

^*Number represents any hospitalizations within the prior 2 months.

Figure 1 illustrates the significant −0.9 MSD difference between intervention and control groups for the research-naive participants (P < 0.001) in the first year of follow-up. In contrast, the difference between treatment groups for the research-experienced participants was only −0.1 MSD (P = 0.64). The difference in the reduction of MSDs between the two subpopulations (i.e., the research experience-by-intervention interaction term of the model) was significant (P = 0.037). MSDs in the second year of follow-up also showed a strong intervention effect for the research-naive population (difference, −0.8; P = 0.001) versus the research-experienced participants (difference, −0.2; P = 0.50); the experience-by-intervention interaction trended toward significance (P = 0.10). Similar trends toward larger treatment effects in research-naive participants were observed for the secondary outcomes of hospitalization and unscheduled visits in both Years 1 and 2, with only unscheduled visits for asthma during the second year of follow-up approaching an interaction P value of 0.11 (data not shown).

Figure 1.

Postbaseline maximum symptom days, by research experience. Bar height represents the adjusted estimated mean (SE) postbaseline maximum symptom days for both control (black bars) and intervention participants (gray bars). The analysis is stratified by prior research experience (naive vs. experienced) and endpoint collection time (Years 1 and 2). The n value represents the number analyzed within each subgroup. The intervention effect (with 95% confidence interval) is shown at the top of each panel; significant within-group effects are shown in bold. The interaction P value is given in the column header. NCICAS = National Cooperative Inner-City Asthma Study.

During the recruitment phase of a trial, researchers often target potential participants who have been enrolled in a previous protocol at their institution or clinic. Benefits of enrolling these participants include the following: Their contact information is already known; the staff is familiar with the participants and their families; participants are familiar with study procedures; less staff time is used for screening; and clinical measurements needed to determine study eligibility may already have been collected in the recent past, thus reducing cost and burden. Previous research experience is generally considered exclusionary only if a previous drug or intervention is likely to have a direct effect on the intervention under investigation. However, it appears that patients with asthma who have participated in an NIAID Inner-City Asthma Consortium research protocol, even a noninterventional epidemiologic protocol, respond differently to a subsequent intervention than newly recruited participants.

The NCICAS II asthma counselor intervention was consistently less effective for the study-experienced participants across a range of outcomes, although the experience-by-intervention interaction effect was not always statistically significant. However, all analyses were post hoc and exploratory, and therefore they were not powered a priori.

We have observed similar trends in another set of clinical trials conducted by the NIAID Inner-City Asthma Consortium, but the sample size of participants who overlapped between the two trials was limited. ICATA (Inner-City Anti-IgE Therapy for Asthma), a 60-week, double-blind, placebo-controlled, randomized clinical trial conducted to determine if guideline-based asthma care would be further improved by the addition of anti-IgE therapy (omalizumab) (6) recruited 39 (9%) of its eligible participants from the ACE (Asthma Control Evaluation) trial, a 46-week, double-blind, randomized clinical trial that determined if exhaled nitric oxide–guided therapy improved asthma control when used as an adjunct to the National Asthma Education and Prevention Program–recommended guideline-based asthma care (8). Although 9% of the ICATA participants were recruited from ACE, we observed a trend consistent with what we report in the present letter from NCICAS using a similar stratified analysis (data not shown).

These findings suggest that research-experienced participants in a clinical trial tend to have fewer asthma symptoms. Increasing the proportion of the sample with reduced morbidity will reduce the power of the study and may bias results toward the null hypothesis, thus warranting consideration when designing protocol entry criteria for intervention trials. Investigators may wish to consider limiting the recruitment of research-experienced participants in their intervention trials or at the very least carefully examining the impact of “research experience” on their outcome measures.