COMPETING COMMITMENTS in CLINICAL TRIALS

Abstract

Most discussion about clinical care in clinical trials has concerned whether subjects’ care may be compromised by research procedures. The possibility that clinical researchers might give priority to helping their “patients” even if that required deviating from the imperatives of the research protocol largely has been ignored. We conducted an on-line survey with clinical researchers, including physicians, research nurses and other research staff, to assess the ways and frequency with which clinical trials may be at risk for being compromised by clinical researchers’ attempting to address the clinical needs of subjects. The survey covered recruitment, clinical management while in the trial, and termination decisions. It produced a 72.0% response rate. Over 20% of respondents agreed that researchers should deviate from the protocol to improve subjects’ care; 28% reported that medications restricted by the protocol were given; 21% reported that subjects who were not eligible had been recruited; and 9% said subjects had been retained in a trial despite meeting termination criteria. Some respondents reported that these deviations from the protocol happened many times. The ramifications of these findings are discussed.

Scholars of the ethics of clinical trials have long recognized tension between clinicians’ “therapeutic obligations” [1] and the scientific demands of clinical trials. [2–4] The basis for this concern is clear: clinical care is focused on improving the condition of the presenting patient, while clinical research seeks valid, generalizable information to help future patients. Although in principle the arms of most clinical trials are designed to have an equal likelihood of efficacy for the typical patient, in many situations research and clinical goals may nonetheless be in conflict.

The relationship between therapeutic and scientific obligations within clinical trials is controversial. Some commentators argue that clinicians should always act in the best interest of their individual patients, even when a patient is enrolled in clinical research. These questions arise, with varying implications, at all stages of a trial. For example, during the recruitment process, clinical researchers may have to decide whether or not to offer participation to a patient who, although technically ineligible, may benefit from trial entry.[5, 6] Once a participant enrolls, researchers concerned with her medical welfare may have to decide whether or not to deviate from protocol requirements with respect to dose of the study medication, provision of adjunctive medications, or performance of procedures to collect required study data. Finally, clinical researchers may have to decide whether or not to keep a participant in a trial who, despite technically meeting a termination criterion, seems to be benefiting from ongoing trial participation.

Whatever the scope of a therapeutic obligation concerning clinical trials, the potential for clinical researchers’ clinical commitments to undercut the research protocols has largely been ignored. With few exceptions [7], ethical and policy discussions of this issue have centered on how research imperatives may interfere with clinical care, not how clinical imperatives may lead to decisions that may compromise the validity of research. What little has been written on gaps between design and practice assumes that violations of research integrity are motivated by selfish interests [8–12]. This is not just a theoretical issue. In some circumstances, decisions motivated by concerns for participants’ medical interests may threaten the validity of the research, waste the financial resources invested as well as the efforts of researchers and subjects, and lead to invalid data entering the medical knowledge base.

Although there has been considerable theoretical debate about potential conflicts between research and clinical care, there are no empirical data to indicate how clinical researchers actually perceive and resolve any tensions that arise between the two sets of obligations. There is, however, evidence from cancer trials that clinicians prioritize patients’ clinical needs in deciding whether or not to offer them trials. Some studies report that many physicians base referrals to research projects on whether they perceive that patients are likely to benefit from participation [13–15] Winn et al. summarized these findings: “The physician’s perception of the nature of a research protocol and its potential benefits and detriments to his patient’s condition may therefore be a significant factor in determining whether a physician referral will transpire ([16] :64).” Though laudable from the point of view of clinical ethics, in some cases such selection factors might lead to selection bias and therefore limit the generalizability of trial results.

These considerations lead to a concern that recruitment biases or clinically motivated protocol deviations may compromise the validity of clinical research. In light of the dearth of empirical data addressing these issues, we sought to determine how often clinical researchers confront such conflicts and how they address them when they arise.

Methods

Study Sample

Survey recipients were sampled from contacts listed on Centerwatch.com, an on-line listing of “over 30,000” clinical trials that, at the time of the survey, was the largest listing of active clinical trials. This list contained 4682 unique contacts of whom 3318 listed an email and a domestic address. Telephone calls to listed contacts who only had telephone numbers identified email addresses for an additional 352 potential subjects. We drew a random sample of 1000 subjects from this pool of 3670 study contacts for inclusion in the survey. Because the percentage of physicians in this sample was inadequate (11%) to permit assessment of whether physicians’ opinions differed from those of non-physicians, we selected an additional 250 physicians who were listed as study contacts, for a total initial sample size of 1250 subjects.

Survey Instrument and Procedures

In order to inform development of the survey instrument, we conducted 27 in-depth interviews with 12 physician-investigators, 12 nurse coordinators and 3 non-nurse research coordinators. In these interviews, participants described how and when clinical and research obligations conflict and when they felt it necessary to violate the protocol in the interests of their “patients.” The interviews helped us to craft the wording of the survey questions and pointed to the importance of nurses and other non-nurse research coordinators--not just physicians--in day-to-day decision-making about the conduct of clinical research. After pilot-testing the survey with 8 additional interviewees, we made several minor modifications.

The survey had 37 questions on various topics including demographic information, and training and clinical trials experience The primary survey items, however, fell into two groups, attitudinal statements and behavioral reports. Eight attitudinal items queried respondents’ beliefs about the appropriateness of actions that, although they may conflict with good research standards in clinical trials, might benefit participants. The questions were framed as statements, and respondents were given the choices of agree, mostly agree, uncertain/neutral, mostly disagree, disagree, and not applicable.

Because attitudes may not accurately predict behavior in a tightly regulated situation like a clinical trial, we also asked respondents about their behavior during the prior two years. The behavioral items were organized in groups of three questions. For each topic, the first question asked how often they had faced a particular conflict between protocol requirements and the subject’s best medical interests during the previous two years.

After each opening question, two follow-up questions asked how often they had resolved the conflict by following the protocol, versus how often they had decided to do what they believed was in the subject’s best medical interest.

The survey also included items querying respondents’ demographic, educational and training characteristics, the duration of their clinical trials experience, the roles that they had played, and the types of trials in which they had worked.

Survey Methods

The survey was conducted utilizing the on-line survey site SurveyMonkey. An internet-based survey was appropriate for this purpose because most clinical researchers use the internet and have email access. The survey was conducted in six steps using a modified version of the Tailored Design Method [17]: 1) a hand-signed mailed letter; 2) an email with a link to the survey; 3) a reminder postcard; 4) a repeat email with a link to the survey; 5) a personal telephone call; and 6) an email sent from our study email address (rather than the automated online system) to minimize the possibility that recipients’ spam blockers prevented our emails from reaching them.

Of the 1250 subjects in the initial sample, 216 had invalid emails. Among 1034 subjects with valid email addresses, 744 (72.0%) responded and completed at least some of the attitudinal or behavioral questions. Sample sizes for individual questions varied due to item non-response. Three respondents who reported having taken part in no clinical trials in the last two years were excluded from the analysis, leaving a final sample of 741.

Human Subjects Review

Participants in the in-depth interviews gave written informed consent after study approval by the University of Massachusetts (UMass) Medical School and Dana-Farber Cancer Institute IRBs. Because of its anonymous nature, the Internet survey was judged by the UMass IRB to be exempt from the requirement for human subjects review.

Frequencies, chi-squares and analyses of variance were computed using SAS and JMP statistical packages.

Results

Respondent characteristics

Table 1 describes characteristics of survey respondents. Most respondents were between 30 and 60 years old. Eighty percent of respondents were female, and 88% were white. Forty-six percent were nurses, whereas 20.3% were physicians. Three-quarters had worked on at least 6 trials in the previous 2 years.

Table 1.

Respondent Characteristics

Characteristic	Number*	Percent
Gender
Female	581	78.8%
Male	156	21.2%
Age
<30	56	7.6%
30–39	206	27.8%
40–49	246	33.2%
50–59	193	26.1%
60+	39	5.3%
Race/Ethnicity
White	648	88.1%
Black	21	2.9%
Asian	45	6.1%
Other	22	3.0%
Training
Physicians	147	19.8%
Other Doctoral	23	3.1%
Nursing	337	44.5%
Other Masters	105	14.2%
Others	129	17.8%
# of CTs in last 2 years
1 to 6	71	9.6% 9.6%
7 to 10	140	18.9%
11 to 15	183	24.7%
16 to 20	127	17.1%
21+	220	29.7%

^*Numbers may not sum to 741 due to missing responses to individual questions

Table 2 describes the roles respondents played in their clinical trials. Most respondents were involved directly with study participants. Eighty-nine percent always or frequently participated in the consent process, 42% always or frequently referred patients, 45% always or frequently decided which trial a subject should participate in, 78% always or frequently managed the care of subjects, and 90% always or frequently participated in data collection. Involvement in the design and reporting of trials was less common. Few respondents (11%) always or frequently participated in designing trials, 21% always or frequently participated in data analysis, and 12% always or frequently participated in writing articles. Not surprisingly, physician-respondents differed from other respondents in the activities they focused on in their clinical trial work. They participated in fewer trials (p<.0001) and were less likely to report involvement in collecting data (p<.0001). In contrast, they were more likely to participate in designing trials (p<.0001), analyzing data (p<.0001), and writing manuscripts (p<.0001) (data not shown).

Table 2.

Study Roles Played by Survey Respondents

	Always		Frequently		Occasionally		Rarely		Never
	MDs	Others	MDs	Others	MDs	Others	MDs	Others	MDs	Others
Participate in the design of the trial N=733 p<.0001	14 9.8%	9 1.5%	34 23.8%	23 3.9%	25 17.5%	65 11.0%	24 16.8%	108 18.3%	46 32.2%	385 65.3%
Refer patients N=728 P<.05	41 29.1%	94 16.0%	30 21.3%	137 23.3%	32 22.7%	169 28.8%	14 9.9%	73 12.4%	24 17.0%	114 19.4%
Decide which study to offer to patients N=735 p<.001	60 41.4%	142 24.1%	48 33.1%	214 36.3%	20 13.8%	131 22.2%	7 4.8%	42 7.1%	10 6.9%	61 10.3%
Participate in the consent process N=732 P<.0001	76 52.4%	418 71.2%	45 31.0%	111 18.9%	18 12.4%	28 4.8%	2 1.4%	13 2.2%	4 2.8%	17 2.9%
Manage clinical care of subjects N= 735 P<.05	77 53.55	295 49.9%	40 27.8%	126 21.3%	15 10.4%	76 12.9%	5 3.5%	18 3.1%	7 4.9%	76 12.9%
Collect data N=737 P<.0001	72 50.0%	471 79.4%	43 29.9%	79 13.3%	12 8.3%	23 3.9%	7 4.9%	6 1.0%	7 4.9%	76 12.9%
Analyze data N=729p<.0001	27 19.0%	60 10.2%	30 21.1%	38 6.5%	24 16.9%	104 17.7%	21 14.8%	114 19.4%	40 28.2%	271 46.2%
Participate in drafting manuscripts N =732 P<.0001	21 14.8%	16 2.7%	27 19.0%	24 4.1%	23 16.2%	52 8.8%	24 16.9%	92 15.6%	47 33.1%	406 68.8%

Attitudes towards conflicts between clinical and research imperatives

Respondents were presented with eight statements that addressed attitudes towards conflict between scientific and clinical imperatives. Table 3 shows responses to these items. Two items addressed investigators’ and centers’ decisions about participation in trials. 14% disagreed or mostly disagreed that “centers should choose which trials to participate in based on how much the trials contribute to science,” whereas 51% agreed or mostly agreed that “researchers should only participate in trials that are likely to help the subjects who take part.”

Table 3.

Attitudes toward Clinical Trials

Question	Agree		Mostly Agree		Uncertain/ Neutral		Mostly Disagree		Disagree
	M.D.s	Others	M.D.s	Others	M.D.s	Others	M.D.s	Others	M.D.s	Others
Research centers should choose which trials to participate in based on how much the trials contribute to science. N=711, p<.0001	7 4.9%	46 8.1%	6 4.2%	42 7.4%	12 8.5%	121 21.3%	67 41.2%	259 45.5%	50 35.2%	101 17.8%
Researchers should only participate in trials that are likely to help the subjects who take part. N =728, p<.005	27 18.8%	88 15.1%	39 27.1%	86 14.7%	17 11.8%	100 17.1%	39 27.1%	176 30.1%	22 15.3%	134 23.0%
Even if patients are technically eligible for a trail, they should only be recruited if being in the trial will be in their best medical interests. N=719, n.s.	14 9.6%	39 6.7%	23 15.9%	64 11.1%	18 12.4%	74 12.8%	46 31.7%	160 27.6%	44 30.3%	242 41.8%
Patients who are not doing well with standard care should be recruited most actively so that being in the trial can help them. N=719, n.s.	28 19.4%	104 18.1%	25 17.4%	83 14.4%	31 21.5%	161 28.0%	44 30.6%	171 29.7%	16 11.1%	56 9.7%
When several subjects at a site do considerably worse than would be expected in ordinary care, that site should stop recruiting for that study. N =719, n.s.	24 16.4%	86 15.0%	24 16.4%	102 17.8%	33 22.6%	140 24.4%	31 21.2%	157 27.4%	34 23.4%	88 15.4%
Researchers should deviate from the protocol if it would improve the subject’s medical care. N=710, n.s.	61 42.7%	254 44.8%	28 19.6%	112 19.6%	18 12.6%	89 15.7%	22 15.4%	77 13.6%	14 9.8%	35 6.2%
The protocol should be used as a guideline rather than something to be strictly followed under all circumstances. N=725, n.s.	83 57.2%	345 59.5%	38 26.2%	135 23.3%	6 4.1%	31 5.3%	12 8.3%	53 9.1%	6 4.1%	16 2.8%
It is okay to ignore minor entry criteria if a patient will benefit from being in the trial. N=719, p<.05	97 66.9%	435 75.7%	22 15.2%	90 15.5%	15 10.3%	29 5.0%	9 6.2%	17 2.9%	2 1.4%	5 0.9%

Three questions concerned which patients should be recruited to trials. Almost 68% agreed or mostly agreed that “even if patients are technically eligible for a trial, they should only be recruited if being in the trial will be in their best medical interests.” Likewise, 40% agreed or mostly agreed with the statement that “patients who are not doing well with standard care should be recruited most actively so that being in the trial can help them.” Finally, 43% percent agreed or mostly agreed that “when several subjects at a site do considerably worse than would be expected in ordinary care, that site should stop recruiting for that study.” These attitudes are not statistically significantly related to how often the respondent reports referring patients to trials or deciding which trial to offer to patients.

The last group of items asked about attitudes towards explicit protocol violations in the service of therapeutic objectives. Almost 21% agreed or mostly agreed with the statement that “researchers should deviate from the protocol if doing so would improve the subject’s medical care,” 12% agreed or mostly agreed that “the protocol should be used as a guideline rather than something to be strictly followed under all circumstances,” and 5% agreed or mostly agreed that “it is acceptable to disregard minor entry criteria if a patient will benefit from being in a trial.”

There are some differences in attitudes between physician and non-physician clinical researchers. Physicians were more likely to suggest that one should choose trials in which to participate based on their contribution to science (p<.0001) and less likely to endorse choosing trials based on whether they help subjects (p<.003). They more often endorsed ignoring minor entry criteria for clinical benefits (p<.005) and were less likely to agree that one should remove subjects from trials on the basis of clinical judgment (p<.0001).

Frequency of and behavioral responses to situations in which clinical and research imperatives conflict

Questions and responses to the behavioral items are presented in Table 4. Ninety-six percent of subjects reported experiencing at least one situation during the previous two years in which clinical and research requirements conflicted.

Table 4.

Clinical Behavior in Clinical Trials “How many times in the past two years….”

Question	0	1–2	3–5	6–10	11–15	16+	DK/N A
Have you had a patient who was eligible for a clinical trial, but being in the trial seemed not to be in the patient’s best medical interests? N=671	15.7%	23.7%	25.3%	11.8%	2.8%	6.3%	14.5%
Was the trial not offered to the patient? N = 665	19.4%	23.9%	15.5%	7.7%	3.0%	4.4%	26.2%
Have you had a patient who was not technically eligible for a clinical trial, but being in the trial seemed to be in the patient’s best medical interests? N=667	19.3%	17.2%	17.7%	15.9%	4.8%	13.0%	12.0%
Was the trial offered to the patient? N=660	62.4%	15.0%	3.8%	1.4%	1.1%	0.2%	16.2%
Was a medication restricted by protocol, but giving the medication seemed to be in the subject’s best medical interests? N=670	26.1%	16.4%	15.8%	10.3%	3.0%	6.9%	21.5%
Was the medication given? N = 664	44.7%	17.8%	5.0%	2.9%	1.5%	0.9%	27.3%
When adjusting the dose of the medication seemed to be in the subject’s best medical interests, but making the adjustment was not permitted by the protocol? N= 670	33.7%	17.2%	16.0%	7.3%	2.4%	5.1%	18.4%
Was the dose of the study medication adjusted? N=661	59.3%	9.6%	3.8%	1.4%	1.1%	0.2%	(24.8 %
When breaking a blind without reporting it seemed to be in the patient’s best medical interests? N=672	75.6%	8.2%	1.9%	0.3%	0.2%	0.6%	13.2%
Have you decided to break the blind without reporting it? N=657	76.0%	0.9%	0.2%				23.0%
Have you had a subject who met termination criteria, but remaining in the trial seemed to be in the subject’s best medical interests? N = 668	50.0%	18.6%	9.9%	4.0%	1.1%	2.7%	13.8%
Was the subject kept in trial? N=655	69.0%	6.7%	1.7%	0.2%	0.2%	0.2%	22.1%

Two questions addressed conflicts that arise during the enrollment process. Almost 70% reported at least one instance in which they had had a patient who was eligible for a clinical trial, but being in the trial seemed not to be in the patient’s best medical interests; 6% reported that this had happened 16 times or more. Among those who reported being in this situation, 55% reported at least one instance in which the trial had not been offered to an eligible patient. Similarly, 69% of respondents reported having had at least one subject who was ineligible, but for whom participation would have been beneficial. Of these, 22% reported that the ineligible subject was recruited, and 2.7% reported that this had occurred 6 times or more. Those who more frequently decided which patients should be recruited for the study were more likely to report having patients who would benefit from being in the trial but did not meet eligibility criteria (p<.0001), and were more likely to report having patients who were eligible but whose medical best interest was not served by being in the trial (p<.0001).

Two questions addressed medication restrictions. Fifty-two percent had faced at least one situation in which a medication that they believed to be in the subject’s best medical interests was restricted by protocol. Of these, 28% reported that the subject had been given the restricted medication at least once, and 5% reported that this had occurred 6 or more times. Similarly, 48% had faced at least one situation in which adjusting dosages of the study medication outside the range prescribed by the protocol seemed to be in the subject’s best medical interests. Of these, 16% reported that the dose had been adjusted outside the prescribed range at least once.

Respondents were also asked how often they had confronted a situation in which breaking a study blind without reporting it seemed to be in the subject’s best medical interest. Although 11% reported having been in such a situation, only 1% reported actually having broken the blind.

The final question asked about study participants who met termination criteria but whose best medical interests would be served by staying in the trial. Thirty-six percent reported confronting this situation at least once during the previous two years; 9% reported that the subject had been kept in a trial despite meeting termination criteria. There were no differences between physicians and non-physicians on any behavioral questions.

Discussion

There has been considerable recent concern about the possibility that involvement in clinical trials may entail compromises in the clinical care provided to participants. Many commentators have noted the potential for conflict between the research goals of clinical trials and the clinical goals of maximizing the quality of care that participants receive. Great effort has gone into both the ethical task of justifying the potential disadvantages that subjects face in clinical trials and the practical task of designing trials that minimize these disadvantages.

The findings reported here suggest another potentially problematic aspect of the conflict between clinical and scientific goals. In this survey, many clinical researchers expressed strong commitments to their “patients.” For example, just over half of respondents agreed or mostly agreed that researchers should only participate in trials that are likely to help the subjects who take part, and almost 40% agreed or mostly agreed that patients who are not doing well with standard care should be recruited most actively so that being in the trial can help them. Smaller numbers of respondents endorsed deviating from the protocol or ignoring minor entry criteria if doing so would be in the interests of subjects.

Responses to survey question about behaviors suggest that commitments to participants’ medical interests can lead clinical researchers to deviate from the letter and spirit of the trial protocols. This is particularly clear in the 26% of survey subjects who reported that research subjects had been given medications that were proscribed by the protocols, the 16% of respondents who said that dosages of medication had been adjusted outside the permitted range, and the 9% of respondents who reported that a subject had been kept in a trial on clinical grounds despite the fact that she or he met termination criteria. The effects of such deviations are unpredictable, but given the right circumstances, they could compromise the validity of trial findings. Although a large majority of respondents did not report being aware of such deviations from protocols, the number of affirmative responses suggests that such behavior is not rare, and raises questions, which this study was not designed to answer, regarding the impact of these actions on the findings of clinical trials.

Potentially just as serious for the generalizability of the results of the clinical trials was the frequency with which survey respondents reported that subjects had been included or excluded from trials based on the likelihood of direct benefit. Fifty-four percent of respondents reported that they were aware of situations in which trials had not been offered because participation was thought not to be in the patient’s best interest. Such decisions may, of course, be ethically justified when viewed from the perspective of a particular subject. However, recruiting subjects who are doing poorly on standard care more vigorously than those who are doing well might bias the results of the trial in two ways: 1) the resulting sample may be more resistant to treatment interventions in general, or 2) if the control group is receiving standard care, the difference between the control and experimental interventions may be exaggerated.. Although decisions not to offer trials to eligible patients do not usually constitute protocol violations, it is important to be aware of potential for such decisions collectively to impact the generalizability of study results. Conversely, 21% of respondents reported that trials had been offered to subjects who did not meet recruitment criteria because participation was perceived to be in their medical interest. Unless approved by sponsors and IRBs and described in study reports, such deviations from prespecified eligibility criteria clearly represent protocol violations and may directly affect the generalizability of study findings.

We note several limitations to these results. First, as with all surveys, it is difficult to be sure that respondents correctly interpreted the survey items. Although we based the questions’ phrasing on detailed preliminary interviews and pilot-tested the survey instrument, the questions may have confused some respondents. Second, the population sampled was drawn from contact persons listed on a consumer-oriented website and does not represent perfectly the population of clinical researchers. Third, the incidence of behaviors reported may reflect an underestimate of their actual occurrence, as respondents may not have been aware of or recalled all of the instances that arose in the studies with which they were involved. Fourth, we do not know how often these deviations from the protocols were reported to sponsors and IRBs, and the subjects involved subsequently excluded from the data analysis. Fifth, the survey method prevented us from collecting data on the various trials that the respondents had worked on, and we were therefore unable to link the specific behaviors with the details of the clinical trials. Finally, because we asked about the number of the behaviors of interest that occurred but not about the number of subjects with whom the respondents had had contact, we cannot offer a base rate for the occurrence of these behaviors. Thus, it is not possible to gauge directly the ultimate impact of these protocol deviations on the results of clinical trials. If the deviations reported represent a sufficiently small proportion of clinical decisions in each trial, their impact might not have substantially altered trial results.

Conclusion

In spite of the above limitations, these data raise questions about the impact of decisions grounded in clinical commitments on the validity of data derived from clinical trials. That most clinical researchers admit that they are aware of selection biases and explicit deviations during the recruitment process, and of violations of the terms of protocols, due to what they perceive to be the best medical interests of their patients, suggests that conflicts between these obligations may adversely influence the validity of findings from clinical trials. There is an urgent need for additional data on the prevalence of protocol deviations, the causal role that clinical imperatives play in prompting these behaviors, and especially the impact of such behaviors on clinical trial results.

It is important to note that many respondents did not experience conflicts between the medical best interests of their subjects and the requirements of the research protocol. This suggests that it is possible to design and implement studies that do not raise such conflicts. However given the frequency with which conflicts are reported, there is a need to consider further actions to reduce clinically motivated protocol deviations. First, investigators developing clinical trials should anticipate situations in which clinical and scientific goals may conflict, and where methodologically feasible should make design choices that minimize the frequency and intensity of those conflicts. Whatever the scientific virtues of tight restrictions on entry criteria, adjunctive treatments and dose modifications, they are of little value if they are not followed. Clinical researchers are clinicians with clinical values and commitments; trials must be designed so that clinicians are comfortable carrying them out. This may involve a move to something closer to the way effectiveness trials are currently carried out, allowing clinicians more freedom to use the intervention in the way in which it would be used in ordinary practice.[18,19] It could also involve new approaches to the analysis of the results of trials since as a consequence of the increased flexibility many variables would need to be statistically controlled.

Equally important, there is a need for renewed attention to the education and training of people who conduct clinical trials. We cannot take for granted that investigators and study staff truly appreciate the importance of the research methods that they are charged with implementing and the impact that failure to adhere to those methods may have on the validity of the research. Such training must deal directly with concrete situations in which the two commitments might conflict and how research staff should respond.