Assessing the Need for a Research Ethics Remediation Program

Abstract

With supplement funding to the Washington University CTSA, the Restoring Professionalism and Integrity in Research (RePAIR) program was developed at Saint Louis University to meet the remediation needs of institutions nationwide regarding investigators who violate research regulations. With the aim of determining the frequency and kinds of wrongdoing at leading research institutions in the United States, as well as institutional responses and levels of interest in a formal remediation program, an online questionnaire was distributed by email to a research integrity officer (RIO) and institutional review board (IRB) chair at all medical schools and comprehensive doctoral institutions in the United States (N = 194). One hundred sixty‐one individuals responded (44%) representing 66% of institutions. For those institutions that had both RIOs and IRB chairs responding, 96% had investigated at least one case over the past 2 years; the modal individual response was 3–5 cases, with a range from 0 to more than 16 cases. The most common forms of wrongdoing were violations of procedure, informed consent, research integrity (fabrication, falsification, plagiarism), privacy, and conflict of interest policies. Most RIOs and IRB chairs expressed interest in the RePAIR program, despite concerns about costs and faculty resistance.

A recently developed taxonomy of wrongdoing in research identifies 14 primary kinds of violations of professional standards for researchers, including research misconduct (which is federally defined as falsification, fabrication, or plagiarism—FFP), informed consent failures, privacy/confidentiality violations, improper care of animals, conflict of interest violations, and others.1

Some forms of wrongdoing appear to be quite common. While determining the frequencies of FFP is controversial,2 it is probably the most studied domain of wrongdoing. A recently published meta‐analysis of survey data on this topic estimates that almost 2% of scientists have admitted fabricating, falsifying, or modifying data at least once and, when asked about colleagues’ behavior, over 14% of scientists surveyed reported knowing of data falsification.3 Applying a conservative rate of 1.5% to the 155,000 researchers supported by the US National Institutes of Health, Titus, Wells, and Rhoades4 estimated 2,325 cases of FFP per year occur that deserve investigation. FFP is just one of 14 categories of wrongdoing in research. When asked about a much broader set of research ethics violations and questionable practices, 33% of researchers self‐reported questionable behavior5 and 84% reported observing questionable behavior among colleagues.6

Wrongdoing in research causes significant problems for multiple stakeholders. Research misconduct or FFP impedes research progress, undermines public trust in research, and wastes public dollars by introducing false information into the scientific literature, distorting meta‐analyses, and straining the scientific publication system as it forces editors to take special effort to detect misconduct.7, 8, 9, 10, 11 The infamous Tuskegee syphilis study chillingly demonstrates how improper exposure to risk and consent violations can harm participants and negatively impact trust in clinical research.12, 13

Although few data on recidivism (or repeated wrongdoing after getting caught) are available in the domain of research, in other professions once violations of professionalism occur, the risk of recidivism is high. For example, while less than 1% of physicians with no sanctions from 1994 to 1998 received a sanction during 1999–2002, more than 20% of physicians receiving moderate/severe sanctions in the first period recidivated in the subsequent time period.14 Our ongoing study of high‐profile cases of wrongdoing in research indicates that many investigators have offended in more than one environment15; oftentimes, earlier offenses are only made public once an investigator is caught at another institutions and these offenses are publicly reported.16, 17, 18, 19, 20, 21, 22 The confidentiality (or secrecy) of institutional responses to wrongdoing often appears to enable further wrongdoing.23

Why would an institution choose to refer an investigator for intensive professional remediation education rather than terminate employment? First, termination has downsides, including: The loss of an investigator in whom the institution, and often funding agencies, have heavily invested; the loss of research funding and oftentimes research staff working in the lab of the terminated investigator; and potential for legal actions. Second, remediation may present a reasonable way of achieving a variety of goals: preventing recidivism; restoring trust; and managing risk by having a reasonable response plan in place.

For physicians who experience lapses in professionalism in medical care, excellent remediation education programs exist. In these programs, participants meet in small groups for several days and engage in exercises that address some of the root causes of unprofessional behavior. Outcomes appear promising: Physicians demonstrate marked improvement in skills and peer evaluations on multiple behavioral measures.24, 25 Until recently, no such program existed for researchers.

Moreover, a small but growing body of evidence gathered across the past two decades indicates that most current instruction programs in the responsible conduct of research (RCR) are not effective in improving ethical decision making or behavior; in fact, for reasons that are currently poorly understood, RCR training may be associated with worse professional behavior.26, 27, 28

We recently received funding from the National Institutes of Health to develop a research ethics remediation program, “Restoring Professionalism and Integrity in Research” (RePAIR). Information on the RePAIR program can be found at www.repairprogram.org. The RePAIR program is not an ethics course, but rather an intensive professional development program that is based on best available evidence regarding the nature of wrongdoing in research and the factors that predict poor ethical decision making. While a full description of the curriculum and the evidence‐base supporting the curriculum is beyond the scope of this needs assessment paper, data indicate that problems in the conduct of research arise in part due to self‐serving biases, faulty mental models for research, stress, and the failure to forecast long‐term consequences of actions, including especially consequences to others.1, 15, 29, 30, 31, 32, 33, 34, 35, 36 The development team includes industrial‐organizational, clinical, experimental, and developmental psychologists, as well as lawyers, researchers, research administrators, and research ethics educators. The program consists of assessment, online training when knowledge deficits are identified, and a 3‐day onsite education program aimed at reducing levels of self‐serving bias in research, fostering ethical decision‐making skills, teaching stress management and work management skills, and developing individualized professional development plans that will be tracked across the following year.

As a first step in the development of the RePAIR program, we conducted a survey of all comprehensive doctoral institutions and allopathic medical schools in the US to assess their needs for a research ethics remediation program. This paper presents details on that survey and explores the implications of our findings.

Methods

The needs assessment survey was reviewed and determined to be exempt by the Institutional Review Board of Saint Louis University (protocol #21655).

Sampling, procedure, and response rates

Using the 2011 Carnegie Classifications, we identified the 194 higher education institutions in the USA with medical schools and/or comprehensive doctoral programs. This included 52 institutions with National Institutes of Health‐funded Clinical and Translational Science Awards (CTSAs). (We do not present separately CTSA institutional data, as they did not differ significantly from other institutions on any variables we tracked.)

For each institution, email addresses for a Research Integrity Officer (RIO) and an Institutional Review Board (IRB) Chair were obtained using Internet resources. In January 2012, emails containing a link to the electronic needs assessment survey were sent to the RIOs and IRB chairs. If a response was not received within 2 days, a follow‐up telephone call was made to discuss the survey opportunity; in most cases, a live person was not reached and a voicemail message was left. If a completed survey was not received within 2 weeks of initial delivery, a follow‐up email was sent. Two weeks after this, any recipient who had not responded received a “last chance to participate” email. Recipients were also given the option of not completing the survey and forwarding it to someone else at the institution who they believed was better suited to respond; this person was generally the Chief Research Officer (CRO).

Overall, 369 individuals were contacted at 194 institutions. Forty‐four percent of individuals (161/369) responded, representing 66% (129/194) of institutions. Among the 52 CTSA institutions, 67% were represented. Two responses from different officials in the same institution were received for 25% of the institutions.

Survey design

The authors developed the final 10‐question needs assessment survey after circulating drafts among the 12‐member RePAIR program advisory committee. Committee members recommended adopting a very brief questionnaire, rather than the initial 37‐item draft, to enhance response rates.

Respondents first read an overview of the RePAIR program presented in a Q & A format. In the survey itself, respondents were asked to identify their primary role with respect to research oversight and how long they had served in that role. A primary needs assessment question was an estimate of the number of credible cases of ethical wrongdoing in research that had been formally investigated by the institution in the past 2 years. If the latter number was ≥1, respondents were asked to indicate the types of research wrongdoing investigated (including multiple types within single cases), selecting from 14 different types of wrongdoing identified in previous research by this team.1 They were also asked to indicate the types of responses made by the institution to the wrongdoing, to rate their level of satisfaction with those responses, and to explain why they were either satisfied or dissatisfied. Lastly, respondents rated the likelihood that their institution would use the RePAIR program and to identify primary obstacles to using the program.

Results

Sample characteristics

As displayed in Table 1, approximately 40% of respondents were RIOs, 40% were IRB Chairs/Directors, and 20% were Chief Research Officers. Eighty‐seven percent had been in their roles over the past 2 years, the timeframe of interest in our survey.

Table 1.

Overview of sample and key responses

Variable	Category	Pct.*
Role	Research integrity officer	40%
	IRB Chair/Director	40%
	Chief research officer	20%
Length of service in role	<2 years	13%
	2–4 years	31%
	5–10 years	41%
	>10 years	15%
Institutional responses to wrongdoing	Letter of reprimand	74%
	Increased oversight	72%
	Education: in‐person, internal	71%
	Education: online	44%
	Termination	13%
	Otherc	14%
	Education: in‐Person, external	11%
Satisfaction with institutional responses	Very satisfied	29%
	Somewhat satisfied	53%
	Somewhat unsatisfied	14%
	Very unsatisfied	4%
Likelihood of using RePAIR	Very likely	8%
	Likely	20%
	Somewhat likely	57%
	Not at all likely	16%

*Percentages rounded to whole number.

Assessment of need

Prevalence and types of research wrongdoing

Figure 1 shows the frequency of wrongdoing indicated by RIOs, IRB chairs, and CROs. The modal frequency of individual respondents was 3–5 cases of wrongdoing investigated over the past year; responses covered the full range from 0 cases to 16 or more.

Figure 1.

Number of cases of investigated in past 2 years.

Figure 2 presents the responses of our two targeted groups—RIOs and IRB chairs—regarding the types of wrongdoing investigated. All 14 types of wrongdoing were represented in the sample. The most common types of research wrongdoing investigated included violations of procedures, consent, research integrity (fabrication, falsification, plagiarism or FFP), oversight, privacy/confidentiality, and conflict of interest policies. As indicated in Figure 2, reports of kinds of wrongdoing are clearly tied to job titles: IRB chairs reported nearly twice as many consent violations and more than three times as many privacy violations as RIOs; RIOs report more than three times as many cases of research misconduct and nearly six times as many animal violations as IRB chairs. This distinction was constant even when we examine responses from RIOs and IRBs chairs from institutions with two respondents (n = 32); such respondents did not differ significantly from individuals at single‐respondent institutions in the kinds of roles they play or the kinds of cases that they reported.

Figure 2.

Frequency of kinds of wrongdoing reported by RIOs versus IRB chairs.

Thirty‐two institutions had two respondents, representing separately oversight of research integrity and human subjects protections. These 64 responses also indicate that individual responses are indeed connected to their job titles and are not reflective of wrongdoing at the entire institution. Five institutions had an individual indicate investigating “no cases of wrongdoing”—a rate nearly identical to our overall sample (16%); however, in all but one of these instances, the response was contradicted by an institutional colleague who had investigated cases within his or her job title. Among the 32 institutions with 2 respondents, 96% had a respondent indicate a case was investigated over the past 2 years.

Current institutional responses to research wrongdoing

Table 1 shows that institutional responses to research wrongdoing are primarily in the form of letters of reprimand, increased oversight of the wrongdoer's work, and education of the wrongdoer offered within the institution. At 87% of institutions, no case within the prior 2 years resulted in termination. Table 1 indicates that less than 30% of respondents were very satisfied with these institutional responses to wrongdoing; open‐ended responses from respondents indicating that they were less than “very satisfied” consistently described reasons for dissatisfaction such as consistency/cooperation issues, diffusion of responsibility, inadequacy of responses, and unclear policies.

Intention to use the RePAIR program

As noted in Table 1, 28% of respondents indicated they would be likely or very likely to use RePAIR for remediation of research wrongdoing; only 16% stated they were “very unlikely” to use the program. However, when respondents reported investigating 10 or more cases over the past 2 years, their interest in using the RePAIR increased to 37%. The primary obstacle to use of RePAIR was cost, followed by potential resistance from faculty or administration, and liability/privacy concerns.

Discussion

Wrongdoing in research is relatively common with nearly all research‐intensive institutions confronting cases over the past 2 years. Only 13% of respondents indicated that a case involved termination, despite the fact that more than 50% of the cases reported by RIOs involved FFP. This means that most investigators who engage in wrongdoing, even serious wrongdoing, continue to conduct research at their institutions.

Given rates of recidivism generally observed in medicine (referenced in the introduction), the authors recognize a need for evidence‐based remediation education that addresses the causes of wrongdoing. Drawing from data from development team members, the RePAIR program hypothesizes that these causes include self‐serving biases, inadequate ethical problem solving skills, and poor stress management and work management skills. Modeled after physician remediation programs, the RePAIR program will provide intensive, face‐to‐face group work with similarly situated individuals, addressing mental models, ethical problem solving, and stress and work management strategies. The RePAIR program will thus provide institutions with a middle ground between termination of employment and mild responses that may not be effective in reducing rates of recidivism.

Our data indicate that research violations are not uniform: they include issues of research integrity (e.g., publishing of falsified data), where the scientific enterprise is victimized; violations of human subjects (e.g., improper or inadequate consent processes), where individual (and potentially vulnerable) human beings are victimized; and animal care violations, where nonhuman animals are victimized. Violations also vary in terms of severity ranging from “procedural violations” to FFP.

However, these variations matter little within the framework of the RePAIR program, which is less focused on blame than on behavioral change. In all instances of research wrongdoing, including persistent noncompliance, there are behaviors that need to change. Even in cases of so‐called procedural violations, behaviors need to change. We believe the most common procedural violations are repeated protocol deviations that have led up to a formal investigation of the researcher, including deviations from approved human subjects protocols (e.g., enrolling participants who do not meet inclusion criteria) or approved IACUC protocols. This is consistent with most participants indicating multiple causes for an investigation; it is also consistent with anecdotes shared informally by RIOs and IRB chairs during a series of four webinars and seminars conducted with more than 300 participants over the past several months.

The RePAIR program is focused on improving the behavior of researchers using the best available evidence from multiple domains of psychology. Thus, the national availability of the RePAIR program for remediation may address demonstrated needs of research‐intensive institutions as they address wrongdoing in research, including persistent noncompliance with approved research protocols.

With a 44% response rate from individuals and only 66% of institutions represented, we cannot be certain that nonrepresented institutions have similar experiences. However, we believe that our sample is representative of the larger population in important regards. Holding a CTSA grant is generally indicative of an institution being research‐intensive in the health sciences; the percent of CTSAs represented in our study (67%) was nearly identical to the percent of institutions represented in the study (66%).

The survey had other limitations: In an effort to achieve a high response rate, we limited the questionnaire to 10 items. This prevents us from answering many “follow‐up” questions regarding the demographics of the individuals investigated or the specific nature of responses (e.g., the specific education programs offered within the institution).

The need for remediation education in research may be even greater than our data suggest: We did not survey representatives from Institutional Animal Care and Use Committees (IACUCs), which means our estimation of cases of wrongdoing at institutions is low given that the roles played by respondents clearly affected the number of cases reported.

The RePAIR program will afford a unique opportunity not only to address a significant problem faced by research institutions, but also to study the effects of remediation education and to compare those investigated for wrongdoing to those who have not been investigated. The RePAIR program held its first face‐to‐face professional development session in January 2013; seminars will subsequently be offered at least three times per year.

Funding

This paper was supported by an administrative supplement award to the Washington University CTSA (3UL1RR024992‐05S2) from the NIH‐National Center for Research Resources (NCRR).