Education Research and Human Subject Protection: Crossing the IRB Quagmire

It used to be…fairly easy to get a proposal through IRB. Now it is easier to write the grant proposal than the IRB proposal.

Anonymous researcher¹

Background

Stamina and mental fortitude are necessary attributes for the present‐day researcher seeking approval from his institution's human subjects protection review board (IRB). Both popular media and respected journals continue to report extremely rare but serious harms to research subjects as well as overly bureaucratic IRB responses. This leads to a pervasive perception of IRB overregulation yet underprotection of human subjects.² Although the Office for Human Research Protections (OHRP) reports that 70% of allegations of research misconduct are ultimately not substantiated,³ the risk of suspension or disruption of research during an investigation, in addition to the possibility of damaging reputations and future funding, creates a national climate of anxiety among researchers and ensures continuation of excessive scrutiny of research processes. In fact, deficiencies detected by OHRP are primarily failures of documentation or failure to follow required procedures, not claims of harm to persons or unethical conduct.⁴

However, a recent article by members of the National Institutes of Health and OHRP⁵ emphasizes that current regulations do permit significant streamlining of ethical review. Options including exemption or expedited review are underutilized: in the past, 25%–77% of United States IRBs were found to review more rigorously than regulations required.⁵ These streamlined processes are particularly relevant to medical education research.

Does Medical Education Research Require IRB Approval?

Yes, these studies usually meet requirements for IRB review, as they entail both (1) research (see box 1) and (2) interventions or interactions with human subjects, or identifiable private information from these subjects. The Code of Federal Regulations Governing the Protection of Human Subjects in Research⁶ is based on the 1979 Belmont Report⁷ and earlier work. The report proposed guidelines for ethical treatment of research subjects guided by 3 ethical principles, beneficence, respect for persons, and justice, which are to be accomplished through attention to informed consent, risk‐benefit assessment, and equitability of subject selection. The primary responsibility of an IRB is to protect the rights and welfare of human research subjects and to ensure that risks undertaken by subjects are reasonable in relation to the potential benefits. If your institution accepts federal funding, you must adhere, even if your research is not federally funded: although not required by the Code of Federal Regulations Governing the Protection of Human Subjects in Research, currently IRBs extend federal regulations to all nonfederal research. Private IRBs and review boards are not subject to this law and are increasingly used in clinical research.

box 1 Definition of Research⁶

• Systematic investigation to develop or contribute to generalizable knowledge

  • Includes testing and evaluation

Ethical issues in medical education research often arise in subject recruitment, informed consent, confidentiality, and use of de‐identified existing medical education data.⁸ Students in particular are considered “at risk” subjects due to the underlying power differential between teachers and students, who may receive grades, recommendations, and promotion ratings from their teacher‐researcher. As a result, students may feel coerced to participate. Although a power differential may not always exist, residents, faculty, other physicians, and members of the health care team are human subjects, and thus IRB review is required when these groups are studied. In contrast, research involving meta‐analyses, systematic reviews, consensus reports, or curriculum proposals does not require IRB review.

Coercion and Informed Consent

Medical education research subjects must not be coerced or unduly influenced to participate but rather allowed to “opt out.” Subjects must be provided informed consent for their participation, unless waived by the IRB (see box 2). Even data previously collected, such as routine course evaluations, are subject to IRB review if the course director will be using the data for research. Ideally, one should anticipate the potential future use of data, although this is not always possible. An IRB may refuse to grant approval retrospectively, although this should be unlikely if data are de‐identified and no harm to subjects can be perceived.

box 2 Criteria Required to Waive Informed Consent¹⁰

• Waiver of consent will not adversely affect rights and welfare of subjects

  Research could not practicably be carried out without the waiver

  • When appropriate, subjects will be provided with additional pertinent information after participation

Are anonymous evaluations or questionnaires, distributed by organizations disconnected to the subjects surveyed, subject to IRB review? An example is the Association of American Medical Colleges (AAMC) medical student graduation questionnaire. In 2004 allegations against AAMC's use of information from the questionnaire were brought.⁹ Allegedly, use of this data represented human subjects research that had not received IRB approval or exemption status. Upon review many allegations were not upheld. However, the AAMC agreed to submit future administrations of the questionnaire for IRB review.⁹ At a university where an IRB decided an educational research project evaluating a new medical school curriculum did not require review, allegations of ethical violations were brought against the research faculty, who had carefully complied with all IRB policies. Despite refutation of the allegations, research data were destroyed and valuable study results lost.³

Course evaluations, if used for a publication, require IRB review and often informed consent. One strategy includes use of a cover sheet, attached to the evaluation, containing a recruitment script. The script will inform students that their answers may be used as part of a research project, that their participation is entirely voluntary, and that if they choose not to participate their grades will not be prejudiced. This process ensures that all medical students complete evaluations, often required by schools as well as essential for course improvement; data from students who opt out will not be used for research or outside presentations. Anonymity must be preserved as well. If demographic data are needed (eg, age), use of ranges rather than actual numbers will ensure no individual can be identified.¹⁰

Education Research Potentially Exempt From IRB Review

The granting of exempt status is always determined by the IRB, not the researcher. The essential elements of exempt research are that risks are minimal and subjects' identities are unknown (see box 3). The Code of Federal Regulations Governing the Protection of Human Subjects in Research⁶ states that research activities are exempt from regulations if the “research is conducted in established or commonly accepted educational settings, involving normal educational practices such as (i) research on regular and special education instructional strategies or (ii) research on the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods.” This definition of exempt research holds unless the data are both identifiable and potentially harmful if disclosed.

box 3 Criteria for Exempt Status

• Anonymous data OR minimal risk to participants

  Trainees must not be coerced

If data are obtained from individuals whose identities cannot be ascertained, the study is not considered human subjects research and thus is not subject to regulation. It is therefore exempt from review. In this category processes must be created that ensure the researcher cannot determine the identity of the participants. Assurances must be provided that the code linking the data to specific individuals will never be disclosed.⁵ Researchers can specify future processes to receive data and remove identifiers to allow future data to be exempt from IRB review as well.⁵

Decision trees for exemption categories are available at the OHRP website, which has a wealth of relevant information (http://research.fiu.edu/compliance/humanResearch/guidelineDocuments/humanSubjectsDecisionCharts.pdf).

Education Research Posing Minimal Risk: Expedited Reviews

Studies that pose minimal risk to participants are eligible for expedited review, usually by a single member of the IRB panel. Educational research that is not exempt usually qualifies for expedited review. Minor changes in already approved research also qualify for expedited review. Minimal risk means that the chance and severity of harm or discomfort anticipated in the research are not more than those encountered in daily life or from routine physical or psychological examinations (see box 4).⁶

box 4 Criteria for Expedited Review: Subjects at Minimal Risk

“The probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests.”⁶

Variability in IRBs

Each IRB is independent and uses individual criteria to judge issues of human safety. Studies have documented variability in review decisions.^1,8,11 Changes requested by an IRB may be minute yet must be done in order to proceed, and substantial delays are common.⁴ Education researchers report frustration with the required paperwork, multiple copies, prolonged delays, and other “hurdles” of the IRB oversight process.^3,12,13 Also, education researchers may have less assistance than biomedical researchers for creating and tracking IRB submission paperwork. Those IRBs lacking members with medical education expertise may not be familiar with education protocols, which also may not fit easily into biomedical‐focused IRB templates and language. The language of medical risk permeates the language of IRB templates and remains the default language of most forms, including consent forms, which often have weak relevance to trainees.¹

Multisite Research

Multisite studies typically entail IRB approval from each site involved in the study, although this is not actually required by the Code of Federal Regulations Governing the Protection of Human Subjects in Research.⁵ Each IRB may request minor changes in the protocol or consent document, which results in researchers making multiple applications before approval by all panels. A 2005 study examined the results of the same education proposal, which included an anonymous survey and focus groups of medical students, submitted to 6 medical school IRBs.¹² Four of the IRBs judged the study appropriate for expedited review, whereas 2 performed a full review. For the reviews by a single member, the time to decision ranged from 1–101 days. By 164 days after proposal submission, 1 IRB had not responded; as a result, the study was reduced from 6 to 5 schools participating. The 5 IRBs made 22 unique requests for additional information and 25 unique changes to the protocol. In addition to reducing the number of schools and students participating, the study had to be truncated due to the greatly delayed start and time‐sensitive nature of the survey. Most striking is that no IRB designated the anonymous survey portion of the project as exempt research.

Many discussions with education researchers demonstrate that delays of this type are not uncommon. Experts from the National Institutes of Health and OHRP advise that only 1 IRB be used for multisite studies and that IRBs from other participating institutions agree in writing to abide by a single IRB review.⁵ Diverting finite resources of time and money to the effort of multiple reviews, particularly of education research, which usually poses at most minimal risk, is “ethically troubling.”⁵

IRBs' Effects on Research

A significant problem for both clinical research and education research has been what experts term “mission creep” or “ethics drift,”,^1,2 in which IRBs are unable to clearly delineate and employ the exempt or expedited categories for work that is extremely low risk to human subjects. Even more concerning are reports that university IRBs have required proposal review and approval for routine academic activities, such as interviews performed by students for a class on investigative journalism,² and a national organization requiring IRB review for kindergarten science fair participants.¹ Experts question whether the driving force behind the noticeable expansion of IRB review since the late 1990s is due more to fears of losing federal funding than to true concerns regarding human abuses.¹ Given limited resources of researchers and universities, a rebalancing of resources is in order to “increase the likelihood that the cases most likely to have serious consequences will be most likely to receive the most thorough level of review.”²

Rather than hypothesizing every conceivable harm, IRBs could direct more resources toward research that represents higher risk. IRBs should look for “identifiable harm,” not every “imaginable harm.”² Since the 1990s, IRBs have grown enormously: at 1 university, from 2 full‐time staff to 26, a single review panel to 6, and a 2‐page protocol template to 15 pages.¹ Yet administrative inefficiency has also continued to expand with major increases in time to approval, even for low‐risk protocols. In addition, faculty are more reluctant to serve as members of IRB panels due to heavy workload, numerous complex and ill‐defined rules, and the mixture of “gratitude and vilification” that IRB members face from faculty communities.¹

Negative effects of “mission creep” include research dropped altogether, major portions removed, diversion of research topic or population to one more likely to pass easily through IRB review, choosing new research themes according to the likelihood of swift IRB approval above inherent importance of the research itself, and choosing methods, such as meta‐analysis, rather than new data collection to avoid IRB review.¹ These are examples in which researchers shied from topics not due to fear of harm to subjects but rather to avoid delays and excessive interference from IRB panels. Time‐limited research, such as student summer projects or time‐restricted funding, is at particular risk from IRB delays and may discourage trainees from working outside of previously approved projects or existing data sets.

Anecdotal and other reports demonstrate that IRB members can identify risks to subjects that researchers have missed. However, to date there are no valid evaluations of the United States' IRB system that demonstrate whether IRB review has successfully protected subjects and, if so, which aspects of the IRB process proved most valuable. Because no other research regulatory system is similar in scope or process to that of the US system, valid comparisons are not feasible.¹

National Consensus on Education Research

Most researchers agree that the US IRB system is in crisis, due to an imbalance between measures to avoid OHRP attention and litigation and the goals of identifying new ethical questions and risks to subjects.^1,4,14 Many groups have called for change, particularly for education research, as well as a national consensus statement on the IRB's role in medical education research.^8,15 The Editorial Board of the Journal concurs with the need for a consensus statement from relevant stakeholders. Box 5 lists recommendations commonly made regarding IRB oversight of medical education research.

box 5 Suggestions for Medical Education Research Review

• IRB has separate education-focused IRB panel¹⁵ (large institutions) or access to an IRB member, full member, or consultant, with education research expertise (smaller institutions)

  Guidelines or decision trees are created to assist IRB members and researchers in determining level of review for education research¹⁵

  IRBs refrain from routinely requesting information that is relevant for clinical but not education research¹²

  Standard electronic application format for all US medical education research¹²

  Standardization of consent letters¹²

  IRBs agree to a single IRB review process for multisite projects16

  Development of central or regional IRBs to facilitate mul‐tiinstitutional trials in medical education¹¹

JGME Policy

The Journal of Graduate Medical Education requires all submitted research manuscripts to include a statement regarding IRB exemption or approval, unless human subjects are not studied (ie, reviews, meta‐analyses, and descriptions of educational materials without evaluation). This policy applies to the United States and countries with similar regulations; papers from countries with different ethical oversight approaches will be reviewed according to the accepted approaches of those countries. For concerns or questions, authors are encouraged to contact JGME or the Editorial Board for assistance (jgme@acgme.org).