Exception from Informed Consent: How IRB Reviewers Assess Community Consultation and Public Disclosure

Abstract

Exception from Informed Consent (EFIC) regulations detail specific circumstances in which Institutional Review Boards (IRB) can approve studies where obtaining informed consent is not possible prior to subject enrollment.

To better understand how IRB members evaluate Community Consultation (CC) and Public Disclosure (PD) processes and results, semi-structured interviews of EFIC-experienced IRB members were conducted and analyzed using thematic analysis.

Interviews with 11 IRB members revealed similar approaches to reviewing EFIC studies. Most use summaries of CC activities to determine community members’ attitudes; none reported using specific criteria nor recalled any CC reviews that resulted in modifications to or denials of EFIC studies. Most interviewees thought metrics based on Community VOICES’s domains (feasibility, participant selection, quality of communication, community perceptions, investigator/IRB perceptions) would be helpful.

IRB members had similar experiences and concerns about reviewing EFIC studies. Development of metrics to assess CC processes may be useful to IRBs reviewing EFIC studies.

Introduction

In 1996, the Food and Drug Administration (FDA) released regulations (21 CFR 50.24) regarding human subjects research to allow an “Exception from Informed Consent” (EFIC) (Administration 2016; Baren et al. 1999; “Electronic Code of Federal Regulations” 2017). The EFIC regulations detail specific circumstances under which research on emergency conditions can be conducted without obtaining patient or surrogate consent prior to study enrollment. EFIC research presents unique obligations and challenges to Institutional Review Boards (IRBs) as well as to investigators (US Dept of HHS et al. 2013). The regulations describe several additional obligations that must be completed prior to executing an EFIC trial, including the processes of “community consultation (CC)” and “public disclosure (PD)” (21 CFR 50.24a.6 & 7; “Electronic Code of Federal Regulations” 2017). Operationalizing these processes and understanding how they should be implemented to show successful execution have proven to be quite challenging (Smirnoff et al. 2018; Ragin et al. 2008; Richardson et al. 2005).

CC requires two-way communication between community members and investigators or IRB representatives (Baren et al. 1999). Researchers must identify the appropriate communities to be consulted based upon both the condition being studied and the study’s geographic catchment area. Researchers must identify and engage relevant community stakeholders in discussions about the proposed study, including study purpose, the risks and benefits of participation, and the reasons why EFIC is needed. CCs should elicit any concerns or objections that the community may have about the proposed EFIC study, however it is important to note that the regulations require CC, not community consent. The IRB may choose to participate in these CC sessions or may receive a report on the CC activities. CC occurs before or during the IRB review and the results are considered by the IRB in determining whether or not to approve the study under EFIC (Baren et al. 1999).

PD differs from CC in that it is a dissemination of information to the public, rather than a dialogue with community members. Prior to study launch, investigators must announce to the public that an EFIC study is being conducted and give information about how to “opt-out” of the study. Once the study is completed, post-study PD shares the study results and outcomes with the public.

Federal regulations provide limited details about methods to meet the CC and PD requirements, and they do not set any standardized metrics to assess successful compliance with these requirements (U.S. Dept of HHS et al. 2013). The determination of successful completion of both CC and PD is up to the IRB(s) that grants the exception and is responsible for overseeing the proposed research. Little is known about how IRBs fulfill their role in evaluating EFIC research proposals and many have called for additional research on how CC influences IRB decision making (Baren et al. 1999). The primary purpose of this investigation is to develop a better understanding of how IRB members, with expertise in EFIC, evaluate the processes of CC and PD, and how they use the results of CCs when reviewing EFIC studies.

Materials and Methods

This was a qualitative investigation that used purposive sampling. As part of Community VOICES 3 (Views On Informed Consent in Emergency Situations), the VOICES 3 team solicited IRB members’ perspectives related to EFIC research at six academic centers in the United States between 2013-2017. VOICES 1 and 2 Site PIs submitted the names of IRB members at their institution who were willing to be approached by the lead VOICES 3 team. IRB members were eligible for the study only if they had personally participated in the initial review of one or more EFIC studies as part of their IRB responsibilities. Consenting IRB members from the VOICES 1 and 2 institutions were interviewed about their experiences reviewing and overseeing EFIC studies, using a semi-structured interview guide (see Appendix for complete interview instrument). Data analysis began during the interviewing process, and the study closed when thematic saturation was achieved, i.e., new interviews no longer uncovered additional themes.

The Institutional Review Board of the Icahn School of Medicine at Mount Sinai approved the study with a waiver of written consent to protect the identity of IRB participants. Several of the IRB members who were approached to participate in this study sought approval from their own IRBs before agreeing to participate. IRB participants were eligible to receive $100 for participation, though some declined to accept this remuneration.

The interview guide was drafted by the VOICES 3 investigative team based upon existing literature, several federally mandated IRB responsibilities, and a conceptual model for evaluation of CC (see Figure 1) that was developed based upon the findings from Community VOICES 1 and 2 studies. The semi-structured interview guide was designed to elicit information about: how IRBs review the CC plan; whether or not IRB members attended CC meetings; how the CC results are reported to the IRB; proposed methods for PD; oversight of the content of PD materials; preferred “opt-out” methods; and issues related to post-enrollment notification. Most questions were open-ended with probes and follow-up questions to elicit details. If not spontaneously mentioned, respondents were specifically asked about the utility of the five domains for evaluation of CC that were developed by the Community VOICES 2 Study: feasibility, selection of participants, quality of communication, community perceptions, and investigator/IRB perceptions. The interview guide was appraised for face validity by the VOICES 3 investigative team and pilot tested prior to being finalized. The interviews were conducted via phone by a seasoned physician-researcher with IRB experience (DN), who was not personally acquainted with any of the respondents. The interviews were audio-recorded, transcribed, and blinded prior to content analysis.

Figure 1.

Conceptual Model: VOICES Domains and Metrics for Evaluation of Community Consultation

To examine the data, members of the investigative team (MCS, DN) cooperatively developed codes, or labels, to apply to similar lines of text so that responses could be grouped and categorized. A set of a priori codes regarding themes related to CC and PD and the VOICES CC domains described in Figure 1. A codebook was developed with definitions of when to use the codes and an example of application. Codes were applied to the transcripts using Atlas.ti (version 7, 2012-2016). Two investigators (MCS, DN) independently coded a randomly selected transcript with the a priori codes and developed additional relevant codes based upon the data. The new codes, with their accompanying definitions and examples, were added to the codebook. After independently coding a different transcript, MCS and DN collaboratively reviewed their coding for agreement and revised the codebook for clarity (constant comparison method). LDR, MRD, JNBC, and CC then reviewed the expanded codebook. Coding continued with each transcript being independently coded by at least two of the authors and additions to the codebook being reviewed by the analytic team as a whole (LDR, MRD, JNBC, CC, IW). Discrepancies of code application were reconciled collaboratively. No new codes were identified after the eighth respondent, but an additional three interviews were conducted to assure saturation had been achieved. MCS examined the frequency of code use, co-occurrence of codes in the same text, and developed a description of the IRB process and a draft of the main themes. MRD, JNBC, LDR, CC, and IW reviewed the coded transcripts to confirm theme significance. All authors reviewed the draft manuscript and made substantive contributions to the final manuscript.

Results

Interviews were conducted with a total of eleven IRB members, from four participating IRBs; all but one had reviewed at least two EFIC studies (one participant had reviewed only one). Many of the respondents had extensive experience with EFIC studies: “I have been on the IRB probably the last 15 years and…I’ve personally been involved in probably between 5 to 10 [EFIC studies]…We get about 1 to 2 per year on my committee” (R10). The length of service on the IRB ranged from three to fifteen years; five of the respondents were current or past IRB chairs.

Overview of IRB EFIC Review Process

All participating IRB members reported similar approaches to the IRB review of EFIC studies (see Figure 2): Investigators are expected to submit a complete research protocol, a detailed CC plan, and a PD plan. If the IRB determines that the study has scientific merit and meets the criteria for an exception from informed consent, and if the IRB approves the CC plan, then the EFIC investigators are permitted to conduct CCs. The results of the CCs are reported to the IRB, which then determines whether or not the EFIC approval is granted. If the exception is granted, the study moves forward. Pre-study PD must occur prior to study launch; it usually occurs after IRB approval of the study, though sometimes it is done concurrently with CC.

Figure 2.

Flowchart of Described EFIC Study Process

How IRBs Review Community Consultation Plans

An abridged data display (see Table 1) demonstrates how the five domains related to codes about CCs, summarizes related findings described below, and shares illustrative quotes. In examining the CC plans before implementation, none of this study’s IRB participants expressed a strong preference for one format of CC over another and all reported that most CC plans they have reviewed use a variety of approaches. Common methods included dedicated town hall-like meetings, condition-specific advocacy or support groups, community fair tabling, focus groups, interviews/surveys with emergency department patients, and joining standing community meetings. Importantly, one participant pointed out that most study resources go to the research study itself, and not the CC which must precede the study: “…Most of these studies do not include significant budgeting for community outreach…[but] it requires resources to really tap into the feelings of the community about studies…” (R6). None of the participants recalled requesting or receiving specific information on how much fiscal support was needed for the CC plan to be feasible for implementation.

Table 1.

Abridged Data Display: Community Consultation Codes and Illustrative Examples of Application

VOICES Domain	Code and Definition	Summary of Findings	Illustrative Example(s)
Feasibility	Review of CC Plan The process of review as described by the IRB	IRB reviews the CC plan before it is implemented and review the results of CC before deciding on the whether to grant the EFIC as summarized in Figure 2. Each CC plan is unique to the specific study. There are no specific quantitative requirements.	“It would be difficult to say we’re looking at a specific thing or for a specific thing because every scientific project is different and so…you have to understand the science in order to know is the plan appropriate” (R4).
Feasibility	Feasibility The resources required to perform the planned community consultation.	The achievability of the CC plan is minimally assessed by the IRB, and given practicability is not assessed in a finite way.	Most of these studies do not include significant budgeting for community outreach…[but] it requires resources to really tap into the feelings of the community about studies…” (R6).
Selection of Participants	Format of CC Activities Implementation of CC by the investigators	They rely heavily on the experience of the investigators to develop a good CC plan, and some IRBs refer new investigators to more experienced ones for guidance.	“Our emergency department here is very experienced with these studies and so their community [consultation] plans are always very extensive, really well thought out.” (R4)
Selection of Participants	Composition of CC The people who participate in the community consultation; potential study participants who are community members.	IRBs expect investigators to have community consultations with people who are at highest risk of or more likely to be enrolled in their EFIC study.	“If it includes patients that are of a certain ethnic group, then we’re going to make sure those community consultations take place where those ethnic groups are well represented.” (R1)
Selection of Participants	Quantification The number of people who participate in the community consultation.	While IRBs review how many participated in an EFIC’s community consultation, there is no minimum threshold that must be met for the CC to be considered valid.	“I think it is very difficult….feeling that you had good communication…that you reached out to the right population, enough of the population’ (R2) “I don’t think we’ve ever actually discussed, like a minimum number, because…our approach is we trust the investigators who know their population better than we do…” (R4)
Quality of Communication	Content The information that is shared with community members, by the study team, at the community consultation.	IRBs review the content of the community consultation but have no requirements about what content is shared or discussed.	“…we wanted more focus on the waiver of consent part of the study. There may have been a lot of discussion of the science but not a lot of discussion about what it means to be enrolled in this trial without your consent.” (R11)
Community Perception of Process	CC participant Comprehension The ability of the CC participants to grasp the purpose of the proposed study and why an EFIC is needed.	CC attendee comprehension of the proposed study, and why an EFIC is necessary, is not a required component of the CC report that the IRB reviews.	“…you can get a gut feeling pretty quickly whether or not the attendees are getting it based on their questions and comments.” (R1)
Community Perception of Process	Community Member Perceptions The attitude of CC participants about the proposed study.	IRBs examine the attitude of CC participants about the proposed study, but rely on gestalt to determine if the participants find the EFIC study in question acceptable.	“…the vast majority of the community is supporting this proposal and if the vast majority of the community didn’t support the proposal, we would certainly be concerned.” (R5)
Investigator/IRB Perception/Response	IRB / Investigator Response The impact of the community consultations on the protocol or on the IRBs approval, disapproval, or study modifications.	On the whole, IRB reviews of CCs have not resulted in modifications to EFIC study protocols, nor outright rejection.	“…we haven’t ever seen…that there has been such a strong pattern of objective that either in the qualitative or, potentially more importantly, in the quantitative component that we’ve asked for anything…” (R4)

Members of one IRB reported having a generic template for investigators to share their CC and PD plans (R4 & R7). One of these respondents reported that they also always require an IRB member to be present at each CC session (R7), but other IRBs do not have a template, do not require the presence of an IRB member at each CC, and most IRB reviewers do not observe the CCs. As one participant said, “…In the last three or four years I’m not aware of IRB members going and observing [CCs]. However, in the CC plan it’s always designated that an IRB representative could be there” (R9).

IRB members reported using a gestalt assessment to determine if the CC plans are feasible and appropriate. They consider “historical data” or what CC plans have “worked in the past” to guide the consideration of a new CC plan (R8). Several IRB participants stated that they rely on the experience of the investigators to develop a good CC plan. “Our emergency department here is very experienced with these studies and so their community [consultation] plans are always very extensive, really well thought out. …We trust the investigators who know their population better than we do” (R4). Sometimes IRBs even refer new EFIC investigators to more experienced ones for guidance in developing their CC plan: “:…A lot of times we’ll have to consult with [name redacted] since she had a lot of experience here at our institution with various studies that involve exception from informed consent and kind of team up with her and glean from her experience” (R9). On the other hand, one respondent did not feel that investigators see CC as an opportunity to meaningfully engage the community about the research: “…I think researchers just [want] to be done with [CCs] and get on with the research…” (R6).

How IRBs Interpret Community Consultation Results

No participants reported their IRB having a quantification requirement for CCs, either in number of consultation sessions carried out or number of community members that must be present (all respondents). One participant said, “I don’t think we’ve ever actually discussed, like a minimum number…” (R4). EFIC investigators report, either in person or in written format, on how the study was viewed by community members, but no institution had quantifiable specifications for how many or what proportion of community members must favor or positively view a proposed EFIC study for it to be approved by the IRB (all respondents). Again, IRBs rely not on specific quantifications but the general concept of apparent acceptability: “…the vast majority of the community is supporting this proposal and if the vast majority of the community didn’t support the proposal, we would certainly be concerned” (R5).

All IRB members reported requiring EFIC investigators to demonstrate that CCs include community members representative of those who would potentially be enrolled in the study. “If [the proposed EFIC study] includes patients that are of a certain ethnic group, then we’re going to make sure those [CCs] take place where those ethnic groups are well represented” (R1). While there is no standard formatting for the CC reports, all IRB reviewers described an expectation of a thorough account of the CCs that includes the number and demographics of the attendees, and a description of community objections and concerns about the proposed EFIC study. “…The study team [is] required to submit a summary or a report of the CC findings along with any proposed revisions to the protocol documents based on…the outcome of the CC…” (R11).

The IRB participants reported that, in reviewing the CC reports, no formal assessment is expected of the CC attendees’ comprehension of the proposed study. One participant shared, “…you can get a gut feeling pretty quickly whether or not the attendees are getting it based on their questions and comments” (R1). When IRBs review the content of the CC discussions, they are specifically concerned that the CC participants reflect on how it feels to be enrolled in a study without the researchers obtaining consent: “We wanted more focus on the waiver of consent part of the study. There may have been a lot of discussion of the science but not a lot of discussion about what it means to be enrolled in this trial without your consent” (R11).

Rather than examining specific objections or concerns that could result in modification of the proposed EFIC investigation’s protocol, IRBs use the summaries of the CCs to determine whether communities accept that the proposed EFIC study should be conducted: “…If they went to a group and talked to 100 people and one person said ‘no’ we don’t ask them to report that one person to the IRB…we just look at the numbers and it is more like a gestalt if there’s overwhelming approval of the study by the people who were consulted” (R10). However, the same participant went on to say: “…If it’s a small number of people who were consulted [and] a significant fraction who had questions about it, we would probably go back to the investigator and ask what they were going to do about that” (R10). Most IRB members’ comments about the sense of community approval or disapproval of the proposed EFIC research implied a conception of CC as a mechanism of gauging community support for the study to move forward. Contrary to the explicit regulatory intent, one IRB member seemed to feel that CCs should allow for the community to reject a proposed EFIC study, rather than provide feedback to the investigators and IRB: “The tricky piece about a CC plan is that I’m not sure that they’re really set up to have the community be a veto” (R2).

The IRB participants commented on the wide diversity of EFIC studies with respect to study populations and clinical questions posed. “It would be difficult to say we’re looking at a specific thing or for a specific thing because every scientific project is different and so…you have to understand the science in order to know is the plan appropriate” (R4). Many IRB members reported grappling with how to assess reports about the CCs. One participant shared: “I think it is very difficult….feeling that you had good communication…that you reached out to the right population, enough of the population, that you came back with a take home message that had any effect on how you were going to conduct the study” (R3). Another said, “I wish I could tell you that there was some sort of checklist that says, ‘If you do 10 consultations, that’s enough,’ or ‘50 consultations, that’s enough.’ I don’t think that having something as rigid as that kind of guideline is going to be meaningful, but that means like so many other IRB judgments that the decision…is a bit subjective” (R2).

Because of the need for flexibility in the guidelines for EFIC study assessment, the federal regulations contain no predetermined definition of “community” nor standardized criteria for assessing the effectiveness of CC. Still, IRB participants reported grappling with how to assess reports about CCs and several expressed a desire for an evaluation tool to guide their decision. In fact, one participant simply stated, “It would be nice to have a rubric” (R9). When specifically asked about the five domains for evaluation of CC developed based on the Community VOICES 1 and 2 studies (see Figure 1), all agreed that these were relevant domains and most thought that metrics based on these domains would be helpful.

Impact of IRB Assessments of Community Consultations

Respondents reported that in their experience, IRB reviews of CCs have not resulted in modifications to EFIC study protocols, nor denial of the exception. One participant stated, “…We haven’t ever seen…that there has been such a strong pattern of objection that either in the qualitative or, potentially more importantly, in the quantitative component that we’ve asked for any changes…” (R4). Another said, “I have never personally seen CC results come back to us and the board felt that it wasn’t sufficient” (R9). Importantly, one reviewer pointed out that when a site is a part of a multi-site study, the local IRB feels less empowered to call for study modifications based upon local CCs: “…We recognized, in this case, that this is part of a national consortium study. We made suggestions but really did not have an expectation that the suggestions would be incorporated in protocol change” (R6).

Public Disclosure Plans

All IRB respondents reported similar methods of reviewing PD plans and content, but, some reported reviewing the PD plan in conjunction with the CC plan. That is to say, at the same time some IRBs consider the CC plan, they also appraise the PD plan. IRB participants who reported that CC and PD plans are jointly considered sometimes conflated the two in their responses. For example, when asked specifically about successful CC plans one IRB member said:

As far as the community consultation process, we do recommend that they visit with the city council, that they visit with our hospital boards, and also that they attend various community outreach programs….We also recommend that they have a public website where people can go to get more information….Also news media releases…and some type of media plan to cover the area that they would have potential subjects come from. (R9)

Here the respondent describes visits that would qualify as CCs, but the use of a website, news releases, and a media plan are PD plan components. Some of the IRBs that reported simultaneous CC and PD plan consideration also reported both plans being implemented at the same time,

They would let us know about any of the hospital committee meetings that they’ve attended, any of the city council meetings that they’ve attended, any public outreach events that they’ve attended. This would include feedback from the community, any surveys that they might have done…they would let us know about any public notifications that they did so that the dates that any news spots were run or any radio ads were done… (R10)

IRB participants reported expecting PD plans to describe broad dissemination and sometimes making recommendations about using more accessible language. No IRB respondents described using specific guidelines to evaluate PD plans or reports, and several participants again mentioned using gestalt to determine the likely effectiveness of the PD plan.

Limitations

This was a small, qualitative investigation about a minimally studied but important topic. Only 11 IRB reviewers from four IRBs participated in this study. Their responses may not represent all EFIC-expert IRB reviewers or EFIC review processes. We also may not have captured the experiences or views of other IRB reviewers who did not want to participate in the study. The findings of this study highlight experiences from IRBs well-versed in EFIC study reviews (of which there are not many) and are meant to describe the experiences of experts in the field.

Additionally, the investigation may suffer from recall bias: the interview relied upon the memory of IRB reviewers, some of whom had not recently reviewed an EFIC study. However, the likelihood of poor recall negatively impacting the quality of the findings is low, as reviewer participants from the same IRBs reported similar information. Moreover, although the sample size is small, the study reached thematic saturation, with the final three interviews not revealing new information.

Discussion

Community Consultations as Community Voice

CC is not meant to replace the informed consent process of traditional research endeavors; it is a practice that allows a group of people, representative of the at-risk (of study inclusion) population and the geographic community, to express their thoughts and voice concerns that investigators may not have considered in the design of the EFIC study (Biros 2013). CC is not meant to be a community-level surrogate to replace the traditional informed consent of the individual. However, in practical application (and as noted in the Results sections entitled, “How IRBs Interpret Community Consultation Results,” and “Impact of IRB Assessments of Community Consultations”), it appears that IRBs represented by our respondents operationalize CC as more of a general referendum about an EFIC study, and not necessarily an opportunity to incorporate community feedback about the specific study into the research plan. It may be that lacking clear evaluation specifications about CC processes does not allow IRBs to properly appreciate non-majority community concerns. The difference in intended purpose and the reality is nuanced but important as it affects the ultimate impact of CCs on IRB consideration of EFIC studies. The CC goal should not be to seek majority “approval” of a proposed study, but rather to provide a forum for community members to raise concerns that the study team and IRB may wish to consider before the study moves forward. IRBs should engage in a critical appraisal of concerns of merit, even if they are not shared by a large number of CC participants. Most IRB members interviewed seemed committed to giving community members an authentic voice while recognizing the difficulty in having such concerns actually impact the proposed EFIC study, particularly in large multi-center trials. Over the course of 11 interviews with IRB reviewers from four IRBs, none described a study in which the discussion of the CC resulted in the IRB or the investigators modifying or rejecting the EFIC study proposed, though their responses did allow for such a possibility. In practice, assessment for a “significant number” of concerns is more common than consideration of relevant but minority objections or trepidations, or of a salient concern voiced by a single individual.

Some IRB participants explained that their IRB reviews the plans for the CCs and the PD simultaneously. According to federal regulations, the CC must come before the EFIC is issued by the IRB, and PD must occur before study launch. Nevertheless, PD was reported as sometimes starting concurrently with CC. The purpose of PD is to inform the geographic community at large about the planned occurrence of the EFIC study, and does not involve soliciting meaningful community member engagement. While IRB participants from IRBs that simultaneously consider CC and PD plans agreed with the technical definitions of CC and PD, during interviews there seemed to be some conflation of the two processes; perhaps this occurred because of simultaneous consideration of the two plans, regardless of which happened first. In these IRBs, it seems the CC is often operationalized as an opportunity for EFIC investigators to inform community members about the study, rather than to seek feedback from community members about the proposed EFIC investigation. It is, of course, necessary to provide CC participants with information about the study and about the EFIC regulations, but it is important to identify the purpose of CC meetings as not being primarily educational. Rather CCs aim to generate a meaningful dialogue with community members about the proposed EFIC study.

Lacking More Specific Guidelines, Needing Flexibility

Recognizing that EFIC investigations generally do not have abundant resources allocated to CC planning and performance, and that CC must take place before the study begins, it is clear that a “good” CC plan can be difficult to achieve, especially for novice EFIC investigators. Absent more specific guidelines on CC planning, IRBs rely heavily on the experience of senior EFIC investigators to guide plans for CCs at their site. Without a rubric for a high-quality CC plan, one IRB has novice EFIC investigators meet with an experienced EFIC investigator before submitting their CC plan to the IRB. These IRBs have admittedly come to trust their seasoned investigators. Another IRB requires at least one IRB reviewer to be present at each CC. Other IRBs report they may also recommend the inclusion of certain community organizations or the use of particular CC formats as part of the CC plan. In this way, the IRBs felt more confident that the “right” community was consulted on the EFIC study proposed. Still, IRBs lack standards to systematically determine feasibility and appropriateness of CC plans; it is not clear how they assure that planned CCs achieve truly meaningful community engagement to facilitate communities’ critical consideration and elicit relevant feedback.

The above innovations in CC plan review do not help the IRBs critically appraise the CCs after they occur. The IRB reviewers interviewed have a keen understanding of the EFIC review process at their sites, yet still, many found it difficult to describe the exact methods of CC evaluation. As reported in another study, these IRB participants reported they rely heavily on a “gut” feeling or gestalt (Dickert et al. 2014). In the same way that qualitative research has specific guidelines to assure rigor, EFIC studies need a clear rubric on how high-quality CCs should be critically appraised by IRB reviewers. Robust CC processes are needed to ultimately examine the appropriateness of granting an EFIC approval to a study, or to inform modifications to the proposed protocol. Based upon the findings of this investigation, we recommend IRBs consider the use of the CC evaluation toolkit found in the Appendix. Based on the findings from Community VOICES 1 and 2 studies (Smirnoff et al. 2018; Ragin et al. 2008; Richardson et al. 2005), VOICES 3 developed these instruments to help EFIC investigators plan for feasible, productive CCs, and to assist IRB reviewers in assessing the results of those CCs. Future research should explore the utility of the VOICES tools and assess the impact of their use on CC planning, resource allocation for CCs, CC evaluation, and CC impact on EFIC studies.

With clear and adaptable standards, EFIC researchers may be better able to develop relevant, achievable, and distinct plans for: EFIC study CCs that result in the acquisition of worthwhile information from the community; and PD plans that disseminate clear and understandable information to the community. While maintaining the necessary flexibility that EFIC studies demand, more robust assessment of CCs may enable EFIC researchers and IRBs to hear people’s voices on the scientific investigations that affect their community, and better operationalize federal regulations and guidelines.

Conclusions

IRB members from different sites had similar experiences of, concerns about, and difficulties with reviewing EFIC studies. In particular, IRB members reported largely using gestalt to ascertain whether conducted CCs were sufficient, and that no CCs impacted EFIC study design or whether the EFIC was granted. Development of standardized metrics to assess CC processes may be useful to IRBs in reviewing EFIC studies.