Residents as Research Subjects: Balancing Resident Education and Contribution to Advancing Educational Innovations

Abstract

Background

Research in education advances knowledge and improves learning, but the literature does not define how to protect residents' rights as subjects in studies or how to limit the impact of their participation on their clinical training.

Objective

We aimed to develop a consensual framework on how to include residents as participants in education research, with the dual goal of protecting their rights and promoting their contributions to research.

Methods

A nominal group technique approach was used to structure 3 iterative meetings held with the pre-existing residency training program committee and 7 invited experts between September 2018 and April 2019. Thematic text analysis was conducted to prepare a final report, including recommendations.

Results

Five themes, each with recommendations, were identified: (1) Freedom of participation: participation, non-participation, or withdrawal from a study should not interfere with teacher-learner relationship (recommendation: improve recruitment and consent forms); (2) Avoidance of over-solicitation (recommendation: limit the number of ongoing studies); (3) Management of time dedicated to participation in research (recommendations: schedule and proportion of time for study participation); (4) Emotional safety (recommendation: requirement for debriefing and confidential counseling); and (5) Educational safety: data collected during a study should not influence clinical assessment of the resident (recommendation: principal investigator should not be involved in the evaluation process of learners in clinical rotation).

Conclusions

Our nominal group technique approach resulted in raising 5 specific issues about freedom of participation of residents in research in medical education, over-solicitation, time dedicated to research, emotional safety, and educational safety.

Objectives

The purpose of this study was to develop a consensual framework for the inclusion of residents as subjects in medical education research.

Findings

Five themes and their related recommendations were identified: freedom of participation, avoidance of over-solicitation, management of time dedicated to participation in research, emotional safety, and educational safety.

Limitations

More residents could have been involved in the process to gather other perspectives and issues and align recommendations consequently; this study was conducted in a single program and in a single center; and our findings might only reflect local and cultural specificities.

Bottom Line

This study provides a framework supporting medical education researchers in involving residents as participants in studies, with the dual goal of fostering meaningful and safe training experiences, while promoting their contribution to the research.

Introduction

Research in health professions education is gaining importance in academic medicine.^1,2 Literature regarding the well-being of learners and ethical aspects of the learning environment in hospitals is expanding.^3-14 Some authors have framed the concept as educational safety and included it in the psychological safety construct in clinical learning environments. However, this has not been intended for considerations about learners as participants in research projects.^14-16

Being a study participant can be time-consuming, and medical training programs are already demanding. There is a need to find balance between resident education and research to improve the quality of training and create new knowledge in the field.^17-21 The literature regarding the recruitment and participation of medical students and residents in research projects as subjects is still emerging and strategies for recruitment of learners in research are described.^22-26 However, uncertainties regarding ethical considerations, such as freedom to participate and exemption from institutional review board (IRB) approval for educational projects, remain frequent.^8,27-31 According to a recent study, the majority of medical students who had participated in at least one project thought they could not decline recruitment (64%) and felt their participation would help their academic grade (74%).³² Another study showed that medical students deem their participation in research a professional responsibility. Nevertheless, their perception of having time available to invest as participants, and risks, mainly related to coercion and confidentiality, remain important factors impacting their decision.³³

To ensure the protection of trainees' consent and to better understand their perspectives on participation in medical education research, some authors suggest careful review of recruitment procedures in multidimensional approaches involving relevant stakeholders.^26,32,33 Furthermore, improving recruitment is not only considered beneficial to learners, but also to researchers, who face growing challenges related to low response rates and participant retention rates for their projects.^25,26,33,34 However, a well-framed approach to guide medical education research involving residents is not currently available, nor are there comprehensive guidelines to protect residents' rights more systematically when they are research subjects.

Our research aim was to develop a consensual framework for the inclusion of residents as subjects in medical education research, with the dual goal of fostering a meaningful and safe training experience, while promoting their contribution to the research. We used a qualitative constructivist approach that allowed us to anchor the project on questions raised in recent studies regarding participation in medical education research while stimulating exploration of the problems raised in clinical interactions or by participants.

Methods

Setting and Population

This study took place within the general pediatrics residency program of the University of Montreal. The program trains 35 residents at CHU Sainte-Justine, a pediatric university hospital in Canada.³⁵

Study Design and Procedure

We developed a procedure based on consensus group methods.^36,37 When faced with incomplete knowledge and uncertainty, different types of consensus group methods can be used in medical education to synthetize opinions and enhance decision-making about curriculum, assessment, definitions of competencies, and educational resources. One of them is the nominal group technique,^37-39 which is a recognized approach in anthropology, ethnography, and qualitative sociology, and it is often used in medical research.⁴⁰ This is also based on the notions of contribution value and information power of qualitative sampling.^41-44 Although nominal group technique is often associated with mixed methods, we used a qualitative exploratory design, comparable to other recent studies.^45-47 It is similar to the Delphi method, with the distinction that it is usually structured in face-to-face interactions involving 5 to 12 participants. It does not require a complete literature review, a questionnaire, or a list of indicators to be rated with predetermined criteria. Instead, it is based on a general problem presented in the form of a question. This nominal question is what participants work from to share ideas and build knowledge throughout the process.^37,48 We used this technique to explore stakeholders' perspectives regarding residents' participation in research as subjects, using the following nominal question: “What are the issues related to residents' participation in research as subjects and how can we prevent and solve problems associated to recruitment and participation in medical education research projects?” We combined idea generation and problem-solving intervention into a process that was integrated into planned meetings.^47,49

We used nominal expert sampling to identify subjects with either experience as residents or with leadership roles in medical education. Two groups of collaborators were solicited. The first included all 20 members of the residency training program committee: 7 residents elected by their peers (3 chief residents and 1 representative for each of the 4 years of training) and the directors of the residency program. The committee also includes faculty responsible for different aspects of the training, research curriculum director, evaluation committee director, faculty in charge of neonatology and pediatric intensive care rotations, director of academic half-day curriculum, director of outpatient clinic, director of the OSCE assessment, director of the competency-based medical education reform, director of the subspeciality match, and faculty from remote clinical teaching facilities (Table 1). At the time of the project, 2 authors were conducting educational research (A.M., L.P.T.). To introduce additional perspectives, a second group of 7 invited experts selected in agreement with the residency training program committee were solicited (director of the medical education research center, CHU Sainte-Justine medical education director, chair of the pediatric residency assessment committee, clinical researcher, expert in clinical ethics, chair of the CHU Sainte-Justine IRB, and a chief resident). Details on the collaborators are summarized in Table 1.

Table 1.

Steps of the Process of the Adapted Nominal Group Technique

Steps	Description	Involved Stakeholders	Main Outcomes
1. Initial meeting within a monthly residency training program committee (09/25/2018, CHU Sainte-Justine, duration: 1h)	Identification, definition, and summary of central issues37,49 Sharing of individuals' ideas in a round-robin format Systematic recording of verbal exchanges to regroup ideas and provide clarification and precision, led by facilitator (L.P.T.)	2 program directors of the pediatrics residency training program Chair of the pediatric university department 4 residents (1 in each training level from PGY-1 to PGY-4) 3 chief residents of the pediatrics residency training program 10 other faculty members responsible for research, evaluation committee, neonatology, and intensive care rotations, classes, longitudinal clinic, CBME reform, OSCE evaluations, sur- and subspecialties match process, and representatives from remote clinical teaching facilities	Consensus on a list of 15 issues to explore Synthesis prepared for team members working on information gathering and literature review
2. Review of literature	Outlining of issues that have already been explored regarding trainees as research subjects and what tools have been proposed to solve the various issues	Authors of the article	Updated internal knowledge-based document (grounded in empirical experience and literature) Synthesis of results from first meeting and review of literature for experts participating to the second meeting
3. Second meeting with experts related to central issues to be discussed (11/15/2018, CHU Sainte-Justine, duration: 1.5h)	Simplification and organization of issues raised during initial meeting, separating issues, and recommendations Revision and critical analysis of issues and recommendations was done individually, after the meeting. Comments were then sent by separate emails.	Director of the medical education research center of the University of Montreal Director of medical education, CHU Sainte-Justine Chair of the pediatric residency evaluation committee 1 faculty member who is a clinical researcher 1 expert in clinical ethics President of the IRB at CHU Sainte-Justine 1 chief resident	Consensual strategic document containing 5 issues and their related recommendations Synthesis of experts' definition of issues and recommendations to present for the final meeting
4. Final meeting within a monthly residency training program committee (04/30/2019, CHU Sainte-Justine, duration: 1h)	Presentation of the proposed final version of the report Discussion, clarifications, and approval of the document	Same collaborators as for Step 1 (initial meeting)	Consensual final version of the document Final version and descriptive synthesis of the process to present to the direction of the program
5. Approval	Final approval of the document	Pediatric residency training program direction	No further modifications were made to the proposed final version before approval (04/30/2019)

Abbreviations: PGY, postgraduate year; CBME, competency-based medical education; OSCE, objective structured clinical examination; IRB, Institutional Review Board.

To ensure the rigor of the process, we used a published list of recommendations for consensus group methods to guide our procedures. We adapted the nominal group technique meetings based on a widely known model used to create a knowledge-based practice with experts.^37,50 The framework of that study is detailed in the Table in the online supplementary data. Three meetings were planned. While this methodology allows for additional meetings, this was not necessary, as consensus was reached after the 3 meetings.

The first meeting, tasked with identifying broad ideas regarding the nominal question, included the members of the residency training program committee. This legitimized the process through collaboration with the head of the program, while identifying the right additional collaborators (the panel of experts), to strengthen idea generation and organization. Moreover, we wanted this first meeting to bring the residents together in a context where they could express themselves as freely as possible, within the committee with which they were familiar. The objective was to minimize both the impact from existing relations of power and censorship related to social desirability while avoiding possible confrontations with collaborators in authority.

The second meeting included the panel of experts only. The simplification, organization, revision, and critical analysis to reach consensus was facilitated within a smaller group.

The third meeting included, again, the members from the residency training program committee. The consensus report developed by the panel of experts was presented, commented on, adjusted consensually, and approved by the committee. Four collaborators were part of both groups and participated in all 3 meetings: the director of the medical education research center (A.M.), the faculty members responsible for the evaluation committee (C.H.) and for research (T.M.L.), and one of the chief residents (L.P.T.). Details regarding meetings are summarized in Table 1.

The number of meetings and time allocated were sufficient to meet the objectives, which were to explore the issues, come to a consensual understanding of the situation, and develop recommendations to address recruitment and participation issues. Collaborators worked together to answer the nominal question. The facilitator (L.P.T.) made sure all collaborators had a chance to voice their opinion during meetings. This made it possible to gather a large range of ideas regarding the involvement of residents in medical education research, hence generating the necessary data to produce the framework to come.⁵¹ Throughout the meetings, themes were generated and the collaborators agreed on areas of focus. These were then re-examined and updated from one meeting to the next, allowing discussed ideas to progressively evolve into main issues and recommendations. The report of this iterative process was deemed final when no further comments arose, meeting our definition of consensus. The report was approved by all collaborators.

Analysis Strategy

The analysis is centered on the development process of the final report. It is aimed at rigorously describing the steps of that process, how results from one step were used strategically as the basis for the following step, and how the group worked in terms of participation, communication, and engagement. The database for the qualitative analysis consists of working papers for the meetings, consecutive versions of the draft, and field notes produced during and after meetings by the facilitator (L.P.T.). Thematic text analysis was conducted on the documents to identify key elements of the process and to validate the coherence between these themes and the content of the final report. This analysis was done for the preparation of documents after each step of the final report development. An overarching analysis and verification were done on all documents to prepare this article. The 5 steps of the intervention, including information about what was done, who the collaborators and experts were, and the main outcomes for every step of the process, are detailed in Table 1. We followed the Consolidated Criteria for Reporting Qualitative Research (COREQ).⁵²

People contributing to this research were considered as collaborators rather than research subjects. The study did not meet the criteria for research on human participants, thus it did not need formal approval by the Research Ethics Board.

Results

The initial meeting (first step) was held within a regular monthly meeting of the residency training program committee on September 25, 2018. Only the 20 members of the committee were present for that meeting. The second step was a rapid literature review (led by L.P.T.) and revised by all co-authors. The third step was a second in-person meeting, held November 15, 2018, with the panel of experts only. At that point, findings were organized in main themes, and related recommendations were developed. The fourth and fifth steps consisted of the presentation of the preliminary report on April 30, 2019, developed by the panel of experts, and the final approval by the members of the residency training program committee. The final report is organized into 5 fundamental issues and their related recommendations, discussed here and summarized in Table 2.

Table 2.

Final Identified Issues and Recommendations

Issues	Recommendations
1. Freedom of participation	Presentation of approved research projects by the researcher should be done in group meetings, hence avoiding individual solicitation of residents. Planning of the meeting with chief residents. Information and consent form should be given to the residents so they have sufficient time to read it thoroughly. 1.2 Solicitation should not be done by a faculty member involved in teaching or evaluating the residents involved. 1.3 The information and consent form should clearly state that neither participation nor non-participation in the project will have an impact on any aspect of the resident's clinical training.
2. Over-solicitation of residents	The research project should be presented to a subcommittee of the Residency Training Program Committee. This subcommittee should include at least 1 resident. The following elements need to be considered for approval: innovation and relevance of the project, educational value, integration within the current clinical rotation, and consideration for concurrent projects. The subcommittee will submit its recommendations to the Residency Training Program Committee. After approval, solicitation and recruitment of learners will be supported by a collaboration letter. This letter will need to be submitted to the IRB. 2.2 Residents are encouraged to transmit their refusal to participate in a project. This will limit over-solicitation.
3. Management of time dedicated to participation to research	Priority is given to successful completion of the residency training program. Ideally, participation in the research project should be done during the clinical rotation in the discipline of the researcher. The day, time, and duration of participation in the project should be communicated to the staff member responsible for coordinating the clinical rotation before the beginning of the rotation.
4. Emotional safety of the learner	Simulation research projects should be followed by a debriefing session. Psychological support should be planned before the beginning of the project and activated as needed.
5. Educational safety	Confidentiality regarding learner's participation and performance as a research subject should be protected. The principal investigator should not be involved in the evaluation process of the learner in their clinical rotation.

Abbreviation: IRB, Institutional Review Board.

Issue 1: Freedom of Participation

Participation, non-participation, or withdrawal from a study should not interfere with teacher-learner relationships. To reinforce this, the information and consent forms provided by researchers in protocols submitted to IRB for projects involving residents as subjects should thoroughly define the process and the timeline regarding solicitation and recruitment, in collaboration with the residency training program committee.

Information about the academic relationship the researcher has with the participants, including the role in the evaluation process during clinical rotations, should also be submitted to the IRB. When recruiting, projects must be presented to the entire group of residents (not individuals) and ideally by a third party, rather than the principal investigator who could play an evaluation role at any point in their training. If this is not possible, the principal investigator should be explicit about the fact that participation or non-participation will not impact the evaluation process. This information must be explicitly included in the consent form. The consent form should be distributed or emailed during the presentation of the project, and residents are encouraged to take the time needed to read the form, sign, and return it in a timely manner.

Issue 2: Over-Solicitation

A limited number of ongoing studies should be presented to residents. The creation of a subcommittee (including a resident representative) within the residency training program committee, responsible for approving the research projects and managing their timelines, should be completed. Residents should be encouraged to transmit their refusal to participate, as an absence of response could trigger the researcher to send multiple reminders.

Issue 3: Management of Time Dedicated to Participation

Education, particularly clinical experiences, should be the priority for the trainees. Research participation should not occur during important clinical exposure. Time allocated for participating in research must not be considered as clinical exposure, unless the project helps reach learning objectives during a specific rotation or when participation to research contributes to the residents' learning process during that clinical rotation (eg, neonatal intubation simulation project during the neonatology rotation).

Issue 4: Emotional Safety

As simulation-based projects can be stressful or emotionally disturbing, mandatory debriefing for these projects should be in place. Medical education research could also involve interviews on difficult topics, which can have emotional effects on participants. Reactions to several questions may also be influenced by the fear of being judged, depending on the given answer. For these reasons, support and information resources should be available, namely a systematic debriefing session and confidential counseling for participants, if needed.

Issue 5: Educational Safety

Data collected during a study should not influence clinical assessment of the resident. Research data should be kept confidential, separate from educational records, and should not contribute to clinical assessment. Faculty playing the dual roles of researcher and clinical supervisor should not be involved in the resident's evaluation during clinical rotations in their specialty.

Discussion

We used the nominal group technique to identify 5 issues and related recommendations: freedom of participation, over-solicitation, time management, emotional safety, and educational safety. This approach was appropriate to summarize opinions of all stakeholders, while promoting rapid decision-making to resolve and prevent problems. Throughout the process, the expert panel contributed to the construct of educational safety,^14-16 which highlights the separation to be maintained between medical education research projects and clinical training activities, especially regarding the evaluation processes.

Our recommendations on freedom of participation seek to foster respect of ethical boundaries and solicitation in research projects, where the subjects (residents) may be in a hierarchical relationship with the researcher. Past publications have shown differing results. Through a survey to allopathic and osteopathic medical students, Forester and McWhorter demonstrated that learners did not necessarily want to participate in research projects as subjects but neither did they feel coerced to participate.²⁸ Other studies, more aligned with our results, described medical students as potentially captive, especially if the solicitation came from a faculty member.^27,29 For example, Sarpel et al²² presented a multicenter study in which they examined the perception of third-year medical students about being subjects of a study and described how students felt pressured by the environment or staff to participate.

Literature is scarce on the specific experience of residents regarding over-solicitation and time investment in medical education research. Our results can fill this knowledge gap. Involving residents in restructuring the solicitation process is an interesting idea that complements the recommendations by Sullivan and by Klitzman about the IRB approval process.^30,53 Allowing residents to have an active role in optimizing procedures in medical education research should also be helpful to address the specific problem of survey fatigue, described by Colbert et al in a recent article describing residents' experience and the decline of response rates in research projects.³⁴ As participation is time consuming for residents, this could considerably reduce their attendance during clinical rotations. These outcomes are consistent with those of Sarpel et al and Forester and McWhorter, who also raised concerns about potential risks on clinical training incurred by the time dedicated to research activities for trainees as participants.^22,28

We highlighted that medical education research projects should provide well-organized psychological support, which has been addressed in the literature about emotional safety. Indeed, emotional distress among medical learners,^3,8 humiliation,⁴ and feeling of powerlessness⁶ are well described. Emotional matters need to be addressed, as they might limit participants' capacity to decline participation or to feel comfortable about sharing sensitive information about their psychological well-being, learning capacities, and confidence.

We have introduced adjustments regarding educational safety, currently defined as a form of freedom from judgment by others in educational settings, to bring more clarity to the concept.¹⁴ Issues related to power, hierarchy, competition, and the hidden curriculum are among the driving forces that influence learners' behaviors and decisions.^3-7,15 Respecting confidentiality during medical education research activities (participants' identity, performance, and data collected) should be added to the construct of educational safety to raise awareness of researchers in their capacity to ensure full educational safety (ie, the performance made visible in research activities, such as simulation, should not influence the assessment process in the clinical settings).¹⁴ What is done or said in simulation remains in the educational setting, notably identified as a safe space.⁵⁴ This posture is also respected when simulation is used as a research setting. Our study sheds new light by encouraging researchers to be explicit about their dual role as researcher and evaluator, if these concurrent roles cannot be avoided. Our results highlight the relevance of studying the concept of educational safety, which is built upon psychological safety⁵⁵ and linked to but distinct from emotional safety. More empirical research on the effects of the double status of trainee and participant is needed.

This study has limitations. First, learners were present in the residency training program committee group but have not been involved in the panel of experts group. One member of this group (L.P.T.) was the chief resident of the pediatric residency training program at the time of data collection and therefore held the perspectives of both learner and administrator. We recognize that this “middle manager role,” as described by Berg and Huot,⁵⁶ did not make him a typical learner. Also, our study has been conducted in a single university hospital center and in a single program. Different findings might have been reached as a result of cultural differences between programs, or in programs with fewer learners.

The issues raised and their related recommendations are not to be considered a toolkit to operationalize these recommendations without any flexibility. For example, concerning freedom of participation, we recognize that it would be almost impossible to fully guarantee that participation or non-participation in a research project will not have an impact on the resident's training if working with the researcher afterward, in the clinical context. Also, while it would be ideal for a researcher not to be involved in the evaluation process of the learner during clinical rotations, we know that this recommendation might be impossible to operationalize. Moreover, in a smaller program, demographics data about participants might expose them to an anonymity breach.

With this study, we reaffirm the importance of awareness among researchers in medical education about the possible and often subtle issues that can arise when residents are subjects of research. Future research should concentrate on comparative studies of our findings in different settings, the development of operationalization guidelines and tools framing and promoting educational safety for residents as research participants in medical education research.

Conclusions

Our nominal group technique approach resulted in raising 5 specific issues about freedom of participation of residents in research projects in medical education, over-solicitation, time dedicated to these projects, emotional safety surrounding their participation, and educational safety.