Learning pediatric emergency medicine over time: A realist evaluation of a longitudinal pediatric emergency medicine clinical experience

Abstract

Introduction

Emergency medicine (EM) practitioners must be proficient at caring for patients of all ages, including pediatric patients. Traditionally, EM trainees learn pediatric emergency medicine (PEM) through block rotations. This is problematic due to the seasonal nature of pediatric diseases and infrequent critical events. Spaced repetition learning theory suggests PEM would be better learned through longitudinal rotations. The transition to competency‐based medical education (CBME) in Canada is accelerating the need to find novel ways to attain competencies in postgraduate training. At McMaster University, senior EM trainees can choose either traditional PEM blocks or longitudinal rotations. Our objective was to understand how learners experience these different rotations given the transition to CBME in Canada.

Methods

Using a realist framework of program evaluation, we conducted semistructured interviews with key stakeholders (trainees, program directors, attending physicians) in EM. The realist framework was used to understand how context interacts with theoretical mechanisms to produce outcomes of interest. Data were analyzed using inductive, conventional content analysis. All investigators coded a subset of transcripts independently and in duplicate to achieve intercoder agreement.

Results

A total of 13 interviews were completed with trainees (n = 11) and staff physicians (n = 2). The learning experience exists within an educational and clinical context, which are logistically distinct but inseparable. The longitudinal learning experience appears to improve learning through spaced repetition, which prevents atrophy of skills and knowledge while also benefitting from the offsetting of seasonal variability associated with many pediatric diseases. Improved feedback and entrustment are facilitated through the building of coaching relationships over time. Barriers to the learning experience are related mainly to logistical difficulties associated with resolving longitudinal and blocked learning experiences. Improved relationships with the interprofessional team may provide distinct learning opportunities and improved team functioning. Block rotations were identified as more valuable to junior trainees learning fundamental concepts.

Conclusions

Longitudinal learning provides numerous advantages to learning PEM, including increased case variety, spaced repetition of core concepts, and a perception of greater entrustment of the learner through formation of coaching relationships over time. Future projects looking to quantify the differences between longitudinal and block learning to objectively show a difference in skills and knowledge are needed.

INTRODUCTION

Emergency medicine (EM) encompasses a wide breadth of knowledge, skills, and attitudes. The nature of EM requires practitioners to be proficient at caring for patients of all ages, including children. This is especially important considering that the majority of children are not cared for in specialized pediatric centers ¹ , ² and that over 20% of a general emergency department (ED)'s visits are pediatric, representing a large portion of their clinical practice ³ , ⁴ The very nature of pediatric emergency medicine (PEM) centers around infrequent high‐stakes events intermixed with a large volume of lower acuity presentations, even among high‐volume centers ⁵ making it imperative that we train EM learners in the most effective manner possible. While alternate learning methods such as simulation‐based education or classroom lectures can help to bridge this gap, real clinical experiences can be invaluable for building capacity in caring for children.

In their call to arms regarding best practices in training EM learners for PEM, Cloutier et al propose that alternate rotation structures may improve the delivery of PEM training. ³ Traditionally, EM training proceeds on a fixed time‐based model, with learners rotating through blocks (typically 4–8 weeks at a time) of specialty training to meet their curricular learning objectives. PEM is typically a required rotation and follows this model in most Canadian EM training programs. Given the seasonality associated with pediatric illness, there is concern that an EM trainee could theoretically finish their training having never managed certain common pediatric emergencies.

At present, there is a shift, with the world's health professions educators now increasingly interested in competency‐based, time‐variable health professions education. In Canada, EM training has transitioned to a competency‐based model ⁶ known as Competency by Design (CBD), in which the framework of training is built around demonstrating competencies rather than immersion in clinical rotations for a fixed quantity of time. Thus, the traditional block method of training may no longer be the optimal method to deliver high‐quality learning experiences.

Longitudinal clinical experiences have the potential to bridge this gap created by the transition to competency‐based medical education (CBME). While longitudinal or integrated models of clinical education have been explored at the undergraduate medical education level, few studies have been done at the postgraduate level. ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ The few studies that have been done tended to focus on components of the postgraduate curriculum, rather than on whole clinical experiences, such as with point‐of‐care ultrasound. ¹² , ¹³ Despite these limitations, however, these studies suggested that longitudinal clinical experiences improve learning outcomes compared to traditional learning.

In response to learner feedback, the McMaster University Division of Pediatric Emergency Medicine has been offering senior EM trainees the option to rotate through traditional PEM blocks or to do PEM longitudinally, where they work a set number of shifts per month for an entire year. This provides a natural opportunity to explore how key stakeholders in EM experience longitudinal learning in the context of the current shift to CBME in Canada. This study aims to help inform the redesigning of educational experiences within EM training programs to ensure trainees are competently trained in the acute care of children.

Our objective was to qualitatively explore the perceived advantages and disadvantages of a longitudinal PEM clinical rotation embedded in a Canadian EM training program that has transitioned to a CBME model. We used the conceptual framework of spaced repetition learning theory as the basis for our exploration, assuming that the advantages and disadvantages of such a program would be grounded in the fact that longitudinal clinical experiences can help to mitigate forgetting curves ¹⁴ and reinforce learning in the long term. ¹⁵ Early studies by psychologist Ebbinghaus showed that when new material is learned by an individual, it is rapidly forgotten. ¹⁴ However, repetitive exposure to that same material spaced over time has the effect of delaying the decay of that learned material and with enough exposures, results in retained knowledge in long‐term memory. Our use of the spaced repetition learning theory assumes that longitudinal clinical experiences are a way to prevent the decay of knowledge and skills over time.

METHODS

To meet our objective, a realist framework was adopted for the evaluation of the program, wherein outcomes of the program were evaluated in the social and historical context in which it was implemented and the theoretical mechanisms that lead to the outcomes. ¹⁶ The realist framework seeks to answer the question “What works for whom, in what circumstances and why?” aiming to define the underlying causal mechanisms through which outcomes occur and the contexts in which those mechanisms are triggered or activated. This relationship can be described more simply as a formula: context + mechanism = outcome, which helped us to organize the findings of our qualitative study. This is important because programs often occur within particular contexts and understanding this relationship is important in how they influence eventual outcomes. ¹⁷ , ¹⁸ Institutional research ethics board exemption was obtained for this study.

Context

The learning environment studied was a single pediatric emergency department (PED) at McMaster Children's Hospital (MCH). This is a tertiary care level hospital associated with McMaster University, in Ontario, Canada. The PED at McMaster University sees approximately 55,000 patients per year, with 15% of patients being triaged to a Canadian Triage and Acuity Score (CTAS) of 1 or 2 (resuscitation or emergent). All shifts in the PED have a significant presence of learners of all levels. The population studied consisted of senior residents (PGY‐4 or ‐5), undergoing specialty training in the Royal College of Physicians and Surgeons of Canada (RCPSC) Emergency Medicine residency program at McMaster University in Hamilton, Canada. The traditional block method of PEM rotations consisted of one PEM block (4 weeks) in PGY‐1, two blocks in PGY‐2, and two blocks in PGY‐4 or PGY‐5. The longitudinal PEM rotation consisted of two clinical shifts per month for 12 months, in PGY‐4 or PGY‐5. The objectives of training of the rotation are identical between both the longitudinal and the blocked formats (see Appendix S1). Supervision is done by faculty of the PED at MCH, which include board‐certified physicians in both PEM and EM. Shift structure is identical between blocked and longitudinal learners and is 9 h in duration, with learners paired with a single supervising faculty for the entire shift. Blocked learners are scheduled evenly between day and evenings shifts with three overnight shifts and two weekends over 14 shifts in a 28‐day block. The longitudinal learners self‐schedule based on availability of shifts and supervisors and have no minimum requirement of nights. The learners in the block format also attend a PEM specific academic half day that includes didactic lectures and simulation sessions of common pediatric emergencies seen in PEM. During the time of the study, the CBME model had only been introduced to junior residents in the RCPSC EM program.

Study design

We used interpretive description ¹⁹ to explore the advantages and disadvantages of longitudinal PEM training from the perspectives of trainees and physician attendings. We used this approach because it allows for comprehensive summaries of the participants' perspectives on PEM longitudinal training and considers the inevitable individual variations among different participants.

Sampling and recruitment

We applied purposive sampling techniques. All attending physicians who supervised senior trainees from the McMaster University PEM rotations, and all trainees who were part of these rotations were eligible. The participants were recommended by the education chair for the PEM division, to capture a cross‐section of learners who participated in the longitudinal program and those who did not, as well as faculty with all faculty associated with teaching trainees while on shift in both the pediatric and the general EDs. This included current and previous residency program directors.

Data collection

We collected data from August 2018 to January 2019. A total of 33 eligible individuals were contacted; 39% (n = 13) agreed to participate in our study. We interviewed 11 trainees and two faculty. Of the 11 trainees, five had completed a year of longitudinal clinical experiences. Data were collected using semistructured, audio‐recorded interviews. The interviewer (SAL) was a PhD candidate whose interest was in program evaluation; she was not affiliated with the residency training program. We pilot tested the interview guide with two eligible individuals for feasibility and clarity of the questions. Three questions required modification but because the modifications were minimal, data from these interviews were included in the qualitative analysis. Data collection and analysis occurred concurrently, and interview questions were subsequently modified to better explore themes that were identified over time. We collected data until thematic sufficiency (no new information from interviews) was reached, independent of the source of the data (i.e., no distinction was made between faculty and trainee data).

Data management

Interviews were transcribed verbatim and deidentified by an independent transcriptionist. The master list that includes the participants' names and their corresponding ID are stored in a separate, password‐protected file in a separate folder.

Data analysis

To analyze the data, we used inductive, conventional content analysis and sensitized ourselves in spaced repetition learning theory. Each transcript was coded independently and in duplicate. The coders held four meetings to share and revise codes, until all coders agreed with all codes. Codes with similar concepts were grouped together to form categories, and we formed themes from the categories. To ensure our findings were trustworthy, we employed a number of strategies. First, we used negative case analysis after we had formed our themes, where we searched the data to understand if any responses did not fit into the overall themes and categories that we had formed. We also maintained reflection summaries in the form of minutes of the coding meetings to reassess how analytical decisions were made and apply any insights to subsequent meetings. Finally, we conducted a member check, wherein we sent a summary of the results to three participants (one faculty participant) to understand whether we had missed any major ideas in our analysis.

RESULTS

A total of 11 participants were EM trainees and two were attending EM physicians. Interviews lasted, on average, 24 min (13 to 36 min). We identified several processes unrelated to the medical expert role that were facilitated by the longitudinal learning experience. While largely positive in nature, there were also drawbacks that became apparent due to the structuring of the learning experience.

Context, mechanism, and outcomes of the longitudinal rotation

Both blocked and longitudinal rotations exist within an educational context (clinical supervision and assessment, bedside, didactic and simulation teaching sessions) and a clinical context (ED culture, interprofessional team), which are logistically separate but intimately intertwined and functionally not separable. However, we find that they interact differently with the mechanisms of the longitudinal rotation to influence the learning experiences of the residents. Within the clinical context, the longitudinal rotation results in outcomes related to how the learner delivers patient care, while within the educational context, the longitudinal rotation provides both barriers and improvements to the attainment of learning objectives in preparation for future clinical practice. We describe the mechanisms within these contexts that shape the learner experience with illustrative quotes in Table 1 and how they interact in Figure 1.

TABLE 1.

Description of CMO and illustrative quotes

Context	Mechanism	Outcome	Illustrative quotes
Educational context	Spaced repetition	Retention/reinforcement of learned concepts Avoids need to reorient to the clinical experience	“Blocks allow you to gain confidence quickly but if the time you spend away from the block is long, like a year, then you forget all that comfort/competence (i.e., choosing right size of resus equipment etc.). Loss of that knowledge seems to be a backward way of doing things” (P4) “… stays fresh in the brain” (P10) “… became rusty when not seeing kids regularly” (P9) “Longitudinal learning gives you consistent exposure, especially in senior years when you are consolidating skills rather than learning for the first time. Pros of block based learning are especially early on—getting volume and becoming comfortable early on …” (P6) (P5 expresses similar)
	Building longitudinal coaching‐type relationships	Entrustment Meaningful coaching in areas for improvement	“… no differences in assessment between PEM and EM but the longitudinal assessors became more attuned to your strengths and areas of growth.” (P3) “It helped people get familiar with you and know you and it, um, it helped you to get to know them and their different styles. And so, um, they were more comfortable with you, uh, taking a more leadership role in terms of running and managing the department and junior learners.” (P10)
	Logistics in implementing longitudinal experiences	Barrier to successful implementation Learner frustration	“… need more shifts to minimize long breaks between seeing pediatric patients.” (P5) “There's a lot of complexity in balancing multiple schedules.” (P9) “The disadvantages are mainly operational … non clinical components need to be more formalized.” (P1) “Longitudinal experiences depend on how flexible your program would be for that.” (P6)
Clinical context	Longitudinal learning offsets seasonal variation of pediatric disease	Exposure to breadth of the clinical cases in the specialty	“… we saw more variety considering the seasonality of peds presentations like bronchiolitis.” (P8)
	Relationship building amongst members of the interprofessional team	Positive relationships in the clinical environment improve clinical care delivery Entrustment	“… staff like it better to work longitudinally with the trainee.” (P8) “Really enjoy the bridging between the emerg and PEM departments, like fostering connections between physicians and sharing and developing policies.” (P3) “You can build better relationships with staff and nursing”

Abbreviation: CMO, context, mechanisms, and outcome.

FIGURE 1.

Illustration of how longitudinal learning affects learning outcomes via different mechanisms.

Educational context

Spaced repetition reinforces concepts and avoids reorientation

Interviewees identified that a disadvantage of block learning was the long periods of time between each of these clinical rotations, resulting in forgotten knowledge and skills. They expressed concerns around forgetting core concepts (e.g., “… becoming rusty when not seeing kids regularly” [P9]) and that the current system of blocked or massed learning, inherently by design, has large gaps in time where a trainee might not see a certain type of patient. This system results in ineffective learning. However, they also identified an advantage of the block rotation system included rapid gains in confidence resulting from immersion in a clinical area and gaining foundational content. This can then be consolidated through later longitudinal learning experiences in later clinical years.

Longitudinal learning builds coaching type relationships that allow for more meaningful feedback and entrustment

Several interviewees did not notice a difference in how their assessments were conducted but commented on the advantage of longitudinal relationships facilitating feedback for learning and in the process of entrustment. Rather than having to reintroduce themselves to their supervisors after a potentially long absence from the PED, learners felt that it was easier for supervising faculty to entrust the learners because they were more familiar with their current level of clinical competence. Other learners commented that relationships allowed for more meaningful feedback given the supervising faculty having a better understanding of a learner's strengths and weaknesses.

Logistics as barriers in implementing a longitudinal clinical experience

Learners identified that planning for the longitudinal clinical experience was difficult if the programs did not directly interact with each other. This resulted in difficulties in scheduling due to the need to balance two or three competing calendars for shift scheduling. This was repeated many times across all interviews and represents an ongoing challenge given the increasing number of commitments and draws on a senior learner's time. Additionally, some learners felt that the commitment to the nonclinical components of the clinical experience (simulation, formal academic conferences) were unclear and thus difficult to navigate. A few learners suggested that more frequent shifts may be required to minimize long breaks in between experiences. None of the interviewees commented on what the optimal number of shifts could be.

Clinical context

Longitudinal learning offsets the seasonal variability of pediatric disease

We found many examples from the participants of how longitudinal learning in the pediatric ED allowed them to be exposed to presentations of disease that they had yet to experience as a junior resident in a block. Because of the seasonal variation that is inherent in pediatric presentations to the ED, ²⁰ this is an important consideration in educational planning for EM trainees. Predictably, winter months bring an increase in infectious diseases and their associated complications, whereas summer months bring about a higher proportion of physical trauma. It is possible for the EM trainee to have all their pediatrics blocks over their training in one or two seasons, resulting in never having managed the spectrum of common pediatric diseases such as asthma and bronchiolitis.

Longitudinal learning builds positive relationships amongst members of the interprofessional and medical team

There was a perception that both physicians and the interprofessional healthcare team enjoyed the experience of working with longitudinal learners more than with those on block rotations and that the working environment was more positive as a result of these improved working relationships. It appears that the positivity may have been due to these longitudinal relationships helping to build trust amongst the interprofessional team, as evidenced by the quote “You can build better relationships with staff and nursing.” Another idea mentioned in this theme was that the relationship building that occurred could foster more collaboration between the PEM and adult EM groups.

DISCUSSION

The implementation of CBD in Canada has brought about the need to reexamine the way we deliver clinical experiences, particularly designed in a way to meet the competencies set out by the specialty. While longitudinal clinical experiences have been extensively described in undergraduate medical education in the literature, ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ their impact on learning at the postgraduate level remains largely unexplored due to a paucity of program evaluation. ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ We present a program evaluation of a longitudinal PEM experience in an EM training curriculum through a realist framework of inquiry, to explore how longitudinal PEM clinical experiences work, for whom, in what circumstances, and the underlying mechanisms.

To answer the question of which learners benefit from longitudinal experiences and in what circumstances, our findings suggest that block learning would be most beneficial for the novice learner who is looking to build a foundation of knowledge, skills, and clinical experience. The longitudinal component would be more useful for the senior trainees who are looking to consolidate their learning, which would be consistent with spaced repetition. The implications of our findings involve residency curricular planning and how clinical rotations are planned for junior versus senior residents. Block rotations would be useful for junior residents initially to acquire skills followed by gradually increasing the spacing between sessions to improve recall as the learners become more senior. This appears to be in line with contemporary ideas of learning science and distributed practice wherein new material learned should be reinforced early before gradually spacing out the intervals of recall. ¹⁵ , ²⁸

No evidence exists that can comment on the optimal number of clinical experiences that are necessary to achieve competency and would suggest that experience alone is a poor proxy for skill. ²⁹ , ³⁰ , ³¹ However, the theme of negating seasonal variability of pediatric medicine would suggest that any longitudinal clinical experience will be more effective in providing a wider variety of opportunities to assess and manage certain types of patients. In a competency‐based model of education, this is important, although we cannot say if this will result in an increased demonstration of competency. Thus, the question of minimum number of clinical experiences should be related back to the idea of spaced repetition—experience should be sufficiently frequent to mitigate the effect of forgetting and that may vary depending on the subject matter. For example, longitudinal clinical experiences in trauma may be required less frequently than pediatric ones because of the more frequent nature of trauma in the general ED.

The opportunity to build relationships longitudinally with faculty through regularly spaced interactions is important as it provides the potential for more meaningful feedback and subsequent learning. This provides potentially an underlying mechanism as to why longitudinal clinical experiences may be a more effective method of learning. Receptivity to feedback is dependent in large part on the credibility of the person providing it and having an ongoing relationship facilitates this process. ³² The staff is more aware of the learner's strengths and weaknesses and can personalize feedback, and the learner may be more receptive to feedback from a staff they know and trust. ³³ The opposite of this may be true given that faculty may be more willing to entrust a longitudinal learner with more clinical responsibility. ³⁴ What is not known is whether longitudinal learners' assessments are affected in any way when compared with their block rotation counterparts.

The theme of improved health care team relationships deserves some further exploration given its importance in high quality patient care and patient safety, likely mediated through improved interprofessional communication. Longitudinal learners commented on how they perceived there to be a preference from both faculty and nursing staff to working with longitudinal learners and that it allowed them to build better relationships. We are unable to determine how this translated into outcomes with respect to improved interprofessional collaboration in actual patient care. We hypothesize that learners may have also had increased learning opportunities given that the nursing team can often direct certain patients toward the trainees or faculty, depending on acuity, trust in the trainee, and a willingness of the trainee to communicate their learning needs to the interprofessional team.

Logistical considerations around planning of the longitudinal learning experience were discussed among every participant. Common in this discussion was the complexity of managing multiple schedules, which was made even more difficult because of the lack of coordination between these clinical sites. Traditionally the clinical sites have been stand‐alone rotations with no need for collaboration in trainee scheduling. However, to ease the process and make it successful, we would suggest schedule coordination. This should also include capacity calculations to ensure that the longitudinal learners have enough opportunities in the schedule to meet the minimum required shifts. As the number of learners increases, so too does the complexity of scheduling. When a schedule is too full of learners, the longitudinal learners often found it frustrating and unable to find shifts. Anecdotally, some learners have dropped out of the longitudinal clinical experience due this logistical difficulty alone. The nonclinical expectations of the learners (i.e., simulation training, attendance, and presentations at academic conferences) also need to be made explicit to minimize the confusion that learners felt during their longitudinal block. A surplus of nonclinical (didactic, simulation, etc.) learning experiences was felt to be a contributing factor to their difficulties in scheduling. Our solution to this problem was to relieve longitudinal trainees of these adjunctive academic duties, which increased the flexibility for their training and allowed them to partake in shifts that were previously unavailable to them.

Our study did not address the question of how effective longitudinal PEM clinical experiences are. There is likely a “dose dependency” relationship such that more longitudinal clinical shifts will result in more effective learning. We speculate that one shift per 4‐week block is too little and that there is likely a point of diminishing returns, but the ideal number is unknown and likely depends on the specific learning outcomes required of the program. Future work should seek to quantify any educational and/or clinical differences in outcomes between those in longitudinal clinical experiences compared with those in traditional block clinical experiences. The literature on longitudinal experiences would suggest some advantages in productivity ³⁵ , ³⁶ but have yet to show clinical or educationally meaningful outcomes. This is especially important as more postgraduate training programs move toward competency‐based models of training. Further, there remains the question of the type of learner that would most benefit from this type of training model and whether it translates well to other clinical contexts. Finally, and perhaps most important in moving this sort of educational intervention forward, is whether it offers measurable advantages relative to traditional models of clinical learning. Watts and Green ¹⁰ interestingly found that their learner's in‐training exam scores improved after they moved to a blocked model of learning from a longitudinal model, although this study was not a randomized trial, introducing potential for confounders to this effect. The findings of our realist evaluation should help to frame future study design when examining measurable educational outcomes of longitudinal versus blocked rotations.

LIMITATIONS

As with most studies, there are several limitations that warrant attention by our readers. As this was an evaluation of a specific program, the context is specific to the experiences of our learners and teachers within our system. Although this will certainly limit the transferability of our findings, we still feel that there are transferable inferences and findings that may inform the construction of similar learning experiences in other locales. Further, the small number of faculty interviewed as well as lack of interprofessional health care team members also limit the transferability of our findings. For example, the thought that longitudinal learners have better team relationships cannot be fully understood without triangulating with interviews with more faculty and allied health care team members. Finally, we acknowledge that our findings are being viewed through the framework of spaced repetition learning theory and that the results could be interpreted through several different learning theories.

CONCLUSIONS

The delivery of longitudinal pediatric emergency medicine rotations that are well coordinated between programs has the potential to improve learning outcomes via spaced repetition of content, exposure to the breadth of clinical presentations in children, improved relationships with faculty that facilitates coaching and feedback, and improved delivery of care through improved interprofessional team relationships. Maximizing this effect requires tight integration between the emergency medicine training program and the pediatric emergency medicine rotation to minimize barriers to the clinical experience. Future exploration should aim to understand measurable learning outcome differences between longitudinal and blocked learning experiences and whether coaching and feedback have any meaningful effect on attainment of entrustment of pediatric emergency medicine learning objectives.

AUTHOR CONTRIBUTIONS

Quang N. Ngo contributed to the study conceptualization and design, analysis of data, and writing and editing of the manuscript. Alex Chorley contributed to the study conceptualization and design, analysis of data, and writing and editing of the manuscript. Shelly‐Anne Li contributed to the study design, data gathering, analysis of data and writing, and editing of the manuscript. Teresa M. Chan contributed to the study conceptualization and design, analysis of data, and writing and editing of the manuscript.

CONFLICT OF INTEREST

All of the authors report that they are employed in part by McMaster University.

Supporting information

Appendix S1

Ngo QN, Chorley A, Li S‐A, Chan TM. Learning pediatric emergency medicine over time: A realist evaluation of a longitudinal pediatric emergency medicine clinical experience. AEM Educ Train. 2022;6:e10822. doi: 10.1002/aet2.10822