Faculty‐lead Opinions on Workplace‐based Methods for Graduated Managerial Teaching (FLOW MGMT): A National Cross‐sectional Survey of Canadian Emergency Medicine Lead Educators

ABSTRACT

Objectives

Patient volumes are increasing in emergency departments (ED), causing issues with long wait times and overcrowding. One strategy to cope with this phenomenon is to focus on improving patient flow through the ED. Building on earlier work that identified how staff physicians manage flow and what techniques they employ to teach managerial skills to residents, we aimed to determine when it was most appropriate to implement these teaching strategies in a resident’s training.

Methods

We employed a Canada‐wide cross‐sectional survey of experienced emergency medicine (EM) teaching faculty to determine when they felt our previously identified teaching strategies would be appropriate to implement. The survey was piloted with local educational experts.

Results

A total of 21 EM (38% female, 62% male) educators from 11 programs responded to the survey. The respondents provided an average of 42.5 endorsements per participant for specific teaching techniques across the stages of training. The core of discipline (35.9%) and transition to practice (39.7%) were the stages of training that received the most endorsement. The top two teaching techniques included the observational teaching technique “attitudinal role modeling (i.e. a strong work ethic)” and the conversational teaching technique “teacher provides clinical pearls, tips, pointers.” The participants showed fairly high agreement, with the advanced in situ techniques showing fairly high reliability as measured by intraclass correlation coefficients ranging from 0.88 to 0.90.

Conclusions

Our results show a trend toward faculty utilizing more didactic and observational teaching techniques early in residency and then progressing toward more experiential techniques in the senior stages of training. This is consistent with a graduated increase in responsibility as residents demonstrate competency and progress through their training. The results of this study will help inform faculty development around teaching managerial skills in the area of competency‐based medical education.

Patient volumes seen in the emergency department (ED) across Canada and the Western world are increasing, which is one of the contributing factors to overcrowding in the ED.1, 2, 3 One means to combat overcrowding is to focus on the workflow of a busy ED. On this individual level, health care providers in a busy ED are tasked with managing multiple competing priorities4 and task switching,5 and must remain focused while handling constant workplace interruptions.6 Optimizing the system and engaging in quality improvement projects will support emergency medicine (EM) residents in the short and long term. However, they still require an understanding of the basic approach and skills to manage multiple patients in a busy ED. Undoubtedly, graduating residents must be well educated on ED management so that they are prepared for this aspect of EM, rather than expecting them to discover these skills after they enter into unsupervised practice.

Competence by Design: The Dawn of a New Era in Canadian EM Education

The Competence by Design system was launched in Canada by the Royal College of Physicians and Surgeons of Canada (RCPSC) in 2017. In July 2018, the first cohort of Competence by Design trainees were officially admitted to the 5‐year RCPSC EM training program.7

Whereas the American system uses a series of Milestones put forth by the ACGME, the new Canadian system takes a different approach. The Competence by Design program seeks to layer competency‐based medical education (CBME) principles8 within a somewhat time‐based system, but more heavily weights decisions for promotion and training on the basis of workplace‐based assessments (known as entrustable professional activities [EPAs]) accrued by faculty observation. The unit of analysis in the Competence by Design system, therefore, is an EPA (which is a piece of work aligned with our specialty’s nuances). Our EM specialty’s 28 EPAs are divided into four stages in a programmatic way, allowing trainees to progressively learn at a unique stage. The data from the EPA acquisition, alongside other more traditional tools like simulation or written tests, generate data that allows us to make aggregate decisions within a competency committee about a trainees progression, but also to initiate tailored learning for individual residents in the system.

At the core of it, Competence by Design seeks to ensure that trainees are provided the educational experiences required to ensure that they are ready for practice on day 1 postgraduation by defining the core pieces of work for the specialty via EPAs. Inevitably, as the Canadian residency education landscape shifts to a focus on trainee competencies, new approaches and instructional methods will need to accompany these new EPAs.9 Curricular changes and faculty development will naturally accompany the shift to CBME.10, 11 However, before curricular changes can be made, we must conduct the research necessary to best inform changes to our educational practice.

Teaching Must Precede Assessment: The Curriculum‐Entrustment Gap

Competence by Design trainees are expected to be entrusted to manage the ED to optimize patient care and departmental flow in the final stage of training (“Transition to Practice”).7, 12, 13 However, for a trainee to be competent in ED management within their final stage of training, this skill must be taught to them long before they are entrusted with the totality of this task.14, 15 But since this is a newly defined EPA within our specialty that is left to be achieved in the final stage of training, it is incumbent of teachers to consider how we might teach these skills well in advance of assessing them near the end of training. Teaching, after all, must precede assessment in any good curriculum. And so, in this present study we must ask: How might the teaching of ED managerial skills be best accomplished?

A previous study from our group undertook a qualitative study to better understand the cognitive processes involved in managing a busy ED.16 We found that most emergency physicians and trainees undertook a three‐step process when evaluating and triaging multiple patients. First, the physicians create a big picture of the department.16 Second, they evaluated individual charts to create a patient story, and finally they developed a relative prioritization list.16

Our research group then examined, through an interview‐based descriptive study, what teaching strategies were being employed by physicians to prepare trainees to handle multipatient environments (such as an overcrowded ED).17 The three main types of strategies that were extracted from that qualitative study included observational learning, in situ ED instruction, and heuristic‐based advice.17 Observational learning took the form of the emergency physician modeling effective managerial skills. In situ ED instruction took many forms such as conversational methods in which physicians imparted their knowledge around ED management or clinical teaching that was prompted by the environment. Additionally, in situ ED instruction involved experiential learning with varying degrees of supervision. Finally, heuristic‐based advice generally took the form of informal conversation around “lessons learned” and team/system‐level advice.17

The cognitive processes involved in managing a busy ED, and the types of teaching approaches used to impart those management skills, have been identified via this previous work.16 The next step of inquiry in our program of research was to determine at which point these instructional strategies should be employed during a learner’s residency to optimize their competency to manage an overcrowded ED in their early career. The objective of this study was to create a survey to determine the training level at which EM residents should be taught about ED management and flow, specifically which strategies should be employed based on the level of training in the age of CBME and specifically Canada’s new Competence by Design (CBD) system.12, 13

METHODS

Overview

We conducted a Canadian‐wide cross‐sectional survey to determine how experienced EM faculty members might stratify previously identified instructional methods for teaching ED management skills within the new RCPSC Competence by Design framework.

Survey Development

An electronic survey was made in Google Forms (Mountainview, CA, USA) was created based on findings from a previous research study undertaken by Chan et al.17 which identified different flow and management teaching strategies that are employed in the ED (see Data Supplement S1, Appendix S1 [available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10429/full], for details). The survey was a 21‐item questionnaire that encompassed five different subgroups of teaching techniques. These techniques include formal teaching (1), observation (2), conversation (4), bedside coaching (7), and experiential (7). The respondents were tasked with assigning which level of training, based on the four Competence by Design stages18 (Transition to Discipline, Foundations of Discipline, Core of Discipline, and Transition to Practice), the teaching technique was most appropriate, or if it was not appropriate for any of the stages. Each survey was accompanied with a document outlining each teaching method in detail if further clarification was needed by those completing the survey, seen in Data Supplement S1, Appendix S2. The survey was piloted with operations and educational experts and amended with their feedback.

Data Collection

The survey was sent out to 14 RCPSC and 27 Canadian College of Family Physicians–Emergency Medicine certificate (CCFP‐EM)‐trained EM educators across Canada. Our request for participation went to our country’s 14 RCPSC and 17 CCFP program directors, and we asked them to send this survey to a maximum of three other educators at each site (the assistant program director, competency committee director, Competence by Design program transition lead). Since some programs had individuals holding dual roles, our maximum response rate was 60 respondents. These survey invitations were distributed with the assistance of program directors across Canada, through social media platforms such as Twitter, and via direct e‐mail from partners involved in the project. Respondents were offered no monetary rewards, but were offered a chance to be acknowledged as contributors to our present work. The online survey was left open for a total of 6 weeks. The survey responses were anonymized prior to analysis. The project was given an exemption by the Hamilton Integrated Research Ethics Board since it was viewed as a curriculum development initiative that would help us align our previous work with CBME curricular changes in the near future.

Data Analysis

Data were entered into Microsoft Excel and analyzed using simple descriptive statistics for each teaching technique. An average measures intraclass correlation coefficient (ICC) with a two‐way mixed‐effects model was calculated to provide a measure of agreement between our participants for each set of advanced techniques (observational, conversational, bedside coaching, experiential). A post hoc subgroup analysis was completed for a particular teaching technique "Throw into deep end (no assistance, resident is thrown in without prep)” as it was the only teaching technique to be felt to never be appropriate to employ by several respondents. An independent‐sample t‐test was conducted to compare number of years in practice for those that felt it was never appropriate (no) to employ the experiential teaching technique compared to those that felt it was appropriate (yes). Similarly, a chi‐square test of independence was performed to examine the relation between gender and appropriateness of the above teaching strategy. As well, a chi‐square test of independence was performed to examine the relation between designation (FRCPC vs. CCFP‐EM) and appropriateness of the above teaching strategy.

RESULTS

Demographics

A total of 21 EM (38% female, 62% male) educators from 11 programs (78% of Royal College training sites) responded to the survey from August to September 2018, representing seven of 10 provinces, with a mean (±SD) years‐in‐practice of 15.2 (±9.7) years. There were an average of 1.8 respondents per site. We had an approximated 35% response rate, based on our calculation of about 60 educational leaders in EM in Canada being the total number. All respondents were involved in resident education; 66% had a current formal educational role, such as program director.

Competency by Design Stages

There were an average of 42.5 endorsements per person (out of a possible 105 endorsements per person) for the various teaching techniques in the four stages of training. An endorsement indicated that the teaching technique was appropriate to employ at that stage of training to educate learners about ED management. Each respondent could select multiple stages where a teaching technique might be suitably used. Transition to discipline received 8.5% of endorsements, foundations of discipline received 15.9% of endorsements, core of discipline received 35.9% of endorsements, and transition to practice received 39.7% of the total endorsements. Figure 1 depicts a frequency chart of all the endorsements as a function of the stages of Competence by Design for each teaching technique.

Figure 1.

A heat map representation of the percent endorsement for each ED managerial skill surveyed.

Endorsement of Specific Teaching Techniques

The observational teaching technique wherein physicians used attitudinal role modeling techniques (i.e., displaying a strong work ethic to trainees) received the most endorsements in total for the Transition to Discipline (n = 10) and Foundations of Discipline (n = 16) stage of training. The conversational teaching technique “informal conversation with staff physician” received the most endorsements in Core of Discipline (n = 21) stage. Finally, the experiential teaching technique “given the reins (resident is placed in charge by attending)” received the most endorsements in the Transition to Practice (n = 21) stage of training.

Examining the total number of endorsements each teaching technique received, as an aggregate across all four stages of training, the top three teaching techniques included the observational teaching technique “attitudinal role modeling (i.e. a strong work ethic)” (n = 60), the conversational teaching technique “teacher provides clinical pearls, tips, pointers” (n = 57), and the conversational teaching technique “informal conversation with staff” (n = 55).

The teaching techniques that received the fewest total endorsements included the experiential teaching technique “thrown into the deep end (no assistance, resident is thrown in without prep)” (n = 17) and “given the reins (resident is placed in charge by attending)” (n = 25). Both experiential teaching techniques received no endorsements for utilizing these techniques in the Transition to Discipline and Foundations of Discipline stages of learning.

Among all the teaching techniques, only the experiential teaching technique “thrown into the deep end (no assistance, resident is thrown in without prep)” was felt to be a technique that was never appropriate to employ to teach residents on how to manage flow in an ED. Among the 21 respondents, 28.5% (n = 6) felt that it was never appropriate to utilize this minimalist technique.

Reliability Analysis

To determine agreement between our participants about the levels at which the various teaching techniques could be used, we calculated the average measures ICC for each of our types of in situ teaching techniques. The observation techniques had the lowest agreement (ICC = 0.77 [95% CI = 0.58 to 0.89]), although the ICC was still fairly good. Our participants agreed even more with the conversational techniques (ICC = 0.85 [95% CI = 0.73‐0.93]), bedside coaching techniques (ICC = 0.90 [95% CI = 0.83 to 0.95]), and experiential techniques (ICC = 0.82 [95% CI 0.68‐0.91]).

Subgroup Analysis

An independent‐sample t‐test was conducted to compare number of years in practice for those that felt it was never appropriate to employ the experiential teaching technique "Throw into the deep end (no assistance, resident is thrown in without prep)” compared to those that felt it was appropriate. There was no significant difference between the number of years in practice in the yes (mean ± SD = 15.9 ± 10.3) and the no groups (mean ± SD = 13.3 ± 8.59; t(19)  = 0.54, p = 0.60). A chi‐square test of independence was performed to examine the relation between gender and appropriateness to employ the above teaching technique (yes vs. no). The relation between these variables was not significant (χ²(1) = 0.42, p = 0.52) suggesting that there is not a gender preference in teachers for employing these techniques. A chi‐square test of independence was performed to examine the relation between designation (FRCPC vs. CCFP‐EM) and appropriateness to employ the above teaching technique (yes vs. no). The relation between these variables was not significant (χ²(1)  = 0.08, p = 0.78), also again suggesting that training route may not directly affect this finding.

DISCUSSION

Our results demonstrate a trend toward teaching this skill set later in practice and specifically using the conversational and experiential techniques later in a resident’s training. Overall, there seems to be less consensus (as suggested by the lower ICC of 0.77) for observational techniques and when these should be employed. However, there seems to be consensus that there are certain types of conversational and experiential techniques that are most consistently endorsed for senior trainees in the latter parts of their training (Core and Transition to Practice periods) as suggested by the much higher ICCs. This is consistent with our experiences as teaching faculty for junior and senior residents. In general, EM training tends to focus on teaching clinical management of the individual patients earlier in residency, and then as a trainee demonstrates competence with individual patients, they are provided with increasing responsibility for multiple patients or nonclinical tasks.19 This is reflected by the low percentage of respondents indicating that they would teach about flow and management in Transition to Discipline or Foundations of Discipline. As residents demonstrate competency, staff physicians increasingly discuss flow and management in the Core of Discipline stage, as is reflected by the significant increase in endorsements of conversational teaching strategies for that stage.

Endorsement for the use of experiential teaching strategies increased significantly in Core of Discipline and for most techniques was even more strongly endorsed in the final stage of training, Transition to Practice. As clinicians gain more experience they develop more advanced heuristics enabling them to more efficiently flow patients in the ED.17, 20, 21 This facet is important to factor in when trying to teach flow and efficiency to trainees.14 As previously described in the literature, this increase in responsibility occurs in a graduated fashion.15 Often a senior resident who has demonstrated competency will be given the task of managing a section of the department or junior learners to supervise. As they progress, these responsibilities and teaching strategies are combined or a more advanced strategy is utilized.

Interestingly, there was more agreement over which advanced techniques to use (often allocated to later in training) than there was for simpler techniques like didactic or observational techniques. This suggests that at this present time, there is high variability across the country as to when and how these important aspects of work are taught. More exploration of other advanced skills in a similar manner will be of use in assisting our nation’s transition into a full implementation of Competence by Design.

Patients First, Flow Later

Effectively, our study essentially defines a predictable pattern that our Canadian EM educators highlight that best aligns with our CBME model in Canada (Competence by Design). Consistent with the core tenets of CBME,22 the use of these ED flow teaching strategies will need to be customized from resident to resident depending on their stage in training. That is to say, if a resident who is a junior in terms of their years of training has demonstrated significant competency in managing individual patients, they may be introduced to ED management before their peers or even before more senior residents who are struggling with individual patient management.

We may foresee that with our study’s proposed stratification of the various teaching strategies for teaching managerial skills, some residents may want to learn about these techniques earlier than staff have indicated their willingness to teach them. We imagine that this paper might serve as a touchstone to open conversations during faculty development to raise the capacity of faculty members to harness the power of the aforementioned teaching strategies. Also, when seeking to take these skills to the clinical environment, we highlight that the application of these educational strategies should be left up to individual trainee–trainer dyads to negotiate when it is safe and appropriate to employ each of these educational strategies. After all, depending on the clinical context, it may not always be safe for trainees to attempt assuming a managerial role due to patient acuity or volume concerns.

This approach often starts as a discussion of how one would manage flow on a particularly busy shift with a more junior trainee and then allow the trainee to start taking on more and more of the responsibility of managing the whole department.17 The way to manage flow may also vary from site to site depending on what resources are available within the ED, what existing protocols exist to help manage flow when near or at capacity, and what levers are available to use when the ED has exhausted or is near exhausting internal resources.

“Thrown Into the Deep End”: A Never Event?

Given that it is ultimately the responsibility of the supervising attending emergency physician to ensure safe and timely care in the ED for all patients, it is not surprising that many faculty members surveyed in our study feel that “throwing the trainee into the deep end” without any preparation is not an appropriate strategy to teach flow and other management skills. Managing flow is often complex and has significant regional/site‐based nuances and so some orientation to strategies and expectations is important prior to giving the trainee the opportunity to try managing the ED on their own.

Allowing the trainee to flounder in an ED managerial role may put patients at risk, especially if there are significant delays in management. Interestingly, another technique that had fewer endorsements also the “given the reins” technique, where the attending places the resident in charge. The low level of endorsement for this fairly independent task may also suggest that there is need for a more explicit curriculum with accompanying faculty development to ensure there is national consistency regarding the skill of ED management. A graduated approach as outlined above, managing a section of the ED, before being responsible for the whole ED is often a better and safer approach.14, 15

All things considered, at some point in a trainee’s final period (e.g., the Transition to Practice period), it may be worthwhile to find ways to create supportive environments where nearly qualified senior trainees are effectively thrown into the mix of a shift. Whereas it may be inappropriate to have a purely laissez faire or minimalist approach to supervision earlier in training,18 during the aptly named Transition to Practice period of training senior trainees must have a way to advance and test their skills. We must develop supportive systems that could empower trainees to safely transition into more usable roles. Such systems might allow some sort of more remote supervision, wherein faculty members might (at a short distance) keep an eye on the status of the ED and intervene either when the trainee asks for help or when there are emerging concerns about their ability to function in their leadership role. As with most experiences, debriefing complicated situations for senior trainees will better prepare them for their entry into practice.

As we move into the full implementation of the Competence by Design program, it is paramount that we continue to both teach and assess the skills that will be eventually required even in the later stages of training. Clearly there is need for a more consistent and harmonized set of teaching strategies for teaching ED flow nationally, to prepare trainees for the eventual goal of completing their Transition to Practice EPAs and enter into unsupervised practice. It is recommended that program directors and competency committee directors begin examining how they might augment the programmatic EPA system with complimentary learning activities and tailor these to the trainees within the system. Only then will we be able to fulfill the promise of CBME. Moreover, it maybe incumbent on the educational leadership to examine alternative methods to teach these critical skills in didactic or simulated environments. One such technique is the GridlockED board game, which is a serious game that has been developed to specifically teach around ED flow management concepts.19 Experimenting with safe, classroom‐based platforms like GridlockED may hold great promise in teaching some of the critical skills to even the most junior trainees well before they are asked to perform them in the real ED.

Future Directions

As we plan ahead in the era of CBME, it is important for us to consider how skills like these can be made more explicit for both trainees and faculty members. If implemented correctly, the above teaching strategies might be employed within a workplace‐based curriculum that layers our proposed instructional methods upon existing competency‐based assessment tools to optimize the skill acquisition of senior trainees in this arena. If so, a natural next step would be to strategically bundle our stratified managerial skills instructional methods into a graduated curriculum that aligns to the RCPSC’s new Competency by Design model. Once deployed such a curriculum might be evaluated by examining the relative productivity of senior trainees (or new graduates) from the CBME model versus that of the previous training model. Within this line of reasoning, once we have the EPA data from different sites, one might imagine deploying a survey of the techniques listed within our paper among various sites with variable amounts of senior resident productivity to discern which techniques that either 1) their faculty had used with them or 2) the trainees recalled experiencing. This might reveal which techniques are optimal for training highly productive emergency physicians.

LIMITATIONS

The limitations of this study include the fact that it was still very early in the Competency By Design implementation during our data collection. This early timing may mean that many participants do not yet have a good mental model for when a particular teaching technique will best fit into the Competence by Design levels. As stated in previous work,10 substantive amounts of faculty development are likely still needed for teachers across the country to be comfortable with the new system. We had attempted to gather a cross‐section of educators across the country to ensure there was some generalizability to our findings, but due to the low number of respondents this meant that we were only harnessing the ideas from one or two individuals per site. In future studies, it may be worthwhile to select a smaller subset of sites to increase recruitment for this study.

CONCLUSIONS

In this study we demonstrate that while there is some variability in teaching around ED flow across Canada, our educators endorsements showed a trend toward faculty utilizing more didactic and observational teaching techniques early in residency and then progressing toward more experiential techniques in the senior stages of training. Our findings are consistent with a graduated plan to increase in responsibility as residents demonstrate competency and progress through their training. Our study results should inform faculty development around teaching managerial skills in the area of competency‐based medical education. This study presents an example of how educators might revalidate previously a derived framework of educational delivery (specifically around managerial skills) within the new Royal College of Physicians and Surgeons of Canada Competency by Design framework.

Supporting information

Data Supplement S1. Supplemental material.

AEM Education and Training 2021;5:19–27