Abstract
Background
Pediatric requirements include procedural skills training such as peripheral intravenous (PIV) catheter placement and bag–mask ventilation (BMV). Clinical experiences may be limited and temporally remote from scheduled teaching. Just‐in‐time (JIT) training prior to utilization can promote skill development and mitigate learning decay. Our objective was to assess the impact of JIT training on pediatric residents’ procedural performance, knowledge, and confidence with PIV placement and BMV.
Methods
Residents received standardized baseline training in both PIV placement and BMV during scheduled educational programming. Between 3 and 6 months later, participants were randomized and received JIT training for either PIV placement or BMV. JIT training included a brief video and coached practice, totaling <5 min. Each participant was videotaped performing both procedures on skills trainers. Blinded investigators scored performance using skills checklists. Pre‐ and postintervention knowledge was assessed using multiple‐choice and short‐answer items, and confidence was reported using Likert scores.
Results
Seventy‐two residents completed baseline training sessions: 36 were randomized to receive JIT training for PIV and 36 for BMV. Thirty‐five residents in each cohort completed the curriculum. There were no significant differences between the cohorts with regard to demographics, baseline knowledge, or prior simulation experience. JIT training was associated with improved procedural performance for PIV (median 87% vs. 70%, p < 0.001) and for BMV (mean 83% vs. 57%, p < 0.001). Results remained significant after using regression models to adjust for differences in prior clinical experience. Improvements in knowledge or confidence were not associated with JIT training in either cohort.
Conclusions
JIT training resulted in a significant improvement in resident procedural performance with PIV placement and BMV in a simulated environment. There were no differences in outcome with regard to knowledge or confidence. Future work might explore how the demonstrated benefit translates into the clinical setting.
INTRODUCTION
The Accreditation Committee for Graduate Medical Education (ACGME) mandates that pediatric residents receive sufficient training in various procedures including peripheral intravenous (PIV) catheter insertion, bag–mask ventilation (BMV), lumbar puncture (LP), and others. 1 Opportunities for pediatric trainees to learn these skills within the clinical setting may be limited; therefore, procedural instruction is often provided through educational conferences, workshops, and simulation‐based training. However, these approaches are conventionally prescheduled and may occur well before the infrequent clinical encounters when they are performed. Given that procedural skills can decay after a period of nonuse, training that is timed closer to the clinical need may be valuable. 2 , 3
Just‐in‐time (JIT) training is an instructional technique that provides timely, on‐demand education to learners. It aims to meet the trainee's educational needs closer to the opportunity to utilize the relevant skill. Using JIT training as an adjunct to traditionally scheduled educational sessions may optimize learning. When applied to procedural skill training, this may translate to improved clinical performance, bolster confidence, and help mitigate patient safety concerns. 4 Previous studies assessing the impact of JIT training across various clinical procedures have had mixed results. 1 , 5 , 6 Rigorously designed studies that assessed the impact of JIT training on procedures with low overall proficiency (LP and intubation) by residents as measured by procedural success on live patients showed no benefit to the intervention. However, a simulation‐based study using a checklist to measure medical student performance on a procedure with high overall success (splinting) did find benefit. These differences in results may be related to the difficulty of the procedure, or the level of the learners, or may reflect the inherent challenges of controlling for the range of factors that may influence clinical outcomes. In addition, all of the JIT interventions in these studies were relatively long. We aimed to evaluate JIT training on procedural skills known to have higher success rates in the simulated setting that are a required part of training for pediatric residents and that include both relatively common and potentially lifesaving procedures. We chose to measure performance using procedural checklists to allow for granular assessment of impact, and used a controlled simulated setting to help isolate the effect of the intervention. In addition, we were deliberate in keeping the educational intervention brief to potentially facilitate utilization in the future. Our specific study objective was to determine the impact of a novel approach to brief JIT training on pediatric resident performance on PIV insertion and BMV in a simulated setting.
METHODS
Study design
This was a prospective, randomized controlled educational trial of a JIT procedural training curriculum in a simulated setting. We followed CONSORT guidelines for simulation‐based studies. 7 The study was approved by the residency program research committee as well as the hospital institutional review board.
Study setting and population
The study was performed within dedicated conference time at a large pediatric residency program at a quaternary care children's hospital (Boston Children's) with approximately 450 inpatient beds. The study took place from January through August 2021.
All 165 residents within the residency program, across all years of training, were invited by email to participate in the study. Prior program feedback had encouraged promoting additional training around procedures, and our included procedures were identified as relevant to all levels of residents. Participation was voluntary. Residents who agreed to participate were invited to attend baseline training sessions in both PIV placement and BMV. Completion of both baseline training sessions was required to be eligible for inclusion in the study. Written consent was obtained from all participants.
Study protocol
Overview of the curriculum
The total instructional and assessment time was standardized for all participants, including 60‐min baseline instructional sessions in PIV insertion and BMV, a single <5‐min JIT training intervention in one of the two included procedures, and a follow‐up assessment session (Figure 1).
FIGURE 1.
Study flow diagram. BMV, bag–mask ventilation; JIT, just‐in‐time; PIV, peripheral intravenous (catheter).
Baseline assessment and instructional sessions
Prior to any educational intervention, residents completed a baseline questionnaire collecting relevant demographic information (age, sex, level of training, prior simulated and clinical experience performing PIV insertion and BMV) as well as a preparticipation knowledge assessment for each of the two procedures. Items on the knowledge assessment were selected from previously published and validated assessments. 8 Question selection and modification for the study setting and population were performed by panels of experts including leaders of the hospital vascular access team and faculty from a national airway management course for the two procedures respectively. Preliminary versions of the knowledge assessments were piloted among nonresident trainees. The final 10‐item knowledge assessment questions for each procedure were approved by consensus among the experts and study investigators.
All participants received standardized baseline procedural training for the two procedures during separate 1‐h educational sessions with a trained facilitator. Each session included completion of the preparticipation knowledge assessment for the respective procedure that day, followed by didactic instruction. Teaching materials were created using an iterative process by study investigators in collaboration with the aforementioned experts and included content related to indications, contraindications, technique, and complications of each procedure. The educational sessions included integrated video segments on PIV or BMV, spliced from the New England Journal of Medicine (NEJM) Videos in Clinical Medicine series. 9 , 10 Following didactic instruction, participants had dedicated time for supervised, hands‐on practice with a trained facilitator for 20–30 min using a vascular access task trainer (Laerdal Medical), or infant airway management trainer (Laerdal Medical), until they demonstrated competence, defined as supervised performance of all the steps included on procedure checklists. Similar to the creation of the study knowledge assessments, previously published and validated procedural skills checklists were identified and adapted for physician trainees by the expert panels. 8
Randomization and JIT training
Residents who completed baseline training in both procedures were randomized into one of two cohorts. Each participant was assigned a unique random number from 1 to 82, using a true random‐number generator tool (https://www.random.org). Those assigned a number from 1 to 41 were designated to receive PIV JIT training (Cohort A) and those assigned numbers from 42 to 82 were designated to receive BMV JIT training (Cohort B). Three to six months after completion of the baseline training, Cohort A received JIT training on PIV but not on BMV, and Cohort B received JIT training on BMV but not on PIV (see Figure 1). JIT sessions for each procedure were composed of 1–2 min of condensed highlights from the video clips used in the didactic portion of the baseline training sessions, followed by a 1‐min standardized, scripted review of core content regarding the relevant procedure. Included teaching points were developed by consensus from study investigators. After the brief review, participants were allowed 1–2 min of supervised practice with real‐time feedback on the relevant procedural task trainers. JIT training intervention sessions for each procedure were limited to a maximum of 5 min.
All facilitators for the JIT procedural training session participated in a “train‐the‐trainer” session, which included instruction on the curricular learning objectives, orientation to the task trainers, and a review of the checklists including the individual steps of each procedure. A teaching script was developed by the study investigators and formed the basis of instruction to standardize the teaching across each session.
Assessment
Immediately following the JIT training, participants from both cohorts were videorecorded performing both procedures (placing a PIV and performing BMV) on the same task trainers used during the baseline training sessions. After completing the two procedures, residents completed surveys assessing their retrospective preparticipation confidence as well as their postparticipation contemporaneous confidence using a 4‐point Likert scale (from 1 = not confident to 4 = extremely confident). The use of a retrospective preassessment has been shown to be a sensitive and valid measure of educational curricular impact, particularly among novice learners. 11 , 12 Study subjects also completed postparticipation knowledge assessments addressing both procedures, identical to the instruments administered in the preparticipation assessment. The postparticipation survey also included open‐ended questions regarding the perceived value of participation in baseline and JIT training sessions.
Data collection
Following training sessions, study investigators, blinded to randomization and JIT assignment, scored videos of participants’ performance for each procedure using the study procedural checklists. To standardize data extraction from video recordings, pilot assessments were performed with study investigators scoring the first five participants independently and then meeting to agree on approaches and ensure standardization around applied definitions for each item on the checklist. In scoring on all subsequent cases, where performance of any individual item was unclear, study investigators reconvened to reach scoring consensus. Time to completion of the procedure was measured directly from the recorded videos. Data from checklists, knowledge assessments, and confidence surveys were all entered into REDCap. 13
Primary and secondary outcome measures
The primary outcome was procedural performance with PIV insertion and BMV, measured as the percentage of checklist items performed correctly for each of the two procedures. Secondary outcomes included time to successful completion of each procedure, performance on the knowledge‐based assessments, and self‐reported confidence. Successful completion of PIV insertion was defined as flash of blood return followed by appropriately securing IV access site. For BMV, completion was measured as the delivery of three effective breaths as determined by chest rise in the airway trainer. Open‐ended comments regarding the value of the procedural curriculum were also collected and analyzed.
Data analysis
Descriptive statistical analyses were performed on baseline demographic and experiential variables. We reported frequencies and proportions for categorical variables (e.g., sex, level of training) and medians with interquartile ranges for continuous measures (e.g., age, previous experience with PIV placement and BMV).
To assess for differences in baseline demographic factors between the cohorts after randomization, we used chi‐square tests and equality‐of‐medians tests for categorical and continuous variables, respectively. PIV and BMV procedural performance scores and time to completion for each procedure were measured at the 3‐ to 6‐month follow‐up assessment and were compared between Cohorts A and B using equality‐of‐median tests. Quantile regression models were used to evaluate these outcomes with adjustment for baseline factors that differed between cohorts.
To evaluate change in PIV and BMV knowledge score assessments from preparticipation (baseline) to the postparticipation (follow‐up), we created delta measures where positive values indicate an increase in knowledge. We compared these knowledge delta scores between Cohorts A and B using equality‐of‐median tests. We similarly created delta scores for the baseline and follow‐up confidence scores, where positive scores indicate an increase in confidence over time. However, the limited distribution of the confidence delta scores (i.e., 94% of scores were 0 or 1 with a range of −1 to 2) precluded a meaningful analysis as a continuous variable. Thus, we dichotomized these delta scores (≤0 vs. >0) to create a binary indicator of confidence improvement. Comparison of improvement in confidence across the cohorts was performed using chi‐square.
We aimed to enroll 40 participants per group, for a total sample size of 80 (i.e., assuming an approximate ascertainment rate of 50% out of the 165 eligible residents). For our power calculation, we used the PIV checklist as the primary outcome given previous existing data for similar learners for that procedure. 8 Assuming an alpha level of 0.05, a total n of 80, equally sized groups, and an overall mean baseline performance score of 65, we had 98% power to detect a 15% mean difference between cohorts (corresponding to a difference of approximately three items). We were also adequately powered (≥94%) to detect this effect assuming smaller sample sizes (total n as low as 60) that could result from lower‐than‐expected recruitment and/or loss to follow‐up. Statistical analyses were performed using Stata Version 16 (StataCorp LP). All tests were two‐tailed and alpha was set at 0.05.
For open‐ended responses, qualitative analysis was performed using a framework approach. 14 Two of the investigators (J.M. and J.N.) used open coding to organize responses into themes. Serial comparison was used and refinement of the coding structure continued until there was agreement on the final categorization of the data.
RESULTS
Of 165 residents in the program, 82 agreed to participate. Seventy‐two completed both baseline training sessions and made up the study population. Thirty‐six were randomized to receive JIT training for PIV (Cohort A) and 36 to receive JIT for BMV (Cohort B). There were no statistically significant differences between cohorts in terms of demographics and no difference in experience in clinical or simulated PIV placement experience (see Table 1). The cohort receiving JIT BMV had greater baseline clinical experience with mask ventilation (p = 0.05).
TABLE 1.
Participant demographics and prior experience.
PIV JIT training (n = 35) | BMV JIT training (n = 36) | p‐value a | |
---|---|---|---|
Sex | |||
Female (%) | 22 (65.7) | 26 (75.0) | 0.39 |
Age (years) | 29 (28–30) | 29 (28–31) | 0.69 |
Year in residency training | 0.58 | ||
PGY‐1 (%) | 54.3 | 41.7 | |
PGY‐2 (%) | 25.7 | 33.3 | |
PGY‐3 (%) | 20.0 | 22.2 | |
PGY‐4 (%) | 0.0 | 2.8 | |
Prior PIV experience (No. of times) | |||
Simulation | 0 (0–2) | 0.5 (0–2) | 0.72 |
Actual | 0 (0–1) | 0 (0–2) | 0.65 |
Prior BMV experience (No. of times) | |||
Simulation | 3 (2–4) | 3 (2–5) | 0.52 |
Actual | 1 (0–3) | 3 (1–7.5) | 0.07 |
Note: Values in table represent frequency (%) or median (IQR).
Abbreviations: BMV, bag–mask ventilation; JIT, just‐in‐time; PIV, peripheral intravenous (catheter).
p‐value derived from chi‐square test for categorical variables and equality‐of‐median test for continuous variables.
Cohort A, randomized to receive JIT training for PIV, had a higher median PIV checklist performance score compared to Cohort B, who were randomized to JIT BMV training (median 90% vs. 73%, p < 0.001; see Table 2). Similarly, JIT BMV training was associated with higher procedural performance on the BMV checklist, compared to Cohort A who received PIV JIT training (83 vs. 58%, p < 0.001). This difference remained significant after adjustment for baseline differences across the two cohorts in clinical experience with BMV.
TABLE 2.
Procedural performance on PIV and BMV by JIT training.
PIV JIT training (n = 35) | BMV JIT training (n = 36) | p‐value | |
---|---|---|---|
PIV checklist score (%) | |||
Median | 90 | 73 | <0.001 |
IQR | 40–100 | 10–95 | |
BMV checklist score (%) | |||
Median | 58 | 83 | <0.001* |
IQR | 29–92 | 54–96 |
Remains significant in regression model after adjusting for prior BMV experience.
Abbreviations: BMV, bag–mask ventilation; JIT, just‐in‐time; PIV, peripheral intravenous (catheter).
JIT training for BMV was associated with a shorter median time to delivery of successful BMVs by more than 20 s (49 vs. 72 s, p < 0.05; see Table 3). No differences were observed between the cohorts for other secondary outcomes, including change in knowledge or confidence.
TABLE 3.
Secondary outcomes: time to successful procedure completion, change in knowledge, and improvement in confidence.
PIV JIT training (n = 35) | BMV JIT training (n = 36) | p‐value | |
---|---|---|---|
Time to successful completion of procedures (s), median (IQR) | |||
PIV | 495 (311–899) | 472 (277–805) | 0.98 |
BMV a | 72 (24–177) | 49 (22–110) | 0.02 |
PIV knowledge, % correct (IQR) | |||
Pretest | 60 (50–70) | 60 (50–70) | |
Posttest | 70 (50–80) | 70 (55–80) | |
Delta b | 10 (−10, 30) | 0 (−10, 0) | 0.41 |
BMV Knowledge % correct, (IQR) | |||
Pretest | 53 (40–60) | 53 (40–60) | |
Posttest | 61 (50–70) | 64 (55–70) | |
Delta b | 10 (0–10) | 10 (0–20) | 0.14 |
Confidence c | |||
Improved PIV confidence, % of participants | 74 | 54 | 0.08 |
Improved BMV confidence, % of participants | 54 | 69 | 0.18 |
Note: Values in table represent median (IQR).
Abbreviations: BMV, bag–mask ventilation; JIT, just‐in‐time; PIV, peripheral intravenous (catheter).
Defined as time to complete three BMV breaths.
The difference between pre‐ and postparticipation.
Change between retrospective pre‐ and postparticipation self‐assessment of confidence.
Responses to the open‐ended questions regarding perceptions of the JIT training curriculum aligned with three categories: content, educational approach, and timing and environment (see Table 4). Residents expressed overwhelmingly positive sentiments following their participation in the curriculum and with regard to the value of JIT training. Common themes included appreciation of “hand‐ on practice,” with “feedback” and timing “prior to immediate implementation of the procedure.”
TABLE 4.
Themes from qualitative analysis of open‐ended questions.
Themes | Examples |
---|---|
Content | |
Steps of procedure |
“Breaking down procedure step by step” “Reminded me of order and appropriate steps” |
Equipment review |
“Reviewing necessary equipment” “Gives you a chance to familiarize yourself with equipment” |
Troubleshooting |
“Remember troubleshooting mechanism” “Running through … pitfalls prior to procedure” |
Educational approach | |
Feedback |
“Real‐time feedback” “Practice with feedback” |
Hands on |
“Hands‐on practice, visual and muscle memory” “The in‐person demo and guidance” |
By experts |
“The review with the fellow” “Experienced people breaking down procedure” |
Timing and environment | |
Controlled setting |
“Practicing without pressures of a case/patient” “Having another opportunity to practice in a controlled environment” |
Timing |
“Being able to review … prior to immediate implementation” “Reviewed the procedure in timely manner” |
DISCUSSION
Developing and maintaining procedural skills in pediatrics is challenging given limited clinical opportunities, and training is often remote from the opportunity to perform the procedure. We found that a brief JIT educational intervention immediately before attempts at PIV insertion and BMV was associated with improved procedural performance and, in the case of BMV, decreased the time needed to achieve effective ventilation by more than 20s. While we did not see similar effects on knowledge or confidence, the impact on procedural performance is most valuable to clinical practice. Importantly, the approach to JIT training used in this curriculum using a brief integrated video and hands‐on coached practice took less than 5 min and could be easily implemented in most learning environments.
Prior studies evaluating JIT training for procedural instruction for pediatric trainees have had mixed results. Some studies looked at the effect of JIT training on clinical performance in the care of actual patients. In a methodologically rigorous, large, multicenter study of JIT for LP, the opportunity to receive 5–10 min of coached practice on a simulator immediately before an infant LP was not associated with higher success, although improvements in some process measures (e.g., use of analgesia and removal of the stylet) were identified. 6 Importantly, LP success rates were relatively low in both the control and the intervention groups (35% and 38%, respectively) based on study definitions and could be affected by a number of factors beyond the JIT intervention. 6 A single‐center, prospective interventional study assessed the effect of JIT training on first‐pass success during endotracheal intubation among pediatric residents in the intensive care unit. Given the often emergent timing for endotracheal intubation, the 30 min of simulation‐based multidisciplinary training with a skills refresher for residents was performed at the start of their on‐call shift rather than immediately prior to the procedural attempt. Investigators found no improvement in first‐attempt success or adverse events related to the intervention. 1 A study assessing JIT training related to orthopedic splint application evaluated medical student performance immobilizing a standardized patient after watching a 10‐min JIT training video or using designated time for self‐study with a textbook. Those receiving JIT training achieved higher performance measured using a procedural checklist and had faster preparation time but did not have faster procedural completion time. 5 Two other JIT studies used simulation‐based outcomes, similar to our study. A study of students and residents using a survey methodology assessed a JIT intervention for intraosseous needle insertion and defibrillation. Investigators found improved comfort and knowledge regarding these two critical procedures after participation; however, no assessment of procedural performance was included. 15 Finally, a recent prospective observational cohort study in a tertiary care pediatric hospital used a unique design for JIT. They delivered a standardized JIT curriculum for trainees participating in procedural sedation on actual patients during their ED rotation and later assessed them on their response to simulated sedation scenarios complicated by apnea and desaturation. The JIT intervention was associated with improvement in residents’ clinical performance and confidence in preparing for and responding to adverse events. 16 The differences in impact across these studies may be related to a number of factors including assessment using actual versus simulated patients, the complexity and acuity of the procedure(s) being studied, the approach to JIT training, and the outcomes being measured.
We were deliberate in the two ACGME‐mandated procedures we chose to include in our study. PIV insertion is fundamental to the care of pediatric patients and is often performed in nonemergent circumstances. In contrast, BMV is performed less frequently but is a lifesaving and often urgently required intervention. The positive impact of the JIT training across the disparate clinical contexts in which our included procedures are performed suggests our approach may have generalizability across a range of procedures in pediatrics.
We also aimed to model a new approach to JIT, with a combined intervention including brief didactic recap, an abbreviated version of a previously viewed instructional video, and coaching on a trainer. Our approach was based on a number of educational principles. The aim of JIT is to address procedural skill decay that occurs after initial learning. 17 In addition, multimodal learning including both didactic and video‐based instruction takes advantage of dual‐channel processing, which has been shown to be effective in procedural instruction. 18 And finally, the hands‐on portion was included to model a growing focus on coaching within graduate medical education. 19 Importantly, our bundle of educational interventions was still performed in less than 5 min and was easily achieved with only a short investment in faculty development for the included instructors.
We were deliberate in choosing a primary outcome related to simulated clinical performance given our ultimate goal of assessing procedural competence. However, we also chose outcomes focused on knowledge, confidence, and attitudes following JIT training. As described previously, prior studies have had mixed results using different outcomes. Therefore, we aimed to broadly assess the impact of JIT training. The majority of participants had an increase in knowledge following participation in the curriculum; however, the JIT training did not appear to influence that outcome. This may be because the greater impact occurred from the more extensive baseline training making it more difficult to detect a difference from the shorter JIT effort, which did not address all items on the knowledge assessment. Similarly, the majority of participants reported higher confidence in performing the procedures following completion of the curriculum. It may be that challenges encountered during the hands‐on portion of the JIT intervention reminded participants of possible gaps in their procedural skills. Our study may also have been underpowered to assess differences in these secondary outcomes. We also recognize that procedural confidence does not always equate with competence. 20 , 21 Although improved confidence can be helpful in overcoming psychological barriers to performing procedures, that may not translate into procedural success. According to the Dunning–Kruger effect, those with relatively lower skill and experience may be at greater risk of overestimating their own capabilities. 22 Given the range of prior experience and procedural performance for participants in this study, this tendency could be relevant for some of our study participants.
While the primary aim of this study was to assess the effectiveness of JIT training as an approach to teaching pediatric residents two procedural skills, the curriculum was designed with deliberate intent of filling in perceived or recognized gaps in procedural training within pediatric residency programs. 23 We used our qualitative analysis of the open‐ended comments to assess the broader impact of the JIT procedural curriculum. We found that the controlled setting and the proximal timing as well as the opportunity for directed feedback are important to the learners. These insights should be considered in future implementation of JIT procedural training.
LIMITATIONS
There were a few limitations to our study. Although we used randomization, we found a difference between cohorts in clinical experience with BMV at entry into the study. However, when adjusting for this difference we still detected significant differences between groups in our primary outcomes. Second, for logistical reasons, the time to intervention was not precisely standardized for learners. We used a range of 3–6 months to allow for sufficient time for potential skill decay but recognize that the impact may be different at 3 months versus 6 months. The final assessment tools we used were not formally validated. The PIV checklist we used was modified from a previously published version with demonstrated reliability and validity and then modified slightly to match our learners and context. Our BMV checklist was compiled using content from previously published studies, created by expert consensus, and piloted, but it has not been formally validated. During our data extraction, we used investigator consensus to address scoring questions but did not separately measure interrater reliability. Additionally, our study was performed in a simulated environment. We cannot comment on how our findings would translate into the clinical environment and patient‐based outcomes, nor can we speak to how performance in the study setting would translate into external measures of competence as defined by the ACGME. However, we are confident our measurement instruments are able to effectively measure changes in performance resulting from our JIT educational intervention. Finally, our study participants were residents from a large residency program in a single‐center, quaternary care facility, which may limit the generalizability of our findings.
CONCLUSIONS
In summary, a brief just‐in‐time educational intervention, which re‐exposed residents to prior procedural training on peripheral intravenous catheter placement and bag–mask ventilation improved their procedural performance on both procedures and improved the time to successful delivery of bag–mask ventilation. Future studies might assess the benefits of just‐in‐time training for additional pediatric procedures and how the impact translates to clinical practice.
AUTHOR CONTRIBUTIONS
Jheanelle McKay and Joshua Nagler conceived the study. Jheanelle McKay, Alexander W. Hirsch, Mollie Wasserman, and Joshua Nagler designed the trial, supervised the conduct of the trial and data collection, recruited participants, and managed the data, including quality control. Michael C. Monuteaux provided statistical advice on study design and analyzed the data. Jheanelle McKay drafted the manuscript, and all authors contributed substantially to its revision. Each author has seen and approved the submission of this version of the manuscript and takes full responsibility for the manuscript.
CONFLICT OF INTEREST STATEMENT
The authors declare no conflicts of interest.
McKay J, Wasserman M, Monuteaux MC, Hirsch AW, Nagler J. Just‐in‐time procedural training for pediatric emergency medicine trainees: A randomized educational interventional trial. AEM Educ Train. 2023;7:e10886. doi: 10.1002/aet2.10886
Funding informationThis study was supported by a Boston Children's Hospital, Division of Emergency Medicine Michael Shannon Research Award.
Supervising Editor: Jaime Jordan
REFERENCES
- 1. Nishisaki A, Donoghue AJ, Colborn S, et al. Effect of just‐in‐time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit. Anesthesiology. 2010;113(1):214‐223. doi: 10.1097/ALN.0b013e3181e19bf2 [DOI] [PubMed] [Google Scholar]
- 2. Lammers RL, Davenport M, Korley F, et al. Teaching and assessing procedural skills using simulation: metrics and methodology. Acad Emerg Med. 2008;15(11):1079‐1087. doi: 10.1111/j.1553-2712.2008.00233.x [DOI] [PubMed] [Google Scholar]
- 3. Arthur W Jr, Winston B Jr, Day EA, McNelly TL. Skill Decay: A Comparative Assessment of Training Protocols and Individual Differences in the Loss and Reacquisition of Complex Skills. Defense Technical Information Center; 2002. doi: 10.21236/ADA417195 [DOI] [Google Scholar]
- 4. Thomas AA, Uspal NG, Oron AP, Klein EJ. Perceptions on the impact of a just‐in‐time room on trainees and supervising physicians in a pediatric emergency department. J Grad Med Educ. 2016;8(5):754‐758. doi: 10.4300/JGME-D-15-00730.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Wang VJ, Cheng YT, Liu D. Improving education: just‐in‐time splinting video. Clin Teach. 2016;13(3):183‐186. doi: 10.1111/tct.12394 [DOI] [PubMed] [Google Scholar]
- 6. Kessler D, Pusic M, Chang TP, et al. Impact of just‐in‐time and just‐in‐place simulation on intern success with infant lumbar puncture. Pediatrics. 2015;135(5):e1237‐e1246. doi: 10.1542/peds.2014-1911 [DOI] [PubMed] [Google Scholar]
- 7. Cheng A, Kessler D, Mackinnon R, et al. Reporting guidelines for health care simulation research. Simul Healthc. 2016;11(4):238‐248. doi: 10.1097/SIH.0000000000000150 [DOI] [PubMed] [Google Scholar]
- 8. Keleekai NL, Schuster CA, Murray CL, et al. Improving nurses' peripheral intravenous catheter insertion knowledge, confidence, and skills using a simulation‐based blended learning program. Simul Healthc. 2016;11(6):376‐384. doi: 10.1097/SIH.0000000000000186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Ortega R, Sekhar P, Song M, Hansen CJ, Peterson L. Peripheral intravenous cannulation. N Engl J Med. 2008;359(21):e26. doi: 10.1056/NEJMvcm0706789 [DOI] [PubMed] [Google Scholar]
- 10. Ortega R, Mehio AK, Woo A, Hafez DH. Positive‐pressure ventilation with a face mask and a bag‐valve device. N Engl J Med. 2007;357(4):e4. doi: 10.1056/NEJMvcm071298 [DOI] [PubMed] [Google Scholar]
- 11. Hoogstraten J. The retrospective pretest in an educational training context. J Exp Educ. 1982;50(4):200‐204. doi: 10.1080/00220973.1982.11011824 [DOI] [Google Scholar]
- 12. Howard GS, Ralph KM, Gulanick NA, Maxwell SE, Nance DW, Gerber SK. Internal invalidity in pretest‐posttest self‐report evaluations and a re‐evaluation of retrospective pretests. Appl Psychol Measur. 1979;3(1):1‐23. doi: 10.1177/014662167900300101 [DOI] [Google Scholar]
- 13. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata‐driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377‐381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Pope C. Qualitative research in health care: analysing qualitative data. BMJ. 2000;320(7227):114‐116. doi: 10.1136/bmj.320.7227.114 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Itoh T, Lee‐Jayaram J, Fang R, Hong T, Berg B. Just‐in‐time training for intraosseous needle placement and defibrillator use in a pediatric emergency department. Pediatr Emerg Care. 2019;35(10):712‐715. [DOI] [PubMed] [Google Scholar]
- 16. Schinasi DA, Colgan J, Nadel FM, Hales RL, Lorenz D, Donoghue AJ. A brief, just‐in‐time sedation training in the pediatric emergency department improves performance during adverse events encountered in simulated procedural sedations. Pediatr Emerg Care. 2022;38(3):e1030‐e1035. doi: 10.1097/PEC.0000000000002657 [DOI] [PubMed] [Google Scholar]
- 17. Wang EE, Quinones J, Fitch MT, et al. Developing technical expertise in emergency medicine‐the role of simulation in procedural skill acquisition. Acad Emerg Med. 2008;15(11):1046‐1057. doi: 10.1111/j.1553-2712.2008.00218.x [DOI] [PubMed] [Google Scholar]
- 18. Clark JM, Paivio A. Dual coding theory and education. Educ Psychol Rev. 1991;3(3):149‐210. doi: 10.1007/BF01320076 [DOI] [Google Scholar]
- 19. Sawyer T, White M, Zaveri P, et al. Learn, see, practice, prove, do, maintain. Acad Med. 2015;90(8):1025‐1033. doi: 10.1097/ACM.0000000000000734 [DOI] [PubMed] [Google Scholar]
- 20. Slubowski D, Wagers B, Kanis J. Characterization of pediatric procedural competency in emergency physicians. Int J Crit Care Emerg Med. 2019;5(4):4‐6. doi: 10.23937/2474-3674/1510083 [DOI] [Google Scholar]
- 21. Miller KA, Monuteaux MC, Roussin C, Nagler J. Self‐confidence in endotracheal intubation among pediatric interns: associations with gender, experience, and performance. Acad Pediatr. 2019;19(7):822‐827. doi: 10.1016/j.acap.2019.06.013 [DOI] [PubMed] [Google Scholar]
- 22. Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self‐assessments. J Pers Soc Psychol. 1999;77(6):1121‐1134. doi: 10.1037/0022-3514.77.6.1121 [DOI] [PubMed] [Google Scholar]
- 23. Schumacher DJ, Frintner MP, Cull W. Graduating pediatric resident reports on procedural training and preparation. Acad Pediatr. 2018;18(1):73‐78. doi: 10.1016/j.acap.2017.08.001 [DOI] [PubMed] [Google Scholar]