Emergency Information Forms for Children With Medical Complexity: A Simulation Study

Abstract

BACKGROUND:

Emergency information forms (EIFs) have been proposed to provide critical information for optimal care of children with medical complexity (CMC) during emergencies; however, their impact has not been studied. The objective of this study was to measure the impact and utility of EIFs in simulated scenarios of CMC during medical emergencies.

METHODS:

Twenty-four providers (12 junior, 12 experienced) performed 4 simulations of CMC, where access to an EIF was block randomized by group. Scenario-specific critical action checklists and consequential pathways were developed by content experts in simulation and pediatric subspecialists. Scenarios ended when all critical actions were completed or after 10 minutes, whichever came first. Two reviewers independently evaluated the video-recorded performances and calculated scenario-specific critical action scores. Performance in scenarios with and without an EIF was compared with Pearson’s χ² and Mann–Whitney U tests. Interrater reliability was assessed with intraclass correlation. Each provider rated the utility of EIFs via exit questionnaires.

RESULTS:

The median critical action score in scenarios with EIFs was 84.2% (95% confidence interval [CI], 71.7%–94.1%) versus 12.5% (95% CI, 10.5%–35.3%) in scenarios without an EIF (P < .001); time to completion of scenarios was shorter (6.9 minutes [interquartile range 5.8–10 minutes] vs 10 minutes), and complication rates were lower (30% [95% CI, 17.4%–46.3%] vs 100% [95% CI, 92.2%–100%]) with EIFs, independent of provider experience. Interrater reliability was excellent (intraclass correlation = 0.979). All providers strongly agreed that EIFs can improve clinical outcomes for CMC.

CONCLUSIONS:

Using simulated scenarios of CMC, providers’ performance was superior with an EIF. Clinicians evaluated the utility of EIFs very highly.

What’s Known on This Subject:

Children with medical complexity (CMC) are vulnerable during emergencies. Emergency information forms (EIFs) have been proposed to provide essential and timely information to emergency providers; however, their potential impact has not been studied.

What This Study Adds:

In simulated scenarios of CMC, access to an EIF improved outcomes independent of provider experience. Clinicians found them useful and desirable in the care of CMC. This evidence supports efforts for development and evaluation of EIFs in clinical settings.

Children with medical complexity (CMC) are at high risk of adverse events, medical errors, and poor outcomes during emergencies.^1–5 These patients often need unscheduled emergency care in both pediatric and general emergency departments (EDs)⁶ and account for an increasing proportion of hospitalizations and resource utilization.^3,4,7–10 They often receive multiple medications, may be dependent on medical devices, and have complicated management plans. In an emergency, it may be difficult for providers to efficiently gather the necessary information to diagnose and treat such complex pediatric patients.^2,11,12 There are many potential barriers to the delivery of timely, effective care to CMC. Parents may not be available, may not speak the provider’s language, or may not know the salient information to provide during emergencies. Subspecialists may not be available for contact, and medical records may be incomplete or disorganized. Furthermore, during an emergency or a disaster, the patient may present to an ED that does not have access to their medical records.^1,13–16

Patient advocacy groups, the American Academy of Pediatrics, and the American College of Emergency Physicians have called for the creation and dissemination of emergency information forms (EIFs) as a means to provide rapid access to a health summary containing the essential information needed to treat patients with special health care needs in an emergency.^{1,2,11,13,17–24} Yet there is paucity of research on the development, implementation, and effectiveness of EIFs.^1,20,25,26 There is no central repository for EIFs for patients or health care providers to quickly access in case of an emergency, and the creation and maintenance of EIFs are limited by concerns about patient privacy and a lack of coordination between primary care physicians and subspecialists. Additional concerns include reimbursement for these activities, medical record ownership, and variation in state regulations.¹⁶

This study is a component of a multidisciplinary research project that aims to develop, implement, and measure the impact of EIFs in emergencies involving CMC. In this study, we measured the impact of EIFs in simulated emergency scenarios involving CMC and evaluated providers’ opinions about the utility of EIFs in the emergency care of CMC.

Methods

This study was approved by the Institutional Review Board at Washington University School of Medicine. Participants were recruited via e-mail and announcements at meetings. Twenty-four pediatric providers volunteered to participate and signed an informed consent document. Participants were not informed about the goals of the study (ie, measuring the impact of an EIF) but were told that investigators were assessing factors that influence the care of CMC. Participants were not subjects in other institutional research efforts on EIFs. Study procedures involved performing 4 consecutive, high-fidelity simulations of emergency scenarios involving CMC. Twelve participants were senior (pediatric emergency medicine fellows, attending physicians, and pediatric hospitalists), and 12 were junior (second- and third-year pediatric residents). Each participant was randomly assigned by study group (junior versus senior) into 1 of 6 treatment groups where access to an EIF for each scenario was randomly predetermined (Table 1) and introduced in different order to adjust for potential learning effects of the simulation. Each participant performed 4 consecutive simulated scenarios and completed a questionnaire to evaluate the format, usefulness, and value of EIFs in the care of CMC presenting to the ED. At the end of the simulations, participants were offered a $15 gift card and were asked not to discuss the study with other people. All simulations were conducted at the Saigh Pediatric Simulation Center at St Louis Children’s Hospital with SimNewB (Laerdal Medical, Stavanger, Norway) and PediaSIM HPS (METI/CAE Healthcare, Sarasota, FL) high-fidelity, computer-driven and preprogrammed manikins, and all sessions were video recorded with the B-Line Medical system (B-Line Medical, Washington, DC). Videos were stored in a password-protected computer, accessible only to the principal investigator and second independent reviewer.

TABLE 1.

Block Randomization Scheme

Group Order	Practice (Diabetic Ketoacidosis)	Metabolic Emergency	Neurology Emergency	Cardiac Emergency
A	Ø	+ EIF	Ø	Ø
B	Ø	+ EIF	Ø	+ EIF
C	Ø	+ EIF	+ EIF	Ø
D	Ø	Ø	Ø	+ EIF
E	Ø	Ø	+ EIF	+ EIF
F	Ø	Ø	+ EIF	Ø

Ø, no EIF; +EIF, emergency information form provided.

Scenario Design

The scenarios were designed to represent realistic cases of patients with medically complex conditions. They were preprogrammed and included consequential pathways with vital sign trends and predetermined outcomes, including adverse events and complications, based on participant actions. Participants were briefed and oriented to the room and manikin in a systematic fashion. They were expected to verbalize or to perform actions by using all equipment available. The first scenario, a diabetic ketoacidosis case, served as practice to acclimatize participants to the manikin and simulation environment and was not scored. The subsequent scenarios were scored and included metabolic (pyruvate dehydrogenase deficiency), neurologic (pyridoxine-dependent epilepsy), and cardiac (supraventricular tachycardia with aberrancy) cases. All scenarios were developed with simulation experts and pediatric subspecialists (cardiology, genetics, and neurology). Critical action checklists with action-weighted scores were developed for each scenario. The study team created scenario-specific EIFs by using a modified American Academy of Pediatrics/American College of Emergency Physicians EIF template.

Outcomes

The primary outcome, the critical action score (CAS), was calculated by assessing the percentage of correctly completed items of the scenario-specific critical action checklist. Two investigators (G.A. and J.F.) independently viewed and scored the video-recorded scenarios. Secondary outcomes were the time to completion of all critical actions (if critical actions were incomplete, simulations were terminated after 10 minutes), the rate of scenario-specific adverse events or complications as predetermined in the consequential pathways programmed in the manikin, and participant evaluation ratings of the utility and value of EIFs via exit interviews.

Analysis

Continuous data were summarized as means with SDs and medians with interquartile ranges (IQRs) for normally and non–normally distributed data, respectively. Categorical data were described as proportions with 95% confidence intervals (CIs). We used the Mann–Whitney U test to compare differences in CAS and time to completion or termination of scenarios (ie, when all critical actions were completed or scenario was terminated after 10 minutes) and the Pearson’s χ² test to compare differences in rates of adverse events or complications between the scenarios performed with and without an EIF. Interrater reliability of the CAS between the 2 independent reviewers was measured via intraclass correlation analysis in a 2-way mixed model.²⁷ Finally, exit interview items used a 5-point Likert scale questionnaire to measure participants’ level of agreement with statements about the utility and value of EIFs for CMC; higher scores indicated stronger agreement with each statement. We used SPSS version 22 (IBM SPSS Statistics, IBM Corporation) for Windows for all analyses.

Results

The scenarios with the EIFs had a median CAS of 84.2% (95% CI, 71.7%–94.1%), compared with CAS of 12.5% (95% CI, 10.5%–35.3%) in the scenarios without an EIF (Table 2). The median time to completion was at 6.9 minutes (IQR 5.8–10 minutes) with an EIF compared with 10 minutes (maximum time allotted per scenario) without an EIF. The group with the EIF had a 30% adverse event or complication rate (95% CI, 17.4%–46.3%), compared with 100% (95% CI, 92.2%–100%) in the group without the EIF. In subgroup analyses, there were no differences in CAS and time to completion of scenarios between junior and senior providers in the EIF versus no-EIF groups (Table 3) or between the groups that received an EIF during their initial scenario versus those who did not (Table 4). Interrater reliability between the 2 independent assessors was excellent at ICC = 0.979 (95% CI, 0.964–0.987).

TABLE 2.

CASs and Complications for the Scenarios With EIFs and Without EIFs, Across All Scenarios

	36 Scenarios With EIF, % (%)	36 Scenarios Without EIF, % (%)	P
Median critical action score (IQR)	84.2 (71.7–94.1)	12.5 (10.5–35.3)	.001
Median time to completion, min (IQR)	6.9 (5.8–10)	10 (constant)	.001
Presence of complications (95% CI)	30.6 (17.4–46.3)	100 (92.2–100)	.001

TABLE 3.

Median Percentage of Critical Actions Completed, by Provider Experience

	Junior Provider, N = 12, % (%)	Senior Provider, N = 12, % (%)	P
Median critical action score (IQR) with EIF	87.5 (80.7–94.1)	81.3 (70.0–94.3)	.406
Median critical action score (IQR) without EIF	11.5 (6.2–22.9)	20.4 (10.5–41.0)	.104

TABLE 4.

Median Percentage of Critical Actions Completed, by Group With EIF Present on First Versus Subsequent Scenario

	Group ABC, N = 12 (EIF Present on First Scenario) , % (%)	Group DEF, N = 12 (EIF Not Present First Scenario) , % (%)	P
Median critical action score (IQR) with EIF	85.9 (76.6–94.0)	84.2 (70.6–94.1)	.863
Median critical action score (IQR) without EIF	22.9 (10.5–41.2)	12.7 (6.3–23.4)	.116

In end-of-study evaluations of the EIFs, all providers agreed or strongly agreed that the EIFs contained sufficient information for the diagnosis and treatment of simulated patients when they had access to them. They all agreed or strongly agreed that EIFs were easy to use and helped manage the patients more effectively and that EIFs could improve clinical outcomes and reduce adverse events and complications. Finally, they all wanted to have access to an EIF when caring for CMC in their practice (Table 5).

TABLE 5.

Practitioner Response to Exit Questionnaire

Exit Interview Question	Strongly Disagree, %	Disagree, %	Neither Agree Nor Disagree, %	Agree, %	Strongly Agree, %
The EIF contained sufficient information for diagnosis and treatment of simulated patients when it was available.	0	0	0	29	71
The EIF was easy to use in the scenarios it was available.	0	0	4	46	50
The EIF helped me manage the patients in the scenarios more effectively compared with the scenarios where the EIF was not available.	0	4	0	12	84
EIFs could improve the clinical outcomes of patients with complex medical conditions in the ED.	0	0	0	0	100
EIFs could reduce medical errors, complications, or adverse events in the ED.	0	0	0	4	96
I would like to have access to an EIF when caring for a patient with a complex medical condition in the ED.	0	0	0	4	96

Discussion

To the best of our knowledge, this is the first study to evaluate the impact of EIFs in simulated scenarios of CMC. We found that physicians performed significantly better in every scenario when they had access to an EIF, regardless of their level of experience and regardless of whether the EIF was introduced during the first scenario or subsequently. Without the EIF, participants were uniformly unable to complete the scenarios within the 10-minute time limit, and preprogrammed adverse events and complications occurred in 100% of these cases. These findings suggest that having access to critical patient-level information is essential to providing optimal care to CMC during an emergency. We used simulation to investigate the impact of EIFs during emergency scenarios to improve patient outcomes, because this research problem cannot be answered safely, ethically, or in a timely fashion in clinical settings. Simulation-based research methods in pediatrics are feasible, safe, and ethical and are increasingly being used for these reasons.²⁸

The limited literature on use of EIFs in CMC includes consensus statements by academies and medical societies specifying contents of EIFs and advocating their use,^1,2 surveys of parents and practitioners,^16,26 and attempts to educate families²⁹ and implement EIFs in selected patient populations.^20,21,30 There is strong consensus among the pediatric and emergency medicine medical societies that CMC are a vulnerable patient population that needs critical information to be readily available to health care providers for delivery of optimal care during emergencies. Countries such as Taiwan, France, and Australia have developed and successfully implemented sophisticated portable health record systems (smart cards) for their populations,^31–33 but EIFs are often not available in the United States, where significant barriers exist, including multiple payers, limited interoperability of electronic health records, and concerns about privacy.¹⁶ Findings of our study contribute new knowledge about the impact of EIFs in caring for CMC during simulated emergency scenarios and support efforts to promote more widespread use of EIFs for CMC.

Our study has several limitations, including the inability to accurately replicate highly complex clinical scenarios by using simulation. Even the best high-fidelity manikins do not always reliably display the physiologic variability noted in real patients, and the context of the ED is not easily reproduced. However, we made every effort to increase the fidelity and realism of the scenarios for all groups, and this study could not have been completed in the clinical setting. Also, our EIFs were very closely related to the simulated scenarios, and the simulated scenarios were closely related to the EIFs, which may not happen in real life and which could overestimate their utility. This study also included a small sample of physicians at 1 hospital and used state-of-the-art simulators; thus, our findings might not be generalizable to findings of simulation studies conducted at other sites or with other equipment. Yet the rigorous study design, the use of multiple scenarios per practitioner including an acclimation scenario, and block randomization of participants to groups and differential order of scenarios were strengths, and we found very large differences in outcomes between physicians who used and did not use an EIF in each scenario, even with this small sample. Also, we conducted this study in a state-of-the-art quaternary care referral pediatric hospital that serves a large population of CMC, and the findings may not be applicable to smaller community-based EDs. However, we hypothesize that the impact of EIFs in such environments may be even more significant given that patients’ records may not be accessible and physicians may be less familiar with CMC.

This study adds to the literature by quantifying the impact of EIFs in simulated scenarios. The data from this study support additional efforts to investigate the clinical application and widespread implementation of the EIFs in this vulnerable population of CMC, who are at high risk of complications and poor outcomes during an emergency. We hope to integrate these efforts with state information exchanges and other state and federal efforts to create standard electronic and personal health records for US patients, particularly CMC.

Conclusions

Use of EIFs significantly improved physician performance and clinical outcomes in simulated emergency scenarios of CMC, independently of provider experience. Clinicians evaluated the utility and desirability of EIFs very highly when caring for CMC. These data can be used to justify the implementation and additional evaluations of efficacy of EIFs in CMC in real-world clinical settings.

Glossary

CAS

critical action score

CI

confidence interval

CMC

children with medical complexity

ED

emergency department

EIF

emergency information form

IQR

interquartile range