Training of Pediatric Critical Care Providers in Developing Countries in Evidence Based Medicine Utilizing Remote Simulation Sessions

Abstract

Background. Remote simulation training provides a unique opportunity to captivate providers despite language, distance, and cultural barriers. Previously we developed a novel electronic decision support and rounding tool, the Checklist for Early Recognition and Treatment of Acute Illness in Pediatrics (CERTAINp). This study was conducted to determine the feasibility and impact of remote simulation training of international PICU providers using CERTAINp. Methods. We conducted train-the-trainer sessions in 7 hospitals based in 5 countries (China, Congo, Croatia, India, and Turkey) between 11/2015 and 11/2016. Providers first took part in a base line simulation session to assess their clinical performance. They had structured hands-on training using CERTAINp, which was done remotely using video conference with recording capabilities. Performance in PICU “admission” and “rounding” scenarios was assessed by their adherence to standard of care guidelines using CERTAINp. After this training, the providers were re-evaluated for performance using a validated instrument by 2 independent trained reviewers. Results. A total of 7 hospitals completed both baseline and post simulation sessions. We observed improved critical task (total 14) completion in the admission scenarios where pre training task completion was 8.2 ± 2.6, while after remote training was 11.2 ± 1.8, P = .01. In rounding scenarios, compliance to standard of care guidelines improved overall from 45% to 95% (P < .01). Conclusion. We observed an improvement in compliance for measures determined as best practice guidelines in simulation rounding and overall improvement in critical tasks for simulated admission cases after remote training.

Background

In spite of rapid advancement of modern medicine and availability of resources and manpower even in developing countries,¹ critical illness in children in low and middle income countries continue to have very high mortality.²

Acute care of critically ill children is a worldwide public health matter, and there is opportunity to enhance this care in the hospitals of the developing countries at all levels. This requires extensive training of the health worker and integrating evidence based medicine with suitable technology, evaluation measures and resources for instructing and funding.³ Critical care is not only early recognition of acute illness or serious injury, but appropriate intervention early in the course of critical illness.

Incomplete knowledge of best practices by frontline health care providers and error-prone care delivery processes can offset the potential benefits of critical care support.⁴ Children in the pediatric intensive care unit (PICU) are particularly prone to such errors because of inherent complexity involving multiple organ systems and the immediacy of the decision-making required for tenuous patients. Adverse events and serious errors in critically ill patients occur frequently and inability to pursue intended therapy is the leading cause of these potentially life-threatening errors.⁴

Strategies like use of checklists and standard order sets have been outlined to educate the health care work force in the resource poor setting.⁵ Checklists have been successful in improving patient outcomes like good surgical practice guielines;⁶ use of reliable and validated checklist has been accomplished for multidisciplinary bedside rounds;⁷ and use of modified version of World Health Organization Surgical Safety Checklist for pediatrics has shown adherence to the process of care.⁸

Simulation training imparts another opportunity to captivate providers despite language, distance, and cultural barriers.^9-11 Telemedicine-based, remote intensivist program has already shown improved clinical outcomes and gains in hospital finance.^12-14 The advances in medical informatics and human factors engineering have provided tremendous opportunity for novel and user-friendly clinical decision support (CDS) tools that could be applied in a complex health care environment.

The adult version of the decision support tool The Checklist for Early Recognition and Treatment of Acute Illness (CERTAIN) has already been designed and beta tested in a simulated environment.¹⁵ Based on the identified need for a pediatric version of the program, pediatric version of the software was designed (CERTAINp).

Our main objective was to determine the feasibility and effectiveness of remote training of international critical care providers that included physicians and nurses. It was to be done by deploying a standardized approach to the evaluation and management of acutely decompensating pediatric patients using CERTAINp remotely. We hypothesized that international PICU providers can apply CERTAIN framework to manage acutely decompensated patient via an online curriculum and hands-on training facilitated by remote 2-way communication.

Methods

This was ancillary analysis of prospective observational study of PICU physicians and nurses from 7 hospitals involved in the CERTAINp study. The CERTAINp is a recently ended multi-center clinical trial to investigate whether trained health care providers can deliver timely and error-free best-practice medicine and minimize preventable death and complications in critically ill children (ClinicalTrials.gov NCT02398981). International PICU providers at each hospital were recruited through a survey sent out through the World Federation of Pediatric Intensive and Critical Care Societies (WFPICCS). Participants were either PICU physicians, PICU nurses, or residents working in the pediatric ICU. After local IRB approval, learner verbal consent was obtained which included an agreement to record the learner’s performance in a private channel on YouTube. We utilized a 2-step verification system to protect those online contents. Information about study subjects were kept confidential and stored in a secure computer system.

All participants were recruited between July 2015 and October 2016. As some of the developing countries have limited internet with language barriers, printable rounding checklist in 4 different languages were developed for those particular countries.

The online and remote training included (Figure 1) (1) Online tutorial of PowerPoint and videos; (2) Online quiz for knowledge of CERTAIN tool; (3) Video based briefing/debriefing (synchronous communication via Google Hangouts/Zoom); (4) Remote simulation scenarios with screen capture for pre- and post-assessment of provider performance using validated instruments by 2 independent raters.

Figure 1.

Online and Remote Training in phases.

We created 3 scenarios (Supplemental Appendix 1) based on commonly encountered emergency situations requiring admission in PICU: (1) Hypotension due to urosepsis; (2) Respiratory distress in the setting of community acquired pneumonia; (3) Status epilepticus; and 1 scenario for rounding. All 4 scenarios were limited to 10 minutes each. They were designed to be led by a team leader assisted by the other 1 to 2 participants at each hospital. The team participated in the session from their local site office. The instructor sent the vitals and labs of the scenarios to leaners via screen sharing using video conference software. Participants received standard instructions during the baseline session. Each scenario commenced with a clinical vignette describing the chief complaint of the patient, age of the patient, gender, brief therapy given so far, and history of arrival to the hospital/ER. The scenario then started with the patient having unstable vital signs for first 4 minutes, while patient getting decompensated toward the end of the case. In this baseline training skills assessment, each learner participated in practice scenarios with remote debriefing. A rounding scenario of typical intubated and sedated septic patient on ventilator was developed and assessments were made with the rounding checklist in the post training.

The effect of learning was measured remotely by a resuscitation skills assessment during the simulation scenario using a valid and reliable instrument from our previous simulation study.¹⁵ After a baseline assessment of resuscitation skills, all providers received a standardized online training tutorial (PowerPoint™ slides, reference papers, and demo videos of how to use CERTAINp for admission and rounding) and were given access to the tool.

Before the post simulation training, each subject underwent structured hands-on training coached remotely using 2-way video communication software.

During the post simulation trainings, the subjects were re-evaluated in a scenario of similar clinical difficulty to assess for an improvement in their clinical performance. Performance was scored using a validated instrument by 2 independent reviewers-DP and AM (Tool). We did not assess the providers separately based on their education or experience given the small number of participants and overall involvement of the entire team during the case scenarios. A mobile version of rounding checklist was also available. To overcome limited internet connectivity in some of the hospitals, we provided the paper version of rounding and admission CERTAINp along with paper booklet for decision support.

Data were summarized as mean (standard deviation) for continuous variables and number (percent) for categorical variables. Pre and post training groups were compared by chi-square test for categorical variables, and 2-sample Student t-test for continuous variables. A P-value of >.05 was considered significant.

Results

Total 7 centers (Croatia, Congo, India, Turkey, and 3 centers from China) completed both baseline and post education sessions. Total 18 (9 baseline + 9 post) providers for “admission” scenarios and 1 per hospital rounding scenario (total 14, baseline 7 + post 7) were included in the analysis. Total 5 “admissions” scenarios session did not have quality videos/no videos and were thus excluded from final analysis.

We observed improved critical task completion in 9 providers in the admission scenarios where number (means ± SD) of pre-training task (total 14) completion was 8.2 ± 2.6, while after remote training it was 11.2 ± 1.8, P = .01. There was significant improvement in exposure assessment, review of home medications, review of allergies, and review of differential diagnosis (all P < .05) There was overall improvement after remote training with CERTAINp (P = .01) (Table 1). The time (in seconds) taken for task completion was improved in post training, when compared to baseline, for disability assessment 188 versus 75, P = .046; for review of exposure assessment was 209 versus 137, P = .22; for review of past medical history, 138 versus 116, P = .74 and review of differential diagnosis was 377 versus 241, P .29. For rest of the task either it was same or there was non-significant increase (Table 3).

Table 1.

Pre and Post Simulation in Critical Tasks in Admission Scenarios.

Item	Pre-training group (N = 9)	Post-training group (N = 9)	P value
Code status discussion	0	4	.02
Airway assessment	6	7	.6
Breathing assessment	8	8	1.000
Cardiac assessment	9	7	.13
Disability assessment	5	8	.11
Exposure assessment	2	7	.02
Evaluation of vitals sign	9	8	.30
Evaluation of temperature	6	5	.63
Review of past medical history	5	6	.63
Review of home medication	3	8	.01
Review of allergies	1	6	.01
Order initial basic lab test	8	9	.30
Start oxygen supplementation	7	9	.13
Review of differential diagnosis	0	4	.02
Personal complete mission (mean ± SD)	8.2 ± 2.64	11.2 ± 1.79	.01

Table 2.

Pre and Post Compliance for Standard of Care Guidelines for Rounding Scenario.

	Baseline (%)	Post simulation (%)	P value
Sedation break discussed	71	100	.13
Delirium discussed	0	100	<.01
Pain treatment discussed	43	71	.28
Lung protective ventilation discussed	28.5	100	.005
Spontaneous breathing trial discussed	0	86	.01
Head of bed elevation discussed	28.5	100	.005
Fluid balance discussed	100	100	1.0
Glucose control assessed	28.5	86	.03
Ulcer prophylaxis discussed	71	100	.13
Nutrition discussed	100	100	1.0
DVT prophylaxis discussed	43	100	.02
Bowel protection discussed	14	100	.01
Skin/wound care discussed	57	86	.24
Need for Devices discussed	14	100	.01
Goals of care discussed	57	100	.05
Family concerns discussed	57	100	.05
Total	44.5	95	<.01

Seven providers of these centers were assessed for completion of 16 critical tasks in rounding scenarios before and after training with CERTAINp. Overall, compliance to standard of care guidelines improved overall from 45% to 95% in all data points (P < .01), with significant improvement in involving discussion of delirium, lung protective ventilation, spontaneous breathing trial, head of bed elevation, glucose control, DVT prophylaxis, bowel protection, need for devices, goals of care, and family concerns (P < .05) (Table 2).

Table 3.

The Time (in seconds) Taken for Task Completion in Pre and Post Training Groups.

Item	Number of cases, N = 9	Pre-training, time (seconds), mean (SD)	Number of cases, N = 9	Post-training, time (seconds), mean (SD)	P value
Airway assessment	6	89 (121)	7	126.9 (192.1)	.69
Breathing assessment	8	83 (77.6)	8	81 (52)	.94
Cardiac assessment	9	110 (142)	7	110 (52)	.99
Disability assessment	4	188 (127)	8	75 (47)	.04
Exposure assessment	2	209 (18)	7	137 (71)	.22
Evaluation of vitals sign	9	50 (49)	8	78 (50)	.27
Evaluation of temperature	6	136 (153)	5	188 (71)	.59
Review of past medical history	5	138 (78)	6	116 (118)	.74
Review of home medication	3	126 (42)	8	151 (136)	.77
Review of allergies	1	193	6	190 (128)	—
Order initial basic lab test	8	173 (111)	9	209 (130)	.55
Start oxygen supplementation	7	118 (121)	9	120 (150)	.98
Review of differential diagnosis	5	377 (328)	9	241 (144)	.29

Reliability assessments: The percentage agreement between 2 reviewers was assessed. The pooled reliability was above 80% for admission scenario using validated checklist while it was 100% for the rounding scenario checklist.

Discussion

This study was designed to test the feasibility and efficacy of using an online curriculum for remote training to teach a standardized approach (CERTAINp) in the admission and rounding of acutely decompensating patients. We observed improved critical task completion in 9 providers in the admission scenarios with overall improvement after remote training with CERTAINp. However, time taken for task completion in the pre- and post-was varied for most critical tasks.

The reason for this finding is difficult to interpret due to a limited number of trainers. We found that compliance to standard of care guidelines improved close to 100% in all data points, with increase in adherence most notable to delirium assessment and spontaneous breathing trial, need for devices and bowel protocol, lung protective strategy, head of elevation and glucose control, pain management, and DVT prophylaxis followed by skin care, goals of care, family concerns and ulcer prophylaxis and sedation break.

A study done by Ikeyama et al showed that remote-facilitated simulation-based learning is technically attainable with minimal cost and simple resources while providers considered this set up as potent as an on-site system.¹⁶ A similar simulation and team training study done in a resource limited setting for maternal and neonatal emergency care displayed encouraging results.¹⁷ Randomized controlled study to determine real-time video communication between the first responder and a remote intensivist via Google Glass demonstrated that this did not reduce no-ventilation and no-compression fractions during the first 5 minutes of a simulated pediatric cardiopulmonary arrest but raised the quality of both the ventilation and compressions.¹⁸ Von Lubitz et al¹⁹ have validated the concept of international simulation-based training based may eventually dispense the most practical stage for a large-scale training of medical personnel in developing countries. Ours is the first study done in pediatrics on global implementation of simulation to train the providers in the pediatric intensive care unit and facilitate the care of delivery in resource limited settings with innovations.

Use of checklists in adult ICU has already demonstrated improvement of compliance measures in trauma and surgical ICUs.^20-22 Similarly, use of demonstrated major improvements in pressure ulcer prevention bundle implementation with a checklist.²³ Use of electronic medical record checklists in pediatric ICU have shown increased compliance with evidence-based catheter care and continued decrease in central line-associated bloodstream infections (CLABSI).²⁴ Improvements in accidental extubations and lung protective strategy has been done with the checklists in PICU.²⁵ Following use of rounding sticker use in a tertiary care PICU, there was decrease in urinary tract infections, rise in GI prophylaxis, and use of DVT prophylaxis.²⁶ Our study has also shown that CERTAINp can be an effective tool to improve high quality evidence-based care to critically ill children in the resource limited settings.

Telemedicine has helped physicians in anesthesia and surgery caring for patients in remote areas who can take advantage from the expertise found in subspecialty centers.^27,28 Integrating the checklist into the EMR boosted substantial clinical improvements in PICU.²⁹ Our remote simulation study has used the principle of tele-education and use of checklists with encouraging results.

Some of the study strengths are; (1) We were able to demonstrate that a low cost remote simulation training of international pediatric intensive care unit providers is feasible; (2) Even with small sample size we could note significant improvement in best care practices in simulated cases; (3) It is innovative because information technology offers pioneering flexibility to connect people around the world with audio/video and screen sharing information on a large scale with minimal cost; (4) The development of this tool in tandem with teaching via online platforms will facilitate the delivery of the evidence based practice to critically ill pediatric patients globally.

The major limitation of our study was the small number of providers in analysis, restraining statistical power. The limited internet connectivity may have added to the difficulty of training sessions. Also, simulation sessions at one of the centers had to redone as videos were inadvertently not saved and due to time constraints of one of the centers, only 1 case scenario was tested. We did not test performances based on training and experience again due to the small number and ultimately this was a team performance. Lastly, we strived to create the simulation as real as possible; the sessions may not completely reflect the actions of these providers in real clinical setting.

Future implications of this study are (1) To implement CERTAINp into clinical practice in variable resource settings and evaluate the impact of the tool on the processes of care and patient outcomes (decrease in ventilator days, decrease in hospital acquired infections, decrease in ICU length of stay, decrease in mortality, etc.) with the implementation. (2) To demonstrate a decrease in deviations from “good quality” and “standard practice” guidelines per patient per day with the implementation.

Conclusion

We observed an improvement in adherence for measures determined as best practice guidelines in simulated rounding and overall improvement in critical tasks for simulation admission cases after remote training.