Introduction
This special issue of Resuscitation Plus focuses on cardiopulmonary resuscitation (CPR) education – a rapidly evolving field of resuscitation science covering a broad range of studies exploring theoretical educational concepts and evaluating interventions designed to enhance provider performance during cardiac arrest.1 Resuscitation education is recognized as a key factor in contributing to improved survival outcomes from cardiac arrest.2 The International Liaison Committee on Resuscitation (ILCOR) has dedicated efforts through the Education, Implementation and Teams (EIT) task force to identify and appraise resuscitation education literature in order to formulate consensus on science treatment recommendations (CoSTR) that ultimately inform council resuscitation guidelines (e.g. European Resuscitation Council, American Heart Association, Heart and Stroke Foundation of Canada, etc.) and resuscitation training programs (e.g. basic and advanced life support training for adults, children, and neonates).3, 4 High quality resuscitation education research addressing key research questions with appropriate study design and outcome measures is required to conduct meaningful ILCOR evidence reviews. As current and former members of ILCOR’s EIT task force, we have encountered hurdles when conducting systematic reviews of existing resuscitation education literature that have made it challenging to effectively translate education science into CoSTRs.
Despite the growing interest in research focused on resuscitation education, many studies fail to get published, or if published, fail to contribute to ILCOR COSTRs due to flaws with their study design. Some studies do not adequately describe the educational intervention, making it hard to discern the key instructional design features contributing to outcomes. Studies may describe interventions of interest to the resuscitation community, but the study population is not reflective of the provider population most affected in real life. Some studies describe outcomes with no supportive validity evidence, or lower level Kirkpatrick outcomes (eg. satisfaction)5, 6; both of which limit the ability for inclusion in meta-analysis due to a lack of relevant outcome measures. Even when a systematic review has many studies contributing to the specific research question, the strength of the final recommendation often needs to be downgraded due to poor quality evidence.3 Failure to address these pervasive weaknesses with resuscitation education research diminishes the likelihood of publication, and ultimately the impact of educational efficiency on cardiac arrest survival outcomes. This editorial describes three categories of weaknesses encountered in our reviews of resuscitation education research that may impact the quality of evidence and therefore hinder meaningful contributions to ILCOR evidence reviews. A better understanding of these issues will enhance awareness and influence change that will eventually strengthen the body of evidence used to inform future ILCOR CoSTRs for resuscitation education.
1. Intervention design: Resuscitation education research often involves exposing the study population to an educational intervention with specific instructional design features. Instructional design features for resuscitation education include (but are not limited to): feedback, briefing and debriefing, spaced learning, mastery learning and deliberate practice, blended learning, contextual learning, assessment, and other innovative educational strategies (e.g. virtual reality, augmented reality).1, 7, 8 In order to assess the contribution of a specific instructional design feature, researchers must standardize the research environment and other elements of the intervention to isolate the independent variable. Studies that fail to do this inherently introduce threats to internal validity that make it difficult to discern the true impact of the educational intervention.9, 10 Studies often report educational interventions that are highly customized to their specific learning group or institution. While these types of studies may demonstrate benefit at the local level, the customized nature of training often limits generalizability, thus minimizing potential contribution to reviews targeting a more global audience and reducing chances of publication. To address these flaws, researchers should reference relevant reporting guidelines (e.g. reporting guidelines for health care simulation research) when designing their study.9 Discussing the key elements of instructional design during the study design process will help researchers determine a standardization strategy to isolate the variable of interest,10 and ensure that the eventual manuscript contains a detailed enough description of the intervention to allow for publication in peer-reviewed journals, and ultimately inclusion in ILCOR evidence reviews.
2. Study Population: The study population and sample size of studies can often influence the likelihood of publication and the degree to which they contribute to ILCOR evidence reviews. Study populations should ideally reflect the population most representative and relevant for the intervention being assessed, and should not be selected out of convenience (e.g. ease of recruitment). For example, a study assessing the impact of a virtual reality-based advanced cardiac life support training should ideally include front-line healthcare providers (i.e. doctors, nurses, paramedics) who are most likely to provide clinical care in this context as the study population, and not medical students who may be easier to recruit, but less likely to be directly involved in cardiac arrest management. One of the biggest challenges is that many resuscitation education studies are small, single-center studies that are underpowered to detect the desired effect for the selected primary outcome. Pooling results from various underpowered studies may be feasible, but the results are less likely to show a significant effect for the intervention of interest. The field of resuscitation education research would benefit from a more collaborative effort, facilitated by the development of a global research network with a focus on conducting large-scale, multicenter, randomized trials to address the most pressing questions in resuscitation education. Global collaboration will promote idea-sharing, the development of robust research protocols more likely to secure grant funding, and the publication of studies that are adequately powered to demonstrate results that can be generalizable across regions.11, 12, 13, 14, 15
3. Selection of outcomes: The nature and types of outcomes reported in a publication directly influence whether or not a study warrants inclusion in an ILCOR evidence review. Studies describing only participant reactions5, 6 (e.g. satisfaction, confidence etc.) are rarely included in ILCOR EIT systematic reviews as these lower-level outcomes are not viewed as sufficiently important to influence treatment recommendations. Behavioural performance outcomes (eg. skill acquisition and retention) in the simulated environment and in the real clinical environment (when available) are key. When an assessment tool is used (e.g. adherence to cardiac arrest protocol), researchers should select a tool with supportive validity evidence. Failure to do so brings the validity of the outcome into question. Alternatively, when objective performance metrics are selected as outcomes (e.g. CPR quality parameters), reporting of metrics that are clinically relevant (e.g. CPR depth, CPR rate, compliance with CPR depth/rate) and not composite scores (e.g. CPR scores with no supportive validity evidence) is strongly recommended. Reporting CPR metrics that are described in clinical studies allows us to build a chain of causality, which over time, will help to establish links between outcomes measured in the training setting, to outcomes measured in real life, and ultimately to patient outcomes.16 Lastly, researchers are encouraged to measure outcomes extending beyond the end-of-course or end-of-training. While there is a substantial body of evidence describing skill acquisition for various instructional design features, there is paucity of evidence detailing skill retention over time. As a potential solution to this issue, ILCOR will be conducting an Utstein consensus process in 2023 to establish reporting outcomes from resuscitation education research, which will provide clear guidance for researchers as they craft outcomes for future educational studies.
In summary, flawed design of educational interventions, poorly representative study populations, inadequately powered studies, and poor selection of study outcomes reduce the chances of publication and hinder meaningful contributions to ILCOR evidence reviews. Addressing these pressing needs through enhanced awareness, research collaborations, and establishment of consensus reporting outcomes will help shape the future of resuscitation education research, allowing for contributions that will inform international guidelines and resuscitation training programs.
Conflicts of Interest
Adam Cheng is Vice-Chair of the International Liaison Committee on Resuscitation (ILCOR)’s Task Force on Education, Implementation and Teams, and member of the American Heart Association’s Resuscitation Education writing group.
Farhan Bhanji is an emeritus member of the International Liaison Committee on Resuscitation (ILCOR)’s Task Force on Education, Implementation and Teams.
Andrew Lockey is President of Resuscitation Council UK and member of the International Liaison Committee on Resuscitation (ILCOR)’s Task Force on Education, Implementation and Team. He is Guest editor of the Special Edition on Resuscitation Education in the Resuscitation Plus journal.
Sabine Nabecker is European Resuscitation Council SEC-IES committee member (Instructor-Educator-Support Science and Education Committee), and Canadian Anesthesiologists’ CEPD (Continuing Education and Professional Development) Committee member. She is currently also Handling Guest editor of the Special Edition on Resuscitation Education in the Resuscitation Plus journal and member elect of the International Liaison Committee on Resuscitation (ILCOR)’s Task Force on Education, Implementation and Teams.
Robert Greif is the Board Director of Guideline and ILCOR of the European Resuscitation Council (ERC) and Chair of the International Liaison Committee on Resuscitation (ILCOR)’s Task Force on Education, Implementation and Team. He is Board member of the Resuscitation Plus journal and is currently also Guest editor of the Special Edition on Resuscitation Education in the Resuscitation Plus journal.
CRediT authorship contribution statement
Adam Cheng: Conceptualization, Writing – original draft, Writing – review & editing. Farhan Bhanji: Conceptualization, Writing – review & editing. Andrew Lockey: Conceptualization, Writing – review & editing. Sabine Nabecker: Conceptualization, Writing – review & editing. Robert Greif: Conceptualization, Writing – review & editing.
Acknowledgements
The authors would like to acknowledge the intellectual contributions of EIT task force members which have contributed to our thinking around the issues highlighted in this editorial.
References
- 1.Cheng A., Nadkarni V.M., Mancini M.B., et al. Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association. Circulation. 2018;138:e82–e122. doi: 10.1161/CIR.0000000000000583. [DOI] [PubMed] [Google Scholar]
- 2.Soreide E., Morrison L., Hillman K., et al. The formula for survival in resuscitation. Resuscitation. 2013;84:1487–1493. doi: 10.1016/j.resuscitation.2013.07.020. [DOI] [PubMed] [Google Scholar]
- 3.Greif R., Bhanji F., Bigham B.L., et al. Education, Implementation, and Teams: 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations. Circulation. 2020;142:S222–S283. doi: 10.1161/CIR.0000000000000896. [DOI] [PubMed] [Google Scholar]
- 4.Cheng A., Magid D.J., Auerbach M., et al. Part 6: Resuscitation Education Science. 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2020;142:S551–S579. doi: 10.1161/CIR.0000000000000903. [DOI] [PubMed] [Google Scholar]
- 5.Kirkpatrick D.L., Kirkpatrick J.D. Berrett-Koehler Publishers; San Francisco, CA: 2005. Transferring Learning to Behavior: Using the Four Levels to Improve Performance. [Google Scholar]
- 6.Kirkpatrick J.D., Kirkpatrick W.K. ATD Press; Alexandria, VA: 2016. Kirkpatrick’s Four Levels of Training Evaluation. [Google Scholar]
- 7.Motola I., Devine L.A., Chung H.S., Sullivan J.E., Issenberg S.B. Simulation in healthcare education: a best evidence practical guide. Med Teach. 2013;35:e1511–e1530. doi: 10.3109/0142159X.2013.818632. [DOI] [PubMed] [Google Scholar]
- 8.Mundell W.C., Kennedy C.C., Szostek J.H., Cook D.A. Simulation technology for resuscitation training: a systematic review and meta-analysis. Resuscitation. 2013;84:1174–1183. doi: 10.1016/j.resuscitation.2013.04.016. [DOI] [PubMed] [Google Scholar]
- 9.Cheng A., Kessler D., Mackinnon R., et al. Reporting Guidelines for Health Care Simulation Research. Simulat Healthc: J Soc Simulat Healthc. 2016;11:238–248. doi: 10.1097/SIH.0000000000000150. [DOI] [PubMed] [Google Scholar]
- 10.Cook D.A., Beckman T.J. Reflections on experimental research in medical education. Adv Health Sci Educ Theory Pract. 2010;15:455–464. doi: 10.1007/s10459-008-9117-3. [DOI] [PubMed] [Google Scholar]
- 11.Cheng A., Auerbach M., Calhoun A., et al. Building a Community of Practice for Researchers: The International Network for Simulation-Based Pediatric Innovation, Research and Education. Simul Healthc. 2018;13:S28–S34. doi: 10.1097/SIH.0000000000000269. [DOI] [PubMed] [Google Scholar]
- 12.Cheng A., Kessler D., Mackinnon R., et al. Conducting multicenter research in healthcare simulation: lessons learned from the INSPIRE network. Adv Simulat. 2017;2:6. doi: 10.1186/s41077-017-0039-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Niles D.E., Duval-Arnould J., Skellett S., et al. Characterization of Pediatric In-Hospital Cardiopulmonary Resuscitation Quality Metrics Across an International Resuscitation Collaborative. Pediatr Crit Care Med. 2018;19:421–432. doi: 10.1097/PCC.0000000000001520. [DOI] [PubMed] [Google Scholar]
- 14.Huggett K.N., Gusic M.E., Greenberg R., Ketterer J.M. Twelve tips for conducting collaborative research in medical education. Med Teach. 2011;33:713–718. doi: 10.3109/0142159X.2010.547956. [DOI] [PubMed] [Google Scholar]
- 15.Marshall J.C., Cook D.J. Canadian Critical Care Trials G. Investigator-led clinical research consortia: the Canadian Critical Care Trials Group. Crit Care Med. 2009;37(1 Suppl):S165–S172. doi: 10.1097/CCM.0b013e3181921079. [DOI] [PubMed] [Google Scholar]
- 16.Cook D.A., West C.P. Perspective: Reconsidering the focus on “outcomes research” in medical education: a cautionary note. Acad Med. 2013;88:162–167. doi: 10.1097/ACM.0b013e31827c3d78. [DOI] [PubMed] [Google Scholar]