Gamified learning for resuscitation education: A systematic review

Abstract

Aim

To systematically review published literature to evaluate the impact of gamified learning on educational and clinical outcomes during life support education.

Methods

This systematic review was conducted as part of the continuous evidence evaluation process of the International Liaison Committee on Resuscitation (ILCOR). A search of PubMed, Embase, and Cochrane was conducted from inception until February 12, 2024. Studies examining incorporation of gamified learning were eligible for inclusion. Reviewers independently extracted data on study design and outcomes; appropriate risk of bias assessment tools were used across all outcomes.

Results

2261 articles were identified and screened, yielding sixteen articles (seven randomized trials, nine observational studies) which comprised the final review. No meta-analyses were conducted due to significant heterogeneity of intervention, population, and outcome. Only one study was found to have a low risk of bias; the remaining studies were found to have moderate to high risk. Fourteen studies were in healthcare providers and two were in laypersons. Most studies (11 of 16) examined the impact of a digital platform (computer or smartphone). Most (15 of 16) studies found a positive effect on at least one educational domain; one study found no effect. No included study found a negative effect on any educational domain.

Conclusion

This systematic review found a very heterogeneous group of studies with low certainty evidence, all but one of which demonstrated a positive effect on one or more educational domains. Future studies should examine the underlying causes of improved learning with gamification and assess the resource requirements with implementation and dissemination of gamified learning.

Introduction

Effective education of both laypersons and healthcare providers in life support is one of the key components in improving survival form cardiac arrest.¹ Published studies have examined many novel approaches to instructional design, course delivery, or technology use to improve the effectiveness and durability of life support education.² More effective teaching strategies for learners may include a greater degree of stimulation and engagement using active participation with and alongside peers. Gamification refers to the use of game-like elements to increase engagement and encourage interactive and intuitive participation by learners. A recent systematic review of the use of ‘serious games’ in healthcare found more than 40 games in multiple genres, but significant heterogeneity and imprecision in measurement of outcomes exist among published studies.

Novel approaches to education in life support have been increasingly prevalent in the literature; this expanding area of knowledge stems from an ongoing need to overcome obstacles to attracting and engaging both laypeople and healthcare providers in life support skills and knowledge training.² Some preliminary studies have found that gamified learning results in improved knowledge and skill during CPR training, either alone or used as pre-training to a standard life support course; other studies have found no significant difference.3, 4

This systematic review assessed the evidence on both educational and clinical outcomes of life-support education studies, comparing courses using elements of gamified learning to courses without such elements.

Methods

This systematic review was part of the continuous evidence evaluation process by the International Liaison Committee on Resuscitation (ILCOR) Education, Implementation and Teams Task Force (EIT) and it was registered with PROSPERO (registration number CRD42023483540).

The systematic review was driven by a research question using the PICOST (Population, Intervention, Control, Outcomes, Study design, and Timeframe) format: In learners training in any form of life support (Population), does instruction using gamified learning elements (Intervention), compared to traditional instruction or other forms of non-gamified learning (Control), affect educational outcomes (skill, knowledge, attitudes) or clinical processes (change in healthcare provider behavior, patient outcomes, cost or resource utilization)?

We defined ‘gamified learning’ as the use of game-like elements in the context of training (e.g. point systems, competition between learners or teams, leaderboards, scaffolded learning with increasing challenge, ‘medals’ or ‘badges’, etc.) aiming to increase learner engagement and to enhance recall of content. Importantly, studies using screen-based content delivery were not automatically included as examples of gamified learning based on the technology alone. Training courses that used these technologies but did not deliver content in a manner that utilized gamification as described above (e.g., a video-based life support training course for individual learners without these aforementioned elements) were excluded from our review.

Study eligibility criteria and data sources

We included randomized controlled trials (RCTs) and non-randomized studies (non-randomized controlled trials, interrupted time series, controlled before-and-after studies, cohort studies) as eligible for inclusion. Unpublished studies (e.g., conference abstracts, trial protocols) and grey literature were excluded from consideration. We searched EMBASE.com, Medline, and Cochrane from inception to February 12, 2024. Citations of included studies were subsequently reviewed for additional eligible studies.

Study selection, risk of bias assessment, and data extraction

The titles and abstracts of all potentially eligible studies were screened for inclusion by pairs of independent reviewers (AD and TS; AO and LT). The full text of included studies were checked against the inclusion and exclusion criteria independently in these pairs of reviewers. Any disagreements between the reviewers at either stage were resolved by discussion finding consensus.

Data from each study were independently extracted by a pair of reviewers and grouped separately according to the predefined outcomes. Two reviewers independently assessed the certainty of evidence of individual studies using the GRADE approach (Grades of Recommendation, Assessment, Development and Evaluation). In case of disagreement, consensus was reached by discussion. Risk of bias was assessed by the same pairs of reviewers. The Cochrane Risk of Bias Tool for randomized trials (RoB 2) was used for randomized studies; the Risk of Bias In Non-randomised Studies of Interventions (ROBINS-I) observational studies.

Quantitative meta-analysis was not possible due to significant heterogeneity of intervention, population, and outcomes. The evidence is summarized in narrative form.

Results

The search identified 2261 articles after duplicates were removed. Of these, 2202 articles were excluded during title and abstract screening, leaving 58 full-text articles to be screened for eligibility. During full-text review, 42 articles were subsequently excluded, leaving 16 articles comprising the final review; 7 randomized trials (RCTs) and 9 observational studies (6 before and after studies, 3 survey-based studies).5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 Table 1 contains the characteristics of these included studies. The PRISMA diagram for the reviewis shown in Fig. 1.

Table 1.

Details of included studies.

Study	Study Design	Setting	Number of participants	Outcome measures	Control group	Intervention group	Gamification elements	Results
Billner-Garcia, 20225	Observational	Nurses; USA	19	S (scoring rubric for NRP scenario, time to initiate PPV)	NA	Online access to game participant portal	“Story” and “quest” elements; inability to progress without perfect score	Scenario score: post better than pre, p = 0.007; time to PPV: post better than pre, p = 0.04;
Boada, 20156	RCT	Nursing students, Spain	109 (IG 67; CG 42)	S (CPR performance) expressed as scaled score (%)	Standard training	Use of LISSA (online gamified platform)	Computer based competitive CPR performance	Improved scores in groups who used LISSA compared to those who didn’t (36 vs 47; p < 0.05 and 36 vs 50, p < 0.05)
Chang, 20197	RCT	Healthcare providers, multinational	920 (IG 601, CG 319)	S (CPR performance)	No access to leaderboard	Availability of leader board	Leaderboard	No effect
Cutumisu, 20198	Observational	Healthcare providers; Canada	30	K (overall score 0–16 points)	NA	RETAIN (board game designed to teach NRP knowledge)	Score-based board game	Overall score: pre 49% vs post 61% (p < 0.001)
Gordon, 19959	Observational	Healthcare providers; USA	11	A (Likert scale survey on effectiveness)	NA	Card game involving NRP knowledge	Card game played between two teams	4.2–4.8 mean scores on 5 point Likert scale
Gutierrez-Puertas, 202110	RCT	Nursing students; Spain	184 (92 per group)	K (ad hoc questionnaire; 10 MCQs)	Standard 2 h training	Training with 90 mins use of game	Phone based app using random keywords that subject gives clues to teammates to identify	Pre-intervention to post intervention: CG 6.9 + 1.5, IG 7.7 + 1.4 (p < 0.05); 3 weeks post training CG 7.8 + 6.4, IG 9.5 + 0.9 (p < 0.05)
Hu, 202111	Observational	Medical students; China	81 (IG 41, CG 40)	K (test; format not described well)	Simulation based NRP training	Same with pre-training access to game	NEOGAMES (screen based NRP game with point system)	Immediate post training score: IG 98 + 3; CG 95 + 7 (NS); 6 months post training: IG 87 + 12; CG 63 + 15 (p < 0.001)
King, 202312	Observational	Paediatric nurses, USA	22	S (preparation of epinephrine dose); K (correct dose); A (comfort)	NA	Repeated practice and testing with leaderboard for best times	Leaderboard	Average decrease in time 27 sec (p = 0.02); proportion completing task in < 2 mins from 23% to 59% (p = 0.03); proportion knowing correct concentration 19% to 73% (p < 0.001); comfort improved by mean 3.6 of ten points (p < 0.001)
MacKinnon, 201513	RCT	HCPs, UK	171 (IG 90, CG 81)	S (CPR performance score)	No refresher training over 6 mos.	Self-motivated CPR refresher training over 6 mos.	Leaderboard	CPR performance change significant in IG (p < 0.001)
Otero-Agra, 201914	RCT	High school students, Spain	489 (IG 151; CGs groups 338)	S (CPR parameters)	EVA (training with grade); VFC (non-compulsory w visual feedback); TC (control)	GAM: team based competition	Competition btw groups of 4 students with scores displayed	QCPR score: GAM 90 + 8; EVA 85 + 20 (p = 0.03); VFC 82 + 21 (p < 0.001); TC 64 + 28(p < 0.001)
Pelletier, 202318	Observational	Residents, USA	32	A (survey on effectiveness)	NA	'Escape room' with sequential tasks to complete	Escape room format (working in groups, solving ‘puzzles’)	18/32 respondents rated 'excellent'; 100% of responses said content was applicable
Phungoen, 202015	RCT	5th yr med students; Thailand	105 (IG 53; CG 52)	K (2 MCQ tests, one on ALS algorithm, one on general ALS knowledge); S (ALS scenario score); A (confidence post course)	2 day ALS course	Use of game before and during 2 day ALS course	Resus Days: smartphone app with video and point based resusc scenarios	ALS algorithm test: IG 17 + 2 vs CG 16 + 2 (p = 0.01); ALS knowledge IG 22 + 2 vs CG 22 + 3 (p = 0.45); Skill test passing score IG 79% vs CG 66% (p = 0.09); Confidence IG 8 + 1 vs CG 8 + 2 (p = 0.51)
Semeraro, 201716	Observational (before and after)	High school students, Italy	65	S (CPR parameters) immediately post course and at 3 months	NA	Use of game interface during CPR “tournament”	Relive: screen based interface with video feedback	Immediate post course: CC depth 45 + 8 mm vs baseline 31 + 12 mm (p < 0.01); CC rate 111 + 10 vs baseline 94 + 32 (p < 0.01); 3 month retention: depth 46 + 15, rate 131 + 37 (vs baseline p < 0.01, vs course NS)
Thomas, 202319	Observational	Residents, USA	16	S (proportion of primary surveys performed by residents)	NA	Training with 'paper doll' model for primary survey	Incentives for voluntary participation ('prizes', notifications in resident newsletter)	Primary survey by residents: 72% pre, 93% post intervention
Toft, 202217	RCT	High school students, USA	491 (IG 286, CG 205)	S (CPR overall performance); K (questions re: recognition of OHCA)	AHA Hands Only CPR Course	Heart Class (HC) – online competition platform	2 teams competing at questions and CPR performance	6 mos. post training: CPR score IG 23%, CG 16% (p < 0.05); recognition IG 12%, CG 7% (NS);
Zanetto, 202320	Observational	Residents, Italy	93	A (survey with Likert scale items regarding confidence and satisfaction with training (1–10 scale))	NA	Neonatal resuscitation training	'Tournament' format with broadcasting of teams performance to other viewing teams	Improved confidence post-training (p < 0.01); median overall rating 9 (IQR 8–10)

Abbreviations: NRP: Neonatal Resuscitation Program; PPV: positive pressure ventilation; NA: not applicable; RCT: randomized controlled trial; IG: intervention group; CG: control group; CPR: cardiopulmonary resuscitation; S: skill; K: knowledge; A: attitude; MCQ: multiple choice questions; ALS: advanced life support; OHCA: out-of-hospital cardiac arrest.

Fig. 1.

PRISMA diagram.

Eleven studies used gamification elements involving digital platforms: 6 studies used an online or screen-based platform5, 6, 11, 13, 16, 17, 3 studies used a digital leaderboard7, 12, 13, and 2 studies used smartphone applications.10, 15 One study used a board game and one used a card game.8, 9 Two studies used gamification elements (tournament format or prizes awarded) to incentivize participation in simulation training.19, 20 One study used an ‘escape room’ format for emergency medicine resident training.¹⁸ Only one study used actual patient care (primary survey patient assessments during actual resuscitations) as an indirect outcome¹⁹; none of the remaining studies examined clinical outcomes. No studies included data regarding cost or resource utilization.

Healthcare providers were the learners in 14 studies5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20; 2 studies examined laypeople (high school students) as learners.16, 17 Five studies examined performance between groups or teams9, 10, 17, 18, 20; and the remaining 12 studies examined individual performance. Six studies used adult scenarios 6, 10, 14, 15, 16, 17; 5 used paediatric scenarios7, 12, 13, 18, 19; 5 used neonatal scenarios.5, 8, 9, 11, 20 The risk of bias assessment (Table 2) found that 15 of 16 studies had moderate to high risk of bias, with one study having low risk.

Table 2.

Assessment of bias tables.

RCTs (Cochrane ROB)
1st Author	Year	Randomization	Deviations from interventions	Missing data	Outcome measurement	Outcome reporting	Overall
Boada6	2015	Some concerns	High	High	High	Some concerns	High
Chang7	2019	Low	Some concerns	Low	Some concerns	Low	Some concerns
Gutierrez-Puertas10	2021	Some concerns	Some concerns	Low	Some concerns	Low	Some concerns
MacKinnon13	2015	Low	Low	Low	Some concerns	Low	Low
Otero-Agra14	2019	Low	Some concerns	Low	Some concerns	Low	Some concerns
Phungoen15	2020	Low	Some concerns	Low	Low	Low	Some concerns
Toft17	2022	Some concerns	Low	Low	Low	Low	Some concerns

Observational studies (ROBINS-I)
1st Author	Year	Confounding	Participant selection	Intervention classification	Deviations from interventions	Missing data	Outcome measurement	Outcome reporting	Overall
Billner-Garcia5	2022	Serious	Low	Low	Low	Low	Low	Low	Serious
Cutumisu8	2019	Serious	Low	Low	Low	Low	Low	Low	Serious
Gordon9	1995	Serious	Low	Low	Low	Low	Serious	Low	Serious
Hu11	2021	Serious	Low	Low	Low	Low	Low	Low	Serious
King12	2023	Serious	Low	Low	Low	Low	Low	Low	Serious
Pelletier18	2023	Serious	No information	Low	No information	Serious	Low	Serious	Serious
Semeraro16	2017	Moderate	Low	Low	Low	Low	Low	Low	Moderate
Thomas19	2023	Serious	Serious	Serious	No information	Serious	Serious	No information	Serious
Zanetto20	2023	No information	No information	Low	Low	No information	Serious	Low	Serious

Skill

Five RCTs reported overall CPR performance: four RCTs with 1689 healthcare professional participants in the gaming groups and 789 in control groups, and one RCT in 491 laypersons with 286 in the intervention groups and 205 in the control group.6, 7, 13, 14, 17 One RCT in nursing students using an online competitive platform found improved CPR performance scores.⁶ Two RCTs in paediatric healthcare providers used a leaderboard to monitor competition in CPR performance during refresher training sessions with mixed results.7, 13 One single center study of healthcare professionals found better performance in leaderboard group¹³; the other multicenter RCT found no effect.⁷ One RCT in laypeople using team competition during training found better CPR performance in the competition group than controls.¹⁴ A cluster randomized study of 491 high school students using a screen-based competition compared to control training found improved performance in the competition group compared to control 6 months after the training.¹⁷

Another observational study of 65 high school students participating in a CPR “tournament” during BLS training via a screen-based interface found immediately post training improved chest compression depth and rate compared to baseline. At 3 months, chest compression depth and rate was still better than baseline but no different compared to immediately post training.¹⁶

One RCT in healthcare providers included 105 study participants using a smartphone-based game involving ALS scenarios before and during an ALS course compared to no gaming, and reported no difference between groups.¹⁵ In contrast, an online gaming portal involving Neonatal Resuscitation Program (NRP) training found improved scenario scores following gaming in an observational study.⁵ The same study found faster time to positive pressure ventilation in a neonatal scenario.⁵ Another observational study of paediatric nurses used a leaderboard during repeated practice of preparing weight-based epinephrine dosing, and found significant decrease in time to dose preparation, and higher proportion of learners completing the task in less than 2 min.¹² One observational study involved residents receiving prizes for participation in supplemental low-fidelity simulation sessions to learn primary survey skills; this study found an increase in residents performing primary surveys in actual patient events.¹⁹

Knowledge

Two observational studies in neonatal healthcare providers report improved knowledge. One study investigating an NRP board game showed improved knowledge scores after playing.⁸ A different screen-based point-system game in NRP led to higher knowledge scores 6 months post training but without a difference compared to immediately post training.¹¹

Two RCTs in healthcare providers with 145 persons in the intervention groups and 144 in the controls reported the effect of gamification on knowledge acquisition. One RCT involved teams in a phone-based game identifying keywords and found higher scores on a multiple-choice questionnaire following training.¹⁰ Another RCT using a smartphone-based game before and during an Advanced Life Support (ALS) course found higher scores on an ALS algorithm test among game users.¹⁵

An observational study of paediatric nurses using a leaderboard during repeated practice of preparing weight-based epinephrine dosing found a significant increase in the proportion of learners knowing the correct concentration of epinephrine.¹²

Affective responses (attitudes)

One RCT using a smartphone-based game in ALS training led to better self-reported confidence among users.¹⁵ Another study of a card game to enhance NRP knowledge reported high levels of perceived usefulness among surveyed learners post-study.⁹ An observational study of emergency medicine training using an ‘escape room’ format was judged to be of high quality and applicability by surveyed participants.¹⁸ Another observational study of resident teams competing in a tournament-format neonatal simulation course found high ratings from the majority of surveyed participants.²⁰

Discussion

This systematic review provides evidence that using elements of gamified learning during life support training may lead to improved educational outcomes. This effect was present in studies involving individual learners as well as team, in neonatal, paediatric, and adult resuscitation scenario teaching, and in laypeople as well as healthcare providers. Given the significant heterogeneity of the studies in terms of intervention, assessment(s), and outcomes, it is not possible to precisely determine what combination of gamified learning elements, life support courses, and learner backgrounds will have the optimal impact of this facet of instructional design. Despite this limitation, all but one of the included studies reported a positive impact on at least one domain of learner experience, and there were no studies that reported a negative effect. Thus, the use of gamified learning elements in resuscitation education of any type can be recommended, based on weak certainty of evidence.

Gamification has been defined as “the use of game elements in non-game contexts”; examples of gamification exist in many different domains including military training, education, and industry.²¹ The effectiveness of gamified learning is based on the premise that inserting game-like elements creates a learning environment that is more entertaining and engaging, leading to a greater degree of collaboration, competition, and ultimately learner engagement. The theory behind gamified learning suggests that learning itself is not directly affected; rather, gamification moderates or influences behavior or attitudes toward learning in a manner that enhances the impact of the instructional content.²² It can be inferred from this theory that gamification is not a substitute for instructional design, but rather a feature to be incorporated. The studies included in our review exemplify this combination, where game-like features were employed in the delivery of other existing forms of life support instruction (either didactic or experiential).

A 2016 systematic review of serious games as training and/or educational tools for healthcare providers by Wang at al. included 48 publications, but only with the use of digital platforms and the use of a form of scoring as their inclusion criteria, covering a wide range of clinical domains and study designs. Similar to our findings, included articles exhibited significant heterogeneity and were of modest quality by the medical education research study quality instrument (MERQSI) criteria.²³ The authors recommended more uniformity and empiricism in development, testing, and dissemination of these games. The 2020 Consensus of Science and Treatment Recommendation (CoSTR) of the ILCOR EIT Task Force combined in one review gamified learning during life support training and the use of virtual and/or augmented reality.24, 25 For the CoSTR 2024 EIT issued two different systematic reviews, the current one on gamified learning, and another on immersive technologies in resuscitation addressing virtual and augmented reality.²⁶ While the majority of the studies in this review involved a digital platform of some sort (either computer or smartphone), the inclusion criteria for this review required that elements of game play needed to be part of the course delivery. A platform that used video or virtual reality but where the material was delivered in a straightforward manner with a typical end of course assessment, even if interactive in nature, would not automatically be included as ‘gamified’ learning. This challenging differentiation led to exclusion of many studies, even if the study description referred to the intervention as a ‘game’ or ‘serious game’. Medical educational researchers should be encouraged to adopt more unified definitions for gamification, based on the theoretical principles outlined above.

Limitations, knowledge gaps, and future research

Several limitations to the findings of this review should be acknowledged. As described above, the seventeen studies included in this review demonstrated marked heterogeneity, making summative conclusions about the impact of gamified learning on educational outcomes nearly impossible. Nine of the included studies reported the use of a summative assessment of knowledge at training conclusion, but in several cases the assessment instruments used for these studies appear to have been created ad hoc for the studies themselves, constituting a significant risk of bias based on inconsistency. The application of unified assessment of the learning outcome in future studies is highly recommended to enable comparative studies and meta-analysis.

The results of this review highlight several important knowledge gaps with regard to gamified learning. Almost all included studies (with one exception) were performed at single institutions. Determining the generalizability of these results would require studying how feasible it is to use a given gamified platform in other settings, both from the perspective of learner needs and instructor training. None of the studies included in the review examined the time requirement or cost of implementing the interventions, of particular relevance in light of the dominant use of digital platforms. There is a growing literature on stress and cognitive load in learners during life support education; future research should use outcome measures from this area of study to attempt to characterize a link, either positive or negative, between gamified learning and the learners’ emotional state. Notably, one study in our review reported on the emotions experienced by laypersons following training, finding that anxiety and relief were more common among the intervention group.¹⁷ Finally, the impact of life support education using gamified learning on patient clinical outcomes remains elusive; only one study in our review had patient care delivery (primary survey) as an outcome, but the study exhibited a high degree of imprecision and indirectness in the reported results.¹⁹

Conclusions

This systematic review provides very low certainty of evidence to support the incorporation of elements of gamified learning into life support education courses of any type in order to improve educational outcomes. Improving the quality of the evidence of gamified learning’s effects will require clearer definitions of gamification, greater uniformity across learning platforms, and consistent use of psychometrically robust assessment methods across studies. Future research on gamified learning should focus on generalizability and implementation, as well as a clearer understanding of learner responses to gamification and possible patient outcomes.

CRediT authorship contribution statement

Aaron Donoghue: Writing – review & editing, Writing – original draft, Methodology, Investigation, Data curation, Conceptualization. Taylor Sawyer: Writing – review & editing, Writing – original draft, Validation, Investigation, Formal analysis, Data curation, Conceptualization. Alexander Olaussen: Writing – review & editing, Writing – original draft, Validation, Methodology, Investigation, Formal analysis, Data curation. Robert Greif: Writing – review & editing, Writing – original draft, Supervision, Project administration, Methodology, Investigation, Formal analysis, Conceptualization. Lorrel Toft: Writing – review & editing, Writing – original draft, Validation, Methodology, Investigation, Formal analysis, Data curation.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: ‘AD, AO, TS are members of of ILCOR’s task force on Education, Implementation, and Teams. RG is the ERC Director of Guidelines and ILCOR, and Chair of ILCOR’s task force on Education, Implementation, and Teams; he is also an Editorial Board member for Resuscitation Plus. LT is the author of one of the studies included in the review, but other members of the task force/writing group reviewed her study.’.