Abstract
OBJECTIVES:
To explore gamification as an alternative approach to healthcare education and its potential applications to critical care.
DATA SOURCES:
English language manuscripts addressing: 1) gamification theory and application in healthcare and critical care and 2) implementation science focused on the knowledge-to-practice gap were identified in Medline and PubMed databases (inception to 2023).
STUDY SELECTION:
Studies delineating gamification underpinnings, application in education or procedural mentoring, utilization for healthcare or critical care education and practice, and analyses of benefits or pitfalls in comparison to other educational or behavioral modification approaches.
DATA EXTRACTION:
Data indicated the key gamification tenets and the venues within which they were used to enhance knowledge, support continuing medical education, teach procedural skills, enhance decision-making, or modify behavior.
DATA SYNTHESIS:
Gamification engages learners in a visual and cognitive fashion using competitive approaches to enhance acquiring new knowledge or skills. While gamification may be used in a variety of settings, specific design elements may relate to the learning environment or learner styles. Additionally, solo and group gamification approaches demonstrate success and leverage adult learning theory elements in a low-stress and low-risk setting. The potential for gamification-driven behavioral modification to close the knowledge-to-practice gap and enable guideline and protocol compliance remains underutilized.
CONCLUSIONS:
Gamification offers the potential to substantially enhance how critical care professionals acquire and then implement new knowledge in a fashion that is more engaging and rewarding than traditional approaches. Accordingly, educational undertakings from courses to offerings at medical professional meetings may benefit from being gamified.
Keywords: behavior, critical care medicine, education, gamification, guidelines
KEY POINTS
Question: How could gamification approaches be applied to critical care education and practice?
Findings: Gamification is ideally suited to crafting an engaging and rewarding competitive approach to acquiring new knowledge or skills relevant for critical care in an immediate feedback setting. It may be uniquely aligned with closing the knowledge-to-practice gap that permeates guideline implementation uptake and utilization.
Meaning: Healthcare and critical care education and practice may benefit from embracing gamification within educational offerings, courses, guidelines, and their implementation toolkits.
Education—both initial and continuing—underpins healthcare clinician professional development and practice. Acquiring didactic knowledge crafts a foundation upon which practical training and implementation skills may be layered. Relatedly, the spectrum of professional aspects of clinical care—including behavior—are all dependent on their underpinning educational anchors. While new knowledge can be readily shared and embraced, the behavior that governs how, when, and perhaps most importantly, if, that knowledge will be applied appears to be the most difficult aspect to influence. The disconnect between knowledge and implementation has been termed the knowledge, or evidence, to practice gap and is one focus for implementation science (1). Accordingly, a host of approaches have been applied to help guide, or most recently, “nudge” clinician behavior; such efforts address domains from antimicrobial stewardship to ventilator settings for patients with the acute respiratory distress syndrome (2, 3). Each of the approaches has met with only partial success that may reflect unique environments or specific sets of clinicians. Instead, an approach that may be more universally applicable is known as gamification—the introduction of game elements into a nongame context (4). Specifically, gamification encompasses the following domains: action language, assessment, conflict/challenge, control, environment, game fiction, human interaction, immersion, and rule/goals (5). Therefore, given the plethora of elements, and the broad appeal that they may engender in “succeeding” within a gamified context, gamification holds the potential to beneficially impact how clinicians acquire, embrace, and implement new knowledge such as that shared with a guideline. Within this narrative review, we explore how gamification applies to critical care medicine (CCM) education and practice.
BEHAVIOR MOTIVATORS IN CCM
There are several readily identifiable behavior influencers at work within CCM that may be grouped into five domains: top-down control processes, collaborative and multiprofessional groupings, resource limitations, incentive-based forces, and penalty-based drivers. All of the aforementioned motivators are external motivators; internal motivators are robust and exceed the scope of this article. Top-down control processes are quite familiar within daily practice and are exemplified by actions or events from which one may visibly gather data. These include hand-hygiene audits, masking mandates, and the prohibition of consumables at workstations or on rounds (6, 7). Importantly, these are rarely durable and are plagued by a Hawthorne effect during the observation period, especially when the observer is readily identifiable. Collaborative and multiprofessional groupings establish boundaries for behavior, may require permission for certain actions (antimicrobial access is dependent on Infectious Disease concurrence), invites participation but does not require engagement (guidelines such as the Surviving Sepsis Guidelines), or may require form completion but not compliance with all elements (unit-based evaluations and certain checklists) (8, 9). Resource limitations are more effective at altering behavior as the desired tool, medication, or consultant is unavailable based on purchasing decisions, supply shortages, or facility complexity (10). Interfacility transfer may be required to address patient needs and therefore influences clinician behavior (11). Since interfacility transfer based on resource limitation may be negatively viewed by the clinicians at the originating facility, this is not an adaptive approach to drive desirable behavior. Incentive-based approaches often tie trackable performance such as care efficiency, work-related relative value units, or billing amounts to financial reward such as salary or bonuses (12). Electronic health record meaningful use finances directly impact facility reimbursement (13). Others such as health maintenance activities may be linked to health insurance credits or incentive pay. Penalty-based drivers reduce reimbursement or salary for failure to meet metrics such as the Severe Sepsis and Septic Shock Management Bundle metric from the Centers for Medicare and Medicaid Services (14). Others such as Performance Improvement Plans that flow from failures noted within professional practice evaluations require additional activities to satisfy their requirements and to regain “good standing.” More recently, anonymous “safety event” reporting systems may be used to address interpersonal conflict with consequences—some of which may be quite intrusive— and only for the individual noted within the report (15, 16).
OUTCOMES OF CURRENT BEHAVIOR MOTIVATORS IN CCM
A systematic review of behavior theories for durable behavioral changes best described five fundamental concepts that include: 1) motives which are the key drivers for volitional behavior change(the motive may be for short-term or long-term gratification); 2) self-regulation/self-determination that includes the ability to monitor and regulate newly adopted behavior; 3) resources that support the ability to maintain behavior; 4) habit that reinforces a behavior to become automatic and habitual; and 5) environmental and social influences that support the behavior and likely have positive feedback that is reflected in the motivational/gratification theme for reinforcing a behavioral change (17). Current behavior influencers achieve variable and often time-limited impacts on clinician behavior including compliance with protocols and guidelines (17). Certain groups may be more strongly influenced rendering the penetrance of current influencers somewhat group specific. For example, guideline compliance is often more robust when pursued by advanced practice providers than physicians (18, 19). Creating an environment of change based on the creation and enforcement of rules is unlikely to actively engage an individual in the needed enjoyment, satisfaction, self-determination, self-evaluation, and perception of personal choice (autonomy) that helps to maintain change. Additionally, in critical care, the lack of resources and staff turnover limits the time needed to create habitual change behavior. Finally, unless an institution or critical care units and their team members embrace a culture of change—as opposed to perpetuating the status quo—the environmental and social drivers that reinforce current behaviors will remain constant. Lasting impact must address influencers that enable and sustain behavioral change.
Commonly, when a specific behavior is externally modified (enforcing selecting an antibiogram driven antibiotic), that modification may be derailed by a work-around that is actively discovered and exploited. Such events are strongly driven by externally imposed behavior modifiers (e.g., required documentation of delivered fluid volume for measure compliance) rather than ones that are internally generated (fluid volume guided by clinician performed Point-Of-Care UltraSound [POCUS]) (14). Nonetheless, restrictions to avoid undesirable behaviors and incentives to encourage desired behaviors seem most effective within systems that are not actively seeking change. When penalties are institutionally-based, they may not impact clinician behaviors. Instead, incentives may more readily impact clinicians if they are tied to compensation. When evaluating incentives versus penalties with regard to care quality and patient outcome, penalty-based programs may be more effective compared with incentive alone or incentive plus penalty driven programs (20). Furthermore, optimal practice may be derailed by existing evidence-to-practice gaps especially when evaluating adherence to current approaches to guideline and best practice implementation.
GAMIFICATION
Critical care training is a challenging process that requires healthcare professionals to acquire and maintain both complex knowledge and skills to provide optimal patient care. Traditional methods of critical care training (e.g., lecture-based education) often lack engagement, fail to provide an immersive learning experience, and may poorly influence behaviors such as prescription practices or protocol compliance. If the desired behavior could be tied to an internal and clinician-agnostic satisfier, then other approaches to behavior modification may be less relevant. Gamification is one approach that leverages visual, cognitive, and sometime auditory cues that may better achieve such a goal. Gamification employs a game style that promotes friendly competition, a competition style feedback and reward system, and avoids a failure mode but may incorporate delayed success (Fig. 1) (21). Gamification has been effectively deployed in a variety of educational settings and primary disciplines outside of CCM where its use is less common (22, 23). Gamification may therefore significantly improve knowledge retention, motivation, and learning outcomes in healthcare education (24). In addition, gamification can provide a safe and controlled environment for learners to practice and apply knowledge in realistic scenarios, without risk of patient harm, thereby complementing simulation-based training (5, 25). In these ways, gamification uses some aspects of adult—as opposed to pediatric—learning theory. These include engaging the learner’s cognitive structures including logical processes (cognitivism), experiential learning, development within a community practice structure, motivation, and reflection (26). Adult learning theories span multiple domains, not all of which will be represented within gamification (26). However, a key aspect of adult learning is its impact on behavior.
A game-based approach to behavior modification offers motivation that is aimed at the individual and not the institution and fits within the intention-change aspect of adult learning theory. Because its approach is visual and interactive, it may be more impactful for post-Generation X learners and healthcare clinicians (27). In most healthcare and wellness-related applications, gamification manifests in four ways: points, leaderboards or progress bars used to measure success, rewards or badges, and feedback systems (28, 29). These game elements can outline clear goals, deliver immediate feedback, and establish a sense of achievement, which can enhance motivation and learning outcomes (30, 31). Furthermore, it enables real-time feedback and performance assessment, allowing learners to track their progress and identify areas for improvement (32). Furthermore, there is a neurocognitive underpinning that supports gamification as a viable approach to modifying clinician behaviors and educational activities.
NEUROCOGNITIVE BASIS FOR GAMIFICATION
Initiating change is grounded in an individual’s intention to enact change (intention-change behavior theory), which is driven by that individual’s attitude toward that behavioral change (attitude-behavior theory) (33). Gamification directly engages the concepts of motivational behavior change, autonomy/self-determination, and social influences of the team that impact enduring behavioral change (34). More importantly, the visual cues and cognitive and emotional impact enable gamification to strongly influence behavior. Rewards or badges have long been recognized to have a neural basis that affects behavior. Studies of event-related brain potentials known as feedback error-related negativity (ERN) identified the basal ganglia and midbrain dopamine system as areas associated with reward prediction and reinforcement. Accordingly, electrophysiology changes induced by differences in positive and negative feedback (i.e., reward or loss) are identifiable. If the basal ganglia revises predictions for the better, it induces a phasic increase in the activity of the midbrain dopaminergic neurons, while predictions for the worse induces a phasic decrease. Dopaminergic signals are relayed to the frontal cortex including the anterior cingulate gyrus generating a feedback ERN responsible for reinforcement learning signals that results in adaptive learning behavior (35). This frontocentral negativity is now known as the feedback-related negativity that appears after negative feedback and affects reward processing and behavioral choice (36). Existing behavior modification approaches appear devoid of the neural links leveraged by gamification.
CURRENT GAMIFICATION USES
While little gamification has been deployed in CCM education training, it has been used more extensively in other venues. Gamification is currently used across multiple disciplines to introduce new knowledge, reinforce existing knowledge, learn new techniques and their application, decision-making, and procedural mentoring and can be extended to patient education as well (23, 24, 37–40). Gamification modalities include escape rooms, simulation competitions, leaderboards, clinical team training exercises, and augmented reality/virtual reality (AR/VR)-based games. One CCM-related medical professional organization (CHEST) developed an educational “escape room” to deliver continuing medical education at their annual meeting (41). Regardless of whether participants have existing relationships, escape room game play drives group problem solving in a zero-risk (e.g., no patients) space. Effective escape room design establishes an overarching goal, and then supports achieving that goal with specific design elements (Fig. 2). Learning objectives may be anchored within a specific, measurable, attainable, relevant, and timely framework that ties to elements of adult learning theory such as self-direction, connection, emotional investment, and social constructivism. The game-based nature of escape room learning ensures that knowledge acquisition occurs by direct experience. Gamification elements include performance stress (time-limit), challenges, and collaborative strategy evaluation with immediate feedback. Importantly, it also provides an opportunity to engage in and explore relational dynamics especially with unfamiliar team members. Successful completion of the escape room should be met with a reward (tangible, cognitive, or emotional), and followed with an opportunity to provide feedback.
While gamification is collaborative, SimWars is a competition-based gamification approach to trainee education that specifically explores learner decision-making (42). Teams compete against one another within a simulated environment to address patient emergencies before an audience. This experiential learning approach values teamwork, accurate assessment, correct decision-making, and presents different simulation-based learning environments. In this way, participants and observers learn together. Performance in front of an audience can be stressful and may be negatively perceived by some. Using only voluntary participants may mitigate such concerns. SimWars is sufficiently well-received that it has been used extensively in emergency medicine (discipline of origin), neurocritical care, obstetrics and gynecology, and multidisciplinary simulation interest groups. In addition to new knowledge, procedural skills and their assessment may be addressed using gamification.
SonoGames is a competition-based gamification training tool that is used by the Society of Academic Emergency Medicine at their annual meeting (43). It spans 4 hours, has multiple elimination rounds, and focuses on POCUS. Post-meeting survey data indicated that participants augmented their ultrasound knowledge-base and enthusiasm, and increased their clinical use of the modality (43). SonoGames incorporates group-based and hands-on learning in a no-risk environment, a combination that may particularly appeal to younger learners. Relatedly, a game-based approach to a variety of Emergency Medicine content was undertaken using randomly selected meeting participants in a novel team-based competition entitled “Challenging Hazards Amidst Observational Simulation in the Emergency Department” (44). Content areas included wilderness medicine, mass casualty incidents, critical care, airway emergencies, and ultrasound diagnosis across three rounds and eight teams. It included cognitive assessments, strategy, and procedural skills with unfamiliar partners while competing against other teams. Most importantly, the game-based approach mimicked what occurs during the process of patient care in a relevant setting for the participants, enhancing their ability to suspend disbelief and enjoy their educational activity.
A number of gamification-relevant behavior modification methods focused on trackable activities are employed with variable penetrance within acute care facilities. These include unit-based leaderboards, dashboards that detail performance metrics, as well as applications that report individual, team, or unit performance. Achieving aspirational goals compared with an established baseline (e.g., reduced time to patient disposition, reduced time to antibiotic delivery) may be encouraged using such competition-driven data displays that provide iterative feedback with visual cues indicating improved or worsened performance. Given the broad applicability of gamification methods to enhance skill and knowledge acquisition or application, its sparse use in CCM represents a broad opportunity for improvement.
FUTURE GAMIFICATION APPLICATIONS IN CCM
There are three primary domains in which gamification may be ideally implemented in CCM—guideline knowledge distribution and uptake, closure of the knowledge-to-practice gap, and team training (Fig. 3). New knowledge discovery continues to explode from cellular and molecular pathways as well as from large data set mining using machine learning/augmented intelligence techniques. Therefore, the doubling time of medical knowledge is now measured in months and continues to accelerate (45). As those data influence current practice, guidelines and pathways must be updated to reflect current evidence. Guidelines generated by professional societies summarize what is known about a scientific or clinical question and integrate a vast body of heterogeneous research into a useable form (46, 47). While guidelines are often straightforward, adherence to a new guideline, or changes to an existing guideline appears more problematic. Gamification can employ external motivators to drive desired behavior (e.g., guideline implementation adherence) (48). Furthermore, a game-based approach can also help reinforce when specific patients fall outside of the guideline to enable deviation from the implementation pathway. Real-world scenarios presented in serious (e.g., professionally relevant) games offer an opportunity to apply guidelines and get instantaneous feedback about whether the guidelines were applied as intended. In this way, gamification of a guideline serves as clinical training in an interactive fashion, partly mirroring the interactive aspects of the “flipped classroom” approach to traditional didactics (49). Additionally, such games can also help players work through the next most appropriate steps if guideline adherence does not repair the clinical problem. AR/VR-based games can also enhance existing knowledge (“upskill”) for clinically active critical care professionals—an approach that may work best after completing parent specialty and fellowship training (50). While guidelines or best practice approaches commonly suggest processes or therapeutics to embrace, some—such as the Choosing Wisely initiatives—also aim to decrease utilization of low value practices (51, 52). Therefore, gamification could also help de-implement specific actions as one method of improving care value as well as quality. Finally, serious games can be updated as guidelines evolve and avoid users potentially accessing an outdated guideline.
Once the information in a guideline is examined and embraced, utilization of that knowledge is key in improving care. Toolkits and bundles serve as implementation guides for the knowledge contained within a guideline (53). The divergence between understanding new knowledge and its clinical utilization (knowledge-to-practice gap) is a prime focus of implementation science. Correctly and rapidly applying evidence presented within a guideline requires confronting and overcoming real world challenges that may be patient, clinician, unit, facility, or health system based. Methods of addressing each of these aspects in a game-based format are termed “serious” games and doing so may be especially important to improve competency during rarely encountered clinical situations (e.g., trauma in nontrauma centers) (54). Importantly, gamification may be leveraged outside of acute clinical crises as it is applicable to routine but evidence-based care such as advanced care planning (55). Because gamification tenets may be applied to nearly every clinical care pathway, games can be tailored to unique clinical roles, and potentially, to preferred learning styles.
While in-hospital rescue teams are often comprised of clinicians who do not routinely work together (e.g., rapid response teams, code teams), other unit-based teams may have a more stable complement (56). Serious games could be used with or instead of simulation to train both kinds of teams to work together (57). Gamification is well suited to team training as individual participants could learn concepts related to teamwork as they interact with other team members, and multiplayer games could mimic ad hoc teams that must work together despite not knowing one another. Because simulation-based training, including using AR/VR approaches, has increasing penetrance in healthcare education, gamifying such training modalities seems like a logical evolution since the AR/VR environment parallels single or multilayer platform-based games embraced for relaxation and stress relief.
GAMIFICATION LIMITATIONS
Despite the multiple potential benefits of using gamification in CCM education and skill acquisition and implementation domains there are important limitations to be addressed. First, not all clinicians may be familiar with game-based education, and some may harbor an aversion to activities that present similarities to recreational gaming. Second, not all gamification structures will appeal to all potential users, nor will all aspects of gamification be applicable to all scenarios. For example, antibiogram compliance may be less appropriate for a competition-based approach that would be time to post-emergency department disposition. Third, the outcome assessment tool must be crafted for the specific event that is being tracked, rendering leaderboards potentially more suitable for a unit-based approach, but a dashboard more ideal for a service line or facility-based outcome metric. Fourth, where a competitive approach is deployed, there is a potential for excessive competition that may derail otherwise ideal interpersonal dynamics, or even clinical care. Fifth, gamification can be resource-intensive, requiring significant planning, equipment, digital resources, and manpower to orchestrate, which may make it difficult to implement on an ad hoc basis or within resource constrained environments. Sixth, because gamification in general uses a digital platform, non-technology savvy individuals may feel disenfranchised or specifically excluded. Finally, developing a gamification approach requires investments of both time and finances. However, it learners become more knowledgeable or skilled and can potentially provide higher quality care, those investments yield highly desirable returns.
CONCLUSIONS
Gamification is an innovative and effective approach to critical care education and practice. By incorporating game-like elements into learning, gamification can improve knowledge retention, motivation, and overall learning outcomes. Furthermore, gamification provides a safe and controlled environment for learners to practice and apply their knowledge without the risk of harming patients. As machine learning/augmented intelligence approaches expand within critical care, gamification approaches may readily adapt to conform to user profiles including their preferred learning style. As we work to close the knowledge-to-practice gap in critical care, gamification should be considered as an alternative approach to traditional education and training methods.
Footnotes
Dr. Bass is a current member of the Executive Board of the European Society of Trauma and Emergency Surgery. Dr. Chang is a current Executive Committee member of the Society of Critical Care Medicine and serves on the editorial board of Critical Care Medicine. Dr. Sorce is the current President-Elect of the Society of Critical Care Medicine. Dr. Subramanian is the Chief Executive Officer of Omnicure, Inc. Dr. Lane-Fall serves on the editorial board of Critical Care Medicine, is Vice President of the Anesthesia Patient Safety Foundation, and serves on the Board of Directors of the Foundation for Anesthesia Education and Research. Dr. Kaplan is a past-President of the Society of Critical Care Medicine (2020–2021), serves on the editorial board of Critical Care Medicine, and is an Associate Editor for Critical Care Explorations. The remaining authors have disclosed that they do not have any potential conflicts of interest.
REFERENCES
- 1.Munro CL, Savel RH: Narrowing the 17-year research to practice gap. Am J Crit Care 2016; 25:194–196 [DOI] [PubMed] [Google Scholar]
- 2.Halpern SD: Using default options and other nudges to improve critical care. Crit Care Med 2018; 46:460–464 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kerlin MP, Small D, Fuchs BD, et al. : Implementing nudges to promote utilization of low tidal volume ventilation (INPUT): A stepped-wedge, hybrid type III trial of strategies to improve evidence-based mechanical ventilation management. Implementation Sci 2021; 16:78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Groh F: Gamification: State of the art definition and utilization. Inst Media Inf Ulm Univ 2012; 39:31 [Google Scholar]
- 5.van Gaalen AE, Brouwer J, Schönrock-Adema J, et al. : Gamification of health professions education: A systematic review. Adv Health Sci Educ 2021; 26:683–711 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Boyce JM: Current issues in hand hygiene. Am J Infect Control 2019; 47:A46–A52 [DOI] [PubMed] [Google Scholar]
- 7.Datta R, Glenn K, Pellegrino A, et al. : Increasing face-mask compliance among healthcare personnel during the coronavirus disease 2019 (COVID-19) pandemic. Infect Control Hosp Epidemiol 2022; 43:616–622 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bassetti S, Tschudin-Sutter S, Egli A, et al. : Optimizing antibiotic therapies to reduce the risk of bacterial resistance. Eur J Intern Med 2022; 99:7–12 [DOI] [PubMed] [Google Scholar]
- 9.Evans L, Rhodes A, Alhazzani W, et al. : Surviving sepsis campaign: International guidelines for management of sepsis and septic shock 2021. Crit Care Med 2021; 49:e1063–e1143 [DOI] [PubMed] [Google Scholar]
- 10.Burry LD, Barletta JF, Williamson D, et al. : It takes a village…: Contending with drug shortages during disasters. Chest 2020; 158:2414–2424 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wilcox SR, Wax RS, Meyer MT, et al. : Interfacility transport of critically ill patients. Crit Care Med 2022; 50:1461–1476 [DOI] [PubMed] [Google Scholar]
- 12.Emanuel EJ, Ubel PA, Kessler JB, et al. : Using behavioral economics to design physician incentives that deliver high-value care. Ann Intern Med 2016; 164:114–119 [DOI] [PubMed] [Google Scholar]
- 13.Wright A, Henkin S, Feblowitz J, et al. : Early results of the meaningful use program for electronic health records. N Engl J Med 2013; 368:779–780 [DOI] [PubMed] [Google Scholar]
- 14.Barbash IJ, Davis B, Kahn JM: National performance on the Medicare SEP-1 sepsis quality measure. Crit Care Med 2019; 47:1026–1032 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Bradley AS, Burgart AM: Write-ups, retribution, and ‘DARVO’. ASA Monitor 2022; 86:25–26 [Google Scholar]
- 16.Feeser VR, Jackson AK, Savage NM, et al. : When safety event reporting is seen as punitive:“I’ve been PSN-ed!”. Ann Emerg Med 2021; 77:449–458 [DOI] [PubMed] [Google Scholar]
- 17.Kwasnicka D, Dombrowski SU, White M, et al. : Theoretical explanations for maintenance of behaviour change: A systematic review of behaviour theories. Health Psychol Rev 2016; 10:277–296 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kleinpell RM, Grabenkort WR, Kapu AN, et al. : Nurse practitioners and physician assistants in acute and critical care: A concise review of the literature and data 2008–2018. Crit Care Med 2019; 47:1442–1449 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Costa DK, Wallace DJ, Barnato AE, et al. : Nurse practitioner/physician assistant staffing and critical care mortality. Chest 2014; 146:1566–1573 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kim KM, Max W, White JS, et al. : Do penalty-based pay-for-performance programs improve surgical care more effectively than other payment strategies? A systematic review. Ann Med Surg (Lond) 2020; 60:623–630 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Deterding S, Dixon D, Khaled R, et al. : From game design elements to gamefulness: Defining“ gamification.” Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments; September 28-30, 2011; Tampere, Finaland, 2011, pp 9–15 [Google Scholar]
- 22.Krishnamurthy K, Selvaraj N, Gupta P, et al. : Benefits of gamification in medical education. Clin Anat 2022; 35:795–807 [DOI] [PubMed] [Google Scholar]
- 23.Chen TS, Hsieh PL, Tung CC, et al. : Evaluation of registered nurses’ interprofessional emergency care competence through the gamification of cardiopulmonary resuscitation training: A cross-sectional study. BMC Med Educ 2023; 23:359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Min A, Min H, Kim S: Effectiveness of serious games in nurse education: A systematic review. Nurse Educ Today 2022; 108:105178. [DOI] [PubMed] [Google Scholar]
- 25.Damaševičius R, Maskeliūnas R, Blažauskas T: Serious games and gamification in healthcare: A meta-review. Information 2023; 14:105 [Google Scholar]
- 26.Mukhalalati BA, Taylor A: Adult learning theories in context: A quick guide for healthcare professional educators. J Med Educ Curricular Dev 2019; 6:2382120519840332 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Halim E, Kristian H, Hebrard M: Investigating the key factors on xyz generations’ higher-order thinking skills in e-learning. 2022 IEEE International Conference on Cybernetics and Computational Intelligence (CyberneticsCom). June 16-18, 2022; Malang, Indonesia, Institute of Electrical and Electronics Engineers, 2022, pp 297–302 [Google Scholar]
- 28.Gentry SV, Gauthier A, L’Estrade Ehrstrom B, et al. : Serious gaming and gamification education in health professions: Systematic review. J Med Internet Res 2019; 21:e12994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Phillips EG, Nabhan C, Feinberg BA: The gamification of healthcare: Emergence of the digital practitioner. Am J Manag Care 2019; 25:13–15 [PubMed] [Google Scholar]
- 30.Sardi L, Idri A, Fernández-Alemán JL: A systematic review of gamification in e-Health. J Biomed Inform 2017; 71:31–48 [DOI] [PubMed] [Google Scholar]
- 31.Hamari J, Koivisto J, Sarsa H: Does gamification work?--a literature review of empirical studies on gamification. 2014 47th Hawaii International Conference on System Sciences. January 6-9, 2014; Waikoloa, Hawaii, Institute of Electrical and Electronics Engineers, 2014, pp 3025–3034 [Google Scholar]
- 32.Khaleel FL, Ashaari NS, Wook TS: The impact of gamification on students learning engagement. Int J Electr Comput Eng 2020; 10:4965 [Google Scholar]
- 33.Webb TL, Sheeran P: Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol Bull 2006; 132:249–268 [DOI] [PubMed] [Google Scholar]
- 34.Sailer M, Hense JU, Mayr SK, et al. : How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Comput Hum Behav 2017; 69:371–380 [Google Scholar]
- 35.Nieuwenhuis S, Holroyd CB, Mol N, et al. : Reinforcement-related brain potentials from medial frontal cortex: Origins and functional significance. Neurosci Biobehav Rev 2004; 28:441–448 [DOI] [PubMed] [Google Scholar]
- 36.Walsh MW, Anderson JR: Learning from experience: Event-related potential correlates of reward processing, neural adaption, and behavioral choice. Neurosci Biobehav Rev 2012; 36:1870–1884 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Lin DT, Park J, Liebert CA, et al. : Validity evidence for surgical improvement of clinical knowledge ops: A novel gaming platform to assess surgical decision making. Am J Surg 2015; 209:79–85 [DOI] [PubMed] [Google Scholar]
- 38.Chen IH, Ghazi A, Sridhar A, et al. : Evolving robotic surgery training and improving patient safety, with the integration of novel technologies. World J Urol 2021; 39:2883–2893 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Garett R, Young SD: Health care gamification: A study of game mechanics and elements. Technol Knowl Learn 2019; 24:341–353 [Google Scholar]
- 40.Hope DL, Grant GD, Rogers GD, et al. : Gamification in pharmacy education: A systematic quantitative literature review. Int J Pharm Pract 2023; 31:15–31 [DOI] [PubMed] [Google Scholar]
- 41.Kaul V, Morris A, Chae JM, et al. : Delivering a novel medical education “escape room” at a national scientific conference: First live, then pivoting to remote learning because of COVID-19. Chest 2021; 160:1424–1432 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Okuda Y, Godwin SA, Jacobson L, et al. : SimWars. J Emerg Med 2014; 47:586–593 [DOI] [PubMed] [Google Scholar]
- 43.Liteplo AS, Carmody K, Fields MJ, et al. : SonoGames: Effect of an innovative competitive game on the education, perception, and use of point-of-care ultrasound. J Ultrasound Med 2018; 37:2491–2496 [DOI] [PubMed] [Google Scholar]
- 44.Salerno N, Papanagnou D, Mahesh P, et al. : Challenging hazards amidst observational simulation in the emergency department: Advancing gamification in simulation education through a novel resident-led skills competition. Cureus 2018; 10:e3563. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Densen P: Challenges and opportunities facing medical education. Trans Am Clin Climatol Assoc 2011; 122:48–58 [PMC free article] [PubMed] [Google Scholar]
- 46.Kane-Gill SL, Winkle J, Kaplan LJ, et al. : SCCM/ACCM guideline and toolkit development pathways. Crit Care Med 2021; 49:1851–1854 [DOI] [PubMed] [Google Scholar]
- 47.Woolf SH, Grol R, Hutchinson A, et al. : Clinical guidelines: Potential benefits, limitations, and harms of clinical guidelines. BMJ 1999; 318:527–530 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.McKeown S, Krause C, Shergill M, et al. : Gamification as a strategy to engage and motivate clinicians to improve care. Healthc Manage Forum 2016; 29:67–73 [DOI] [PubMed] [Google Scholar]
- 49.Sezer B, Abay E: Looking at the impact of the flipped classroom model in medical education. Scand J Educat Res 2019; 63:853–868 [Google Scholar]
- 50.Gasteiger N, van der Veer SN, Wilson P, et al. : How, for whom, and in which contexts or conditions augmented and virtual reality training works in upskilling health care workers: Realist synthesis. JMIR Serious Games 2022; 10:e31644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Halpern SD, Becker D, Curtis JR, et al. ; Choosing Wisely Taskforce: An official American Thoracic Society/American Association of Critical-Care Nurses/American College of Chest Physicians/Society of Critical Care Medicine policy statement: The Choosing Wisely® top 5 list in critical care medicine. Am J Respir Crit Care Med 2014; 190:818–826 [DOI] [PubMed] [Google Scholar]
- 52.Zimmerman JJ, Harmon LA, Smithburger PL, et al. : Choosing wisely for critical care: The next five. Crit Care Med 2021; 49:472–481 [DOI] [PubMed] [Google Scholar]
- 53.Martin GS, Kane-Gill SL, Nadkarni V, et al. : The evolution of toolkits and bundles to improve the care of sepsis patients. Crit Care Med 2021; 49:1849–1850 [DOI] [PubMed] [Google Scholar]
- 54.Mohan D, Fischhoff B, Angus DC, et al. : Serious games may improve physician heuristics in trauma triage. Proc Natl Acad Sci U S A 2018; 115:9204–9209 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Mohan D, MacMartin MA, Chelen JSC, et al. : Development of a theory-based video-game intervention to increase advance care planning conversations by healthcare providers. Implement Sci Commun 2021; 2:117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Fernandez-Moure J, Pascual JL, Martin ND, et al. : Emergency response teams in and outside of medicine—structurally crafted to be worlds apart. J Trauma Acute Care Surg 2019; 86:134–140 [DOI] [PubMed] [Google Scholar]
- 57.Billebot M-N, Cotteret M-A, Visier P, et al. : Measurement and knowledge in health. In: Connected Healthcare for the Citizen. Picard R. (Ed). Paris, France, Elsevier, 2018, pp 59–83 [Google Scholar]