Taking Serious Games Forward in Curriculum and Assessment: Starting Infusions Right Every Time

Abstract

Technology-driven educational modalities are increasingly utilized today in a variety of forms. Different combinations of the spectrum of simulation-based learning, the use of virtual reality, augmented reality, mixed reality, and serious gaming continue to gain traction on various educational platforms. In this paper, we share the formation of our project team to plan and execute a serious game on starting infusions and the use of infusion pump for nursing and health-care staff. The incorporation of element of assessment is also discussed.

The various phases we went through included:

1. Learning needs assessment and conceptualization

2. Assembly of project team

3. Transfer of medical concepts

4. Storyboard and content production

5. Learners' experiential mapping

6. Testing of the prototype

7. Beta testing and release of the final product

The collaborative work and coordination between the subject-matter experts together with the technical production team is critical. Issues such as assessment and debriefing in serious gaming were also addressed, not forgetting the need to ensure that, above all, learning must take place.

INTRODUCTION

The educational environment surrounding health care continues to evolve and change. The armamentarium of learning and teaching tools as well as methodologies has increased over the years. The options for both teachers and learners too have been enhanced. Besides these, health-care professional educators have to grapple with the intergenerational differences and preferences in learning methods of their learners in order to keep them engaged. Engagement refers to learners' interest and involvement in their learning, learning environment, and educational interventions. Sustained engagement is also important throughout the life span of practice of the health-care professional, which may span decades, whereby they have to be involved in ongoing and continuing professional development. Thus, newer modalities are constantly being sought to enhance teaching and learning in the health-care professions. This will eventually impact patient care and health-care outcomes.[1,2,3,4,5,6,7,8]

Of late, more and more technology-enhanced learning tools are being utilized. These have added a paradigm shift to the way health-care education is planned, strategized, and executed today. Some of these examples include simulation-based learning, virtual-, augmented-, and mixed reality platforms as well as serious gaming. The rationale for these is to make real-world tasks the basis of a learning environment, promote the application of knowledge, and transfer to actual practice. The focus of these is on “learning from doing.”[1,6,7,8,9]

SERIOUS GAMES IN HEALTH CARE

Serious games (SGs) are digital games which prioritize learning needs rather than entertainment. It represents an interactive computer application with or without a hardware component that has a challenging goal, is fun, and incorporates concepts of scoring or competition. The goal of its core design is to teach and train a spectrum of learners and some even focus on assessment and tracking of learners' progress. SG can be used to teach a variety of skills, train for particular behavior, and even inspire critical thinking or decision-making process in its fully controllable environment.[2,3,6] [Figure 1] SG can cover the spectrum of treatment, management, recovery, and rehabilitation in health care.[4,5,10,11] [Table 1]. The science behind SG is interesting and is based on human characteristics of being immersed in a “world” whereby they can be involved and are engaged to solve a mystery or problem, while looking for cues and clues. It can also represent a form of “escape.”[6,11]

Figure 1.

An example of serious game in health care. Hazardous scenario that may potentially happen: Bringing hazardous object into Zone 4 when the MRI is still in operation mode. Players will then be able to decide the needed responses based on the scenario. From: NNI-MRI Serious Game, 2020

Table 1.

Spectrum of the use of serious games

Serious games: Spectrum of use
Performance management ± feedback
Knowledge reinforcement
Real-world applications, for example, remote hands-on training for skills
Risk mitigation, for example, serious games based on real-world situations can help mitigate risk and potential hazards
Mistake-driven learning (similar to deliberate practice to reach mastery)

In SG, learners (or players) interact with the game world through probing, seeing, and experiencing things in certain contexts. The learning is organized in a systematic and stepwise fashion such that it makes sense to the learners. Just like simulation, it can allow repetitive practice to enable mastery of skills in a safe environment. Learners' confidence can be enhanced through reinforcements with immediate, real-time, and dynamic feedback on their performance [Figure 2]. Adult learners can also use these for reflection. The presence of cognitive rewards in some games can further boost engagement levels. SG, which is often thought of as the intersection of learning, simulation, and gaming, can harness the power of competition and motivate participants to optimize their own performance. The SG platforms also allow for team-based learning and opportunities for collaborative learning (such as interprofessional education), with real-time communications in a coordinated fashion.[5,12,13,14]

Figure 2.

Examples of the dynamic feedback based on the decision/action made by the player in an MRI Safety serious game. Left: From NNI-MRI serious game, 2020: Positive feedback given with elaboration on the likely outcome from a wrong decision. Right: From Infusion Pump serious game, 2020: Highlighting the important of specific action. As seen from the Infusion Pump game, timer could be added on the screen to induce stress, testing players' ability to make correct decision when stressed. NNI: National Neurosciences Institute. MRI: Magnetic Resonance Imaging

The level of technological development today has given us the capability to create an immersive environment, very close to the actual working environment in which the healthcare provider works. These SGs can be used for “anywhere, anytime learning” and they provide the learners with the flexibility and independent practice.[7] This type of learning appeals especially to the younger generation as well as millennial learners, who tend to be more mobile and dynamic in their learning approach. It is a form of active learning through solving clinical problems, which means that learners can develop analytical skills and strategize thinking, knowledge, decision-making, communications as well as psychomotor skills. SG may also utilize various emerging technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR) in various combinations.[5,11] Studies have shown that the integration of the elements of fun and competition in SG can help reduce the stress of learning, increase intrinsic motivation, and make learners look forward to learning. It is important to understand that in many contexts, the use of SG is a supplement to face-to-face learning and training.[5,6,11,13,14]

SGs have good potential to provide quality, cost-effective, and a novel approach to learning, which is portable. It is important to make the appropriate choice of which SG to use for the various learning goals if there are plans to introduce them to supplement the curriculum [Supplementary Figure 1 ^{(279.6KB, tif)} ]. It is important to not lose focus that learning is the goal of SG, despite all the attractive edutainment capabilities that technology can offer today.[15,16,17]

ALL ABOUT INFUSIONS

Starting infusions is a fundamental nursing task, which is conducted very frequently for a variety of medications and fluid. In some areas of work such as intensive care units, critical care areas, and emergency departments' resuscitation rooms, critical drugs and infusions must be started quickly and in a timely fashion, especially for the very ill patients and those with life-threatening conditions, thus the need for familiarity, adequate knowledge, and versatility.[18] In view of all this, regular, deliberate practice for starting infusions will be of value. Current practice and learning is by didactic sessions, followed by hands-on practice in the actual work environment, where the senior nursing staff will coach the junior ones. Starting infusions and understanding the workings of infusion pumps are part of the compulsory nursing task assessment which is part of their skill accreditation process.[19] Looking at new ways of learning and assessment is part of the dynamic process in adult learning. Following a comprehensive review by our team, it was decided that a SG format for starting infusion training would provide a new and interesting perspective [Figure 3]. From our initial scan, there were no suitable commercially available SG for learning about and starting infusion, which included assessment, in the market. During our search, we specifically looked at the elements in the framework in Supplementary Figure 1 ^{(279.6KB, tif)} to try to find the most comprehensive available game. The purpose of our infusion SG would be to help nurses achieve a given set of goals which are required in their job by playing the game. It will: (1) provide training, instruction as well as performance assessment and (2) allow limited repeated attempts/practice with feedback. This SG will have the knowledge sharing (cognitive) and instructions on the procedure (skills) and be able to motivate (attitude) the learners. It would have appeal to both the younger nursing staff, new recruits, and the senior staff.

Figure 3.

The infusion pump interface and settings are very similar to the actual infusion pump. This allows the player to explore the functions and learning through in-game application. From Infusion Pump serious game, 2020

Our initial decisions involved:

1. Identifying the need for the SG

2. Identifying the stakeholders and categories of users/learners

3. Identifying the game functionality and constraints, which also involves defining the interaction mechanisms.

In planning this SG, it was also important to understand the principles of administration of intravenous medications as well as the operations of the infusion pumps.[18,19,20,21] These are available in the institution's policy and procedure documents.[18,19] For administration of intravenous medications/infusions, there are the following steps[18,19] [Figure 4]:

Figure 4.

Testing the player on the “7 rights for verification of medication administration” with our electronic health record system. From Infusion Pump serious game, 2020

1. Ensuring the seven rights for verification of medication administration (right patient, right medication, right dose, right time, right route, right diluent, and right rate of infusion)

2. Prescription and preparation

3. Administration

4. Monitoring and documentation (into the electronic health record system)

FROM INFUSION PUMP SG, 2020

Upon retrieval of the medication, it is also crucial to check the drug name and its dose on the vial. The integrity of the vial, the expiry date, the consistency of the medication (e.g., not discolored or cloudy) as well as the manufacturing company can all be checked on the vial itself. As for the operations of the infusion pumps, the steps the nurses need to master would include:[20]

1. Setting up the infusion pump

2. How to change the infusion rate/dose while the pump is in progress

3. How to deliver bolus dosing

4. How to put the pump on standby mode and to deactivate the standby mode when needed

5. Switching off the pump

6. Follow-up documentation (in the electronic health record system)

All of the above steps have to be considered and shared with the relevant parties in the project team for our SG.[20,21]

PHASES OF DEVELOPMENT OF THE INFUSION SERIOUS GAME

In planning and starting a new SG from scratch, the following represent the phases we went through:

Learning needs assessment and conceptualization of serious game

This is the first step but extremely critical. The level of technological development today has given us the capability to create an immersive environment, very close to the actual working environmentin which the healthcare provider works. Focused group discussions and feedback were also obtained from different groups of nurses. Another source of input was from the review of the errors related to starting and running infusions over the last few years. Errors such as wrong dilution and wrong configuration of the delivery rate on the infusion pump were observed. This gap analysis helped us planned in getting from the current state to the desired state. We also assessed the different training options. The focus must be on meeting the needs for training nurses to carry out their roles, within the priorities of the ED. The choice must also be evidence based, with added value compared to the current way of doing it. SG methodology was decided following review of all the necessary factors, the time savings, thrust toward individualized learning in one's own time, and also the ability for deliberate practice.[20,21,22] Other factors which may have to be considered would include the available technology, cost considerations as well as the return of investment (i.e., the value of expenditure related to the training/development) for the project.[23]

Assembly of the project team

The members of the project team, their commitment, expertise, and attitude are all important factors. These personnel may be involved at the different stages of the development of the SG, but their coordination and expertise is crucial.[1,8] Some may serve as subject-matter experts while others provide technical expertise. Subject-matter experts must establish the objectives of the SG, take charge of scenario development, and scope the project. Technical developers must be made to understand the objectives, establish the timeline and tasks to be completed, and be involved in documentation. Regular meetings and updates are important to ensure continuity. During the COVID-19 pandemic, the team met frequently on virtual platforms. When face-to-face meetings were arranged, these were only for a few members to come together to brainstorm certain parts of the game development. For this project, the team comprised of:

• The principal investigator, an emergency physician

• Three ED nurses

• Video game developer/interactive media developer

• Graphic artist (for both two-dimensional [2D] and 3D)

• Testers (for the prototype)

• Supporting administrative staff

Transfer of medical/health-care concepts [Supplementary Figure 2 ^{(270.9KB, tif)} ]

Transfer will happen at different phases.[24,25] Doctors and nurses represent the subject-matter experts in this SG. They have to share the concepts and principles with the other team members. They will play a crucial role in transferring the key concepts in the learning theory in terms of the knowledge, skills, and attitude. This builds upon the clinical reasoning which the nurses who play the game will apply. As most health-care-related tasks are considered complex tasks, each individual step or phase would have to be planned carefully and accurately in order to reflect as close as possible what happens in the real situation in the ED. The concept transfer will orientate the other members to the medical/clinical information to be used. Conveying medical information to nonmedically trained developers and technical experts can be very challenging. Thus, our team brought the latter on-site visits, performed demonstrations to illustrate the actual steps to be undertaken by the nurses, and shared relevant equipment and photographs with them. For the infusion pump and its working mechanism, short tutorial sessions were also planned. The project developer used these concepts to develop the next phases of the SG, for example, the storyboard.

The other phase where transfer will occur is after the nurses “play” the completed SG, they will be expected to transfer the knowledge and skills to the actual ED environment. This transfer is classified as: [16,22,25]

1. First class transfer: Whereby the SG reflects and portrays the real-world practice very closely and accurately, both in the literal and specific senses

2. Second class transfer: This is where what is learned from the SG may be applied in part or in principle to a known complex real-world situation. Here, the ideas, concepts, and knowledge will need to be juxtaposed to some new problem or situation in the workplace.

Storyboard and content production [Supplementary Figure 2 ^{(270.9KB, tif)} ]

The storyboard sketches the ideas for characters and scenes and places them in an organized order such that they can be used systematically to make sense to the sequence of events. Storyboards originated in the filmmaking industry and are powerful enough to be able to narrate the storyline in sequence. The initial and simple storyboard sketch can be responsible for the eventual effective storytelling in the SG. It encompasses the conceptualization of plans, visualization of the scenes as well as the graphic representation as to how the SG will unfold. All details can be included in the sketch which will help guide the subsequent production steps. Complex algorithms and guidelines should be simplified to enhance understanding. It is necessary for the subject-matter experts to work with the SG producer in order to come up with the best possible and comprehensive storyboard.[26,27]

Experience mapping for learners

This refers to the personalized learning experience mapped out for the learners.[28,29] In this case, it will be mapping the learning path for the nurses in the ED. This process will be for new nursing recruits, junior nurses as well as the more senior, experienced nurses. This is part of their learning journey and reflects the learner-centric approach of the whole project. In planning our SG, we had to bear in mind the diffusion of innovation model,[30] which described in general categories of learners/players as:

• Innovators: 2%–5%

• Early adopters: 13.5%

• Early majority: 34%

• Late majority: 34%

• Laggards: 16%

Diffusion is described as a social process in response to learning about an innovation. The necessary information is often shared among people of a certain social system or group. In this case, it will refer to the nurses in the emergency department. The dependent variable here is the time to adoption. Many factors may affect this, for example, do the learners have choices available to them, the ease of the use of the SG or new technology, the accessibility, and time taken. For our SG, we plan to eventually migrate the current mode of practice and assessment onto this platform.[30,31]

We may then have to allocate appropriate training time and closer supervision for certain groups. At the start, we will also give briefing to help create awareness, understanding and buy-in from all learners/nurses concerned. In order to ensure the best play and learning experience, all team members will discuss and determine the game functionality, flow, feedback to learners, and the game modeling. This phase may take a while, and we encountered multiple changes during these steps. It takes some effort, but it is worthwhile doing this adequately to have the best outcomes.

Prototype testing

This must be done in order to ensure the proper usability and playability of the SG. The prototype testing can be carried out at different phases such as the design phase, creation phase, or the final testing phase. User acceptance is key. Reviewing partial systems early in the development phase facilitated our team's discussion on the design and functionality. Changes were then made accordingly. There were very active inputs for content accuracy and assessment as well. We also used a spreadsheet to communicate the inputs and expected outcomes for every user action. All sections of our game had to be tested adequately, errors identified and corrected. We found this feedback very useful to address any bugs, usability problems, and content modifications.[32] All revisions were reviewed before the next meeting by members. Even as these steps were time-consuming, it is important not to rush these stages. It has been shown that when sufficient time is allocated to testing and trying out the SG, the better the learning process and learning outcomes. This was also linked to the ability for repetitions and continuous feedback.[33,34]

Beta testing and final product

With all the edits and corrections completed, the final usability testing can be done. Following this, the final product is ready for beta testing and then release. The actual validation and assessment of our SG will then be conducted.[32,34] Our team will be following up closely on this aspect of the implementation of the SG (the term implementation science refers to what happens before, during, and after an innovation such as our SG is adopted).

DEPLOYMENT AND DEVELOPMENT

Being the more technical parts of the SG, these aspects have critical inputs from the appointed game developer.

1. Deployment devices

   1. Tablets were the primary device we planned for (primary iPads/Android tables). Sizes of 9” and above are recommended for maximal impact of the game. Based on our earlier feasibility studies on the nursing staff, providing a convenient way to access the training courseware is of significant priority, also ensuring that training can take place anywhere and at anytime of the day is important. With the shift work hours put in by our nurses, this flexibility will appeal to them.

2. Development platform

   1. Web-based application
      1. Considering the need for ease of deployment and ease of the access, web-based application is recommended as the deployment platform. We anticipate that there may be large numbers of nurses accessing the courseware during certain peak hours, thus planning for this is important as it can increase the window of training time. It should also be accessible from both home and their workplace.
   2. HTML5, CSS3, JavaScript, and Web Canvas
      1. Following the web standards, the solution is able to be deployed on any server easily without having to install complex server configurations. The minimum recommended server specs is Windows/Linux server with 16 or 32GB ram. This is sufficient to push the application to the targeted client devices.

3. Development tools

   1. Game engine – Scirra Construct 3
      1. For web-based game, Scirra Construct 3 is one of the more matured game engines in the industry. The popularity reflects the ease of use and the powerful features to create the SG which can meet the training needs. It is able to handle the graphics elements and maintain the performance (not laggy and dropping of frame rates).
   2. 3D modeling tools – Blender is the main 3D modeling tool utilized to re-create the hospital and emergency ward scenes. 3D props like the Infusion Pump is modeled and textured to match the real infusion pump. Other 3D props like the GTN ampoule, the syringe, and plunger are modeled very close to the actual thing. This gives the trainee the sense of handling the “real” thing and enhances their learning experience.

4. Vulnerability test

   1. This is done by scanning the code for vulnerabilities to ensure that the courseware are free from any vulnerability attacks from external hacking.

5. Application performance test

   1. Ensuring that the loading time is in keeping to the minimum and gameplay is smooth is critical so as to give the trainee/student the best possible user experience

THE ASSESSMENT IN SERIOUS GAME

One critical differentiating factor between SG and entertainment games is the presence of assessment. How this will be done and tracked is crucial for our SG as it will impact frontline nurses' capabilities and their accreditation. We focused on the fact that measuring, discussing, and reasoning about the gameplay effectiveness is critical in order to lead to a SG which will enable the players to reflect. The validity and efficacy of the SG, if poorly designed, can make the conclusions invalid, especially if there is no evidence to support the efficacy. Grounding the game in well-defined theories is important. In some cases, whereby ready-made and available games are being considered, teams may execute a randomized controlled trial using the traditional method versus the proposed new method.[14,22]

Some aspects of the validity we went through with the developers for our SG included: [27,33,34]

• Content validity: This is the degree to which the SG content adequately converts the dimensions of the medical construct it has to educate the learners/players with

• Face validity: This refers to the degree of resemblance between the medical constructs featured in our game with that in the real situation

• Construct validity: This refers to the inherent differences in the outcomes of assessment of experts versus novice in the gameplay

• Predictive validity: This is the degree of concordance of the SG outcome and task performance in reality based on a validated scoring system.

Our team went through some of these questions when planning our assessment format:

1. What outcomes are the SG expected to deliver? (e.g., engagement, practice, and increased participation rate)

2. What assessment methods would be used?

3. How many levels or phases will the SG be divided into? Can learners join at certain levels or do they have to commence from the beginning every time they restart gameplay?

4. How many times can each level or phase be played?

5. Are there time limits set for each of the levels, for example, 5–8 min, 10–15 min, or 15–25 min?

6. Will the context be set only in English or other languages?

DEBRIEFING

Proper debriefing is an important part of the game playing activity. The reflection and sharing are the crucial elements that convert the game experience into learning. Some degree of learning does occur when the SG is being played, but the deeper lessons are drawn out during a debriefing session. In fact, debriefing sessions can last longer and be more engaging than the gameplay itself, even if it is less flashy and sexy than the game itself. Debriefing must be a design consideration right from the time the SG is conceptualized. It can be built into almost every single player game either on the computer or via the Internet. Designers can even set up discussion forums as needed for players to discuss, share, and cross-fertilize their ideas and generalize their learnings from the SG. Debriefing after SG is an area that requires more research.[11,27,32,35]

DISCUSSION

As more and more technology is used in education, it is important to realize that learning must continue to take place effectively despite all the frills and “entertainment.” For our infusion SG, we were cognizant of this and strived to ensure that the four pillars of learning (attention, active learning, feedback, and consolidation) were addressed from the conceptualization stage.[32,36]

We share how it was done in the following section:

Attention

In any learning activity, learners must pay attention and stay alert in order to ensure understanding. This is the same with SG. It has been shown that with at least a medium level of arousal in SG, there will be the highest gain in knowledge.[36,37] With our SG, we added graphics, avatars and used sounds as well as interactive activities to ensure engagement. This way, we can capture their attention and motivation.[34,38] We also took time to carefully select the challenges to be included, with inputs from the nurses on our project team, as we felt this can help hold the attention of the players with a positive impact on their learning. A good SG will immerse the player in the task environment. This can facilitate gaining mastery, but we must ensure it enables the achievement of the learning outcomes as well.[33]

For each stage of the SG, we also ensure adequate and clear instructions were provided.[39] This is important so the person playing the game will not get “lost” as to what to do and how to negotiate the SG. We also moved a step further to provide tutorial to the nurses as we rolled out the SG to show them the details and steps, with supportive instructions. Such instructional support has been shown to be associated with better learning outcomes [Figure 5].

Figure 5.

Example of a supportive instruction used in serious game, with guideline to help player on the needed procedure. From Infusion Pump serious game, 2020

Active learning

SG can facilitate active learning more effectively than traditional didactic teaching and lectures. This is mostly due to the active engagement and participation of the players. We planned many interactive segments in our SG and ensured these were integrated strategically at the right points to stimulate action on the part of our nurses playing the game.[7,12,32]

Feedback

Feedback is known to be important for learning to take place. Its positive impact is generally well accepted. With SG, we train our faculty to provide feedback on the task at hand and not on the learner as this is more effective. Various SGs integrate feedback differently, for example, through scores, the use of number of stars awarded or experiential points. As this is a relatively new area, we plan to follow up closely with our SG implementation and feedback system to be able to share further on these.[2,3,8]

Consolidation

This refers to the process that enables our memory to consolidate the learning that has taken place. Usually, our first experience at an activity such as a SG will be a “conscious” one. Repeated attempts will make us more familiar and we will begin to assimilate the activity and experience into our subconscious neural networks. The latter is also linked to pattern recognition, utilization, and ability to apply the knowledge learned at our workplaces or elsewhere. As with simulation-based learning, the repetitions are effective to reach mastery. It has also been shown that rest periods between repetitions also enhance learning and memory.[32,34,40]

CONCLUSION

With technology-driven education, it involves a process of trial and error in incorporating them into our existing curriculum. We went through the process of searching and exploring the most suited platform for our SG. We also looked at the evidence for the use of the various modalities. Working collaboratively with industry partners is crucial, just as we need to have an open mindset to adopt change in bringing on newer techniques and methodologies to keep our learning exciting, relevant, adaptable, and accessible. Choosing the right SG or planning one on your own is a decision you have to make, depending on your capabilities, funds, and timeline. Most importantly, the desired outcome of meeting educational key performance indicators must be met, no matter which methodology, platform, or SG you choose.

Financial support and sponsorship

This project is funded by the Emergency Medicine, Academic Clinical Programme Research Funding, SingHealth.

Conflicts of interest

There are no conflicts of interest.

Supplementary Figure 1

Framework on the elements to consider when choosing a serious game

Supplementary Figure 2

Elements to consider when planning or evaluating a serious game. These are some of the elements that will determine how “responsive” the serious game is to the learners. For game designers, these are the elements to be integrated during game production