Vax the Max, a Gamification Intervention for COVID-19 Vaccination Task Engagement in the Inpatient Setting

Abstract

The development of vaccines for SARS-CoV-2 has offered game-changing protection from severe disease and death from COVID-19. Despite efforts to vaccinate individuals in the ambulatory setting, a sizable minority of the US population remains unvaccinated for COVID-19. For unvaccinated patients, hospitalization for non-COVID-19 illness offers another opportunity for vaccination. In the summer of 2021, the authors noted that COVID-19 vaccination rate for medicine inpatients at their hospital had fallen to 5.3 vaccine doses administered per 4-week block. In response, they created Vax the Max, a gamification program of COVID-19 vaccination tasks where internal medicine resident teams were awarded points for completing these tasks. Residents were anonymously surveyed after participation. The hospital demonstrated higher rates of administering the initial COVID-19 vaccine dose and completing the vaccine series in the inpatient setting per 4-week plan-do-study-act cycle after implementation of Vax the Max (5.3 versus 8.8 doses per plan-do-study-act cycle). Among residents, 76.8% reported that Vax the Max spurred their COVID-19 task engagement, and 66% reported that a similar gamification model could be utilized for a different clinical task in the future. An increase was observed in the COVID-19 vaccination rate for medicine inpatients after launching the Vax the Max competition. This occurred in the setting of resident turnover every 4 weeks, which normally makes practice sustainment more challenging. Despite this, a high degree of engagement was produced by itinerant residents. There is potential to explore similar gamification approaches involving resident physicians in areas of quality improvement and patient safety.

Introduction

Problem Description

Since first being detected, infections from SARS-CoV-2 have resulted in millions of hospitalizations and more than one million deaths in the United States.¹ The development of safe² and effective COVID-19 vaccines has been game-changing during the pandemic.^3,4 Since first being introduced, there have been many outreach campaigns aimed at persuading unvaccinated individuals to accept the COVID-19 vaccination. Locally our health care system and other established innovative COVID-19 vaccination venues⁵ and conducted targeted outreach to patients with unstable housing.⁶ Despite these efforts, a sizable minority of the population in the United States remains unvaccinated against COVID-19.¹ Hospitalization for non-COVID-19 illness presents an underutilized opportunity to address patients’ and families’ questions and concerns related to COVID-19 vaccination.

Although hospitalization represents an opportunity to engage with patients about their vaccination status, there are competing interests that divert clinicians away from preventative medicine efforts. Previous work has demonstrated a wide degree of heterogeneity to hospital-based vaccine programs.^7,8 These efforts may be further complicated for resident trainees due to the increasing density of tasks needed to care for acutely ill patients.^9,10 This presents a formidable challenge to pursuing dialog around COVID-19 vaccination status in the inpatient setting at training facilities.

The use and implementation of quality improvement skills are key competencies in medical education for resident physicians,^11,12 including a recent mandate by the American Council for Graduate Medical Education.¹³ While substantial work has been undertaken to increase the prevalence of quality improvement medical education,^14,15 challenges remain to encouraging trainee engagement in quality improvement in the clinical environment.^16–18 The etiology for these challenges is multifactorial and includes competing educational and clinical demands placed on trainees, time constraints, and limited personnel and technological support availability.¹⁹

Available Knowledge

Toolkits and protocols have been developed to aid in the decision-making and operationalization of ambulatory COVID-19 vaccination.²⁰ Hospital operations and inpatient populations differ from those of ambulatory settings. Numerous barriers limit the ability to translate and implement outpatient methodology to the hospital setting.

Work completed before the COVID-19 pandemic has demonstrated effective hospital-based strategies for increasing immunization for some vaccine-preventable diseases.^21,22 Some hospital systems implemented large-scale inpatient COVID-19 vaccination programs that relied on electronic medical record (EMR) integration and nurse-driven protocols.²³ These programs faced challenges such as competing inpatient priorities and changing vaccination eligibility. Furthermore, these programs were labor intensive, with staff dedicating close to an hour and a half per vaccine administered in one program.²⁴ While many hospitals have created inpatient COVID-19 vaccination programs, none appear to have utilized residents as the primary agents of this change.

Rationale

An examination of admitted patients at the study hospital in the summer of 2021 found that only 57% had documentation of a COVID-19 vaccination. The team also found that the average rate of COVID-19 vaccine administration to medicine inpatients was 5.3 doses per 4-week block. In the EMR, each patient’s COVID-19 vaccination status was prominently displayed in a banner located at the top of the chart. However, this data were often incomplete as it did not include vaccinations administered outside of the health care system. Discussions with frontline resident physicians in the summer of 2021 determined that COVID-19 vaccine status and ordering COVID-19 vaccination was inconsistently prioritized and completed. At that time, the study hospital did not have a standard process to identify patients eligible for a COVID-19 vaccination and ensure vaccine administration. A committee was formed to address the suboptimal rate of COVID-19 vaccination among inpatients. To address this problem, a program was created to increase COVID-19 vaccination engagement with acute care medicine patients through tailored engagement with internal medicine resident (IMR) trainees over a 7-month intervention period.

Specific Aims

The primary aim of this study was to assess the impact of a quality improvement intervention, specifically the gamification of COVID-19 vaccine related tasks, on inpatient COVID-19 vaccine administration for inpatients cared for by trainee physicians on a medicine acute care service. Secondary outcomes included determination of patients’ COVID-19 vaccination status, offering counseling to vaccine-hesitant patients, and the rate of wasted vaccines doses.

Methods

Context

The study hospital is integrated into a regional and national health care system in Washington State that serves United States military veterans as part of the Veterans Health Administration. The facility has 210 licensed acute care beds, admitted 4827 patients in 2020, and treats patients with a variety of complex medical and surgical problems. It is also a training site for the University of Washington IMR program. Four of the five medicine teams at the hospital are teaching teams composed of resident physicians and medical students. These teaching teams care for the majority of acute care medicine patients admitted to the medical center. Acute care medicine teams are comprised of an attending physician, 1 PGY2 or PGY3 (senior resident who is 2 or 3 years into their postgraduate medical education), 2 PGY1s (intern who is 1 year into their postgraduate medical education), a clinical pharmacist, and 1 or 2 medical students. Each medicine team can care for a maximum of 18 patients at any given time.

The hospital has a dedicated quality improvement team that includes clinical and support staff for patient safety and quality improvement interventions. This project was championed by the Chief Resident for Quality and Safety (CRQS) based at the medical center.²⁵ This position is a 1-year training program developed by the Veterans Administration National Center for Patient Safety in partnership with the University of Washington. During this year, the CRQS engages in quality improvement and patient safety endeavors at the hospital.

For this project, a quality improvement team was formed consisting of the CRQS, local director of quality improvement, inpatient chief residents, hospitalist section chief, chief of hospital and subspecialty medicine, and the chief nursing officer. Outcomes for success were developed by this multidisciplinary committee. Before the summer of 2021, the hospital had utilized a nursing-driven protocol to screen for COVID-19 vaccine eligibility. As the pandemic evolved, nursing was no longer able to perform COVID-19 vaccine screening due to nursing staff redeployment and turnover related to the COVID-19 pandemic. It was decided that IMRs, given their broad footprint within the hospital and direct patient contact, would be the agents best suited to encourage vaccine acceptance.

IMRs rotate through the hospital for four weeks before moving to another training site, therefore their replacements must be trained monthly. To coincide with this turnover, each plan-do-study-act (PDSA) cycle was four weeks.

Intervention

The process for COVID-19 vaccination necessitates that a clinician first determines patient eligibility and then confirms their interest in receiving the vaccine (Figure 1). For most patients, this can happen at several points during their hospitalization: on admission, during their treatment course, or at discharge. Having defined these parameters, they were mapped onto Michie’s Behavior Change Wheel.²⁶ These efforts were paired with discussions with IMR and supervising physicians regarding opportunities and barriers to COVID-19 vaccine administration and identified persuasion and incentivization as potential mechanisms for behavioral modification.

Figure 1.

Process map of IMR workflow for vaccinating a patient for COVID-19 during an inpatient admission. Abbreviation: IMR, internal medicine resident.

The central intervention was the creation of Vax the Max, a gamification of COVID-19 tasks. Gamification has been utilized in medical education contexts,^27–29 but only sparingly in quality improvement contexts.^30,31 Vax the Max was developed after the primary and secondary drivers for increasing inpatient COVID-19 vaccination rates were explored (Figure 2). Residents were informed of this intervention during the in-person orientation to their rotation at the hospital. This was followed by an email to them that included details about Vax the Max participation. Vax the Max awarded points to various forms of COVID-19 task engagement (Appendix 1, Supplemental Digital Content 1, available at http://links.lww.com/AJMQ/A85). Points awarded per task were weighted differently. Initial vaccination or completion of the vaccine series was allotted the most points as this was the primary outcome of the program, and frequently required substantial time investments. Administration of booster vaccines was awarded fewer points than the initial series as individuals who were eligible for a booster vaccine rarely demonstrated vaccine hesitancy. One point was awarded for offering counseling to a patient who subsequently declined vaccination, and 2 points if the trainee was able to determine a vaccine hesitancy subtype for the patient. The latter was to encourage the use of a communication framework provided to trainees that offered curated engagement strategies for patients exhibiting several common COVID-19 vaccine hesitancy subtypes.³²

Figure 2.

Driver diagram for primary aim of increasing rate of COVID-19 vaccine administration in the inpatient setting.

As part of Vax the Max, trainees were asked to self-tally their COVID-19 vaccine tasks engagement scores on scorecards taped to their team room door. In addition to tracking progress, these scorecards served as visual cues. Scorecards were collected weekly and the team with the highest score was given a trophy for their team room. The trophy rotated location depending on the weekly Vax the Max team winner. Weekly winners were also announced at educational conferences and displayed on the chief resident office door. At the end of each 4-week block, the cumulative 4-week scores for all the teams were tabulated. The team with the highest cumulative score received a congratulatory certificate signed by the hospital and IMR program leadership. In the event of a tie, the team with the most COVID-19 vaccines administered during the block was declared the winner.

The first PDSA cycle involved the implementation of the Vax the Max gamification of COVID-19 vaccine tasks. The second PDSA cycle included the implementation of a COVID-19 vaccine hesitancy workshop. This workshop was created after many IMR reported patients who had COVID-19 vaccine hesitancy on the Vax the Max scorecards during the first PDSA cycle. This interactive workshop provided learners with a framework for categorizing vaccine-hesitant patients according to thematic subtypes that were generated from an evidence-based national survey.³² The workshop also incorporated an interactive role-playing exercise where learners practiced tailored counseling to each hesitancy subtype. External factors that influenced Vax the Max results during the second PDSA cycle included the US Centers for Disease Control and Prevention (CDC) expanding their recommendation for COVID-19 vaccine booster doses to the general population. Additional PDSA interventions can be found in Appendix 2, Supplemental Digital Content 2, available at http://links.lww.com/AJMQ/A86.

During PDSA cycles 1–3, all 3 COVID-19 vaccines approved for use in the United States were available at the study medical center (Pfizer, Moderna, and Johnson & Johnson).^33–35 The same number of points was awarded for each primary vaccine series dose regardless of manufacturer. Data on specific vaccine types administered were not collected. During the fifth PDSA cycle, only the Pfizer vaccine was available at the medical center.

Measures

Primary and secondary outcome measures, along with balance measures, were analyzed. The primary outcome measure was the rate of COVID-19 vaccination administration to patients admitted to the medicine acute care service at the hospital. Other outcome measures included the number of patients confirmed as fully vaccinated, and provision of COVID-19 vaccine counseling to those who were vaccine-hesitant. Wasted COVID-19 vaccine doses were monitored as a balance measure. Participating trainees were further surveyed regarding their perspective of the intervention and any perceived impact it had on their behavior.

Analysis

Seven months of data were analyzed during the implementation of Vax the Max. COVID-19 vaccination administration data was extracted from the EMR for patients who were located on wards that care for acute care medicine patients during the study period. Data was also tabulated from weekly scoresheets on which IMRs tallied COVID-19 task engagement. Scoresheets included data confirming previous vaccination status, scheduling COVID-19 vaccination after discharge, and patients who were counseled for COVID-19 vaccination but declined. Trainee participants were emailed an anonymous survey after participation in Vax the Max to assess their engagement. The total number of trainee participants involved in Vax the Max was obtained from an administrative log.

Ethical Considerations

This quality improvement project was reviewed jointly by the relevant authorities at the institution and determined to not constitute human subject research.

Results

Seven groups of trainees participated in Vax the Max over a seven-month period. A baseline average of 5.3 initial vaccines per 4-week block was observed in the summer of 2021, 3 months before implementation of Vax the Max. After implementation of Vax the Max, the rate of initial vaccine administration increased to an average of 8.8 per 4-week block. A control chart (Figure 3) shows the number of initial series COVID-19 vaccines administered when compared with the preintervention period. This control chart demonstrates how the process of COVID-19 vaccine administration reached a new mean after implementation of Vax the Max. An average of 19.7 booster COVID-19 vaccines was administered per PDSA cycle after implementation of Vax the Max. Booster vaccines were approved by the CDC after the start of Vax the Max so no baseline data could be obtained for comparison. An average of 1.5 patients per PDSA cycle were scheduled for vaccination after hospital discharge after implementation of Vax the Max. Residents tallied other COVID-19 task engagement including confirmation of previous vaccination status and offering counseling for vaccine-hesitant patients (Figure 4). The run chart displays the number of completed COVID-19 tasks that occurred in each PDSA cycle and allowed the study team to observe possible impacts from the interventions. Baseline data were not available for these forms of COVID-19 task engagement.

Figure 3.

Vax the Max control chart of COVID-19 vaccine administration by PDSA cycles. Abbreviation: PDSA, plan-do-study-act.

Figure 4.

Vax the Max run chart of COVID-19 vaccine task completion including discarded vaccines (blue), confirming previous COVID-19 vaccination (black), booster vaccines administered (gray), and COVID-19 vaccine counseling offered without ultimate acceptance of vaccination (purple) by PDSA cycles. Abbreviation: PDSA, plan-do-study-act.

An average of 67 wasted vaccine doses was observed in the 4, four-week blocks that preceded implementation of Vax the Max (Figure 4). The average number of wasted vaccines increased to 144 per 4-week block after implementation of Vax the Max. These vaccine counts include all three approved COVID-19 vaccines. The team was unable to determine the setting of care (ie, inpatient or outpatient) of the wasted vaccines within the medical center as this data was not collected by the pharmacy department.

The anonymous postparticipation survey was emailed to 123 participants and 45 responded (36.6%). The training level of respondents was as follows: medical students (N = 16, 36%), PGY1 (N = 15, 33%), and PGY2/PGY3 (N = 14, 31%). All respondents (N = 45, 100%) were aware of Vax the Max during their clinical rotation at the hospital. The majority of respondents reported that Vax the Max gamification spurred increased COVID-19 vaccine engagement (N = 37, 83.7%), and that a desire to win the competition motivated them (N = 27, 65.1%). The majority of respondents also thought that a gamification model could be utilized on a clinical task other than COVID-19 vaccines (N = 28, 67%). Other survey results can be seen in Table 1.

Table 1.

Trainee Responses to Postparticipation Survey.

Survey prompts	Agree, N (%)
I was aware of the Vax the Max competition during my rotation.	45 (100%)
I administered more COVID-19 vaccines because of Vax the Max.	34 (77.3%)
I counseled more patients on the importance of COVID-19 vaccination as a result of Vax the Max.	36 (81.4%)
I think the gamification of COVID-19 tasks has been helpful in increasing the engagement of COVID-19 tasks.	37 (83.7%)
Screening patients for COVID-19 vaccination status was a significant clinical burden.	2 (4.7%)
I was motivated to vaccinate patients for COVID-19 because I believe this is my responsibility as part of the COVID-19 pandemic response.	44 (100%)
I was motivated to vaccinate patients for COVID-19 because I wanted to win Vax the Max.	27 (65.1%)
I think the gamification model could be used for a different clinical task.	28 (67%)

Discussion

When confronted by flagging COVID-19 vaccination rates among inpatients at the study institution, combined with an already-stretched nursing workforce unable to accommodate additional assignments, a gamification model was successful at recruiting IMRs to lead inpatient vaccine efforts. This intervention successfully increased the COVID-19 vaccination rate to acute care medicine inpatients by gamifying COVID-19 vaccination tasks within a quality improvement framework.

The observed increase in initial vaccine series administration rate to admitted patients is especially notable, given the number of patients eligible for the initial vaccine series dropped from the time Vax the Max was started to when this new baseline was established. Specifically, 41% of Washington State residents were eligible to start or complete the initial COVID-19 vaccine series when Vax the Max was launched, as compared with 29% at the end of the study period.³⁶ While other inpatient COVID-19 vaccination programs exhibited declining vaccination rates over time,^23,24 involved residents were able to counter this trend and increase COVID-19 vaccine uptake.

The utilization of trainees as the central agents of change for increasing inpatient rates of COVID-19 vaccination was also novel. Most participants reported that the gamification process encouraged positive behavioral changes around inpatient COVID-19 vaccine engagement. This is a striking outcome given the barriers integrating residents into quality improvement efforts that have been observed previously,^16–18 including monthly resident turnover making practice sustainment more challenging. Furthermore, these results were obtained without utilizing financial incentives for residents which have been employed previously to engage trainee involvement in quality improvement endeavors.³⁷ The use of gamification with trainees may have had a synergistic relationship. The gamification model produced a high degree of engagement by itinerant residents despite limited training and also had a subjective positive impact on the morale of trainees. This was noteworthy given that Vax the Max added tasks to residents at a time when health care workers were experiencing elevated fatigue and burnout related to the COVID-19 pandemic. Despite these promising results, the authors recognize the importance of context, and that COVID-19 may have been a potent motivator with which gamification paired especially well.

During most PDSA cycles, it was noted that the winning teams were propelled by several highly motivated individuals, supporting an additive effect from intrinsic desire. Interestingly these positive outliers were frequently the junior members of the teams: medical students and interns. In 1 case, a third-year medical student spearheaded the efforts for a single team to vaccinate 18 individuals during one PDSA cycle.³⁸ The leadership role which medical students assumed in Vax the Max suggests there is an under-tapped potential in involving junior learners in bedside quality improvement activities.

Residents noted formidable patient and family COVID-19 vaccine hesitancy. This was the most significant barrier to vaccine administration, followed by clinical task saturation. Recognizing this, a COVID-19 vaccine hesitancy workshop was embedded into Vax the Max.³⁹ In this workshop, residents were introduced to a COVID-19 vaccine hesitancy framework that subtyped common motivations for COVID-19 vaccine hesitancy. After this, learners were offered strategies for productive engagement on COVID-19 vaccine hesitancy based on each subtype. Currently, limited efforts have been made regarding the introduction of effective and impactful training for IMRs to counter COVID-19 vaccine hesitancy. Further work is needed to empower IMRs to counsel vaccine-hesitant patients.

This program had local limitations that may complicate its replication in other clinical contexts. Notably, Vax the Max did not lead to higher vaccination rates across all participating teams. During most PDSA cycles, 1 or 2 teams had diminished participation. This may be due to a variety of factors including higher patient census and/or acuity, reduced receptivity to gamification, residual lack of awareness about the project or its details, or higher overall burnout. Local environmental and workforce factors may dampen or amplify the effect of a gamification program when transferred to another setting. Variable engagement between PDSA cycles was noted, evidenced by the declining trend in COVID-19 vaccine administration in the final two PDSA cycles. It is unclear if this represented similar variability as was observed earlier in the study, or if this demonstrated a trend back to preintervention mean. Quantifying the fidelity of resident participation in the various elements of Vax the Max is an area the team hopes to investigate in the future. Nevertheless, in their experience, a gamification program paired with focused skills training around vaccine hesitancy yielded improved performance across the system as a whole, even if the influence on individuals was heterogeneous.

As the COVID-19 pandemic matures, the need for programs like Vax the Max will modulate. Despite this, there are lessons to be drawn from this experience that may be applied to other clinical scenarios. By stimulating participant interest through gamification, Vax the Max increased completion of an important clinical task in a novel clinical setting and among unaccustomed clinical staff who also turned over frequently. This program was also developed and implemented quickly, at low cost, and was able to adapt to internal and external factors. These features make similar programs most effective for problems that are new, time-sensitive, and high priority. The empowerment of trainees in quality improvement is an underutilized resource, and there is potential to explore gamification models for trainee involvement in other clinical scenarios.

Acknowledgments

AVR is the Chief Resident for Quality and Safety at the Veterans Administration Puget Sound, Seattle Division Hospital. His salary is furnished by the Veterans Health Administration.

Conflicts of interest

The authors have no conflicts of interest to disclose.

Authors Contributions

The initial development for Vax the Max was developed by AVR, PC, and CW. Ongoing development and stewardship of Vax the Max was completed by AVR, KA, KB, CW, PC, KS, and RR. Daily tasks for Vax the Max implementation were carried out by AVR, KA, and KB. Organization leadership for Vax the Max contributed by PC. KS, and RR. The initial manuscript draft was written by AVR. AVR, KA, KB, PC, CU, RR, and KS all critically edited the manuscript. All authors had access to the included data.

Supplementary Material

jmq-38-47-s001.docx ^{(12.7KB, docx)}

jmq-38-47-s002.docx ^{(14.1KB, docx)}