Abstract
Background
To avoid misuse of personal protective equipment (PPE), ensure health care workers’ safety, and avoid shortages, effective communication of up-to-date infection control guidelines is essential. As prehospital teams are particularly at risk of contamination given their challenging work environment, a specific gamified electronic learning (e-learning) module targeting this audience might provide significant advantages as it requires neither the presence of learners nor the repetitive use of equipment for demonstration.
Objective
The aim of this study was to evaluate whether a gamified e-learning module could improve the rate of adequate PPE choice by prehospital personnel in the context of the coronavirus disease (COVID-19) pandemic.
Methods
This was an individual-level randomized, controlled, quadruple-blind (investigators, participants, outcome assessors, and data analysts) closed web-based trial. All emergency prehospital personnel working in Geneva, Switzerland, were eligible for inclusion, and were invited to participate by email in April 2020. Participants were informed that the study aim was to assess their knowledge regarding PPE, and that they would be presented with both the guidelines and the e-learning module, though they were unaware that there were two different study paths. All participants first answered a preintervention quiz designed to establish their profile and baseline knowledge. The control group then accessed the guidelines before answering a second set of questions, and were then granted access to the e-learning module. The e-learning group was shown the e-learning module right after the guidelines and before answering the second set of questions.
Results
Of the 291 randomized participants, 176 (60.5%) completed the trial. There was no significant difference in baseline knowledge between groups. Though the baseline proportion of adequate PPE choice was high (75%, IQR 50%-75%), participants’ description of the donning sequence was in most cases incorrect. After either intervention, adequate choice of PPE increased significantly in both groups (P<.001). Though the median of the difference in the proportion of correct answers was slightly higher in the e-learning group (17%, IQR 8%-33% versus 8%, IQR 8%-33%), the difference was not statistically significant (P=.27). Confidence in the ability to use PPE was maintained in the e-learning group (P=.27) but significantly decreased in the control group (P=.04).
Conclusions
Among prehospital personnel with an already relatively high knowledge of and experience with PPE use, both web-based study paths increased the rate of adequate choice of PPE. There was no major added value of the gamified e-learning module apart from preserving participants' confidence in their ability to correctly use PPE.
Keywords: personal protective equipment, COVID-19, electronic learning, prehospital, randomized controlled trial, protection, equipment, safety, gamified, online learning, communication
Introduction
Background and Importance
Adequate use of personal protective equipment (PPE) is of paramount importance to ensure health care workers’ safety and to avoid shortages of such equipment in the context of the coronavirus disease (COVID-19) pandemic [1,2]. Protection guidelines against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection have rapidly evolved following publication of new evidence regarding its mode of transmission, making prompt adaptation of the guidelines both frequent and necessary [3-5]. Prehospital personnel is particularly at risk of contamination as they usually work in a challenging environment and have to stay next to their patients in the narrow space of the ambulance for extended periods of time. This risk is further increased when high-risk procedures such as endotracheal intubation have to be performed [6-8].
To avoid misuse of PPE, effective communication of the corresponding guidelines to frontline health care workers is necessary [9]. However, continuous education has been massively disrupted due to the cancellation of continuous education sessions to limit disease transmission [10]. In this challenging context, electronic learning (e-learning) might provide significant advantages as it requires neither the presence of learners nor the repetitive use of equipment for demonstration as could be the case for live simulation [11,12].
Goal of This Investigation
The purpose of this study was to evaluate whether a specifically designed gamified e-learning module [13] could improve the rate of adequate PPE choice by prehospital personnel in the context of the COVID-19 pandemic. Our hypothesis was that knowledge of PPE guidelines would be inconsistent between prehospital personnel, and that an e-learning module may increase and standardize knowledge regarding the use of PPE. This could help limit both underuse and overuse of such equipment.
Methods
Study Design
This was an individual-level, stratified, randomized, controlled, quadruple-blind (investigators, participants, outcome assessors, and data analysts) closed web-based trial (Figure 1) designed following the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth) guidelines [14,15] and incorporating relevant elements from the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) statement [16].
The regional ethics committee issued a “Declaration of no objection” (Req-2020-00374) as the population studied was not considered vulnerable according to the Swiss federal law on human research [17]. As the purpose of the study was to examine the effect of two different study paths only on providers’ knowledge and attitude toward PPE, registration of the trial was not performed as it is not deemed necessary by the International Committee of Medical Journal Editors [18].
Setting
The study took place in April 2020 in Geneva, Switzerland; the detailed organization of local prehospital emergency services has already been described elsewhere [19]. There are five levels of increasing expertise working in the prehospital field. Ambulance drivers all have a basic life support certificate but are never called upon to deal with emergency situations. Emergency medical technicians (EMTs) are certified after 1 year of training, and can either transport stable patients on their own, or team up with a paramedic to deal with more difficult prehospital situations. The highest level of nonphysician care is provided by paramedics, who complete a 3-year curriculum. Whenever a potentially life-threatening emergency is identified by emergency dispatchers, paramedics are sent in teams of two. A medical reinforcement by way of a light vehicle, Service Mobile d’Urgence et de Réanimation (SMUR), staffed by a paramedic and an emergency physician can either be dispatched at the same time as the ambulance or be called upon by paramedics if specialized medical care is required on site. Emergency physicians are either senior residents or fellows working in one of the following departments: emergency medicine, anesthesiology, or general internal medicine. They can be supervised, either by call or on-site, by a senior specialist emergency physician, which represents the highest level of prehospital care in this setting.
Apart from the physician-staffed prehospital medical service, there are seven different ambulance companies in Geneva, two of which belong to public organizations while the remaining five are privately owned. As each company has its own continuous education structure as well as its own equipment, medical devices, and protocols, one cannot expect all paramedics to share the same knowledge level regarding all aspects of prehospital emergencies.
Online Platform
An online platform [20] developed under the Joomla! 3.9 content management system (Open Source Matters) was created specifically for the purpose of this study. A mailing management component (AcyMailing 5.10, Acyba), a survey component (Community Surveys Pro 5.4.0, CoreJoomla), and a form builder component used to issue completion certificates (BreezingForms Pro 1.9.0, Crosstec) were installed on the website. L Suppan was the only author with access to the platform’s administration console. No maintenance or update was planned on the server, platform, or content during the study period.
Study Material
A previously described gamified e-learning module created under Storyline 3 (Articulate Global) was used in this study [13]. The module contains 19 sections and embeds 7 video sequences. Within the module, trigger mechanisms are used to check that the user had accessed and completed all required steps before being allowed to proceed to the following section. The e-learning package is available on the study website where it can be viewed and downloaded freely. The comprehensive prehospital COVID-19 guidelines from the Geneva University Hospitals, version 1.11, was also used in this study [21].
Two quizzes were created by BG and L Suppan: a preintervention quiz designed to establish the participants’ baseline knowledge regarding PPE, their use and indication, and a postintervention quiz to assess whether these parameters had changed. Both quizzes contained 10 closed questions, either multiple choice or multiple answer. Questions designed to assess PPE choice were preceded by short clinical scenarios. Each quiz was displayed over 5 pages. The number of questions was limited to reduce attrition. These quizzes were tested and validated by all coinvestigators.
Consistency of specific “free-text” questions, such as age, was checked by means of regular expression (“regex”) rules. All answers were automatically checked for completeness by the system before participants were allowed to proceed to the next page. Custom validation messages were displayed to inform users who had not answered a question. Participants were not allowed to correct or review their answers once a page had been completed.
Subjects and Inclusion and Exclusion Criteria
Chief ambulance officers of all services were asked to provide one of the investigators (L Suppan) with a list of all the professional email addresses of their EMTs, paramedics, and emergency physicians. All the email addresses received from these officers were included.
Email addresses of ambulance drivers were excluded as these drivers usually only deal with interhospital transfers and almost never don PPE. Senior specialist emergency physicians were also excluded, as they are few in number and are usually involved in the writing of the guidelines or in the creation of the learning material; in addition, some of them are authors of this study. Finally, the email addresses of the paramedics who participated in the creation of this study or the learning material were also excluded.
Randomization and Allocation Concealment
Before performing a 1:1 randomization, stratification was achieved according to professional status (EMTs, paramedics, and emergency physicians). Email addresses were then sorted according to alphabetical order, and an investigator (MS) who did not have access to the email addresses database was given the number of participants by category and performed the randomization using a computer-generated table. The randomization key was then combined with the list of email addresses and entered in the mailing component by the only investigator who had access to the system (L Suppan). As the list of email addresses and allocations were solely present in an encrypted database, all other investigators were blinded as to group allocation.
Enrolment and Consent
Individual emails that were identical for all participants except for the unique links that pointed to one of the two study paths were sent on April 13, 2020 (Multimedia Appendices 1 and 2). These unique links were automatically created by the survey component but were not recorded in the mailing component other than in generic form (there was no way the individual tokens could be linked to the email addresses). The webmaster could therefore only know whether the email had been opened and the survey started, but was prevented from reconciling the email address with the survey answers, thereby guaranteeing user anonymization. Using unique links served two purposes: they allowed the participants to resume the course in case they were interrupted and also avoided double entries. Given the current circumstances, and as the vast majority of prehospital professionals had had their holidays cancelled and were therefore at work, participants were only given 12 days to complete the study, with email reminders sent on days 3, 7, and 10.
Apart from the survey link, the emails contained information regarding the study length and objectives as well as a short data protection statement. Participants were informed that they would be presented with the most recent version of the prehospital COVID-19 guidelines as well as with an e-learning module, though the order in which these materials would be shown was not explained. They were informed that, by clicking on the survey link, they consented to participate in the study and were provided with the names and electronic addresses of five investigators (BG, EG, L Stuby, L Suppan, and PC), whom they could contact at any time. As collected data were irreversibly anonymized, it was impossible for users to ask for their own answers to be deleted once the survey had been completed.
To improve the response rate, the chief ambulance and medical officers of all companies were asked to encourage their paramedics, EMTs, and emergency physicians to participate in the study. Participation reminders were also sent to all prehospital personnel along with a daily COVID-19 information newsletter.
Participation was not mandatory. No monetary incentive or prize was offered to the survey participants. As the e-learning module was akin to a continuous education session, participants were informed before beginning the survey that they would be able to print a continuous education certificate upon completion. As the certificate component was independent from both the mailing and the survey components, participants were ensured they could generate the certificate without their identity being disclosed.
Study Sequence
After clicking on the survey link, a welcome screen containing detailed information about the study was displayed. This welcome screen was identical for both groups and, similarly to the email messages, did not convey any information regarding the study sequence to ensure the participants were adequately blinded.
After clicking on the start button, participants were asked a series of questions designed to gather demographic-related data. Adaptive questioning was used in this section to avoid displaying irrelevant questions. Participants were then asked a series of general questions related to SARS-CoV-2 and the COVID-19 pandemic.
The control group was then shown the prehospital COVID-19 guidelines. They were then asked to answer a second set of questions before being prompted to evaluate the learning path up to this point. Only then could they access the e-learning module and download their certificate.
The e-learning group followed the same path at first, but accessed the e-learning module immediately after being shown the guidelines. This group then completed the same second set of questions, was asked to evaluate the learning path (which in this case included the e-learning module), and was finally allowed to download the completion certificate.
Outcomes
The main outcome was defined as the difference in the proportion of correct choice of PPE before and after the course, assessed by means of short clinical scenarios.
Secondary outcomes were stratification of the main outcome by profession and by personal history of COVID-19 (whether or not the participant had been infected), accuracy of donning and doffing sequences reconstitutions, differences in the rates of overuse and underuse of PPE, confidence in one’s ability to use PPE, perceived usefulness of the course, and satisfaction regarding the course. The latter three outcomes were assessed by means of a 5-point Likert scale.
Data Collection
Data was electronically recorded and securely stored in an encrypted MariaDB database (Version 5.5.5; MariaDB Foundation). At the end of the study, all data was extracted to CSV format by the only investigator who had access (L Suppan). No personally identifiable data (including name, date of birth, email, or IP address) was ever asked for or recorded.
Data Curation and Availability
The extracted data were imported into Stata (StataCorp LLC). Variables were renamed to facilitate their understanding by the blinded data analysts. All data that could have enabled data analysts to identify group allocation were removed, and data fields were sorted accordingly. Incomplete questionnaires were excluded at this stage. The control and e-learning groups were renamed using city names (Moscow and Nairobi), and all relevant data were exported by L Suppan to a Stata .dta file and sent to L Stuby and MA for analysis.
Sample Size
As guidelines differ from region to region, we decided to only include prehospital staff working in the Geneva emergency medical system, thereby using a convenience sample rather than performing a sample size calculation.
Outcomes Assessment
Though most outcomes were electronically recorded and their interpretation therefore was generally independent of subjective human evaluation, comments had to be assessed by outcome assessors. Two assessors (L Stuby and PC), blinded as to group allocation, were asked to independently assess all comments. The nature of comments were to be rated as “positive,” “negative,” or “neutral” regarding the study, and as to whether they challenged Infection Prevention and Control (IPC) guidelines (binary, yes versus no). Disagreements were solved by sending the unclear comments to a third outcome assessor (BG), who was blinded to the previous assessments.
Statistical Analysis
Statistical analysis was performed using Stata 15. Continuous independent outcomes were assessed using the Student t test or the Mann-Whitney rank-sum test depending on normality. Categorical outcomes were assessed using Fisher exact test. A two-sided P<.05 was considered significant. Normality of distribution was first assessed graphically. In case of doubt, the Shapiro-Wilk test was applied.
Continuous paired data were assessed using either the paired Student t test or the Wilcoxon matched-pairs signed-rank test depending on normality. The sign test for matched pair was used if symmetry could not be proven. Categorical paired data were analyzed using asymptotic symmetry and marginal homogeneity tests. Binomial paired data were assessed using the McNemar test.
Stratification was defined a priori based on expertise level (EMTs, paramedics, or physicians) and COVID-19 status (negative; confirmed, quarantined; confirmed, cured or unknown). Two post hoc sensitivity analyses were conducted.
Data Availability
The original data has been deposited to Mendeley Data [22].
Results
Characteristics of Study Subjects
Of the 291 randomized participants, 176 (60.5%) completed the trial (Figure 2). No change was made to the web platform after the first batch of emails was sent. There was no significant downtime (the server was available more than 99% of the time throughout the study period). Just three participants were unable to complete the trial owing to technical problems, the exact nature of which could not be determined: two were not able to access the e-learning module, and one could not access the guideline (in PDF format). The study path was completed in one session by 82.7% of the participants (143/173).
The blinded data analysts (L Stuby and MA) excluded two surveys from the e-learning group as the participants’ professions did not match the target population (one “ambulance driver” and one “other”). They also excluded one survey from the control group because of incoherent answers (Multimedia Appendix 3). Participant characteristics are described in Table 1.
Table 1.
Characteristics | Control (n=85) | E-learning (n=88) | |
Profession, n (%) | |||
|
Student paramedic | 5 (5.9) | 10 (11.4) |
|
Emergency medical technician | 11 (12.9) | 12 (13.6) |
|
Paramedic | 61 (71.8) | 60 (68.2) |
|
Physician | 8 (9.4) | 6 (6.8) |
Gender, female, n (%) | 32 (37.6) | 28 (31.8) | |
Age (years), median (Q1-Q3) | 35 (30-42) | 34 (28-40) | |
Professional experience (years), median (Q1-Q3) | 9 (3-15) | 7 (2-12) | |
Prior infection prevention and control course, n (%) | |||
|
No/does not remember | 73 (85.1) | 70 (79.6) |
|
Yes | 12 (14.1) | 18 (20.4) |
Coronavirus disease status, positive, n (%) | 7 (8.3) | 6 (6.8) | |
Local guideline seen, yes, n (%) | 79 (92.9) | 84 (95.4) | |
Last time guideline seen (days), median (Q1-Q3) | 5 (3-10) | 5 (2-8) | |
Specific coronavirus disease course followed, yes, n (%) | 28 (32.9) | 32 (36.4) |
aTotals may not equal to 100% due to rounding.
Main Results
There was no significant difference in baseline knowledge between groups. Though the baseline proportion of adequate PPE choice was high (75%, IQR 50%-75%), description of the donning sequence (assessed preintervention) was in most cases incorrect, as only 7 (4%) of the participants were able to reconstitute it accurately, with a similar proportion between groups. The donning sequence initially displayed in the survey was left unchanged by 7% of the participants (12/173, 6 per group).
Adequate choice of PPE was significantly increased in both groups after the intervention (P<.001; Figure 3). Though the median of the difference was higher in the e-learning group (17%, IQR 8%-33% versus 8%, IQR 8%-33%), it did not reach statistical significance (P=.27). This difference was similar regardless of stratification by profession or history of COVID-19 (Table 2). No participant was able to describe the correct doffing sequence, which was assessed postintervention. The doffing sequence originally displayed in the survey was left unchanged by 8% of the participants (14/173, 7 per group).
Table 2.
Outcomes | Control (n=85) | E-learning (n=88) | P value | |
Main outcome: difference in percentage of correct answers, median (Q1-Q3) | 8 (8 to 33) | 17 (8 to 33) | .27 | |
Main outcome by profession (%), median (Q1-Q3) | ||||
|
Paramedic | 8 (0 to 25) | 17 (8 to 33) | .15 |
|
Paramedic student | 25 (8 to 33) | 29 (25 to 33) | .49 |
|
Emergency medical technician | 25 (8 to 33) | 8 (–8 to 33) | .88 |
|
Physician | 25 (13 to 38) | 13 (–8 to 42) | .30 |
Main outcome by coronavirus disease status (%), median (Q1-Q3) | ||||
|
Negative for coronavirus disease | 8 (8 to 33) | 25 (8 to 33) | .20 |
|
Positive for coronavirus disease | 8 (–8 to 33) | 0 (–8 to 17) | .37 |
Confidence in the ability of using PPE was identical before and after the course in the e-learning group (P=.27), but was significantly lower after the course in the control group (P=.04).
While most participants found the course useful (68.5%, 95% CI 61.5%-75.3%), the proportion of participants finding the course “very useful” was significantly higher (P=.01) in the e-learning group (33.0%, 95% CI 23.2%-42.8% versus 11.8%, 95% CI 4.9%-18.7%). Participants were generally satisfied regarding the course (60.0%, 95% CI 49.6%-70.4% for the e-learning group versus 62.5%, 95% CI 52.4%-72.6%; P=.28; Figure 4).
The proportion of positive comments was similar between groups (14%, 95% CI 7%-21% in the e-learning group versus 19%, 95% CI 11%-27%; P=.16). Participants who expressed disagreements with IPC recommendations were also evenly divided (5.7%, 95% CI 3.2%-8.2% in the e-learning group versus 4.7%, 95% CI 0.2%-9.2%; P>0.99).
Two post hoc sensitivity analyses were performed. The first was achieved by excluding all participants who answered they were either “very unsatisfied” or “unsatisfied” with the course. The second was performed by excluding participants whose compliance regarding COVID-19–specific IPC guidelines could be doubted; this included those who did not answer “systematic wearing of a double pair of gloves” to the question “Which of these measures is NOT one of the infection prevention measures?” (40 [45%] in the e-learning group versus 35 [41%]). These analyses did not demonstrate a significant change regarding the main outcome (P=.89 and P=.29, respectively).
Discussion
Main Results
In this study, the proportion of those making an adequate choice regarding PPE increased significantly after prehospital personnel followed either web-based study path. Though the median of the increase was twice that of the e-learning group, this difference was not statistically significant.
Failure to reach significance might be explained by many different factors. The high baseline proportion of adequate choice of PPE in both groups may have dampened any relative impact of the intervention as there was little room left for improvement. Although this result might seem counterintuitive given the high rate of participants who responded that they did not attend (or did not recall attending) an IPC course prior to this study, more than 90% declared they had consulted the local guidelines recently. Most participants were therefore aware of the local recommendations, and the anxiety generated by the COVID-19 pandemic probably acted as a catalyst regarding their interest in such guidelines [23].
Another factor that might partly explain the lack of a significant difference is the sample size, which limited the power to detect small differences. Nevertheless, as PPE guidelines vary not only between countries, but also between Swiss cantons, it might have been inadequate to draw conclusions from a pooled group of paramedics working in different cantons with different guidelines [24]. Moreover, though increasing the sample size would increase the chances of finding a significant difference, too small an increase would not be clinically relevant regardless of the P value obtained [25]. Lack of significance might therefore also be consecutive to lack of effect of the e-learning module regarding knowledge acquisition in this particular population.
Though necessary to assess participants’ knowledge and attitudes regarding PPE prior to the interventions, the first set of questions, along with the study title, might have acted as a primer and focused the participants’ attention on the specific contents that would be tested postintervention [26]. This effect might have further dampened the potential impact of the e-learning module.
The relatively low level of satisfaction displayed by the participants should also be taken into account. Though e-learning modules and serious games usually increase participant satisfaction when compared to more traditional methods [27,28], such an effect could not be found in this study. Disagreement with IPC guidelines might, at least in part, explain why some participants were dissatisfied [5,29,30]. Indeed, the vast majority of paramedics working in Geneva have had some prior training regarding the use of PPE in situations other than the COVID-19 pandemic. Preparations for high-risk transfers during the 2014 Ebola epidemic [31] and practical exercises in the context of simulated major incidents involving the presence of hazardous materials [32] have presented paramedics with many different PPE guidelines and protocols over the last few years. As the latter situations require more stringent donning and doffing procedures than those described in the COVID-19 context, some paramedics seem to feel that IPC recommendations do not go far enough. There is however a delicate balance between underprotection and overuse of scarcely available resources that must be maintained [33,34]. Moreover, it can hardly be expected to have all paramedics don maximal PPE for all interventions. This would not only considerably increase intervention time, but also make delicate interventions more difficult given the bulk of the PPE and lead to more difficult communications. It could also increase fatigue as such equipment has been shown to be quite uncomfortable [35].
Secondary Outcomes
Though confidence in the ability to use PPE was maintained in the e-learning group, it significantly decreased after reviewing the guidelines in the control group. Participants who initially felt confident in their knowledge might have felt it was challenged after being asked specific questions [36,37]. The e-learning module probably helped restore their altered confidence, as interactive presentations, as well as gamification, have been shown to increase this feeling [38,39]. Nevertheless, most participants were unable, regardless of their assigned intervention, to reconstruct either the donning (assessed preintervention) or the doffing (assessed postintervention) sequences. This potential limitation has already been highlighted in the paper describing the development of the gamified e-learning module used in this study [13], though cut scenes were embedded to provide direct demonstration [40,41].
Though incorrect answers regarding the donning sequence are easily understandable as they might primarily result from a lack of knowledge, an inability to correctly rebuild the doffing sequence is questioning. No less than four different broad categories of causes could be involved: ineffectiveness of the teaching material, inadequate sequence, flawed means of assessing the participants’ knowledge, or disagreement with the procedure outlined in the guidelines and in the e-learning module. As this result was unforeseen, the method used in this study was unfortunately ill-suited to the evaluation of the underlying causes. Nevertheless, with less than 10% of participants having left the initially displayed sequences unchanged, a technical flaw can reasonably be ruled out.
Limitations
Apart from the abovementioned limitations, the ever-increasing knowledge regarding SARS-CoV-2 and COVID-19 might render both the guideline and the gamified e-learning module used in this study obsolete. However, current technological tools might mitigate this effect as they allow for a quick adaptation, even of highly interactive content [42].
Another limitation is the relatively small number of questions asked pre- and postintervention. Keeping the total number of questions and the time required to complete either study path relatively low was necessary to limit attrition [16,43]. This strategy was altogether successful as dropouts amounted to less than 15% in either group, a rather lower proportion than generally reported [44]. Attrition was further limited thanks to the use of unique identifiers, which allowed as many as 30 participants (17.3%) to complete their assigned study path in more than one session.
Despite its limitations, this study also has some strengths, among which the quadruple-blind design and the relatively high response rate should be acknowledged. Moreover, as all answers were electronically recorded, there was no risk of an outcome assessment bias. Finally, as neither the control nor the e-learning path requires the physical presence of either participants or instructors, the framework used in this study could serve as the building ground for courses in the context of an epidemic or a pandemic such as the current COVID-19 situation.
Perspectives
The larger impact such a web-based study might have had, regardless of the effect of a specific intervention such as the gamified e-learning module, should be assessed, as a possible change in PPE consumption or infection rate among prehospital providers could ensue.
The potential impact of this gamified e-learning module on less experienced and less primed participants should also be evaluated to confirm (or refute) the theories outlined in this discussion. The gamified e-learning module can thus be freely downloaded from the study website in both a web (HTML5 with Flash fallback) and SCORM (Shareable Content Object Reference Model) format.
Conclusions
Among prehospital personnel with an already relatively high knowledge and experience regarding PPE use, both web-based study paths increased the rate of adequate choice of PPE. There was no major added value of the gamified e-learning module apart from preserving participants' confidence in their ability to correctly use PPE.
Acknowledgments
The authors would like to thank Mr Michel Hofer for his support.
Abbreviations
- CHERRIES
Checklist for Reporting Results of Internet E-Surveys
- CONSORT-EHEALTH
Consolidated Standards of Reporting Trials of Electronic and Mobile HEalth Applications and onLine TeleHealth
- COVID-19
coronavirus disease
- E-learning
electronic learning
- EMT
emergency medical technician
- IPC
infection prevention and control
- PPE
personal protective equipment
- SARS-CoV-2
severe acute respiratory syndrome coronavirus 2
- SCORM
Shareable Content Object Reference Model
- SMUR
Service Mobile d’Urgences et de Réanimation
Appendix
First email sent to participants belonging to the control group.
First email sent to participants belonging to the e-learning group.
Details of the answer set excluded by the blinded data analysts.
Footnotes
Conflicts of Interest: Most of the authors participated in the development of the gamified e-learning module tested in this trial. Nevertheless, as this module is freely available, the authors deny any financial conflict of interest.
Editorial Notice: This randomized study was not prospectively registered. As the purpose of the study was to examine the effect of two different study paths only on providers’ knowledge and attitude toward PPE, registration of the trial was not performed as it is not deemed necessary by the International Committee of Medical Journal Editors. The editor is therefore following the guidelines suggested by the ICMJE and not mandating prospective registration of this randomized trial. However, readers are advised to carefully assess the validity of any potential explicit or implicit claims related to primary outcomes or effectiveness, as retrospective registration does not prevent authors from changing their outcome measures retrospectively.
References
- 1.Livingston E, Desai A, Berkwits M. Sourcing Personal Protective Equipment During the COVID-19 Pandemic. JAMA. 2020 Mar 28;:E1–E3. doi: 10.1001/jama.2020.5317. [DOI] [PubMed] [Google Scholar]
- 2.Ranney ML, Griffeth V, Jha AK. Critical Supply Shortages - The Need for Ventilators and Personal Protective Equipment during the Covid-19 Pandemic. N Engl J Med. 2020 Apr 30;382(18):e41. doi: 10.1056/NEJMp2006141. [DOI] [PubMed] [Google Scholar]
- 3.Wax RS, Christian MD. Practical recommendations for critical care and anesthesiology teams caring for novel coronavirus (2019-nCoV) patients. Can J Anaesth. 2020 May;67(5):568–576. doi: 10.1007/s12630-020-01591-x. http://europepmc.org/abstract/MED/32052373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Guan L, Zhou L, Zhang J, Peng W, Chen R. More awareness is needed for severe acute respiratory syndrome coronavirus 2019 transmission through exhaled air during non-invasive respiratory support: experience from China. Eur Respir J. 2020 Mar 20;55(3):2000352. doi: 10.1183/13993003.00352-2020. http://erj.ersjournals.com:4040/cgi/pmidlookup?view=long&pmid=32198275. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rhee C, Baker MA, Klompas M. The COVID-19 infection control arms race. Infect Control Hosp Epidemiol. 2020 May 08;:1–3. doi: 10.1017/ice.2020.211. http://europepmc.org/abstract/MED/32381140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Teter J, Millin MG, Bissell R. Hand hygiene in emergency medical services. Prehosp Emerg Care. 2015 Nov 21;19(2):313–9. doi: 10.3109/10903127.2014.967427. [DOI] [PubMed] [Google Scholar]
- 7.Valdez MK, Sexton JD, Lutz EA, Reynolds KA. Spread of infectious microbes during emergency medical response. Am J Infect Control. 2015 Jun;43(6):606–11. doi: 10.1016/j.ajic.2015.02.025. http://europepmc.org/abstract/MED/26042849. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Lockhart SL, Duggan LV, Wax RS, Saad S, Grocott HP. Personal protective equipment (PPE) for both anesthesiologists and other airway managers: principles and practice during the COVID-19 pandemic. Can J Anaesth. 2020 Apr 23;:1005–1015. doi: 10.1007/s12630-020-01673-w. http://europepmc.org/abstract/MED/32329014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Oh HS, Uhm D. Occupational exposure to infection risk and use of personal protective equipment by emergency medical personnel in the Republic of Korea. Am J Infect Control. 2016 Jun 01;44(6):647–51. doi: 10.1016/j.ajic.2015.12.022. http://europepmc.org/abstract/MED/26874405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Rose S. Medical Student Education in the Time of COVID-19. JAMA. 2020 Mar 31;323(21):2131. doi: 10.1001/jama.2020.5227. [DOI] [PubMed] [Google Scholar]
- 11.Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis. 2020 Jul;20(7):777–778. doi: 10.1016/S1473-3099(20)30226-7. http://europepmc.org/abstract/MED/32213335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wittich C, Agrawal A, Cook D, Halvorsen A, Mandrekar J, Chaudhry S, Dupras D, Oxentenko A, Beckman T. E-learning in graduate medical education: survey of residency program directors. BMC Med Educ. 2017 Jul 11;17(1):114. doi: 10.1186/s12909-017-0953-9. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-017-0953-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Suppan M, Gartner B, Golay E, Stuby L, White M, Cottet P, Abbas M, Iten A, Harbarth S, Suppan L. Teaching Adequate Prehospital Use of Personal Protective Equipment During the COVID-19 Pandemic: Development of a Gamified e-Learning Module. JMIR Serious Games. 2020 Jun 12;8(2):e20173. doi: 10.2196/20173. https://games.jmir.org/2020/2/e20173/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Eysenbach G, CONSORT-EHEALTH Group CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. 2011 Dec 31;13(4):e126. doi: 10.2196/jmir.1923. https://www.jmir.org/2011/4/e126/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.de Kraker MEA, Abbas M, Huttner B, Harbarth S. Good epidemiological practice: a narrative review of appropriate scientific methods to evaluate the impact of antimicrobial stewardship interventions. Clin Microbiol Infect. 2017 Nov;23(11):819–825. doi: 10.1016/j.cmi.2017.05.019. https://linkinghub.elsevier.com/retrieve/pii/S1198-743X(17)30279-3. [DOI] [PubMed] [Google Scholar]
- 16.Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) J Med Internet Res. 2004 Sep 29;6(3):e34. doi: 10.2196/jmir.6.3.e34. http://www.jmir.org/2004/3/e34/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.CC 810.30 Federal Act of 30 September 2011 on Research involving Human Beings (Human Research Act, HRA) [2020-05-12]. https://www.admin.ch/opc/en/classified-compilation/20061313/index.html.
- 18.Clinical Trials Registration. International Committee of Medical Journal Editors. [2020-06-10]. http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/
- 19.Chan M, Fehlmann CA, Pasquier M, Suppan L, Savoldelli GL. Endotracheal Intubation Success Rate in an Urban, Supervised, Resident-Staffed Emergency Mobile System: An 11-Year Retrospective Cohort Study. J Clin Med. 2020 Jan 16;9(1):238. doi: 10.3390/jcm9010238. http://www.mdpi.com/resolver?pii=jcm9010238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Gamified E-Learning Module. [2020-08-18]. https://coronavirus.anesth.ch.
- 21.Brigade Sanitaire Cantonale Coronavirus de la maladie COVID-19: prise en charge préhospitalière. [2020-04-22]. https://smur.hug-ge.ch/sites/smur/files/guideline_covid19_1_11c.pdf.
- 22.Suppan M, Stuby L, Abbas M, Gartner B, Suppan L. COVID-19 - Prehospital PPE Dataset. Mendeley Data. 2020. [2020-08-17]. https://data.mendeley.com/datasets/ffh678bv42/1.
- 23.Pappa S, Ntella V, Giannakas T, Giannakoulis VG, Papoutsi E, Katsaounou P. Prevalence of depression, anxiety, and insomnia among healthcare workers during the COVID-19 pandemic: A systematic review and meta-analysis. Brain Behav Immun. 2020 Aug;88:901–907. doi: 10.1016/j.bbi.2020.05.026. http://europepmc.org/abstract/MED/32437915. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Glass GV. Primary, Secondary, and Meta-Analysis of Research. Educational Researcher. 1976 Nov 1;5(10):3–8. doi: 10.3102/0013189X005010003. [DOI] [Google Scholar]
- 25.Ranganathan P, Pramesh C, Buyse M. Common pitfalls in statistical analysis: Clinical versus statistical significance. Perspect Clin Res. 2015;6(3):169–70. doi: 10.4103/2229-3485.159943. http://www.picronline.org/article.asp?issn=2229-3485;year=2015;volume=6;issue=3;spage=169;epage=170;aulast=Ranganathan. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Latimier A, Riegert A, Peyre H, Ly ST, Casati R, Ramus F. Does pre-testing promote better retention than post-testing? NPJ Sci Learn. 2019 Sep 24;4(1):15. doi: 10.1038/s41539-019-0053-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Rouleau G, Gagnon M, Côté J, Payne-Gagnon J, Hudson E, Dubois C, Bouix-Picasso J. Effects of E-Learning in a Continuing Education Context on Nursing Care: Systematic Review of Systematic Qualitative, Quantitative, and Mixed-Studies Reviews. J Med Internet Res. 2019 Oct 02;21(10):e15118. doi: 10.2196/15118. https://www.jmir.org/2019/10/e15118/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Lozano-Lozano M, Fernández-Lao C, Cantarero-Villanueva I, Noguerol I, Álvarez-Salvago F, Cruz-Fernández M, Arroyo-Morales M, Galiano-Castillo N. A Blended Learning System to Improve Motivation, Mood State, and Satisfaction in Undergraduate Students: Randomized Controlled Trial. J Med Internet Res. 2020 May 22;22(5):e17101. doi: 10.2196/17101. https://www.jmir.org/2020/5/e17101/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Godlee F. Protect our healthcare workers. BMJ. 2020 Apr 02;:m1324. doi: 10.1136/bmj.m1324. [DOI] [Google Scholar]
- 30.Houghton C, Meskell P, Delaney H, Smalle M, Glenton C, Booth A, Chan XHS, Devane D, Biesty LM. Barriers and facilitators to healthcare workers' adherence with infection prevention and control (IPC) guidelines for respiratory infectious diseases: a rapid qualitative evidence synthesis. Cochrane Database Syst Rev. 2020 Apr 21;4:CD013582. doi: 10.1002/14651858.CD013582. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Schibler M, Vetter P, Cherpillod P, Petty T, Cordey S, Vieille G, Yerly S, Siegrist C, Samii K, Dayer J, Docquier M, Zdobnov E, Simpson A, Rees P, Sarria F, Gasche Y, Chappuis F, Iten A, Pittet D, Pugin J, Kaiser L. Clinical features and viral kinetics in a rapidly cured patient with Ebola virus disease: a case report. The Lancet Infectious Diseases. 2015 Sep;15(9):1034–1040. doi: 10.1016/s1473-3099(15)00229-7. [DOI] [PubMed] [Google Scholar]
- 32.Lettre d'information ERNS 19. Confédération suisse. 2019. [2020-08-18]. https://www.vbs.admin.ch/content/vbs-internet/fr/verschiedene-themen-des-vbs/die-schweizer-sicherheitspolitik/die-sicherheitsverbundsuebung--svu19-/_jcr_content/contentPar/tabs/items/documents/tabPar/downloadlist/downloadItems/30_1567670674944.download/Lettre%20information%20ERNS%2019%20-%20Aout%202019.pdf.
- 33.Rowan NJ, Laffey JG. Challenges and solutions for addressing critical shortage of supply chain for personal and protective equipment (PPE) arising from Coronavirus disease (COVID19) pandemic - Case study from the Republic of Ireland. Sci Total Environ. 2020 Jul 10;725:138532. doi: 10.1016/j.scitotenv.2020.138532. https://linkinghub.elsevier.com/retrieve/pii/S0048-9697(20)32045-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Delgado D, Wyss Quintana F, Perez G, Sosa Liprandi A, Ponte-Negretti C, Mendoza I, Baranchuk A. Personal Safety during the COVID-19 Pandemic: Realities and Perspectives of Healthcare Workers in Latin America. Int J Environ Res Public Health. 2020 Apr 18;17(8):2798. doi: 10.3390/ijerph17082798. https://www.mdpi.com/resolver?pii=ijerph17082798. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kang J, O'Donnell JM, Colaianne B, Bircher N, Ren D, Smith K. Use of personal protective equipment among health care personnel: Results of clinical observations and simulations. Am J Infect Control. 2017 Jan 01;45(1):17–23. doi: 10.1016/j.ajic.2016.08.011. [DOI] [PubMed] [Google Scholar]
- 36.Serra M, DeMarree K. Unskilled and unaware in the classroom: College students' desired grades predict their biased grade predictions. Mem Cognit. 2016 Oct;44(7):1127–37. doi: 10.3758/s13421-016-0624-9. [DOI] [PubMed] [Google Scholar]
- 37.Osterhage JL, Usher EL, Douin TA, Bailey WM. Opportunities for Self-Evaluation Increase Student Calibration in an Introductory Biology Course. CBE Life Sci Educ. 2019 Jun;18(2):ar16. doi: 10.1187/cbe.18-10-0202. http://europepmc.org/abstract/MED/31025916. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Tan AJQ, Lee CCS, Lin PY, Cooper S, Lau LST, Chua WL, Liaw SY. Designing and evaluating the effectiveness of a serious game for safe administration of blood transfusion: A randomized controlled trial. Nurse Educ Today. 2017 Aug;55:38–44. doi: 10.1016/j.nedt.2017.04.027. [DOI] [PubMed] [Google Scholar]
- 39.Dankbaar MEW, Richters O, Kalkman CJ, Prins G, Ten Cate OTJ, van Merrienboer JJG, Schuit SCE. Comparative effectiveness of a serious game and an e-module to support patient safety knowledge and awareness. BMC Med Educ. 2017 Feb 02;17(1):30. doi: 10.1186/s12909-016-0836-5. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-016-0836-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Koka A, Suppan L, Cottet P, Carrera E, Stuby L, Suppan M. Teaching the National Institutes of Health Stroke Scale to Paramedics (E-Learning vs Video): Randomized Controlled Trial. J Med Internet Res. 2020 Jun 09;22(6):e18358. doi: 10.2196/18358. https://www.jmir.org/2020/6/e18358/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Brame CJ. Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content. CBE Life Sci Educ. 2016;15(4):1–6. doi: 10.1187/cbe.16-03-0125. http://europepmc.org/abstract/MED/27789532. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Kwag K, González-Lorenzo M, Banzi R, Bonovas S, Moja L. Providing Doctors With High-Quality Information: An Updated Evaluation of Web-Based Point-of-Care Information Summaries. J Med Internet Res. 2016 Jan 19;18(1):e15. doi: 10.2196/jmir.5234. https://www.jmir.org/2016/1/e15/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Eysenbach G. The law of attrition. J Med Internet Res. 2005;7(1):e11. doi: 10.2196/jmir.7.1.e11. http://www.jmir.org/2005/1/e11/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Hochheimer CJ, Sabo RT, Krist AH, Day T, Cyrus J, Woolf SH. Methods for Evaluating Respondent Attrition in Web-Based Surveys. J Med Internet Res. 2016 Nov 22;18(11):e301. doi: 10.2196/jmir.6342. http://www.jmir.org/2016/11/e301/ [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
First email sent to participants belonging to the control group.
First email sent to participants belonging to the e-learning group.
Details of the answer set excluded by the blinded data analysts.
Data Availability Statement
The original data has been deposited to Mendeley Data [22].