Impact of a Focused Online Teaching Module on Airway Intervention: Can an Online Teaching Module Enable Knowledge Acquisition and Increased Confidence in Airway Management?

Abstract

OBJECTIVE

Teaching procedural skills via digital platforms is challenging. There is a paucity of literature on the feasibility of implementing an online asynchronous web-based learning (WBL) module for endotracheal intubation in Emergency Medicine. Learners completed a pre-test questionnaire prior to reviewing the module to assess their current knowledge. After completion of the online module, another assessment on airway management competency was completed. The purpose of our pilot study was to determine the feasibility of implementing an online airway module and investigate knowledge acquisition among learners who completed it. Additionally, we compared the relationship between pre-module confidence and knowledge between various training levels of those who completed the module.

METHODS

The study was IRB exempt. We conducted a quasi-experimental pre- and post-test study, where learners took a multiple-choice question-based test before watching content module, and after completion of modules, they went on to complete post-test questions. All responses were collected using Google survey and the data were collected over a period of 6 months. We performed descriptive statistics for the pre- and post-module. Frequency distribution was used for data summarization and chi-square test was used to assess the difference between variables.

RESULTS

We received 366 responses in the pre-test module and 105 in post-test module. Responses were summarized into 5 broad categories which assessed knowledge about airway technique, anatomical landmarks, formulas for selecting blade size, tube size, depth of tube, and case-based scenarios. All questions showed a higher percentage of correct answers in the post-assessment compared to the pre-assessment.

CONCLUSION

The results demonstrated that this WBL airway module resulted in significant knowledge acquisition, as well as increased confidence when approaching airway management. The study demonstrated that a WBL airway module is a feasible method of asynchronous education for healthcare providers in all levels of training.

Introduction

Web-based media and digital study tools are finding larger audiences each year.^1,2 Increasing amounts of medical information and its rapid digital dissemination have changed traditional teaching methods. This has also enabled more self-directed and online learning approaches. ³

Web-based learning (WBL) is defined as the “use of computers and networks in education.” ⁴ It is an increasingly popular method of instruction in higher education. In an era where learners move at their own pace and complete objectives on their own time, WBL provides a unique opportunity.^5,6 WBL is a form of asynchronous learning, which occurs outside the constraints of time and space. The learner has the flexibility of using resources across different digital platforms, at their own pace, whether at home or at the library. There are multiple studies that have examined WBL as a form of asynchronous learning. In WBL, the learner utilizes various multimedia for additional self-guided education, as opposed to traditional methods such as didactic lectures and primary textbooks.^4,6–8 Learners find higher satisfaction with WBL asynchronous learning, often citing flexibility as a major advantage.^9–11

As the use of WBL expands into various medical specialties, one type of WBL that can be difficult to implement in multimedia format is teaching procedural skills. Overall, the measurement of procedural competency is difficult to standardize and assess. ¹² Indeed, psychomotor skills are required in many medical procedures, further challenging the development of effective WBL modules.

There are certain procedures an Emergency Medicine (EM) physician must be capable of performing, as defined by the Accreditation Council for Graduate Medical Education (ACGME). ¹³ Airway management is a core skill of any emergency physician. The inability to properly manage an airway results in significant morbidity and mortality. ¹⁴ Endotracheal intubation is consistently cited as a critical procedural skill EM physicians must display competency to avoid poor patient outcomes.^15,16 Endotracheal intubation is one of the most commonly performed procedures in the ED, further making its competency critical to obtain. ¹⁷

With WBL, there have been studies demonstrating the feasibility of online airway simulation and live training courses.^18–20 Additionally, multiple anesthesiology-based studies have shown the benefit of offering online lectures and videos, in combination with case-based approaches.^21,22 However, to our knowledge, there are not any studies discussing the feasibility of implementing an online WBL module for endotracheal intubation in the field of EM.

Emergency Medicine Board Bombs (EMBB) was launched in 2018 as an asynchronous online educational tool to assist residents and attendings taking their in-service exam and boards, respectively. EMBB is a peer-reviewed resource and functions at no cost to the learner. Since its founding, the website has expanded to offer a broad number of resources, including podcasts, study guides, and practice questions. In May 2020, the website launched an airway module, titled “Airway Supremacy.” The module is offered at no cost to the learner. It has a pre and post self-assessment test and provides education on adult and pediatric intubations. Given that a large portion of endotracheal intubation involves the knowledge of anatomy, equipment, pharmacology, and pre-procedural patient positioning, we sought to create a WBL that taught these skills to learners. These are skills that must be understood before approaching any patient needing endotracheal intubation.

The purpose of our study was to determine the feasibility of implementing an online airway module and investigate knowledge acquisition among learners who completed the module. Additionally, we sought to compare the relationship between pre-module confidence and knowledge between various training levels of those who completed the module.

Methods

This pilot study was conducted at Wake Forest School of Medicine. We conducted a quasi-experimental pre- and post-test study, where learners took a multiple-choice question-based test before watching content module, and after completion of modules, they went on to complete post-test questions.

Data were collected from the airway module from its release in May 2020 to September 2020. This study was IRB exempt and no waiver was required as per our IRB at our institution after communicating with them (Wake Forest Baptist Health Hospital IRB). The need for informed consent was waived by the IRB as well.

In constructing the module, the study team reviewed all recorded intubation videos from an academic Emergency Department's Karl Storz C-Mac^© video laryngoscope carts. All intubations were performed by an EM physician, either resident or attending. All videos were downloaded from the cart using a SIM card to a secure, HIPPA-compliant hospital computer. Videos were excluded if they failed to show endotracheal intubation being performed. Videos were included if they demonstrated either a successful or unsuccessful endotracheal intubation performed. Both pediatric and adult intubations were included.

The videos did not display any patient or healthcare staff identifying information. To ensure this, the videos were reviewed, and only content detailing endotracheal intubations were retained. The videos had no sound associated with them.

The videos were added to the EMBB website. ²³ A special landing page was created titled “Airway Supremacy,” for the airway module. Select videos were edited into distinct, shorter clips with timestamps and description boxes next to them. These videos educate on the steps of endotracheal intubation in detail, allowing the learner to pause at their convenience and read the corresponding information.

The target audience included EM physicians, both residents and attendings, and medical students. The module was open to paramedics, physician assistant, and nurse practitioners as well. Recruitment methods included emails to EMBB subscribers and internally to those practicing at the institution. However, as this was the first module to be created on an open educational website platform, it was open to all internet users and no verification of healthcare status was required. Therefore, demographics were not tracked.

Some videos demonstrated unique airway situations, such as trauma, use of the gum elastic bougie, and pediatric airways. Multiple failed endotracheal intubation videos were also included. In total, 17 videos totaling 11 min were embedded in the module.

Throughout the length of the module, various tables, photos of endotracheal intubation equipment, and review questions were incorporated to test the learner's knowledge acquisition. A drop-down tab allowed answers to be revealed when clicked on by the learner (Figure 1; Appendix images 1–3).

Figure 1.

Embedded video on the module with drop-down tab and video timesheet (https://vimeo.com/583870201).

Pre- and post-self-assessments were inserted prior to the beginning of the module and at the end, respectively. We developed the questions and created the assessments using Google Forms, a survey administration application that is included with Google Docs. ²⁴ It allows for the collection of responses from a personalized quiz or survey that the user creates. ²⁵ The pre- and post-self-assessments were evaluated and reviewed by the authors of this manuscript before the module went live.

The pre-test and post-test modules comprised of 14 multiple choice questions each in random order. Questions were classified into 4 broad categories. These assessed knowledge about airway technique, anatomical landmarks, formulas for selecting blade size, endotracheal tube size, depth of endotracheal tube, and finally case-based scenarios.

The pre-assessment had 2 demographic questions. One asked the learner their level of training and the other asked the learner how confident they were with the process of intubation on a scale of 1 to 5.

The post-assessment asked 2 unique questions: How confident the learner was with the process of intubation on a scale of 1 to 5 and to rate the impact of the training module on the learner's education on a scale of 1 to 5.

Before its release, the module was reviewed by 5 academic faculty EM physicians for clarity and ease of use, including download speed, navigation, and accessibility.

The module was released on the website in May 2020. To assess the module, the learner had to complete the pre-assessment first. They were then directed to the module. The post-assessment was available for completion at the end of the module.

There was no collection of personal or contact information on the website. Responses were collected for a 5-month period from May 2020 to September 30, 2020.

Statistical analysis

Responses were reviewed from Google Forms and exported to Microsoft Excel for analysis. We performed descriptive statistics for the pre- and post-module. Frequency distribution was used for data summarization. To assess the difference between continuous and categorical variables, Wilcoxon rank sum test and chi-square test were used, respectively. All computations were done using STATA 12.0.

Results

We received 366 responses in the pre-test module and 105 responses in post-test module. The pre-test and post-test modules comprised of 14 multiple choice questions each. After completing a pre-test questionnaire, learners would complete a comprehensive reading assignment which would comprise details on above-mentioned categories, which included illustrations and videos to enhance the learning experience. For airway technique, the correct responses improved from 54% in the pre-test module to 77% in the post-test module. For anatomical landmarks, we observed an increase from 70% to 88%. Case-based scenarios and formulas increased 30% each, from 36% to 66% in the former and 48% to 78% in the latter (Table 1).

Table 1.

Comparison of pre- and post-quiz performance based on the key quiz categories.

Category	Pre-test	Post-test
Airway technique	54%	77%
Anatomical landmark	70%	88%
Formulas	48%	78%
Case-based scenarios	36%	66%
Pediatric scenarios	45%	70%

We observed higher confidence in learners with greater years of training such as attending or senior residents versus EM interns or medical student (91% vs 44%, P < .000). In our pre-test, we also found that learners with more years of training had a higher median score in comparison to learners with less years of training (7 vs 5, P < .000).

Discussion

Principal results

The results demonstrated that this WBL airway module resulted in improved knowledge acquisition, as well as increased confidence when approaching airway management. All quiz questions showed a higher percentage of correct answers when comparing the pre-assessment to post-assessment. This pilot study demonstrated a WBL airway module is a feasible method of asynchronous education for healthcare providers in all levels of training.

Our results illustrate a range of training levels completed the module, from attendings to medical students. Several participants indicated they had increased confidence when it came to airway management (Figure 1). The participants viewed the module favorably.

Our findings are consistent with prior studies of WBL which demonstrated higher satisfaction and higher evaluation scores overall with WBL compared to text-based materials.^11,22 Additionally, sufficient evidence has demonstrated that well-designed WBL can enhance learner's confidence. ²²

Comparison with prior work

The true value of this module is not only affirmed by the high score on the post-self-assessment, as well as the post-self-assessment scores, but with how unique it is. To date, there is no online, free video module on airway management with actual performed endotracheal intubations. To our knowledge, it is the only WBL that has recorded intubation videos from both failed and successful procedures in both adults and pediatrics. By utilizing video laryngoscopic and fiberoptic recording software, we offer critical airway teaching principles.

It is well known that well-designed WBL is as effective as traditional educational methods. ²⁶ A systematic review published in 2008 identified the most effective strategies for learning outcomes in WBL. They discovered that the most successful WBL modules utilize rich multimedia, engaging content, and the opportunity to perform self-assessment. ²⁷ Other studies found that accessibility, download speed, ease of navigation, and overall attractiveness of the module greatly matter when recruiting learners.^28–33

Unsuccessful WBL modules rely too heavily on traditional methods of teaching: long text manuscripts based off books or lectures, and poorly interactive exercises. ³⁴ Other barriers to developing successful WBL include time, cost, and technical expertise. ³⁵

During our airway module's development, we were careful to ensure it would be engaging to all levels of training, with no more than a few sentences per text block, frequent use of pictures and tables, as well as review questions within each content section. The videos were spaced out throughout the module, each with an instructive timesheet that discussed the intubation step-by-step. Special coding ensured navigation and accessibility was carefree for the learner. Intuitive drop-down tabs and boxes which revealed answers when clicked on provided for an attractive, inviting module. Prominent website designers have stated that even the background color of a website is critical to being welcoming to learners. The best color scheme was found to be white or a light background, with bold dominant text to keep the learner's focus on the teaching points.^36–39

Time was dedicated to ensuring page download speed was not slow, as to not discourage learners from finishing the module. Multiple studies demonstrate that download speed is as critical to learner satisfaction as content.^22,40–43 Our airway videos are distinctive in that the learner witnesses endotracheal intubation pitfalls and failures from real-life intubations. Our module was low cost, with the only financial commitment being a website platform fee (less than $100 annually). The construction of the online module including the edit of all videos took approximately 1 month of dedicated work. We found that revisions were rare, averaging once every 6 months.

Future research utilizing our module combined with bedside or simulated technique instruction are critical to further procedural competency. As the volume of medical information continues to exponentially grow and digitally distribute, learners are constantly seeking the most up-to-date information in an interactive way. ⁴⁴ This pilot study provides valuable insight into a WBL module that is unique, easy to implement, and features an innovative strategy to provide airway management education.

Limitations

There are several limitations of this study. First, the module is open to all internet users and no verification of healthcare status was required. Consequently, any person could take the quiz, regardless of their medical background.

This pilot study establishes the feasibility of an online module regarding intubation. No analytics have been established regarding the formal performance of this module and its assessment. Further research will be directed at establishing question analytics to evaluate whether the assessment is a valid measure of learning outcome.

While increased confidence is welcome when teaching procedures, proper technique is critical for procedure success. Being an online module with no in-person follow up, we were unable to test proper technique and psychomotor skills. Future research is needed to objectively evaluate learner skill development.

Another limitation includes the self-assessment completion rate and sample size. We were not able to directly control the completion rate of the pre- and post-self-assessments, and therefore, could not pick the sample size. We also could not control if a user chose to skip the pre-assessment and only complete the post-assessment. The pre-self-assessment had substantially more completions than the post-self-assessment. As with most online surveys and modules with questionnaires, this is to be expected. Of the total number who completed the pre-assessment (346 entries), there were only 105 who completed the post-assessment (32.9% completion rate). This is consistent with the average internet survey response rate of 29%. ⁴⁵ Further examination should be directed at contributing factors such as module length, technical problems, or quiz complexity dissuading learners from completing the activity.

Future research should be directed at measuring clinical outcomes in those who have completed this airway module. Further investigation should be performed on this module's effect on procedure performance in real time at the bedside.

Conclusions

Our results indicate this novel WBL airway module enabled significant knowledge acquisition, as well as increased confidence when approaching airway management. Overall, every post-test question showed a higher percentage of correct answers when compared to pre-test. This pilot study is the first to demonstrate how a novel WBL airway module is a feasible method of asynchronous education for healthcare providers in all levels of training.

Abbreviations

ACGME

Accreditation Council for Graduate Medical Education

EM

emergency medicine

EMBB

emergency medicine board bombs

WBL

web-based learning.