Improving neuromuscular monitoring and reducing residual neuromuscular blockade via e‐learning: A multicentre interrupted time‐series study (INVERT study)

Abstract

Background

Neuromuscular monitoring should be applied routinely to avoid residual neuromuscular block. However, anaesthetists often refrain from applying it, even when the equipment is available. We aimed to increase neuromuscular monitoring in six Danish anaesthesia departments via e‐learning.

Methods

Interrupted time series study, with baseline data from a previous study and prospective data collection after implementation of the module, which was available for 2 weeks from 21 November 2016. We included all patients receiving general anaesthesia with muscle relaxants until 30 April 2017. Main outcome was application of acceleromyography, grouped as succinylcholine only and non‐depolarising relaxants. Secondary outcomes were last recorded train‐of‐four ratio (non‐depolarising) relaxants and score on a ten‐question pre‐ and post‐course multiple‐choice test.

Results

The post‐intervention data consisted of 6525 cases (3099 (48%) succinylcholine only, 3426 (52%) non‐depolarising relaxants). Analysing all departments, we found a positive pre‐intervention trend in application of acceleromyography for both groups, of estimated 7.5% and 4.8% per year, respectively (p < .001). The monitoring rate increased significantly for succinylcholine in two departments post‐intervention (p = .045 and .010), and for non‐depolarising relaxants in one department (p = .041), but followed by a negative trend of −37.0% per year (p = .041). The rate was already close to 90% at the time of the intervention and the mean last recorded train‐of‐four ratio was 0.97 (SD 0.21), also without a significant change. The median score on the post‐course test increased from 7 (IQR 5–8) to 9 (IQR 8–10) (p < .001, Wilcoxon Signed‐Ranks Test).

Conclusion

We found no overall effect of the e‐learning module on application of neuromuscular monitoring, although the post‐course test indicated an effect on anaesthetists’ knowledge in this field.

Trial registration

Trial registration: Clinicaltrials.gov identifier: NCT02925143. https://clinicaltrials.gov/ct2/show/NCT02925143

Editorial Comment.

It is unclear if a knowledge gap for the clinician is the reason that there can be persistent neuromuscular blockade effects for patients in the post‐anaesthesia recovery area. In this study, an e‐learning tool which aimed to increase neuromuscular monitoring during anaesthesia was implemented in several hospitals. The tool was well received but a large measurable effect could not be demonstrated probably due to a pre‐intervention increase in monitor use. Additional testing is needed on sites with low use of neuromuscular monitoring.

1. INTRODUCTION

Neuromuscular monitoring should be applied to avoid residual neuromuscular block. ¹ However, anaesthetists do not apply neuromuscular monitoring consistently. ² , ³ , ⁴ Especially, when administering succinylcholine as the sole neuromuscular blocking agent (NBMA), monitoring is often omitted, ⁵ , ⁶ although this may have serious consequences for the patients. ⁷ In 2014, following a year‐long quality improvement project, an increase in objective neuromuscular monitoring rate of approximately 95% was reported in one institution. ⁸ In another institution, an increase in the rate of neuromuscular monitoring from 2% to 60% was reported in the years 1995 to 2004, also with an ongoing quality improvement effort. ⁹ In Denmark, objective neuromuscular monitoring is an integrated part of anaesthesia machines in most operating rooms and in some institutions, anaesthesia personnel now monitor the neuromuscular block routinely. ⁵ , ⁶ However, many anaesthetists continue to experience challenges with fluctuating measurements and difficulties interpreting the results of monitoring, and request further training in using the equipment. ³ , ⁶

An internet‐based learning module, often referred to as e‐learning, allows participants to access educational material independent of time and place and may include instructional aids such as animations, tests and adaptive content based on participants’ individual needs. In a meta‐analysis of published studies examining the effect of e‐learning, it was found that the modality may produce results similar to traditional teaching methods. ¹⁰

We aimed to investigate whether an e‐learning module implemented simultaneously in six anaesthesia departments could increase the use of neuromuscular monitoring and reduce the proportion of patients awakened with a train‐of‐four (TOF) ratio <0.9.

2. METHODS

We designed a multicentre interventional study to assess the effect of an e‐learning module, developed specifically for the study. The Committees on Health Research Ethics for the Capital Region of Denmark waived approval because of the educational nature of the intervention (protocol 16028220). The Danish Patient Safety Authority approved the use of routinely collected patient data for the study without individual patient consent (3–3013‐1467/1). We published the study protocol, ¹¹ and pre‐registered the study at clinicaltrials.gov (NCT02925143), and with The Danish Data Protection Agency (HGH‐2015‐063/04364) before implementation of the intervention.

We included six of seven anaesthesia departments in the Zealand Region of Denmark, where objective acceleromyographic monitoring was integrated in the anaesthesia machine in all operating rooms. We recruited local investigators at the six departments and aimed to have all clinically active anaesthesiologists and nurse anaesthetists (referred to collectively as anaesthetists in the following) to complete the e‐learning module in the intervention period defined as 2 weeks from the 21 November 2016. We included records of all patients receiving general anaesthesia with use of NMBAs in the study period. Patients who received anaesthesia on multiple occasions were included as multiple cases.

The primary outcome was use of acceleromyography in cases involving an NMBA. We divided the outcome in cases with succinylcholine only and cases with non‐depolarising NMBA (with or without succinylcholine), because the agents are typically used in different clinical situations (rapid sequence induction and elective intubation or surgical paralysis, respectively). Secondary outcomes included assessment of the effect of the e‐learning module on anaesthetists’ knowledge as assessed by a pre‐ and post‐course test, the last recorded TOF ratio in cases involving a non‐depolarising NMBA, reversal of neuromuscular block with neostigmine or sugammadex, respectively, and length of stay in the post‐operative care unit.

The e‐learning module was developed in collaboration with Regional Unit for Development and Evaluation of Learning Technologies in the Capital Region of Denmark. ¹¹  The topics included theory of NMBA, residual neuromuscular block and reversal, as well as practical tips on neuromuscular monitoring and troubleshooting equipment malfunction. Interactive animations of neuromuscular block and reversal at the level of the synaptic cleft were included to give an in‐depth understanding of the mechanisms involved (Figure 1). The animations were split into several smaller sections with key elements repeated in order to reduce cognitive load. ¹² Clickable optional content made the module adaptable to participants’ different learning needs as guided by research in e‐learning. ¹³ Examples of acceleromyography monitor output was included to increase relevance to the clinicians’ daily practice. Cases and quotes from patients who had experienced being awakened while paralysed because of butyrylcholinesterase deficiency were included with the aim of changing clinicians’ attitude towards the relevance of monitoring the depolarising neuromuscular block. The course duration was approximately 30 min. The effect of the e‐learning module on anaesthetists’ knowledge was assessed by a ten‐question pre‐ and post‐course multiple choice test (Appendix S1).

FIGURE 1.

E‐learning module. Examples of interactive content from the e‐learning module

Data were recorded automatically with an entry per minute in the anaesthesia information management system MetaVision (iMDsoft^®, Düsseldorf, Germany). Acceleromyography data were recorded automatically when the TOF stimulation was initiated, while anaesthetics and other medicines were recorded manually by the anaesthetist by pressing a software button. Baseline data before implementation of the intervention was obtained from a descriptive study using the same database. ⁵  Variables were defined as described in the baseline study. We collected post‐intervention data in the period 21 November 2016 to 30 April 2017.

We used interrupted time series analysis of the routinely collected data, because it allowed us to describe and account for any pre‐existing trends in the use of neuromuscular monitoring. ¹⁴  The method involves using segmented regression analysis to assess two possible changes that may occur with an intervention: a sudden change in the level of the pre‐existing trend at the time of the intervention and, secondly, a change in the trend after the intervention. Sample size calculation showed that a change in acceleromyography use from 40% to 60% would require three data collection periods before and after the intervention, with approximately 100 cases from each department. It was estimated that 9 weeks of data collection before and after the intervention would satisfy this requirement. Data collection was extended to 30 April 2017 to enable assessment of a wearing off of any potential effect. Non‐normally distributed paired data will be analysed using Wilcoxon Signed‐Ranks Test. Analyses were performed by authors JLDT and LTS in SPSS version 22 (SPSS Inc., Chicago, IL, USA). We considered a two‐tailed p value of < .05 statistically significant.

3. RESULTS

One week after the introduction of the e‐learning module, the mean completion rate across all departments was 24%. After 1, 2 and 3 additional weeks, this increased to 45%, 63% and 76%, respectively. Department 3 had a final completion rate of 34%, while the remaining departments had a mean of 85%. A total of 429 anaesthetists completed the e‐learning module. Complete data on the ten‐question pre‐ and post‐course multiple choice test were available for 367 (86%) anaesthetists. The median score was seven (IQR 5–8, range 1–10) on the pre‐course test and nine (IQR 8–10, range 3–10) on the post‐course test (p < .001, Wilcoxon Signed‐Ranks Test). For the individual anaesthetist, the median change was two (IQR 0–3, range (−3)‐7). Figure 2 shows the change in percentage of correct answers in the ten questions of the pre‐ and post‐course test.

FIGURE 2.

Pre‐ and post‐course test results. The columns show the percentage of correct answers in the ten questions of the pre‐ and post‐course test, respectively. Answers from 367 anaesthetists

The baseline dataset consisted of 30,430 cases, of which 13,905 (46%) received succinylcholine only and 16,525 (54%) received a non‐depolarising NMBA (with or without succinylcholine). ⁵ We collected 6525 cases post‐intervention from the period 21 November 2016 to 30 April 2017, of which 3099 (48%) involved succinylcholine only, and 3426 (52%) involved a non‐depolarising NMBA (with or without succinylcholine) (Figure 3). Baseline and post‐intervention patient characteristics were comparable (Table 1).

FIGURE 3.

Study diagram

TABLE 1.

Patient characteristics

	Baseline 5 n = 30,430		Post‐intervention n = 6525
	Mean or number	% or SD	Mean or number	% or SD
Age; y	57.0	19.9	57.8	19.3
Sex
Male	13,529	44.5%	2928	44.9%
Female	16,901	55.5%	3597	55.1%
Height; cm	170.2	11.1	170.3	11.1
Weight; kg	78.4	21.0	78.9	20.6
ASA physical status
I	7569	24.9%	1341	20.6%
II	14,726	48.4%	3226	49.4%
III	7613	25.0%	1818	27.9%
IV	460	1.5%	122	1.9%
V	11	0.0%	1	0.0%
Unknown	51	0.2%	17	0.2%
Priority
Elective	19,240	63.2%	3728	57.1%
Emergency or urgent	11,110	36.5%	2395	36.7%
Unknown	80	0.3%	402	6.2%

3.1. Primary outcome

3.1.1. Neuromuscular monitoring in cases involving succinylcholine

A positive pre‐intervention trend in use of acceleromyography in cases involving succinylcholine only was found in the analysis of all departments as a whole with a calculated estimate of 7.5% per year (p < .001) (Figure 4). There was no statistically significant difference in the trend after implementation of the intervention, though visual inspection of the plot could give the impression of a change (p = .264).

FIGURE 4.

Percentage of applied acceleromyography in cases involving succinylcholine only and cases involving a non‐depolarising muscle relaxant, respectively. E‐learning implemented at week 108 from the beginning of the baseline period

The individual departments showed a wide variability in the use of acceleromyography in cases involving succinylcholine only. Departments 2, 3, 4 and 6 showed significant positive pre‐intervention trends, with department 6 having a final monitoring rate of approximately 90% (Figure 5). After the intervention, there was a significant sudden change in departments 2 and 4, and in department 4 this change was followed by a highly significant increase in the trend to estimated 109.4% per year (p = .005).

FIGURE 5.

Percentage of applied acceleromyography in cases given succinylcholine only in individual departments. E‐learning implemented at week 108 from the beginning of the baseline period

3.1.2. Neuromuscular monitoring after non‐depolarising NMBAs

A positive pre‐intervention trend in use of acceleromyography in cases involving a non‐depolarising NMBA was found in the analysis of all departments as a whole, with a calculated estimate of 4.8% per year (p < .001) (Figure 4). At the time of the intervention, acceleromyography was used in 92.8% of all cases involving non‐depolarising NMBAs and there was no detectable change in trend after the intervention. For the individual departments, we found a significant pre‐intervention positive trend in departments 2, 4 and 5 (Figure 6), with estimates 12.9%, 5.1% and 2.1% per year, respectively. At the time of the intervention, the acceleromyography rates at departments 2 to 6 were already above 85%. In department 2, there was a significant change following the intervention, but it was followed by a significant decreasing trend of −37.0% per year (Figure 6). There were no significant changes in the remaining departments.

FIGURE 6.

Percentage of applied acceleromyography in cases given non‐depolarising NMBA in individual departments. E‐learning implemented at week 108 from the beginning of the baseline period

3.2. Secondary outcomes (non‐depolarising NMBAs only)

3.2.1. TOF‐ratio before extubation

Analysing all departments as a whole, we found no significant pre‐intervention trend in the median last recorded TOF value (p = .129). At the time of the intervention, the mean last recorded TOF ratio was 0.97 (SD 0.21). There was no significant change with the intervention (p = .180). The proportion of patients with a last recorded TOF ratio <0.9 showed a significant negative pre‐intervention trend of −6.6% per year (p < .001) (Figure 7), decreasing from 31.8% at the beginning of the baseline period to 22.2% at the time of the intervention. There was a sudden further negative change after the intervention (p = .007), but it was followed by a positive trend post‐intervention (p = .006).

FIGURE 7.

Proportion of patients with a last recorded TOF ratio <0.9. E‐learning implemented at week 108 from the beginning of the baseline period

Analysing the individual departments, we found a negative pre‐intervention trend in the median last recorded TOF ratio for department 4, decreasing 1.6% per year (p = .028) and a positive trend for department 6, increasing 1.9% per year (p < .001), but no change in trend in the other departments. We found a significant negative trend in proportion of cases with a last recorded TOF ratio <0.9 in departments 1 (−7.9% per year, p = .009), 3 (−9.5% per year, p = .003) and 6 (9.9% per year, p < .001). There was no significant sudden change with the intervention in any of the departments, but after the intervention, there was a small positive trend in department 6 (p = .045).

3.2.2. Reversal with neostigmine and sugammadex, and length of stay in the post‐operative care unit

Analysing all departments as a whole, we found a positive trend in the proportion of patients receiving neostigmine, with an increase from 51.6% to 60.6% before the intervention (p < .001). There was no change with the intervention. A significant trend was found for department 2 increasing 6.7% per year (p = .001), department 3 decreasing 10.5%, department (p = .002), department 4 increasing 8.6% (p = .004), department 5 increasing 3.6% (p = .010) and department 6 increasing 3.5% (p = .010). There was no significant change in the timing of reversal of neuromuscular blockade with neostigmine.

Analysing all departments as a whole, we found a positive pre‐intervention trend in the proportion of patients receiving sugammadex, with an increase from 1.1% to 2.1% before the intervention (p < .001). A significant trend was found for department 2 increasing 1.4% per year (p = .021) and department 5 increasing 1.1% per year (p = .008). A total of 230 cases received sugammadex.

There was no change in the median length of stay in the post‐operative care unit after the intervention.

4. DISCUSSION

We implemented an e‐learning module on neuromuscular monitoring simultaneously in six anaesthesia departments. There was a significant positive pre‐intervention trend in use of acceleromyography both in cases involving succinylcholine only and cases involving non‐depolarising NMBAs, but we could not detect an effect of the e‐learning module on the monitoring rate in general. We did, however, detect a decrease in the proportion of patients with a last recorded TOF ratio <0.9 after the intervention, but it appeared to be evened out shortly thereafter.

The present study is the first aiming to increase the use of objective neuromuscular monitoring by means of e‐learning and, therefore, our results cannot be directly compared with other studies. However, other researchers have successfully managed to increase the use of objective neuromuscular monitoring in a single department using more traditional learning methods. Baillard and colleagues conducted four 3‐month audits in the years 1995 to 2004, measuring TOF ratio on patients arriving to the recovery area. They reported the incidence of residual neuromuscular block to anaesthetists at the department and repeatedly trained them in neuromuscular monitoring and reversal. By the end of the intervention period, use of objective neuromuscular monitoring had increased from 2% to 60% and the incidence of residual neuromuscular block decreased from 62% to 3.5%. ⁹ Using a similar approach in 2011, Todd and colleagues also managed to increase the use of monitoring and decrease the incidence of residual block over a 2‐year period. ⁸ In both studies, objective neuromuscular monitoring was implemented in the same year as the study started.

There are several possible explanations why the e‐learning module in our study generally did not affect the use of acceleromyography. Compared with the studies by Baillard and Todd, ⁸ , ⁹ there was little room for improvement with regard to monitoring the non‐depolarising block, since the monitoring rate was already above 85% in 5 of 6 departments before the intervention. The effectiveness of e‐learning may be assessed using Kirkpatrick's model which defines four levels of evaluation: Satisfaction, learning, change in learner behaviour, and organisational change or patient outcome. ¹⁵ It is considered easier to obtain a change in the first categories compared with the latter, which is also reflected in the results of the pre‐ and post‐course test. ¹⁰ However, we also aimed at changing behaviour, i.e. use of monitoring, and affect the patient outcome of last recorded TOF value, which proved more difficult. We chose e‐learning to be the sole intervention to increase clarity of a potential effect, but perhaps a greater effect could have been obtained by including hands‐on‐training and audits on the incidence of residual neuromuscular block. ¹⁶

The small, transient effect detected on proportion of patients with a last recorded TOF ratio <0.9 after the intervention and the small changes observed in some departments could be explained by the anaesthetists paying extra attention to applying acceleromyography immediately after the intervention, i.e. the Hawthorne effect, but then returning to usual behaviour. Observed small effects in individual departments could also be due to multiple testing. Regarding the monitoring of depolarising block, there was considerably more room for improvement. While it did look like, visually, that there was an overall effect of the intervention, it was not significant. It is possible that the post‐intervention data collection was simply too short to establish such a change. Unfortunately, it was not possible to collect more post‐intervention data, because the Zealand Region of Denmark implemented a different anaesthesia information management system mid‐2017.

The observed positive trends in almost all outcomes pre‐intervention also deserve mentioning. The increasing use of neuromuscular monitoring in the 2‐year period could be attributed to years of continued focus on neuromuscular monitoring in Denmark. ² , ³  The subject is taught in both the anaesthesia residency programme and in the nurse anaesthesia specialisation.

4.1. Strengths and limitations

The major strengths of the study include the multicentre design and the use of automatically collected objective data on the use of acceleromyography. We chose to apply interrupted time series analysis of the data, which has been described as the ‘the strongest, quasi‐experimental approach for evaluating longitudinal effects of interventions’. ¹⁴  This design allowed us to both describe the underlying pre‐intervention trend and assess a potential effect of the intervention more precisely than if using a simple before and after design. The design did not, however, include a control group. A cluster randomised design would have allowed this, but would require more departments. ¹⁷

4.2. Future studies

Though we did not show an overall effect of the e‐learning module, we believe that e‐learning is a suitable format for teaching anaesthetists the basics of neuromuscular monitoring as demonstrated by the increase in post‐course test score. It is often considered a difficult subject and requires expert personnel, which is not available in all departments. The e‐learning module also allows clinicians to re‐access the material as needed. Future studies aiming to increase use of objective neuromuscular monitoring could combine implementation of an e‐learning module with repeated surveys of the incidence of residual neuromuscular block, as described in the recent consensus statement on perioperative use of neuromuscular monitoring. ¹⁶ Such studies would probably be most likely to show an effect if conducted in settings where objective neuromuscular monitoring is used less frequently.

5. CONCLUSION

In this multicentre study investigating the effect of an e‐learning module on neuromuscular monitoring, we found no overall effect of the e‐learning module on application of neuromuscular monitoring. We did, however, find a significant change in the application of acceleromyography after the intervention in one of the six anaesthesia departments, and a significant increase in a post‐course multiple choice test, indicating an increase in knowledge in this field. Finally, we found a pre‐intervention positive trend in the rate of neuromuscular monitoring for both types of NMBAs, resulting in a monitoring rate that was already close to 90% for non‐depolarising NMBAs before implementation of the e‐learning module.

Conflicts of interest

JLDT: received research grants and speaker's fees from Merck. OM: none declared. DHP: none declared. LTS: none declared. DØ: none declared. DHB: none declared. TB: none declared. TT: none declared. HG: none declared. JB: none declared. SDT: none declared. AJ: none declared. MRG: received research grants and speaker's fees from Merck.

PRESENTATION

This study was presented as a poster presentation at the ASA Anesthesiology Conference, 13–17 October 2018, San Francisco, USA.

Supporting information

Supplementary Material

Thomsen JLD, Mathiesen O, Hägi‐Pedersen D, Skovgaard LT, Østergaard D, Gätke MR; the INVERT collaborator group . Improving neuromuscular monitoring and reducing residual neuromuscular blockade via e‐learning: A multicentre interrupted time‐series study (INVERT study). Acta Anaesthesiol Scand. 2022;66:580–588. doi: 10.1111/aas.14038