Implementation of an experiential learning strategy to reduce the risk of ventilator-associated pneumonia in critically ill adult patients

Abstract

Objective

We evaluated the impact of an experiential learning strategy on both the adherence to the use of bundles and the incidence of ventilator-associated pneumonia in critically ill adult patients.

Methods

Longitudinal, quasi-experimental interrupted time-series study in a tertiary teaching hospital in Buenos Aires, Argentina. Successive measurements were made before and after the intervention was implemented between January 2016 and December 2018. Our main exposure was experiential learning, which was based on a combination of play activities, simulation models, knowledge and attitude competencies, role-playing and feedback. The adherence to the bundle for the care of mechanically ventilated critically-ill adult patients and the occurrence of ventilator-associated pneumonia were the main outcomes of interest. We used generalized linear models including time as a linear spline to estimate the effect of the experiential learning strategy both on the adherence to the bundle of care and the occurrence of ventilator-associated pneumonia during long-term follow-up.

Results

The overall proportion of adequate bundle use before and after the implementation of the intervention was 60.8% (95% CI: 56.9–64.7) and 85.6% (95% CI: 81.2–90.1), respectively. The incidence rate of ventilator-associated pneumonia before and after the intervention was 6.11 (95% CI: 5.82–6.40) and 3.55 (95% CI: 2.96–4.14) every 1000 days of mechanical ventilation, respectively. The estimated baseline monthly change in the adherence to the mechanical ventilation bundle was 0.4% (95%CI: −0.3–1.2%, p = 0.31) and 1.1% (95% CI: 0.2–2.2%, p < 0.01) before and after the implementation of the intervention, respectively. These results were consistent across our statistical quality control analysis.

Conclusions

The implementation of experiential learning strategies improves the adherence to bundles in the care of mechanically ventilated critically ill adult patients. Such strategies also decrease the incidence rate of ventilator-associated pneumonia. Both effects appear to remain constant during long-term follow-up.

Introduction

Health care-associated infections (HCAIs) constitute a worldwide public health problem.^1,2 Approximately 1 of every 20 patients admitted to an acute care hospital will experience a HCAI.¹ This represents about 4.1 million patients a year in the European Union and over 2 million in the United States alone^1,2 with a total cost of approximately $4.5 and 5.7 billion, respectively.³ HCAI are generally more challenging to treat, mainly owing to the fact they are caused by antibiotic-resistant microorganisms.⁴ Hence, the prevention of HCAI remains not only an ongoing challenge but also a potentially widely applicable cost-effective intervention.

Within the spectrum of HCAI, ventilator-associated pneumonia (VAP) remains as one of the most frequent infections in the intensive care unit (ICU) and is associated with increased in-hospital mortality, longer lengths of stay and increased antimicrobial exposure, with the consequent increase in the risk of antimicrobial resistance.⁵ Hence, the prevention of this infection remains a priority. To this end, the training of health personnel involved in preventive measures, diagnosis and treatment of VAP is essential.⁶ Moreover, this generally includes the widely known bundle of care comprised by: (1) elevation of the head of the bed, (2) thrombo-prophylaxis, (3) stress ulcer prophylaxis, (4) daily sedation interruption and (5) awakening trials and protocol-guided weaning strategies.⁷ However, the most frequent problem encountered is the poor adherence with safety practices over time,⁸ which is also common in otherwise seemingly simple tasks such as hand hygiene.⁹ Ineffective communication and learning strategies and the consequent lack of knowledge translation into clinical action yield compliance rates that are, on average, lower than 50%.^8,10

Hence, the incorporation of learning modalities focused on the evaluation of one's own experiences and the comparison with tasks performed by peers can be considered as a useful modality to reduce the aforementioned gap.¹¹ A specific case is that of experiential learning, which with techniques such as gamification, debriefing and simulation, seeks to tackle the usual disconnection between knowledge and implementation.¹² Consequently, the present work evaluated the impact of an experiential learning strategy in the adherence to the use of bundles of care and the incidence of VAP among mechanically ventilated, critically ill adult patients. Our overall goal is to improve the safety practices within health care institutions in an effective fashion and with a sustained effect over time.

Material and methods

Study design and population

This was a longitudinal, interrupted time-series, quasi-experimental study¹³ conducted in three ICUs totaling 103 beds at the Hospital Italiano de Buenos Aires, a tertiary teaching hospital in the city of Buenos Aires, Argentina. The population subject to the educational strategy implemented was the entire unit's assistance team, which includes all registered nurses, respiratory therapists, physiotherapists, physicians and physician assistants. The study was carried out between January 2016 and December 2018. It was approved by the local research ethics committee and was conducted according to the amended declaration of Helsinki.

Main exposure and outcomes of interest

Our main exposure of interest was the occurrence of experiential learning (Table 1). This intervention was carried out in all of the included ICUs, was started on 2 January 2017, and focused on play activities, simulation models, knowledge and attitude competences, role-playing and feedback. The work of the team was focused on the identification of unit-specific factors and barriers that could potentially drive the incidence of VAP. These were carried out face-to-face in the different units and taking into account the different nursing shifts, in order to include all of the health-care team involved in patient centered care. Finally, the intervention was developed and carried out by the staff of the quality improvement department of our hospital (four registered nurses and one physician).

Table 1.

Exposure and main outcomes (indicators) components and definitions.

Exposure (experiential learning) component	Description
General actions	Team formation including registered nurses, physicians, pharmacists, physiotherapists and other personnel. General definition and description of components in individual bundles of care (including bundles for central venous catheters, urinary catheters, prevention of ventilator associated pneumonia and hand hygiene).
Tracers	Analyze individual patients exposed to specific risks (e.g. VAP) in order to evaluate the current level of knowledge and its application (clinical action).
Security walks	Walking around the individual unit as a group, understanding the context of care, equipment use and inter-individual behavior.
Jeopardy	Question and answer game, structured around > 200 clinical scenarios.
Video watching	Learning based on prior errors. Videos are compiled by hospital staff based on past real-cases.
Role playing and simulation	Simulation scenarios that aid in the identification of both one’s and peer’s role in increasing the safety culture of the institution.
Security rally	Competition between units. External evaluators measure outcomes of interest. Overall winner and the unit with the greatest improvement are identified.
Feedback and debriefing	Each activity is followed by constructive feedback to aid in the continuous process of quality improvement.
Outcome measure	Numerator	Denominator (at risk)
Adherence to a bundle of care for mechanically ventilated patients	Definition of adherence includes the compliance with all components of the bundle: - Weaning strategy - Oral hygiene - Head of bed at 30° - Cuff pressure between 20 and 30 mmHg	All adult patients admitted to one of the three critical care units undergoing mechanical ventilationa
Ventilator-associated pneumonia (VAP) – incidence rate (VAP episode/days of mechanical ventilation)	The occurrence of VAP was defined by the hospital’s infectious control committee, following the CDC’s National Healthcare Safety Network 2010 guidelines.	All adult patients admitted to one of the three critical care units undergoing mechanical ventilationa

^aPatients undergoing non-invasive ventilation and high flow nasal cannula were excluded.

VAP: ventilator-associated pneumonia.

The main outcomes of interest were process indicators (the adherence to the use of the bundle of care for mechanically ventilated patients) and the incidence rate of VAP (Table 1). For the unit of analysis, it was assumed that all patients admitted to the ICU were at risk if they were mechanically ventilated and hence were included as total patient-days of mechanical ventilation in both the measurement of adequate bundle use and infection rates. For the calculation of VAP incidence rates, we included all patients with a new VAP within the critical unit or within 72 h of discharge (Table 1). VAP was defined following the National Healthcare Safety Network Definition from the CDC.¹⁴ The outcome measurement was not blinded and performed by staff of the infectious control committee of our hospital (including registered nurses and physicians).

Statistical analysis

In order to analyze changes in both outcomes of interest, statistical quality control was used.¹⁵ We used both graphs of proportions and ratio. Specifically, proportion graphs were used for measurements of dichotomous variables and ratio graphs were used for the analysis of incident VAPs.

Moreover, we used simple linear generalized models to estimate the proportion of the adequate use of the bundle of care and the incidence rate of ventilator-associated pneumonia before and after the implementation of the intervention. The sandwich estimator was used to construct robust confidence intervals and the exposure was included as a dichotomous variable in the model. Furthermore, in order to estimate the effect of time and exposure on the use of bundles for mechanically ventilated patients and the incidence rate of VAP, multiple generalized linear models were constructed. These models also used the sandwich estimator to construct robust confidence intervals considering the potential non-independence and correlation of the observations in our study. For these models, time was modeled as a linear spline with a pre-specified knot at the 12th month of follow-up (when the intervention was first initiated).

We used a p value < 0.05 to declare statistical significance and all reported p-values are two-tailed. We used SAS v.9.4 (SAS Institute Inc. Cary, NC) for all our analysis.

Results

Our results reflect both the initial 12 months of follow-up prior to the start of the intervention (period between January and December 2016), and the subsequent months until December 2018. As previously said, the experiential learning intervention was launched on January 2017 in all units simultaneously and was incorporated as a work modality in all of the included ICUs. The average proportion of bundle use for mechanically ventilated patients was 60.8% (95%CI: 56.9–64.7) and 85.6% (95%CI: 81.2–90.1) for the periods before and after the intervention was started, respectively. Moreover, the average pre- and post-intervention VAP incidence rate was 6.11 (95%CI: 5.82–6.40) and 3.55 (95% CI: 2.96–4.14) every 100 days of mechanical ventilation, respectively (Table 2). The average number of ventilator-days/month for the included ICUs was 788.

Table 2.

Prevalent use of the bundle of care for mechanically ventilated patients and ventilator-associated pneumonia (VAP) incidence rate before and after the introduction of the experiential learning strategy.

Main outcome	Before the intervention	After the intervention	p valuea
Adherence to bundle of care – % (95% CI)	60.8 (56.9–64.7)	85.6 (81.2–90.1)	<0.01
VAP incidence rate/1000 days (95% CI)	6.11 (5.82–6.40)	3.55 (2.96–4.14)	<0.01

^ap value based on a generalized linear model including the exposure as a binary variable.

Figure 1 shows the proportion-graphs in relation to the use of bundles in mechanically ventilated adult patients. From month 13 of the study on, and concurrent with the time when the intervention began, the proportion in the adherence to the use of bundles for mechanically ventilated patients started to rise in a distinct fashion. Several significant patterns of variation are observed, such as the presence of seven or more consecutive points above the average of proportions and points outside of the higher control limit. Figure 1 also shows the graph for the VAP incidence rate, where it is also evident that once the intervention started, the incidence rate decreases, marking an instability in the process (e.g. more than six points below the central line).

Figure 1.

Control chart of main outcomes for mechanically ventilated adult patients over time (intervention started January 2017). Upper panel depicts adherence to bundle of care and lower panel the occurrence of ventilator associated pneumonia. CL: control limit; LCL: lower control limit; UCL: upper control limit.

Finally, the monthly change in adherence to the use of bundles during the pre-intervention period was 0.4% (95% CI: −0.3 – 1.2%, p = 0.31) while post intervention it increased to 1.1% (95% CI: 0.2–2.2%, p < 0.01) (Table 3). A similar effect was evident for the VAP incidence rate (Table 4).

Table 3.

Effect of an experiential learning strategy on the adherence to bundles of care for adult critically ill mechanically ventilated patients.

Exposure	Change in the proportion of adherence to bundle of care (95% CI)	p valuea
Time (in months) – before the initiation of the exposure	0.4% (−0.3% – 1.2%)	0.31
Time (in months) – after the initiation of the exposure	1.1% (0.2%–2.2%)	<0.01

^ap value based on a generalized linear model including time as a linear spline with a pre-specified knot at the time of exposure initiation.

CI: confidence interval.

Table 4.

Effect of an experiential learning strategy on the ventilator associated pneumonia incidence rate in critically-ill adult patients.

Exposure	Change in ventilator associated pneumonia (95% CI)	p valuea
Time (in months) – before the exposure	−0.08 (−0.15 – −0.01)	0.02
Time (in months) – after the exposure	−0.09 (−0.18 – −0.01)	0.02

^ap value based on a generalized linear model including time as a linear spline with a pre-specified knot at the time of exposure initiation.

CI: confidence interval.

Discussion

Our study shows a beneficial association between an experiential learning strategy and the adherence to a bundle of care for critically ill, mechanically ventilated adult patients and the subsequent sustained effect on both the use of the bundle and the decrease in the incidence rate of VAP over time. To our knowledge, this is the first study evaluating an experiential learning strategy as a quality of improvement intervention for critically ill adult patients.

Our findings are specially relevant considering that VAP remains as one of the main causes of nosocomial infection in critically ill adult patients.¹⁶ In addition to its magnitude, its importance also lies within its impact on morbidity, mortality and associated costs.¹⁷ Bundles (a selection of simple interventions with a high level of evidence that in addition to being effective, might also enhance each other) have been proposed as a way of decreasing the risk of HCAI in general and VAPs in particular.¹⁸ Training professionals on the importance of bundle compliance and its impact on nosocomial infection remain as the other pillar of such improvement strategy and the importance of a timely feedback in such regard cannot be overstated.¹⁹ Our study reinforces not only the efficacy but also the long-term applicability of an experiential strategy in the use of bundles of care for mechanically ventilated patients and the consequent decrease in the incidence rate of VAP. Moreover, the present work shows, in a novel way, that the combination of gamification, debriefing and simulation techniques – elements that make up the experiential learning strategy and are shaped as a modality centered on the concept of “learning by doing” – impacts both the adherence to safety practices and clinical outcomes among critically ill adult patients. This strategy is seldom used in health-care processes and our study may lead to wider applications in distinct settings and types of HCAI. These strategies can help to narrow the gap between knowledge and clinical action.⁹ It should be noted that in order to achieve this, experiential learning should include four central aspects: (1) people need to be involved in their learning process, (2) learning must be through experiences in the areas in which they work, (3) learning must be relevant to those involved and (4) learning should facilitate, for those who learn, their preparation for an ever-changing and challenging clinical task.²⁰

Several previous papers have shown a decrease of VAP after the application of bundles of care.^21,22 Overall, studies show a decrease in VAP rates and a strong relationship between the latter and the compliance rate with the bundle itself. However, the benefits were usually observed for short periods of time and hence showing that bundles, while being used at the beginning, were not incorporated into the work culture as a daily tool. Specifically, Eom et al.,²³ in a quasi-experimental study like ours, showed that the implementation of the bundle decreased the VAP rate from 4.08 to 1.16 cases per 1000 days of mechanical ventilation. Furthermore, Talbot and Carr²⁴ showed that after the intervention, the rate of VAP decreased significantly at a rate of 0.20/1000 ventilation days per month.²⁴ In line with our results, overall compliance with the bundle improved after the implementation of the intervention. However, our long-term findings have special relevance when we consider that more than 70% of organizations fail to sustain such improvements over time.²⁵ The reasons for this situation include the lack of follow-up with the interventions, the decreased performance of isolated interventions, and the lack of generation of a “culture of change.”²⁶

Several limitations should be taken into account when analyzing the results of our study. First, since it is a quasi-experimental time series study, our results can be explained at least partially by the Hawthorne effect (that is, the response induced by the participants' knowledge that they are being studied).²⁷ However, our long-term follow-up both before and after the intervention and our modelling of the time variable allow for the estimation of the effect even in the face of secular trends and behavioral changes. Moreover, our organization has worked on patient safety certification processes for several years prior to conducting this study, which renders a health care team that is used to being observed on a daily basis. Second, our results may be explained by the regression to the mean,²⁸ which is the tendency of individuals to obtain values closer to the mean of the distribution when the same variable is measured repeatedly over time. However, our prolonged follow-up demonstrates a clear trend both before and after intervention. Finally, as our study was carried out in a single center, our findings may not be extrapolated to settings with different characteristics (for example, specific composition of the medical team or distinct methodologies for the clinical decision making process in daily practice).

On the other hand, our work has several strengths. First, our design maximizes the estimation of causal effects in this given scenario. When the subjects of the experiment are working groups, randomization or the existence of a control group can be problematic, so quasi-experimental studies emerge as a potential alternative. In this scenario, the main difficulty lies in differentiating the specific effects of treatment from those nonspecific effects that result from the lack of comparability of the groups at the beginning and during the study period, which may compromise the internal validity of our results. A strategy used in our study consisted in taking multiple records of the same subject or unit to be investigated over time (in our work, for a period of over 12 months prior to the intervention). Moreover, multiple characteristics of our study increase the robustness of our findings, such as: (1) serially sequenced time points before and after the intervention, (2) observations that are equally spaced and (3) the high overall number of time points. Second, the combination of the use of regression models and statistical quality control allowed us to verify the robustness of our results. Third, the incorporation of the use of statistical quality control allows to propose such tool as a method of analyzing processes within health-care organizations in order to continuously improve their quality of delivered care.²⁹ In the particular case of our study, the application of the control chart with pre-intervention measurements allows to correctly assess the stationary variations, whether they are due to the disease loads linked to time variables (e.g. influenza), or to changes in human resources (e.g. new health personnel income). Fourth, the use of debriefing as part of our intervention is novel and potentially relevant for the implementation of our intervention in other areas. Debriefing is currently the highlight of teaching by simulation and our study may position it as a useful tool in the management of health-care processes.^30,31 Fifth, the use of strategies focused on playfulness fosters commitment and creative thinking, making difficult activities rewarding, motivating and potentially improving the sustainability of the safety practices in the long term. Gamification is a term used to describe the use of game design elements in non-gaming contexts.³² In health care, gamification has been used to influence personal health behavior and to promote the training of professionals and procedural learning,³³ but few studies have used simulation and gamification techniques in the implementation of quality and safety improvement programs.

Conclusion

In conclusion, our work shows that an experiential learning strategy improves the adherence to a bundle of care for critically ill, adult patients undergoing mechanical ventilation and that it leads to a decrease in the incidence rate of VAPs that is sustained over time. Our findings can help in the implementation of quality of improvement programs that positively impact both the daily care and clinically relevant outcomes of critically ill adult patients. Future studies should confirm these findings and evaluate such interventions in a broader population of units and patients.