Failure mode and effects analysis of the universal anaesthesia machine in two tertiary care hospitals in Sierra Leone

Abstract

Background

Anaesthesia care in developed countries involves sophisticated technology and experienced providers. However, advanced machines may be inoperable or fail frequently when placed into the austere medical environment of a developing country. Failure mode and effects analysis (FMEA) is a method for engaging local staff in identifying real or potential breakdowns in processes or work systems and to develop strategies to mitigate risks.

Methods

Nurse anaesthetists from the two tertiary care hospitals in Freetown, Sierra Leone, participated in three sessions moderated by a human factors specialist and an anaesthesiologist. Sessions were audio recorded, and group discussion graphically mapped by the session facilitator for analysis and commentary. These sessions sought to identify potential barriers to implementing an anaesthesia machine designed for austere medical environments—the universal anaesthesia machine (UAM)—and also engaging local nurse anaesthetists in identifying potential solutions to these barriers.

Results

Participating Sierra Leonean clinicians identified five main categories of failure modes (resource availability, environmental issues, staff knowledge and attitudes, and workload and staffing issues) and four categories of mitigation strategies (resource management plans, engaging and educating stakeholders, peer support for new machine use, and collectively advocating for needed resources).

Conclusions

We identified factors that may limit the impact of a UAM and devised likely effective strategies for mitigating those risks.

Editor's key points.

• Staff expertise and maintenance of medical devices are likely to be limited in developed countries.

• Human factors analysis was used to address individual and organizational factors that limit the safe use of a universal anaesthesia machine.

• Engagement of local providers was a crucial component of the process.

Safe anaesthesia care in developed countries relies on sophisticated technology and experienced providers. Advanced anaesthesia machines enable high degrees of safety in the contexts of use for which they were designed. However, these same machines may be inoperable or fail frequently when placed into the austere medical environment of a developing country.^1,2 Unfavourable environmental conditions, unreliable electrical grids, lack of access to maintenance and repair resources, insufficient training of anaesthesia providers, and scarcity of compressed gas and proper drugs are all factors that can result in machine failure or improper/unsafe use, potentially compromising patient safety.^3–5 The World Health Organization recently has focused on the persistent lack of basic medical devices that can function correctly and reliably under these conditions and the potential contribution that adequate equipment could make to effective and safe medical care.⁶ To meet this growing need, device designers must know the performance requirements for the devices and the constraints under which they will be operating. Thus, designers require a systematic understanding of austere medical environments, but almost nothing has been published on the subject.

Failure mode and effects analysis (FMEA) is a method for identifying likely breakdowns in processes or work systems and developing strategies to mitigate risks (Table 1). It originated in the aviation domain and has been applied to a wide range of safety issues in healthcare.^7,8 FMEA has not yet been reported in an austere medical environment, but attributes of the process make it potentially well suited for application in such settings. Specifically, FMEA engages local providers in the process of identifying a wide range of system-related factors that can impact performance.

Table 1.

Overview of the general steps in the FMEA process, data from DeRosier and colleagues.⁸ UAM, universal anaesthesia machine

General FMEA step	Application in the current project
Define the goals and form a team	The goal of the session was to identify any issues that may interfere with the maintenance and repair of the UAM and to develop strategies for mitigating those risks. The team was composed of Sierra Leonean nurse anaesthetists, a human factors professional, an anaesthesiologist, and two physicians with public health backgrounds
Conduct a task analysis	The task analysis was performed as a part of the session. The focus was kept narrow (use processes of the UAM) to address project goals and make the most out of the limited time available
Brainstorm potential failure modes	The group reviewed the processes outlined, and nurse anaesthetists were prompted to identify failure modes by asking questions such as: What makes performing this step difficult or impossible? Why would things happen differently than we have outlined here?
List potential effects of each failure mode	Consequences of failure modes were discussed, but many were immediately apparent to the entire team given the relatively simple processes identified
Assign severity, occurrence, and detectability ratings; derive risk index	Risks were rated qualitatively (e.g. does this happen frequently or infrequently?). A formal risk index was not calculated because the intent was to target a relatively simple process and explore it in detail
Brainstorm actions to eliminate risks	Session facilitators prompted nurse anaesthetists to think about solutions to the risks identified, whether or not they had direct control over the primary causal factors
Assign effectiveness ratings	Formal feasibility and effectiveness ratings were not performed. Instead, the group focused on factors that were controlled locally and those that were under less direct control of local staff
Revise risk priorities	This step of a traditional FMEA was not carried out because a formal risk index was not calculated initially
Implement changes	Ongoing

Starting in November 2011, the Safe Anaesthesia & Surgery in Sierra Leone Initiative (SASSL), a collaborative effort between the Sierra Leone Ministry of Health and Sanitation and the Johns Hopkins University Austere Anaesthesia Health Outcomes Research Group, implemented the use of two universal anaesthesia machines (UAMs) in Freetown, Sierra Leone (one in each of the major tertiary-care hospitals). The UAMs were donated by their manufacturer (Gradian Health Systems, LLC, NY, USA). Gradian Health Systems, LLC funded this evaluation of the implementation of UAMs in Sierra Leone, but was not involved in the design, data collection, or analysis of the study reported in this article. FMEA was chosen as a component of the implementation plan to enable inclusion of frontline providers' insights into the broader work system barriers that may influence safe and effective use of the UAM. In this study, we demonstrate the feasibility and utility of applying FMEA as one component of a device implementation process. Additionally, we extract themes from analysis of the sessions that contribute to the knowledge base of work system pressures faced by workers in austere medical environments.

Methods

SASSL assessed the impact of the UAMs on the safety and quality of perioperative medicine. FMEA was included as one component of this evaluation. This study was approved by the institutional review boards of the Johns Hopkins University School of Medicine and the Sierra Leone Ministry of Health and Sanitation.

Participants and setting

Fourteen nurse anaesthetists from two tertiary-care government hospitals in Freetown, Sierra Leone, participated in three separate FMEA sessions of 90 min each. The sessions were facilitated by a human factors psychologist and an attending anaesthesiologist from the Armstrong Institute for Patient Safety and Quality and the Johns Hopkins School of Medicine, respectively. Ten nurse anaesthetists participated in two sessions at Princess Christian Maternity Hospital, a referral maternity hospital, and four nurse anaesthetists participated in one session at Connaught Hospital, a referral and trauma hospital. These 14 anaesthesia care providers represent approximately half of the nurse anaesthetists employed by the two hospitals.

FMEA sessions were part of a larger implementation process that included lectures, computer-based training, and in situ simulation exercises in which providers used the UAM at their local work areas. The FMEA sessions took place ∼3 months after the initial introduction of the UAM into the hospitals as part of an ongoing training and implementation support process.

Procedure

Each FMEA session lasted ∼90 min and proceeded as follows. The session facilitator (M.A.R.) welcomed participants and gave a brief introduction to the purpose of the session. Participants then completed an informed consent form. Next, the facilitator presented an overview of the FMEA process and an example of FMEA applied to patient monitoring and documentation on the wards in Sierra Leonean hospitals. The study team then facilitated a dialogue with the nurse anaesthetists, who mapped the current process of UAM use, identified failure modes or barriers to use, and generated solutions to overcome the identified barriers. The study team consisted of a human factors professional (M.A.R.), an anaesthesiologist (J.B.S.), and two physicians with public health backgrounds (A.M.C., O.U.O.).

Data collection and analysis

Sessions were audio recorded, and the session facilitator mapped the group discussion graphically for analysis and commentary. Study team members annotated the commentary with word processing software and stored the information on the hard drive of a laptop computer used during the FMEA sessions. Photographs were taken of all visual artifacts and stored on the same laptop computer. These data sources were combined to summarize the common ideas that emerged across the FMEA sessions. This process consisted of grouping similar ideas across sessions to reduce redundancy in the presentation of results. In general, there was a high degree of consistency in barriers identified and in solutions generated. We note in the results when a specific barrier or solution surfaced in only one group.

Results

Process description

Figure 1 illustrates the process of use for the UAM defined by participating clinicians. Seven main process steps were identified: daily machine check; pre-case preparation; anaesthetic induction, maintenance, and emergence; intraoperative monitoring and patient care handoff(s); and machine maintenance/preparation between surgical cases.

Fig 1.

Process description of UAM use generated by Sierra Leonean Nurse Anaesthetists.

Failure modes identified

Table 2 provides a list of the failure modes identified for each step in the process. These can be grouped into four principal categories. First, resource availability was a recurring failure mode. This category included access to needed drugs (e.g. neuromuscular blocking agents, anaesthetic reversal agents, inhalation agents) and supporting devices/equipment (e.g. suction, non-invasive arterial pressure monitor, pulse oximeter probes, tracheal tube stylets). The second major concern pertained to environmental issues, such as frequent power failures, which caused monitoring devices and oxygen concentrators to fail. Thirdly, workload and staffing issues were perceived as barriers to efficiency and quality care. Specifically, the UAM requires manual ventilation of patients during the administration of anaesthesia. Staff indicated that for cases of long duration, an individual could become fatigued. Similarly, nurse anaesthetists perceived that performing manual ventilation made it difficult to perform other tasks (e.g. monitoring, documentation) and that two anaesthetists were required for portions of a case when only one would be needed if a functional ventilator was present (which it was not at either location). Fourthly, staff knowledge and attitudes surrounding the machine were perceived as large barriers to use. Specifically, at the maternity hospital, surgeons reportedly pressured nurse anaesthetists not to use general anaesthetics when an acceptable alternative because they perceived the process to take too long. This barrier only surfaced at the maternity hospital, and is discussed in more detail below. Because of the current perioperative standard of care and the experience of anaesthesia care providers, some nurse anaesthetists preferred to use spinal anaesthesia. They considered managing an awake patient to be less complicated than providing general anaesthesia care. Additionally, several examples of deficits in nurse anaesthetists' skill, knowledge, or confidence were provided in the sessions as potential failure modes.

Table 2.

Failure modes identified. AP, arterial pressure; GA, general anaesthesia; NA, nurse anaesthetist; UAM, universal anaesthesia machine

Relevant process steps (Fig. 1)	Failure modes identified
1f, g, k, l; 2d; 3a, b, d–f, h, i; 4b–e; 5a–e; 7d, e	Resource availability
	Equipment No access to a reliably functioning suction device AP machine not working Pulse oximetry probes broken Stylet missing/broken
	Drugs Absence of acting neuromuscular blocking agents Paralytic medications expired or insufficient Absence of neuromuscular blocking agent reversal medications Inhalation agents (halothane is inconsistently available and may have expired by the time of distribution to the hospitals; isoflurane is rarely available)
1a–c, h; 2a–c, e; 3b; 4b	Environmental
	Power failures cause the oxygen concentrator to shut down  Power failures cause cardiac monitors to turn off
4a–e	Workload and staffing
	Fatigue from manual ventilation with bellows during long cases  When manually ventilating a patient, they currently use two anaesthesia providers (one to ventilate, the other to monitor/document/perform needed tasks). At times this was perceived to impact their ability to perform the number of operations desired if they were to have a functioning ventilator
All steps, as these failure modes impact the decision to use the UAM and appropriate execution of tasks	Staff knowledge and attitudes
	Surgeon attitudes and perceptions regarding the risk of GA (believed that GA took too much time and would pressure NAs not to use the machine)  NA attitudes (lack of confidence with the new machines/processes; NAs believing it was safer to have patients awake so they could be responsive)  NAs lack familiarity with how to use the new machine (need for skill building)  NA fear of injuring patient from barotrauma when oxygen tanks were connected to the UAM

Solutions generated

As detailed in Table 3, participants generated four categories of solutions to the failure modes described above. First, resource management strategies were developed to make the most of the existing devices. Specifically, the multiparameter intraoperative patient monitor for the UAM (which measures heart rate, arterial pressure, and pulse oximetry) has an internal backup battery. However, during a recent case, a nurse anaesthetist experienced a power failure, and the patient monitor did not function. It was discovered that the monitor had not been plugged into the machine or a wall outlet to charge when it was not in use. A plan for ensuring that backup batteries for the intraoperative patient monitor remain charged at all times was implemented. Secondly, participants devised a prioritized list of required items (suction and drugs) and a plan for advocating collectively to hospital management and other external sources of required equipment and supplies. Thirdly, they proposed a plan to engage and educate stakeholders to address the identified gaps in staff knowledge and attitudes. This plan included informal one-on-one conversations between nurse anaesthetists and surgeons and formal multidisciplinary meetings to educate and advocate for situations in which general anaesthesia (and use of the UAM) is appropriate and recommended for patient care. Specifically, engagement of surgeons at the maternity hospital was chosen as the best strategy to build awareness of the risks associated with local and general anaesthesia for different types of cases. For most procedures at the maternity hospital, general anaesthesia is not appropriate. The majority of general anaesthesia cases at the maternity hospital are for emergent Caesarean section. However, there is a small but significant per cent of elective cases as well (fibroids, ectopic, abscesses, and cysts). Lastly, they discussed peer support and teamwork strategies for overcoming machine fatigue and balancing provider tasks when manually ventilating a patient and delivering inhaled anaesthetics.

Table 3.

Solutions generated. UAM, universal anaesthesia machine

Failure modes identified	Solutions generated
Resource availability; environmental	Resource management strategies
	Implement process and checks to ensure that the UAM and all monitors and backups are charged at all times to maximize the amount of monitoring time available during a power failure  Build a closer collaboration with the newly formed biomedical engineering department to ensure that they know when suctioning devices are malfunctioning and that suctioning devices are critical for patient care
Resource availability	Advocate for additional resources
	Work for anaesthesia representation within the government medical procurement group for a better understanding of anaesthetic drug and equipment needs  Write a letter as a collective group (14 local nurse anaesthetists) to hospital management to educate them on the need for drugs and equipment (priority items were suctioning devices and paralytic and reversal drugs)  Write a letter as a collective group to LifeBox to advocate for additional monitoring devices  Obtain a ventilator for the UAM through similar advocacy approaches
Staff knowledge and attitudes	Engage and educate stakeholders
	Start an ongoing dialogue with surgeons that incorporates one-on-one conversations and formal meetings to educate them about risk and appropriate anaesthetic care
Workload and staffing	Peer support for new machine use
	Help one another work through cases with the new machine  Teamwork for long cases—shift roles and actively ask for assistance when fatigued

Discussion

This study indicates that FMEA was a practical strategy in the austere environment of two, tertiary-care government hospitals in Sierra Leone and likely is a useful methodology for other low-resource settings. In the relatively brief FMEA sessions, nurse anaesthetists identified a broad array of factors that may limit the impact of the UAM and devised innovative, and likely effective, strategies for mitigating those risks. The process was easily understood by the session participants and actively engaged stakeholders involved in front-line care of patients. Although alternative strategies are available for analysis of risks or hazards, FMEA is a potentially simple and straightforward method to engage local providers in process improvement. This study did not use local facilitators, but in-country subject-matter experts likely can be trained to use the FMEA methodology to examine other clinical care processes. Previous literature has identified a failure to consider local constraints and conditions as an impediment to the design and implementation of devices for austere medical settings.^1–6 As part of an implementation strategy, FMEA can help to fill this gap by generating knowledge about austere medical contexts and how they interact with device use, maintenance, and repair. Interestingly, most of the failure modes identified by the nurse anaesthetists did not deal with the machine itself, but with broader socio-technical systems issues, primarily resource availability (drugs, ancillary devices) and the knowledge and attitudes of stakeholders in surgical services and hospital administration.

Head nurse anaesthetists at each participating facility were contacted ∼3 months after the FMEA sessions were conducted to gather information on the implementation and effectiveness of strategies identified in the sessions during this study. Neither hospital secured an additional battery due to lack of funds, but both had new processes in place to ensure that the monitor's batteries were fully charged before each case. Both sites had actively engaged their hospital leadership in ongoing efforts to secure reliable supply lines for needed drugs and equipment. At the referral and trauma hospital, surgeons had been receptive to engagement, cooperative in implementation, and satisfied with the machine. Most cases conducted at the maternity hospital are performed under regional anaesthesia, but the presence of the UAM reportedly increased the ability and confidence of anaesthetist staff to provide general anaesthesia when required. At both hospitals, nurse anaesthetists have adapted to the increased burden associated with continuous ventilation of the patient using the UAM and instituted steps to reduce fatigue during use, such as having more than one person manning every case so that fatigued persons can be relieved intermittently.

This follow-up reveals mixed results for the impact of FMEA on practice. However, solutions generated with a high degree of local control (i.e. things local anaesthesia providers had direct influence on) were implemented effectively (i.e. ensuring charged monitor battery, engaging and educating other stakeholders, developing new staffing models, and teamwork patterns) while those requiring larger capital or supply chain management were less effective (i.e. obtaining back-up batteries, the ongoing process of coordinating with central government supply offices).

Although the sample size in this study is relatively small, it does represent a significant proportion of the total number of nurse anaesthetists who work in these two hospitals, including night and weekend staff. As these two hospitals are located in the same country, it is unclear whether the findings can be generalized to other settings. However, our findings align well with and expand upon barriers previously identified in healthcare systems of developed countries^7,8 and support the limited literature on austere medical environments.

Authors’ contributions

M.A.R.: study concept and design, data collection and analysis, drafting and critical review, and approval. J.B.S.: study design, data collection, critical review, and approval. E.V.J.: study design, critical review, and approval. R.K.: study design, critical review, and approval. A.M.C.: data collection and analysis, critical review, and approval. O.U.O.: data collection and analysis, critical review, and approval. M.K.M.: data collection and analysis, critical review, and approval. M.K.: participant recruitment, data collection, critical review, and approval. B.H.L.: study concept and design, drafting and critical review, and approval.

Declaration of interest

None declared.

Funding

This work was supported by a grant from Gradian Health Systems, Inc.; IO #90047879, PI to B.H.L. The funds for the Open Access are being provided by Gradian Health Systems, LLC, 160 5th Avenue, 7th Floor, New York, NY 10010.