Improved FMEA Methods for Proactive Health Care Risk Assessment of the Effectiveness and Efficiency of COVID-19 Remote Patient Telemonitoring

Abstract

The COVID-19 pandemic exposed the need to more effectively harness and leverage digital tools and technology for remote patient monitoring (RPM). RPM gained great popularity given the need to provide effective, safe, efficient, and remote patient care. RPM is based on noninvasive digital technologies aimed at improving the safety and efficiency of health care delivery. We report on an RPM program in which 200 COVID-19 patients were followed remotely to evaluate the effectiveness in treating and monitoring patients in home settings. We analyzed the inherent risks using mixed methods, including failure mode and effect analysis, a prospective, team-based risk management methodology structured to identify high-risk process system failures before they occur in telemonitoring of remote patients. The RPM saved lives and improved decision-making during the pandemic and helped prevent the health system’s collapse. The failure mode and effect analysis-based assessment offers important insights and considerations for evaluating future RPM implementation and direction. RPM solutions are technically feasible, staff friendly, and can achieve high adherence rates. Rigorous and ongoing evaluation of devices and platforms is essential to clarifying their value and guiding national health and insurance health coverage decisions and adoption programs.

Introduction

The COVID-19 pandemic exposed the urgent need to harness and leverage digital tools and technology for remote patient monitoring (RPM). RPM involves the monitoring of biometrics outside the hospital with the transmission of data to clinicians. Digital technologies provide an opportunity to effectively manage patients, while limiting the use of hospital facilities to acute or complex interventions. RPM will be limited in its impact until engagement and adherence can be maximized.¹ It is necessary to identify critical issues that arise with RPM use and design strategies to address them quickly and effectively, before they lead to patient and staff repercussions.

Digital health technologies offer an opportunity to envision a new model of care in which the need to offer care remotely is an opportunity to use time more efficiently and reduce the risks incurred by physical proximity.^2,3 However, studies have highlighted that to successfully implement telemedicine in a sustainable manner, it is necessary to identify the technical and organizational barriers that arise and to design strategies capable of addressing them quickly.⁴

Telemedicine involves the secure transmission of medical information and data in a variety of forms: text, sound, images, or other forms necessary for the prevention, diagnosis, treatment, and subsequent follow-up of patients.² Over the past decade, these solutions have been coupled with the ability to reliably transmit data to central collection systems where clinicians can view the data continuously.⁵ Data can be collected through devices that harbor cellular chips and with Bluetooth capacity for direct transmission of data. There are many benefits of RPM such as ease of access, ability to provide higher levels of care to more patients with less risk to caregivers, lower costs, and greater efficiency.⁶

The European Commission funded the development of robust telemedicine tools^7,8 despite many aspects yet to be clarified and validated (ie, medical liability, human factors, usability, privacy and security, real efficacy for health care improvement). The aim of this study was to assess critical issues related to widespread implementation of telemonitoring with a special focus on remote monitoring of COVID-19 patients at home to avoid hospital overload and potential further contagion.

Methods

Study Design

Two hundred COVID-19 patients were monitored remotely from March 2020 to July 2021 in Tuscany, Italy. The intervention consisted of 2 components: telephone coaching and home-telemonitoring of oxygen saturation, blood pressure, heart rate, and other symptoms using a device which transferred patient data in real-time to the Tuscany 112 Operations Control Center (OCC) Emergency Medical Service through a Bluetooth device.

A failure mode and effects analysis (FMEA) methodology was used to identify the risks of RPM failure modes, effects, and causes, and assess the safety of RPM. Figure 1 describes an overview of the study and monitoring process: installation of the device and the app, alignment of the device with the OCC, receipt of the data, and display of the data to the nursing physician staff. The OCC made decisions to have patients examined by a primary care physician, send a nurse from the emergency department, or advise unstable and sick patients to come directly to the emergency department (Figure 1).

Figure 1.

Overview of the study and monitoring process.

Setting and Participants

The population of Italy is 60 million people. Each of the 20 Italian regional governments is independently responsible for overseeing planning, delivery, and management all health services in their local Health Trusts. The Southeastern Tuscany area has a catchment area of 818,934 people and is managed by one local Health Trust. The hospital resources are concentrated in the 3 centers of Siena, Arezzo, and Grosseto. The hospitals of Southeast Tuscany provide 211 beds, of which 23 are intensive care beds and 72 beds support intermediate care organized in a hub-and-spoke model. The local Health Trusts have an important territorial extension, compared to the other health agencies in Tuscany, with a low population density due to the geographical characteristics of the area and their elderly population with their unique comorbid conditions.

The COVID-19 pandemic in early 2020 stressed the regional emergency departments, leading the Health Authority to rapidly adopt telemonitoring of COVID-19 patients. Patients were monitored remotely at home using arm bracelets, thus avoiding crowding of the emergency departments and collapse of the hospital services as occurred in the adjacent Lombardy province in March 2020.⁹

Patient Population

Two hundred patients in Tuscany were eligible to participate based on the following qualifications: (1) a positive COVID-19 test; (2) one or more diseases that may complicate COVID (ie, diabetes mellitus, high blood pressure, bronchial chronic obstructive pulmonary disease, ischemic heart disease, obesity, cancer); (3) the presence of other chronic diseases, as long as the patient’s health condition was not debilitating; or (4) signs of pregnancy. Participants had to agree to comply with the technology and provide written informed consent in Italian.

Patients were excluded from the study if they did not meet the parameters described above. The telemonitoring was activated only for users who developed a complex clinical course and who could manage the RPM device provided, independently or with the assistance of a family member. In all other cases (eg, young patients, asymptomatic, absence of pathologies, or technological incompatibility), the management of patients affected by COVID-19 was done through traditional hospital channels or through their general practitioner.

Intervention

The intervention consisted of 2 components: regularly scheduled telephone coaching and home telemonitoring of oxygen saturation, blood pressure, heart rate, and symptoms. The patients were monitored with the device for a time not exceeding 10 days from the outcome of their positive COVID-19 test.¹⁰

The electronic equipment (Bluetooth enabled; Masimo’s Radius Tº) consisted of a wireless wearable device which continuously measures and tracks a patient’s body temperature, heart rate, blood pressure, and oxygen saturation. The data was sent through the Bluetooth connection to an application installed on the patient’s cell phone. The app was connected to the OCC, located in Arezzo. This allowed health care professionals to telemonitor the patient’s status and prioritize safety events in near real-time.

Perturbations of monitored vital signs (ie, oxygen saturation, SpO₂ <94%, and heat rate <60 and >100 beats at rest, and BP alteration) generated an alarm that was displayed to the OCC medical staff. The operators evaluated whether to send a physician to the patient’s home, call the patient by telephone to perform a stress test or send the patient directly to a dedicated COVID-19 hospital ward.

Outcome Measures

The following outcome measures were used to evaluate the effectiveness and efficacy of telemonitoring in the management of the study’s noncritical COVID-19 patients:

1. Average days of hospitalization, in cases of monitored patients admitted to the hospital, compared to the average hospitalization duration of COVID-19 patients in Tuscany.

2. Rate of monitored patients admitted to the hospital who had a new hospitalization 30 days after hospital discharge.

3. Rate of patients who directly accessed the emergency department of the total monitored patients.

4. Estimated costs savings by reduced hospital length of stay (LOS) using telemonitoring for the Southeast Health Authority.

FMEA Risk Analysis

The National Academy of Medicine recommends conducting prospective risk analysis studies to bolster baseline epidemiological studies.¹¹ The FMEA is a reliable mixed methodology that can identify potential failures before harmful events occur.^12,13 FMEA offers a proactive approach to fault detection in contrast to incident analysis and root cause analysis, which are performed retrospectively. It is a popular technique in industries such as aviation, aerospace, nuclear power, and car manufacturing.¹⁴ Lately, it has been used in many health care specialties including in chemotherapy,^15–18 pediatric anesthesia,¹⁹ pharmacy, and in various settings such as inpatient settings, intensive care units,^20,21 and community clinics.²² The team could not find reports on the use of FMEA to assess the safety and reliability of telemedicine and remote monitoring of home-based patients.

FMEA is a systematic, step-by-step methodology that begins with the selection of a clearly defined process and the assembly of a multidisciplinary team. The method involves a quantitative analysis of failures by creating a series of links between potential failures (Failure Modes), the impact on the mission (Effects), and the causes of the failure (Analysis). We mapped the processes and sub processes of the selected COVID care process by querying the team’s collective knowledge and by focusing on key components of the process. After mapping the process, the team brainstormed and identified potential failure modes for each sub process. The team identified the effects, causes of potential failure modes, and entered the results into a spreadsheet. Professional knowledge and personal experience of team members about the effects, causes of potential failure modes, and literature examples were useful in this process mapping. The team prioritized the potential failure modes, considering the severity, frequency, and detectability of failure modes. Finally, the team proposed a redesign of the clinical process to avoid or minimize the failures. This type of analysis can proactively assess what could go wrong (failure modes) and what the possible consequences could be (effects analysis). Adopting this method for each phase of the study allowed the authors to define the:

1. Possible causes that give rise to each failure mode;

2. Possible consequences of failure; and

3. Calculate the risk priority index (RPI) or the index of necessity of intervention, which allows for prioritization of the various findings based on severity and probability.

The FMEA 5 steps were conducted as follows:

Step 1. Assembling a team to conduct FMEA

Step 2. Mapping the process and sub processes of dispensing

Step 3. Brainstorming to identify potential failure modes in each subprocess of dispensing, as well as their effects and causes

Step 4. Giving a numerical value (scoring) for the severity, frequency, and detectability of each failure mode and calculating the risk priority number (RPN)

Step 5. Suggesting corrective actions for selected failure modes

Ethics

The study was submitted to the ethics committee of Guglielmo Marconi University and was exempted from full review as only anonymous administrative data were evaluated.

Results

Two hundred patients were evaluated between March 2020 and July 2021 after giving consent to participate in the study. The median age of participants was 57.5 years, and 33% were female. Only 12% (24/200) of the telemonitored patients required hospitalization. Hospitalized patients were all admitted to a COVID-19 dedicated hospital ward without going through the emergency department. This maintained continuity of care for patients with other conditions.

None of the 200 patients in the study were readmitted to the hospital within 30 days after discharge from the hospital but unfortunately one patient died. The Tuscany region mortality of Covid positive patients during the same period was 0.87%, and in this study, it was 0.005%.²² All patients identified by the Regional Health Agency (ARS) of Tuscany as Category A, that is, “patients admitted in non-critical condition to medical wards,” were managed at home.²² Data from the analysis were compared with data from the Tuscany region were available on the ARS Tuscany website,²² where regional statistics on various indicators are monitored. Table 1 presents the main characteristics of the population analyzed.

Table 1.

Study Population Characteristics

Average age of the population	57.46 y
Gender representation	2/3 men (137/200) 1/3 women (66/200)
Re-evaluation by OCC after device installation	10% (20/200)
Hospital LOS shorter than 7 d	12 % (24/200)
Hospital LOS longer than 30 d	4 % (8/200)
Deaths	1 patient

FMEA Steps

Step 1. Assembling a Team to Conduct the FMEA

The FMEA was conducted (M.M., G.S., S.P.), in collaboration with the team of the Emergency Medical Service of the Southeast Healthcare Trust (a doctor and a nurse specialist in emergency medicine) to verify criticalities or improvement actions to be implemented. The multidisciplinary analysis team was composed of a doctor (R.T.) with competence in clinical risk management, a researcher in engineering management with competences on FMEA methodology (C.P.), and an engineer with medical device expertise (M.R.). The group documented the clinical, technological, and organizational aspects within the process and the causes of possible process criticalities.²³

Step 2. Mapping the Process and Subprocesses of Dispensing

The first necessary step after establishing the analysis team, was to map the work process being analyzed. An expert in collaboration and qualitative methods conducted the process mapping with the team (C.P.). Through the individual narratives of operational team members and guided questions, it was possible to reconstruct the process of receiving and managing patients. We observed how physicians worked in the operations center, observed the tools they used, read their reports, and listened to their phone calls with patients. Because of the health isolation imposed on COVID-19 patients, the authors did not observe the physicians’ work at the patients’ homes. In this way, the process map was constructed. Figure 2 shows the flow chart in which all the phases of the process analyzed by FMEA are described. Each step describes the operations to be performed from the moment of COVID-19 positivity until the end of a patient’s remote monitoring, which coincides with the end of the device battery. The wearable device used for monitoring was designed to have a usable battery for up to 10 days. At the end of this period, the device ended its function and turns off¹⁰ (Figure 2).

Figure 2.

Study monitoring flow chart.

Step 3. Brainstorming to Identify Potential Failures of Each Subprocess, Effects, and Causes

The project team met face to face several times to define the steps and substeps of the monitoring process and identify any failures, consequences, and their causes. Since this study was about managing sick COVID patients, it was not possible to involve patients directly, for reasons of protection and isolation. The critical issues highlighted by the patients were collected by the practitioners and reported and discussed within the research group.

Step 4. Assigning a Numerical Value (Scoring) for Severity, Frequency, and Detectability of Each Failure Mode and Calculating the RPN

After mapping the work process, the team brainstormed to identify potential failure/fault modes for each phase and sub phase. Each failure mode was assigned a score from 1 to 10, as required by FMEA methodology, with respect to:

1. severity of the effects the failure would cause on the process or patient;

2. probability—the frequency with which the failure would occur; and

3. detectability—the possibility of detecting the failure before it generates an adverse event.

The product of these indicators allows the calculation of the RPN which can then be used to identify the riskiest steps, that is, those with a higher RPN. The severity and probability of occurrence and detectability scores were assigned by each team member based on their extensive clinical experience. The brainstorming session was used to converge on a score that is representative and agreed upon by all participants.

Table 2 shows the main criticalities highlighted by the FMEA in order of criticality. The steps that were found to be most at risk, that is, those with the highest RPN, are the steps related to the technology environment and device management. These steps have a major impact on the process because they can compromise the ability to reliably monitor the patient. Other critical issues highlighted include the inability for clinicians to consult the patient’s electronic medical record due to privacy issues. This implies that the clinical team relies on the history collected during patient interviews (albeit not always comprehensive and complete) without the possibility of viewing the health data directly in the electronic medical record. In addition, at the end of the monitoring period, the data collected by the device cannot be transferred or shared with other clinicians due to strict privacy laws (Table 2).

Table 2.

Phases of the Analyzed Process and Related RPN

	Process phases	RPN	Main criticalities	Improvement action	Result
1	Device installation	164.9	Impossible to activate the device	Development of a practical guide for device alignment with the operations center	Abatement of errors in device alignment practice with the operations center
2	Remote monitoring feasibility	137.9	Incorrect therapeutic program or technology inadequacy	Extend the technological compatibility of the monitoring system to more devices.	increase in the number of patients enrolled for remote monitoring
3	Patient enrollment	125.4	Inability to correctly assess the patient’s clinical/therapeutic situation	Development of a checklist to guide the patient interview	reduction of the number of errors in the therapeutic framework
4	Intake at 112 Operations Center	69	Impossible to evaluate trends from the remote terminal	Develop continuity systems so that the system is never blocked	Reduction in the number of interruptions on monitoring
5	Data download from portal	65.9	The medical record was not transferred to the General Practitioner	Provide for the possibility of downloading the data recorded on the portal either in computer or paper form to be transferred to patients general practitioners	Provide patients with information that they can share with doctors
6	Trend evaluation	61.7	Portal blocked, impossible to evaluate the user	Develop continuity systems so that the system is never blocked	Reduction in the number of interruptions on monitoring
7	End of monitoring	49.7	The user cannot be contacted to inform him of the end of monitoring	Provide a message for the system to conclude monitoring	Inform patients of the end of monitoring

Step 5. Suggesting Corrective Actions for Prioritized Failure Modes

Several critical points emerged from the analysis, some of which can be tackled with minimum investment, while others must be considered constraints, as they imply significant changes needed in authorizations and regulatory actions. According to the assigned priority, we intervened with improvement actions based on the most critical phases and those with a high error rate:

1. Operations center app alignment. The development of a checklist to be delivered to USCA (Special Continuing Care Units) operators with the tasks to be performed in sequence. The checklist sequence must be checked off.

2. Patient enrollment: clinical evaluation. A checklist should be developed and given to OCC clinicians. The checklist needs to show the range of values to guide inexperienced operators.

3. Device Installation. The development of a checklist delivered to frontline professionals with the operations to be performed in sequence. The development of tutorials for the installation and alignment of the app. The tutorials were present in the device package but could be made available on other mediums within the operator’s reach. If they had access to tablets, the tutorials would be uploaded directly to the tablets.

4. Integrate with the electronic medical record. At the end of the 10-day patient monitoring period, the patient’s data were not transmitted to the GP’s, leading to a lack of communication between the OCC and the patient’s GP.

Discussion

This study is one of the largest studies of remote patient telemonitoring of sick COVID-19 patients at home. The study was designed to determine the effectiveness of the RPM intervention using a broad population of patients with COVID-19 in Italy, consistent with real world practice. The study underscores the criticality of specific RPM processes. The criticalities detected through the FMEA analysis are characteristic of telemonitoring and must be taken into account in future planned rollouts and uptake of similar RPM device programs.

The application of FMEA is very useful in the emerging form of health care delivery, namely, telemonitoring of remote patients. FMEA adoption is especially useful in the presence of costly implementation, as it proactively evaluates the cost-benefit ratio of RPM device uptake. The assessment tools are based on risk quantification, such as those used in this case, and also allow for the identification of corrective actions to ensure optimal quality of care as well as error prediction, both systemic and those related to human factors. This mechanism has the advantage of defining critical areas in advance, making it possible to identify and implement the adaptive interventions before problems and harm arise.²⁴

In this study, the team also made an estimate of the impact of expanded implementation of telemedicine could have, both on the quality of care and on health care costs. Unfortunately, the emergency during which the work was undertaken did not allow exhaustive collection of data. We propose preliminary analyses, carried out through comparison indicators with data from the Tuscany region during the same period. The most important hospital indicator (taken as a reference in the analysis) that impacts increased hospitalization costs is the patient’s length of stay (LOS). The team found that the hospital LOS was reduced by 5 days as compared to the average hospital stay for the region. The average LOS of telemonitored patients was 7 days, whereas the average LOS rates in the Tuscany region for equally sick COVID-19 patients was 12 days.²² While it was not possible to do a detailed financial analysis of the cost savings associated with the adoption of RPM, the costs of a patient in low intensity care in Tuscany is estimated at €427.77 per day²⁵ (Table 3). With the use of RPM, the system saved an estimated €2000 per hospitalized patient, since the implementation costs for each mobile health device was €200 per device for 10 days. It is important to consider that, in addition to the direct costs of hospitalization, one must also consider further costs related to the costs of personal protective equipment (PPE) essential for the safety of health care personnel (Tables 3 and 4).^26,27

Table 3.

Cost Per Day of Hospitalization

Economic resources required
Low-intensity care	427.77 €
Medium-intensity care	582.28 €
High-intensity care	1278.50 €

Table 4.

Estimate Costs of Equipment and Supplies

		Medical ward	Subintensive care ward	Intensive care unit	Media
Personal protective equipment	Cost per day of hospitalization	8.85€	7.36€	19.78 €	12 €
	% of hospitalization costs	2.35 €	1.18€	1.80 €	1.81€
Equipment	Cost per day of hospitalization	0.03 €	0.15 €	0.31 €	0.16 €
	% of hospitalization costs	0.05 €	0.03 €	0.02 €	0.03€

This exploratory study has several limitations by its nature. First, the FMEA methodology results are closely linked to the experience of the practitioners involved in the analysis itself, so they might vary with different practitioners. Second, it would have been optimal to include more patient and family input in the process mapping, but due to the dire COVID-19 situation and the sickness of the patients. The authors were only able to include some input from patients, coupled with detailed ethnographic observations of providers at the operations center, where the team observed the tools providers used, and listened to their phone calls with patients. Third, the RPN index is a qualitative index, as it is the result of the personal assessments of experienced operators involved in the analysis. For this reason, it is necessary to compare the data obtained with data in the literature of similar cases. There are known limitations well described in the literature.²⁷ Despite this, the system criticalities detected through the analysis are characteristic of telemonitoring and should be taken into account in the future adoption of RPM devices.^28,29 Fourth, although this study refers to a particular situation related to the COVID-19 pandemic, assessing the use of similar devices can be an excellent solution for other clinical chronic disease contexts such as patients suffering from asthma, heart disease, etc.³⁰ Finally, the data analysis was possible only on a sample of convenience, as the Italian privacy legislation dictates strict rules that do not allow access of identified data recorded by the telemedicine device. Privacy regulations represent an administrative barrier that is hindering telemedicine adoption. The European Union GDPR³¹ legislation provides for very strict rules and requires a privacy impact assessment (PIA) that involves all organizational and technological aspects that influence data processing. Simpler tools are needed to ensure adequate levels of data security that do not hinder the study, improvement and implementation of telemedicine.³²

A particular finding of this study is that the percentage of women who used telemonitoring was only 35%, while worldwide it is recorded that women contracted COVID-19 at a higher percentage of 50% or more than men. However, it is necessary to emphasize in the study that the patients deemed eligible for monitoring were all patients with chronic diseases, and that nationally women have a lower incidence of chronic illness.²²

Challenges of Conducting FMEA in the Health Care Delivery Context

The application of the FMEA methodology is very useful in the field of telemedicine, given its widespread use in health care systems. This is a methodology that originated in industry, and its adoption has spread widely in health care and has proven to be effective for clinical risk management. Its adoption is particularly useful where there is a high cost of implementing a new technology, as it allows proactive analysis to assess cost-effectiveness. In addition, multiple services can be compared on how well they meet actual utilization needs. The assessment tools based on risk quantification, such as those used in this study, make it possible to identify corrective actions to ensure optimal quality of care, including with respect to latent errors that would otherwise not be visible except at the time of the adverse event. The recent WHO consensus statement on the implementation of the “Global Plan of Action for Patient Safety 2021-2030”³³ puts a strong emphasis on the implementation of effective telemonitoring.

The COVID-19 pandemic has provided a strong impetus to the spread of telehealth and telemedicine, but some considerations remain. Currently, there are no shared protocols to ensure technology choices, so each health facility and regional authority relies on its own expertise. Studies to date have shown that more fragile user groups may have more equitable access to care using these systems. At the same time, low technology awareness and agency may be an additional barrier. Specific legislation addressing these concerns is lacking, raising legal and liability issues.

Future Areas of Research and Unanswered Questions

The analysis carried out is not exhaustive with respect to the countless applications of telemedicine. The FMEA methodology has limitations. In fact, the analysis is linked to the specific process and the composition of the team. The team plans to extend the analysis to other telemedicine processes, with different groups of patients across a variety of illnesses. This will allow a better comparison of the results and help to develop effective solutions capable of mitigating risks.

Conclusions

We found that a combination of patient monitoring tools using remote health care provider oversight and home-based monitoring of COVID-19 patient management was effective in providing safe and reliable care. We demonstrated that the telemedicine device reduced the number of hospitalizations and costs significantly. FMEA is a valuable method for addressing critical issues related to RPM deployment and successful uptake. This analysis demonstrates that among the most relevant critical issues in the use of telemonitoring is the need to consider technological compatibility. Indeed, digital tools represent an opportunity, but in the presence of frail people, they can be a barrier. Training of health care personnel and the sharing of supporting tools (eg, guidelines, checklists, structured data collection templates) are essential for the identification of the most appropriate devices and their proper use by both patients and caregivers. These initial data on the risks of telemonitoring will need to be further explored through additional research that the team is planning in the coming months.

Monitoring current clinical scenarios at the start can lead to predicting effective clinical scenarios, facilitate targeted interventions, and potentiate preventative care using large datasets and machine learning tools. Strong support of telemonitoring directly relevant to other processes of patient care was observed, including:

1. the data were accurate and reliable and could be used to make clinical decisions within tight time-frames;

2. patients remained in their own home with ability to perform daily actions;

3. there were no false positives;

4. health care workers could perform remote walking stress tests; and

5. data flowed continuously with no need to call patients or send a physician to their home to check on their health status.

It is important to emphasize the value of the data collected by RPM in managing COVID-19 and other chronically ill patients and for the planning and optimization of social services. The possibility of future integration of the electronic medical record and monitoring devices represents an innovative solution for data collection and monitoring processes. Conducting research on the costs and performance of hospitals during COVID-19 is still at an early stage, and further studies are needed to determine if these results are consistent across other hospitals, regions, and health care systems.

Conflicts of Interest

The authors have no conflicts of interest to disclose.