Fully Automated Surveillance of Healthcare-Associated Infections with MONI-ICU

Abstract

Objective

Expert surveillance of healthcare-associated infections (HCAIs) is a key parameter for good clinical practice, especially in intensive care medicine. Assessment of clinical entities such as HCAIs is a time-consuming task for highly trained experts. Such are neither available nor affordable in sufficient numbers for continuous surveillance services. Intelligent information technology (IT) tools are in urgent demand.

Methods

MONI-ICU (monitoring of nosocomial infections in intensive care units (ICUs)) has been developed methodologically and practically in a stepwise manner and is a reliable surveillance IT tool for clinical experts. It uses information from the patient data management systems in the ICUs, the laboratory information system, and the administrative hospital information system of the Vienna General Hospital as well as medical expert knowledge on infection criteria applied in a multilevel approach which includes fuzzy logic rules.

Results

We describe the use of this system in clinical routine and compare the results generated automatically by MONI-ICU with those generated in parallel by trained surveillance staff using patient chart reviews and other available information (“gold standard”). A total of 99 ICU patient admissions representing 1007 patient days were analyzed. MONI-ICU identified correctly the presence of an HCAI condition in 28/31 cases (sensitivity, 90.3%) and their absence in 68/68 of the non-HCAI cases (specificity, 100%), the latter meaning that MONI-ICU produced no “false alarms”. The 3 missed cases were due to correctable technical errors. The time taken for conventional surveillance at the 52 ward visits was 82.5 hours. MONI-ICU analysis of the same patient cases, including careful review of the generated results, required only 12.5 hours (15.2%).

Conclusion

Provided structured and sufficient information on clinical findings is online available, MONI-ICU provides an almost real-time view of clinical indicators for HCAI – at the cost of almost no additional time on the part of surveillance staff or clinicians.

Introduction

Surveillance of healthcare-associated infections (HCAIs) is a key parameter for good clinical practice (GCP), especially in intensive care medicine [1, 2]. Assessment of clinical entities such as HCAIs is a time-consuming task for highly trained experts who, in clinical settings, neither are available nor affordable in sufficient numbers for continuous surveillance services. Many published surveillance studies have been performed with either additionally budgeted (scientific) staff or with information technology (IT) tools specifically developed for such studies and not installed for regular use afterwards. Healthcare authorities increasingly demand from healthcare institutions the installation and regular use of HCAI surveillance [3] as a part of quality management (QM). However, this sound demand is often overruled by financial constraints or simply by the unavailability of a suitable workforce at the local or regional level. This situation has catalyzed the development of automated surveillance strategies. These automated surveillance algorithms often use diagnosis codes and inpatient medication administration data and in combination with results of microbiological analysis the sensitivity and objectivity of surveillance improves [4]. For a long time now, we have also been attempting to bridge these gaps by establishing a fully automated computer-based system for early recognition and continuous monitoring of HCAIs [5–7]. The foremost challenge was to obtain reliable surveillance data from intensive care units (ICUs) without the need to employ additional documentation staff and statisticians.

Scope

When developing MONI-ICU, our main approach was from the clinical viewpoint with the following key requirements: Reliability and accuracy in clinical terms and its acceptance and adoption by clinical experts; possibly lowering infection rates and costs through (almost) real-time monitoring. We tried to achieve compliance with international standards and HCAI case definitions such as those issued by the Centers for Disease Control and Prevention (CDC) in Atlanta, USA [8], or the European Centre for Disease Prevention and Control in Stockholm criteria [9, 10]. Very important was timeliness of the obtained results for early identification of infection in the individual patient and in the patient population of the healthcare institution. It was imperative to realize the full technical and organizational feasibility with no need for additional staff for documentation or analysis.

Technical Aspects

MONI-ICU relies on the following three components in terms of both method and practice:

• 1.Data sources that provide the respective structured medical data;
• 2.a medical knowledge base with computerized knowledge about all relevant clinical entities in the system; and
• 3.a processing algorithm that evaluates, aggregates, and interprets medical data in a stepwise manner until they can be mapped into the given HCAI definitions.

Technically, MONI-ICU possesses the following components and characteristics to enable its fully automated mode of operation and wide acceptance by clinical users:

There are data import interfaces to the patient data management systems (PDMSs) at the ICUs, the laboratory information system of the microbiology department and, last but not least, to the respective hospital information system. The measured and observed medical data coming from these information systems are entered into a stepwise pipeline of aggregation and interpretation, eventually to draw conclusions with regard to the presence or absence of HCAI conditions according to their definitions (cf., [7,11]). An extensive adoption of fuzzy set theory and fuzzy logic methodologies allows for graded intermediate and final results [11]; this permits immediate identification of borderline cases and trends.

MONI-ICU is deployed as modern web-based, Java-programmed system with routine operation within the intranet of the healthcare institution [12]; it is based on a service-oriented architecture. The Arden Syntax is used for representation of the necessary medical knowledge and processing steps [12–14]. This is a standard programming language fostered by Health Level Seven (HL7) and adopted as an American National Standards Institute (ANSI) standard. For the physicians acceptance of MONI-ICU the program offers explanations that describe how and why the intermediate and final results were calculated. Additionally, graphical user interfaces display the daily results while reporting tools summarize the patient-oriented outcome. Specifically, the MONI-ICU system consists of the following:

Data sources

MONI-ICU is connected to 12 ICUs at the Vienna General Hospital where adult intensive care patients are treated. These ICUs are all equipped with Philips Care Vue PDM systems that collect clinical, laboratory, and nursing data over time. The patients’ administrative data is transferred from the hospital’s information system into the PDMSs and is thus available for MONI-ICU. However, a separate data interface was established between the laboratory information system (LIS) of the microbiology department (the LIS there was developed by the municipal authorities of Vienna) and MONI-ICU.

Furthermore, all PDMS data are restored daily and made available for further software systems, such as MONI-ICU, in a so-called information support mart (ISM). ISM is a relational data base containing a number of MONI-ICU specific tables, and is filled at night. In the morning about 5–6,000 data items are waiting to be processed by MONI-ICU.

Medical knowledge base

MONI-ICU contains an extended number of algorithm- and rule-based knowledge to recognize and interpret relevant data constellations that finally contribute to the decisions as to whether a certain HCAI is present or not. By the use of fuzzy set theory and fuzzy logic, presence of HCAI is tagged with its calculated certainty. The required medical knowledge was defined by a small team consisting of an infection control specialist and a knowledge engineer and was facilitated by the availability of standards [8–10]. For more details the reader is referred to [7, 11].

Processing algorithm

The inference process is started daily at 5 am. For each patient (a maximum of 96 in the 12 ICUs), the entire knowledge base is applied. Processing is done in a stepwise manner: first, medical data are checked for plausibility and algorithms are applied to calculate intermediate numerical values such as means and scores; second, the patients’ measured, observed, and calculated data are interpreted and classified into normal or the respective pathological classes (increased, decreased, highly increased, etc.). Then, the abstracted and intermediate results are aggregated by the use of clinically meaningful rules. Finally, all included HCAI definitions are evaluated. As a result the (definitions of) HCAIs are fulfilled, not fulfilled, or fulfilled to a certain degree by the respective patient data. Quite often, patient data from the last few days are also taken into account.

A surveillance screen allows the infection control user to obtain an overview about all 12 ICUs, the patients, and the HCAI results. Moreover, detailed explanations containing the intermediate clinical results, and – if requested – the measured and observed patient data can be demanded by opening the explanation screens.

The results of MONI-ICU on the surveillance screen (we call it cockpit surveillance) are accessed from the rooms of the infection control unit at the Vienna General Hospital. Clinicians at the ICUs are directly contacted by the infection control staff when necessary.

Aims of the Study

The objective of the clinical evaluation presented here is to perform the following two comparisons:

We contrasted surveillance results generated automatically by MONI-ICU with those generated in parallel by trained surveillance staff and attending clinical experts using patient chart reviews and other available information. The data collected by human staff were taken as a clinical “gold standard” and then compared with the Moni-ICU results.

According to the key requirements we performed a comparison of the amount of time taken to manually analyze patient charts on the one hand, and applying Moni-ICU as well as reviewing the results presented on screen on the other.

Methods

The clinical “gold standard”

From November 2006 to February 2007, trained surveillance staff, at least one physician specialized in infection control, together with attending clinical experts reviewed systematically patient charts and other available information, such as laboratory results, twice or three times weekly, thus collecting data from 1007 patient days (two ICUs with together 16 beds, for adult patients, 99 admissions of >48 h duration; refer to ►Table 1). All data were collected at the Vienna General Hospital, a 2,200-bed teaching and tertiary-care hospital.

Table 1.

Patient data

	ICU 1	ICU 2	total
#Admissions >48 h	56	43	99
Patient days	471	536	1007
Average duration of stay (days)	8.4	12.5	10.2

For the identification of HCAI episodes the European HELICS [9] definitions for HCAI were applied (which after minor changes are now adopted [10] by ECDC, the European Centre for Disease Prevention and Control, and are largely compatible with US-based CDC [8] definitions). We asked for the following entities (with onset >48 h after admission):

• •septicemia (blood stream infection, BSI),
• •central venous catheter-related infection (CRI),
• •pneumonia (PN), and
• •urinary tract infection (UTI).

For more details on these entities and several variants of them, we refer the reader to [9].

Interpretation of the results was also done in cooperation with the physicians, and all collected data were reviewed again by the senior staff of the Clinical Institute for Hospital Hygiene.

Only the time taken by the surveillance staff to review patient charts was recorded and summed up. The recording was performed in 6-minute units.

MONI-ICU surveillance

The above selection of patient data was again analyzed. This time the analysis was performed by automatically accessing the respective PDMS data, the data from the microbiology laboratory, and data from the admission department of the Vienna General Hospital. The complete analysis of the selected cases was done with MONI-ICU [actually, a programmed prototype of the present system].

Again, the time taken to load the results on the screen, select patient data and review the results, including the backward-chained inference and calculation path to the original patient data, was measured and recorded for comparison.

Both, the results of infection surveillance and the time taken for two independent cycles were then compared.

Results

The following results were obtained: 68 patients had no HCAI and in 31 patients the following HCAI were found:

• •septicemia (blood stream infection, BSI) n = 1,
• •central venous catheter-related infection (CRI) n = 18,
• •pneumonia (PN) n = 7, and
• •urinary tract infection (UTI) n = 5.

MONI-ICU correctly identified the presence of one of the above-listed HCAI conditions in 28/31 cases (sensitivity 90.3%) and their absence in 68/68 of the non-HCAI cases (specificity 100%). Thus, an overall accuracy of 97% was achieved (cf., ►Table 2). Of the three undetected cases, two were due to missing microbiological data in the MONI-ICU database (a transfer error in data input) while one was due to a missing parameter in one rule definition. The positive predictive value accounts for 100% and the negative predictive value amounts to 95.7%.

Table 2.

HCAI conditions correctly / falsely identified or missed by MONI-ICU

	Condition present “gold standard”	Condition absent “gold standard”
Condition present “MONI-ICU”	28/31 (90.3%)	0/68 (0%)
Condition absent “MONI-ICU”	3/31 (9.7%)	68/68 (100%)

The time taken for conventional surveillance was 52 ward visits comprising 82.5 hours (incl. 7.2 hours of walking) for human data collection and analysis. MONI-ICU analysis of the same 99 admissions took 12.5 hours at the MONI cockpit, which was roughly 15% of the time taken for conventional surveillance (cf., ►Table 3).

Table 3.

Time expenditure for conventional (human) and MONI-ICU surveillance

	Conventional surveillance	MONI-ICU surveillance
Time spent	82.5 h (100%)	12.5 h (15.2%)

Discussion

Automated surveillance by MONI-ICU is much faster and less dependent on human factors than conventional (manually operated) surveillance. High specificity of the results of surveillance is of paramount importance, as false alarms would rapidly and strongly discourage clinicians from accepting such a tool.

As the missing cases in MONI-ICU surveillance were due to rectifiable technical errors, a sensitivity of 100% can be achieved.

Investing resources in the development and programming of MONI-ICU is meaningful, as it provides reliable surveillance data rapidly. In general and also for the future, time and know-how must be invested so that MONI-ICU can keep pace with advancing clinical expertise and gets adapted to the users’ specific needs. The users, in turn, benefit from this time investment because their daily surveillance becomes rapid and precise.

Requirements for MONI-ICU and challenges

• •Availability of a suitable electronic PDMS is crucial, because in the future any additional manual data entries will have not the ghost of a chance due to shortage of staff.
• •Sufficient data of adequate quality must be stored and be accessible in PDMS. MONI-ICU functionality is hindered or blocked by improper use or changes in interfaces to other clinical and institutional data systems. Similarly, sudden software and/or hardware changes in remote components of the healthcare institution’s IT network may cause unexpected breakdowns. MONI-ICU requires a smoothly functioning IT environment.
• •Clinical experts must be available and willing to cooperate in tuning and updating the system. Continuous cooperation between the MONI-ICU provider, the local IT management, the surveillance team, and clinical experts is indispensable. Therefore a steering team will be established.
• •Funding the development and installation as well as continuing support of the system to keep in pace with advancing clinical expertise and case definitions are also necessary.
• •Being understood and accepted by intended users. The reluctance of medical and other experts to entrust knowledge to an electronic system, the fear of being replaced by it in the long run and similar prejudices may prevent potential users from getting acquainted with its qualities.

Conclusions and Perspectives

MONI-ICU generated no false positive cases, thus demonstrating high specificity. Its sensitivity was reasonable even with the applied MONI-ICU prototype and after having modified the rule definition and corrected the data input, its sensitivity has now been optimized, and re-evaluation of the presented data is under way. Regular updating is a basic feature of MONI-ICU in order to keep pace with advancing clinical expertise. In fact, MONI-ICU may be used as a tool for challenging current HCAI definitions and for investigating their validity. Although it was primarily developed for continuous surveillance, its features will offer clinical decision support directly at the ICUs in the form of alerts and reminders.

Clinical Relevance Statement

With MONI-ICU as an add-on to PDMSs, regular and continuous surveillance of HCAIs is feasible even with a small workforce. This opens great possibilities for GCP, QM, and benchmarking routines in healthcare institutions. Likewise, MONI-ICU may be used as a tool for clinical science.

Conflict of Interest

The scientific development of Moni was carried out at the Medical University of Vienna, programming work by Medexter Healthcare GmbH, which will also commercialize the software.

Protection of Human Subjects

This study was carried out according to the Medical University of Vienna rules of “Good Scientific Practice”. Only retrospective patient data that were routinely acquired through medical information systems at the Vienna General Hospital were applied. The study had no influence on any diagnostic or therapeutic decisions.