Accurate Automated Apnea Analysis in Preterm Infants

Abstract

Objective

In 2006 the apnea of prematurity (AOP) consensus group identified inaccurate counting of apnea episodes as a major barrier to progress in AOP research. We compare nursing records of AOP to events detected by a clinically validated computer algorithm that detects apnea from standard bedside monitors.

Study Design

Waveform, vital sign, and alarm data were collected continuously from all very low-birth-weight infants admitted over a 25-month period, analyzed for central apnea, bradycardia, and desaturation (ABD) events, and compared with nursing documentation collected from charts. Our algorithm defined apnea as > 10 seconds if accompanied by bradycardia and desaturation.

Results

Of the 3,019 nurse-recorded events, only 68% had any algorithm-detected ABD event. Of the 5,275 algorithm-detected prolonged apnea events > 30 seconds, only 26% had nurse-recorded documentation within 1 hour. Monitor alarms sounded in only 74% of events of algorithm-detected prolonged apnea events > 10 seconds. There were 8,190,418 monitor alarms of any description throughout the neonatal intensive care unit during the 747 days analyzed, or one alarm every 2 to 3 minutes per nurse.

Conclusion

An automated computer algorithm for continuous ABD quantitation is a far more reliable tool than the medical record to address the important research questions identified by the 2006 AOP consensus group.

Apnea is common in preterm infants and is a major reason for prolonged stay in neonatal intensive care units (NICUs). In 2006, the National Institutes of Health and the Food and Drug Administration convened a consensus group to define the issues associated with apnea of prematurity (AOP), with the assignment to define future research goals for improving diagnosis, understanding etiology, and developing more effective treatments. The group identified the following issues requiring attention: “1) lack of standardization for definition, diagnosis, and treatment of AOP, 2) unproven benefit of intervention, 3) lack of real-time data documenting AOP events, 4) unevaluated sustained treatment improvement at 7 days or later, 5) failure to address confounding conditions, 6) unsubstantiated AOP–gastroesophageal reflux disease relationship, and 7) undetermined role of AOP affecting long-term neurodevelopmental outcomes (p. 47).”¹

A major barrier to progress has been the accurate counting of apnea episodes. Nearly all published studies on AOP involving the incidence, efficacy of treatments, and predictions for safe NICU discharge regarding AOP have relied on tallies of nursing records of AOP events. Only a few have utilized electronic databases of recorded NICU monitor tracings^2–4 or portable home monitors,⁵ and these were limited by either small sample size and short monitoring times, or they involved infants studied after discharge, when the risk of AOP events was quite low. There is consensus that nurses underreport AOP.^2–4,6–9 This may be attributed to inaccuracy of bedside monitor alarms, sensory overload from other alarms, and confounding chest impedance signals from skeletal muscle and the heart.^2,3,7,9,10

The current study reports on data collected from a large cohort of preterm infants, where respiratory, heart rate, and oximetry waveforms were collected continuously throughout the infants’ NICU hospitalization, subjected to analysis using automated computer algorithms to detect and quantify apnea, and compared with apneic events as recorded by nurses. We believe that this new technique of automated analysis of very large computer databases, using clinically validated algorithms, can be used in future research to answer many of the questions posed by the consensus group.¹

Methods

The Institutional Review Board at the University of Virginia approved the analysis of patient cardiorespiratory waveforms and data abstraction from patient charts and classified the information as exempt from requiring consent.

Patients

All very low-birth-weight (VLBW) infants (birth weight < 1,500 g) undergoing continuous cardiopulmonary monitoring in the NICU at the University of Virginia from January 23, 2009, to March 4, 2011, were included in the analysis. Analysis of the waveform data and the nursing records was restricted to periods when the infants were spontaneously breathing without ventilator assistance. We included time periods when infants were receiving nasal cannula, continuous positive airway pressure, or oxygen hood therapies.

Data Acquisition and Study Design

There were 298 VLBW infants admitted in 771 days. We entered birth weight, gestational age, and times and types of respiratory support throughout the NICU stay for all patients in a relational clinical database. We collected and stored waveform data continuously from all VLBW infants throughout their hospital stay. There were periods of time where waveform data were not available due to technical failure. Also we excluded any time when the patient was receiving mechanical ventilation. We were able to analyze 19.9 patient-years of signal data for 276 patients while they were breathing spontaneously, and we also restricted our analysis of the nursing records to those same time periods.

All infants in the NICU have three electrode leads connected to GE monitors, GE Healthcare, Milwaukee, WI (models Solar 8000M and I, and Dash 3000), from which three electrocardiograph (EKG) and chest impedance (CI) signals are generated. Every 2 seconds the monitors compute the heart rate (HR) by averaging the RR intervals of the preceding eight beats, and the respiratory rate by counting the impedance oscillations. A pulse oximetry probe provided a pulse oximetry (SpO₂) and a HR value every 2 seconds from the oximeter signal. CI, EKG, oximeter waveforms, vital sign data, and alarm occurrence were captured from NICU bedside monitors, using the Bed-Master (Excel Medical, Jupiter, FL, United States) system. Data were collected continuously, date stamped, and stored on a computer cluster, with the capacity to store over 100 terabytes of data, for subsequent analysis and comparison with the clinical database.^11,12 A custom Matlab-based graphical interface was used to inspect data and develop and test data analysis algorithms. A typical event is shown in ►Fig. 1, where the top tracing represents the HR, which falls to < 100 beats per minute (bpm) starting near time 0. The third tracing represents the CI signal. The SpO₂ is represented by the fourth tracing and fell below 80% (horizontal dashed line) as the HR fell.

Fig. 1.

Algorithm-detected episode of central neonatal apnea. This constellation of signals is consistent with an episode of central apnea with bradycardia and desaturation. Time 0 is the center point of the apnea event. The results of the automated detection appear below the clinical data tracings as described in text. Abbreviations: ABD, apnea, bradycardia, and desaturation event; bpm, beats per minute; EKG, electrocardiograph; IP, impedance pneumography; SpO₂, pulse oximetry.

We compared nurses’ documentation and alarms generated by the bedside monitors, with apnea events found by analysis of the continuous electronic waveform data.

Data of nurses’ recordings were collected from an apnea and bradycardia document located at each patient’s bedside and entered retrospectively into the relational clinical database used for this study, excluding periods when there was no waveform data and excluding times that the patient was receiving mechanical ventilation. Our NICU nursing guidelines define a recordable apnea as a respiratory pause > 20 seconds, or a respiratory pause > 10 seconds with associated cyanosis, pallor, hypotonia, or bradycardia, with specific HR and SpO₂ limits undefined. Nurses are to note every event on the apnea and bradycardia record and to include the date and time, presence of apnea, lowest HR and oxygen saturation, skin color, duration, level of stimulation and/or ventilation support, and other comments relevant to the episode. Events that were recorded within 5 minutes of another event were considered a single event. Nurse-recorded events during times when electronic data were missing for technical reasons were excluded from analysis.

The automated algorithm has been described in detail elsewhere.¹¹ In brief, apnea is detected as low-variance epochs of a digitally filtered CI signal. A notch filter removes the cardiac artifact of the CI, resampled using the QRS signal as a clock. A high-pass filter removes artifact due to patient movement. In ►Fig. 1, the algorithm-defined event appears below the clinical data tracings. The fifth tracing is the impedance pneumogram after removal of cardiac artifact. The sixth tracing is the impedance pneumogram after high-pass filtering to remove artifact caused by patient movement. This renormalized signal is converted to the probability of apnea as a function of time, represented by the bottom tracing. The algorithm-detected apnea event begins when the probability of apnea increases through 0.1 and ends when it decreases through 0.1, which correlated with length of respiratory pause as measured by clinicians’ examination of the raw impedance signal. A validation process that compared algorithm-defined apneas with clinician analysis of the waveforms disclosed a 91% agreement and 5% false-positive rate.¹¹

Because of the lack of consistency in previously published definitions of AOP,^{1,2,6,13–16} we chose to use a very conservative threshold for defining an algorithm-detected event, requiring all three criteria to be present, that is, apnea duration > 10 seconds plus associated bradycardia (HR < 100 bpm), plus desaturation (SpO₂ < 80%) as described in Lee et al.¹¹ If all three conditions were met, the event was labeled as ABD-n, defined as central apnea lasting at least n seconds and accompanied by bradycardia and desaturation. Hence, each event was termed ABD-10, ABD-20, and ABD-30 if it lasted greater than 10, 20, or 30 seconds, respectively (ABD-20 includes ABD-30 and ABD-10 includes the other two). For comparison with nursing records, we counted events that occurred within 5 minutes of another event as a single event, and we designated the duration as equal to the longest event within that 5-minute period, as we decided that it would be impractical and therefore unlikely for nurses to record all the individual elements of an apnea “cluster” as multiple events.

For the purposes of matching the nurse-documented and algorithm-documented events, we selected criteria that would favor the nurse. First, we counted all events recorded by nurses, regardless of duration or completeness of the nursing entry. Second, when seeking an algorithm-recorded event to compare with a nurse-recorded event, we accepted all ABD-10s, -20s, and -30s. Third, when seeking a nurse-recorded event to compare with an algorithm-recorded event, we selected only ABD-20s and -30s, reasoning that these conservative criteria should have activated at least one type of alarm and that it would be reasonable to expect clinicians to consider these events as clinically significant and warrant recording in the patient record.

Alarm data were collected electronically from the monitors and stored in the database. The monitor is default-programmed to sound the alarm for apnea as soon as the monitor detects no breaths by CI for > 15 seconds, for bradycardia when there are eight consecutive RR intervals less than the set alarm threshold (default = 90 bpm), and for desaturation when the SpO₂ threshold is less than the set limit for > 5 seconds (default = 85%). Nurses do not routinely change monitor alarm default settings except the SpO₂ lower alarm limit, which is sometimes increased for older babies with severe bronchopulmonary dysplasia at risk for pulmonary hypertension.

Results

Patient Population

Patient demographics are presented in ►Table 1.

Table 1.

Characteristics of study population (n = 276)

	Mean±SD	Median	25th percentile	75th percentile
Gestational age at birth (wk)	27.6±3.0	27	25	29
Birth weight (g)	999±294	1,000	760	1,250
Ventilator days (d)	15.4±38.5	2.7	0	28.9
Length of stay (d)	65.6±42.9	57.6	32.3	94.7
PMA at discharge (wk)	37.7±4.7	37	35	39

Abbreviations: PMA, postmenstrual age (defined as gestational age at birth + chronological age); SD, standard deviation.

Comparison of Nursing Records with Algorithm-Detected Events

Nurses reported 3,019 events occurring during times for which there were valid algorithm data. For each event recorded by nurses, we searched for an algorithm-detected ABD-10 event within 60 minutes before or after the nurse-recorded event. We selected 60 minutes before and after the nurse-recorded event to allow for the fact that busy nurses may not document events immediately and may have only approximate recall of the time an event occurred. Of the 3,019 events documented in the nursing records, the computer algorithm validated an ABD-10 event in 2,078 cases. Thus, for more than 30% of nurse-recorded events the computer algorithm did not find an ABD-10 event.

The computer algorithm detected many ABD events not documented in the nursing records, as shown in ►Table 2. When considering the clearly significant events—apnea events that lasted more than 30 seconds and were accompanied by bradycardia and desaturation—more than 70% of algorithm-detected events were not accompanied by a nurse-recorded event.

Table 2.

Comparison of nurse-recorded events to algorithm-detected events

ABD Duration	Algorithm-detected events (n)	Algorithm-detected events that had an associated nursing record, n (%)a
> 20 s	10,133	2,057 (20.3%)
> 30 s	5,275	1,375 (26.1%)

Abbreviation: ABD, apnea, bradycardia, and desaturation event.

^aAny event recorded within 60 minutes of algorithm-detected event.

Comparison of Monitor Alarms with Algorithm-Detected Events

►Table 3 gives the incidence of bedside monitor alarms registered during the 19.9 patient-years that waveform data were analyzed. ►Table 4 compares the bedside monitor alarms to algorithm-detected events.

Table 3.

Monitor alarms during algorithm analysis time

Type of Alarm	n
A	19,604
B	37,965
D	313,364
Simultaneous A, B, and D	1,244
Simultaneous B and D	22,051

Abbreviations: A, apnea; B, bradycardia; D, desaturation.

Table 4.

Comparison of monitor alarms to those clinically significant events detected by computer algorithm

ABD Duration	No. of ABD	Apnea alarms, n (%)	Bradycardia alarms, n (%)	Desaturation alarm, n (%)	Apnea, bradycardia, and desaturation alarms, n (%)	Any alarms, n (%)
> 20 s	10,133	1,827 (18%)	6,549 (65%)	6,841 (68%)	853 (8%)	7,363 (73%)
> 30 s	5,275	1,217 (23%)	3,555 (67%)	3,646 (69%)	596 (11%)	3,911 (74%)

Abbreviation: ABD, apnea, bradycardia, and desaturation event.

Note: % refers to the percentage of ABD events associated with the specific monitor alarm.

For ABD-20 and ABD-30 events, the apnea alarm on the monitor sounded only 18 and 23% of the time, respectively. The bradycardia and desaturation alarms had better performance, but even considering only the most severe events (ABD-30), the monitor did not sound any alarm in 26% of cases.

Frequency of Monitor Alarms in the NICU

In a separate analysis, we considered all alarms of all descriptions that were activated by the bedside monitors throughout the NICU from all patients, regardless of gestational age or diagnosis. During the 747 days for which we had electronic alarm data, there were 8,190,418 alarms. On average, there was a daily census of 37.8 infants cared for by 17 nurses per shift. Thus, there were 27 alarms per nurse per hour, or an alarm approximately every 2.2 minutes per nurse.

Discussion

This study, the largest of its kind, confirms reports from other groups that nursing records do not provide sufficiently reliable documentation of AOP.^2–4,6–9 The goal of our work was to test the hypothesis that the nursing record is not sufficient for the accurate accounting of episodes of central neonatal apnea required for clinical research studies. The results point to clear superiority of the automated algorithm for accurate central apnea counting and therefore would be a better way to answer many of the outstanding questions posed by the Summary Proceedings from the Apnea of Prematurity Group.¹

Comparison of Nursing Records with Algorithm-Detected Events

We found two kinds of discrepancy between nursing records and algorithm-detected events. First, nurses reported many events that the algorithm did not verify, and second, the algorithm detected many events that were not documented in the nursing record.

Overreporting

Apnea is a term that has routinely been used to apply both to complete cessation of phrenic movement (“central apnea”) and to obstruction of airflow in the presence of phrenic movement (“obstructive apnea.”) Events with elements of both types are called mixed apnea.¹⁷ Without a means of detecting airflow, our algorithm is restricted to detecting and recording only those events that have central components, with cessation of diaphragmatic movement lasting more than 10 seconds. Because nurses may record any obstructive, central, or mixed events as an “apnea event” on the bedside record, it is likely that this inability to distinguish central from obstructive apnea is responsible for much of the overreporting.

Underreporting

A certain amount of underreporting is not surprising, given the intensity of the clinical workload, the frequency of self-resolved events not requiring nurse intervention, and uncertainty about the validity of the bedside monitor alarms. However, of the more than 10,000 hypoxemic bradycardic apneic events lasting more than 20 seconds that the algorithm identified in 19.9 patient-years of data, documentation of only about one in five events appeared in the medical record. Only one in four episodes lasting more than 30 seconds—a clinically significant central apnea event by any standard—was documented., It is highly unlikely that the algorithm overreported events as the algorithm was repeatedly and rigorously validated. In a random sample of 100 algorithm-detected ABD-30 events inspected by three experienced clinicians, the algorithm had only approximately 5% false-positive rate.¹¹ One possible explanation for lack of medical record documentation of long apnea events is that nurses might record the time of the event incorrectly, especially if their work is interrupted, accounting for inaccurate times on the nurses’ record. Nurses may also interpret a self-resolved monitor alarm or an apnea event that is mixed or occurs during feeding as a questionable or not clinically relevant event, and therefore not document the event. Whatever the reason for underreporting, the occurrence of many long central apnea events associated with both bradycardia and desaturation that are not documented in the medical record demonstrates that a computerized system provides a more accurate quantitation of central AOP.

Muttitt et al studied 27 infants and showed that the nurses documented 54% of computer-detected apneas that included central, obstructive, and mixed etiologies.⁴ The nurses were best at recording those of central origin and those of longer duration. Southall et al studied 14 infants with 24-hour tape recordings of EKG and CI.² They showed that nurses did not record 67% of apnea events that lasted more than 20 seconds, a result similar to our study of over 10,000 automatically detected events, more than 5,000 of which lasted more than 30 seconds.

Comparison of Monitor Alarms with Algorithm-Detected Events

We found the most unreliable alarm to be the apnea alarm, which activated in only 23% of the most severe events (ABD-30). As previously described by Southall et al, the apnea alarm is very insensitive due to the conflicting effect of cardiac movement on the CI signal.^2,10 This effect can be interpreted by the monitor as a breath and becomes an increasingly significant artifact when there is bradycardia associated with the apnea.¹¹ In our study we were able to remove cardiac artifact and random movements from our CI signal and therefore determine true central apneic events.

In addition, we found that monitors alarm for only 60 to 70% of the bradycardias associated with these events. Possible explanations of these apparent alarm failures are that the monitor’s bradycardia alarm limit was set to a number lower than the 100 bpm used by our algorithm or that the bradycardia was shorter than the 8 EKG beats required by the monitor alarm algorithm. Similar explanations might apply to the desaturation alarms—the computer algorithm we used requires the SpO₂ to be less than 80% for only a brief period of time, but the monitor algorithm requires that the SpO₂ be lower than the set limit for > 5 seconds. This latter explanation appears less likely, as the algorithm threshold was set at 80% saturation and the standard alarm setting is significantly higher (85%). Nevertheless, the fact that 26% of the most severe events (ABD-30 events) led to no apnea, bradycardia, or desaturation alarm constitutes good evidence that improvements are needed.

Assessment of the Frequency of Monitor Alarms throughout the NICU

The finding that there is on average one alarm every 2 to 3 minutes per nurse shows the need for improvement in monitoring and alarm systems in NICUs. This is an extraordinary rate of alarms. Moreover, it has been suggested by others that fewer than 10% of ICU alarms indicate truly dangerous clinical situations, and others and the current study have found that the monitor may not alarm at all during clinically significant events.^18–21 Varpio et al examined alarm fatigue among nurses on an inpatient pediatric unit and found an average of one alarm every 6 to 7 minutes.²² Similarly, Bitan et al studied this phenomenon in the NICU and found an alarm every 3 to 4 minutes.²³ Both studies concluded that nurses do not specifically respond to every alarm, but incorporate them as part of the clinical scenario.^22,23

In our previous work, we found that about two-thirds of the apnea alarms from the monitor were false alarms.¹¹ In a recent national survey, clinical engineering, nursing, and technology professionals agreed that frequent false alarms were a problem and disrupted patient care.²⁴ Moreover, the majority felt that false alarms led to “distrust” and led to ignoring or disabling the alarms. Therefore, it is reasonable to speculate that significant underreporting of events can be ascribed both to desensitization because of false alarms, as well as to distrust because of failure of the monitors to detect serious events.

Limitations

A limitation of this study is that we do not know the monitor alarm limits settings in all instances. The default monitor alarm for HR is 90 bpm and our algorithm used a higher threshold of 100 bpm. This could explain why the monitor alarm failed to activate for an algorithm-defined bradycardia but does not explain why the monitor does not alarm for the apnea and/or desaturation component of an ABD-20 or ABD-30, given that the default alarm thresholds are much greater than the algorithm threshold.

Conclusions

The current gold standard of documentation of AOP often used for research purposes, the nursing record, does not document the majority of prolonged central apnea events detected by our rigorously validated computer algorithm. Monitor alarms are not activated in about 25% of prolonged events. Possible reasons for inaccuracy of apnea documentation include insensitivity of standard monitors to reliably detect central apnea, insensitivity of monitors to detect obstructive and many mixed apneas, an abundance of alarms leading to alarm fatigue by caretakers, and the impracticality of manually recording a large number of events. Continuous computer analysis of existing NICU monitor cardiorespiratory signals, such as the system we developed, provides more accurate quantitation of apnea events. This, or a similarly validated system, would help to answer many of the questions posed by the Apnea of Prematurity Group in future research including true benefit of therapies, the relationship of gastroesophageal reflux and AOP, and the role of AOP affecting long-term neurodevelopmental outcomes.¹