A Ballistographic Approach for Continuous and Non-Obtrusive Monitoring of Movement in Neonates

Abstract

Continuously monitoring body movement in preterm infants can have important clinical applications since changes in movement-patterns can be a significant marker for clinical deteriorations including the onset of sepsis, seizures, and apneas. This paper proposes a system and method to monitor body movement of preterm infants in a clinical environment using ballistography. The ballistographic signal (BSG) is acquired using a thin and a film-like sensor that is placed underneath an infant. Manual annotations based on video-recordings served as a reference standard for identifying movement. We investigated the performance of multiple features, constructed from the BSG waveform, to discriminate movement from no movement based on data acquired from 10 preterm infants. Since routine cardiorespiratory monitoring is prone to movement artifacts, we also compared the application of these features on the simultaneously acquired cardiorespiratory waveforms, i.e., the electrocardiogram, the chest impedance, and the photoplethysmogram. The BSG-based-features consistently outperformed those based on the routinely acquired cardiorespiratory waveforms. The best performing BSG-based feature-the signal instability index-had a mean (standard deviation) effect size of 0.90 (0.06), as measured by the area under the receiver operating curve. The proposed system for monitoring body movement is robust to noise, non-obtrusive, and has high performance in clinical settings.

This paper presents a system and method for monitoring movement in neonates using ballistography. The proposed approach is robust to noise and can be used in real time in clinical settings.

I. Introduction

With 15 million infants being born too soon every year (>10% of all births), prematurity of birth is a major public health concern and the leading cause of neonatal deaths [1]. Such vulnerable infants, owing to their physiological immaturity, are often admitted to neonatal intensive care units (NICUs) where vital signs such as the heart rate (HR), breathing rate (BR) and oxygen saturation (SpO2) are continuously monitored. These vital signs of cardiorespiratory origin are reflective of the regulation of the autonomic nervous system. Monitoring vital signs is important so that dysregulation, as measured by vital signs crossing predetermined thresholds, can be brought to the attention of clinicians and the underlying cause of deterioration can be addressed.

In addition to continuous cardiorespiratory monitoring, as a part of routine care, intermittent clinical observations at the bedside help provide information on muscle tone, skin color, and gross body movement -- important clinical features that are not captured in cardiorespiratory monitoring yet capable of serving as early warning signs of clinical deterioration. Body movement or the quality thereof has long been considered important, as can be deduced from its use in various clinical scoring systems such as the ComfortNeo [2], Assessment of Preterm Infant Behavior [3] and the Newborn Individualized Developmental Care and Assessment Program [4]. While monitoring vital signs reflects the functioning of the autonomic nervous system, monitoring body movement indirectly provides insights into the functioning of the motor system. Motor activity changes (e.g., overall body movement, lethargy) in response to many clinically significant events including the onset of sepsis, seizures, apneic episodes as well as during the sleep-wake cycle. For instance, lethargy, the absence of spontaneous movement, but also less apparent changes in movement activity over time, have been identified as strongly predictive symptoms of sepsis in multiple studies [5]–[8]. Concerning seizures, some seizure-types cannot be identified using electroencephalograph monitoring but do have motor manifestations [9], [10]. Also, the presence or absence of body movement preceding, during, or in response to apnea can help discriminate central apnea from obstructive apnea [11]–[13]. Monitoring movement can also facilitate identifying sleep-wake cycles in preterm infants, which can be used to help synchronize periods of nursing care to minimally disturb the sleep of preterm infants – an important factor in neurodevelopment [14], [15]. Therefore, continuously monitoring movement can be important -- for example, over long durations for identifying sepsis (timescale of hours, up to 1 or 2 days), medium durations for the tracking of sleep-wake cycles (tens of minutes, up to an hour) and short durations for identifying apnea (seconds or minutes). Here, intermittent observational scoring of movement by clinical personnel at the bedside is of insufficient frequency, necessitating the need for automated technology to monitor movement continuously.

Recently two reviews have discussed, in detail, wearable sensor systems (e.g., accelerometers) and camera-based solutions for monitoring movement in neonates, along with potential clinical use-cases [16], [17]. They highlight the challenges for sensor technologies, including the requirement of sufficient sensitivity to subtle movements and the necessity of integration within the neonatal environment given the fragility and vulnerability of preterm infants. The limitations of wearable sensor systems such as accelerometers include difficulties in finding the optimal location(s) for the placement of the sensor(s), i.e., body parts likely to undergo acceleration due to movement. Further long-term monitoring using wearable sensors is associated with skin irritation and discomfort, to which preterm infants are especially vulnerable [18]. Finally, wearable sensors are associated with multiple wires attached to the infant that increase the sense of detachment between parents and infants [19]. On the other hand, video-based monitoring of movement, while non-obtrusive, face challenges with regard to occlusion of view, changes in lighting conditions, light-reflection from incubator walls and the fact that NICU environments are often dim and that blankets cover incubators.

Any approach for capturing forces generated by the body can be termed ballistography (BSG) and includes forces due to body movement, breathing motion and the mechanical action of the beating heart (also called ballistocardiography, BCG) [20]. BSG-based clinical monitoring solutions using different types of sensors such as pressure sensors, weighing scales, optical sensors, accelerometers, and piezoelectric sensors have been described for several clinical applications, including monitoring cardiac performance, hemodynamic changes, sleep monitoring and respiration-monitoring [21]–[26].

Currently, there are no clinically tested solutions suitable for monitoring movement in the NICU environment, potentially a valuable source of physiological information. In this paper, we propose a non-obtrusive BSG-based system, for monitoring body movement of infants in real time. Based on the BSG signal, we develop and compare the performance of several BSG-based features for capturing movement with video-based visual annotations of movement. The system and method described herein can enable the automation and quantification of movement. Furthermore, since movement artifacts are a common occurrence in routinely monitored cardiorespiratory waveforms such as the electrocardiograph (ECG), chest impedance (CI) and the photoplethysmograph (PPG), we also quantify the potential of these waveforms for monitoring movement.

II. Methods

A. Sensor Specifications

In this study, we used an electromechanical film sensor (EMFi, Emfit, Kuopio, Finland) first described by Paajanen et al. [27]. This sensor is made of three layers – two smooth and homogenous surface layers enclosing a thicker midsection consisting of polypropylene layers separated by air voids (Fig. 1a). Similar to piezoelectric materials, the EMFi sensor produces an electric charge when force is applied perpendicular to its surface. However, the charge generation mechanism is not piezoelectric, since the change in the internal electric field, as a result of external forces being applied is caused by the mutual movement of static charges that were injected into the film using a corona discharge method in the manufacturing process. In other words, the EMFi is an elastic, permanently charged film in which the air voids act as electrical dipoles with the film being sensitive to forces normal to its surface [24], [28]. This sensor is well suited to measure pulsatile and transient forces, such as those of biological origin but unsuitable for measuring static forces [21], [26], [29], [30]. Notably, the sensor produces one output signal, effectively integrating different forces that may be applied to it.

FIGURE 1.

Part a shows the EMFi sensor which was used by placing it on a mattress underneath the infant (part b). Part c shows the sensor in clinical use in a NICU, but without the mattress cover for visibility. The sensor was connected to the signal acquisition system which was powered by a battery pack (part d). Part e shows the various components of the signal acquisition system which stored the BSG signal for offline signal processing in a computer (part f). Part g shows 30 minutes of the BSG signal while part h shows the breathing signal over a short period.

The sensor used in this study (L-series sensor, EmFit) had dimensions of mm. The sensitivity of the sensor is 25 pC/N, and the frequency response is flat up to 20 KHz. The specifications for the range of operating temperature are between −20 to 70 °C. The soft, flexible nature of the sensor, as well as the fact that it was electrically passive, made it well suited for use in the NICU environment where safety concerns are paramount. Before use, the sensor was placed on top of the mattress and was covered by a bedsheet (Fig. 1b). Fig. 1c shows the sensor being used in the NICU without a bedsheet for illustrative purpose.

B. Circuit Design for Signal Acquisition

A signal acquisition system was designed for processing, digitizing and storing the electrical signals that were acquired from the EMFi sensor. Fig. 1d shows the entire signal acquisition system in the packaging in which it was housed for the study. Fig. 1e shows the different parts of the signal acquisition system in a block diagram format. First, the signal from the EMFi sensor was amplified by a factor of 100 followed by low-pass filtering using a 4th order Butterworth filter with a cutoff frequency of 100 Hz (anti-aliasing filter). Both the amplification and the filtering were carried out using operational amplifiers (MCP 6242, Microchip). Digitization was carried out using a 16-bit analog-to-digital converter (ADS1115, Texas Instruments) and the digitized data was written onto a secure digital card using a microcontroller (Arduino Uno, Arduino). The data was time-stamped by the microcontroller, which in turn was time-synchronized to a laptop computer to which it was connected before starting every measurement. A rechargeable power bank capable of supplying 5V output was used to power the entire signal acquisition system.

C. Signal Quality and Signal to Noise Ratio

An infant placed on the EMFi sensor can generate forces due to body movement, breathing movements and the beating of the heart. Forces generated by the beating heart are maximally along the longitudinal axis of the infant, effectively parallel to the EMFi sensor. Thus, a majority of the power corresponding to the heart rate is filtered out by design. For breathing and body movement, a fraction of the force is always perpendicular to the surface of the sensor and is defined as the BSG signal (Fig. 1g and 1h). Fig. 1g shows the BSG signal (blue) with an algorithm to track movement (in green, described later). Typically, the breathing signal (Fig. 1h) is always present, and the signal corresponding to infant motion is superimposed upon it. Any external influence that would reduce the quality of the signal is defined as noise. We calculate the signal to noise ratio (SNR) as the ratio of the variance (normalized over time) of the zero-mean signals, corresponding to an infant placed on the sensor within the incubator, in comparison with the typically expected external ‘noisy scenarios’ in the NICU environment. These noise-like scenarios include the baseline signal acquired by the sensor while it is placed in the incubator without an infant on it, the effect of mechanical vibrations in the proximity of the incubator due to walking and hopping, as well as nurse-handling of the incubator. Nurse-handling includes opening the portholes of the incubator, opening the lateral side of the incubator, and operating the incubator while seated adjacent to the incubator with both feet placed at the base of the incubator, as is typical during nursing care. We tested all these effects on the SNR.

D. Signal Acquisition

Routine patient monitoring was carried out using patient monitors (Philips IntelliVue MX 800, Germany) from which the ECG (250 Hz), CI (62.5 Hz) and PPG (125 Hz) waveforms were acquired via a data warehouse (DWH, IIC iX, Data Warehouse Connect, Philips Medical Systems, Andover, MA). The BSG waveform (250 Hz) was acquired using the custom-made signal acquisition device (Fig. 1d). Furthermore, patient-monitor parameter data corresponding to HR, BR and SpO2 were acquired at 1 Hz from the DWH. Video monitoring of infants was carried out using a camera placed on a tripod, near the incubator. The data acquired from the DWH were manually synchronized with the BSG signal and the video data, to a precision of within 1s, based on matching the noise characteristics of the waveforms with the timestamps of the internal clock of the BSG acquisition device and the video camera, respectively.

E. Patient Population

Ten infants of varying body weights were enrolled in this study that took place in the level III NICU (private room design) of the Máxima Medical Center, Veldhoven, the Netherlands between May-June, 2017. The mean (standard deviation, SD) weight and postmenstrual age of the infants on the day of the study were 1422g (402g) and 31.4 weeks (2.3 weeks), respectively. All infants were recorded for approximately 6–8 hours during daytime hours, because of higher ambient lighting, desirable for video monitoring. From the recordings of each infant, two continuous segments of approximately one hour or longer were selected for inclusion in the analysis. The selection criterion was minimal obstruction of the infant due to nursing care or parental presence, and sufficient visibility of the infants in the video-data, which varied with lighting conditions. When possible, segments were selected with infants in different positions to determine whether the system under consideration worked sufficiently for all body positions of the infants. All infants received routine care. Infants were supported by positioning materials (SnuggleUp, Philips, Amsterdam, the Netherlands) in 18 out of 20 recorded segments, and partially covered by a blanket in 17 out of 20 segments, respectively. Eight of the ten infants received non-invasive ventilator support (continuous positive airway pressure or high flow nasal cannula) while two infants did not need supplementary oxygen. Infant positions in the recordings included eight right-lateral positions, seven left-lateral positions, three prone and two supine positions. In accordance with the Dutch law on medical research with humans (WMO), the medical ethical committee provided a waiver (registered as N16.068) for the study since it was of an observational nature and low-risk.

F. Movement Annotations

While qualitative movement scoring systems of a clinical nature based on visually observing various aspects of movement such as spontaneity, speed, force, etc. are available, they are not readily applicable for this analysis using the BSG waveform due to its purely quantitative nature [31]. Therefore, to determine whether the BSG waveform and the routinely acquired cardiorespiratory waveforms can be used to monitor body movement, a reference standard for measuring movements is required. Theoretically, body movement of an infant can be considered as a signal that ranges from zero to infinity. The signal is zero if there is no body movement, no breathing motion, and no heartbeats, thereby creating no micro-motion of the body. On the other hand, movement can be large with no upper bound (theoretically infinity) making gross body movement particularly hard to annotate and for which there exist no conventions.

For this study, as a reference standard, two annotators independently annotated infant motion at a frequency of 1 Hz, based on retrospective visual observation of video recordings, and categorized movement in a dichotomous manner. They were instructed to consider as a movement any motion of the head, trunk, arms, legs, and digits, no matter how small, but were required to ignore facial movements, such as grimaces and breathing-based motion of the chest. All periods of a limited view of the infant due to occlusion (e.g., parents or staff between infant and camera) and periods of nursing care were noted. Annotations were categorized into three classes -- no movement (‘0’), movement (‘0.1’) and occlusion/nurse handling (‘0.2’). Following this, the inter-rater agreement was estimated using the chance-corrected metric, Cohen’s kappa coefficient (), and expressed as mean (SD). For further analysis, only those periods were retained where the annotators agreed. The periods corresponding to occlusion or nursing care were removed from further analysis.

G. Signal Processing and Features for Tracking Movement

All waveform signals – the BSG, ECG, CI, and the PPG signals were digitally band-pass filtered using a Butterworth filter of the second order with lower and upper cutoff frequencies of 0.001 Hz and 0.40 Hz respectively. The lower cutoff was chosen to eliminate any low-frequency baseline drifts that may occur over time, while the upper cutoff was chosen to preferentially capture any movement information in the signal while limiting breathing and heart-rate based movement signal. The upper cutoff frequency was swept from 0.1 Hz to 1 Hz with a step size of 0.1 Hz to determine the most suitable frequency band for capturing movement information, for each of the four waveform signals under consideration. It was expected that upon increasing the upper-cutoff, an increasing amount of breathing and heart-rate based information would be captured. Finally, features to obtain different indices of movement were developed for use with the filtered signals.

A total of six features were used to quantify movement with the intention to integrate over both the quantity and duration of motion. These were based on the signal instability index (SII), Teager energy (TE), Hilbert transform (HT), approximate entropy (ApEn), Kolmogorov complexity (KC) and permutation entropy (PeEn), as discussed in the following paragraphs.

The rationale for using the aforementioned features was based on an expected difference in sensitivity to movement in comparison to the infant lying still. The use of TE and HT were motivated by their expected sensitivity to movement amplitude (signal power), while KC and the entropy-based features were expected to be sensitive to signal regularity with movement expected to be of an irregular nature. The SII was expected to capture aspects of both, signal amplitude and signal regularity. Frequency-based features were not considered since the corresponding signal is expected to be non-stationary, even in short time-windows. All feature values were estimated at a frequency of 1 Hz using a moving window of 15s since motion is typically considered to change over several seconds.

The signal instability index is a non-parametric measure based on the kernel density estimate (KDE) for determining the probability density function of the underlying signal. The probability density is approximated by the superposition of a number of Gaussian kernels centered on equidistant points (here 100) on the windowed signal. The estimated bandwidth of the superposition of these Gaussian kernels forms the SII [32]. The following equation was used to calculate the KDE, the bandwidth of which was the SII.

where is the KDE of the underlying signal , n is number of equidistant points – here 100 – used to divide the length of signal , is the Gaussian kernel centered at points and is the bandwidth of the Gaussian kernels, optimally estimated by the expression where is the standard deviation of the Gaussian kernel. The SII is low if the mean and variance of the underlying Gaussian kernels are comparable and would be reflective of the absence of motion.

The Teager energy operator is a non-linear operator to estimate energy using only a small time-window, making it ideal for local time analysis [33]. The TE feature, calculated every second is defined as the maximum value of the TE estimate applied at every point on the windowed data. The TE is calculated according to the following equation,

where, is the sample of the underlying signal .

The Hilbert transform was used for local envelope detection [34]. The HT feature, calculated every second, was defined as the mean value of the modulus of the complex analytical signal obtained from the window under consideration. The analytical signal () was defined as the sum of the complex signal, comprising the original signal () and its Hilbert transform () as follows,

where n is the samples of the underlying signal .

Approximate entropy, a regularity statistic, was used as a feature because it quantifies the unpredictability of time-series data in a manner not captured by moment statistics [35]. A time-series with repetitive patterns tends to have low ApEn, while a less predictable time-series has a high ApEn. The ApEn feature was calculated on the entire length of the windowed data, with the sequence-length for comparison, also known as the embedded dimension, equal to 1.5s of data (empirical choice) and a tolerance value defined as 0.2 times the standard deviation of the windowed data [35].

The feature based on Kolmogorov complexity was calculated for the entire length of windowed data after binarizing the time-series corresponding to the windowed data, based on the median value of the windowed data [36].

Permutation entropy is a complexity measure that considers the temporal order of data by using symbolic dynamics. It is known to be less affected by small amounts of noise as opposed to other entropy measures [37]. The PeEn feature was calculated for the moving window by averaging the PeEn calculated at every sample while using an embedded dimension equal to 2s of data and a permutation order of 5.

First, the performance of all six features was compared to annotations for all four waveforms corresponding to the original frequency band of 0.001-0.4 Hz. Second, for the frequency sweep, corresponding to each frequency band, the best performing features were determined for each of the waveforms.

H. Correlation Between Movement and Vital Signs

The maximum and minimum cross-correlation between the best performing movement index (SII) and the vital signs (acquired directly from the patient monitor) were calculated over ten-minute windows by first standardizing each signal (subtracting mean and dividing by standard deviation) and calculating the normalized correlation coefficient within a time lag of ±30 seconds at every data-point, i.e., every second. The correlations and corresponding lags were averaged for each of the 20 recordings and are presented when the mean ± SD for both the correlation values and the corresponding time lags did not include zero. High absolute values for correlation are indicative of coupling between the motor and the autonomic nervous systems (ANS).

I. Statistics

The feature-values for movement and no movement were visually compared using box-plots. Since the range of signal amplitudes corresponding to the movement was large, box plots were constructed using the logarithm of the feature values. The predictive accuracy of the algorithms was evaluated using effect size – a measure of performance that determines the magnitude of the difference between the two groups. We measured the effect size using a non-parametric approach by calculating the area under the receiver operating curve (AUROC). The AUROC is a standardized measure (no untis) and offers a threshold-independent method to evaluate the performance of the features. The mean and SD of the AUROC of the 20 recordings were calculated for all six features for all waveforms – the BSG, ECG, CI, and the PPG. All data were analyzed using Matlab (MathWorks, Natick, Massachusetts, USA).

III. Results

In this study, movement forces generated by preterm infants were captured using an EMFi sensor. For all 20 recordings that were acquired, the order of magnitude of the variance of the BSG signals was comparable, indicating that the differences in weight and position of the infants did not adversely affect the BSG signal. With regard to the SNR, the signal was defined as the BSG signal obtained with a 1500g infant lying on the sensor. The SNR for both the scenario without an infant lying on the sensor and for walking in the proximity of the incubator was 1000 while hopping in the proximity of the incubator and opening the incubator’s portholes led to an SNR of 100. Opening an entire side of the incubator and resting one’s feet at the base of the incubator, as is typical during nursing care, led to an SNR of 10.

The mean (SD) duration of the 20 recordings that were selected for analysis was 72 (4.5) min. For this data, the inter-rater agreement as measured by the score was 0.72 (0.07). After excluding periods of disagreement, occlusion, and nurse-handling, 82% (10%) of the BSG-data remained and was used for comparing the performance of the various features. Additionally, based on observations of the video recordings, the annotators identified infants to be moving for 43.7% (19.8%) of the time.

Fig. 2a and 2b, show 80 minutes of typical BSG signals obtained from two different infants along with one of the six calculated features – the SII. Here, the SII values can be considered a proxy for movement, with the area under its curve acting as a cumulative measure of movement over a time-period. Fig. 3 shows the box plot for the values of the SII corresponding to no-movement and movement, respectively, for the SII feature corresponding to Fig. 2b. Based on the box-plot we can observe that the SII can discriminate movement from no movement.

FIGURE 2.

Parts a and b show approximately 80 minutes of the BSG signal (blue) of two different infants with annotations for movement (red; 0 = no movement, 0.1 = movement and 0.2 = external handling) and the SII calculated every second using a running window of 15s.

FIGURE 3.

The box-plots show the logarithm of the SII calculated from the BSG signal shown in Fig. 2b for the time-periods corresponding to no-movement and movement. The SII responds differentially to the periods annotated as no-movement, versus those annotated as a movement. The y-axis is in arbitrary units (a.u.).

Fig. 4 shows the mean (SD) AUROC of the 20 recordings corresponding to all four waveform signals calculated for the six features– SII, TE, HT, ApEn, KC, and PeEn for the frequency band of 0.001-0.40 Hz. Amongst all features, the best performing features for detecting movement are the SII and the HT which are comparable to each other and outperform other features for all waveforms. The best-performing feature for both the BSG and ECG waveforms is the SII, with a mean AUROC of 0.90 (0.06) and 0.79 (0.15), respectively. The best-performing feature for the CI and the PPG waveforms is the HT, with a mean AUROC of 0.78 (0.15) and 0.77 (0.18), respectively. Amongst all waveform signals, the BSG is the best at monitoring movement for all six features. Particularly noteworthy is that the SD of the AUROC corresponding to the BSG signal, irrespective of the feature under consideration, is considerably smaller than the SD of the AUROCs corresponding to other waveforms, indicating repeatability in monitoring movement in comparison with other waveforms.

FIGURE 4.

The mean (SD) AUROCs corresponding to the BSG (black), ECG (red), CI (blue) and the PPG (green) waveforms for different features. The AUROC is a measure of the effect size of the feature’s ability to discriminate no-movement from movement.

Fig. 5 shows the performance of the best features for each of the waveforms when the upper cutoff frequency was swept from 0.1-1 Hz with a step size of 0.1 Hz, and the lower cutoff frequency was held constant at 0.001 Hz. The best performance of the BSG and ECG waveforms across all cutoff frequencies is with the SII while that of the CI and PPG waveforms is with the HT. The best performing of all features, irrespective of the waveform, is the SII for BSG waveform with an AUROC of 0.90 (0.06).

FIGURE 5.

The y-axis shows the effect size, as measured by the AUROC, of the best performing feature for each of the four waveforms after they were band-pass filtered to range from 0.001-1 Hz in step sizes of 0.1 Hz (x-axis). The error-bars are staggered about the x-tick values for readability.

Fig. 6 shows the SII, HR, BR and, SpO2 signal corresponding to a recording of approximately 80 min duration. Visually, we can observe that there are sustained periods of increased movement or movement-bursts that are often associated with an increase in HR and decrease in BR and SpO2, respectively. The correlation results indicate the same – a correlation of 0.40 (0.18) with time lag of −10.3s (6.9s) between SII and HR, a correlation of −0.41(0.17) with a time lag of −6.6s (4.1s) between the SII and the BR, and a correlation of −0.24 (0.13) with a time lag of −14.7s (10s) between SII and SpO2. The negative time lags indicate that movement precedes changes in vital signs.

FIGURE 6.

The SII, HR, BR and SpO2 signals for approximately 80 minutes of data. Visually it is apparent that often movement-bursts (black arrows) are associated with an increase in HR and a decrease in BR and SpO2 respectively.

IV. Discussion

We have developed and clinically tested a setup that can be used to non-obtrusively monitor body movement of preterm infants in clinical settings. Continuous monitoring of movement in preterm infants has many potential applications including the early detection of sepsis, seizures, apneas and sleep-wake cycles. Features based on the BSG waveform acquired from the EMFi sensor performed well for monitoring movement and outperformed the same features applied to the routinely acquired cardiorespiratory waveform. Not only do the features based on the BSG waveform yield the highest AUROC – a measure for quantifying effect size – but also the lowest variability from one measurement to the other (small SDs). The small SD points to repeatability in monitoring movement and indicates that, irrespective of external factors such as the position and weight of infants, movement monitoring with BSG was reliable.

Since the BSG waveform is acquired from a pressure sensor, it is not surprising that it is more effective than cardiorespiratory waveforms for tracking movement. Also, compared to the ECG, CI, and the PPG waveforms, the superior performance of the BSG waveform may be because, irrespective of which body part of the infant is moving, the sensor, as a result of being placed underneath the infant, is always sensitive to movement. The ECG-electrodes, which generate both the ECG and CI waveforms, may have limited sensitivity to movement in the lower extremities. Also, they might be more prone to noise due to poor electrode contact with the body. The PPG sensor, which is typically placed on one of the feet, may have limited sensitivity to movement in the upper body. Combining information from different sensors could prove useful in discriminating movement between the lower and upper extremities of the body. Another possible reason for the poorer performance of cardiorespiratory waveforms is that the corresponding sensors are not optimized to detect movement and may, in fact, have hardware or embedded software solutions to reduce motion artifacts. Nevertheless, while routinely monitored cardiorespiratory waveforms are less reliable than BSG for monitoring movement, they may still prove useful for certain applications such as longitudinally tracking movement.

Determining the best-performing features after pre-filtering the waveform signals – the BSG, ECG, CI, and PPG – to limit the upper-frequency content, revealed that for all waveforms, performance is best for an upper cutoff between 0.3-0.5 Hz, in accordance with the literature [13]. Increasing the cutoff frequency further reduces performance, likely due to a decrease in the SNR, because of increased breathing and heartbeat-related contribution. Increasing the upper cutoff frequency beyond 1 Hz reduces performance for all waveforms (data not shown). Overall, the best performing feature is the SII, closely followed by the HT. The HT-feature detects the instantaneous amplitude of the waveform and performs better than entropy and complexity-based measures, indicating that amplitude changes due to movement are more informative than changes in the signal’s regularity. Likely, the SII performs best because it combines both information based on signal amplitude, as well as its regularity. The battery of features used in this work are easily interpretable and perform well, but for future work, other features such as those based on the wavelet transform may be considered [38]. The data presented in this study uses a window length of 15s to calculate and evaluate the performance of the features. Depending on the intended application, the window length may also be reduced – for example, if sensitivity to sharper movements, such as startles, is required.

A remarkable finding, based on monitoring movement, as can be seen in Fig. 2 and Fig. 6, is that there were periodic movement-bursts that lasted 20–60s, but occasionally up to 2 minutes, and typically occurred every few minutes. These movement-bursts, in fact, occurred in all 20 recordings. Based on visual observation of the corresponding video-data, movement in these periods were large body movements of the head, trunk, and limbs. Perhaps, the nature of these movements is also the reason why motion detection based on the ECG, CI and PPG waveforms perform reasonably well on average, but with considerable variation in performance (large SD). Likely the overall superior performance of the BSG waveform is due to its ability to capture smaller movements, even if they are localized to a body part. The repetitive nature of the movement-bursts suggests a physiological origin. We hypothesize that these are reflective of the arousal state of the sleep-wake continuum as discussed in Werth et al. [14]. Occasionally in the 20 hours of recorded data, there were periods of tens of minutes without any movement bursts, and these might have corresponded to periods of quiet sleep [14]. This hypothesis of movement-bursts corresponding to arousals is strengthened by the positive correlation between movement bursts and increases in heart rate, as is expected during arousals [14].

The fact that motor response often precedes changes in vital signs can be exploited for the early generation of alarms – an important issue in the NICU – potentially increasing the window of opportunity for nurses to respond to critical alarms [39], [40]. Literature also shows that spontaneous movement can lead to various physiological perturbations, including increased oxygen consumption due to metabolic demands, movement-induced hyperventilation, and hypocapnia. These perturbations can lead to a destabilizing effect on breathing and therefore tracking movement can be an important marker for predicting apnea, which, if minimized, leads to neurodevelopmental benefits [12], [41], [42]. Further applications of monitoring gross body movement include its use in filtering heart rate variability and as a physiological marker to study growth and development in neonates [41], [43]. Polysomnographic studies may also benefit from a ballistographic approach for monitoring movement, since visually identifying movement is not easy, as supported by the inter-rater agreement obtained in this study. The challenges include infants being partially covered by blankets and low ambient visibility to minimize external stimuli to the infant – problems that persist in automated video-based actigraphy monitoring as well.

The unobtrusive and reliable solution for monitoring body movement of preterm infants in a clinical environment, as detailed in this study, can function in real time and may be useful in identifying both acute and longer-term clinical deteriorations. In the future, we will evaluate the potential of this BSG-based approach for monitoring breathing in the NICU setting.

V. Conclusion

This paper presents a system and method for monitoring movement in neonates using ballistography. The proposed approach is robust to noise and can be used in real time in clinical settings. Multiple features for identifying movement using the BSG waveform were evaluated and performed well when compared to visually observed movement and outperformed features based on routinely acquired cardiorespiratory waveforms. We recommend using the SII based on the BSG waveform for tracking movement.