Using sensor-fusion and machine-learning algorithms to assess acute pain in non-verbal infants: a study protocol

Abstract

Introduction

Objective pain assessment in non-verbal populations is clinically challenging due to their inability to express their pain via self-report. Repetitive exposures to acute or prolonged pain lead to clinical instability, with long-term behavioural and cognitive sequelae in newborn infants. Strong analgesics are also associated with medical complications, potential neurotoxicity and altered brain development. Pain scores performed by bedside nurses provide subjective, observer-dependent assessments rather than objective data for infant pain management; the required observations are labour intensive, difficult to perform by a nurse who is concurrently performing the procedure and increase the nursing workload. Multimodal pain assessment, using sensor-fusion and machine-learning algorithms, can provide a patient-centred, context-dependent, observer-independent and objective pain measure.

Methods and analysis

In newborns undergoing painful procedures, we use facial electromyography to record facial muscle activity-related infant pain, ECG to examine heart rate (HR) changes and HR variability, electrodermal activity (skin conductance) to measure catecholamine-induced palmar sweating, changes in oxygen saturations and skin perfusion, and electroencephalography using active electrodes to assess brain activity in real time. This multimodal approach has the potential to improve the accuracy of pain assessment in non-verbal infants and may even allow continuous pain monitoring at the bedside. The feasibility of this approach will be evaluated in an observational prospective study of clinically required painful procedures in 60 preterm and term newborns, and infants aged 6 months or less.

Ethics and dissemination

The Institutional Review Board of the Stanford University approved the protocol. Study findings will be published in peer-reviewed journals, presented at scientific meetings, taught via webinars, podcasts and video tutorials, and listed on academic/scientific websites. Future studies will validate and refine this approach using the minimum number of sensors required to assess neonatal/infant pain.

Trial registration number

ClinicalTrials.gov Registry (NCT03330496).

Strengths and limitations of this study.

• An innovative and objective approach for continuous pain monitoring in infants including term and preterm neonates is described.

• To discriminate between noxious and non-noxious events, we use sensor-fusion and machine-learning algorithms.

• Multimodal assessments may be more sensitive and specific for identifying pain and quantifying its intensity than the subjective assessments currently used in pain scales.

• Accurate, objective pain assessments may help reduce infant pain and suffering, enhance recovery, avoid untreated pain versus analgesic overuse, and allow evaluation of newer analgesics or other therapies in randomised clinical trials.

• Multiple sensors are used for the first step of this study, but recording artefacts may require data corrections and sensor variability may generate a need for recruiting more patients.

Introduction

Being non-verbal, hospitalised infants are particularly vulnerable to inadequate pain management. Repetitive pain in newborns leads to short-term and long-term neurodevelopmental consequences including behavioural and cognitive sequelae.^1–5 Conversely, the safety and efficacy of some analgesics in neonates and their negative consequences on the neonatal brain have raised concerns.^6–9 Objectively assessing the pain responses in infants is thus necessary to assess the efficacy of analgesics in infants in order to avoid overtreatment but also undertreatment and the consequences of repetitive pain exposure.

Composite pain scales including behavioural and physiological measures are the most widely used surrogate measures of infant pain and are currently recommended for the clinical practice.¹⁰ However, they provide a one-time measurement and their use can be challenging for the bedside staff leading to low inter-rater reliability, with overestimation or underestimation of infant pain.^{11 12} Depending on the context, behaviours or physiological responses may mirror non-noxious stimuli, leading to misinterpretation and a lack of specificity in subjective pain scales.^13–15

Pain from clinically required invasive procedures leads to well-described neurophysiological responses in term and preterm infants.¹⁵ These responses imply that the central, peripheral, and autonomic nervous systems can be monitored using various behavioural and physiological modalities.^16–19 Since pain is a complex process, multimodal measurement may improve the accuracy of pain assessment, also suggested by studies reporting the presence of pain-evoked potentials in some neonates showing no facial expressions of pain or others pointing out different profiles of pain responses.^{13 20}

Thus, developing new methods to assess the responses to pain in infants deserves a high priority. Multimodal measurements that provide an objective estimate of real-time and continuous pain monitoring at the bedside will avoid the subjective bias and limitations associated with clinical pain scales, especially when behavioural assessment is limited by the medical conditions.²¹

Our study protocol was designed to develop a multimodal pain assessment system, using sensor-fusion and novel machine-learning algorithms to provide an objective estimate of acute pain intensity in infants that is patient centred, context dependent and observer independent.

Methods and analysis

Study design

We designed a prospective observational study enrolling subjects from Lucile Packard Children’s Hospital at Stanford. The study does not involve any intervention other than those clinically required. The study started on 30th October 2017 and will be completed on 30th November 2025.

Study population

We will collect data from 15 study subjects in each of the following age groups: (1) preterm infants (34–37 weeks of corrected gestational age, postnatal age 3–30 days); (2) term newborns (37–42 weeks of corrected gestational age, less than 1 month of age); (3) infants from 1 to 3 months of age; (4) older infants from 3 to 6 months of age.

Eligibility criteria

After obtaining parental consent, we will include all infants less than 6 months of corrected chronological age requiring an acute painful procedure for routine clinical care in Lucile Packard Children’s Hospital.

We will exclude newborns with birth trauma, intrapartum asphyxia (5-minute Apgar score <4 or cord pH <7.01), fetal growth restriction (birth weight <5th percentile for gestation), congenital anomalies or metabolic disorders, or any kind of brain injury; if their mothers had a history of heavy smoking or drug abuse (alcohol, cocaine, ketamine and heroin/other opiates) or psychiatric drugs used during this pregnancy; infants requiring positive pressure ventilation using a face mask (Bilevel Positive Airway Pressure) or endotracheal tube; those receiving continuous infusions of opioid drugs (morphine, fentanyl and others) and nerve blocks or neuraxial analgesia affecting the site of the invasive procedure in the 24 hours prior to study entry; infants with facial anomalies (cleft lip), injuries or other pathologies affecting the facial area; and infants breast fed to alleviate pain during the painful procedure.

Objectives/outcomes

Primary objective

Our primary objective is to identify the specific signals and patterns from each sensor that correlate with the pain stimulus. This pilot study is designed to exclusively assess acute pain responses during routine, clinically required skin-breaking procedures—it measures the intensity of acute pain from the physiological responses of each subject.

Primary outcome

We will extract pain-related information using non-invasive multimodal sensors. Specific features of the physiological/behavioural indicators of infant pain will require unique processing algorithms. We will record pain signals using facial electromyography (EMG), ECG, electrodermal activity (skin conductance/SC), oxygen saturation (SpO₂) and electroencephalography (EEG) in real time. Dedicated algorithms for each sensor will extract pain-related information such as facial grimacing or heart rate variability (HRV). The reliability and validity of these algorithms will be tested prospectively on data from preterm and term neonates, and infants aged 1–6 months experiencing acute pain during invasive procedures.

Secondary objectives

We aim to identify if multiple sensors will provide overlapping information, which a sensor-fusion framework can integrate to identify ‘pain’-related and ‘no pain’-related features. These features will be used to train machine-learning algorithms that will finally provide reliable, objective assessments of pain intensity in real time.

We will also study if the pain intensity measured by the sensor-fusion framework will show clinical validity, inter-rater reliability, as well as responsiveness to pain relief using analgesic drugs or non-pharmacological therapies. However, no interventions are planned in this study.

Secondary outcomes

We will develop a sensor-fusion framework designed to integrate data from different sensor modalities. No single sensor is capable of measuring neonatal pain. Rather, skilled clinicians draw on multiple sources of information to estimate pain. A machine-learning algorithm will be developed to test if the sensor-fusion framework (1) can ‘calibrate’ itself to the unique physiology of each newborn, (2) handle missing (eg, sensor failure) or unreliable data (eg, movement artefact), and (3) determine specific features from each modality to reach asymptotic levels of sensitivity and specificity. We hypothesise that this automated sensor-fusion approach will be able to estimate neonatal pain intensity with greater specificity and sensitivity than the pain scales clinically used at the bedside.

To assess the reliability and validity of the pain intensity measured by the sensor-fusion framework, we will compare the objective pain measure with the pain scores assessed by skilled research staff. Variations in the objective pain scores before and after the clinical use of analgesic therapies will also be assessed to explore whether this device can also identify pain relief.²²

Sample size calculations

We assume an α-error=0.05, 1–β error=0.8, and a mean:SD ratio of 2:1 for clinical and sensor-based pain scores (based on prior studies of clinical assessments of procedural pain in infants).^{23 24} Based on these assumptions, to detect a slope that corresponds to a 0.5-unit change in the outcome (eg, sensor pain score) per 1-unit change in the predictor (eg, clinical pain score), we will require a minimum of 40 infants in the training dataset. For a binary predictor with 25% or 50% prevalence (eg, mild vs moderate pain, or male vs female neonate), with this sample size, we will be able to detect a 0.7-unit or 0.8-unit change in the outcome, respectively. Given the number of sensors used for the first step of this study, some artefacts may occur requiring data corrections or greater variability, with a need for recruiting more patients. We plan to recruit 60 patients in this study.

The sample size calculated for the first step of the study may represent a challenge for machine learning by limiting samples sizes for the training and testing datasets. Therefore, the results provided by our analyses will be confirmed in larger sample sizes within the next steps of the study.

Interventions/experimental design

After applying inclusion/exclusion criteria and parental consent, infants' medical data will be recorded including: date of birth, perinatal/medical history, birth weight, gestational age, Apgar scores, congenital anomalies, metabolic defects, other diagnoses, previous surgeries, recent labs, prior imaging, major physical findings, number and types of painful procedures, and all medications used in the 24 hours immediately preceding the study. We will document the type of procedure, time of day, its location and duration, number of attempts and behavioural state of the infant before and after the procedure.

Prior to a clinically indicated procedure such as a heel stick, subcutaneous or intramuscular injections (vaccine, drug shot), we will attach SC leads to measure galvanic skin responses either on a hand palm or foot sole, and EMG to record facial muscle activity from cheek and forehead, and directly obtain recordings from clinical monitors (ECG and SpO₂). In addition, the infant will wear a cap with EEG leads. For study procedures, research staff will clinically assess pain using the Neonatal Pain and Sedation Scale (N-PASS) before and after the procedure, the Premature Infant Pain Profile-Revised (PIPP-R) and the Neonatal Facial Coding System (NFCS) during the procedure. For older patients, the FLACC (Face, Legs, Activity, Cry, Consolability) Scale or Visual Analogue Scale (VAS) will be scored to assess pain during the procedure.^25–28 Physiological and audio recordings will start 10–30 min before a planned procedure and continue for up to 20–30 min after the procedure.

All sensors will be monitored and displayed on the same laptop. We will use the Brain Vision software to display and record EEG, EMG and SC responses and the MediCollector software for ECG and SpO₂. The recording time of the two softwares will be synchronised based on the laptop digital clock. The event marker will be triggered by the researcher using a dedicated function of the Brain Vision software to time lock and record times of noxious and non-noxious events for all sensors (Brain Vision and MediCollector).

At the completion of recording, all the above sensors will be disconnected from each patient and study procedures will be terminated. The entire study will last approximately 30–60 min; however the study may be stopped earlier if any infant shows signs of distress or if a bedside nurse or parent has any concerns.

Data collection includes physiological recordings and medical information. Other than the facial EMG, SC leads and 32-channel BrainAmp EEG data acquisition system, all other sensors are used routinely as the standard of care. All sensors are non-invasive and safe. We will use standard electrodes which are routinely used for recording vital signs in the hospital (ECG, SpO₂). All recordings will use sticky pads attached to the skin. Other than a potential for mild skin irritation from the adhesives used, there are no significant risks associated with these devices, or other study-related procedures.

Data analysis

Signal-filtering and information extraction from sensors

For each sensing modality, we will develop a software algorithm to process signals recorded by sensors and extract the clinically relevant information related to pain. Proof-of-concept algorithms will be tested using the collected dataset. The feasibility of each sensing modality will be based on: (1) sensitivity and specificity of detecting clinically relevant pain-related changes, and (2) robustness of sensor readings in the presence of non-pain-related disturbances (eg, movement artefact).

In order to address artefacts due to movement or suboptimal electrode-skin contact, we will initially use filtering techniques (eg, to remove power line interference). In addition, we will identify channels exhibiting artefacts by considering the range of signal values, where signals showing extreme deviation from average values or channels showing virtually zero activity will be excluded. Specifically for EEG analysis, we will identify and remove EMG-related artefacts using well-established techniques such as filtering and independent component analysis (ICA).

Windows of different lengths will be used for the analysis. Specifically, for ECG, SpO₂, EMG and SC signals an analysis window of 1–5 min will be used to extract appropriate features. The window lengths ranging from 400 ms to 5 s will be used for EEG signals.

Pain behaviours

Newborn behaviours (eg, facial expressions, body movements) are accepted as the most sensitive and valid indicators of pain.^{13 29} Facial expressions like brow bulge, eye squeeze, nasolabial furrow, and horizontal mouth stretch were verified as the most valid and discriminative components of neonatal pain scales.^30–32 Using facial EMG in real time, we will detect the presence of muscle activity in neonatal pain-associated facial movements.³³ Given the multiple overlapping layers of facial muscles, facial EMGs record signals from a facial region as opposed to any specific muscle.³³ Previous studies of startle and blinking in infants used miniature silver (Ag/AgCl) periorbital surface electrodes for recording EMGs.^34–36 We will focus on infant forehead and cheek areas to detect EMG activity associated with brow bulge, eye squeeze and nasolabial furrow.³³ We will iteratively refine our algorithms by using multimodality sensing and developing robust feature extraction and classification frameworks that address the challenges specific to neonatal/infant pain detection. We will exclude mechanically ventilated infants due to the challenges associated with identifying facial features (occluded by securing tape, ventilator tubing or devices) and their need for ongoing sedation/analgesia.

Skin conductance

Acute pain stimulates the sympathetic post-ganglionic cholinergic neurons,³⁷ leading to diaphoresis, palmar sweating and increased SC.³⁸ Eliminating painful stimuli results in sweat reabsorption and decreased conductivity. The amplitude of changes in palmar SC reflects increased sympathetic nervous system activity, which tracks with pain intensity.^39–42 SC can change with body temperature,^{43 44} but not with the ambient temperatures.⁴⁴ Specifically, the number of fluctuations of SC per second (NFCS) was correlated with pain intensity in children,⁴⁵ and was more sensitive than pain scores in preterm and term neonates.18 40–42 46 We will use SC using the BrainAmp system (Brain Products GmbH, Gilching, Germany),

Electrocardiography

HR changes are components of many pain assessment scales and recent studies have established correlations between HRV and pain.^{47 48} A number of linear time-domain (HR mean, SD) and frequency-domain (power spectral density) metrics and non-linear metrics (sample entropy, approximate entropy and others) can detect painful stimuli.^{47 48} We will record the infant’s ECG before, during, and after an acute pain event to extract the linear and non-linear metrics (listed above) from the ECG signal for further analyses.

Electroencephalography

EEG studies to assess neonatal pain have met with variable results.^49–51 Opioid analgesia in adults leads to slowing of the EEG, whereas painful stimuli activate brain regions identified by neuroimaging studies⁵² such as the primary sensory cortex (S1).⁵³ EEG amplitudes and frequencies decreased when analgesics were given to newborns in pain.^{20 54–56} Using the BrainAmp EEG system (Brain Products GmbH, Gilching, Germany), we will apply 32 active EEG electrodes using the infant-sized ActiCap (Brain Products GmbH, Gilching, Germany). Although Hartley et al have selectively used the vertex (Cz) lead for neonatal pain studies,^{20 54} we believe that infant pain processing is widely distributed across many brain regions and the current evidence is not sufficiently strong enough to exclude information from other EEG leads. The BrainAmp is similar to other EEG monitors, however, it uses 32 active electrodes allowing for placement of the ActiCap on the infant’s head with minimal preparation. Each active electrode amplifies the signal recorded from the skin and records also indicate the impedance of each electrode at the start of the recording to improve the quality of recorded signals. Over the past 10 years, this device has been used for research purposes in all age groups including infants and newborns. No side effects were reported from its use in newborns and small infants.

In order to analyse EEG signals and extract appropriate features, we will first remove noise and artefacts using standard techniques such as ICA and wavelet denoising.⁵⁷ After artefacts are removed, we will investigate the correlation between features extracted from EEG data and pain. Specifically, we will use spectral decomposition and extract features such as mean power in different frequency bands (delta, theta, alpha and beta) as well as asymmetry measures for each homologous pair and functional connectivity measures for further investigation.

Pulse oximetry (SpO₂)

Changes in SpO₂ occur frequently following acute pain and, therefore, have been included in the PIPP and other pain scales.^{23 58 59} Newer generation monitors (Masimo, Irvine, California, USA) use multiwavelength technology to provide more reliable SpO₂ and pulse rate signals, with parallel signal processing engines and adaptive filters to separate the arterial from venous signals, patient motion or skin perfusion.⁶⁰ Changes in skin blood flow were also used as physiological markers for neonatal pain or morphine analgesia.^{61 62} We will test the utility of the SpO₂ and peripheral perfusion index provided by pulse oximetry monitors as possible signals for neonatal pain.

Statistical approach

Sensor fusion

We have previously investigated using machine learning to detect pain in neonates using facial expressions recorded by a camera.⁶³ We will develop a sensor-fusion framework to detect pain in non-verbal infants based on machine learning to detect pain using multimodal sensor data.

Feasibility of this new framework will be assessed based on its sensitivity and specificity to detect pain events in infants and further refined into a prototype for validation in future studies.

A ‘calibration’ period will be used to establish a baseline for these multiple sensor modalities by monitoring neonates who are not in pain. The clinical staff at the bedside will identify the pain state of each neonate/infant using validated pain scales (N-PASS, NFCS and PIPP-R; FLACC, VAS) and record the timing of pain-inducing clinically indicated procedures. Our sensor-fusion framework will classify the neonatal/infant responses to infer pain intensity based on observed changes from baseline. A probabilistic relationship between pain intensity and sensor measurements can be established, where the unknown parameters of the statistical relationships are identified by a training dataset. The training dataset will also be used to estimate the importance of each feature, which can then be used to identify the optimal set of sensors.⁶¹

Gestational age, postnatal age, and days of life and in hospital will be taken into account in the statistical analyses. We will initially focus on recruiting term neonates who are studied within 1 week after birth and have minimal exposures to prior painful events. This will increase the homogeneity of our sample and minimise the variability in physiological responses due to gestational age, postnatal age, days in the hospital and long-lasting effects of previous painful experiences.

Interventions to manage pain will be allowed including non-pharmacological and pharmacological treatments apart from continuous infusions of opioid drugs. This will be considered in the statistical analysis.

Pain intensity scores computed by our sensor-fusion framework will be compared with pain scores measured concurrently by skilled research staff. All sensors (facial EMG, EEG, ECG, SpO₂ and SC) will be connected with an event recorder to mark ‘pain’ versus ‘no-pain’ states. To make the best use of our data, the sensor-fusion framework will use standard cross-validation methods to establish the generalisability of this framework.

The extracted features from each modality will be used to train a machine-learning algorithm. Specifically, we will train a binary classifier to assign ‘pain’ and ‘no pain’ class labels based on the extracted features. We will specifically investigate using the random forests classifier given their robustness to outliers and its classification performance when a large number of features are used for classification.

Data from patients will be divided to a training set and a test set. The training set is used for model training and optimisation of model parameters. A leave-one-patient-out cross-validation technique will be used, where the machine-learning classifier is trained on data from all but one patient and the performance of the classifier is assessed on the remaining patient. Once the appropriate machine-learning classifier and its associated parameters are selected using the training set and the associated cross-validation procedure, the performance of the machine-learning classifier will be assessed on the test set.

Validation and correlation with pain intensity

We will compare clinical pain scores from nursing assessments with scores from the sensor-fusion framework. First, we will examine clinical pain scores to verify agreement with the pain scores assessed by the research staff. Internal consistency will be evaluated by Cronbach’s α, with values >0.8 to show good internal consistency. Second, we will conduct multivariable linear regressions or generalised estimating equations (GEE)^{64 65} to understand the agreement between the device pain scores and the clinical pain scores, as well as the contribution of each modality to the device pain scores. We will examine if these associations vary after adjustment for covariates such as pain medications, age, sex, duration or invasiveness of the procedure. A covariance matrix will examine the degree of correlation between individual sensor inputs, types of procedures, clinician pain scores and analgesic therapies used during the procedure. Finally, due to the limited understanding of factors contributing to pain in newborns, linear regression or GEE models will examine the association of the sensor-fusion pain scores reported by the device with the demographic and clinical variables of neonates and infants.

Content validity depends on the sensors and sensor variables that we have chosen for the sensor-fusion framework. Concurrent validity will depend on the pain scores of skilled research staff using validated pain scoring methods. Construct validity will rest on: (1) the range of objective pain scores from procedures causing mild, moderate or severe pain; (2) changes in pain scores with analgesic drugs or non-pharmacological therapies; and (3) variation in pain scores over time consistent with the expected, natural course of acute procedural pain.

We will also develop a machine-learning algorithm to predict subjective pain. As part of the validation, we will evaluate the machine-learning pain assessment algorithm which has been trained on clinical classification of pain based on validated pain scales and compare the results with the results provided by the machine-learning pain assessment algorithm which has been trained on data involving objective pain events (eg, heel stick).

Future studies will test the validity of this approach to pain assessments in larger populations of newborns, older infants and also extend these studies to smaller preterm neonates. Future applications may also include patient populations incapable of expressing pain (children with disability, adults with dementia or mechanically ventilated patients).

Ethics and dissemination

The Institutional Review Board of the Stanford University approved the protocol (Protocol #39076). The ethics approval includes anonymity and written consent will be provided by the parents.

Study findings will be published in peer-reviewed journals and presented at national and international scientific conferences. Practical use of this methodology will be taught at conference workshops, or via webinars, podcasts and video tutorials.