Behavioral Observation of Infants with Life-Threatening or Life-Limiting Illness in the Neonatal Intensive Care Unit

Abstract

Background:

Infants in the neonatal intensive care unit experience aversive stimuli that cause pain and distress. Maintaining adequate relief from pain and distress is challenging because of infants’ varying ages and stages of development and their nonverbal status. Thus, pain and distress must be interpreted by a health care provider or other proxy from their own observations or perceptions. There is no standard research or clinical measure for pain and distress in infants.

Objective:

The purpose of this analysis was to evaluate the use of the COMFORT-Behavior Scale as a measure of pain and distress in infants diagnosed with life-threatening or life-limiting illness in the neonatal intensive care unit in comparison with the nurse-documented Neonatal Pain, Agitation and Sedation Scale, the infants’ Technology Dependence Scale, and the mothers’ report of total perceived symptom scores.

Methods:

Infants diagnosed with life-threatening or life-limiting illnesses and hospitalized in a level IV neonatal intensive care unit in the Midwestern United States and their parents were included. Measurement of pain and distress in infants was collected weekly from enrollment through 12 weeks or until discharge whichever occurred first. Observations for the COMFORT-Behavior Scale were conducted before and after standard caregiving activities. Pearson r correlations were used to compare means between pain and distress, technology dependence, and mothers’ total perceived symptom scores over time.

Results:

Data from seventy-eight infants (46 males, 32 females) ages 23–41 weeks gestation at birth were analyzed. No correlations were found among the COMFORT-Behavior Scale; Neonatal Pain, Agitation, and Sedation Scale; and mothers’ total perceived symptom scores. Moderate correlations were found among the Technology Dependence Scale; mothers’ total perceived symptom scores; and the Neonatal Pain, Agitation, and Sedation Scale.

Discussion:

Performing COMFORT-B observations can be challenging, and it is unclear whether the information obtained from the COMFORT-B added to the assessment of the infant’s pain and distress that is typically recorded in the health record or from parents. Further evaluation is needed to determine if it is more reliable to collect the Neonatal Pain, Agitation, and Sedation Scale scores rather than conduct observations using the COMFORT-Behavior Scale in studies of infants with life-threatening and life-limiting illness in the NICU.

Every year in the United States about 4 million infants are born with more than 500,000 of these infants being subsequently admitted to a neonatal intensive care unit (NICU) due to birth defects, prematurity, or problems associated with delivery (Bettegowda et al., 2014). Advances in technology have allowed for the survival of infants who in the past would have died soon after birth (Marc-Aurele & English, 2017). These very ill infants are stabilized and then may undergo treatment over days, weeks, or even months depending on gestation and underlying condition. During this time, critically ill infants can experience pain or other non-pain related distress that is difficult for clinicians to assess and manage and for parents to comprehend, causing emotional distress (Shultz et al., 2017).

Relief from pain and distress is a critical component of care for an infant admitted to a NICU. Much progress in research has been made over the last 30 years in regard to understanding of infant experiences of pain that has described underlying mechanisms, epidemiology, and effects on the brain and infant development, as well as advancing guidelines for assessment, interpretation, and management (Anand, 2017; van Dijk & Tibboel, 2012). However, despite research regarding pain (Anand, 2017) and pediatric palliative care (Institute of Medicine [IOM], 2003; 2014), the total symptom experience, including non-pain-related distress of infants with life-threatening (LT) and life-limiting (LL) illnesses, is still not well described. Issues such as variations in the expression of symptoms by gestational age and the infant’s nonverbal status have led to underlying gaps in our knowledge of the symptom experience and symptom assessment in these vulnerable infants (Boyle et al., 2018).

Accurate assessment of both pain and non-pain-related distress is essential to ensure effective clinical management of the infant as well as to monitor the effectiveness of interventions given to reduce pain and/or distress (Harris et al., 2016). The Synactive Theory of Development, which guides much of NICU practice, proposes that infant behavior indicates distress more objectively than opinions of health care providers and parents (Als, 1982; Pritchard et al., 2010). Most behavioral pain scales score multiple behavioral aspects including the infant’s facial expression, vocalizations, and body movements; these scales are used frequently in clinical setting to assess pain or distress in nonverbal patients.

Seminal work by Pillai Riddell & Racine (2009) and Walden et al. (2001) identified several factors that may affect the accurate identification of infant symptoms by health care providers and parents. First, the infant must be able to effectively relay the cues. Then, the health care provider or the parent must interpret those cues correctly. This interpretation requires knowledge of normal neurodevelopment and pathophysiology, experience with the infant, as well as intuition (Pillai Riddell & Racine, 2009). That some infants might exhibit few obvious cues (Walden et al., 2001) or that one physiologic indicator may indicate a number of symptoms, may further complicate accurate assessment. For example, crying can be evidence of pain, agitation, a soiled diaper, hunger, or a need to be held. Further, the subjective nature of symptom assessment in infants may lead both health care providers and parents to subconsciously infer that an infant is experiencing pain or other non-pain related distress, as well as the perceived causal factors, based on their own experiences. Although accurate assessment of pain and distress is the first step toward symptom management, there has been debate about the quality and accuracy of the tools that are most frequently used for these assessments in clinical settings (Maaskant et al., 2016).

Tools used in the NICU for the assessment of pain and/or distress may be chosen based on the infant’s gestational age or clinical status or NICU preference (Cong et al., 2013). For example, the Face, Legs, Activity, Cry, and Consolability (FLACC), the CRIES (C-Crying; R-Requires increased oxygen administration; I-Increased vital signs; E-Expression; S-Sleeplessness), the Neonatal Infant Pain Scale (NIPS), the Premature Infant Pain Profile (PIPP), and the Behavioral Assessment Scale of Newborn Individual Development Care and Assessment Program (NIDCAP) are widely used in the NICU to assess infant pain and distress. However, these tools measure varying characteristics of pain and most have not been compared with other behavioral measures or with other pain indicators, such as physiological measures. Another commonly used tool, the Neonatal Pain, Agitation, and Sedation scale (N-PASS), is often used to assess both acute and chronic pain, agitation, and sedation in all ages of infants, including preterm infants. Also, while these standardized assessment tools have been developed for infant symptom assessment, they still contain subjective components that may limit assessment objectivity. Typically, little training or competency maintenance is provided for NICU staff who complete the assessments. Thus, variability is inherent in how the assessment scores are obtained and documented by the many different nurses who care for infants throughout their hospitalization. While some of these scales have been used in research studies, they were developed primarily for use in the clinical setting and not for research purposes. Moreover, clinical assessments, while typically documented in the electronic health record (EHR), may be missing entries, making this source of data particularly problematic for researchers (Fortney & Steward, 2015).

The Comfort-Behavior (COMFORT-B) Scale is an adaptation of the COMFORT scale initially validated in 1992 by Ambuel and colleagues. The goal of the original COMFORT scale was to provide a research measure that would be sensitive not only to acute pain, but also to more subtle signs of distress (Ambuel et al., 1992). The original scale included the measurement of both physiologic and behavioral criteria. Behavioral distress was defined as encompassing all negative effects of pain and distress, as well as distress that may occur in the absense of pain (van Dijk et al., 2009). Following evidence demonstrating that measurement of physiologic data could be omitted and still provide a valid assessment of pain and distress, the COMFORT-B scale was developed, including only behavioral assessment (van Dijk et al., 2009). The COMFORT-B scale offers a formalized training protocol for observers that is advantageous for use in research. However, the COMFORT-B has not been used in research with infants with life-threatening or life-limiting conditions.

As part of a larger study, the purpose of this analysis was to evaluate the use of the COMFORT-B scale as a measure of pain and distress in critically ill infants with LT and LL illness in the NICU. The COMFORT-B scores were compared to the nurse-documented N-PASS, which was used by nurses in the research setting, the Technology Dependence Scale (TDS), which was developed as a proxy measure for painful or distressing exposures, and mothers’ report of total perceived symptom (TPS) scores.

Methods

Procedures and Participants

Following IRB approval infants and their parents who met inclusion criteria were enrolled in a prospective, longitudinal study in a large, Midwestern, level IV, regional referral NICU between March of 2017 and June of 2019. All eligible infants enrolled in the study had a) been diagnosed with at least one potentially life-threatening or life-limiting condition on or during admission to the NICU including congenital heart disease, multiple congenital anomalies, life-limiting genetic syndrome, surgical necrotizing enterocolitis, hypoxic ischemic encephalopathy at Sarnat Stage ≥ 2 requiring mechanical ventilation with multiple organ system involvement and/or seizures, congenital diaphragmatic hernia, overwhelming sepsis with multiple organ failure, and extreme prematurity of ≤ 27 weeks’ gestation requiring >30% FiO2 with significant comorbidity; or b) received a palliative care consult while hospitalized; and c) were born ≥ 23 weeks’ gestation; and d) had a parent over 18 years old who could speak and read English. Infants with neonatal abstinence syndrome or who had a positive drug screen were excluded from the study as these infants often exhibit signs that complicate the understanding of symptoms.

Data Collection Procedures

Written informed consent was obtained from parents prior to the start of data collection. Pain and distress in infants were measured using the COMFORT-B, N-PASS scores, the TDS, and mothers’ TPS score, which was collected by paper-and-pencil survey. Measurements began at enrollment when one of four trained NICU research nurse observers performed a baseline (week 0) COMFORT-B observation before and after a scheduled infant care time. Additional COMFORT-B observations were conducted each subsequent week (1–12) at the infant’s bedside until death, discharge, or a maximum of 12 weeks. Completion times for the COMFORT-B observations ranged from 15 minutes to two hours for a single observation depending on how the infant was tolerating caregiving and/or whether there were additional therapies the nurse needed to provide during that particular caregiving episode.

NICU research nurse observers watched training videos, completed practice observations at the bedside with the principal investigator, and completed inter-rater reliability checks prior to data collection. Interrater reliability of nurse observers administering the COMFORT-B was assessed using weighted kappa statistic, Cohen’s kappa. This method assesses agreement for numerical ratings and adjusts for chance agreement. Interrater reliability between the four observers was assessed prior to study start, after 25 participants were enrolled, and again after 50 participants were renrolled with weighted kappa’s ranging between .76 and .93, indicating moderate to almost perfect agreement (McHugh, 2012).

N-PASS scores (recorded by bedside nurses during the same caregiving episode as the COMFORT-B observation) and technology dependence score data (based on the previous week’s exposure) were extracted from the EHR. Mothers’ TPS scores were obtained weekly and were not temporally in relationship to the other measures.

Measures

The COMFORT scale has been found to be valid and reliable as a measure of pain and distress in children and infants 0–3 years old (van Dijk et al., 2000). Associations between the COMFORT scale and physiological measurements (i.e. heart and respiratory rates) were tested in the same sample and positive correlations were found. These relationships were not affected by physical health condition (van Dijk et al., 2001). In 2005, the COMFORT scale was adapted to the COMFORT-B to add “crying” as a category to enable the use of the tool in infants who are spontaneously breathing (van Dijk et al., 2005). On the COMFORT-B scale, “crying” is included in the respiratory subscale and is not scored for infants receiving mechanical ventilation (van Dijk et al., 2005). Instead these infants are scored according to their respiratory response during observation.

The COMFORT-B consists of six behavioral categories (muscle tone, facial tension, alertness, calmness/agitation, respiratory response, and physical movement). Each category uses a behaviorally-anchored interval rating scale scored 1 to 5 to produce a total score of 6 to 30 (van Dijk et al., 2009; Veopel-Lewis et al., 2010). The lower the score, the lower the pain and distress level; conversely, the higher the score, the higher the pain and distress level. The measure was found to have adequate interrater reliability (Κ = .79 – .96), internal consistency (α = .84 – .99), and moderate construct validity (r = .42 – .59) for measuring distress in the NICU population (Maaskant et al., 2016).

The N-PASS scale was developed as a non-invasive assessment tool of both acute and prolonged pain, agitation, and sedation in postoperative and mechanically ventilated infants in the NICU, including those born preterm (Hummel et al., 2008). Five criteria are graded 0, 1, or 2 for pain/agitation and 0, −1, or −2 for sedation. The N-PASS is currently the only tool that has been used to assess sedation in critically ill neonates. It has been found to be valid and reliable for infants with acute pain (Hummel et al., 2010) as well as those with chronic post-operative pain (Hummel et al., 2008). Its use in a broader population of non-ventilated or sedated infants for routine assessment of pain has also been established (Hillman et al., 2015).

The Technology Dependence Scale (TDS) is a validated measure of an infant’s medical complexity and can identify changes in technology dependence over time, in different settings, and across phases and types of illness (Docherty et al., 2011; Vance et al., 2020). Scores are obtained by summing items across 11 categories of technology support: care environment, invasive lines, nutrition, monitors, blood draws, respiratory assistance, skin care, specialized oral care, external drains/catheters, colloid therapy, and medications. Items are given scores from 1 to 5 with the higher score given to the most technologically burdensome (i.e. invasive lines). The higher the score, the more technological dependence the infant is deemed to have. Scoring in this study was based on data extracted from the infant’s EHR.

Mothers’ total reported symptoms scores (TPS) were obtained weekly using an investigator-developed Parent Perceptions of Infant Well-being survey that was based on literature review and expert clinician guidance (Shultz et al., 2017). Six symptoms identified in previous work as experienced by infants and distressing to parents (respiratory distress, pain, agitation, skin swelling/breakdown, lethargy, and feeding intolerance) were assessed to create a total symptom score. Mothers were asked to report on the presence (yes/no) of each of the 6 symptoms. If present, then mothers were asked to rate on a 4-point Likert scale the severity (1=slight, 2=moderate, 3=severe, and 4=very severe) and the frequency (1=almost never, 2=sometimes, 3=a lot of the time, and 4=almost always) of the symptom. Mothers were also asked to rate how much each present symptom bothered the infant, as well as herself, on a 5-point Likert scale (0=not at all, 1=a little bit, 2=somewhat, 3=quite a bit, and 4=very much). Data were also collected about whether treatment for the symptom was received (yes/no) and how well that treatment worked (0=not at all, 1=a little bit, 2=somewhat, 3=quite a bit, and 4=very much).

Statistical Analysis

Demographic data and study variables were analyzed with descriptive statistics for means, standard deviation, and frequencies. Pearson’s correlation was used to compare associations among the COMFORT-B, N-PASS, TDS, and mothers’ TPS scores. All variables were tested for normality prior to analysis. Weekly correlation analysis was conducted for all four measures and little variability was found across weeks taking the standard error into account. Therefore, we conducted our final correlational analysis using means for each participant across time for each of these four measures. Visual inspection and descriptive analysis of the COMFORT-B and N-PASS data revealed that for the vast majority of participants after week 4, scores decreased and remained low for the remaining weeks, thus not producing statistically meaningful results. For the purposes of this paper, we chose to analyze study measures for the first five weeks only (weeks 0–4).

Results

For this analysis, 78 infants (46 males, 32 females) ages 23–41 weeks gestation, and 76 mothers were included in the analyses. Demographic data can be found in Table 1. We completed 288 COMFORT-B observations during weeks 0–4 for each infant. Mothers of the infants completed 279 symptom surveys; while 281 N-PASS scores and 280 TDS scores were computed from data collected from the EHR. Pearson’s r correlations, displayed in Table 2, revealed no correlations among the COMFORT-B scores, mothers’ TPS, or nurse-documented N-PASS scores. There was a small negative correlation (r = −.183, 95% CI [−.442, .108]) between the TDS and the COMFORT-B. There were moderate correlations between the TDS and mothers’ TPS scores (r = .353, 95% CI [.108, .553]) and between the TDS and nurse-documented N-PASS scores (r = .401, 95% CI [.122, .628])

Table 1.

Infant Demographics (n=78)

Variable	n	%
Mothers	76	100
Infant Sex
Females	32	38
Males	46	61
Gestational Age
≤ 28 weeks (extremely preterm)	17	21.8
29–38 weeks (pre-term)	49	62.8
39–40 weeks (full-term)	11	14.1
≥ 40 weeks (post-term)	1	1.3
Race
White	51	67.1
Black/African American	19	25
Other	6	7.9
Ethnicitya
Non-Hispanic or Latino	71	93.4
Hispanic or Latino	5	6.6
Mothers’ Education
Some High School	3	3.9
High School Graduate	22	29
Trade School	4	5.3
Some College (1–3 yrs)	26	34.2
Bachelors (4 yrs)	12	15.8
Graduate (4+ yrs)	9	11.8
Total Household Incomeb
≤$25,000	23	31.1
$25,000 to $50,000	29	29.2
$50,001 to $75,000	12	16.2
$75,001 to $100,000	3	4.1
>$100,000	7	1
Prenatal Diagnosis of Severe Illness
Yes	35	44.9
No	43	55.1
Primary Diagnosis at Admission
Prematurity	30	39
Congenital Anomalies/CHD	16	20.8
Respiratory	15	19.5
Otherc	16	20.7

Note. CHD = congental hearth disease.

^aMissing data for 2 infants; mothers’ race and ethnicity used to define infants’ race and ethnicity.

^bMissing data for 4 infants

^cOther diagnoses include hypoxic ischemic encephalopathy (HIE), congenital diaphragmatic hernia (CDH), and neurologic conditions.

Table 2.

Pearson’s r correlation results.

	Mother TPS	N-PASS	COMFORT-B	Tech Burden
	r [CI]	r [CI]	r [CI]	r [CI]
	p value	p value	p value	p value
Mother TPS	1	.208	.063	.353
		[.040, .384]	[−.194, .308]	[108, .553]
		.117	.639	.007
N-PASS	.208	1	.080	. 401
	[.040, .384]		[−.273, .507]	[.122, .628]
	.117		.552	.002
COMFORT-B	.063	.080	1	−.183
	[−.194, .308]	[−.273, .507]		[−.442, .108]
	.639	.552		.168
Tech Burden	.353	. 401	−.183	1
	[108, .553]	[.122, .628]	[−.442, .108]
	.007	.002	.168

Note: Mother TPS = Mother’s total perceived symptoms; N-PASS = Neonatal Pain, Agitation and Sedation Scale; COMFORT-B = COMFORT Behavioral Scale; Tech = technology; r value, Pearson’s correlation; p value, 2-tailed (95% level); CI = confidence interval (95% level).

Discussion

The purpose of this analysis was to evaluate the use of the COMFORT-B scale as a measure of pain and non-pain-related distress in critically ill infants in the NICU as compared to the nurse-documented N-PASS, the infant’s level of technology dependence, and the mother’s report of total perceived symptom scores. While no relationships were found among the COMFORT-B score, mothers’ TPS, or nurse-documented N-PASS scores, positive correlations were found between the TDS and mothers’ TPS scores, and the TDS and nurse-documented N-PASS scores. The COMFORT-B was inversely correlated with the TDS score.

It was surprising that no correlation was found between COMFORT-B and N-PASS scores given that the two scales measure similar constructs. However, the COMFORT-B data were collected by trained research personnel and the N-PASS data were collected by bedside nurses, who may have scored infant behaviors much differently than research personnel. We might expect that independent scoring by trained research staff may provide more accurate information about about infant pain and distress. However, this is an untested assumption and reasons for this lack of correlation remain unknown. It may be beneficial to further analyze both measures to determine if it is more feasible, reliable, and cost-effective to collect the N-PASS scores rather than conduct observations using the COMFORT-B for future studies of critically ill infants with LT and LL illness in the NICU. Further evaluation of the value of the N-PASS tool for research is also warranted.

It is unclear whether the COMFORT-B observations added any additional information to the assessment of the infant’s overall pain and non-pain-related distress. The benefit of using the COMFORT-B in this study was that it was an additional non-invasive measure of pain and non-pain-related distress for critically ill infants experiencing life-threatening or life-limiting illness that had a training component for research staff. However, several challenges were encountered while performing COMFORT-B observations in the NICU setting, including that it could be very time consuming. Given the busy nature of the NICU, scheduling COMFORT-B observations was difficult, especially when trying to conduct the observations both before and after usual care assessments. Observation times were pre-arranged with nurses, however sometimes nurses forgot they had scheduled the observation, the infant woke early, or the parent wanted to hold the infant or participate in caregiving activities at the scheduled time. Rescheduling added to the time necessary for completion of weekly observations. Further, completion of pre- and post-care COMFORT-B observations could take anywhere from 15 minutes to 2 hours depending on how much care the infant required during that caregiving episode. Research costs were incurred for observer training time, conducting behavioral observations, and performing inter-rater reliability checks throughout the study. Because there was little association between the COMFORT-B and any of the other measures over weeks 0–4, it is unclear whether the COMFORT-B scale added any additional information regarding the infant’s total symptom experience.

The N-PASS, which is widely used clinically, is administered several times per day by bedside nurses, usually with every scheduled caregiving episode. Thus, the N-PASS scores summarize a longer time period during which symptoms could be observed. However, bedside nurses receive minimal training in scoring the N-PASS and retraining is infrequent if done at all, and there is usually no assessment of interrater reliability between nurses. Thus, using N-PASS scores recorded by bedside nurses in research is of questionable value. Moreover, since the N-PASS is completed by bedside nurses who may change often, there may be little consistency in observations among nurses. There are also documented issues with using the EHR for research purposes (Fortney & Steward, 2015), which could mean that use of the N-PASS scores documented in the EHR have the potential to be biased, unreliable, or missing altogether.

The relationship between the TDS, a measure of invasive procedures and technology experienced by an infant within a 1-week time frame, and mothers’ TPS scores may be related to mothers’ perception of the level of technology and invasive procedures used in the NICU. That is, it is possible that when technology used in the care of her infant increased, so did a mother’s perception that her infant was experiencing increased pain and distress. It is also possible that the correlation between the TDS and the N-PASS scores could be a result of the increasing pain and distress an infant experiences with an increase in need for technological therapies to treat their illness.

Limitations

The generalizability of the results is limited due to the recruitment of study participants from one Midwestern NICU. Future studies should include infants from multiple NICU settings in order to evaluate the generalizability of these findings. While all infants in the study had been diagnosed with illnesses that were potentially LT or LL, several different diagnoses and ages of infants were represented, including preterm infants >23 weeks gestation, so heterogeneity of the sample may have also affected the study results.

Conclusion

Behavioral observation in the neonatal intensive care unit is challenging and costly in terms of research personnel time and cost. However, more research on symptom assessment and management is crucial in order to alleviate the burden and suffering of vulnerable infants. Measuring symptoms in young infants, both clinically and for research purposes, is challenging, in large part due to the general inadequacies of proxy measures. Further research, including testing current clinical measures such as the N-PASS for research purposes or developing new symptom identification measures, is needed to provide a stonger basis for treatment plans and care decisions for very ill infants.