Assessment of Safe Sleep: Validation of the Parent Newborn Sleep Safety Survey

Abstract

Purpose

Sudden Infant Death Syndrome (SIDS) and suffocation account for more than half of all Sudden Unexpected Infant Deaths (SUID). The American Academy of Pediatrics (AAP) recommendations describe the safest environments to protect infants. This study compared parent responses on the Newborn Sleep Safety Survey and observational assessments (N=72) of infant sleep environments in families thought to be at high-risk for non-compliance with AAP recommendations.

Design and Methods

A naturalistic study of participants enrolled in two home visitation support programs was used. Observed risks ranged from 36.6% (never use pacifier) to 4.3% (never use firm mattress).

Results

Results comparing report to observation demonstrated acceptable concordance. Five items had fair concordance (Kappa > .4), four showed moderate concordance (Kappa > .6), and one excellent concordance (Kappa > .8). Although direct observation of safety behaviors is the gold standard in the injury prevention field, direct observation is logistically difficult, time consuming, and costly.

Conclusions

Research and interventions aimed at a reduction of Sudden Infant Death Syndrome (SIDS) and suffocation require accurate assessment of the infant sleep environment. This study provides acceptable evidence for the use of the Newborn Sleep Safety Survey as an alternative to direct observation to assess parent adherence to recommendations. Limitations are discussed.

Introduction

Sudden Infant Death Syndrome (SIDS) and suffocation account for more than half of all Sudden Unexpected Infant Deaths (SUID) and represent the first and third leading causes of post neonatal infant death (death in an infant age 28 days to 1 year) in the US(Centers for Disease Control and Prevention [CDC], 2011; CDC, 2011). The Centers for Disease Control and Prevention reports that nearly 3,500 US infants die suddenly and unexpectedly annually. In 1992, the American Academy of Pediatrics (AAP) released its original recommendation for infants to be placed in a non-prone position for sleep (Kattwinkel, Brooks, Keenan, & Malloy, 1994; Li et al.., 2003). The AAP recommendation, and the associated “Back to Sleep” (BTS) campaign, was followed by a marked decline in the incidence of SIDS which plateaued in the late 1990’s. The decline was concurrent with and likely related, at least in part, to a shift in the classification system for SUID in the US in response to epidemiologic reports in other western countries. This was evidenced by co-occurring increase in other causes of SUID that occur during sleep such as suffocation, asphyxia, and entrapment (Malloy & MacDorman, 2005; Moon, 2011; Shapiro-Mendoza, Tomashek, Anderson, & Wingo, 2006).

Typically, the etiology of sudden infant deaths cannot be explained; however, because the majority occur during sleep in an unsafe sleep environment, addressing potential risks in this environment is critical (Carlin, R. F., & Moon, 2016). The Triple Risk Model characterizes SIDS as the intersection between a period of critical newborn development during the first months of life, underlying and often invisible vulnerability in the infant and risky environmental conditions including sleep position (Filiano & Kinney, 1994; National Institute of Child Health and Human Development[NICHD], 2013). While research continues in the biological aspects of infant sudden death, infant sleep environments linked with unintentional suffocation are important targets of safety intervention (Hunt, Waters, Rodriguez, & Machaalani, 2015; Scheers, Rutherford, & Kemp, 2003).

Infants who succumb to suffocation are unable to respond and protect themselves to challenges in the environment, such as non-supine sleep position and materials that block airflow (Garcia, Koschnitzky, & Ramirez, 2013). Still, many parents placed infants in a non-supine sleep position (Colson et al., 2009), and bed sharing between infants and adults continues (Willinger, Ko, Hoffman, Kessler, & Corwin, 2003). Unfortunately, parents report that they do not often receive safe sleep recommendations and when they do they are often inconsistent with AAP recommendation (Eisenberg et al., 2015). With continued focus on the environment, AAP expanded its guidelines for infant sleep safety in 2011 to include additional parent actions such as breastfeeding, use of a pacifier, infant immunization, and discontinued use of bumper pads in cribs (Moon, 2011). Each has been associated with a reduction in SUID or injury (Committee, Stratton, Almario, Wizemann, & McCormick, 2003; Garcia et al., 2013; Gartner et al., 2005). For example, there is no evidence that bumper pads prevent injuries, but there is a potential risk of suffocation, strangulation or entrapment (Moon, 2011). On the other hand, there is evidence that room sharing on a separate sleep surface decreases the risk of SIDS by as much as 50% (Task Force On Sudden Infant Death Syndrome, 2016).

The guidelines are organized into those recommendations focused on the home environment and parenting and those that target policy makers, researchers and professionals (Moon, 2011). The recommendations for parents and other caregivers are often the target for newborn safety interventions and surveillance. To assess intervention success or assess risk in subpopulations, a reliable assessment tool is required. While observation of the infant sleep environment in the home is ideal, it is often not practical because of cost and logistic reasons. Therefore, because there are nopublished, commonly used tools, research teams typically develop their own assessment interviews. This leads to results with unknown reliability or validity. The lack of a common tool results in research findings that cannot be clearly compared across studies.

Our purpose was to develop a survey tool that could be used by professionals to identify families in need of program support and to assess intervention outcomes targeting safe sleep. In this naturalistic study of a convenience sample, we developed a parent survey, Newborn Sleep Safety Survey, to evaluate risk of infant SIDS and suffocation. We hypothesized that carefully worded survey questions could serve as reliable more feasible alternative to expensive home visits. We also examined the survey with the assumption that parents may be hesitant to use extreme scores. For example, participants may find it difficult to respond with the extreme response selection (“Never” or “Every Time”) because they are not the only caregiver of the infant. Parents may be hesitant to report ‘Every Time’ when the infant has other caregivers or if they did not adopt the safest option one or two times.

In summary, this study began with the following hypotheses:

• H1: Responses on the Newborn Sleep Safety Survey will not differ in rating from the observed on six APA safe sleep recommendations.

• H2: The most extreme responses on the Newborn Sleep Safety Survey will indicate less risk than observed.

Methods

Procedures

Study Site

We targeted low -income women and adolescent mothers because of the higher risk status of the infants in these families (National Library of Medicine, 2013). Study participants were recruited from a pool of women who were enrolled in one of two on-going home visiting programs: Teen Thrive (Thrive) or Healthy Families America (HFA)(DuMont et al., 2008). Thrive is a multi -site prenatal and parenting home visit program serving low-income, typically geographically isolated pregnant and parenting adolescents. HFA is a similar program for women of all ages who are pregnant or have young children. HFA recruits women who are low-income, single, or have other parenting risk factors (e.g., domestic violence). All participants qualified for Thrive or HFA based on pregnancy or parenting status and income level. Study sites were located across a wide rural geographic area of one southern US state.

Training

Home visitors were trained by expert research study staff to conduct the Newborn Sleep Safety Survey and the observational assessments. Training (6 hours) including guidance in introducing this study during the phone call to schedule the home visit, standardized introduction of the Newborn Sleep Safety Survey, integration of the study protocol into the home visit, interview skills, role play scenarios, use of the doll simulation, and scoring the observational items. Data were collected with approval of an Institutional Review Board. This study’s protocol required that participants be clients in the program with infants less than or equal to 6 months of age. The study was introduced using a standardized script prior to a formal written consent process.

Recruitment

Data were collected by 21 home visiting staff across 13 program locations. Home visitors recruited participants from the pool of women with infants from birth to 6-months of age who were scheduled to receive Thrive or HFA home visits during the period January 2012 to July 2012. Home visitors were instructed to integrate the protocol script in the first appointment possible and continue recruiting every eligible family for 6 months.

Measures

Newborn Sleep Safety Survey

The survey, designed by study investigators for mothers of infants, assessed the living arrangements of the participant and infant. Survey items were revised until there was 100% consensus within the study team. The survey assessed key APA recommendations but this study focused on: a) supine sleep position, b) firm sleep surface, c) separate sleep location and room sharing, d) bedding safety -soft or loose bedding, e) avoid overheating, and f) use of a pacifier. Questions were drawn from existing research tools identified in outreach to experts in the field, literature review, and translation of AAP guidelines into lay language.

Specifically, the survey began with a standardized introduction asking the parent to focus on how she and others put the baby to sleep. The interview included 37 questions organized into seven sections with each section introduced by a standardized script. Each section addressed a specific area of interest. Section 1 included questions about where the infant slept, Section 2 asked about the sleep surfaces, Section 3 asked about the crib and mattress, Section 4 focused on co-sleeping, Section 5 asked about clothing, use of a pacifier, and breastfeeding, Section 6 asked about the other caregivers that were involved in care taking during bedding, and Section 7 covered demographic information about the family.

For this study, we report on the 14 survey questions (see Table 1) that assess five of the AAP recommendations related to sleep practices. The 14questions include three types of responses. Most questions allowed responses on a 4-point scale of “Every Time”, “Most of the Time,” “Some of the Time.” or “Never”(e.g., How often do you put your baby down to sleep in an adult bed?). Some questions asked the respondent to select from a list of options (e.g., select the sleep location used most often). Finally, one question allowed a yes/no/don’t know response (e.g., Do you think the mattress in the crib is firm?).

Table 1.

Comparison of Safe Sleep Recommendation by Self Report and Observational Survey

Recommendation	Survey Question a	Observed Behaviorb	Reported Behavior Safest Definitionc		Reported Behavior Lenient Definitiond
		% Safe	% Safe	Agreee	% Safe	Agreee
Sleep Position	1. How often do you put your baby down to sleep on his/her back?	72.9%	39.7%	.38**	67.6%	.65**
Sleep Surface	2. The mattress in the (CRIB) is firm?	95.7%	85.1%	.41**	-	-
Sleep Location	3. At night, the baby sleeps in the same room as you or another adult?	90.3%	83.3%	.58**	90.3%	.84**
	4. How often do you and your baby sleep in the same bed? With someone else?	87.5%	27.1%	.13*	72.9%	.55**
	5. How often do you put your baby down to sleep in an adult bed?	87.5%	34.5%	.13	72.7%	.58**
	6. Where do you put your baby down most? (response of Adult Bed)	87.5%	-	-	76.4%	.59**
Bedding safety-soft or loose bedding	7. When your baby is sleeping, how often is there a pillow in the (CRIB) for him/her to rest his/her head on?	71.6%	74.6%	.62**	85.1%	.44**
	8. When your baby is sleeping, how often is there a bumper pad around the edges of the (CRIB)?	76.1%	74.6%	.72**	76.1%	.75**
	9. How often is there a cushion, pillow, heavy blanket, or sheepskin, on top of the mattress but underneath the baby?	68.4%	54.4%	.49**	70.2%	.46**
	10. How often is there a stuffed animal in the (CRIB) with your baby?	68.7%	64.2%	.63**	79.1%	.58**
	11. When your baby is sleeping, how often does the (CRIB) have a mattress in it – one that fills up the whole bottom of the (CRIB) and doesn’t leave any space between the mattress and the edges of the (CRIB)? f	92.5%	92.5%	NS	92.5%	NS
	12. How often do you use a wedge or something else to hold the baby in a particular position while s/he sleeps, or to keep him/her from rolling? f	92.5%	76.1%	NS	94.0%	NS
Overheating	13. How often is there a heavy blanket, comforter, or bedspread in the (CRIB) with your baby?	77.6%	74.6%	.34*	89.6%	.26+
Offer a pacifier at nap time and bedtime.	14. How often do you give your baby a pacifier when you put him/her down for sleep?	63.4%	15.5%	.19*	25.4%	.33**

⁺p< .05,

^*< .01,

^**p< .001;

Sample sizes range from 67 to 72 with the exception of # 2 where there were only 49 used cribs/bassinette, and #11 where there were only 65 with identified mattress.

^aAll have response options of Every time to Never except #2 (Yes/No/DK) and #6 (name the most common place).

^bSimulation with doll.

^cPercent of Always for safe or never for unsafe.

^dPercent of Always or Most of time for safe or Sometimes or never for unsafe.

^eCohen’s kappa Inter-rater reliability coefficient.

^fOnly 4 women reported risk behaviors.

To address our hypothesis (H2) that extreme response options would vary in usefulness across the survey, we considered two definitions of safe sleep based on the parent responses. The safest response was defined as a report consistent with the strict definition for safety (e.g., “Every Time” put baby down to sleep on his/her back). This represented the behaviors most consistent with the AAP recommendations. A second definition was more lenient but may better reflect the ability of the parent to report. For example, a safe response was defined in the lenient definition as at least ‘Most of the Time’ in response to how often the baby is put to sleep on their back by the parent. That is, mothers that responded ‘Every Time’ or ‘Most of the Time’ would be considered in compliance with the AAP recommendation.

Observational assessment

The observational assessment, designed by study investigators, was developed to match as nearly as possible the survey items to confirm. Each observation was examined for barriers and a protocol for training created to address. The observation occurred during the home visit after the Newborn Sleep Safety Survey and was organized into three sections with 15 observation items.

The assessment begins with a standardized script to put the parent at ease and asks to be shown the baby’s sleep area and to see the pacifier (if one is used) that is offered to the child. Once in the sleep area, the parent is provided an infant doll and asked to put the doll to sleep in the baby’s normal sleep location and position using the same routine that the parent typically used. The observer completed a detailed examination of the sleep location (e.g., crib) and position for AAP recommendation compliance. For example, the observer completed a checklist of the type of contents of the sleep area (e.g., blanket, bumper pad, pillow), items near the sleep environment (e.g., cords, wedge), and the type of sleep area (e.g., adult bed, couch, floor, crib). A separate check list was used to examine the repair status of the crib (e.g., damaged, mattress supported) and the sleep surface (e.g., firm, sheet tight).

Statistical Analysis

SPSS version 22.0 was used to calculate kappa statistics using the crosstab procedure. (IBM Corp., 2013) Rates of compliance with safe sleep recommendations were computed from the interview and the observational assessment data. The observed sleep environment for each recommendation assessed during a single home visit was compared with the subject’s reported behaviors (using both the safest and lenient definition interview responses). P values less than .05 were considered trends and less than .01 were considered sufficient to reject the hypothesis. Inter-rater agreement to determine the consistency between the self-report interview and the observer rating was computed using Cohen’s kappa coefficient (Viera & Garrett, 2005). Kappa statistics above 0.40 are considered to demonstrate fair agreement, above 0.60 show moderate agreement, and above 0.80 substantial agreement (Viera & Garrett, 2005). Table 1 reports the responses reflecting lower risk for SIDS for each recommendation and the corresponding Kappa statistic.

Results

Study Sample

Of 155 eligible home visits, 78 consented and were assessed (40 were outside of the target age, 21 had visits already scheduled disrupting the protocol using the introduction script, and 16 did not have complete consent paperwork). A total of 78 women completed the Newborn Sleep Safety Survey and received the observational assessment. Six (6) children were over 30 weeks of age and were excluded from this analysis, for a total sample size of 72 infants. Infants included in this analysis ranged from 1 week to 28 weeks of age (M= 9.5 weeks, SD= 7.3 weeks). Women ranged in age from 13 to 36 years of age (M= 18.8 years, SD= 3.9 years, only 2 women were between 24 and 26 years of age). For most women (71%), this was their first child and 55.6% of the infants were female.

Like many southern states, the rural area included in this study has high unemployment rates (over 60%), high poverty rate (over 25% of children live in poverty), and a high teen birth rate (ranked among the highest in the nation). Other demographics could only be obtained at the program level for all clients served by the program during the year of the study (n = 220). In this larger sample, women served were majority white (57.7% white with 30.2% African American, 12.1% other/mixed). Nearly half of women (47.6%) had not finished high school (40.2% had only a high school degree or GED). Most (71.3%) were not working and of the remainder that were working 13.4% were working less than 20 hours a week. Regarding the biological father, 30.5% lived in the home, 38.6% lived outside the home but had contact with the family, and the remainder were not involved with the family.

Recommendation 1: Sleep Position – Back to Sleep

Ten percent (10.0%) of women reported never putting the baby to sleep on his or her back and 39.7% reported putting the baby down to sleep every time on their back. During the simulation, the women most often placed the doll on its back (72.9%) with the other options nearly evenly used (12.9% on side and 14.3% on stomach).

Our analysis indicated moderate and statistically significant concordance in the observed and self-report sleep position for the lenient definition of risk (response at least most of the time; Kappa .65, p = .00) and only fair agreement with the safest definition (Kappa .38, p = .00). For the lenient definition, the discrepancy between simulation and self-report was balanced. That is, 5 women reported typically placing the child on their back but did not place the child on their back during the simulation; and another 4 women did and reported the reverse.

Recommendation 2:Sleep Surface – Firm Surface

Most parents reported using a crib with a mattress (N = 62, 86.1%). Of those with a crib, most parents reported the mattress in the crib was firm (83.9%). Observers were able to observe 47 cribs with mattresses and sheets. Of these, 85.1% were observed and reported as firm. Five parents did not report the mattress as firm but it was observed as firm and 2 parents correctly indicated that their infant’s mattress was not firm. Agreement statistics indicate fair concordance (Kappa 0.41, p < 0.01).

Recommendation 3: Sleep Location– Room sharing, not adult bed

In the simulation, the observer noted the location the doll was placed for sleep. The most common locations observed were bassinette (27.8%) and full-size crib (27.8%). Other options included a portable crib (12.5%) and adult bed (12.5%). An additional 20% placed the doll in some other place such as car seat, couch, chair, or floor.

For comparison with parent report, observations of adult bed were considered to indicate co-sleeping. Sleep location was assessed by a comparison of room sharing to bed sharing with multiple questions to assess bed sharing. As seen in Table 1, most parents reported that the infant slept in the same room with them or another adult all (83.9%) or most (90.3%) of the time. This was consistent with the observation that the infant sleeping space was in the same room as the parent sleep space (90.3%). Agreement statistics indicate substantial concordance, in particular for the more lenient definition of risk.

Subjects were asked two questions related to co-sleeping (How often do you and your baby sleep in the same bed? How often does your baby sleep in the same bed as someone else?). These were combined to assess co-sleeping frequency. Of subjects reporting that the infant ever sleeps with someone besides the mother (N = 23), the other person identified was evenly split between the child’s father (N = 9) and other family members (N = 9, grandmother, sister, aunt). Concordance was not acceptable when based on the extreme ‘never’ response but was at a moderate level when based on the lenient definition of some of the time or never.

Subjects were asked two questions related to where the baby slept (How often do you put your baby down to sleep in an adult bed? Which location do you use most?). For the first question, 28.4% of women responded never and 40.3% responded some of the time. Adult bed was selected as the most used by 30.6% of subjects. Concordance was not acceptable based on the safest response definition but was moderate based on the lenient definition.

Recommendation 4: Keep soft objects and loose bedding out of the crib

Six questions were observed and included in the survey as seen in Table 1. These include the use of a pillow in the crib, bumper pads, material under the baby (i.e. loose bedding, heavy blanket), soft toys, gaps between the mattress and the bed, and use of a wedge. Four questions showed concordance between observation and parent report at a moderate level or above. Of the six questions assessing the bedding safety, two items had only four mothers report risky behavior (that is, only 4 persons reported a bed with dangerous gaps between the mattress and bed frame and only 4 reported using a wedge). While the percent of women reporting behaviors was similar, the overlap in the observed and reported behaviors was not enough to provide a statistically significant Kappa coefficient.

Recommendation 5: Avoid Overheating

Subjects were asked how often heavy blankets or other covers typically used for cold weather were used in the crib. The concordance of the report and observation of a heavy blanket use was poor based on the Kappa. Under observation, 77.6% of infants were not exposed to blankets in the crib, and by self-report, between 74.6% and 89.6% were not exposed; however, the overlap was not high (Kappa = .26 to .34 depending on risk definition).

Recommendation 6: Use of Pacifier

The interviewer asked the parent to show them the pacifier typically used to assess the level of pacifier use;63.4% of the women were able to show one. Only a few (15.5%) reported always giving the baby a pacifier when they put them down to sleep and 38% reported never using a pacifier. More women reported giving the child a pacifier at least some of the time (25.4%); however the Kappa statistic was not of an acceptable level of concordance for either response definition.

Discussion

Sudden Infant Death Syndrome and suffocation are among the top three causes of infant mortality in the United States (Centers for Disease Control and Prevention [CDC], 2011; CDC, 2011). Risks to infants for SIDS and suffocation in the physical and social environments are frequently included in efforts to reduce SIDS because, unlike genetic or developmental risks, they are modifiable. However, there is inconsistency in the assessment of the risk in the home sleep environment, hampering research and monitoring efforts. This study examined the reliability of Newborn Sleep Safety Survey, a parent interview, as a tool to identify children at risk in the home sleep environment. Overall, our results supported H1 that the Newborn Sleep Safety Survey had adequate concordance with observed behaviors supporting the reliability of it as a tool to use as an alternative to expensive home visit observational assessments.

Our second hypotheses (H2) that some survey questions would be more sensitive to the selection of extreme values was supported. Our results showed that adequate reliability for some items relied on maternal response that was more lenient than the AAP guidelines. That is, our study also indicated that, in some cases, the response consistent with the AAP guidelines was useful in the identification of risks. In other cases, the strictest interpretation of the response would likely have over- or under-identified infants at risk. Using a more lenient definition, concordance was high for sleep position, sleep surface, and sleep location. Of the four survey questions with acceptable concordance related to bedding safety, the definition consistent with AAP guidelines had the most support. This will allow us to provide clear instructions to other users of the Newborn Sleep Safety Survey to identify children most at risk.

We hypothesized the need for the more lenient definition because we expected parents to find it difficult to respond to the most extreme value when they were not the only caregiver. It may also have been a way for mothers aware of the recommendations to be honest in the face of the pressure of social desirability. However, other explanations are possible. For example, response option use has been tied to participant culture (Lee, Jones, Mineyama, & Zhang, 2002) and education level. Particularly important in the study of high-risk populations is evidence that when respondents have low socioeconomic status, groups are sometimes biased toward agreement with the questions (regardless of content). There is evidence that personal characteristics (Hui & Triandis, 1989) are linked with the selection of end -points or a tendency not to select end-points. Regardless, the degree of concordance of the Newborn Sleep Safety Survey with the observations provided evidence that the Survey shows promise as an alternative to expensive observational assessments.

Limitations

Our study was limited in that we focused only on safe sleep recommendations and did not collect reported and observed information on all aspects of safe sleep recommendations of AAP. For example we omitted smoke exposure during and after pregnancy, breast feeding, cardio respiratory monitor use, and prenatal care. However, we collected parent report and observational data on all the physical environment factors addressed by the AAP guidelines. We feel this is a major strength of this study. Further the study was limited because the Newborn Sleep Safety Survey was developed by investigators for this study and has no published psychometric testing before this study. Limited study resources meant that we were unable to conduct inter-rater reliability assessments of the data collection. The generalizability is limited in that the subjects in this study were drawn from mothers eligible for home visiting services that were specifically for young and/or low-income women. While the sample size is adequate, a larger more diverse sample would provide stronger evidence of the psychometric of the survey.

Clinical Implications

The safety of the infant sleep environment and parenting practices during early infancy is critical for healthy development. Education regarding recommended practices should begin prenatally and continue with pediatric providers. The American Academy of Pediatrics has provide guidance on the areas related to SIDS that should be addressed; however in practice barriers may exist (Kattwinkel, Brooks, Keenan, & Malloy, 1994; Li et al.., 2003). For example, there is the potential confusion and concern in urging parents to insist on some of the practices that may be protective. For example, the use of the pacifier should be offered and encouraged, not forced. Further, there is ongoing research and discussion regarding some recommendations, such as the perception of bed-sharing as a tool to facilitate breast feeding (Mileva-Seitz, Bakermans-Kranenburg, Battaini, & Luijk, 2016).

Conclusions

To our knowledge, there is no prior literature published on comparing reported and observational data using safe sleep surveys. Our study demonstrates that the Newborn Sleep Safety Survey is useful in obtaining valid information on the physical environment of the infant’s sleep area. The overlap between the observations and Survey responses suggest that the ability of the parent to report (i.e., remember and avoid socially desirability) was sufficient to provide a reliable assessment. The consistent use of a reliable instrument to assess infant sleep has the potential to improve research by allowing meaningful comparisons of study results at relatively low cost, allowing for wider dissemination of intervention activities.

Practical Implications.

This study provided evidence of the usefulness of the Newborn Sleep Safety Survey, a parent survey of infant sleep environments. This tool will provide medical and research professionals a reliable, inexpensive tool to evaluation of the quality of sleep environments using a standard definition.