Effectiveness of an mHealth Intervention for Infant Sleep Disturbances

Abstract

Bedtime problems and night wakings are highly prevalent in infants. This study assessed the real-world effectiveness of an mHealth behavioral sleep intervention (Customized Sleep Profile; CSP). Caregivers (83.9% mothers) of 404 infants (age 6 to 11.9 m, M = 8.32 m, 51.2% male) used the CSP (free and publicly available behavioral sleep intervention delivered via smartphone application, Johnson’s® Bedtime® Baby Sleep App). Caregivers completed the Brief Infant Sleep Questionnaire–Revised (BISQ-R) at baseline and again 4 to 28 days later. Changes in sleep patterns were analyzed, based on sleep problem status (problem versus no problem sleepers; PS; NPS). Sleep onset latency improved in both groups. Earlier bedtimes, longer continuous stretches of sleep, as well as decreased number and duration of night wakings, were evident in the PS group only. The BISQ-R Total score, total nighttime sleep, and total 24-hour sleep time improved for both groups, with a greater change for the PS group. Further, caregivers of infants in the PS group decreased feeding (bedtime and overnight) and picking up overnight, and perceived better sleep. Bedtime routine regularity, bedtime difficulty, sleep onset difficulty, and caregiver confidence improved for both groups, with the PS group showing a greater magnitude of change. Thus, a real-world, publicly available, mHealth behavioral sleep intervention was associated with improved outcomes for older infants. Intervention recommendations resulted in changes in caregivers’ behavior and improvements in caregiver-reported sleep outcomes in infants, in as few as 4 days.

SLEEP PROBLEMS ARE HIGHLY prevalent for infants and young children (Sadeh, Mindell, & Rivera, 2011). Child sleep disruptions, such as bedtime problems and night wakings, for instance, are reported by 20%–30% of parents of infants and are related to the functioning of not only children themselves (Hysing, Sivertsen, Garthus-Niegel, & Eberhard-Gran, 2016; Mindell, Leichman, DuMond, & Sadeh, 2017), but also to the sleep and functioning of family members (Gradisar et al., 2016; Lam, Hiscock, & Wake, 2003; Sinai & Tikotzky, 2012). Sleep problems throughout infancy and childhood are typically based on caregiver report on both specific sleep parameters (e.g., bedtime, sleep duration) and parent perception of sleep problems (e.g., Mindell et al., 2011a, 2011b; Quach, Hiscock, Ukoumunne, & Wake, 2011; Stevens, Splaingard, Webster-Cheng, Rausch, & Splaingard, 2019). Behavioral interventions have been shown to be effective in treating these problems (Gradisar et al., 2016; Mindell, Kuhn, Lewin, Meltzer, & Sadeh, 2006), and can be delivered through a variety of mechanisms, including web-based platforms (e.g., Customized Sleep Profile, CSP; Mindell et al., 2011a, 2011b).

MOBILE ACCESS AND MHEALTH

mhealth defined

As defined by the World Health Organization (WHO), mHealth, or mobile health, is a type of telehealth that utilizes mobile devices to support medical or public health practices (WHO, 2011). Simply, mHealth is the use of wireless and mobile technologies for health (WHO, 2016). mHealth smartphone applications (apps) have been developed to support health promotion initiatives in a variety of areas. These apps address everything from physical activity and weight control to mental health support and medication management (Zhao, Freeman, & Li, 2016), to general parenting (Davis et al., 2017) and prenatal care (Mauriello, Van Marter, Umanzor, Castle, & de Aguiar, 2016). Sleep, an important health domain across the lifespan, is also a common area for app development (Grigsby-Toussaint et al., 2017; Lee-Tobin, Ogeil, Savic, & Lubman, 2017). Considering the individual, familial, and societal toll poor sleep can precipitate, there is promising value in disseminating information and evidence-based intervention strategies through a widespread, accessible, mobile outlet.

Mobile Prevalence

In the U.S., it is estimated that 96% of people own a cell phone and 81% (up from 35% in 2011) own a smartphone (Pew Research Center [PRC], 2019). The health domain is a primary area of app development. In the first quarter of 2019, over 45,000 mHealth apps were estimated to be available in the Apple store alone (Statista, 2019), and over 97,000 health and fitness apps for mobile and tablet use were reported in 2018 (Baluni, 2018). Thus, considering its widespread use and availability in the current U.S. culture, a smartphone is a common vehicle for health information delivery and potential behavior change within the context of mHealth.

mHealth and Sleep Apps

Existing sleep apps have diverse functionality that address many aspects of sleep (e.g., duration, quality, architecture, and sleep problems such as insomnia or obstructive sleep apnea; Lee-Tobin et al., 2017), and have primarily targeted adults. App components and functionality range from information delivery and psychoeducation, to self-monitoring and data collection, to intended behavior change through intervention. However, evidence supporting use of these apps, or individual functions within the apps, is not nearly as prevalent as the apps themselves. For instance, in a review of 76 sleep apps found in the Google Play store, less than one-third had any empirical evidence supporting claims, and only 13.2% offered links to sleep-related literature (Lee-Tobin et al., 2017). Similarly, of the 51 sleep apps reviewed by Ong and Gillespie (2016), all reported on sleep duration and over 65% reported on sleep structure. According to the authors, however, no information was provided regarding sleep structure algorithms and no scientific research was cited regarding their accuracy. Thus, empirical validation of functions within these apps is warranted.

Finally, although no large-scale study has reviewed existing apps exclusively targeting the sleep of infants and young children (and their parents), a review of 46 parenting apps related to infant and child health included two apps that targeted sleep for young children (Davis et al., 2017). Although these two infant sleep apps reportedly referenced university or sleep clinic professionals, both also included opinion-based information, and the degree to which the available information was research-validated was unclear. The specific content that was noted as “opinion-based information” in these two apps was not reported.

mHealth and the Customized Sleep Profile

One evidence-based online intervention targeting infant and toddler sleep (6–36 months) that has been shown to be efficacious is the Customized Sleep Profile (CSP; Mindell et al., 2011a, 2011b). The Customized Sleep Profile is an algorithm-based internet tool that allows a caregiver to complete a sleep analysis, which is an expanded and revised form of the Brief Infant Sleep Questionnaire-Revised (BISQ-R; Mindell, Gould, Tikotzky, Leichman, & Walters, 2019). Based on responses from the sleep analysis, individually tailored recommendations are provided onscreen immediately upon completion of the analysis, targeting areas such as bedtime, bedtime routine, parent behavior in response to night wakings, sleep duration, sleep space, and overall sleep health. Detailed information regarding CSP intervention recommendations has been published previously (i.e., Mindell et al., 2011a).

The CSP has been adapted to be delivered via a mobile app, and is available on Johnson’s® Bedtime® Baby Sleep app (Johnson & Johnson Consumer Inc., Skillman, NJ, USA), a free and publicly available smartphone application. The primary adaptation was formatting to an app-based platform, with no changes in the algorithms or the content. Additionally, the previous stand-alone CSP is now provided within the context of other tools, such as sleep tracking and an Ask the Expert function where caregivers can submit individual sleep questions.

In the initial validation of the CSP, mothers of 264 infants and toddlers were randomized into one of two intervention groups (CSP or CSP plus prescribed bedtime routine) or a control group (Mindell et al., 2011a). Within 2 weeks of provision of recommendations, infants and toddlers in both intervention groups had improved sleep onset latency, night waking frequency and duration, more nighttime sleep, and less difficulty at bedtime. Sleep consolidation, or duration of an infant or child’s longest continuous stretch asleep, improved for both intervention groups. At 1-year follow-up, mothers reported maintained improvements in falling asleep, number and duration of night wakings, and sleep consolidation in their infants (Mindell et al., 2011b). Although efficacy of the CSP was supported at the time of intervention as well as at 1-year follow-up, effectiveness and real-word application of the intervention has not been examined.

Considering the prevalence of sleep problems in infancy as well as the potential benefits and widespread availability of mHealth apps, there is a surprising paucity of research validating the effectiveness of interventions within sleep apps. Further, there is need for additional validation of the CSP intervention within the context of mobile app delivery. Thus, the aim of this study was to examine the effectiveness of the CSP using exploratory real-world data, specifically assessing the magnitude of change following provision of individualized recommendations, with a focus on comparing outcomes for those infants considered to be problem sleepers (PS) by their caregivers versus those who were not considered to be problem sleepers (NPS). It was expected that a greater magnitude of change would be evident in the PS group post-intervention across sleep parameters, given that there was room for improvement and specific recommendations regarding sleep-related issues were more likely to be provided.

Method

PARTICIPANTS

Caregivers (83.9% mothers) of 404 infants age 6 to 11.9 months (M = 8.32; SD = 1.61) from the U.S. participated in this study. Participants were included in this real-world study if the CSP was completed twice, with the second completion between 4 and 28 days later. There were no other inclusion or exclusion criteria. Table 1 contains complete participant demographic information.

Table 1.

Demographic Information of Current Sample

	Total sample	No problem (NPS)	Problem (PS)
	Mean (SD)	Mean (SD)	Mean (SD)	t/χ2
	n = 404	n = 131	n = 273
Child age (mos)	8.32 (1.61)	8.33 (1.66)	8.31 (1.59)	0.85
Caregiver age (yrs)	29.34 (6.13)	27.96 (6.13)	30.00 (6.03)	3.18**
Days between assessment	10.59 (6.00)	10.96 (6.32)	10.42 (5.84)	0.10
Child gender	% (n)	% (n)	% (n)
Male	48.8 (197)	47.3 (62)	53.1 (145)	1.19
Female	51.2 (207)	52.7 (69)	46.9 (128)
Caregiver relationship
Mother	83.9 (339)	84.7 (111)	83.5 (228)	0.10
Father	1.7 (7)	2.3 (3)	1.5 (4)
Grandmother	0.5 (2)	0.8 (1)	0.4 (1)
Nanny, Au Pair, Babysitter, other caregiver	13.9 (56)	12.2 (16)	14.7 (40)

Note.

^*p<.05;

^**p<.01;

^***p<.001;

NPS = No problem sleeper group; PS = Problem sleeper group; Caregiver χ² test analyzed 2 × 2 table (i.e., NPS, PS, Mother, All other).

PROCEDURE

This study was approved by a university Institutional Review Board. The free and publicly available Johnson’s® Bedtime® Baby Sleep app (Johnson & Johnson Consumer Inc., Skillman, NJ, USA) was utilized as an intervention delivery mechanism within the current study.

All naturally acquired app users provided consent to participate in a research study and were able to decline or withdraw participation. If users declined research participation they were still able to use the app and CSP. The CSP (described above) is an optional component of the Johnson’s® Bedtime® smartphone application. Within the app, users had the ability to access the CSP as many times and as frequently as they wished. Tailored recommendations were provided based on their answers each time the CSP was completed.

As part of the CSP, the BISQ-R (Mindell et al., 2019) was completed by participants each time they accessed this part of the app. Participants were selected based on the time between their first (initial assessment) and second use of the tool (between 4 to 28 days later). The selection of the minimum number of days after intervention delivery was determined based on a previous study indicating that rapid changes in infant sleep outcomes following provision of behavioral recommendations can occur within the first 4 days (Mindell, Leichman, Lee, Williamson, & Walters, 2017). The 28-day (1-month) maximum was selected based on previous intervention studies addressing sleep in infancy commonly including follow-up within a 1-month time frame (i.e., Eckerberg, 2002; Stevens et al., 2019). Further, limiting the follow-up period to 1 month decreased the likelihood of developmental influences on sleep.

Participants were grouped based on whether or not caregivers rated their child as being a problem sleeper (PS) or a non-problem sleeper (NPS) at their initial app entry (“Do you consider your child’s sleep a problem?”), based on a 5-point Likert scale with scores of 1 and 2 (“no problem” and “very small problem”) denoting non-problematic sleep and 3 through 5 (“small problem” to “serious problem”) indicating a parent-perceived sleep problem (Sadeh, 2004; Mindell et al., 2019). Of note, parent perceived sleep problem has been found to be predicted by a set of sleep parameters, including longest stretch asleep, night waking frequency and duration, and total sleep time (AUC = .88; Sadeh et al., 2011).

Brief Infant Sleep Questionnaire–Revised

The BISQ-R (Mindell et al., 2019), a component of the Bedtime app, is a parent-completed measure of infant and toddler sleep (Sadeh, 2004; Sadeh, Mindell, Luedtke, & Wiegand, 2009). As noted above, the BISQ-R was completed by caregivers in conjunction with the Customized Sleep Profile (CSP; Mindell et al., 2011a, 2011b) intervention. The BISQ-R was used to generate the following commonly analyzed infant sleep parameters: start of the bedtime routine, bedtime, sleep onset latency, night waking frequency, night waking duration, longest stretch a child remains asleep, total nighttime sleep, nap duration, and total amount of time a child sleeps in a 24-hour period. Finally, the BISQ-R utilizes 19 items from the full assessment to yield a total score. Infant sleep parameters, elements of parent perception of sleep, and parent behavior representing sleep ecology are taken into account to generate the total sleep score. Higher scores on the BISQ-R Total score indicate better sleep. Development of normative values for the BISQ-R and validation of the measure has been conducted (Mindell et al., 2019).

INTERVENTION

The CSP provides individually tailored recommendations and psychoeducation within the app immediately after completion of the BISQ-R. For example, if a caregiver indicates that her baby usually falls asleep at bedtime while being held and sleeps poorly, psychoeducation is provided regarding sleep onset associations and recommendations are presented regarding helping the baby learn to fall asleep independently at bedtime. Alternatively, if a caregiver indicates that her baby usually falls asleep at bedtime while being held but sleeps well, psychoeducation would be given regarding the future possibility of bedtime sleep training, but that there is no need to make the change since the baby is sleeping well. Recommendations span multiple aspects of sleep, including sleep schedules, bedtime routines, sleep onset associations, and limit setting. Additional detail regarding the intervention recommendations can be found in a previous paper (i.e., Mindell et al., 2011a).

ANALYSES

Data were analyzed using SPSS Version 24 (IBM Corp., 2016). To determine differences by group on change in sleep from baseline to follow-up (i.e., pre- and post-intervention repeated measures on each sleep parameter), analyses were conducted using general linear modeling. Child age, caregiver age (which differed between groups), and number of days between assessments were held constant when examining between group differences. We examined the main effects of time (baseline to follow-up) and group (PS versus NPS), with a focus on the group by time interaction to identify whether groups differed in their magnitude of change from pre- to post-intervention exposure. For significant time by group interaction effects, post-hoc Bonferroni-adjusted pairwise comparisons were used to probe the direction and magnitude of within-group change from baseline to follow-up, as well as statistically significant differences between groups pre- and/or post-intervention exposure. Estimated marginal means, standard errors, and effect sizes (partial eta-squared) are presented for these analyses. McNemar tests, utilized to compare paired nominal data, were used to compare categorical sleep variables from baseline to follow-up, for the whole sample as well as by sleep group (i.e., PS versus NPS). Effect sizes in the form of odds ratios are also provided (Graphpad QuickCalcs, Prism 8, accessed 2019).

Results

SLEEP PROBLEMS

At baseline assessment, 67.6% (n = 273) of infants were identified as problem sleepers (PS). Child age, t(402) = 0.85, p = 0.39, and days between assessment, t(402) = 0.10, p = 0.92, did not differ between PS and NPS groups (see Table 1).

SLEEP-WAKE PATTERNS AND BISQ-R TOTAL

Summary results are provided in Table 2. At baseline, the PS group showed significantly more frequent and longer night wakings, shorter stretches of continuous sleep overnight, less sleep per 24-hour period, and lower BISQ-R total scores as compared to the NPS group. In terms of within-group statistically significant change from pre- to post-intervention exposure, the following parameters improved only for the PS group: earlier bedtime routine start time and bedtime, decreased night waking frequency and duration, and increased longest stretch of continuous overnight sleep. Both PS and NPS groups improved in terms of BISQ-R total score, sleep onset latency, total nighttime sleep, and total sleep per 24-hour period; of note, however, the PS group showed greater improvement in sleep duration than the NPS group. No changes in either group from pre- to post-intervention were observed for nap duration. Further, at follow-up, the PS group had earlier routine start times and bedtimes than the NPS group, and similar total sleep time per 24-hour period.

Table 2.

Individual Infant Sleep-Wake Patterns and BISQ-R

	Time 1 M (SE)	Time 2 M (SE)	Within group ES	Repeated Measures	Wλ	F	P	ES
Start bedtime routine (time, min)
NPS	7:52 p.m. (6.6)	7:55 p.m. (6.6)	-	Time	0.99	4.32	.038	.01
PSa***,c**	7:45 p.m. (4.8)	7:32 p.m. (4.2)	.05	Interaction	0.97	9.55	.002	.03
Whole Sample	7:47 p.m. (3.9)	7:38 p.m. (3.7)		Group	-	2.02	NS	-
Bedtime (time, min)
NPS	8:40 p.m. (6.6)	8:40 p.m. (6.6)	-	Time	0.97	10.34	.001	.03
PSa***, c*	8:31 p.m. (4.8)	8:16 p.m. (4.8)	.08	Interaction	0.97	10.23	.001	.03
Whole sample	8:34 p.m. (3.8)	8:24 p.m. (3.8)		Group	-	1.73	NS	-
Sleep onset latency (min)
NPSa***	34.58 (2.31)	24.78 (1.91)	.05	Time	0.91	39.39	<.001	.09
PSa***	34.39 (1.60)	27.26 (1.38)	.05	Interaction	0.99	0.96	NS	-
Whole sample	34.46 (1.31)	26.44 (1.09)		Group		0.96	NS	-
Night waking frequency
NPS	1.90 (0.11)	1.75 (0.11)	-	Time	0.94	26.68	<.001	.06
PSa***,b***,c***	3.01 (0.08)	2.58 (0.08)	.10	Interaction	0.98	6.18	.013	.02
Whole sample	2.65 (.07)	2.31 (.07)		Group	-	64.68	<.001	.14
Night waking duration (min)
NPS	38.45 (4.41)	32.21 (3.87)	-	Time	0.94	22.61	<.001	.06
PSa***,b***,c*	61.12 (2.83)	44.09 (2.48)	.10	Interaction	0.99	5.01	.026	.01
Whole sample	54.50 (2.45)	40.62 (2.11)		Group	-	15.41	<.001	.04
Longest stretch asleep (min)
NPS	391.54 (13.11)	408.24 (13.98)	-	Time	0.96	15.59	<.001	.04
PSa***, b***,c***	286.78 (8.93)	325.96 (9.51)	.06	Interaction	0.99	2.20	NS	-
Whole sample	320.00 (7.75)	352.05 (8.12)		Group	-	42.80	<.001	.10
Total nighttime sleep (min)
NPSa**	551.42 (10.49)	578.38 (9.19)	.02	Time	0.89	46.10	<.001	.11
PSa***	531.07 (7.21)	581.50 (6.30)	.14	Interaction	0.99	4.24	.04	.01
Whole sample	537.59 (5.95)	580.50 (5.19)		Group	-	1.03	NS	-
Nap duration (min)
NPS	170.60 (5.55)	162.17 (5.41)	-	Time	0.99	2.52	NS	-
PS	157.00 (3.77)	156.03 (3.67)	-	Interaction	0.99	1.40	NS	-
Whole sample	161.29 (3.03)	157.97 (3.14)		Group	-	2.54	NS	-
Total sleep in 24-h (min)
NPSa*	721.44 (11.55)	744.92 (10.62)	.01	Time	0.94	23.33	<.001	.06
PSa***,b**	687.92 (7.62)	730.83 (7.01)	.08	Interaction	0.99	2.00	NS	-
Whole sample	698.08 (6.40)	735.10 (5.85)		Group	-	4.83	.029	.01
BISQ-R: Total Score
NPSa***	65.10 (0.97)	68.70 (1.28)	.03	Time	0.76	126.28	<.001	.24
PSa***, b***,c***	47.10 (0.67)	57.48 (0.89)	.35	Interaction	0.93	29.76	<.001	.07
Whole sample	52.94 (0.69)	61.12 (0.77)		Group	-	145.81	<.001	.27

Note.

^*p<.05;

^**p<.01;

^***p<.001;

NS = not significant (i.e., p ≥ .05); Wλ – Wilks’ lambda; M=mean; SE=standard error; ES=effect size; partial Eta² where 0.10 = small effect, 0.25 = moderate effect, and 0.40 = large; NPS = no problem sleeper group; PS = problem sleeper group.

^aSignificant difference between time 1 and time 2, within group;

^bSignificant difference between groups at time 1;

^cSignificant difference between groups at time 2 Child age, caregiver age, and days between assessment held constant in models to interpret between-subject effects.

Bedtime and Falling Asleep

There was a significant time by group interaction effect (Wilks’ λ = 0.97; F(1, 372) = 9.55, p < .05; ES = 0.03) for the timing of the bedtime routine. The PS group had a significantly earlier bedtime routine start time of 13.2 minutes at follow-up (M= 7:32 p.m., SE = 4.2 min) than at baseline (M = 7:45 p.m., SE = 4.8 minutes). The NPS group’s routine start time did not differ significantly at follow-up. Similarly, there was a significant time by group interaction (Wilks’ λ = 0.97; F(1, 382) = 10.23, p < .01; ES = 0.03) for infant bedtimes, with the PS group showing a statistically significant, earlier bedtime from baseline (M = 8:31 p.m., SE = 4.8 minutes) to follow-up (M = 8:16 p.m., SE = 4.8 minutes), by 15.6 minutes. However, no group or time by group interaction effects were evident for sleep onset latency, with both PS and NPS having a comparable reduction in sleep onset latency (Wilks’ λ = 0.91; F(1, 385) = 39.39, p < .001; ES = 0.09; baseline M = 34.49 min, SE =1.40, follow-up M = 26.01 min, SE = 1.16, combined groups).

Night Wakings and Sleep Consolidation

Significant time by group interactions were found for night waking frequency (Wilks’ λ = 0.98; F(1, 402) = 6.18, p < .05; ES = 0.02) and duration (Wilks’ λ = 0.99; F(1, 361) = 5.01, p < .05; ES = 0.01), with only the PS group showing reductions in both outcomes. In the PS group, night waking frequency decreased (baseline M = 3.01, SE = .08, follow-up M = 2.58, SE = .08) and night waking duration reduced by 17.03 minutes on average (baseline M = 61.12, SE = 2.83, follow-up M = 44.09, SE = 2.48). The PS and NPS groups both increased in their longest stretch asleep during the night (time effect Wilks’ λ = 0.96; F(1, 389) = 15.59, p < .001; ES = 0.04), yielding an average increase in longest stretch asleep of 28.18 minutes for both PS and NPS groups combined (baseline M = 339.22, SE = 7.91, follow-up M = 367.41, SE = 8.47, combined groups). On average across time points, the NPS group had longer continuous overnight sleep periods than the PS group by 97.5 minutes (PS M = 402.61 min, SE = 8.32; NPS M = 305.11 min, SE = 12.26; F(1, 352) = 42.80, p < .001; ES = 0.10).

Sleep Duration

There was a significant time by group interaction effect for total nighttime sleep in minutes (Wilks’ λ = 0.99; F(1, 361) = 4.24, p < .05; ES = 0.01). Both PS (baseline M = 531.07, SE = 7.21, follow-up M = 581.50, SE = 6.30, p < .001; ES = .14) and NPS (baseline M = 551.42, SE = 10.49, follow-up M = 578.38, SE = 9.19, p < .01; ES = .02) groups showed statistically significant increases in total nighttime sleep by 50.43 minutes and 26.90 minutes, respectively, although the effect size was larger for the PS group. A significant main effect of time for total 24-hour sleep time (Wilks’ λ = 0.94; F(1, 354) = 23.33, p < .001; ES = .06) indicated that infants’ total sleep increased similarly across groups by 33.2 minutes on average (baseline M = 704.68, SE = 6.92, follow-up M = 737.87.41, SE = 6.36, combined groups). There was also a group effect for total sleep time (F(1, 352) = 4.38, p < .05; ES = 0.01), indicating that caregiver-identified problem sleepers did not sleep as much as nonproblem sleepers. Although the time by group interaction effect did not reach statistical significance for total sleep time in 24 hours, it is noteworthy that the effect size for the PS group average increase was greater than that of the NPS group, and there was no statistically significant difference between groups post-intervention. No statistically significant time, group, or time by group interaction were found for nap duration.

BISQ-R Total

At baseline, BISQ-R Total scores ranged from 40.45 to 96.69 (M = 65.10; SD = 11.86) in the NPS group and from 19.80 to 80.36 (M = 47.10; SD = 10.73) in the PS group (higher scores denote better sleep). Post-intervention, BISQ-R Total scores ranged from 36.18 to 97.89 (M = 68.70; SD = 14.36) in the NPS group and from 20.66 to 98.88 (M = 57.48; SD = 14.77) in the PS group There was a significant time by group interaction for the BISQ-R Total Score (Wilks’ λ = 0.93; F(1, 402) = 29.76, p < .001; ES = 0.07). Infants in both the PS (baseline, M = 47.10, SE = 0.67; follow-up, M = 57.48, SE = 0.89, p < .001; ES = .35) and the NPS (baseline, M = 65.10, SE = 0.97, p < .001; follow-up, M = 68.70, SE = 1.28, p < .001; ES = .03) groups significantly improved their total scores from baseline to follow-up, with those in the PS group showing a greater magnitude of change. On average across time points, the NPS group had a higher BISQ-R Total score than the PS group by 15 points (PS M = 52.16, SE = .70; NPS M = 67.16, SE = 1.12; F(1, 399) = 145.81, p < .001; ES = 0.27).

CAREGIVER BEHAVIOR AND PERCEPTIONS

McNemar tests were used to compare sleep-related parent behavior from baseline to follow-up for the whole sample, as well as within PS and NPS groups (see Table 3). When considering the whole sample, statistically significant improvements were evident across all indicators of sleep-related parent behavior. Of note, reductions in feeding to sleep (χ² = 28.89, p < .001), overnight feeding (χ² = 16.57, p < .001), and picking up an infant following night wakings (χ² = 13.04, p < .001) were observed in the PS group only. Routine regularity increased (PS, χ² = 26.65, p < .001; NPS, χ² = 9.50, p < .01), bedtime was easier (PS, χ² = 18.19, p < .001; NPS, χ² = 13.79, p < .001), falling asleep was easier (PS, χ² = 34.29, p < .001; NPS, χ² = 5.62, p < .05), and caregiver confidence in managing child sleep increased (PS, χ² = 29.41, p < .001; NPS, χ² = 4.78, p < .05) for both groups from baseline to follow-up. Larger effect sizes were observed in the PS group across measures. Caregivers in the PS group perceived their infant’s sleep to be better at follow-up than they did at baseline (χ² = 28.32, p < .001). Across groups the proportion of caregivers rating their child as being a problematic sleeper significantly decreased from baseline (67.6%) to follow-up (54.2%; (McNemar χ² = 26.04, p < .001). Within the group of infants identified by their caregiver as problematic sleepers at baseline, 30.4% (n = 83) were no longer identified as problematic sleepers by their caregivers at follow-up. Over half of the caregivers in the PS group (52.4%) rated their infant’s sleep to have improved post-intervention.

Table 3.

Comparisons of Parent Behavior and Perception by Group

Dichotomous comparisons	Time 1 % (n)	Time 2 % (n)	McNemar χ2	Odds Ratio (ES, 95% CI)
Feed to fall asleep: NPS	71.0 (93)	67.2 (88)	.52	-
PS	73.3 (200)	57.1 (156)	28.89**	0.19 (0.08, 0.37)
Whole sample	72.5 (293)	60.4 (244)	24.25***	0.32 (0.19, 0.52)
Feed overnight: NPS	78.6 (103)	71.0 (93)	3.12	-
PS	84.2 (230)	74.0 (202)	16.57***	0.22 (0.09, 0.49)
Whole sample	82.4 (333)	73.0 (295)	19.55***	0.30 (0.16, 0.53)
Pick up overnight: NPS	65.6 (86)	57.3 (75)	2.86	-
PS	83.9 (229)	72.5 (198)	13.04***	0.38 (0.21, 0.66)
Whole sample	78.0 (315)	67.6 (273)	16.16***	0.43 (0.27, 0.66)
Routine regularity: NPS	57.3 (75)	72.5 (95)	9.50**	3.22 (1.49, 7.74)
PS	60.1 (164)	76.9 (210)	26.65***	4.07 (2.28, 7.70)
Whole sample	59.2 (239)	75.5 (305)	37.06***	3.75 (2.40, 6.15)
Sleep well: NPS	90.1 (118)	94.7 (124)	1.56	-
PS	44.3 (121)	61.9 (169)	28.32***	4.20 (2.36, 7.94)
Whole sample	59.2 (239)	72.5 (293)	29.89***	3.70 (2.23, 6.40)
Bedtime difficulty: NPS	23.7 (31)	7.6 (10)	13.79***	0.16 (0.04, 046)
PS	47.3 (129)	31.5 (86)	18.19***	0.39 (0.24, 0.61)
Whole sample	39.6 (160)	23.8 (96)	31.50***	0.33 (0.21, 0.49)
Difficulty falling asleep: NPS	32.8 (43)	20.6 (27)	5.62*	0.43 (0.20, 0.87)
PS	57.1 (156)	34.8 (95)	34.29***	0.27 (0.16, 0.43)
Whole sample	49.3 (199)	30.2 (122)	39.83***	0.31 (0.20, 0.45)
Confidence: NPS	48.1 (63)	59.5 (78)	4.78*	2.15 (1.08, 4.53)
PS	16.8 (46)	35.5 (97)	29.41***	4.00 (2.33, 7.26)
Whole sample	27.0 (109)	43.3 (174)	33.53***	3.20 (2.10, 5.00)

Note:

^***p < .001;

^**p < .01;

^*p < .05;

ES = effect size; CI = confidence interval; NPS = No problem sleeper group, n = 131; PS = problem sleeper group, n = 273. Routine regularity is ≥5 nights per week.

Discussion

The CSP is an empirically validated, publicly available, mHealth psychoeducational and behavioral intervention for bedtime problems and night wakings in infants. Utilizing the CSP within the framework of a free, publicly available smartphone app allows for analysis of real-world data. Overall, use of the CSP was associated with improved sleep outcomes for older infants (ages 6 to 12 months), especially for sleep consolidation. Thus, this study expanded previous research to support not only efficacy but also the real-world effectiveness of the CSP.

After CSP use and intervention recommendations were delivered, statistically significant change was evident for all measured sleep-wake variables and the complementary BISQ-R Total score, with the exception of daytime sleep duration. As expected, some of these improvements were found within the PS group only, without corresponding change in the NPS group. Within the PS group there was a shift to an earlier bedtime, decreased night waking frequency, and reduced night waking duration. Although both PS and NPS groups obtained significantly more total nighttime sleep at follow-up, effects were larger for the PS group with the PS group gaining almost twice as much sleep as the NPS group (i.e., 50.43 min for the PS group vs. 26.96 min for the NPS group). Further, by post-intervention, the average nighttime sleep duration in the PS group exceeded that of the NPS group by 3.12 minutes. Both the PS and NPS groups showed statistically significant reductions in sleep onset latency and improvements in the longest continuous stretch of overnight sleep and their total (24-hour) sleep duration. Although the problem sleep group remained significantly behind with regard to consolidated sleep (i.e., longest sleep stretch), the PS group gained an average of 39 minutes of sleep whereas the NPS group gained an average of 17 minutes.

Overall, the changes noted were potentially clinically meaningful, with reductions in night waking duration of approximately 17 minutes for the PS group and nighttime sleep increasing by approximately 50 minutes for the PS group and 27 minutes for the NPS group. Change in total sleep time, which did not differ by group, increased by around 30 minutes overall. Interestingly, there was no significant difference between groups at follow-up in terms of total sleep duration (24 hours). This finding suggests that infants in the PS group appeared to “catch up” with the NPS infants by follow-up, going from an approximate 34-minute difference at baseline to an approximate 14-minute average difference between groups at follow-up. Further-more, there was a meaningful reduction in overall caregiver perceptions that their child had a sleep problem in that just under one-third of the infants in the baseline PS group were no longer identified as problematic sleepers after the intervention. Additionally, although both the PS and NPS groups significantly improved in their BISQ-R Total scores from baseline to follow-up, the magnitude of change was greater for infants in the PS group.

Overall, resulting effect sizes from the primary analyses in the current real-world study were lower than those found in the extant literature when considering mHealth interventions for youth health promotion in general, and when comparing to more controlled efficacy trials for sleep intervention in young children. For instance, across 37 studies and almost 30,000 participants included in a recent meta-analysis assessing the effectiveness of mHealth interventions targeting health outcomes in youth (i.e., infancy through adolescence), an aggregate effect size of .22 was found (Fedele, Cushing, Fritz, Amaro, & Ortega, 2017). Further, in a meta-analysis of behavioral interventions for pediatric insomnia (primarily non-mHealth), the standard mean deviation, an effect size measurement, ranged from .24 to .34 when considering interventions effects for night waking frequency, night waking duration, and sleep onset latency in young children (Meltzer & Mindell, 2014). However, compared to the results of the initial CSP efficacy trial (Mindell et al., 2011a), this study found similar patterns of sleep improvement. That is, all sleep-related variables that overlapped across the current study and the Mindell et al. (2011a) efficacy trial improved with use of the intervention, with effect sizes ranging from .24 to .48 on those common sleep parameters. Similarly, no changes in daytime sleep were found in either study. However, effect sizes were more modest across outcomes in the current study, ranging from very small to small, with one moderate effect.

Smaller effect sizes could be due to a variety of factors. Primarily, the original efficacy study only included parent-identified problematic sleepers, as defined by both parental perception of a sleep problem (similar to this study) and bedtime difficulties. These children likely had more distinctive sleep issues than those in this real-world study and may have constituted a more homogeneous group of problem sleepers, as those in the Mindell et al. (2011a) study were recruited in the context of controlled clinical trial with specified inclusion and exclusion criteria. Other differences in the study paradigms that may have affected the differences in the magnitude of results included a predefined 2-week time line (i.e., intervention delivery for 2 weeks for all participants), comparison to a true control condition, and participant compensation in the initial CSP validation study. Nonetheless, it is noteworthy that potentially clinically meaningful changes across both groups emerged for all sleep parameters, as well as improvements in parent-reported confidence in managing sleep issues, ease of bedtime and sleep onset, and increased bedtime routine regularity. Overall, total sleep time increased almost 45 minutes for the PS group and almost 25 minutes for the NPS group. Further, the PS group caught up with or surpassed the NPS group post-intervention for bedtime routine start time, time of lights out, and total sleep time.

The changes in sleep outcomes align with caregiver report of changes in their own behavior. At follow-up, there were significant changes in caregiver behavior at bedtime and in response to night wakings. For problematic sleepers only, caregivers were much less likely to feed to sleep at bedtime and back to sleep following night wakings, as well pick up overnight upon their infant awakening. These changes align with specific recommendations provided to parents. Further-more, an increase in bedtime routine regularity was found across all families. This is a recommendation provided to all families within the CSP, and one that has been shown to be highly beneficial for healthy child sleep patterns and potentially for promoting broad child development (Mindell & Williamson, 2018).

It is important to note that this study expanded prior research on the CSP, primarily with the inclusion of infants identified as nonproblematic sleepers. As mentioned, sleep benefits for these infants were also associated with use of the CSP in that they had a shorter sleep onset latency, longer continuous stretch of sleep, and more total sleep in 24 hours at follow-up. Previous CSP research only targeted young children who had sleep problems. Thus, provision of sleep education and recommendations may help all families, not just those who perceive that their infant has a sleep problem, as noted by the improvement in parental confidence regarding sleep management across all families. These changes align with a recent finding that 91% of parents of young children wish to change something about their child’s sleep (Mindell & Leichman, 2018), regardless of whether or not the parent perceives their child’s sleep to be problematic. To further test this possibility, a randomized controlled trial that includes a wide spectrum of infants is warranted, as this design would help control for developmental changes that may be occurring. Further examination of intervention responders and nonresponder characteristics is also warranted in future studies.

These findings further provide support for the utility and importance of mHealth applications delivered via publicly available and accessible modalities, such as those that are internet or app-based. These applications have been noted to be cost-effective, with the broader digital healthcare culture (e.g., telehealth, mHealth, etc.) projected to potentially save the U.S. up to $300 billion (Roman & Conlee, 2015), generally by availability of high-quality technological health tools to a larger number of people and a lower cost. mHealth utilization also has the potential to improve access to clinical expertise, improving the well-being of children and families, and potentially improving family engagement with health-related assessment and intervention services (Law & Schueller, 2019). Widespread dissemination of validated interventions and research-based information is critically important to improve child health outcomes and sound infant development. Further, caregivers must be explicitly informed about whether apps are valid and empirically supported. Apps and other mHealth intervention platforms should include statements about empirical validation (or lack thereof) as a method to increase transparency and promote the use of evidence-based strategies in real-world contexts.

There are several limitations in this study that are linked to important directions and caveats for future research. First, inability to randomize participants within this context prevents the ability to make definitive conclusions that changes in sleep and caregiver behavior were, in fact, due to use of the CSP. Second, there were no measures of intervention adherence between the two assessments, or additional insight into overall app use. Third, limited demographic information is available for the sample. Although there were no differences found based on demographic and other study variables (e.g., days between assessments) except for caregiver age, there may be other differences based on group that are currently undetectable due to the limited demographic information. Future research should include more thorough information regarding sample characteristics within the real-world research application paradigm and a focus on specificity of the intervention and target behaviors. That caregivers of infants in the PS group were, on average, 2 years older than caregivers of infants in the NPS group also warrants future research, as previous studies have not found parental age as a predictor of parent-perceived sleep problems (e.g., Sadeh et al., 2011). Fourth, although utilizing caregiver perception in the field of infant sleep research is commonplace, capturing an objective measure of sleep through actigraphy and/or videosomnography would enhance study methodology. Fifth, the current study design did not allow for collecting information (e.g., general app use, overall user experience) that might help improve future use of the CSP within the app. Finally, including users who elected to utilize the CSP intervention tool once, or did so a second time but outside of the 4-to 28-day window, may have affected the results. For instance, those who utilized the tool only once may not have found improvements in sleep. Alternatively, they may have done so because it worked well and they did not feel the need for additional recommendations.

CONCLUSION

Overall, use of the CSP was associated with improved sleep and caregiver behavior change in the context of preliminary real-world results. Improvements were also evident in global parent perception, with fewer parents perceiving their child’s sleep as problematic after the intervention. Recommendations were associated with changes in caregivers’ behavior and improvements in caregiver-reported sleep outcomes in infants identified as problem sleepers, with improvements observed in as few as 4 days. These results expand previous research beyond efficacy to the effectiveness of an mHealth intervention for infant sleep disturbances.