Response patterns and intra-dyadic factors related to compliance with ecological momentary assessment among mothers and children

Compliance with real-time surveys was acceptable overall, but higher when mothers and children were together and lower when participants identified as Hispanic or low-income.

Abstract

Ecological momentary assessment (EMA) is a real-time sampling strategy that may address limitations in health research, such as the inability to examine how processes unfold on a daily basis. However, EMA studies are prone to limited data availability due to difficulties in implementing sophisticated protocols and systematic non-compliance with prompts, resulting in biased estimates and limited statistical power. The objectives of this study were to describe the availability of data, to examine response patterns, and to analyze factors related to EMA prompt compliance in a dyadic EMA study with mothers and children. Participants (N = 404) each received up to eight EMA prompts (i.e., audible pings) per day for a total of 7 days. Each EMA survey consisted of items assessing affect, perceived stress, and social context. Participants responded to approximately 80% (range: 3.4%–100%) of prompted EMA surveys, and completed 92.6% of surveys once started. Mothers and children identifying as Hispanic, as well as mothers in lower-income households, were less likely to comply with any given EMA prompt. Participant dyads were more likely to comply with prompts when they were together. Understanding factors related to systematic EMA prompt non-compliance is an important step to reduce the likelihood of biased estimates and improve statistical power. Socioeconomic factors may impede mothers’ compliance with EMA protocols. Furthermore, mothers’ presence and involvement may enhance children’s compliance with EMA protocols.

Implications

Practice: Behavioral health is increasingly utilizing mobile technology as a tool for just-in-time adaptive interventions, but studies must cautiously examine how findings may be influenced by various contexts, and among individuals.

Policy: Research using ecological momentary assessment is optimal for policy use due to its implementation of technology and high ecological validity, therefore, a more comprehensive understanding of factors related to response patterns may reduce bias in data.

Research: Given the time-intensive longitudinal nature of studies using ecological momentary assessment, it is important to understand barriers to protocol implementation, overall data availability, and differential response patterns in order to reduce burden on participants, maximize sample size, and mitigate response biases.

INTRODUCTION

Cross-sectional studies and longitudinal studies with infrequent sampling are unable to capture intra-individual variation throughout a measurement period, especially for mood states and activity that are expected to change throughout the day. Ecological momentary assessment (EMA) [1] is a real-time self- report sampling strategy that addresses methodological concerns found in such studies. Modern protocols are typically deployed on smartphones as multiple brief surveys throughout the course of a day that can be used to collect information about stress, emotional states, physical activity, and eating as they occur [2]. However, issues associated with data availability and systematic non-compliance rates are significant limitations in EMA studies. Complex and novel protocols using sensors may collect as little as half of the anticipated data [3]. Furthermore, even studies with acceptable data availability have reported systematically varying patterns of EMA prompt non- compliance as a function of time, across demographic factors, and dependent on study outcomes [4, 5].

Both technical issues and participant non-compliance may influence availability of data in EMA studies. Technical issues that affect data availability include interrupted network connectivity where data does not reach servers, excessive battery draining resulting in the device not being in operation, and runtime errors caused by unknown bugs that crash the software. Participant forgetfulness, protocol burden, and schedule incompatibility are among many factors thought to affect compliance with EMA protocols [6]. While non-compliance directly results in missing data, systematic differences may still exist in response patterns such as: survey completion rate, defined as the proportion of surveys completed to those started; completion time, defined as the length of time it takes to finish a survey; and response latency, defined as the number of auditory/tactile prompts received or the length of time before a participant responds to an EMA survey [6].

Understanding data availability can help overcome technical and practical challenges to implementation in future EMA studies. Moreover, missing data from low compliance rates are a potentially significant problem in EMA studies because analyses performed with missing data can produce biased estimates and limit the statistical power of multilevel models [1]. Given the increased potential for differential response rates in studies that sample participants in their everyday lives, it is important to fully understand whether missing data due to EMA non-compliance are correlated with time-varying and time-invariant factors. Time-varying factors include any study outcomes and predictors such as affect and social context that may impact results if systematically missing. Time-invariant factors, such as male gender and non-White ethnicity have also been associated with non-compliance across EMA studies [7]. The results from these analyses can be used to determine how to best mitigate missing data to reduce biases [8].

While previous EMA studies have been conducted in children and adults independently [2,9], no known study to date has deployed EMA in dyads, defined as mother–child pairs. Dyadic studies using EMA allow researchers to examine within-dyad or intra-dyadic covariation and cross-over effects of psychological factors on eating and physical activity behaviors. Yet, simultaneous compliance between parent and child is necessary at each time point in dyadic studies, subsequently compounding problems arising from data availability. However, even less is known about the availability of data and correlates of non-compliance in EMA studies involving parent–child dyads. Therefore, there is a critical need to explore data availability in the context of dyadic studies to guide future study development.

The first objective of this study was to describe patterns of data availability that arise from mobile phone hardware and software technical issues. The second objective was to examine differences in response patterns such as prompt compliance rates, survey completion rates, response latency, and completion times between mothers and children. The third objective was to attempt to explain patterns of non-compliance with survey prompts using time-varying and time-invariant covariates.

MATERIALS AND METHODS

This study used data from the first measurement period of six planned waves of the Mothers and Their Children’s Health (MATCH) study. MATCH is an ongoing longitudinal investigation of parenting factors and obesity in a sample of mothers and children. The study examines the effects of mothers’ stress on children’s physical activity and eating behaviors. The study employs an observational, dyadic, case-crossover design, with each dyad serving as their own control to assess associations among repeatedly measured dependent variables [10]. A more detailed description of the protocol for the MATCH study has been published elsewhere [11].

Participants

Participants resided in the greater Los Angeles metropolitan area and consisted of an ethnically diverse sample of mothers and their children. Mother–child dyads were included based on the following criteria: (i) the child was in fourth or fifth grade; (ii) the mother had ≥50% custody of the child; and (iii) mother and child could read English or Spanish. Mother–child dyads were excluded if: (i) either mother or child was taking medications for thyroid function or psychological conditions; (ii) either mother or child had health issues that limited physical activity; (iii) the child was enrolled in a special education program; (iv) either the mother or child was using oral/inhalant corticosteroids for asthma; (v) the child was underweight based on age–sex adjusted BMI percentile; (vi) the mother worked more than two weekday evenings or more than 8 hr on any weekend day; or (vii) the mother was pregnant.

Procedures

Participants were initially recruited at local school events and screened for eligibility by phone. Mothers reported motivation to participate due to the novelty of the study and the relatability of the study objectives to their lives. Eligible participants traveled to a nearby school or community center for an in-person data collection session during which they completed a paper-and-pencil questionnaire and anthropometric assessments, and received instructions for the smartphone-based EMA protocol. A total of 416 mother–child dyads expressed initial interest in participating in the study by returning completed recruitment sheets with contact information. Of these, 269 mother–child dyads were screened by phone for eligibility and a subgroup of 250 dyads were subsequently determined to be eligible to participate. From this group, a total of 202 mother–child dyads consented to complete Wave 1. Mothers and children each received up to eight signal-contingent EMA prompts on a smartphone across the 7-day study period per wave. The study assessed participants in everyday situations, and dyads were instructed to proceed with typical daily routines while answering all surveys during the afternoons and evenings. Technical and non-compliance issues were monitored and addressed twice weekly by research staff members via phone calls to the mother for the dyad. If the mother could not be reached, a customized text message was sent as a reminder and included a compliance progress report. Each dyad received $200 for each completed wave. Mothers provided informed consent and parental permission, and children assented to participation. Procedures were approved by the Institutional Review Boards at the University of Southern California and Northeastern University.

A custom software application (app) for smartphones running Android OS (Google Inc., Mountainview, CA) collected EMA data from participants. The app wirelessly uploaded EMA data and stored it on a secure internet-accessible server that allowed investigators to monitor non-compliance. Mothers and children who owned compatible Android smartphones were given the option to download the EMA app and complete the EMA surveys directly from their personal phones; 11% of participants chose this option. Participants without a compatible smartphone (e.g., iPhone) or not wishing to use their own smartphone for the study were loaned a Motorola MotoG (Motorola Mobility, Chicago, IL) smartphone. Mothers were given the option to use the app in English or Spanish (6% of mothers chose Spanish), while children could only use the app in English. Participants were instructed to complete an EMA survey lasting approximately 3 min consisting of up to 31 items for mothers and 13 items for children upon receiving a prompt. The app issued a prompt as a standard Android notification with a survey link, signaled by an auditory alert or vibration. The app re-prompted participants two additional times at 3-min intervals if participants did not complete the survey. The survey was inaccessible if the participant did not answer or did not complete the survey in a timely manner, defined as 10 min after the initial prompt. The app prompted EMA surveys across six full days and two half days for a total of 7 days of data. The app delivered prompts up to seven times per day on weekend days (7–8am, 9–10am, 11am–12pm, 1–2pm, 3–4pm, 5–6pm, and 7–8pm windows) and up to three times per day on weekdays (during non-school time: 3–4pm, 5–6pm, and 7–8pm windows). Surveys asked mothers and children to indicate daily bed-time and wake-time, and the app tailored survey schedules to occur during waking hours accordingly. The app prompted mothers and children in pairs, with mothers receiving a randomly scheduled prompt in the first half of the hour and children receiving a randomly scheduled prompt in the second half of the hour to avoid contamination effects from simultaneously completing surveys. Mothers also completed an additional unpaired late-evening survey prompt each day (9–9:30pm).

Measures

EMA response patterns

For each mother and child, response patterns were characterized and assessed as the primary outcome of interest using the following four outcomes with survey as the unit observation: (i) Prompt compliance (i.e., 1 = compliant and 0 = non-compliant) was defined as responding to a survey within the 10 min of the initial prompt and answering at least one question on that survey; (ii) Survey completion rate was defined as the proportion of fully completed surveys to answered surveys per participant throughout the study period; (iii) Response latency was defined as the number of prompts a participant received prior to starting a survey; (iv) Completion time was defined as the length of time between receiving an initial prompt and finishing the last question on the survey for that prompt. The compliance rate represents the availability of surveys and the completion rate represents the completeness of available surveys, while response latency and completion time characterize timeliness and length of response as a data attribute.

Time-varying covariates of compliance

EMA items assessed positive and negative affect, perceived stress, number of specific stressors, and social context in mothers and children. EMA items assessing positive and negative affect were based on the circumplex model composed of valence and arousal [12] in mothers and children with four response options: “not at all,” “a little,” “quite a bit,” and “extremely.” Positive affect was assessed with two items in mothers (“happy” and “calm/relaxed”) and children (“happy” and “joyful”). Negative affect was assessed with three items in mothers (“stressed,” “sad/depressed,” and “frustrated/angry”) and children (“stressed,” “sad,” and “mad”). Perceived stress was measured using two items adapted from the Perceived Stress Scale (PSS) [13]: “How certain do you feel that you can deal with all the things that you have to do RIGHT NOW?” and “How confident do you feel about your ability to handle all of the demands on you RIGHT NOW?” In mothers, exposure to specific types of psychosocial stressors was assessed using items adapted from the Daily Hassles Scales [14]. Children were asked to indicate whether they had experienced specific types of psychosocial stressors using items modified from a children’s stress scale (e.g., “Having a lot of homework to do,” “Not doing well at something”) [15].

Time-invariant covariates of compliance

Height and weight were measured in duplicate using an electronically calibrated digital scale (Tanita WB-110A) and professional stadiometer (PE-AIM-101). Body mass index (BMI; kg/m²) was calculated for mothers. For children, CDC age- and gender-specific BMI z-scores were determined using the Stata extension zanthro [16]. Mothers completed paper questionnaires assessing age, race/ethnicity for themselves and their child, their child’s eligibility for free or reduced-price lunch at school, their own marital status, and their average hours worked per week. Children reported age and sex.

Data processing and analyses

To address the first objective of describing data availability, an EMA prompt-level consort diagram was developed, which reported the flow of data availability and reasons for missing data. To meet the second objective of examining EMA response patterns, a multilevel logistic regression was used to test differences in prompt compliance and separate multilevel linear regressions were used to test differences in survey completion rate, response latency, and completion time between mothers and children. Additionally, gender, race, ethnicity, annual household income, time of day, and day of week were tested as factors related to prompt compliance, survey completion rate, response latency, and completion time, independently for mothers and children.

To address the third objective of exploring patterns of non-compliance with EMA survey prompts, time-varying factors related to prompt compliance were also analyzed using multilevel logistic regressions separately for mothers and children. A time-lagged prompt was defined as the previous prompt received by the target participant on the same day, approximately 2 hr prior to the original prompt. A concurrent prompt was defined as a prompt received by a mother within the same 1-hr prompting window as her child. Each time-varying factor was disaggregated into between-subject (person-level, grand mean centered) and within-subject (observation level, person-level mean centered) terms and entered into one model [17]. Models predicting co-occurring compliance adjusted for average compliance rates. Time-invariant factors related to compliance to any given prompt were analyzed using separate multilevel logistic regressions for each factor, separately for mothers and children. All models were screened for autoregressive residuals and variance–covariance structures were specified as exchangeable or auto-regressive.

RESULTS

Descriptive statistics

Demographic characteristics for the sample with some EMA data available for both dyad members are shown in Table 1 (N = 190). Descriptive statistics for measures of perceived stress, specific types of stressors, negative and positive affect, stressful event, eating and activity behaviors, and social context between mothers and children are shown in Table 1 (n = 4,564 and n = 3,552 for mothers and children, respectively).

Table 1.

| Participant Characteristics and Descriptive Statistics for Ecological Momentary Assessment (EMA) Variables

	Mothers		Children
Variable	N	%	N	%
Sex
Male	0	0.0	90	47.4
Female	190	100.0	100	52.6
Annual household incomea
Less than $35,000	50	26.5
$35,000–$74,999	57	30.2
$75,000–$114,999	37	19.6
$115,000 and above	45	23.8
Reduced/Free lunch at schoolb
Yes			82	43.8
No			105	56.2
Ethnicity
Hispanic	93	51.1	103	45.8
Loaned study phone
Non-Hispanic	97	48.9	87	54.2
Own device	36	18.9	7	3.7
Loaned device	154	81.1	183	96.3
Marital status
Married	128	67.4
Single or separated	62	32.6
EMA social context
With others or alone	1,987	46.6	1,328	40.9
With dyad member	2,276	53.4	1,919	59.1
	M	SD	M	SD
Agec	40.8	6.1	9.6	0.9
Body mass index (BMI)d	28.6	6.4
BMI percentilee			63.9	28.8
Hours worked per weekf	33.7	12.7
EMA positive affectg	2.65	0.76	3.05	0.94
EMA negative affectg	1.35	0.49	1.26	0.50
EMA perceived stressg	1.91	0.73	1.15	0.30
EMA number of specific stressors	0.54	0.82	0.37	0.75

N = 190 dyads.

^aIncome was missing for one mother.

^bLunch status was missing for three children.

^cAge was missing for one mother.

^dBMI was missing for six mothers.

^eBMI was missing for seven children.

^fHours worked per week was missing for 39 mothers.

^gHigher scores indicate greater positive, negative affect, or perceived stress.

Data availability

A flow chart displaying data availability and sources of missing EMA data for the 202 dyads that completed Wave 1 is shown in Fig. 1. Of the 12,726 programmed surveys (35 surveys each across 202 mothers, and 28 surveys each across 202 children), 318 (2.5%) surveys were not prompted due to the phone being powered off, the battery being drained, or the app being disabled due to a software bug. On 1,118 (8.8%) occasions, participants were not prompted because the prompt was scheduled outside of study parameters including prompts that were outside of the participant’s tailored sleep and wake schedule, before a participant received their device, or after the participant dropped off their device. Miscellaneous issues such as a protocol change early in the study, data loss due to server upload issues, and other unknown technical problems accounted for the remaining 787 (6.2%) missing programmed prompts. Overall, two mothers did not receive any survey prompts, one mother failed to answer any survey prompts, and one mother did not receive more than two prompts during the 8-day monitoring period. Likewise, two children did not receive any survey prompts, two children did not answer any survey prompts, and one child received no more than two prompts during the 8-day monitoring period. All of these participants were excluded from further analyses. Thus, there were 198 mothers and 192 children who received at least three prompts. Of these participants, 162 (81.8%) of mothers and 184 (95.8%) of the children were loaned a smartphone for the study. Of the loaned smartphones, one was locked by the participant without knowledge of the passcode and two were damaged beyond repair; data were retrieved from server uploads for all three devices. There were no differences in smartphone usage among Hispanic or non-Hispanic mothers or children (χ²(1) = 0.77, p = .38 and χ²(1) = 0.87, p = .35, respectively).

Fig. 1.

| Flow chart of ecological momentary assessment (EMA) data availability. Lv.1 refers to EMA prompt-level observations (n) and Lv. 2 refers to person-level data (N). Solid lines indicate available data. Dashed lines indicate data lost due to reason indicated within each box.

EMA response patterns

Prompt compliance rates

Mothers with valid data answered 4,730 of the 5,829 surveys that were prompted, resulting in an average compliance rate of 81.1% (SD = 20.6%) among mothers. Children with valid data answered 3,590 of the 4,595 surveys that were prompted, resulting in a mean compliance rate of 78.0% (SD = 19.9%) among children. Mothers were more likely (OR: 1.37, 95% CI: 1.01–1.85) to respond to a prompt than children (z = 2.02, p < .05). Overall, there were 190 mother–child dyads with at least some valid data for each member of the pair; dyads with no data available for both mother (N = 8) and child (N = 2) were excluded from subsequent analyses. Children were more likely to comply with EMA prompts on weekend days (OR: 1.34, 95% CI: 1.15–1.56) than on weekdays (p < .001). On weekdays, both children (OR: 0.71, 95% CI: 0.57–0.89) and mothers (OR: 0.58, 95% CI: 0.46–0.72) were less likely to comply with EMA prompts in the afternoon than in the evening (p < .01 and p < .001, respectively). However, on weekend days, there was no statistically significant difference with compliance to a prompt across the day for mothers and children (p’s > .05).

Survey completion rates

Mothers completed 4,263 of the 4,564 survey prompts that were answered, yielding an overall completion rate of 93.4%. Children finished 3,247 of the 3,552 EMA survey prompts that were answered, yielding an overall survey completion rate of 91.4%. However, after adjusting for clustering of observations within participants, children (M = 94.0%, SE = 0.01%) were more likely (OR: 1.92, 95% CI: 1.11–3.32) to finish all questions after starting a survey than mothers (M = 90.8%, SE = 0.01%) (p ≤ .05).

Response latency

Indicators for response latency revealed that 4,718 (58.1%) of answered EMA surveys were responded to after the first prompt, 1,957 (24.1%) after the first re-prompt (3 min later) and 872 surveys (10.7%) after the second re-prompt (6 min later). The likelihood of responding to a survey after the first prompt versus one of the subsequent re-prompts was greater for children (68.7%, SE=1.3%) versus mothers (47.6%, SE = 1.4%) (OR: 2.71, 95% CI: 2.24–3.27), (p < .001).

Completion time

Completion time was calculated for surveys completed at the first prompt (n = 3,398). On average, mothers completed each EMA survey in 124.0 s (SD = 39.3, range: 1–603 s) and children completed each EMA survey in 81.5 s (SD = 34.7, range: 18–241 s). Age was positively associated with completion time (B = 0.59, SE = 0.23) among mothers (p < .05). However, age was inversely associated with completion time (B = −4.25, SE = 1.42) among children (p < .01). Among mothers and children, there were no statistically significant differences in EMA completion time by sex or ethnicity (p’s > .05). Mothers with a reported annual household income between $35,000 and $74,999 or greater than $105,000 completed surveys faster (B = −11.69, SE = 3.77 and B = −10.62, SE = 3.99; respectively) than mothers making less than $34,999 yearly (p’s < .01). Mothers and children completed surveys 4.05 and 5.51 s faster, respectively (SE = 0.50; SE = 0.39, respectively) per chronological day in the study (p’s < .001). Children completed surveys 10.55 s (SE = 1.26) faster on weekends than on weekdays (p < .001). Similarly, mothers completed surveys 3.85 s (SE = 1.56) faster on weekends than on weekdays (p < .05). Children completed evening surveys 4.98 s (SE = 2.20) faster than afternoon surveys on weekdays (p < .05); similarly, they completed evening surveys 4.1 s (SE = 1.87) faster than morning surveys on weekends (p < .05). Mothers completed surveys 7.02 s (SE = 2.43) slower on weekend evenings than on weekend mornings (p < .01), but not throughout the day on weekdays (p > .05).

EMA compliance factors

Time-varying factors related to EMA prompt compliance

Table 2 shows the results of multilevel logistic regression analyses examining time-varying factors that are associated with likelihood of EMA prompt compliance for mothers and children. A greater likelihood of complying with a prompt was found among mothers when a child reported being together with them at the concurrent prompt (p < .05). Likewise, children were more likely to respond to a prompt if their mother reported being together at the concurrent prompt (p < .001). Compared with children whose mothers did not comply with the concurrent prompt, children with mothers that complied concurrently were twice as likely (OR: 2.38, 95% CI: 1.92–2.95) to comply with a prompt (p < .001). Similarly, compared with mothers whose children did not comply with the concurrent prompt, mothers with children who complied to the concurrent prompt were twice as likely (OR: 2.41, 95% CI: 1.94–2.98) to respond to a prompt (p < .001).

Table 2.

| Results of Multilevel Logistic Regression Models of Time-Varying Predictors of Ecological Momentary Assessment (EMA) Compliance

		Mothers						Children
Variable		Between-Subjects			Within-Subjects			Between-Subjects			Within-Subjects
		OR	95% CI	z	OR	95% CI	z	OR	95% CI	z	OR	95% CI	z
Social context	Mother							1.59	0.63–4.03	0.98	1.69	1.35–2.11	4.58***
	Child	1.68	0.63–4.47	1.04	1.33	1.02–1.73	2.10*
Positive effect	Mother	1.21	0.79–1.85	0.88	1.01	0.84–1.21	0.13	1.95	0.93–4.07	1.78	1.24	0.94–1.65	1.52
	Child	1.31	0.89–1.93	1.38	0.94	0.79–1.11	−0.76	1.31	0.86–1.98	1.25	0.94	0.80–1.11	−0.71
Negative effect	Mother	0.71	0.33–1.54	−0.87	1.10	0.84–1.45	0.70	1.13	0.84–1.52	0.78	0.97	0.81–1.17	−0.29
	Child	0.55	0.22–1.34	−1.32	0.81	0.64–1.03	−1.70	1.16	0.55–2.42	0.39	1.07	0.83–1.39	0.55
Perceived stress	Mother	1.02	0.71–1.47	0.12	1.08	0.86–1.35	0.64	0.66	0.32–1.33	−1.17	1.24	0.93–1.66	1.48
	Child	0.32	0.09–1.14	−1.76	1.00	0.64–1.56	−0.01	0.88	0.62–1.26	−0.68	0.96	0.78–1.18	−0.40
Number of specific stressors	Mother	0.77	0.48–1.24	−1.09	1.04	0.89–1.20	0.45	0.72	0.26–2.01	−0.63	1.25	0.76–2.06	0.88
	Child	0.78	0.44–1.37	−0.86	0.97	0.80–1.16	−0.36	0.96	0.61–1.51	−0.17	1.00	0.87–1.14	−0.07

Each time-invariant predictor was tested in separate models. The dependent variable, EMA compliance, was coded as a binary (1 = EMA compliance, 0 = EMA non- compliance). All models were screened for autoregressive residuals.

***p < .001.

*p < .05.

Time-invariant factors related to EMA prompt compliance

Table 3 shows the results of multilevel logistic regression analyses examining time-invariant factors that are associated with likelihood of EMA prompt compliance for mothers and children. Ethnicity was significantly related to response rate, such that Hispanic mothers and children were less likely to respond that non-Hispanic mothers and children (p’s < .05). Mothers with an annual household income greater than $35,000 but less than $105,000 were more likely to comply with EMA prompts than those with income less than $35,000 (p < .05). Lastly, BMI percentile was inversely associated with EMA prompt compliance in children (p < .05).

Table 3.

| Results of Multilevel Logistic Regression Models of Time-Invariant Factors Related to Ecological Momentary Assessment (EMA) Compliance

		Mothers			Children
Time-Invariant Variable		OR	95% CI	z	OR	95% CI	z
Age	Mother	1.01	0.98–1.05	0.76	1.00	0.97–1.03	−0.01
	Child	0.92	0.72–1.17	−0.65	0.92	0.74–1.14	−0.80
Child’s gendera		0.91	0.58–1.43	−0.42	0.74	0.49–1.10	−1.50
Ethnicityb	Mother	0.49	0.32–0.77	−3.14**	0.54	0.37–0.81	−2.99**
	Child	0.48	0.31–0.76	−3.17**	0.58	0.39–0.86	−2.68**
Loaned study phonec		0.93	0.52–1.67	−0.23	1.28	0.43–3.81	0.44
Household incomed
$35,000–$74,999		1.88	1.05–3.37	2.11*	1.42	0.84–2.41	1.30
$75,000–$104,999		2.08	1.08–4.01	2.20*	1.65	0.91–2.99	1.65
>$105,000		1.64	0.88–3.06	1.55	1.38	0.79–2.43	1.13
Child’s reduced/free lunche		0.76	0.48–1.18	−1.23	0.81	0.54–1.22	−1.02
Mother’s BMI		1.01	0.97–1.05	0.56	0.99	0.96–1.03	−0.40
Child’s BMI percentile		0.99	0.99–1.00	−1.87	0.99	0.99–1.00	−2.04*
Hours worked per week		1.00	0.98–1.02	−0.05	0.99	0.98–1.01	−0.65
Marital statusf		1.17	0.72–1.90	0.65	1.10	0.71–1.69	0.42

Each time-invariant factor was tested in separate models. The dependent variable, EMA prompt compliance, was coded as binary (1 = at least one item in prompt answered, 0 = prompt missed). All models were screened for autoregressive residuals.

^aBoy vs. girl (ref.).

^bHispanic vs. Non-Hispanic (ref.).

^cLoaned study phone vs. not loaned a study phone (ref.).

^dRef.: <$34,999.

^eChild receives free lunch vs. child does not receive free lunch (ref.).

^fMarried vs. not married (ref.).

*p < .05.

**p < .01.

DISCUSSION

The study illuminated potential challenges associated with using EMA methods to examine dyadic within-day variations in psychological and health behavior outcomes in a socioeconomically and racially diverse population. The research was unique in its intensive exploratory data analysis of time-variant and invariant factors related to compliance with EMA protocols and is one of the first known studies to examine intra-dyadic factors related to compliance with EMA protocols. The study described in detail how approximately 18% of scheduled EMA prompts are not administered due to technical issues such as data loss and battery life. Analyses of response patterns revealed that children and mothers showed differences in compliance, completion rate, response latency, and completion time. Analyses with time-varying covariates revealed systematic compliance when mothers and children reported being with each other or complied to prompts concurrently. Income, ethnicity, and BMI were identified as time-invariant factors related to compliance.

Data availability was consistent with other non- dyadic EMA studies, but also provided distinctive analyses of dyadic compliance data. Compared with dyadic studies in adult couples [18], the data availability was somewhat lower, with retention of data from approximately 65% of EMA prompts programmed to be delivered by the smartphone app. However, this number may be due to the substantial amount of missing data from software errors and battery life, as participant compliance was similarly high, the number of questions per survey was comparable, and other reported protocol parameters were similar to short-term EMA studies [19]. This source of data loss indicated that software errors and phone compatibility should be priorities in the testing and refining phases of complex EMA implementation. Given the amount of data lost due to sleep schedule adjustments, future studies should consider software features allowing precise tailoring to unique sleep schedules.

Children were less likely to comply with any given EMA prompt than mothers, but were more likely to respond to the first EMA prompt in a series than mothers. These data suggest that children may have been less receptive of EMA re-prompting protocols than mothers. Both mothers and children completed surveys faster with each chronological day of measurement period, indicating that participants may have learned to complete surveys more efficiently over time. However, it is not known whether the decrease in survey completion time occurred because of study fatigue, such that participants were no longer providing valid answers. Similarly, improvements in survey completion time on weekends were a function of fatigue, as opposed to adaptation [6]. Consistent with EMA protocols in non-dyadic studies, mothers and children were less likely to respond to prompts in the afternoon compared with evenings [5]. Participants missed prompts occurring in the afternoon before 5pm likely due to school or work obligations that could not be addressed by study design, since most work days occur on weekdays, and no temporal association was observed on weekends. Custom prompting times designed around work schedules like the tailored wake and bed times used in this study could address this finding. Next, older children and younger mothers completed surveys faster than younger children and older mothers, respectively. Additionally, low annual household income was associated with decreased EMA completion time in mothers. These findings may reflect maturation in children and socioeconomic differences in familiarity with technology in adults that warrant consideration in future EMA development [20].

Mothers and children were more likely to comply with any given prompt if the other dyad member reported being together compared with when the other dyad member reported being apart. Future studies may seek to use this finding advantageously when incorporating momentary measures of critical importance in dyads, such as glucose measures, by scheduling these prompts at times where parents and children are likely to be together. Mothers and children were more than twice as likely to respond to any given EMA prompt if the other dyad member also responded to a prompt within 1 hr, whether or not dyad members were together. In settings where low compliance is observed for a dyad, these findings suggest that interventions, such as reminder phone calls, may be effective even when one dyad member cannot be reached due to reasons such as participation in after-school sports.

EMA prompt compliance in mothers and children did not differ by most person-level covariates examined, consistent with studies examining similar health behaviors [19]. Hispanic mothers and any mothers with Hispanic children were less likely to comply with EMA prompts, while Hispanic children and any children with Hispanic mothers were less likely to comply with EMA prompts. Similarly, mothers with a reported annual income of $34,999 or less were less likely to comply with EMA prompts than their higher-income counterparts. These findings corroborate those of other non-dyadic EMA studies and suggest that addressing differential compliance across socioeconomic factors in future studies may be key to improving validity of results in diverse populations [18, 21]. Moreover, results indicated that smartphone familiarity, based on smartphone loan rates, did not seem to be a key contributor to differential compliance rates, nor did usage differ among Hispanic and non-Hispanic mothers. Researchers should further investigate ethnic differences to elucidate behavioral processes that may influence compliance. Similarly, children with higher BMI were also less likely to comply with EMA prompts, but more research is needed to determine whether a similar relationship exists with accelerometer-recorded physical activity or whether different explanatory mechanisms may drive children with higher BMI to not answer health-related questions [2].

The study is limited first by design and second by its exploratory nature. First, concurrent windows occurred in a period of 60 min with mothers always receiving the initial survey. Mothers and children could, in theory, have received prompts up to 60 min apart. Therefore, concurrent associations may have some directionality. Furthermore, time-lagged responses used data from approximately 2 hr prior; effects may either be attenuated over time or represent an aggregate person-level factor. Second, the study used sequential bivariate tests without adjusting for multiple comparisons. Therefore, the results should only be interpreted as exploratory in nature and serve to guide future confirmatory research in EMA methodology. Moreover, a relatively high compliance rate results in unstable estimates for some predictors when cell counts were low for a particular comparison group (e.g., household income). Lastly, compliance was defined as answering at least one item in a timely manner. Future research may seek to use three categories of compliance to distinguish non-compliance, partial compliance, and completion, as children were more likely to complete surveys than mothers.

The findings in this study present an important contribution to translational research as research progressively adopts real-time assessment methodology. The study suggests that conducting smartphone-based dyadic studies in mothers and children with time-intensive assessments of psychological and behavioral variables may provide useful data given the acceptable EMA compliance rates [7]. These patterns of EMA non-response suggest that comprehensive analyses of missing data should be conducted as a standard step in EMA studies [19]. Moreover, standardized reporting of data availability, compliance, and other response patterns would allow for systematic reviews and meta-analyses of EMA studies to elucidate factors associated with compliance. EMA methods research should consider using passive data from smartphone sensors to examine factors such as activity or proximity to one’s home, without the need for wearables and other devices. Lastly, the association that time-invariant factors such as income and ethnicity may have with compliance warrants considerable analyses, as inclusion of a population in a study may not necessarily mean inclusion of their data.

Compliance with ethical standards

Conflict of interest: The authors report no conflicts of interest.

Ethical statement: No animals were used in this study.