Measurement Reactivity in Ecological Momentary Assessment Studies of Movement-Related Behaviors

Abstract

Measurement reactivity has implications for behavioral science, as it is crucial to determine whether changes in constructs of interest represent true change or are an artifact of assessment. This study investigated whether measurement reactivity occurs for movement-related behaviors, motivational antecedents of behavior, and associations between them. Data from ecological momentary assessment studies of older adults (n = 195) and women in midlife (n = 75) lasting 8–10 days with 5–6 prompts/day and ambulatory monitoring of movement were used for this secondary data analysis. To examine potential drop-off patterns indicative of measurement reactivity, multilevel models tested whether behavior, antecedents, and associations changed after the first or first 2 prompts compared with remaining prompts and the first, first 2, or first 3 days compared with remaining days. Older adults’ sedentary behavior was lower, and time spent upright and intentions and self-efficacy to stand/move were higher on the first 2 and first 3 days compared with remaining days. Associations between intentions and self-efficacy and subsequent sedentary behavior were weaker earlier in the study compared to later. For women in midlife, light physical activity was higher at the first and first 2 prompts compared with remaining prompts, and physical activity motivation was higher across all prompts and days tested. There was a stronger association between intended and observed minutes of moderate to vigorous physical activity on the first 2 days compared with remaining days. Measurement reactivity appeared as expected for movement-related behaviors and motivational antecedents, though changes in associations between these constructs are likely do not reflect measurement reactivity.

Device-based monitoring and intensive smartphone-based assessments (e.g., ecological momentary assessment [EMA]) are increasingly used to capture movement-related behaviors such as physical activity (PA) and sedentary behavior (SB) and their motivational antecedents in everyday life (O’Reilly & Spruijt-Metz, 2013). These approaches are advantageous for either eliminating the need for recall entirely or shortening the recall time-frame, as well as enhancing ecological validity of findings. Asking a participant to wear a device or repeatedly report on their movement-related behaviors or motivation may introduce changes in one’s behavior, cognitions, or beliefs as a result of its measurement—a phenomenon known as measurement reactivity (French & Sutton, 2010). Measurement reactivity has implications for behavioral science, as it is crucial to determine whether changes in constructs of interest represent true change or are an artifact of assessment.

Measurement reactivity occurs when the act of assessing a phenomenon of interest changes that phenomenon (French & Sutton, 2010). Reactivity is thought to be induced at the beginning of a study protocol, when there is a heightened awareness of one’s behaviors, cognitions, or beliefs (Barta et al., 2012). Individuals may experience self-consciousness or evaluation anxiety, present themselves in a favorable light or in line with a positive self-image, or increase effort necessary to reflect on and rate constructs of interest, ultimately resulting in modified behavior, cognitions, or beliefs (Barta et al., 2012). Fortunately, measurement reactivity dissipates and typically resolves after the first 1–2 days of a study protocol, returning to “normal” levels thereafter (French & Sutton, 2010). As estimates of behavior, cognitions, and beliefs are often aggregated over this period; however, measurement reactivity may bias these estimates by skewing them meaningfully upward or downward (French et al., 2021).

Measurement Reactivity in Movement-Related Behaviors

Measurement reactivity has been documented in studies of movement-related behaviors (Miles et al., 2020), which have operationalized reactivity in multiple ways. Some researchers used an experimental design and cover stories to conceal the purpose of the PA or SB monitoring device (Clemes & Parker, 2009); others sealed the display of devices, to prevent feedback from self-monitoring (Clemes & Deans, 2012). In contrast, most studies used observational designs, in which participant behavior was tracked for several days. Reactivity is usually defined in one of two ways: (a) as a linear trend, assuming desired behaviors (e.g., PA) would gradually decrease as the study progresses, while undesired behaviors (e.g., SB) would gradually increase or (b) as a “drop-off’ after initial days or observations and then consistency. A recent review concluded that, regardless of operationalization of reactivity or PA (e.g., steps, moderate to vigorous PA [MVPA]), the effects of measurement reactivity are small but meaningful (König et al., 2022). Most studies included in this review aggregated movement-related behaviors at the day level to determine whether daily PA behavior changes between (or across) days.

In contrast, there has been little attention to measurement reactivity when PA behavior is examined more intensively (i.e., within days, such as in hour intervals). Examining measurement reactivity in movement-related behaviors for protocols that use multiple within-day assessments is important, given the increased focus on within-day fluctuations in motivational constructs and behavior using intensive methods such as EMA (Bittel et al., 2023; Perski et al., 2022). EMA is a real-time data capture methodology in which individuals are intensively and repeatedly assessed within days on some phenomenon of interest (Stone & Shiffman, 1994). EMA protocols can include single items or brief assessments related to the antecedents of the phenomenon of interest (Stone & Shiffman, 1994). Because data from EMA studies are date- and time-stamped, it facilitates pairing of data from other sources such as device-based measures of PA and SB. Of those EMA studies that have examined measurement reactivity in device-based behavior, measurement reactivity has been investigated in the context of acute behavior changes in response to a single EMA prompt (i.e., comparing behavior immediately before to immediately after the EMA prompt) as opposed to changes in behavior as time in the study goes on (Dunton et al., 2012; Maher et al., 2018, 2021). Therefore, it is unclear the extent to which movement-related behaviors following an EMA prompt may change over the course of an intensive study.

Measurement Reactivity in Motivational Antecedents of Movement-Related Behaviors

In addition to behavior, measurement reactivity can affect motivational constructs (Barta et al., 2012). For instance, smokers completing an EMA protocol before the quit attempt had higher levels of self-efficacy immediately prior to the quit attempt, compared with smokers who completed the protocol after the quit attempt or who did not complete any EMA protocol, suggesting that the act of assessing motivational constructs may impact motivation itself (Rowan et al., 2007). University students also showed decreases in their positive affect and emotional awareness in a 14-day EMA protocol (Eisele et al., 2023). This broader EMA literature shows that the pattern of responses in mean levels of internal states “flatten” after a period, rather than changing linearly over time (Eisele et al., 2023), which would suggest measurement reactivity that aligns with a drop-off pattern. Though, a coordinated analysis of seven EMA studies in the broader psychological literature suggests that the phenomenon of measurement reactivity is not sufficiently reliable, robust, or generalizable across designs and constructs (Cerino et al., 2022).

With respect to movement-related behaviors, there has been little investigation of potential reactivity in motivational antecedents of PA and SB. To our knowledge, only one study has explored changes in PA motivation by time in study; findings showed that when assessed weekly, the strength of behavioral intentions does decrease with time, though this decrease follows neither a linear nor flattening pattern (Conroy et al., 2011). Yet, assessing behavioral intentions weekly may be less sensitive to understanding more dynamic changes in motivation because of its assessment than more intensive measurements (e.g., momentary). Further investigation of potential reactivity in PA or SB motivational constructs is needed. Moreover, whether measurement reactivity in motivational constructs and movement-related behaviors produces changes in associations between these constructs is unknown.

The Present Study

The overarching goal of this study was to determine whether measurement reactivity in either movement-related behavior or its motivational antecedents (i.e., self-efficacy, intentions, general motivation) affects estimates of within-day associations between antecedents and behavior. To address this question, we conducted a secondary analysis of two existing EMA data sets that captured movement-related behaviors and their motivational antecedents in the context of daily life, among older adults (Study 1) and women in midlife (Study 2). Both of these groups show high rates of physical inactivity and SB, and there is considerable attention to promoting their movement (Badon et al., 2021; Troiano et al., 2008). These groups may be particularly susceptible to measurement reactivity in a study of movement-related behaviors, as there is a greater deficit between their current level of behavior and desired or expected levels of behavior (Arigo & König, 2024). Our first aim was to determine whether measurement reactivity occurs for movement-related behaviors and motivational antecedents of these behaviors. Our second aim was to determine whether the association between motivational antecedents and movement-related behaviors show evidence of measurement reactivity (and how long it takes for this to dissipate), and the extent to which any reactivity might bias conclusions about the overall associations. Consistent with a “drop-off” pattern seen in prior work (Zhu & Haegele, 2019), if reactivity was present, we expected to see reports of higher motivation, intentions, and self-efficacy earlier in each observation period than later (e.g., higher on the first day vs. others, or at the first prompt vs. others). We expected similar drop-offs in PA (and conversely, increases in SB), as well as in the associations between motivational antecedents and movement-related behavior. Understanding potential changes in movement-related behaviors, motivational antecedents of these behaviors, and associations between them because of measurement reactivity can have important implications for EMA study design and interpretation of findings from such studies.

Methods

The secondary analyses presented in this manuscript were preregistered on Open Science Framework (https://osf.io/s2rby), and code for major analyses is available at the same website. Access to the data is available by request.

Study 1

Recruitment and Participants

Data were compiled from two signal-contingent EMA studies with ambulatory monitoring of older adults (≥60 years). Study 1a took place in Los Angeles County, CA, from 2016 to 2017, and Study 1b took place in Guilford County, NC, in 2018 and 2019. Older adults were excluded if they (a) did not speak or read English, (b) had a mobility limitation that prevented standing/walking on their own, (c) were not able to see/operate a smartphone’s basic functions, or (d) were diagnosed with dementia or Alzheimer’s disease. In addition, in Study 1b, participants had to self-identify as a member of a racial or ethnic minority group and were excluded if a telephone-based screener indicated mild cognitive impairment. Briefly, participants were 195 older adults (Study 1a: n = 104, Study 1b: n = 91). Most participants were women (70% female and 30% male) with an average age of 71.35 years (SD = 6.85) and body mass index (BMI) of 29.1 kg/m² (SD = 6.21; 70.6% overweight or obese). The sample was diverse with respect to race (45.9% Black/African American, 39.2% White, 6.2% two or more races, 5.7% Asian, 3.1% other).

Measures

Participants completed a baseline survey and self-reported demographic information including age, sex, ethnicity, race, height, and weight. ActivPAL3 micro accelerometers (PAL Technologies Ltd.) were used to provide a device-based measure of PA and SB. This activPAL device uses a triaxial accelerometer and has been shown to be a valid and reliable measure of posture and movement in adult populations (Grant et al., 2006). ActivPAL proprietary algorithms and PAL analysis Generation 7 software were used to calculate time spent upright and sitting in 15-s epochs. R syntax created for this study was used to calculate the amount of time spent upright or sitting in the 2 hr after EMA prompts. Observations were excluded if participants indicated that they were not wearing the activPAL device for at least half (1 hr) of the 2-hr window after the EMA prompt based on activity monitor logs kept by participants during the study. PA was operationalized as time spent sitting in the 2 hr after the EMA prompt. Motivational constructs assessed in this study included intentions and self-efficacy to stand or move for at least 30 min over the next 2 hr as well as to limit time spent sitting to less than an hour and a half over the next 2 hr. Single items were used to assess each construct. These items appeared in every EMA prompt, and participants responded on a scale of 1 (strongly disagree) to 5 (strongly agree). Items were adapted from previous EMA research (e.g., Maher & Conroy, 2016). For more details on EMA assessments of motivational constructs, please see Maher & Dunton (2020a, 2020b).

Procedures

Participants received six EMA prompts/day on a loaned smartphone. Participants were also given an activPAL3 micro accelerometer to wear on their anterior thigh during sleeping and waking hours for the duration of the study. ActivPAL devices were waterproofed, but participants were asked to remove the monitor if it would be submerged underwater. Participants in Study 1a and 1b completed these observational study procedures for 10 and 8 days, respectively. Participants were instructed to go about their normal daily lives and not change behaviors because of the study procedures. All procedures were approved by the University of Southern California (HS-16–00572) and the University of North Carolina Greensboro (17–579) Institutional Review Boards. Further details on these studies can be found elsewhere (Maher et al., 2018, 2021).

Study 2

Recruitment and Participants

Data collection took place between February 2019 and March 2020 in southern New Jersey and southeastern Pennsylvania. Study procedures were approved by the Rowan University and Rowan School of Osteopathic Medicine Institutional Review Boards (Approval: Pro2018002377). Women in midlife (age 40–60 years) with one or more risk factors for cardiovascular disease were recruited. Eligible participants were also fluent in English, not currently pregnant, owned a mobile device that could be used to complete electronic prompts (i.e., smartphone or tablet), and did not have health conditions contraindicating participation in PA. Briefly, the sample comprised 75 women (M_age = 51.61, SD = 5.5). Of these, 2% identified as Black, 2% as Latina, 1% as Asian American, and 73% as White, with 3% identifying as mixed race or a racial identification not listed. The largest subsets of participants reported a diagnosis of hyperlipidemia or hypercholesterolemia (52%). Average BMI was 34.02 (SD = 7.13), and most participants had BMIs in the obese category (69%).

Measures

Participants completed a baseline survey and self-reported demographic information, including age, cardiovascular disease risk conditions, education level, menopause status, ethnicity, and race. Research staff measured participant height and weight during an in-person visit. Movement-related behaviors were captured using the ActiGraph GT3X accelerometer (ActiGraph Corporation). This triaxial accelerometer was worn on the participant’s dominant hip during waking hours for 10 consecutive days following their in-person baseline appointment. Observations were considered missing if ActiGraph accelerometer wear for the relevant period was not validated (i.e., activity count was 0 for the 2-hr period following each EMA prompt). Behavioral data were processed in 10-s epochs using proprietary algorithms in ActiLife (version 6) software. Behavior in 2-hr windows subsequent to each EMA prompt was processed using the ActiPro package for RStudio (Dzubur, 2020). PA behavior outcomes included total activity counts and minutes of light-, moderate, and vigorous-intensity activity (moderate and vigorous summed to generate minutes of MVPA). Thresholds for MVPA (Arigo et al., 2020; Matthews et al., 2008) and minutes of MVPA were used to determine the cardiovascular PA intention–behavior association.

PA motivation and intentions were captured via four items at each EMA prompt. First, participants were asked to report how motivated they were to be physically active in the next few hours, at any activity intensity (1 = not at all,4 = very much; prompts 1–4 of the day). They were then asked whether they had intentions of doing cardiovascular exercise in the relevant timeframe. If yes, participants recorded the number of minutes they planned to engage in and the type of PA they planned to do (e.g., walking or taking an exercise class). EMA items were pilot-tested and refined during formative stages of research (Arigo et al., 2021). The last prompt of the day (Prompt 5) came close to participants’ bedtimes and assessed PA motivation/intentions for the following day; this prompt was not included in analyses to maintain consistency in correspondence between motivational antecedents and subsequent movement-related behaviors.

Procedure

To assess eligibility, interested participants completed a phone screening and initial electronic survey. During a baseline appointment at a research center, participants then provided written documentation of informed consent and reviewed study procedures. Participants were instructed on how to wear and care for the accelerometer and to complete the EMA prompts within 1 hr of receipt. Participants’ height and weight were measured at this visit. Participants who enrolled completed a 10-day signal-contingent EMA protocol. During the study, which started the day following their baseline appointment, participants were signaled via their personal mobile device to complete EMA prompts at five semi-random times per day. Participants also wore the waistband accelerometer during waking hours. Participants were instructed to go about their normal daily lives and not change behaviors because of the study procedures. After 10 days of observation, participants came back to the research center for a brief follow-up visit that included returning the accelerometer and providing feedback on their experience. Additional details about the procedures for this study can be found elsewhere (Arigo et al., 2020).

Statistical Analyses: Study 1 and Study 2

Temporal variables were created using date- and time-stamps of each EMA prompt. Specifically, date- and time-stamps were used to created dichotomized variables representing whether an EMA prompt was on (a) first prompt of the study, (b) first 2 prompts of the study, (c) first day of the study, (d) first 2 days of the study, or first 3 days of the study. Prompts occurring within the timeframe referenced were coded as “1” (reference group in all models described below) with all other occasions coded as “0.” Temporal variables were chosen for consistency with previous research investigating measurement reactivity (König et al., 2022).

Data Analyses

In both studies, empty multilevel models were used to generate estimates of average levels for motivational antecedents and movement-related behavioral outcomes and to calculate intraclass correlation coefficients (ICCs). ICCs provide estimates of the degree to which variance in intensively assessed constructs is attributable to between-person (vs. within-person) variability. All models controlled for age and BMI (grand mean centered) and time of day and day of week to account for factors associated with missingness as well as movement behaviors across both studies (Arigo et al., 2022; Maher et al., 2018, 2021). Study 1 models also controlled for study and participants’ gender to account for data being pooled across studies and the inclusion of both sexes in Study 1.

We first examined prompt- and day-level patterns in individual predictors and outcomes, computing separate models per movement-related behavior and motivational antecedent. To align with our hypotheses regarding drop-off in behaviors and motivation following an initial assessment period, we tested for evidence of meaningful change in the constructs of interest, after (a) the first prompt or first 2 prompts of the study and (b) the first day, first 2 days, or first 3 days of the study, as indicating measurement reactivity. We then examined the temporal associations between predictors and outcomes, and whether these associations change after the initial prompt(s) and/or day(s), by testing the interaction between motivational antecedent and prompt/day variables for predicting movement-related outcomes. Statistical significance was set at p < .01 and effect sizes are interpreted in minutes of change.

Multilevel models were tested using the PROC MIXED command in SAS (version 9.4, SAS Institute). Multilevel models had occasions (Level 1) nested in participants (Level 2). All models used maximum likelihood estimation techniques to account for missingness. A total of 9,319 observations across 193 participants from Study 1 and 1,490 observations across 75 participants from Study 2 were used in analyses. Detailed compliance information can be found elsewhere (Arigo et al., 2020; Maher et al., 2018, 2021).

Results

Tables 1 and 2 summarize findings regarding temporal changes in movement-related behaviors and motivational antecedents as well as changes in associations between motivational antecedents and movement-related behavior, respectively. Complete multilevel model results are in the Supplementary Materials (available online). Figure 1 displays displays prompt- and day-level estimates for changes in associations between motivational antecedents to quantify changes in associations across the first 10 prompts (top panel) or days (bottom panel).

Table 1.

Summary of Temporal Changes in Movement-Related Behaviors and Motivational Antecedents in Studies 1 (n = 195) and 2 (n = 75)

	First prompt vs. others	First 2 prompts vs. others	First day vs. others	First 2 days vs. others	First 3 days vs. others
Study 1
Time spent sitting	NS	NS	NS	+	+
Time spent upright	NS	NS	NS	−	−
Intentions—limit sitting	NS	NS	NS	NS	NS
Self-efficacy—limit sitting	NS	NS	NS	NS	NS
Intentions—standing/moving	NS	NS	NS	−	−
Self-efficacy—standing/moving	NS	NS	NS	−	−
Study 2
Light physical activity	−	−	NS	NS	NS
Moderate/vigorous physical activity	NS	NS	NS	NS	NS
Physical activity motivation	−	−	−	−	−
Physical activity intentions (Y/N)	NS	NS	NS	NS	NS
Physical activity intentions (minutes)	NS	NS	NS	NS	NS

Note. Symbols in cells represent the direction of the association between the temporal variable and movement-related behavior or motivational antecedent. For all temporal variables, the reference group represents the other category. + = construct is significantly higher in the other category compared to the timeframe referenced; − = construct is significantly lower in the other category compared to the timeframe referenced; NS = no change or difference in construct based on timeframe referenced. Significant associations, p < .01.

Table 2.

Changes in Associations Between Motivational Antecedent and Movement-Related Behavior by Temporal Variables in Studies 1 (n = 195) and 2 (n = 75)

	First prompt vs. all others	First 2 prompts vs. all others	First day vs. all others	First 2 days vs. all others	First 3 days vs. all others
Study 1  Predicting time spent sitting   Intentions × Time	–	–	–	–	–
Self-efficacy × Time	–	–	–	–	–
Predicting time spent upright   Intentions × Time	–	NS	NS	NS	NS
Self-efficacy × Time	–	NS	NS	NS	NS
Study 2  Predicting minutes of light activity   Motivation × Time	NS	NS	NS	NS	NS
Predicting minutes of moderate to vigorous physical activity   Motivation × Time	NS	NS	NS	NS	NS
Intentions (Y/N) × Time	NS	NS	NS	NS	NS
Intentions (minutes) × Time	NS	NS	NS	+	NS

Note. Symbols in cells represent the moderating effect of the temporal variable on the association between motivational antecedents and movement-related behavior. For all temporal variables, the reference group represents the other category. + = significantly stronger association in the timeframe referenced compared to the other category; – = significantly weaker association in the timeframe referenced compared to the other category; NS = no difference in construct based on timeframe referenced. Significant associations, p < .01.

Figure 1 —

Individuals estimates of associations between motivational antecedents and movement-related behaviors by prompt (A) or day (B). Darker shades of blue indicate stronger negative associations. Darker shades of yellow indicate stronger positive associations. *For Study 2, prompts 5 and 10 were not included in the model since this was always the last prompt of the day and assessed motivation and intentions to engage in physical activity the next day.

Study 1

Movement-Related Behaviors and Motivational Antecedents

On average, older adults spent most of their time in the 2 hr following an EMA prompt sitting (B = 78.57 min, SE = 1.13). Approximately, 41.28 min in the 2 hr after the EMA prompt was spent upright (SE = 1.13), with 30.87 min spent standing (SE = 0.94) and 10.41 min present stepping (SE = 0.36). Based on ICCs, between-person stability in these movement-related behaviors was generally low (ICCs = .21–.28). There were no differences in movement-related behaviors in the 2 hr following the first prompt and the first 2 prompts compared with all other prompts (ps > .01). Consistent with measurement reactivity, time spent sitting in the 2 hr following the EMA prompt was lower, and time spent upright was higher on the first 2 days, sitting: B = –2.68 min, SE = 0.66, F(1, 192) = 16.66; upright: B = 2.60 min, SE = 0.65, F(1, 192) = 15.72, ps < .001, and first 3 days, sitting: B = –2.68 min, SE = 0.55, F(1, 191) = 23.21; upright: 2.74 min, SE = 0.55, F(1, 191) = 24.37; ps < .001, compared with all remaining days.

On average, older adults reported moderate to high intentions (B = 3.37, SE = 0.04, on a 1–5 scale) and self-efficacy (B = 3.55, SE = 0.04) to limit SB over the next 2 hr as well as intentions (B = 3.60, SE = 0.04) and self-efficacy (B = 3.80, SE = 0.04) to stand or move over the next 2 hr. These constructs had low to moderate between-person stability with ICCs ranging from .38 (intentions to limit SB) to .46 (self-efficacy to stand or move). Intentions and self-efficacy to limit SB in the 2 hr after the EMA prompt did not significantly differ by prompt or day in study (ps > .08). There were no differences in intentions or self-efficacy to stand or move across prompts (ps > .44), or to stand or move on the first day compared with all other days (ps > .65). However, consistent with reactivity, intentions and self-efficacy were significantly higher on the first 2 days, intentions: B = −0.08, SE = 0.02, F(1, 192) = 19.08, p < .001; self-efficacy: B = −0.06, SE = 0.02, F(1, 190) = 10.49, p = .001, and the first 3 days, intentions: B = −0.09, SE = 0.02, F(1, 192) = 29.80; self-efficacy: B = −0.06, SE = 0.02, F(1, 190) = 14.62, ps < .001, compared with all other days.

Associations Between Motivational Antecedents and Movement-Related Behaviors

The association between intentions to limit SB and minutes of SB in the subsequent 2 hr across all prompts and days was −7.17 (SE = 0.33); between self-efficacy to limit SB and minutes of SB in the subsequent 2 hr across all prompts and days was −6.60 (SE = 0.35). Contrary to expectations, the associations between intentions and self-efficacy to limit SB and subsequent SB were weaker at the first prompt, intentions: B = −7.40, SE = 1.60, F(19, 105) = 21.24; self-efficacy: B = −8.69, SE = 1.70, F(19, 023) =26.06, ps < .001) and first 2 prompts, intentions: B = −4.55, SE = 1.14, F(19,105) = 15.83; self-efficacy: B = −5.54, SE = 1.20, F(19, 023) = 21.13, ps < .001, compared with all other prompts in the study. Similarly, associations between intentions and self-efficacy to limit SB and subsequent SB were weaker on the first day, intentions: B = −4.09, SE = 0.85, F(19, 105) = 23.36; self-efficacy: B = −4.25, SE = 0.89, F(19, 023) = 22.73, ps < .001, first 2 days, intentions: B = −3.21, SE = 0.62, F(19, 105) = 26.46; self-efficacy: B = −2.90, SE = 0.66, F(19, 023) = 19.46, ps < .001, and first 3 days, intentions: B = −2.48, SE = 0.55, F(19, 105) = 20.64; self-efficacy: B = −1.84, SE = 0.57, F(19, 023) = 10.34, ps < .001, compared with all other days.

The association between intentions to stand or move and time spent upright in the subsequent 2 hr across all prompts and days was 8.04 (SE = 0.35); between self-efficacy to stand or move and time spent upright in the subsequent 2 hr across all prompts and days was 7.14 (SE = 0.37). At the first prompt of the study, but not the first 2 prompts of the study, the relation between intentions and self-efficacy to stand or move and subsequent time spent upright was weaker compared with all other prompts, intentions: B = 4.41, SE = 1.61, F(19, 102) = 7.56, p = .006; self-efficacy: B = 4.75, SE = 1.82, F(19, 021) = 6.78, p = .009, ps < .01. Neither the associations between intentions or self-efficacy to stand or move and time spent upright differed by day in study (ps > .01). Overall, in Study 1, the strength of the associations between motivational antecedents and movement-related behaviors was not meaningfully influenced by measurement reactivity.

Study 2

Movement-Related Behaviors and Motivational Antecedents

As expected, women in midlife with elevated cardiovascular disease risk showed minimal movement-related behavior across the 2-hr windows after the first 4 EMA prompts of the day. On average, they spent 12.63 min in light activity (SE = 0.48) and 8.84 min in MVPA (SE = 0.41) per 2-hr window, and spent 78.09 min in SB (SE = 0.89), though stability in these behaviors within-person was low (i.e., ICCs .01–.14). SB did not differ between days in the study (first day vs. remaining, first 2 days vs. remaining, or first 3 days vs. remaining; Bs < 1.00, ps > .67), or between the first prompt and remaining prompts (p = .85) but did differ between the first 2 prompts and remaining prompts, F(1, 69) = 9.47, p = .0003: consistent with measurement reactivity, participants’ SB was 8.87 min higher after the first 2 prompts than after remaining prompts (SE = 2.88). Minutes of light activity did not differ between days in the study (ps > .08) but were higher after the first prompt (B = 4.02, SE = 1.15) and after the first 2 prompts (B = 3.41, SE = 0.83) than after remaining prompts (ps < .001; also consistent with reactivity). In contrast, minutes of MVPA did not differ between days (ps > .29) or between prompts (ps > .02).

Average motivation to be physically active in the next few hours was fairly low (B = 2.20, SE = 0.07 on a scale of 1–4) and minimally stable within person (ICC = .39). Consistent with measurement reactivity, motivation was higher at the first prompt, B = 0.38, SE = 0.10, F(1, 60) = 15.92, and the first 2 prompts, B = 0.46, SE = 0.07, F(1, 70) = 44.76, ps < .001, than at all other prompts, with the highest rating on Day 1. Day 1 in study showed a similar pattern: significant differences between the first day and all other days, the first 2 days and all other days, and the first 3 days and all other days (ps < .0001), with higher motivation on earlier days of observation. Direct contrasts showed that these differences were equivalent to ≤0.35 units of motivation (on a 4-point scale). The likelihood of reporting intentions to be physically active (vs. no intentions) was 0.12 (SE = 0.02) and variable within person (ICC = .31). The likelihood of reporting intentions did not differ between the first prompt and remaining prompts, or between the first 2 prompts and remaining prompts (ps > .82). The likelihood of reporting intentions also did not differ between the first day and all other days, the first 2 days and remaining days, or the first 3 days and remaining days (ps > .14). Finally, the intended minutes of cardiovascular exercise at times when intentions were reported (B = 35.46, SE = 2.84) was somewhat more stable than other experiences of interest (ICC = .49) and showed no differences between days in the study or between initial and later prompts (ps > .61).

Temporal Associations Between Motivational Antecedents and Movement-Related Behaviors

Our primary interest was in potential reactivity patterns for the temporal associations between movement-related predictors and outcomes. The association between PA motivation and minutes of light activity across all prompts and days was 1.21 (SE = 0.30); between PA motivation minutes of MVPA in the 2 hr after each prompt, the association was 0.80 (SE = 0.29) across all prompts and days. The strength of these associations showed no differences between initial and later days in the study or between initial and later prompts (ps > .08). The same was true for setting intentions for cardiovascular activity (yes/no) and minutes of MVPA in the 2 hr after each prompt: no differences between days in the study or between initial and later prompts (ps > .75), and little distinction in observed MVPA at times when intentions were versus were not set (across all days and prompts; B = 0.19, SE = 0.60).

With respect to the association between intended minutes of cardiovascular activity and observed MVPA in the 2 hr following each prompt, the association across all prompts and days was not significant (B = 0.01, SE = 0.02, p = .60). As was the case for motivation and PA behavior, there were no differences in this association between days or between the first prompt and remaining prompts (ps > .12). However, the association between intended length of time in cardiovascular activity and subsequent MVPA did differ between the first 2 days and remaining days, B = −0.14, SE = 0.05, F(1, 243) = 7.63, p = .006). It was more positive on the first 2 days, primarily due to a stronger-than-usual positive association on Day 2 (B = 0.33, SE = 0.08) but hovered within 0.10 of 0 on all other days and showed no consistent pattern. Thus, in Study 2, the overall strength of the associations between psychological antecedents and movement-related behaviors was not meaningfully skewed by measurement reactivity.

Discussion

In studies that use repeated assessments of the same participants, measurement reactivity can bias estimates of movement-related behaviors (French & Sutton, 2010; König et al., 2022; Miles et al., 2020). Potential reactivity in the motivational antecedents of these behaviors has received less attention, particularly in the context of intensive assessments (e.g., as with EMA), and the extent to which there is reactivity in associations between motivational antecedents and movement-related behaviors is unknown. To address these lesser-understood aspects of measurement reactivity in the context of movement-related behaviors, the present study capitalized on existing datasets to conduct a two-study series of preregistered secondary analyses.

We observed evidence of measurement reactivity in both studies, with respect to individual motivational antecedents and movement-related behaviors, though patterns were distinct. Consistent with a reactivity response, older adults (women and men) showed higher self-efficacy for and intentions to stand/move, as well as greater time spent upright, earlier in the study (vs. later). Their time spent sitting was also lower earlier in the study (vs. later); these differences appeared between early and later days, but not between early and later EMA prompts. Women in midlife also showed greater PA motivation on earlier days and at earlier EMA prompts, versus later days and prompts, and engaged in more minutes of light PA at earlier (vs. later) EMA prompts, though there were no differences between days of observation.

As in previous work (Conroy et al., 2011), these differences between timeframes aggregated across considerable variability in antecedents and behaviors, particularly later in the study, rather than the expected pattern of drop-off and then consistency. For motivational antecedents, it is possible that people only begin to show consistency after longer periods (Eisele et al., 2023), and this may translate to associations between antecedents and behavior. For instance, among older adults in this study, associations between motivational antecedents and behavior appeared to become stronger later in the study. This increasing strength of association is unlikely to reflect reactivity, for which we expected a gradual decoupling of antecedents and behavior, returning to prestudy levels. Instead, participants may develop more realistic expectations over time and become better at anticipating opportunities for movement-related behaviors in the next hours due to increased awareness induced by self-monitoring (Wilde & Garvin, 2007).

The present findings reveal more consistent temporal differences in associations between motivation and behavior for SB compared with other movement-related behaviors. Weaker associations at the start of an EMA study may be because SB is thought to be strongly regulated by habits (i.e., well-learned cue behavior associations; e.g., Maher & Conroy, 2016). Therefore, individuals may not be able to accurately gauge their SB or motivation to limit SB at the start of a study. However, as time goes on, prompting individuals to reflect on their motivation and behavior may increase awareness, thereby improving one’s ability to accurately forecast their behavioral engagement and motivation for limiting SB. Again, this pattern and its likely contributors do not reflect measurement reactivity but underscore the short-term power of self-monitoring for follow-through on efforts to reduce SB (Compernolle et al., 2019).

The present analyses also revealed an interesting pattern in the association between intended and observed minutes of MVPA among women in midlife. Our planned contrasts confirmed that the association was significantly higher on the first 2 days than on the remaining days. However, this association was very close to and highly variable around 0 on all days, except for Day 2. On Day 2, there was a modest but noteworthy positive association (B = 0.33), suggesting that women appeared to follow through on their specific MVPA intentions more so on this day than on other days. Previous EMA work with this population shows that women achieve their intended minutes of MVPA for the next 2 hr only 13%–18% of the time (Arigo et al., 2022), and it is not clear why they would be more successful on Day 2 than other days.

Although the pattern observed here does not reflect an initial elevation bias, which is one manifestation of measurement reactivity (Shrout et al., 2018), this pattern may still indicate a measurement reactivity response. Given the pattern of intended MVPA minutes, which decreased from Day 1 to Day 2, it is possible that women became aware of their lack of follow-through on MVPA intentions on Day 1 of participation and subsequently adjusted their intentions downward to reflect their behavior. This change was extremely short-lived (i.e., 1 day) in a period with a consistent intention-behavior gap; however, it may merely reflect that these women have extremely busy and unpredictable lives (Hendry et al., 2010) and have little impact in the context of attempts to modify their movement-related behavior. Importantly, however, comparisons between the overall association and the association between the first 2 days and subsequent days suggests that an aggregated estimate of the association between MVPA intentions and behavior across an entire EMA period is not meaningfully affected by an elevation early in the study.

Interestingly, although EMA can reveal time-varying changes in associations between motivational antecedents and movement-related behaviors across an observation period (Perski et al., 2022), few published reports include this. Instead, authors typically report fixed effects for associations aggregated across the observation period, without reporting whether associations change within or across days. One exception is Maher et al. (2016), who documented that associations between motivational antecedents and MVPA change within and across days using time-varying effect modeling; however, changes in associations earlier (vs. later) in the observation period were not reported. Future studies should explicitly test for and report time-varying relationships across observation periods to advance our understanding of potential reactivity effects as well as provide insights for theory refinement and intervention development.

As noted, existing studies have used different approaches to test measurement reactivity. We chose to compare behavior between prompts and days to test for drop-offs (or sudden increases; Cerino et al., 2022) as this approach may provide insights into the persistence of the effect. To date, however, there is no uniform definition or best practice, which is needed to arrive at more definite conclusions regarding the existence and magnitude of the effect across studies. With respect to the magnitude of the effect, it is also important to note that although many of the hypothesized differences between early and later in each study surpassed a predetermined level of significance, these differences likely have limited practical significance due to small cumulative changes in motivation, behavior, or associations between them.

Implications for Future EMA Research in the Context of Measurement Reactivity

Based on the results of the present study, measurement reactivity may not always present meaningful confounds in EMA studies of movement-related behavior in midlife and older adults. Indeed, the magnitude of the effect might depend on the population, timescale, motivational antecedent, or movement behavior. Therefore, future research should still attempt to mitigate and/or investigate potential measurement reactivity within their data. Current best practices for mitigating reactivity in advance include randomizing the order of EMA survey items (Arslan et al., 2021) and blinding participants to real-time feedback from a movement monitor (Clemes & Deans, 2012). Of note, devices in the present studies did not provide real-time feedback (and still showed some reactivity), though the order of EMA survey item presentation was consistent. Researchers can also consider eliminating data from the earliest prompts in their analyses, to avoid biasing estimates of relations between antecedents and behavior. This is a drastic step for many researchers, as it can waste scarce resources and introduce unnecessary burdens on participants—particularly if differences between early and later observations are small and will not meaningfully change conclusions. This approach also ignores a subset of participant experiences, including their observed (and likely actual) behavioral engagement. We recommend researchers test for potential differences in associations and use context-specific judgment about steps such as planned sensitivity analyses and statistical control for prompt and/or day (French et al., 2021), or if effect sizes are known to the reactivity effect itself (Bendtsen & McCambridge, 2021), versus dropping observations.

Strengths and Weaknesses of the Present Study

Strengths of the present study included the investigation of measurement reactivity across a diverse group of participants with respect to age and movement-related behaviors. Further, the use of intensive, within-day assessments paired with device-based measures of behavior and the examination of changes in associations between these constructs is unique among studies of measurement reactivity in this domain. However, the limitations should be noted. It is unclear the extent to which other populations might experience measurement reactivity in associations between motivational antecedents and movement-related behaviors. Both smartphone and wearable device use tends to be higher in adolescents and emerging adults compared to midlife and older adult populations (Pew Research Center, 2021), and exposure to technology may mitigate some elements that drive measurement reactivity (Arigo & König, 2024). There are conflicting findings with respect to differences in behavioral measurement reactivity among adults versus younger populations (Davis & Loprinzi, 2016; Zhu & Haegele, 2019) and warrant further investigation.

In addition, Study 2 had limited racial, ethnic, and socioeconomic diversity. Individuals from marginalized communities may be more suspicious or distrustful of health research settings (Jaiswal, 2019) and may show distinct measurement reactivity patterns as a result, if they do participate. Motivational constructs in this study focused on general motivation as well as behavior-specific intentions and self-efficacy. It is unclear whether measurement reactivity is present for other movement-related constructs. Finally, because we utilized data from existing EMA studies with a similar number of prompts per day, we are unable to compare potential differences in measurement reactivity based on the sampling frequency of EMA assessments per day. Previous research has investigated potential differences in measurement reactivity based on the frequency of assessments per day, concluding that higher sampling frequencies do not produce stronger reactivity on various affective and cognitive states (Eisele et al., 2023). Future research regarding movement-related behaviors may benefit from examining measurement reactivity across different EMA study designs (including sampling frequencies) to better understand this phenomenon.

Conclusion

This study is the first to simultaneously investigate measurement reactivity in movement-related behaviors, motivational antecedents, and associations between them in midlife and older adults. There was some evidence of measurement reactivity regarding changes in movement-related behaviors and motivational antecedents, but these differed by timescale, study, behavior, and antecedent. Some associations between motivational antecedents and behavior did change at the beginning of the study relative to time points later in the study. However, these associations were not consistent with the phenomenon of measurement reactivity and did not meaningfully bias overall estimates. Other participatory effects, as opposed to measurement reactivity, likely drive these changes in associations. Future research should continue to test reactivity patterns, though measurement reactivity may not present meaningful confounds in EMA studies of movement-related behavior and their antecedents in midlife and older adults.

Data availability statement:

Analysis was pre-registered on Open Science Framework: https://osf.io/s2rby. Computer code needed to reproduce major analyses is also available at this site. Data will be made available upon request.