Effort in daily life: relationships between experimental tasks and daily experience.

Abstract

Recently, experimental tasks have been developed which index individual differences in willingness to expend effort for reward. However, little is known regarding whether such measures are associated with daily experience of effort. To test this, 31 participants completed an ecological momentary assessment (EMA) protocol, answering surveys regarding the mental and physical demand of their daily activities, and also completed two effort-based decision-making tasks: the Effort Expenditure for Rewards Task (EEfRT) and the Cognitive Effort Discounting (COGED) Task. Individuals who reported engaging in more mentally and physically demanding activities via EMA were also more willing to expend effort in the COGED task. However, EMA variables were not significantly associated with EEfRT decision-making. The results demonstrate the ecological, discriminant, and incremental validity of the COGED task, and provide preliminary evidence that individual differences in daily experience of effort may arise, in part, from differences in trait-level tendencies to weigh the costs versus benefits of actions.

Introduction

Daily decision-making involves choices about behaviors that require physical or cognitive effort. Prior theories (e.g., motivation intensity theory) provide descriptions of factors that influence effort mobilization and expenditure (Brehm & Self, 1989). Further, questionnaires (e.g., the Need for Cognition Scale: Cacioppo & Petty, 1982) and lab-based experimental paradigms have been developed to assess individual differences in desire to engage in cognitively demanding activities and willingness to expend effort for reward (e.g., Treadway et al., 2009; Westbrook, Kester, & Braver, 2013). Such tasks, have been implemented to better understand motivational dysfunction in psychiatric illnesses, and may be relevant to understanding motivation processes in more applied areas of psychology (e.g., sports science, education). However, it is not known whether such tasks and questionnaires have ecological validity. In particular, it remains unclear whether performance on these experimental tasks relates to the experience of effort during daily life.

In one exception, Moran and colleagues asked individuals with schizophrenia to self-report levels of enjoyment and interest with daily activities (Moran, Culbreth, & Barch, 2017). They found that individuals with schizophrenia who reported greater interest and enjoyment in daily activities were more willing to expend effort on a physical effort-based decision-making task (EEfRT: Treadway et al., 2009) (Moran et al., 2017). However, no study to date has examined this question in healthy adults, testing whether experimental tasks predict experience of effort in daily life. Likewise, no study has included tasks that focus on decision-making about cognitive effort, even though the development of such tasks has been a focus of recent research attention (e.g., COGED: Westbrook et al., 2013; Culbreth et al., 2016).

The current study examined the ecological validity of effort-based decision-making tasks in a community sample. We collected participants’ self-report of effort during daily activities. We then related daily reports to performance on both physical and cognitive effort-based decision-making tasks. We hypothesized that individuals who demonstrated greater willingness to exert effort on experimental tasks would self-report greater experience of effort during daily life.

Methods

Participants

We recruited 31 individuals from the St. Louis community (Table 1). Exclusion criteria included (a) diagnosis of a current mood (e.g., major depressive disorder, bipolar disorder), substance use, or psychotic disorder as defined by the Diagnostic and Statistical Manual of Mental Disorders – Fourth Edition using the Structured Clinical Interview for DSM-IV (b) prescription of psychiatric medications. The Washington University Institutional Review Board approved the study. Participants provided written, informed consent in accordance with Washington University’s Human Subject Committee’s criteria. Power analyses for HLM analyses yielded approximately 73% power to detect an effect size of r = 0.46 or greater with an error rate of p<0.05.

Table 1:

Demographic Information

	(N = 31)
	MEAN	SD
Age (years)	35.2	10.3
Sex (number female)	11
Ethnicity, (n)
African American	13
Asian	6
Caucasian	12
Education (years)	15.9	1.9
WTAR (raw score)	35.2	10.3
WTAR (standard score)	103.4	16.1

Abbreviations: WTAR: Wechsler Test of Adult Reading

Study Design

The current study was a part of a larger design (see S1 in Online Supplemental Materials (OSM) for full methods). The EEfRT task was administered during the first testing session. The COGED task and NCS were administered in the second testing session, approximately one week later. The EMA protocol was completed in between testing sessions.

Questionnaire

Participants completed the Need for Cognition Scale (NCS). The scale was used as it assesses the tendency for individuals to engage in and enjoy demanding cognitive tasks (Cacioppo & Petty, 1982).

EMA

Participants were either provided an Android-enabled smartphone or used their personal smartphone. During the seven-day protocol, participants received five text messages per day between 10:00 a.m. and 8:00 p.m., approximately every 2–3 hours. Text messages contained hyperlinks to a Qualtrics online survey (Snow & Mann, 2013). Participants were given 15-minutes, following text message receipt, to begin each survey. Participants were paid $1.75 for each survey completed within this 15-minute window.

On each survey, participants indicated their current activities from a predetermined list (Figure 1). Next, they indicated the level of (a) physical demand; (b) mental demand; (c) enjoyment they experienced from these activities on a 5-point scale ranging from 1 (not at all) to 5 (extremely). Participants also indicated their activities, physical demand, mental demand, and enjoyment (a) since the last prompt (last 2–3 hours), as well as (b) what they expected to do in the upcoming 2–3 hours. Current, past, and future self-reports were summed for each survey for physical demand, mental demand, and enjoyment separately. A hierarchical linear model (HLM) predicting self-reported mental demand from physical demand revealed that the two questions were highly related (S2 in OSM). We averaged these self-report questions, for each survey, to reduce multiple comparisons, creating a single EMA effort measure (EMA Effort). However, we provide models predicting mental and physical demand, individually, as well as models examining current, past, and future EMA questions separately (S3–4 in OSM).

Figure 1:

Frequency of Self-Reported Activities during the EMA Protocol

Effort-Based Decision-Making Tasks

1. EEfRT

Participants performed a modified version of EEfRT (Barch, Treadway, & Schoen, 2014). In the task, participants make repeated choices between completing an easy or hard task. The easy task involves making 20 dominant index finger button-presses within 7 seconds for the opportunity to win $1. The hard task involves making 100 non-dominant pinky finger button-presses within 21 seconds for the opportunity to win between $1.24-$4.30. At trial onset, easy/hard task reward offers and probability of reward receipt (50% or 88%) are presented. Participants completed a total of 57 trials. For each participant, the percentage of hard task choice across all trials was calculated (EEfRT Average), and used in all analyses as a measure of global willingness to expend physical effort. S5 in OSM lists average trial completion.

2. COGED

Participants completed a modified version of the COGED task (Westbrook et al., 2013). In this task, participants first practiced increasingly difficult versions of a cognitively demanding task (N-Back: 1—4 Back). Specifically, participants completed two 64-trial (16 targets and 48 non-targets) runs of each N-back level. Next, individuals made a series of choices between completing a more demanding level of the N-back (2—4 back) for a greater monetary reward or a less demanding level (1-back) for a smaller reward. After each choice the reward amount for the 1-back was titrated until participants were indifferent between the base offer for the harder task and the offer for the 1-back. This indifference point was then divided by the base offer amount for the hard task in order to quantify a subjective value for each hard task-base amount pair. In the current study, three high-demand N-back levels (N = 2–4) and 2 base reward amounts ($2 and $4) were used. For each participant, we averaged subjective values across the 6 task-amount pairs (COGED Average), and used this average in all analyses as a measure of global willingness to expend cognitive effort. S5 in OSM lists N-back performance.

Data Analysis

1. Task Behavior

We first tested for choice effects reported in prior studies (Barch et al., 2014; Westbrook et al., 2013). For EEfRT, after grouping trials into hard task offer value quartiles (<$1.86; $1.96 to <$2.77; $2.77 to <$3.58; >= $3.58), we conducted a repeated-measures ANOVA with both value quartiles and reward probability as within-subjects factors to test whether the percentage of hard task choices was influenced by reward probability and hard task reward value.

For COGED, we used an HLM to account for the hierarchical nesting of indifference points within participants. Specifically, we tested whether task level, reward value, and their interaction predicted subjective value for each task-amount pair, using fully-random models. Models were fit in R using the lme4-package, version 1.1–7 (Bates & Sarkar, 2007).

2. EMA Associations

We examined relationships between EMA variables (Level 1) and effort-based decision-making task variables (Level 2) using HLM. We fit HLMs to test whether task metrics, day of survey (1–7), survey number of the day (1–5), and EMA survey completion rate predicted EMA Effort. We implemented a maximal random effects structure (Barr et al., 2013). Specifically, day of survey and survey number of the day, as well as the intercept were allowed to randomly vary across participants. In contrast, variables with only one observation per participant (EMA completion rate, EEfRT Average, NCS Total, and COGED Average) were entered at the participant level. We first analyzed separate models for COGED and EEfRT. Next, we examined whether associations between tasks and EMA Effort were driven by relationships between effort and enjoyment by conducting a HLM that included both EMA enjoyment and task variables as simultaneous predictors of EMA Effort. Finally, we conducted a follow-up analysis to test whether COGED, EEfRT, and NCS scores were independently associated with EMA Effort. We included survey day, survey time of day, EMA completion rate, and EMA enjoyment as nuisance variables.

Open Science Framework

The data and the main analyses of this manuscript are available for public use on the Open Science Framework (Project Title: “Effort-Based Decision-Making in Daily Life”).

Results

Task Performance

Choice behavior on effort-based decision-making tasks exhibited effects largely consistent with previous reports (Figure 2) (Treadway et al., 2009; Westbrook et al., 2013). For EEfRT, individuals selected the hard task with greater frequency as reward value (F(3,90)=41.7, p<0.001, partial eta squared=0.58) and reward probability (F(1,30)=23.2, p<0.001, partial eta squared=0.44) increased, but we unexpectedly did not find a significant interaction reward by probability interaction (F(3,90)=1.6, p<0.2, partial eta squared=0.05). For COGED, subjective values decreased with higher N-back levels (beta=−0.18, SE=0.03, p<0.001). However, subjective value did not vary by hard task offer amount (beta=−0.001, SE=0.013, p=0.95). Bivariate correlations between COGED Average and EEfRT Average revealed a modest, but not significant, positive association (r = 0.30, p=0.10).

Figure 2: Effort-Based Decision-Making Task Performance:

Top (EEfRT): individuals selected the hard task with greater frequency as reward value and probability of reward receipt increased, but we did not find a significant interaction reward by probability interaction. Bottom (COGED): subjective values decreased with higher N-back levels (i.e., they were discounted more steeply with increasing N-back load) However, subjective value did not vary by hard task offer amount (i.e., participants discounted $2 and $4 offer values similarly).

EMA

On average, participants completed 75% of EMA surveys within 15 minutes, similar to previous reports (e.g., Gard, Sanchez, Starr, et al., 2014). EMA completion rate negatively correlated with Average COGED (r = −0.37, p =0.04), but not Average EEfRT.

Task and EMA Associations

COGED Average significantly predicted EMA Effort. Specifically, individuals who were more willing to exert effort reported greater daily experience of effort (Table 2A). This relationship remained significant when controlling for n-back performance (S6 in OSM). EEfRT Average did not significantly predict EMA Effort (Table 2B). Interestingly, neither COGED Average nor EEfRT Average significantly predicted enjoyment with daily activities (S7 in OSM). Finally, NCS did not significantly moderate prediction of EMA effort by EEfRT or COGED Average (S8 in OSM).

Table 2:

Models Predicting EMA Effort

Parameter	Estimate	Standard Error	t-value	p-value
Intercept	4.36	1.33	3.28	<0.001
Time of Day	−0.21	0.06	−3.93	<0.001
Day of Survey	0.01	0.05	0.21	0.83
COGED Average	2.28	1.00	2.28	0.02
EMA Completion Rate	1.18	1.34	0.88	0.38
Parameter	Estimate	Standard Error	t-value	p-value
Intercept	6.14	1.15	5.32	<0.001
Time of Day	−0.21	0.05	−3.90	<0.001
Day of Survey	0.01	0.04	0.23	0.82
EEfRT Average	0.51	1.01	0.51	0.61
EMA Completion Rate	−0.06	1.42	−0.04	0.96
Parameter	Estimate	Standard Error	t-value	p-value
Intercept	4.23	1.30	3.30	<0.001
EMA Enjoyment	−0.01	0.05	−0.28	0.78
Time of Day	−0.21	0.05	−3.94	<0.001
Day of Survey	−0.02	0.04	−0.35	0.72
COGED Average	2.79	0.90	3.10	0.002
EMA Completion Rate	1.29	1.19	1.09	0.27
Parameter	Estimate	Standard Error	t-value	p-value
Intercept	4.65	1.68	2.77	0.01
Time of Day	−0.21	0.06	−3.82	<0.001
Day of Survey	−0.02	0.04	−0.51	0.61
EMA Enjoyment	−0.01	0.06	−0.26	0.80
EEfRT Average	−0.82	0.83	−1.00	0.32
COGED Average	2.72	1.02	2.66	0.01
Need for Cognition	0.01	0.02	0.44	0.66
EMA Completion Rate	0.73	1.24	0.59	0.55

We conducted analyses to further examine the significant relationship between COGED Average and EMA Effort. First, we tested whether COGED Average predicted EMA Effort independently of self-reported enjoyment with daily activities (Table 2C). Here, we observed that COGED Average significantly predicted EMA Effort when holding EMA Enjoyment constant. Second, we tested whether COGED Average predicted EMA Effort even when including other laboratory-based measures in the model simultaneously. Results showed non-significant effects of NCS and EEfRT, but COGED Average remained significantly predictive (Table 2D).

Supplemental analyses were conducted with models predicting mental and physical demand individually from task behavior, as well as models examining current, past, and future EMA questions separately (S2–S3 in OSM). These analyses showed that that COGED Average significantly predicted current levels of EMA Effort, as well as physical demand with daily activities. However, unexpectedly, we failed to observe significant relationships between COGED and experience of mental demand, though the effect was in the predicted direction. EEfRT Average did not significantly predict either physical or mental demand.

Finally, we addressed whether participants with extreme task behavior may have influenced observed associations. While no extreme outliers (+/−3 SD) were identified, several participants solely chose one option (EEfRT: N=4, COGED: N=2). When excluding these participants, the relationship between COGED Average and EMA Effort remained in a similar direction with similar beta weight values but was no longer statistically reliable (p=0.15; S9 in OSM). The relationship between EEfRT Average and EMA Effort remained non-significant. However, this approach may be overly conservative, since it is ambiguous whether these participants are actually outliers, or failed to comply with instructions, or instead that their choices reflected a true extreme preference for a particular task.

Discussion

The current study examined whether laboratory-based measures of willingness to expend effort predicted self-reported experience of effort outside the lab. We found a significant positive association between COGED performance and EMA Effort. Unexpectedly, associations between EEfRT and EMA Effort were not statistically significant. Associations between COGED and EMA Effort remained significant when including EEfRT and NCS into the statistical model, providing modest support for an independent relationship. Further, COGED predicted EMA Effort even when statistically controlling for the enjoyment of daily activities, suggesting that COGED indexes the subjective experience of daily effort, independent of how enjoyable such activities may be. These data provide initial evidence for the ecological validity of the COGED paradigm.

Little is known regarding associations between experimental measures of effort and experience of effort in daily life. The current report makes a significant contribution to the literature, by examining this issue in healthy adults. The results are somewhat consistent with a previous report that demonstrated a positive association between EEfRT and self-reported enjoyment/interest with daily activities in individuals with schizophrenia (Moran et al., 2017). However, in the current study, we did not observe a relationship between EEfRT performance and any EMA variables. The lack of correlation with EEfRT may be due to reduced variability in EEfRT choice behavior.

Future Studies

First, future work with larger samples is needed to replicate the current study. Future studies may also benefit from collecting both objective (e.g., actigraphy) and subjective indicators of motivational experience. Studies could also benefit from examining associations between EMA Effort and biological indices of reward function. For example, a study recently found positive associations between reward-induced striatal dopamine release and tendency to engage in enjoyable behaviors (Kasanova et al., 2017). Relatedly, a recent neuroimaging study found that individuals with increased BOLD activation of ventral striatum to high reward amount offers exhibited a greater willingness to expend effort on COGED (Westbrook, Lamichhane, & Braver, 2019). However, no study to date has examined the association between neuroimaging during effort-based decision-making and daily motivational experience.

Limitations

In the current design, experimental tasks and EMA questions indexed potentially different aspects of motivational experience. Specifically, tasks indexed an individual’s willingness to expend effort, while EMA questions indexed experience of effort during completion of daily activities. Using Brehm’s theory of motivation intensity (Brehm & Self, 1989), the task metrics may be akin to potential motivation, while the EMA questions may be akin to motivation intensity. Thus, associations between task performance and EMA metrics in the current manuscript must be interpreted in the context of this distinction (Wright, 2008). Future reports would benefit from examining similar aspects of motivational experience through task and EMA to glean more precise associations. Further, it is difficult in the current EMA design to independently quantify the subjective experience of effort in daily life and whether the daily activity was objectively demanding. Additionally, experimental task order effects must be considered due to the fixed nature of the task presentation.

Summary

In conclusion, the current study provides preliminary support for the ecological and discriminant validity of the COGED paradigm. Future work will be needed to replicate, as well as extend this finding using objective measures of daily motivational experience and biological measures of reward function, for example ventral striatal BOLD activation measured via fMRI.

Figure 3:

Associations between Effort-Based Decision-Making and Self-Reported Daily Effort

Comment: Shaded region indicates 95% confidence interval