Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jun 1.
Published in final edited form as: J Psychiatr Res. 2021 Apr 30;138:436–443. doi: 10.1016/j.jpsychires.2021.04.022

DIGITAL PHENOTYPING ADHERENCE, FEASIBILITY, AND TOLERABILITY IN OUTPATIENTS WITH SCHIZOPHRENIA

Ian M Raugh 1, Sydney H James 1, Cristina M Gonzalez 1, Hannah C Chapman 1, Alex S Cohen 2, Brian Kirkpatrick 3, Gregory P Strauss 1,*
PMCID: PMC8192468  NIHMSID: NIHMS1699641  PMID: 33964681

Abstract

Digital phenotyping has potential for use as an objective and ecologically valid form of symptom assessment in clinical trials for schizophrenia. However, there are critical methodological factors that must be addressed before digital phenotyping can be used for this purpose. The current study evaluated levels of adherence, feasibility, and tolerability for active (i.e., signal and event contingent ecological momentary assessment surveys) and passive (i.e., geolocation, accelerometry, and ambulatory psychophysiology) digital phenotyping methods recorded from smartphone and smartband devices. Participants included outpatients diagnosed with schizophrenia (SZ: n = 54) and demographically matched healthy controls (CN: n = 55), who completed 6 days of digital phenotyping. Adherence was significantly lower in SZ than CN for active recordings, but not markedly different for passive recordings. Some forms of passive recordings had lower adherence (ambulatory psychophysiology) than others (accelerometry and geolocation). Active digital phenotyping adherence was predicted by higher psychosocial functioning, whereas passive digital phenotyping adherence was predicted by education, positive symptoms, negative symptoms, and psychosocial functioning in people with SZ. Both groups found digital phenotyping methods tolerable and feasibility was supported by low frequency of invalid responding, brief survey completion times, and similar impediments to study completion. Digital phenotyping methods can be completed by individuals with SZ with good adherence, feasibility, and tolerability. Recommendations are provided for using digital phenotyping methods in clinical trials for SZ.

Introduction

There has been increased interest in incorporating digital phenotyping (i.e., using mobile technology to collect data in everyday life) methods into psychiatric research and clinical trials (Insel, 2017; Onnela & Rauch, 2016). Digital phenotyping is typically divided into “active” and “passive” recording modalities (Insel, 2017; Onnela, 2020). Active refers to data collection that requires intentional task completion (e.g., survey, video, cognitive test) and is similar to methods including experience sampling methods and ecological momentary assessment. Passive refers to data that is collected in the background via mobile device sensors (e.g., accelerometry, geolocation, phone usage, ambulatory psychophysiology). Digital phenotyping may address limitations inherent to traditional clinical rating scales and questionnaires (e.g., cognitive impairment, halo effects, social desirability, low resolution)(Ben-Zeev, McHugo, Xie, Dobbins, & Young, 2012; Cohen et al., 2019; Cohen, Schwartz, et al., 2020; Cohen et al., 2021; Mote & Fulford, 2019; Torous & Keshavan, 2018). Specifically, passively collected geolocation accounts for variance in negative symptoms beyond subjective report (Raugh et al., 2020) and digital phenotyping methods provide scalability (i.e., “zooming” in or out on how a given construct changes over time) not available in clinical interviews or questionnaires (Cohen et al., 2021).

There is preliminary evidence supporting the reliability and validity of these methods in schizophrenia (SZ), suggesting utility as outcome measures in clinical trials and experimental psychopathology studies (Ben-Zeev et al., 2016; Buck et al., 2019; Cohen, Cowan, et al., 2020; Depp et al., 2019; Granholm et al., 2020; Granholm, Loh, & Swendsen, 2008). However, several important methodological questions remain to be addressed. For example, what is the frequency with which participants complete active tasks (e.g., surveys, videos) when expected to do so (i.e., adherence)? How frequently are passive data (e.g., geolocation) collected at the programmed intervals? Which person-related (e.g., age, education, cognitive impairment, employment status) and study-design related (e.g., survey length, number of days of recordings) factors influence rates of data collection in SZ (i.e., feasibility)? Are there unique barriers to using mobile technology that people with SZ face but healthy controls (CN) do not (and vice versa)? How tolerable do people with SZ find these methods? Answering these questions is critical for informing clinical trial study design and determining how appropriate digital phenotyping methods may be for specific subsets of the SZ population.

Prior digital phenotyping studies in SZ have primarily focused on adherence for active digital phenotyping. Results indicate that levels of survey completion are generally high in people with SZ and comparable to CN; however, there is considerable variability in rates of survey completion (60%−98%) (Ben-Zeev, Frounfelker, Morris, & Corrigan, 2012; Ben-Zeev, McHugo, et al., 2012; Brenner & Ben-Zeev, 2014; Collip et al., 2011; Culbreth, Moran, Kandala, Westbrook, & Barch, 2020; Gard, Kring, Gard, Horan, & Green, 2007; Granholm et al., 2020; Granholm, Ben-Zeev, Fulford, & Swendsen, 2013; Granholm et al., 2008; Johnson et al., 2009; Moran, Culbreth, & Barch, 2017, 2018; Moran, Culbreth, Kandala, & Barch, 2019; Myin-Germeys, Delespaul, & deVries, 2000; Myin-Germeys, Krabbendam, Jolles, Delespaul, & van Os, 2002; Myin-Germeys, Nicolson, & Delespaul, 2001; Oorschot, Lataster, Thewissen, Wichers, & Myin-Germeys, 2012; Pishva et al., 2014; Reininghaus et al., 2016; Sanchez, Lavaysse, Starr, & Gard, 2014; Strauss et al., 2020; Thewissen, Bentall, Lecomte, van Os, & Myin-Germeys, 2008; Visser, Esfahlani, Sayama, & Strauss, 2018). A recent meta-analysis of 79 experience sampling studies found a mean adherence rate of 78.7% (Vachon, Viechtbauer, Rintala, & Myin-Germeys, 2019). These adherence estimates are likely an over-estimation of rates observed in the broad schizophrenia population due to the common practice of reporting survey completion rates only for participants exceeding an a priori cut-off (typically ~33%). Adherence rates for other active methods that have only become feasible more recently (e.g., ambulatory videos and cognitive tests) and have yet to be systematically evaluated. Preliminary evidence suggests that adherence for completing cognitive tests via smartphone is relatively high (86%, Holmlund et al., 2020), but rates for ambulatory videos are considerably lower (27%, Cohen, Cowan, et al., 2020). Recently developed passive methods (e.g., geolocation and accelerometry) have yet to be evaluated for adherence. Across both active and passive recording methods, it will be important to determine whether adherence is influenced by the person-related and study-related factors outlined above. Feasibility (i.e., obstacles to data collection and methods for ensuring validity) and tolerability (i.e., whether the methods are experienced as aversive or intrusive) have also received relatively little attention in SZ (Ben-Zeev et al., 2016; Granholm et al., 2008), and it will be important to address these issues for both active and passive variables before they are used in clinical trials.

To address this critical gap in knowledge, the current study systematically evaluated adherence, feasibility, and tolerability of active and passive digital phenotyping methods in outpatients with SZ. It was hypothesized that adherence would be lower in SZ than CN for both active and passive digital phenotyping methods recorded via smartphone and smartband. It was hypothesized that greater age, number of children, being employed, positive symptoms, and negative symptoms would be associated with lower adherence rates, while greater education and functioning would be associated with greater adherence rates. Both active and passive methods were hypothesized to be feasible for use in SZ and CN, as indicated by short survey duration, low rates of exclusion for participants below adherence cut-offs, low rates of invalid survey responses, and low barriers to completing study procedures. Variables predicted to influence feasibility included: technology problems, unavailability, motivation, and tolerability. Tolerability was expected to be high and comparable in SZ and CN, as indicated by not finding the completion of digital phenotyping study procedures to be aversive.

Methods

Participants

Participants included 54 outpatients diagnosed with SZ and 55 CN. The groups did not differ on age, sex, parental education, or race, but CN had higher personal education (see Table 1). Participants were recruited from the local community using printed and online advertisements; SZ participants were also recruited from outpatient mental health centers. SZ diagnosis was made using the Structured Clinical Interview for the DSM-5 (SCID-5; First, Williams, Karg, & Spitzer, 2015). CN had no current major psychiatric diagnoses as determined via SCID-5, no current schizophrenia-spectrum personality disorders as determined via the SCID-PD (First, Spitzer, & Williams, 2015), no lifetime history of psychotic or bipolar disorders, no family history of psychosis, and were not currently prescribed any psychotropic medications. All participants denied lifetime neurological disorders and substance abuse within the last six months. All participants provided written informed consent for a protocol approved by the University of Georgia Institutional Review Board.

Table 1.

Sample characteristics

SZ (n = 54) CN (n = 55) Test statistic p
Age; M (SD) 39.17 (12.41) 39.07 (10.62) F = 0.02 .966
Male; n (%) 19 (35.2%) 17 (30.9%) χ2 = 0.22 .635
Personal education; M (SD) 13.24 (2.27) 15.4 (2.82) F = 19.33 < .001
Parental education; M (SD) 12.9 (2.91) 13.63 (2.85) F = 0.21 .647
Ethnicity; n (%) χ2 = 9.16 .103
 African American 17 (31.5%) 16 (29.1%)
 Asian American 0 4 (7.3%)
 Biracial 3 (5.6%) 3 (5.5%)
 Caucasian 32 (59.3%) 24 (43.6%)
 Hispanic/Latino 2 (3.7%) 6 (10.9%)
 Other 0 2 (3.6%)
Cognition; M (SD) 39.04 (15.22) 49.86 (10.95) F = 17.11 < .001
PANSS Positive 13.58 (5.13)
BNSS Total 15.38 (13.47)
Open in a new tab

Note. The SZ group consisted of 22 individuals diagnosed with schizophrenia, 29 with schizoaffective disorder, and 3 with bipolar disorder with psychotic features. Within the SZ group, 55.56% were prescribed an antipsychotic at the time of the study.

Procedures

Study procedures were completed over three visits, see Figure 1. Participants were provided with smartphones programmed using the mEMA application from Ilumivu (ilumivu.com) and the Alert app from Empatica. Participants were shown how to use the necessary applications and completed a practice survey with a research assistant to ensure they understood the response formats. See Supplemental Materials for a list of items and their response formats. Surveys were designed to take less than five minutes to complete and used skip logic (i.e., inclusion or exclusion of items based on previous responses) to minimize time burden. Within the momentary (i.e., signal-contingent) surveys, valid responding was assessed using infrequency items from the Chapman Physical and Social Anhedonia Scales (Chapman, Chapman, & Raulin, 1976). Infrequency items evaluate attentive responding based on how participants respond to commonly endorsed experiences (i.e., seeing a child playing or going to bed early when tired). One infrequency item was randomly selected and presented approximately three quarters of the way through each momentary survey. Additionally, response time for each item was recorded in seconds after the item appeared on the screen. As there could be several items on any given screen, response time per screen was calculated.

Figure 1.

Figure 1.

Open in a new tab

Summary of study procedures

CN = Control; SZ = Schizophrenia; SCID-5 = Structured Clinical Interview for the DSM-5; SCID-5-PD = SCID-5 Personality Disorders; BNSS = Brief Negative Symptom Scale (Kirkpatrick et al., 2011); PANSS = Positive and Negative Syndrome Scale (Kay, Fiszbein, & Opler, 1987); LOF = Level of Functioning Scale (Hawk, Carpenter, & Strauss, 1975); MCCB = MATRICS Consensus Cognitive Battery (Nuechterlein et al., 2008). Participants were assigned 11 total surveys per day. To allow for greater flexibility, event-contingent surveys (i.e., morning, event, and evening) were available “on-demand” (i.e., could be completed at any time) rather than scheduled. See main text for details on survey design and passive data recording.

To examine feasibility, evening surveys assessed the reasons that surveys were skipped that day. Participants could select more than one reason, including: driving, bathing, exercising, meeting or class, busy, misplacing the phone, technical issues, “phone battery died”, “annoyed with the prompts”, “didn’t notice the alarm”, “didn’t want to”, or other.

Passive digital phenotyping measures were collected from the smartphone (accelerometry, geolocation, and ambient sound) via the mEMA app and smartband (accelerometry, skin conductance, and skin temperature) via the Alert app. Phone sensors collected accelerometry (ACL) with one sample recorded for each change in XYZ coordinate motion. A total accelerometry value per sample was calculated using the root of the sum of the squares from each axis. Geolocation (GPS) was sampled every 10 minutes or when participants moved more than 10 meters. A participant’s location was also recorded upon completing a survey. The GPS data also provided an estimate of accuracy; only samples accurate to less than 35 meters were retained. Ambient sound (VOX) was recorded for five seconds every 10 minutes at 16 kHZ.

Smartband sensors collected accelerometry as gravitational force (g units) at a rate of 32 Hz and skin conductance and temperature at 4 Hz. Accelerometry from the smartband was summarized as a single value per sample using the same method as phone accelerometry. Participants were instructed to wear the band as much as possible except when it would be submerged. They were also instructed to charge the band every other night to account for its limited battery life (~ 30 hours). Samples where skin conductance or skin temperature were outside of physiological bounds (skin conductance < 0.1 or > 39.95 μS; skin temperature < 20 or > 40 °C were deleted. Data was transmitted from the smartband to the phone via Bluetooth connection, but the band was able to store up to 14 hours of data. Smartband data was epoched into one-minute periods.

Data Analysis

Analyses were conducted with R version 3.6 (R Core Team, 2020); see Supplemental Materials for a link to download the code.

Aim 1 (Adherence).

Active adherence was examined using a two Group (SZ, CN) X four Survey Type (momentary, morning, event, evening) mixed-models ANOVA. Passive adherence was evaluated using a two Group (SZ, CN) X four Data Type (ACL, GPS, VOX, and smartband) mixed models ANOVA. Significant effects of Survey Type or Data Type were followed up by post-hoc pairwise contrasts. Significant interactions were followed up by post-hoc between-group and within-group pairwise contrasts. Additional mixed-model ANOVAS were conducted using Survey Instance (1–8), Study Day (1–6), and Weekday. Significant effects of Survey Instance and Study Day were followed up by post-hoc contrasts to Instance 1 and Day 1, while significant effects of Weekday were followed up by post-hoc pairwise contrasts. All eight models for aim 1 were conducted using a multilevel approach due to repeated observations per person (one adherence value per person per predictor) with random intercepts by person and using Tukey’s method for multiple comparisons in post-hoc contrasts.

Predictors of overall adherence were examined as a series of multiple regression models. Survey adherence was analyzed as a multivariate multiple regression. Significant omnibus effects were followed by post-hoc univariate multiple regression models. Passive data adherence was examined in a series of multiple regression models.

Aim 2 (Feasibility).

Several survey timing variables were calculated, including total survey time, mean screen response time, total items completed, and variability in screen response time. As this data contains one observation per person at each survey instance, linear mixed models (LMM) using an uncorrelated covariance structure were used to examine Group X Survey Type differences on each variable. Analyses were nested within day, survey type, and person. These models used maximum likelihood estimation and random intercepts. Significant interactions were followed up by post-hoc between-group and within-group pairwise contrasts. See Supplemental materials for data analysis procedures for adherence cut-offs, infrequency item endorsement, reasons for missed surveys, and barriers to survey completion.

Aim 3 (Tolerability).

One-way ANOVAs were used to determine whether the groups differed in tolerability ratings (positive and negative) in debriefing interviews.

Results

Aim 1: Adherence

Active.

Mixed-models ANOVA indicated significant main effects of Group and Survey Type on active adherence with a medium effect size; however, the Group X Survey Type interaction was nonsignificant (see Table 2). The effect of Group was such that SZ had lower adherence than CN. The effect of Survey Type was such that momentary and evening surveys were not completed at significantly different rates, but were completed significantly less than morning and event surveys. As can be seen in Table 2, rates of adherence for active surveys were lower in SZ than CN, and in both groups adherence was highest for morning surveys, followed by event surveys, and then momentary and evening surveys.

Table 2.

Adherence rates for active and passive digital phenotyping methods.

SZ CN F
M (SD) k M (SD) k Group Data Type Group X Data Type Group Cohen’s d
Active 63.77% (34.12%) 2162 75.3% (28.71%) 2563 6.11* 15.66*** 0.13 0.53
 Momentary 55.71% (28.23%) 1444 66.67% (23.76%) 1768
 Morning 75.31% (28.36%) 267 85.45% (25.67%) 290 A
 Event 64.81% (39.21%) 253 78.79% (31.01%) 273 A, B
 Evening 59.26% (36.87%) 198 70% (30.66%) 232 B
Passive 2.53 118.96*** 0.31 0.24
 ACL 86.65% (31.5%) 1910 90.27% (25.76%) 2345
 GPS 72.65% (40.02) 1581 77.54% (39.08%) 2014 C
 VOX 39.69% (32.98%) 869 49.93% (29.5%) 1311 C, D
 Band 19.97% (30.06%) 432 27.96% (32.33%) 711 C, D, E
Open in a new tab

Note. ACL = Accelerometry, GPS = Geolocation, VOX = Vocal variables; A = Contrast to momentary surveys, B = Contrast to morning surveys, C = Contrast to ACL data, D = Contrast to GPS data, E = Contrast to VOX data; k = Number of active digital phenotyping samples

* =

p< .05,

** =

p< .01,

*** =

p< .001.

LMM was conducted to evaluate the impact of instance, study day, and weekday on active adherence (Figure 1). The Group X instance (F(7, 749) = 1, p = .431), Group X study day (F(5, 535) = 1.47, p = .197), and Group X weekday (F(6, 538) = 0.95, p = .457) interactions were nonsignificant. The main effect of instance was nonsignificant (F(7, 749) = 0.23, p = .978); however, there were significant effects of study day (F(5, 535) = 2.37, p = .038) and week day (F(6, 538) = 2.36, p = .029). The effect of study day was such that adherence decreased after day one, but was only significantly decreased on days four and five (see Figure 1B). The only significant contrast for Weekday was from Thursday to Saturday (t = 2.97, p = .048). A significant main effect of Group was observed in models of instance (F(1, 107) = 4.25, p = .042), but not study day (F(1, 107) = 3.59, p = .061) or weekday (F(1, 107) = 3.77, p = .055).

Passive.

Mixed-models ANOVA indicated a significant main effect of Data Type on passive adherence; the effects of Group and Group X Data Type were nonsignificant (see Table 2). The effect of Data Type was such that each type had a significantly different adherence rate (ACL > GPS > VOX > Band; see Table 2).

LMM was conducted to evaluate the effects of instance, study day, and weekday on passive adherence. The Group X Weekday interaction on passive adherence was significant (F(6, 1482) = 2.68, p = .01), but Group X Instance (F(21, 2213) = 0.51, p = .966) and Group X Study Day (F(15, 1471) = 0.71, p = .774) interactions were nonsignificant. The Group X Weekday interaction was such that adherence in CN was lower on Saturday compared to Monday (t = 3.62, p = .006), Tuesday (t = 5.01, p < .001), Wednesday (t = 3.97, p = .002), Thursday (t = 4.4, p < .001), and Friday (t = 3.26, p = .019); however, differences between days were nonsignificant in the SZ group. The main effect of instance was nonsignificant, F(7, 2216) = 0.64, p = .72. Significant main effects were observed for Study Day (F(5, 1476) = 6.12, p < .001) and Weekday (F(6, 1482) = 4.55, p < .001). A significant main effect of Group was found in models of instance (F(1, 104) = 4.44, p = .037), study day (F(1, 105) = 5.76, p = .018), and weekday (F(1, 105) = 6.35, p = .013), such that CN had greater adherence than SZ. Thus, passive adherence is generally lower in SZ than CN, decreases over the study period, and varies by the day of the week in CN but not SZ (see Figure 1).

Predictors of adherence.

Predictors of survey adherence are presented in Table 3. In SZ, higher functioning predicted greater event survey adherence. In CN, greater age predicted lower event survey adherence, while greater mean survey time was associated with lower event and evening survey adherence. All other predictors were nonsignificant in both groups.

Table 3.

Multivariate regressions predicting survey adherence per group.

Predictor Omnibus Momentary Morning Event Evening
SZ
Age 0.44 0.14 0.19 0.39 0.31
Personal Education 0.57 −0.5 −0.42 −0.48 −0.29
Cognition 1.14 0.3 0.39 0.14 0.06
Number of children 0.26 0.08 0.08 0.11 0.05
Employed 0.92 0.27 −0.02 −0.06 −0.45
Mean survey time 0.89 −0.19 −0.02 −0.27 −0.24
PANSS Positive 1.46 −0.07 −0.12 −0.21 −0.38
BNSS Total 0.81 0.06 −0.09 0.32 0.44
LOF 3.9* 0.1 −0.21 0.5* 0.32
CN
Age 4.81** −0.15 −0.17 −0.34* 0.12
Personal Education 0.95 0.23 0.15 0.12 0.15
Cognition 1.12 −0.12 −0.06 0.03 −0.02
Number of children 0.58 0.01 −0.15 0.09 −0.19
Employed 3.57* 1.08 −0.89 −0.65 0.18
Mean survey time 5.16** −0.14 0.02 −0.35** −0.32*
Open in a new tab

Note. PANSS = Positive and Negative Syndrome Scale, BNSS = Brief Negative Symptom Scale, LOF = Level of Functioning scale. Omnibus values are approximate F while all others are β. Employment coded 0 or 1, 1 = Unemployed/ Retired. CN were not rated on PANSS, BNSS, or LOF. Significant effects in bold.

* =

p< .05,

** =

p< .01,

*** =

p< .001.

Predictors of passive data adherence are presented in Table 4. In SZ, greater positive symptoms predicted less ACL data. Higher band adherence was predicted by higher education. Higher negative symptoms and greater functioning were associated with reduced VOX data. Passive data adherence was not significantly predicted by age, personal education, cognition, number of children, or employment status in CN, regardless of data type.

Table 4.

Regressions predicting passive data adherence per group.

Variable ACL GPS VOX Band
SZ
Age 0.2 0.01 0.25 −0.07
Personal education −0.53 0.04 0.13 0.48*
MCCB 0.09 −0.1 −0.17 −0.09
Number of children −0.14 0.01 −0.02 −0.07
Employment −0.73 −0.42 −0.48 −0.65
PANSS Positive −0.42* 0 −0.09 0.06
BNSS Total −0.33 −0.23 −0.46* 0.27
LOF −0.29 −0.26 −0.45* 0.22
Overall fit; F (R2) 1.6 (.29) 0.37 (.09) 1.31 (.25) 1.8 (.32)
CN
Age 0.19 −0.39 −0.07 −0.28
Personal Education −0.06 0.04 −0.2 −0.17
MCCB −0.24 0.1 −0.07 0.26
Number of children 0.02 0.27 0.38 0.26
Employed −0.78 −0.07 1.22 1.04
Overall fit; F (R2) 0.66 (.07) 0.85 (0.8) 1.79 (.16) 0.76 (.08)
Open in a new tab

Note. ACL = Accelerometry, GPS = Geolocation, VOX = Vocal variables, PANSS = Positive and Negative Syndrome Scale, BNSS = Brief Negative Symptom Scale, LOF = Level of Functioning scale. Unless otherwise specified, all values are β. Employment coded 0 or 1, 1 = Unemployed/ Retired. Band data consisted of accelerometry, electrodermal activity, and skin temperature.

* =

p< .05,

** =

p< .01,

*** =

p< .001.

Exploratory analyses were conducted to evaluate the influence of sex on adherence. No significant main effects or interactions with sex were observed, see Supplemental materials.

Aim 2: Feasibility

Survey duration.

Results related to survey duration variables are presented in the Supplemental materials. SZ were slower to complete surveys and less variable in their response times compared to CN; however, there was not a significant difference in the number of items completed by each group.

Adherence cutoffs.

Possible cut-offs for survey adherence are presented in the Supplemental Materials. Results indicated that using a cut-off of 50% or more may exclude more participants from the SZ group than CN group. However, effects were comparable at lower cutoffs.

Infrequency.

Rates of endorsing infrequency items was low and did not differ between groups (CN = 1.12%, SZ = 1.01%, χ2 = 1.4, p = .237); however, infrequency item endorsement was predicted by the number of items (β = 0.37, p = .002) and total survey time (β = 0.38, p = .001).

Obstacles.

Obstacle analyses are presented in the Supplemental materials. Obstacles were endorsed at similar rates in both groups, and some obstacles (e.g., meetings and technology problems) were related to adherence.

Aim 3: Tolerability

There was no group difference in positive (F(1, 59) = 0.82, p = .37) or negative (F(1, 58) = 1.07, p = .306) tolerability ratings in debriefing interviews. Positive ratings were high (SZ: M = 8.81, SD = 0.91; CN: M = 8.56, SD = 1.16) and negative ratings were low (SZ: M = 0.96, SD = 1.27; CN: M = 1.29, SD = 1.19).

Discussion

The current study evaluated whether individuals with SZ and CN differed in rates of adherence, feasibility, and tolerability for active and passive digital phenotyping methods. Results provide support for active and passive digital phenotyping methods and suggest areas for optimizing adherence.

Individuals with SZ had lower active survey adherence than CN for every survey type, consistent with meta-analytic findings (Vachon et al., 2019). Across both groups, adherence was highest for morning surveys, followed by evening surveys, and then momentary and event surveys. In SZ, social/vocational functioning was the only significant predictor of survey adherence. Rate of survey adherence in SZ was not predicted by several plausible variables, including: age, education, cognition, number of children, employment status, mean survey time, positive symptoms, or negative symptoms. These null results are consistent with previous research regarding active digital phenotyping (Granholm et al., 2008) and use of a smartphone intervention in SZ (Ben-Zeev et al., 2014). Adherence did not differ based on time of day, diminished across days of participation, and was higher on weekdays than weekends.

Rates of passive adherence do not appear to differ between SZ and CN. Within passive recording methods, rates of adherence were highest for accelerometry, followed by geolocation, ambulatory acoustics, and smartband data in both groups. Predictors of passive adherence in SZ include personal education, positive symptoms, negative symptoms, and functioning. There are several reasons why rates of data acquisition may be low for certain passive variables. For example, low smartband adherence may reflect difficulty using the smartband, forgetting to keep the device properly charged, or difficulty troubleshooting Bluetooth connectivity problems between the band and phone. Smartband data adherence was notably lower than that observed by other studies (88.5%, Raugh, Chapman, Bartolomeo, Gonzalez, & Strauss, 2019) due to Bluetooth connectivity and troubleshooting difficulties. Phone based accelerometry collected in this study was based on detected movement; therefore, missing samples may reflect participants leaving the phone on surfaces (not capturing valid movement), being stationary (capturing a valid lack of movement), or both. Additionally, lower phone accelerometry adherence was associated with greater severity of positive symptoms. This association could reflect not carrying the phone due to concerns of being monitored or a valid lack of movement due to the effects of positive symptoms on activity. The present study does not provide clear evidence of either possibility and this should be explored in the future. Geolocation is computationally demanding and based on the ability of the phone to communicate with satellites. Ambulatory acoustic data may be impacted by phone placement (microphone covered or not) and computational demands. Passive data types that are less computationally demanding (i.e., accelerometry) may be more consistently recorded than those that require more device processing power (i.e., geolocation).

Results also support the feasibility of active and passive digital phenotyping measures in SZ and CN. Specifically, for active surveys the time metrics indicated that SZ could complete the surveys quickly (average completion time was 2.5 minutes in SZ and 2.17 minutes in CN; approximately one minute faster than in Granholm et al., 2008). Importantly, SZ did not complete fewer items than CN even though they were slower and more variable in completion time. Feasibility and validity of active surveys was also supported by low rates of endorsing infrequency items and a similar rate and type of self-reported obstacles to survey completion. Obstacles most frequently reported were difficulties with the smartband and being “busy.” While our results do not provide a clear answer as to ideal survey length or duration, evidence from nonpsychiatric populations suggests that shorter surveys are better for adherence (Eisele et al., 2020). It may also be useful to examine activity responses in future studies in order to detect invalid responses indicated by participants reporting more activities than could reasonably be accomplished between surveys (e.g., visiting a doctor, exercising, and watching a movie all within a 90 minute time-period).

Another critical study design decision involves whether to use a cut-off for being considered a valid participant to include in analyses. Typical cut-offs for ecological momentary assessment surveys range from 20–33%, which results here indicated would not disproportionately remove participants from either group or result in substantial data loss for active or passive methods. However, extending that to 50% may remove more SZ participants than CN. Thus, regarding cut-offs, the best practice may be 25% for both forms of data collection. However, researchers should make sure to check that their cut-off does not exclude either group disproportionately and may consider employing cut-offs based on daily adherence, rather than total study adherence, in order to retain more active data. Regardless of the cut-off choices used, they must be clearly reported (Trull & Ebner-Priemer, 2020).

Tolerability is an important consideration with digital phenotyping designs to respect the time, energy, and resources of participants, and to make sure the experience is not aversive. In our sample, both groups found digital phenotyping highly tolerable, as evidenced by comparably favorable positivity and comparably low negativity ratings. While other research has identified tolerability with active digital phenotyping methods (Granholm et al., 2008), these ratings indicate that tolerability is still high with passive methods and ambulatory psychophysiology.

This study has several limitations. First, context variables (e.g., location, activities, social interactions) were only collected when surveys were completed, making it impossible to study the effects of context on adherence. However, this limitation was somewhat addressed by the evening surveys, which assessed reasons why surveys were skipped during the day to probe contextual influences. Second, smartphones and bands were provided to participants, and results may not generalize to “bring your own device” study designs. Although this limits generalizability, it does come with the advantage of ensuring consistent technology across participants, allowing a more controlled initial evaluation of adherence in this study. Third, participants were compensated for survey completion, as is most common in the literature but is associated with greater adherence (Vachon et al., 2019). Fourth, estimates of adherence may be specific to the design of the present study. It is unclear how findings might differ using a different number of surveys per day or a longer collection period (e.g., a clinical trial lasting several months). These results may be most informative for clinical trials using a “burst” design, where data is collected at set intervals with intervening periods without data collection. This burst design may be ideal for studies wanting to combine the power of active and passive digital phenotyping concurrently to obtain a richer understanding of passive data in the contexts where it is most relevant. Such burst designs may only collect for a few days at a time or may employ a similar six day period. Finally, outpatients in the chronic phase of illness were evaluated, and it is unclear how results differ among inpatients and those in the early course of illness.

Despite these limitations, findings have important implications for the design and implementation of digital phenotyping in SZ. We conclude with several recommendations for digital phenotyping research, presented in Table 5.

Table 5.

Recommendations for active and passive digital phenotyping methods.

Active digital phenotyping
  • 1

    Contact participants approximately three to four days into the study period to prevent drops in adherence. For longer studies, weekly check-ins may be necessary.

  • 2

    When designing a study, plan extra data collection to account for missing data. Piloting an extensive digital phenotyping battery will help with setting these expectations and identifying common technology problems and solutions.

  • 3

    Use skip logic to reduce unnecessary items and attempt to keep surveys short. However, surveys should include context items (e.g., location, social activity) in addition to items related to symptoms and constructs of interest (Cohen, Cox, et al., 2020).

  • 4

    Survey items should be evaluated psychometrically and multiple items should be used for a given construct (Trull & Ebner-Priemer, 2020).

  • 5

    Include infrequency or “catch” items, especially for longer surveys, to detect invalid responses. Combining infrequency items with item or survey reaction times may be a valuable means of identifying invalid responses.

  • 6

    When using an adherence cut-off, make sure it is not biased against any groups in the data. Overall adherence cut-offs exclude entire individuals from data analysis, which can impact the generalizability of the results. By excluding individual days, rather than entire participants, only days that have inadequate data for analyses are lost. This approach may be valuable to clinical trials where discarding entire participants for low adherence would be costly.

  • 7

    Researchers should consider presenting active data summaries to participants for their own enrichment, engagement, and possible therapeutic benefit. However, what data is seen as important by participants and how this can be presented should be considered in future studies that involve consumer focus groups representing the population(s) of interest.
Passive digital phenotyping
  • 8

    Continuous passive data collection should be paired with some form of daily active data collection (e.g., surveys, videos) in order to maximize the interpretability of passive data.

  • 9

    Passive data is a valuable adjunct to clinician and self-report data which may be used to identify individual symptom variation (i.e., those at elevated risk for relapse; (Barnett et al., 2018) and other psychological constructs of interest. However, the current evidence does not suggest that passive digital phenotyping should be used in place of clinician ratings and diagnostic evaluation.

  • 10

    Researchers should consider the utility of providing feedback on passive data collection to participants and the research team. Specifically, the use of notifications may improve adherence to passive data by allowing participants to be informed (through an app or by the researchers) that passive data is not being collected when expected. Commercial fitness trackers may provide useful examples of how to provide this feedback in ways that participants find engaging and useful.

  • 11

    Consideration should be given to the ethical conduct of passive data collection and ensuring participants recognize what types of data are collected and how that data is used. We recommend including questions to assess participant understanding of passive data collection when working with populations who may have difficulty providing informed consent due to cognitive impairments (e.g., schizophrenia). The aforementioned notifications may also aid informed participation in research by reminding participants that they are providing specific forms of data.

  • 12

    Passive data streams (e.g., accelerometry, geolocation, phone usage) should be examined with a three-fold validation process consisting of verification, analytic validation, and clinical validation (see Goldsack et al., 2020). Smartbands and ambulatory psychophysiological recording devices can vary widely in validity and should be compared to laboratory psychophysiology measures in order to evaluate data quality (Raugh et al., 2019).
Open in a new tab

Supplementary Material

1

Figure 2.

Figure 2.

Open in a new tab

Effects of instance, study day, and weekday on adherence in SZ vs CN Groups.

Error bars represent standard errors. EMA = Ecological momentary assessment; ACL = Accelerometry, GPS = Geolocation, VOX = Vocal variables. Panel A: Survey adherence based on survey instance. Panel B: Survey adherence based on study day. Panel C: Survey adherence based on weekday. Panel D: Passive data present by survey instance. Panel E: Passive data present by study day. Panel F: Passive data present by weekday. Panels B and E significance markers indicate difference in adherence relative to Day 1.

* = p < .05, ** = p < .01, *** = p < .001.

Conflict of Interest

This work was funded by NIMH grant R21-MH112925 to Dr. Strauss. Study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

G.P.S. and B.K. are original developers of the Brief Negative Symptom Scale (BNSS) and receive royalties and consultation fees from ProPhase LLC in connection with commercial use of the BNSS and other professional activities; these fees are donated to the Brain and Behavior Research Foundation. BK has received honoraria and travel support from ProPhase LLC for training pharmaceutical company raters on the BNSS, consulting fees and travel support from Genentech/Roche, Minerva Neurosciences, Lundbeck, and ProPhase LLC, consulting fees from anonymized pharmaceutical companies through Decision Resources, Inc. and from an investment capital company through Guideposts, and from Wockhardt Bio AG for consulting on a legal issue. GPS has consulted for Minerva Neurosciences and Lundbeck. Dr Cohen, Ms. Gonzalez, Mr. Raugh, Ms. James, and Ms. Chapman have no conflicts to report.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Barnett I, Torous J, Staples P, Sandoval L, Keshavan M, & Onnela J-P (2018). Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology, 43(8), 1660–1666. doi: 10.1038/s41386-018-0030-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ben-Zeev D, Brenner CJ, Begale M, Duffecy J, Mohr DC, & Mueser KT (2014). Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophrenia Bulletin, 40(6), 1244–1253. doi: 10.1093/schbul/sbu033 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Ben-Zeev D, Frounfelker R, Morris SB, & Corrigan PW (2012). Predictors of Self-Stigma in Schizophrenia: New Insights Using Mobile Technologies. Journal of dual diagnosis, 8(4), 305–314. doi: 10.1080/15504263.2012.723311 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Ben-Zeev D, McHugo GJ, Xie H, Dobbins K, & Young MA (2012). Comparing retrospective reports to real-time/real-place mobile assessments in individuals with schizophrenia and a nonclinical comparison group. Schizophrenia Bulletin, 38(3), 396–404. doi: 10.1093/schbul/sbr171 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Ben-Zeev D, Wang R, Abdullah S, Brian R, Scherer EA, Mistler LA, … Choudhury T (2016). Mobile behavioral sensing for outpatients and inpatients with schizophrenia. Psychiatric Services, 67(5), 558–561. doi: 10.1176/appi.ps.201500130 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brenner CJ, & Ben-Zeev D (2014). Affective forecasting in schizophrenia: comparing predictions to real-time Ecological Momentary Assessment (EMA) ratings. Psychiatric rehabilitation journal, 37(4), 316–320. doi: 10.1037/prj0000105 [DOI] [PubMed] [Google Scholar]
  7. Buck B, Scherer E, Brian R, Wang R, Wang W, Campbell A, … Ben-Zeev D (2019). Relationships between smartphone social behavior and relapse in schizophrenia: A preliminary report. Schizophrenia Research, 208, 167–172. doi: 10.1016/j.schres.2019.03.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chapman LJ, Chapman JP, & Raulin ML (1976). Scales for physical and social anhedonia. Journal of Abnormal Psychology, 85(4), 374–382. doi: 10.1037/0021-843X.85.4.374 [DOI] [PubMed] [Google Scholar]
  9. Cohen AS, Cowan T, Le TP, Schwartz EK, Kirkpatrick B, Raugh IM, … Strauss GP (2020). Ambulatory digital phenotyping of blunted affect and alogia using objective facial and vocal analysis: Proof of concept. Schizophrenia Research. doi: 10.1016/j.schres.2020.03.043 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cohen AS, Cox CR, Masucci MD, Le TP, Cowan T, Coghill LM, … Elvevåg B (2020). Digital phenotyping using multimodal data. Current behavioral neuroscience reports. doi: 10.1007/s40473-020-00215-4 [DOI] [Google Scholar]
  11. Cohen AS, Schwartz E, Le TP, Cowan T, Cox C, Tucker R, … Elvevåg B (2020). Validating digital phenotyping technologies for clinical use: the critical importance of “resolution”. World psychiatry : official journal of the World Psychiatric Association (WPA), 19(1), 114–115. doi: 10.1002/wps.20703 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cohen AS, Schwartz E, Le TP, Cowan T, Kirkpatrick B, Raugh IM, & Strauss GP (2021). Digital phenotyping of negative symptoms: the relationship to clinician ratings. Schizophrenia Bulletin, 47(1), 44–53. doi: 10.1093/schbul/sbaa065 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Cohen AS, Schwartz E, Le TP, Fedechko T, Kirkpatrick B, & Strauss GP (2019). Using biobehavioral technologies to effectively advance research on negative symptoms. World psychiatry : official journal of the World Psychiatric Association (WPA), 18(1), 103–104. doi: 10.1002/wps.20593 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Collip D, van Winkel R, Peerbooms O, Lataster T, Thewissen V, Lardinois M, … Myin-Germeys I (2011). COMT Val158Met-stress interaction in psychosis: role of background psychosis risk. CNS Neuroscience & Therapeutics, 17(6), 612–619. doi: 10.1111/j.1755-5949.2010.00213.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Culbreth AJ, Moran EK, Kandala S, Westbrook A, & Barch DM (2020). Effort, avolition, and motivational experience in schizophrenia: analysis of behavioral and neuroimaging data with relationships to daily motivational experience. Clinical Psychological Science, 8(3), 555–568. doi: 10.1177/2167702620901558 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Depp CA, Bashem J, Moore RC, Holden JL, Mikhael T, Swendsen J, … Granholm EL (2019). GPS mobility as a digital biomarker of negative symptoms in schizophrenia: a case control study. npj Digital Medicine, 2, 108. doi: 10.1038/s41746-019-0182-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Eisele G, Vachon H, Lafit G, Kuppens P, Houben M, Myin-Germeys I, & Viechtbauer W (2020). The effects of sampling frequency and questionnaire length on perceived burden, compliance, and careless responding in experience sampling data in a student population. Assessment, 107319112095710. doi: 10.1177/1073191120957102 [DOI] [PubMed] [Google Scholar]
  18. First MB, Spitzer RL, & Williams JBW (2015). Structured Clinical Interview For DSM-5(R) Personality Disorders (SCID-5-PD) (1st ed., p. 52). American Psychiatric Publishing. [Google Scholar]
  19. First MB, Williams JBW, Karg RS, & Spitzer RL (2015). Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). American Psychiatric Association. [Google Scholar]
  20. Gard DE, Kring AM, Gard MG, Horan WP, & Green MF (2007). Anhedonia in schizophrenia: distinctions between anticipatory and consummatory pleasure. Schizophrenia Research, 93 (1–3), 253–260. doi: 10.1016/j.schres.2007.03.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Goldsack JC, Coravos A, Bakker JP, Bent B, Dowling AV, Fitzer-Attas C, … Dunn J (2020). Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). npj Digital Medicine, 3(1), 55. doi: 10.1038/s41746-020-0260-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Granholm E, Ben-Zeev D, Fulford D, & Swendsen J (2013). Ecological Momentary Assessment of social functioning in schizophrenia: impact of performance appraisals and affect on social interactions. Schizophrenia Research, 145(1–3), 120–124. doi: 10.1016/j.schres.2013.01.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Granholm E, Holden JL, Mikhael T, Link PC, Swendsen J, Depp C, … Harvey PD (2020). What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia. Schizophrenia Bulletin, 46(2), 242–251. doi: 10.1093/schbul/sbz070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Granholm E, Loh C, & Swendsen J (2008). Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophrenia Bulletin, 34(3), 507–514. doi: 10.1093/schbul/sbm113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hawk AB, Carpenter WT, & Strauss JS (1975). Diagnostic criteria and five-year outcome in schizophrenia. A report from the International Pilot Study of schizophrenia. Archives of General Psychiatry, 32(3), 343–347. doi: 10.1001/archpsyc.1975.01760210077005 [DOI] [PubMed] [Google Scholar]
  26. Holmlund TB, Chandler C, Foltz PW, Cohen AS, Cheng J, Bernstein JC, … Elvevåg B (2020). Applying speech technologies to assess verbal memory in patients with serious mental illness. npj Digital Medicine, 3, 33. doi: 10.1038/s41746-020-0241-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Insel TR (2017). Digital phenotyping: technology for a new science of behavior. The Journal of the American Medical Association, 318(13), 1215–1216. doi: 10.1001/jama.2017.11295 [DOI] [PubMed] [Google Scholar]
  28. Johnson EI, Grondin O, Barrault M, Faytout M, Helbig S, Husky M, … Swendsen J (2009). Computerized ambulatory monitoring in psychiatry: a multi-site collaborative study of acceptability, compliance, and reactivity. International journal of methods in psychiatric research, 18(1), 48–57. doi: 10.1002/mpr.276 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kay SR, Fiszbein A, & Opler LA (1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13(2), 261–276. doi: 10.1093/schbul/13.2.261 [DOI] [PubMed] [Google Scholar]
  30. Kirkpatrick B, Strauss GP, Nguyen L, Fischer BA, Daniel DG, Cienfuegos A, & Marder SR (2011). The brief negative symptom scale: psychometric properties. Schizophrenia Bulletin, 37(2), 300–305. doi: 10.1093/schbul/sbq059 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Moran EK, Culbreth AJ, & Barch DM (2017). Ecological momentary assessment of negative symptoms in schizophrenia: Relationships to effort-based decision making and reinforcement learning. Journal of Abnormal Psychology, 126(1), 96–105. doi: 10.1037/abn0000240 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Moran EK, Culbreth AJ, & Barch DM (2018). Emotion regulation predicts everyday emotion experience and social function in schizophrenia. Clinical psychological science : a journal of the Association for Psychological Science, 6(2), 271–279. doi: 10.1177/2167702617738827 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Moran EK, Culbreth AJ, Kandala S, & Barch DM (2019). From neuroimaging to daily functioning: A multimethod analysis of reward anticipation in people with schizophrenia. Journal of Abnormal Psychology, 128(7), 723–734. doi: 10.1037/abn0000461 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Mote J, & Fulford D (2019). Ecological momentary assessment of everyday social experiences of people with schizophrenia: A systematic review. Schizophrenia Research. doi: 10.1016/j.schres.2019.10.021 [DOI] [PubMed] [Google Scholar]
  35. Myin-Germeys I, Delespaul PA, & deVries MW (2000). Schizophrenia patients are more emotionally active than is assumed based on their behavior. Schizophrenia Bulletin, 26(4), 847–854. doi: 10.1093/oxfordjournals.schbul.a033499 [DOI] [PubMed] [Google Scholar]
  36. Myin-Germeys I, Krabbendam L, Jolles J, Delespaul PA, & van Os J (2002). Are cognitive impairments associated with sensitivity to stress in schizophrenia? An experience sampling study. The American Journal of Psychiatry, 159(3), 443–449. doi: 10.1176/appi.ajp.159.3.443 [DOI] [PubMed] [Google Scholar]
  37. Myin-Germeys I, Nicolson NA, & Delespaul PA (2001). The context of delusional experiences in the daily life of patients with schizophrenia. Psychological Medicine, 31(3), 489–498. doi: 10.1017/s0033291701003646 [DOI] [PubMed] [Google Scholar]
  38. Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, … Marder SR (2008). The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. The American Journal of Psychiatry, 165(2), 203–213. doi: 10.1176/appi.ajp.2007.07010042 [DOI] [PubMed] [Google Scholar]
  39. Onnela J-P (2020). Opportunities and challenges in the collection and analysis of digital phenotyping data. Neuropsychopharmacology. doi: 10.1038/s41386-020-0771-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Onnela J-P, & Rauch SL (2016). Harnessing Smartphone-Based Digital Phenotyping to Enhance Behavioral and Mental Health. Neuropsychopharmacology, 41(7), 1691–1696. doi: 10.1038/npp.2016.7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Oorschot M, Lataster T, Thewissen V, Wichers M, & Myin-Germeys I (2012). Mobile assessment in schizophrenia: a data-driven momentary approach. Schizophrenia Bulletin, 38(3), 405–413. doi: 10.1093/schbul/sbr166 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Pishva E, Drukker M, Viechtbauer W, Decoster J, Collip D, van Winkel R, … Kenis G (2014). Epigenetic genes and emotional reactivity to daily life events: a multi-step gene-environment interaction study. Plos One, 9(6), e100935. doi: 10.1371/journal.pone.0100935 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Raugh IM, Chapman HC, Bartolomeo LA, Gonzalez C, & Strauss GP (2019). A comprehensive review of psychophysiological applications for ecological momentary assessment in psychiatric populations. Psychological Assessment, 31(3), 304–317. doi: 10.1037/pas0000651 [DOI] [PubMed] [Google Scholar]
  44. Raugh IM, James SH, Gonzalez CM, Chapman HC, Cohen AS, Kirkpatrick B, & Strauss GP (2020). Geolocation as a digital phenotyping measure of negative symptoms and functional outcome. Schizophrenia Bulletin, 46(6), 1596–1607. doi: 10.1093/schbul/sbaa121 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Reininghaus U, Kempton MJ, Valmaggia L, Craig TKJ, Garety P, Onyejiaka A, … Morgan C (2016). Stress sensitivity, aberrant salience, and threat anticipation in early psychosis: an experience sampling study. Schizophrenia Bulletin, 42(3), 712–722. doi: 10.1093/schbul/sbv190 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Sanchez AH, Lavaysse LM, Starr JN, & Gard DE (2014). Daily life evidence of environment-incongruent emotion in schizophrenia. Psychiatry Research, 220(1–2), 89–95. doi: 10.1016/j.psychres.2014.07.041 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Strauss GP, Esfahlani FZ, Granholm E, Holden J, Visser KF, Bartolomeo LA, & Sayama H (2020). Mathematically modeling anhedonia in schizophrenia: A stochastic dynamical systems approach. Schizophrenia Bulletin. doi: 10.1093/schbul/sbaa014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Thewissen V, Bentall RP, Lecomte T, van Os J, & Myin-Germeys I (2008). Fluctuations in self-esteem and paranoia in the context of daily life. Journal of Abnormal Psychology, 117(1), 143–153. doi: 10.1037/0021-843X.117.1.143 [DOI] [PubMed] [Google Scholar]
  49. Torous J, & Keshavan M (2018). A new window into psychosis: The rise digital phenotyping, smartphone assessment, and mobile monitoring. Schizophrenia Research, 197, 67–68. doi: 10.1016/j.schres.2018.01.005 [DOI] [PubMed] [Google Scholar]
  50. Trull TJ, & Ebner-Priemer UW (2020). Ambulatory assessment in psychopathology research: A review of recommended reporting guidelines and current practices. Journal of Abnormal Psychology, 129(1), 56–63. doi: 10.1037/abn0000473 [DOI] [PubMed] [Google Scholar]
  51. Vachon H, Viechtbauer W, Rintala A, & Myin-Germeys I (2019). Compliance and Retention With the Experience Sampling Method Over the Continuum of Severe Mental Disorders: Meta-Analysis and Recommendations. Journal of Medical Internet Research, 21(12), e14475. doi: 10.2196/14475 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Visser KF, Esfahlani FZ, Sayama H, & Strauss GP (2018). An ecological momentary assessment evaluation of emotion regulation abnormalities in schizophrenia. Psychological Medicine, 48(14), 2337–2345. doi: 10.1017/S0033291717003865 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1699641-supplement-1.docx (49.1KB, docx)

RESOURCES