Skip to main content
PMC home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2022 Oct 25;39(12):794–804. doi: 10.1002/da.23286

Prospective associations of text‐message‐based sentiment with symptoms of depression, generalized anxiety, and social anxiety

Caitlin A Stamatis 1,✉, Jonah Meyerhoff 1, Tingting Liu 2,3, Garrick Sherman 2, Harry Wang 4, Tony Liu 4,5, Brenda Curtis 3, Lyle H Ungar 2,4, David C Mohr 1
PMCID: PMC9729432  NIHMSID: NIHMS1841600  PMID: 36281621

Abstract

Objective

Language patterns may elucidate mechanisms of mental health conditions. To inform underlying theory and risk models, we evaluated prospective associations between in vivo text messaging language and differential symptoms of depression, generalized anxiety, and social anxiety.

Methods

Over 16 weeks, we collected outgoing text messages from 335 adults. Using Linguistic Inquiry and Word Count (LIWC), NRC Emotion Lexicon, and previously established depression and stress dictionaries, we evaluated the degree to which language features predict symptoms of depression, generalized anxiety, or social anxiety the following week using hierarchical linear models. To isolate the specificity of language effects, we also controlled for the effects of the two other symptom types.

Results

We found significant relationships of language features, including personal pronouns, negative emotion, cognitive and biological processes, and informal language, with common mental health conditions, including depression, generalized anxiety, and social anxiety (ps < .05). There was substantial overlap between language features and the three mental health outcomes. However, after controlling for other symptoms in the models, depressive symptoms were uniquely negatively associated with language about anticipation, trust, social processes, and affiliation (βs: −.10 to −.09, ps < .05), whereas generalized anxiety symptoms were positively linked with these same language features (βs: .12–.13, ps < .001). Social anxiety symptoms were uniquely associated with anger, sexual language, and swearing (βs: .12–.13, ps < .05).

Conclusion

Language that confers both common (e.g., personal pronouns and negative emotion) and specific (e.g., affiliation, anticipation, trust, and anger) risk for affective disorders is perceptible in prior week text messages, holding promise for understanding cognitive‐behavioral mechanisms and tailoring digital interventions.

Keywords: anxiety, depression, digital phenotyping, personal sensing, sentiment analysis

1. INTRODUCTION

Depression and anxiety disorders are common mental illnesses worldwide, affecting millions of adults and producing significant burden for patients and family members (Moreno‐Agostino et al., 2021; Yang et al., 2021). Better understanding how shared and distinct mechanisms of depression and anxiety symptoms surface in patients' daily lives may enhance the ability to deliver targeted treatments.

Depression and anxiety symptoms are expressed in day‐to‐day behaviors such as social interactions and language use (Brockmeyer et al., 2015; Geyer et al., 2018), including online interactions. Advantages of examining online verbal expression to model latent symptoms of depression, general anxiety, and social anxiety symptoms is that the language is recorded, represents an objective record of verbal expression, and can be categorized and analyzed.

The majority of studies on online linguistic markers of affective symptoms have been conducted with language from social media platforms, which allow people to share their daily experiences, express feelings and thoughts, and record diverse information about their social interactions (Chancellor & De Choudhury, 2020). Linguistic analysis of social media has proven useful in predicting depression (Eichstaedt et al., 2018), anxiety (Al‐Mosaiwi & Johnstone, 2018), loneliness (Guntuku et al., 2019), personality (Mairesse et al., 2007), and other mental health issues (Guntuku et al., 2017). Findings indicate that people with depression are more likely to use more first‐person pronouns, talk about pain and rumination, and express anger and aggression (Eichstaedt et al., 2018). Symptoms of general anxiety disorder have been linked to greater tentativeness, likely reflecting a higher degree of uncertainty (O'Dea et al., 2021). However, these studies on online language and affective symptoms are largely cross‐sectional and specific to social media, which may not reflect day‐to‐day communication patterns or signal prospective risk for symptoms.

Distinct from public‐facing social media data, text messages offer a novel corpus of private, directed communications for examining language associated with affective symptoms. Text messages are more frequent than social media posts (Smith, 2015) and less influenced by social desirability, facilitating more granular visibility into changes in linguistic patterns over time and across social relationships and tie strength. One recent study identified correlations between depression symptoms and text message‐based depression, emotional, and personal pronoun language (Liu et al., 2022). However, no studies have simultaneously considered prospective associations of text message sentiment with depression and anxiety symptoms. From a translational perspective, given that text messages are used to deliver digital interventions for depression (Senanayake et al., 2019) and anxiety (Anstiss & Davies, 2015), better identifying language within texts that signals distinct and common risk for depression and anxiety symptoms could lay the groundwork for refining digital interventions via targeted, in‐the‐moment interventions (Wilhelm et al., 2020).

Prior research points to several theory‐driven language markers as strong candidates for predicting affective symptoms, including first‐person pronouns and negative, absolutist language. The response styles theory of rumination in depression (Watkins & Nolen‐Hoeksema, 2014), which has also been applied to worry in generalized anxiety (Watkins & Roberts, 2020), asserts that these symptoms are driven by repetitive, self‐focused negative thinking that becomes habitual over time. Applied to language, response styles theory would suggest that people with high levels of depression and anxiety might communicate using more negative words, fewer positive words, and greater self‐focused language (e.g., “I” pronouns), and that the tendency to communicate in this manner may be habitual or outside of conscious awareness. In line with this notion, reductions in the use of “I” and present‐tense verbs have been linked with symptom improvement during text‐based therapy, despite language not being a central focus of treatment (Nook et al., 2022). Additional language markers that predict symptom change during therapy for depression include positive emotion words, negative emotion words, and certainty (Hernandez‐Ramos et al., 2022), the latter of which may reflect cognitive distortions. Absolutist language (superlatives and intensifiers) has also been linked with suicidal thoughts, and greater pronoun use has been linked with suicidal behavior (Homan et al., 2022). Again, however, these associations have never been tested using in‐vivo private data streams such as text messages.

The overarching goal of the present study was to test associations between text message sentiment and subsequent symptoms of depression, generalized anxiety, and social anxiety. Our specific aims were to (1) identify common linguistic features that predict depression, generalized anxiety, and social anxiety symptoms the following week; we hypothesized that the use of absolutist words (Al‐Mosaiwi & Johnstone, 2018), personal pronouns (particularly I/we), nonfluencies, and tentativeness (O'Dea et al., 2021) would be associated with common risk for affective psychopathology; (2) compare text message sentiment features that differentially predict symptoms of depression, generalized anxiety, and social anxiety the following week, with one hypothesized distinction being an association of depression versus anxiety symptoms with past versus future lexica, respectively; and (3) evaluate the degree to which accounting for text message sentiment improves overall ability to predict symptom severity relative to the predictive power of related symptoms alone. We chose to examine prospective associations, rather than concurrent relationships, because we believe the ability to predict the near‐future onset of symptoms using sensing features would be clinically useful and a foundational step for future just‐in‐time interventions.

2. METHODS

2.1. Participants and procedures

Participants were recruited from across the United States using social media and online advertisements, as well as an internally maintained registry of people who have indicated interest in participating in digital mental health clinical trials and completed a variety of prescreening assessments. We also recruited participants through Focus Pointe Global (which merged with the Schlesinger Group during this study). Focus Pointe Global panel members were sent email invitations to participate in the study, including a link to the screening site. We enrolled 659 participants in this study over two periods: February–April, 2020 (n = 370) and January–April, 2021 (n = 289).

To attain a sample with elevated affective symptoms, we oversampled for people having at least moderate depressive symptom severity on the Patient Health Questionnaire‐8 (PHQ‐8) ≥ 10 (Kroenke et al., 2009). Eligibility criteria for the present study included: living in the United States; being able to speak and read English at a level that enabled the participant to provide informed consent in English and participate in all study procedures and assessments; and having an Android smartphone with a data plan. Exclusionary criteria were: a self‐reported diagnosis of bipolar disorder, schizophrenia, or other psychotic disorder; sharing a smartphone with another person; and not being willing to share smartphone data necessary for sensor analyses.

All study protocols and procedures were approved by Northwestern University's Institutional Review Board, and all participants provided electronic informed consent before beginning study procedures. Participants were informed that they had the option not to provide consent or to withdraw from the study at any point if privacy concerns arose. Participants were compensated up to $142 for completing assessments, with compensation prorated according to percent completion.

Each wave of data collection took place over 16 weeks. Participants completed online symptom assessments at baseline and every 3 weeks through the end of the study (i.e., Weeks 4, 7, 10, 13, and 16), including measures of depression symptoms (PHQ‐8; Kroenke et al., 2009), generalized anxiety symptoms (Generalized Anxiety Disorder 7‐item scale [GAD‐7]; Spitzer et al., 2006), and social anxiety symptoms (Social Phobia Inventory [SPIN]; Connor et al., 2000). Depression symptoms were measured via smartphone‐based ecological momentary assessment, once at the beginning and end of the assessment week, whereas generalized and social anxiety symptoms were assessed using online questionnaires. Text message sentiment data were collected using the LifeSense app, built on the Passive Data Kit platform (Audacious Software, 2018). The platform conducted on‐device processing of text message data to calculate sentiment scores for each message sent and received; this allowed us to protect privacy by transmitting only sentiment score data, and not the raw text, off the devices. Sentiment scores were computed using the LIWC 2015 lexica categories (Pennebaker et al., 2015), the NRC Emotion Lexicon (Schwartz et al., 2014), and the Depression and Stress Lexica previously developed using Facebook data (Guntuku et al., 2017; see Supplementary Materials for details).

2.2. Data analysis

Analyses were conducted in R (version 4.1.0), using sentiment scores from outgoing messages only. We did not analyze texts that were part of group text conversations; only one‐on‐one texts were included. Sentiment data from text messages were aggregated over 2‐week intervals (Figure 1). While there was no minimum length of text messages required, to increase the reliability of sentiment score estimates, we excluded data from a given person for a given 2‐week period if the person had sent fewer than 50 outgoing messages for that interval.

Figure 1.

Figure 1

Open in a new tab

Overview of study design. GAD‐7, Generalized Anxiety Disorder 7‐item scale; PHQ‐8, Patient Health Questionnaire‐8; SPIN, Social Phobia Inventory.

Using the lmerTest package (Kuznetsova et al., 2017), we ran multilevel regression models to test associations of sentiment scores across weeks t and t + 1 with subsequent symptoms at t + 2. Sentiment predictors were person‐mean centered, and for each sentiment category, both a person mean term and within‐person deviation term were included in the model. Additional predictors included time (week) and the random intercept of person, as well as age and gender as covariates, which were included in keeping with prior studies (Eichstaedt et al., 2018; Preoţiuc‐Pietro et al., 2017; Yaden et al., 2018). For Aim 1, we tested a series of models evaluating the influence of each sentiment category on next‐week PHQ‐8, GAD‐7, and SPIN, controlling only for age and gender. In the second set of models (Aim 2), we repeated these models while also controlling for the other two symptom scales (e.g., in models where PHQ‐8 was the outcome, GAD‐7 and SPIN were included as covariates), which allowed us to evaluate the specificity of effects. A Benjamini–Hochberg correction for multiple comparisons was applied (Benjamini & Hochberg, 1995).

For the exploratory aim, we again tested a series of multilevel regression models, this time with a focus on overall variance explained in each symptom outcome based on (1) the other two symptoms alone, and (2) the additive effects of sentiment scores. In the first set of models, we predicted each symptom outcome based on the other two symptom measures, controlling for age and gender. In the second set of models, we added in all bottom‐level sentiment categories simultaneously as additional predictors, allowing us to test the question of whether including language features improves overall symptom prediction (e.g., of PHQ‐8) beyond the effects of other symptoms (e.g., GAD‐7 and SPIN). As in the previous aims, sentiment predictors were person‐mean centered, and for each sentiment category, both a person mean term and within‐person deviation term were included in the model.

3. RESULTS

3.1. Participants

Of the 659 who were eligible and signed up for the study, 107 withdrew. An additional 36 were excluded from analysis due to missing symptom inventories, and 181 were excluded from analysis after applying the criterion of having at least 50 outgoing messages for a 2‐week period. As a result, 335 participants (M age = 40.4, SD = 12.0, range = 18–73; 75.8% female gender) were included in analyses (see Table 1 for full demographics).

Table 1.

Participant demographics

Variable Total n = 335
Age, mean (SD) 40.4 (12.0)
Sex (assigned at birth), n (%)
Female 260 (77.6%)
Male 75 (22.4%)
Gender identity, n (%)
Female 254 (75.8%)
Male 74 (22.1%)
Nonbinary 4 (1.2%)
Transgender 2 (0.6%)
Genderqueer/gender nonconforming 1 (0.3%)
Race, n (%)
White 277 (82.7%)
Black/African American 35 (10.4%)
More than one race 11 (3.3%)
Asian 7 (2.1%)
Prefer not to answer 3 (0.9%)
Native American/Alaskan Native 2 (0.6%)
Ethnicity, n (%)
Non‐Hispanic/Non‐Latinx 312 (93.1%)
Hispanic/Latinx 22 (6.6%)
Unknown/prefer not to answer 1 (0.3%)
Education, n (%)
Bachelor's degree 97 (29.0%)
Some college, no degree 87 (26.0%)
Associate's degree 65 (19.4%)
Graduate degree 62 (18.5%)
High school/GED 22 (6.6%)
Some high school 2 (0.6%)
Marital status, n (%)
Married 115 (34.3%)
Single/never married 107 (31.9%)
Divorced 50 (14.9%)
Living with partner 50 (14.9%)
Separated 9 (2.7%)
Domestic partnership 3 (0.9%)
Unknown/prefer not to answer 1 (0.3%)
Household income, n (%)
$20,000–$39,000 73 (21.8%)
$60,000–$99,000 72 (21.5%)
$40,000–$59,000 66 (19.7%)
>$100,000 60 (17.9%)
$10,000–$19,000 28 (8.4%)
<$10,000 28 (8.4%)
Unknown/prefer not to answer 8 (2.4%)
Employment, n (%)
Employed 209 (62.4%)
Unemployed 53 (15.8%)
Disability 33 (9.9%)
Other 30 (9.0%)
Retired 9 (2.7%)
Prefer not to answer 1 (0.3%)
Open in a new tab

All reported p‐values are Benjamini–Hochberg corrected. All reported results reflect between‐subject effects; after correction for multiple comparisons, none of the within‐person associations between language and affective symptoms were significant.

3.2. Aim 1

From the first set of models, there was significant overlap in sentiment categories associated with the three symptom scales (Table 2; “Baseline Model”), particularly for PHQ‐8 and GAD‐7.

Table 2.

Associations between text message sentiment and symptoms the following week

Sentiment category Outcome
PHQ‐8 GAD‐7 SPIN
Baseline model β (SE) Controlling for GAD‐7, SPIN β (SE) Baseline model β (SE) Controlling for PHQ‐8, SPIN β (SE) Baseline model β (SE) Controlling for PHQ‐8, GAD‐7 β (SE)
LIWC 2015 (Pennebaker et al., 2015)
Function words .11 (0.05) .05 (0.03) .10 (0.05) .01 (0.03) .11 (0.05) .08 (0.05)
Pronouns .14 * (0.05) .05 (0.03) .15 * (0.05) .03 (0.03) .16 * (0.05) .11 (0.04)
Personal pronouns .13 * (0.05) .04 (0.03) .16 ** (0.05) .05 (0.03) .16 * (0.05) .11 (0.04)
I (First person singular) .13 * (0.05) .06 (0.03) .14 * (0.05) .04 (0.03) .10 (0.05) .06 (0.05)
She/He (Third person singular) .07 (0.05) −.01 (0.03) .13 * (0.05) .05 (0.03) .17 ** (0.05) .14 * (0.04)
Articles −.06 (0.05) .01 (0.03) −.13 * (0.05) −.08 (0.03) −.07 (0.05) −.04 (0.04)
Negations .17 ** (0.05) .09 (0.03) .14 * (0.05) .01 (0.03) 0.14 * (0.05) 0.10 (0.04)
Common adverbs .15 * (0.05) .08 (0.04) .12 (0.05) .01 (0.03) .11 (0.05) .07 (0.05)
Common verbs .10 (0.05) .04 (0.03) .10 (0.05) .02 (0.03) .11 (0.05) .08 (0.04)
Quantifiers .09 (0.05) .07 (0.03) .02 (0.05) −.05 (0.03) .07 (0.05) .06 (0.04)
Affective processes
Negative emotion .22 *** (0.05) .08 (0.03) .25 *** (0.04) .07 (0.03) .25 *** (0.04) .19 ** (0.04)
Anxiety .12 * (0.05) .02 (0.03) .17 ** (0.04) .07 (0.03) .13 * (0.05) .10 (0.04)
Anger .14 * (0.04) .04 (0.03) .18 *** (0.04) .06 (0.03) .19 *** (0.04) .14 ** (0.04)
Sadness .21 *** (0.05) .11 * (0.03) .17 ** (0.05) .03 (0.03) .14 * (0.05) .09 (0.04)
Social processes −.04 (0.05) −.09 * (0.03) .11 * (0.04) .13 *** (0.03) .01 (0.05) −0.02 (0.04)
Family .09 (0.05) .03 (0.03) .10 (0.05) .03 (0.03) .10 (0.05) .08 (0.04)
Female references .06 (0.05) .01 (0.04) .09 (0.05) .01 (0.03) .14 * (0.05) .11 (0.05)
Male peferences .07 (0.05) .00 (0.03) .13 * (0.04) .06 (0.03) .11 (0.04) .09 (0.04)
Cognitive processes .19 ** (0.05) .10 * (0.04) .15 * (0.05) .01 (0.03) .16 * (0.05) .12 (0.05)
Discrepancy .10 (0.05) .04 (0.03) .11 * (0.05) .03 (0.03) .11 (0.05) .08 (0.04)
Tentativeness .11 (0.05) .07 (0.03) .05 (0.05) −.03 (0.03) .11 (0.05) .09 (0.04)
Differentiation .18 ** (0.05) .10 * (0.03) .14 * (0.05) .01 (0.03) .14 * (0.05) .10 (0.04)
Perceptual processes
Hear .10 (0.05) .04 (0.03) .13 * (0.04) .06 (0.03) .05 (0.05) .02 (0.04)
Biological processes .11 * (0.04) .04 (0.03) .12 * (0.04) .02 (0.03) .14 * (0.04) .11 (0.04)
Body .12 * (0.05) .05 (0.03) .13 * (0.04) .02 (0.03) .12 * (0.04) .09 (0.04)
Health .11 (0.05) .05 (0.03) .11 * (0.05) .03 (0.03) .08 (0.05) .05 (0.04)
Sexuality .11 * (0.04) .02 (0.03) .15 ** (0.04) .05 (0.03) .16 ** (0.04) .12 * (0.04)
Drives −.06 (0.04) −.10 * (0.03) .09 (0.04) .13 *** (0.03) −.02 (0.04) −.04 (0.04)
Risk .12 * (0.04) .05 (0.03) .14 ** (0.04) .06 (0.03) .07 (0.04) .03 (0.04)
Affiliation −.06 (0.05) −.09 * (0.03) .08 (0.04) .13 *** (0.03) −.05 (0.04) −.05 (0.04)
Time orientations
Past focus .12 (0.05) .08 (0.03) .06 (0.05) −.02 (0.03) .06 (0.05) .04 (0.04)
Present focus .07 (0.05) .01 (0.03) .10 (0.05) .04 (0.03) .10 (0.05) .08 (0.04)
Informal language .09 (0.05) .01 (0.03) .13 * (0.05) .06 (0.03) .11 (0.05) .08 (0.05)
Swear words .12 * (0.04) .04 (0.03) .16 ** (0.04) .05 (0.03) .16 ** (0.04) .12 * (0.04)
Netspeak .16 * (0.05) .05 (0.04) .19 ** (0.05) .08 (0.03) .12 (0.05) .07 (0.05)
Assent −.10 (0.05) −.05 (0.03) −.09 (0.05) −.03 (0.03) −.03 (0.05) .00 (0.04)
Fillers .14 * (0.04) .09 * (0.03) .10 (0.04) .01 (0.03) .08 (0.04) .05 (0.04)
NRC Emotion Lexicon (Schwartz et al., 2014)
Sentiment −.22 *** (0.05) −.11 * (0.03) −.21 *** (0.05) −.05 (0.03) −.17 ** (0.05) −.11 (0.04)
Negative sentiment −.20 ** (0.05) −.09 (0.03) −.21 *** (0.05) −.05 (0.03) −.20 ** (0.05) −.13 * (0.04)
Anger .15 * (0.05) .05 (0.03) .18 ** (0.04) .06 (0.03) .18 ** (0.05) .13 * (0.04)
Disgust .11 (0.05) .04 (0.03) .12 * (0.04) .02 (0.03) .15 ** (0.04) .11 (0.04)
Fear −.14 * (0.05) −.08 (0.03) −.12 * (0.05) −.02 (0.03) −.08 (0.05) −.05 (0.04)
Sadness .08 (0.05) .00 (0.03) .14 * (0.05) .07 (0.03) .10 (0.05) .07 (0.04)
Positive sentiment −.15 * (0.05) −.09 (0.03) −.14 * (0.05) −.03 (0.03) −.08 (0.05) −.04 (0.04)
Anticipation −.10 (0.04) −.10 * (0.03) .03 (0.04) .12 *** (0.03) −.11 (0.04) −.10 (0.04)
Trust −.05 (0.04) −.09 * (0.03) .09 (0.04) .13 *** (0.03) −.04 (0.04) −.05 (0.04)
Depression and Stress Lexica (Guntuku et al., 2017)
Depression .19 ** (0.05) .07 (0.03) .25 *** (0.05) .12 ** (0.03) .13 * (0.05) .07 (0.05)
Stress .15 * (0.05) .07 (0.03) .15 * (0.05) .03 (0.03) .13 * (0.05) .08 (0.04)
Open in a new tab

All p‐values are Benjamini–Hochberg corrected. For each outcome (PHQ‐8, GAD‐7, or SPIN) “Baseline Model” reflects the association between a given sentiment category and the outcome of interest the following week, controlling for age and gender, from multilevel regression models including the random effect of person. The column “Controlling for…” reflects the association between a given sentiment category and the outcome of interest the following week, controlling for age, gender, and the other two symptom scales, from multilevel regression models including the random effect of person. Note that LIWC separates the categories “I” and “We,” so these pronouns were extracted separately; the LIWC “I” category also includes “me” and “my,” as well as other variations on these pronouns (e.g., “I've” and “myself”).

Abbreviations: GAD‐7, Generalized Anxiety Disorder 7‐item scale; LIWC, Linguistic Inquiry and Word Count; PHQ‐8, Patient Health Questionnaire‐8; SPIN, Social Phobia Inventory.

*

p < .05

**

p < .01

***

p < .001.

3.2.1. LIWC linguistic dimensions

In line with hypotheses, all three symptom scales were linked with greater use of personal pronouns in the preceding three weeks (β: .13–.16, p: .006–.035). PHQ‐8 and GAD‐7 were specifically linked with the use of “I” (PHQ‐8: β = .13, p = .039; GAD‐7 β = .14, p = .019), whereas SPIN was uniquely linked with the use of third‐person pronouns (she/he; β = .17, p = .007). In terms of additional function words, all three symptom scales were associated with greater use of negations (β: .14–.17, p: .004–.016), and reduced use of adverbs was positively associated with subsequent PHQ‐8 (β = .15, p = .024).

3.2.2. LIWC psychological processes

Regarding affective processes, greater incidences of all negative emotion word subcategories (anxiety, anger, and sadness) were predictive of all three symptom scales the following week (anxiety β: .12–.17, p: .002–.035; anger β: .14–.19, p: <.001–.013; sadness β: .14–.21, p: <.001–.019). In terms of social processes, male references were linked with higher GAD‐7 symptoms (β = .13, p = .018), and female references were linked with higher SPIN symptoms (β = .14, p = .033). For the cognitive processes category, higher instances of differentiation words were associated with higher levels of all three symptoms the subsequent week (β: .14–.18, p: .004–.025), and higher instances of discrepancy words were specifically associated with subsequent GAD‐7 symptoms (β = .14, p = .036). Contrary to hypotheses, the associations between tentativeness and symptoms were nonsignificant after correction for multiple comparisons.

We also saw several significant links of symptoms with preceding language about perceptual and biological processes. A greater incidence of biological process words was predictive of all three symptoms the following week (body [e.g., cheek, hands, spit] β: .12–.13, p: .018–.030; sexual [e.g., horny, love, incest] β: .11–.16, p: .001–.035). GAD‐7 symptoms were additionally linked with greater instances of health‐related (e.g., clinic, flu, pill; β = .11, p = .043) and hearing‐related (e.g., listen, hearing; β = .13, p = .017) words. For the drives categories, greater use of risk‐related words was positively associated with subsequent PHQ‐8 (β = .12, p = .035) and GAD‐7 (β = .14, p = .007).

There were also several significant associations of informal language categories and subsequent symptoms. All three symptom scales were linked with higher use of swear words (β: .12–.16, p: .001–.017). Greater use of netspeak was associated with later PHQ‐8 (β = .16, p = .018) and GAD‐7 (β = .19, p = .002) symptoms. The use of filler words was linked with subsequent PHQ‐8 symptoms (β = .14, p = .015).

While we saw initial support for our hypothesis around depression symptoms and the past language category with uncorrected analyses, associations between temporal (past/present) language categories and symptoms became nonsignificant after correction for multiple comparisons.

3.2.3. NRC emotion lexicon sentiment

All three symptoms were linked with greater negative sentiment (β: −.20 to −.21, p: <.001–.001) and a bias toward negative sentiment relative to positive sentiment (β: −.17 to −.22, p: <.001–.004). Higher use of disgust words was associated with subsequent GAD‐7 (β = .12, p = .026) and SPIN (β = .15, p = .008), whereas lower use of fear words was associated with subsequent PHQ‐8 (β = −.14, p = .021) and GAD‐7 (β = −.12, p = .039). The sadness category was uniquely linked with GAD‐7 symptoms (β = .14, p = .011). In terms of positive sentiment, we saw specific negative associations with PHQ‐8 (β = −.15, p = .014) and GAD‐7 (β = −.14, p = .014), but not SPIN.

3.2.4. Depression and stress lexica

All three symptom categories were significantly linked with greater scores on the depression (β: .13–.25, p: <.001–.040) and stress (β: .13–.15, p: .013–.041) categories.

3.3. Aim 2

Results from the second set of models indicated specific associations between sentiment and a given set of symptoms after controlling for the other two symptom scales (Table 2).

3.3.1. LIWC linguistic dimensions

After controlling for the other two symptom scales, the only remaining significant association between a pronoun category and symptoms was between third‐person pronouns (she/he) and subsequent SPIN (β = .14, p = .033).

3.3.2. LIWC psychological processes

Two unique associations between symptoms and affective processes remained after controlling for the other two symptom scales: anger was linked with subsequent SPIN (β = .14, p = .009), and sadness was linked with subsequent PHQ‐8 (β = .11, p = .027); all other associations became nonsignificant. Although specific associations between social processes (female/male categories) and symptoms all became nonsignificant after controlling for other symptoms, GAD‐7 remained linked with higher social references overall (β = .13, p < .001), and PHQ‐8 became negatively linked with social processes overall (β = −.09, p = .042). For the cognitive processes category, higher instances of differentiation words remained significantly associated with PHQ‐8 after controlling other symptoms (β = .10, p = .042), but were no longer associated with GAD‐7 or SPIN. The link between discrepancies and GAD‐7 symptoms became nonsignificant.

After controlling for other symptoms, use of sexual words was significantly and uniquely associated with subsequent SPIN (β = .12, p = .031), and remaining biological and perceptual process associations became nonsignificant. Regarding the drives categories, the links between greater use of risk‐related words and PHQ‐8/GAD‐7 became nonsignificant after controlling for other symptoms; however, affiliation words were negatively linked with subsequent PHQ‐8 (β = −.09, p = .033) and positively linked with subsequent GAD‐7 (β = .13, p < .001).

Two specific links between informal language categories and subsequent symptoms remained after controlling for other symptoms. Filler words remained associated with subsequent PHQ‐8 (β = .09, p = .042), and swear words remained associated with SPIN (β = .12, p = .033).

3.3.3. NRC emotion lexicon sentiment

After controlling for other symptoms, SPIN was uniquely linked with greater negative sentiment (β = −.13, p = .038) and anger (β = .13, p = .033), and PHQ‐8 was uniquely linked with a bias toward negative sentiment relative to positive sentiment (β = −.11, p = .033). There was a dissociation of PHQ‐8 and GAD‐7 with respect to positive sentiment: PHQ‐8 was negatively linked with anticipation (β = −.10, p = .027) and trust (β = −.09, p = .033), whereas GAD‐7 was positively linked with anticipation (β = .12, p < .001) and trust (β = .13, p < .001).

3.3.4. Depression and stress lexica

After controlling for other symptoms, only GAD‐7 remained significantly associated with depression words (β = .12, p = .002); all associations with stress became nonsignificant.

3.4. Exploratory aim

When language features were added to the model predicting PHQ‐8 from other symptom features (GAD‐7 and SPIN), the model fit significantly improved (X 2 (100) = 126.88; p = .036), indicating an improved ability to predict PHQ‐8. Similarly, the model with SPIN and PHQ‐8 as predictors of GAD‐7 was significantly improved by the addition of language features into the model (X 2 (100) = 131.08; p = .020). Conversely, language features did not significantly improve the prediction of SPIN beyond the effect of the other two symptom scales alone (X 2 (100) = 118.89; p = .096).

4. DISCUSSION

The present study examined passively sensed language in text messages as a potential indicator of general affective psychopathology as well as an indicator of disorder‐specific vulnerability. Our results replicate findings from cross‐sectional, social media‐based studies indicating that linguistic features convey common and disorder‐specific risks of affective disorder symptoms (Edwards & Holtzman, 2017; Eichstaedt et al., 2018; Tackman et al., 2019). We extend this literature in several ways, most notably by leveraging text messages, a novel source of data involving private and directed communications, and by testing prospective associations between language use and symptom severity.

We found that linguistic features such as personal pronoun use, cognitive processes, and negative emotion words conveyed common risk for affective symptom severity. These results are consistent with prior literature demonstrating first‐person personal pronoun use as one of the most robust linguistic indicators of affective symptoms (Edwards & Holtzman, 2017), as well as associations between cognitive process‐related words and negative emotion‐related words as strong indicators of depression (Eichstaedt et al., 2018). Importantly, we found that biological process‐related words (e.g., eat, blood, and pain) were also associated with common risk. Biological processes are a superset of sexual words, which have substantial overlap with swear words that have previously been associated with depressive symptoms (De Choudhury et al., 2013; Eichstaedt et al., 2018; Liu et al., 2022), and thus it is possible that the association of the superset with some common risk of affective symptoms is driven to some extent by the sexual subset. Importantly, our results on common risk extend prior studies, which have typically used cross‐sectional methods, by showing that certain sentiment use prospectively predicts general (i.e., disorder nonspecific) risk of affective symptoms.

Our results also highlight disorder‐specific sentiment patterns. Increased use of anticipation, trust, social process, and affiliation words were associated with lower subsequent depressive symptom severity after controlling for generalized and social anxiety symptoms. The negative relationships of trust, social process, and affiliation words with next‐week depression symptom severity align with the notion that increased sociality, more active social ties, and closer social bonds are generally found to be protective against depressive symptoms (Santini et al., 2015). The finding that increased use of anticipation words was predictive of lower depressive symptoms the next week likely reflects an increase in motivation and drive, or the opposite of certain central depressive symptoms (Fried et al., 2016), which may reflect a reduction in drive state (Nusslock & Alloy, 2017).

Interestingly, we found these same linguistic features to be significantly, but positively, associated with next‐week generalized anxiety disorder symptoms when controlling for the effects of depressive and social anxiety symptoms. While increased use of anticipation words in the context of depression likely represents increased drive and motivation (Nusslock & Alloy, 2017), in the context of generalized anxiety, arousal is frequently experienced as increased anxiety (Fisher et al., 2010), in line with the tripartite model. For individuals who experience generalized anxiety disorder symptoms, increased arousal can be associated with negative cognitive interpretations of symptoms or situations (Hallion & Ruscio, 2011). Thus, while an individual with depression may experience arousal as excitement (a pleasant change from the low arousal state that characterizes depression), the individual with anxiety may experience arousal as fear, an unpleasant worsening of symptoms (Joiner et al., 1999). Similarly, increased use of affiliation, trust, and social process words in people with generalized anxiety symptoms could signal increased social engagement, which—paradoxically—may exacerbate anxiety symptoms if individuals negatively appraise this increased social contact. At the same time, we cannot rule out the possibility that the associations of depression and anxiety symptoms with the affiliation, anticipation, and trust lexica are suppression effects, as they emerged only after controlling for associated symptoms.

Our results also indicated that the use of anger, sexual, and swearing language was uniquely associated with greater next‐week social anxiety symptom severity. Of note, sexual and swear lexica have significant overlap and are dominated by profanities and hence can be indicative of anger (Liu et al., 2022). The finding that these lexica are associated with increased social anxiety symptoms is consistent with the literature on anger among people with social anxiety disorder (Versella et al., 2016). Prior work hypothesizes that expressions of anger among individuals with social anxiety may be a functional mechanism to avoid rejection: by rejecting others, one effectively insulates themself from the core social anxiety fear of rejection or negative evaluation (Leary et al., 2006). However, prior research suggests that while individuals with social anxiety disorder experience greater anger (Versella et al., 2016), they are also likely to suppress this anger. It may be that within private communications, such as text messages, anger is more readily expressed among individuals with social anxiety disorder symptoms; future research could test this question directly.

Results have implications for intervention development. While digital mental health treatments are efficacious for myriad mental health conditions (Firth et al., 2017; Lattie et al., 2019; Moshe et al., 2021), an important step in advancing such interventions is tailoring content to user subgroups based on symptom profiles. Tailoring is currently achieved via user input, but there is growing promise for the automatic delivery of tailored content to users based on individual context in the form of just‐in‐time‐adaptive‐interventions (JITAIs). JITAIs rely on networked sensors in smartphones to detect low‐level behavioral and smartphone use patterns to “sense” real‐world actions or behaviors that are correlated with both individualized and higher‐order risk factors such as psychopathology symptom worsening or improvement. Early implementations of JITAIs are promising (Teepe et al., 2021); however, the complexity and heterogeneity of mental health symptoms presents a barrier to applying JITAIs that are sufficiently specific to the treatment mechanisms they target while also accommodating the range of behaviors that can be sensed and leveraged to customize content based on individual circumstances. This paper represents a first step to disentangling disorder‐specific passively sensed signals, contributing an improved understanding of shared and unique passively sensed language markers of depression, generalized anxiety, and social anxiety symptoms that can help narrow the range of potential treatment approaches and targets that will be most effective in the moment. JITAIs could even focus on language directly, as prior research suggests that linguistic changes (namely, reductions in the use of “I” and present‐tense verbs) track symptom improvement in therapy, potentially reflecting attempts to distance oneself from negative stimuli in service of emotion regulation (Nook et al., 2022).

This study has several strengths and limitations that point to future directions for research. Strengths of this study included the prospective, longitudinal approach and use of multilevel models to parse within‐ and between‐person associations between language and subsequent symptom severity. While our findings generally aligned with prior associative findings between linguistic features and psychopathology from the social media literature, our results had modest effect sizes. This may reflect the challenge of isolating specific effects associated with a single disorder in light of the high comorbidity between depression, generalized anxiety disorder, and social anxiety disorder, such that little remaining variance is explained after controlling for other symptoms. Additionally, to increase robustness, we chose to exclude data from periods with fewer than 50 outgoing text messages; it is conceivable that sending a reduced number of text messages could reflect clinically relevant information (e.g., anhedonia; social isolation; and avoidance), suggesting a topic for future study. Further, the context of a text message is important. We were underpowered to test the ways in which a person's relationship to the recipient of a text may moderate associations between language and psychopathology (e.g., if associations between social anxiety and the use of anger language, or between depression and the use of positive emotion language, vary according to whether someone is corresponding with a close or more distant contact). Another consideration is that we used the PHQ‐8 (and not an anxiety measure) as a means of oversampling for people with affective pathologies; while this resulted in a high proportion of people over the clinical cutoffs for both depression and anxiety symptoms, a conservative interpretation of our anxiety findings would be that they represent the relationship between language and anxiety symptoms in a sample recruited on the basis of depressive symptoms. Moreover, our study involved a predominantly White, non‐Hispanic sample, raising the question of whether similar language features would predict psychopathology symptoms across racial and ethnic groups. Finally, to preserve participant privacy, we used on‐device processing of text messages and only transmitted aggregate LIWC sentiment scores, rather than raw text messages, reducing the signal that can be extracted from the messages. For example, sexual and biological process‐related words contain many different types of words that serve a variety of functions including as expletives, to convey medical or physical conditions, and/or to convey anger, excitement, or emphasis.

Overall, our findings suggest both common and disorder‐specific linguistic signals that correspond with symptom worsening. These linguistic markers, in combination with other data such as networked sensors and regular self‐reports, may be useful for just‐in‐time interventions (Liu et al., 2022; Meyerhoff et al., 2021) that can be personalized to identify periods of risk (Kaurin et al., 2022). Future idiographic models could leverage the message‐level data and symptom specificity information we have explored here, along with other disorder‐specific passively‐sensed risk signals, to deliver key intervention components depending on individual risk status.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

ETHICS STATEMENT

All study procedures protocols and procedures were approved by Northwestern University's Institutional Review Board.

Supporting information

Supporting information.

Click here for additional data file. (32.5KB, docx)

ACKNOWLEDGMENTS

This study was funded by the National Institute of Mental Health (NIMH) R01 MH111610, R34 MH124960, and T32 MH115882 grants, and by the Intramural Research Program of the National Institutes of Health (NIH), National Institute on Drug Abuse (NIDA).

Stamatis, C. A. , Meyerhoff, J. , Liu, T. , Sherman, G. , Wang, H. , Liu, T. , Curtis, B. , Ungar, L. H. , & Mohr, D. C.  (2022). Prospective associations of text‐message‐based sentiment with symptoms of depression, generalized anxiety, and social anxiety. Depression and Anxiety, 39, 794–804. 10.1002/da.23286

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request. All code is publicly available on RPubs. Data are available upon request.

REFERENCES

  1. Al‐Mosaiwi, M. , & Johnstone, T. (2018). In an absolute state: Elevated use of absolutist words is a marker specific to anxiety, depression, and suicidal ideation. Clinical Psychological Science, 6(4), 529–542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Anstiss, D. , & Davies, A. (2015). ‘Reach Out, Rise Up’: The efficacy of text messaging in an intervention package for anxiety and depression severity in young people. Children and Youth Services Review, 58, 99–103. [Google Scholar]
  3. Audacious Software . (2018). Passive Data Kit. Published online. https://passivedatakit.org/
  4. Benjamini, Y. , & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57(1), 289–300. [Google Scholar]
  5. Brockmeyer, T. , Zimmermann, J. , Kulessa, D. , Hautzinger, M. , Bents, H. , Friederich, H. C. , Herzog, W. , & Backenstrass, M. (2015). Me, myself, and I: Self‐referent word use as an indicator of self‐focused attention in relation to depression and anxiety. Frontiers in Psychology, 6, 1564. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chancellor, S. , & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on social media: A critical review. NPJ Digital Medicine, 3(1):43. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. De Choudhury, M. , Gamon, M. , Counts, S. , & Horvitz, E. (2013). Predicting depression via social media. Proceedings of the International AAAI Conference on Web and Social Media, 7(1), 128–137. [Google Scholar]
  8. Connor, K. M. , Davidson, J. R. T. , Churchill, L. E. , Sherwood, A. , Weisler, R. H. , & Foa, E. (2000). Psychometric properties of the Social Phobia Inventory (SPIN): New self‐rating scale. British Journal of Psychiatry, 176(4), 379–386. [DOI] [PubMed] [Google Scholar]
  9. Edwards, T. , & Holtzman, N. S. (2017). A meta‐analysis of correlations between depression and first person singular pronoun use. Journal of Research in Personality, 68, 63–68. [Google Scholar]
  10. Eichstaedt, J. C. , Smith, R. J. , Merchant, R. M. , Ungar, L. H. , Crutchley, P. , Preoţiuc‐Pietro, D. , Asch, D. A. , & Schwartz, H. A. (2018). Facebook language predicts depression in medical records. Proceedings of the National Academy of Sciences, 115(44), 11203–11208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Firth, J. , Torous, J. , Nicholas, J. , Carney, R. , Rosenbaum, S. , & Sarris, J. (2017). Can smartphone mental health interventions reduce symptoms of anxiety? A meta‐analysis of randomized controlled trials. Journal of Affective Disorders, 218, 15–22. [DOI] [PubMed] [Google Scholar]
  12. Fisher, A. J. , Granger, D. A. , & Newman, M. G. (2010). Sympathetic arousal moderates self‐reported physiological arousal symptoms at baseline and physiological flexibility in response to a stressor in generalized anxiety disorder. Biological Psychology, 83(3), 191–200. [DOI] [PubMed] [Google Scholar]
  13. Fried, E. I. , Epskamp, S. , Nesse, R. M. , Tuerlinckx, F. , & Borsboom, D. (2016). What are'good'depression symptoms? Comparing the centrality of DSM and non‐DSM symptoms of depression in a network analysis. Journal of Affective Disorders, 189, 314–320. [DOI] [PubMed] [Google Scholar]
  14. Geyer, E. C. , Fua, K. C. , Daniel, K. E. , Chow, P. I. , Bonelli, W. , Huang, Y. , Barnes, L. E. , & Teachman, B. A. (2018). I did OK, but did I like it? Using ecological momentary assessment to examine perceptions of social interactions associated with severity of social anxiety and depression. Behavior Therapy, 49(6), 866–880. [DOI] [PubMed] [Google Scholar]
  15. Guntuku, S. C. , Schneider, R. , Pelullo, A. , Young, J. , Wong, V. , Ungar, L. , Polsky, D. , Volpp, K. G. , & Merchant, R. (2019). Studying expressions of loneliness in individuals using twitter: An observational study. BMJ Open, 9(11), e030355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Guntuku, S. C. , Yaden, D. B. , Kern, M. L. , Ungar, L. H. , & Eichstaedt, J. C. (2017). Detecting depression and mental illness on social media: An integrative review. Current Opinion in Behavioral Sciences, 18, 43–49. [Google Scholar]
  17. Hallion, L. S. , & Ruscio, A. M. (2011). A meta‐analysis of the effect of cognitive bias modification on anxiety and depression. Psychological Bulletin, 137(6), 940–958. [DOI] [PubMed] [Google Scholar]
  18. Hernandez‐Ramos, R. , Altszyler, E. , Figueroa, C. A. , Avila‐Garcia, P. , & Aguilera, A. (2022). Linguistic analysis of Latinx patients' responses to a text messaging adjunct during cognitive behavioral therapy for depression. Behaviour Research and Therapy, 150, 104027. [DOI] [PubMed] [Google Scholar]
  19. Homan, S. , Gabi, M. , Klee, N. , Bachmann, S. , Moser, A. M. , Duri', M. , Michel, S. , Bertram, A. M. , Maatz, A. , Seiler, G. , Stark, E. , & Kleim, B. (2022). Linguistic features of suicidal thoughts and behaviors: A systematic review. Clinical Psychology Review, 95, 102161. [DOI] [PubMed] [Google Scholar]
  20. Joiner, T. E. , Steer, R. A. , Beck, A. T. , Schmidt, N. B. , Rudd, M. D. , & Catanzaro, S. J. (1999). Physiological hyperarousal: Construct validity of a central aspect of the tripartite model of depression and anxiety. Journal of Abnormal Psychology, 108(2), 290–298. [DOI] [PubMed] [Google Scholar]
  21. Kaurin, A. , Dombrovski, A. Y. , Hallquist, M. N. , & Wright, A. G. C. (2022). Integrating a functional view on suicide risk into idiographic statistical models. Behaviour Research and Therapy, 150, 104012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kroenke, K. , Strine, T. W. , Spitzer, R. L. , Williams, J. B. W. , Berry, J. T. , & Mokdad, A. H. (2009). The PHQ‐8 as a measure of current depression in the general population. Journal of Affective Disorders, 114(1–3), 163–173. 10.1016/j.jad.2008.06.026 [DOI] [PubMed] [Google Scholar]
  23. Kuznetsova, A. , Brockhoff, P. B. , & Christensen, R. H. B. (2017). Lmertest package: Tests in linear mixed effects models. Journal of Statistical Software, 82, 1–26. [Google Scholar]
  24. Lattie, E. G. , Adkins, E. C. , Winquist, N. , Stiles‐Shields, C. , Wafford, Q. E. , & Graham, A. K. (2019). Digital mental health interventions for depression, anxiety, and enhancement of psychological well‐being among college students: Systematic review. Journal of Medical Internet Research, 21(7), e12869. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Leary, M. R. , Twenge, J. M. , & Quinlivan, E. (2006). Interpersonal rejection as a determinant of anger and aggression. Personality and Social Psychology Review, 10(2), 111–132. [DOI] [PubMed] [Google Scholar]
  26. Liu, T. , Meyerhoff, J. , Eichstaedt, J. C. , Karr, C. J. , Kaiser, S. M. , Kording, K. P. , Mohr, D. C. , & Ungar, L. H. (2022). The relationship between text message sentiment and self‐reported depression. Journal of Affective Disorders, 302, 7–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Mairesse, F. , Walker, M. A. , Mehl, M. R. , & Moore, R. K. (2007). Using linguistic cues for the automatic recognition of personality in conversation and text. Journal of Artificial Intelligence Research, 30, 457–500. [Google Scholar]
  28. Meyerhoff, J. , Liu, T. , Kording, K. P. , Ungar, L. H. , Kaiser, S. M. , Karr, C. J. , & Mohr, D. C. (2021). Evaluation of changes in depression, anxiety, and social anxiety using smartphone sensor features: Longitudinal cohort study. Journal of Medical Internet Research, 23(9), e22844. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Moreno‐Agostino, D. , Wu, Y. T. , Daskalopoulou, C. , Hasan, M. T. , Huisman, M. , & Prina, M. (2021). Global trends in the prevalence and incidence of depression: A systematic review and meta‐analysis. Journal of Affective Disorders, 281, 235–243. [DOI] [PubMed] [Google Scholar]
  30. Moshe, I. , Terhorst, Y. , Philippi, P. , Domhardt, M. , Cuijpers, P. , Cristea, I. , Pulkki‐Råback, L. , Baumeister, H. , & Sander, L. B. (2021). Digital interventions for the treatment of depression: A meta‐analytic review. Psychological Bulletin, 147(8), 749–786. [DOI] [PubMed] [Google Scholar]
  31. Nook, E. C. , Hull, T. D. , Nock, M. K. , & Somerville, L. H. (2022). Linguistic measures of psychological distance track symptom levels and treatment outcomes in a large set of psychotherapy transcripts. Proceedings of the National Academy of Sciences, 119(13), e2114737119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Nusslock, R. , & Alloy, L. B. (2017). Reward processing and mood‐related symptoms: An RDoC and translational neuroscience perspective. Journal of Affective Disorders, 216, 3–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. O'Dea, B. , Boonstra, T. W. , Larsen, M. E. , Nguyen, T. , Venkatesh, S. , & Christensen, H. (2021). The relationship between linguistic expression in blog content and symptoms of depression, anxiety, and suicidal thoughts: A longitudinal study. PLoS One, 16(5), e0251787. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Pennebaker, J. W. , Boyd, R. L. , Jordan, K. , & Blackburn, K. (2015). The Development and Psychometric Properties of LIWC2015.
  35. Preoţiuc‐Pietro, D. , Liu, Y. , Hopkins, D. , & Ungar, L. (2017). Beyond binary labels: Political ideology prediction of Twitter users (pp. 729–740). Association for Computational Linguistics. [Google Scholar]
  36. Santini, Z. I. , Koyanagi, A. , Tyrovolas, S. , Mason, C. , & Haro, J. M. (2015). The association between social relationships and depression: A systematic review. Journal of Affective Disorders, 175, 53–65. [DOI] [PubMed] [Google Scholar]
  37. Schwartz, H. A. , Eichstaedt, J. , Kern, M. , Park, G. , Sap, M. , Stillwell, D. , Kosinski, M. , & Ungar, L. (2014). Towards assessing changes in degree of depression through facebook (pp. 118–125). Association for Computational Linguistics. [Google Scholar]
  38. Senanayake, B. , Wickramasinghe, S. I. , Chatfield, M. D. , Hansen, J. , Edirippulige, S. , & Smith, A. C. (2019). Effectiveness of text messaging interventions for the management of depression: A systematic review and meta‐analysis. Journal of Telemedicine and Telecare, 25(9), 513–523. [DOI] [PubMed] [Google Scholar]
  39. Smith, A. (2015). U.S. smartphone use in 2015. Pew Research Center.
  40. Spitzer, R. L. , Kroenke, K. , Williams, J. B. W. , & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD‐7. Archives of Internal Medicine, 166(10), 1092–1097. [DOI] [PubMed] [Google Scholar]
  41. Tackman, A. M. , Sbarra, D. A. , Carey, A. L. , Donnellan, M. B. , Horn, A. B. , Holtzman, N. S. , Edwards, T. S. , Pennebaker, J. W. , & Mehl, M. R. (2019). Depression, negative emotionality, and self‐referential language: A multi‐lab, multi‐measure, and multi‐language‐task research synthesis. Journal of Personality and Social Psychology, 116(5), 817–834. [DOI] [PubMed] [Google Scholar]
  42. Teepe, G. W. , Da Fonseca, A. , Kleim, B. , Jacobson, N. C. , Salamanca Sanabria, A. , Tudor Car, L. , Fleisch, E. , & Kowatsch, T. (2021). Just‐in‐time adaptive mechanisms of popular mobile apps for individuals with depression: Systematic app search and literature review. Journal of Medical Internet Research, 23(9), e29412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Versella, M. V. , Piccirillo, M. L. , Potter, C. M. , Olino, T. M. , & Heimberg, R. G. (2016). Anger profiles in social anxiety disorder. Journal of Anxiety Disorders, 37, 21–29. [DOI] [PubMed] [Google Scholar]
  44. Watkins, E. R. , & Nolen‐Hoeksema, S. (2014). A habit‐goal framework of depressive rumination. Journal of Abnormal Psychology, 123(1), 24–34. spit. [DOI] [PubMed] [Google Scholar]
  45. Watkins, E. R. , & Roberts, H. (2020). Reflecting on rumination: Consequences, causes, mechanisms and treatment of rumination. Behaviour Research and Therapy, 127, 103573. [DOI] [PubMed] [Google Scholar]
  46. Wilhelm, S. , Weingarden, H. , Ladis, I. , Braddick, V. , Shin, J. , & Jacobson, N. C. (2020). Cognitive‐behavioral therapy in the digital age: Presidential address. Behavior Therapy, 51(1), 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Yaden, D. B. , Eichstaedt, J. C. , Kern, M. L. , Smith, L. K. , Buffone, A. , Stillwell, D. J. , Kosinski, M. , Ungar, L. H. , Seligman, M. E. P. , & Schwartz, H. A. (2018). The language of religious affiliation: Social, emotional, and cognitive differences. Social Psychological and Personality Science, 9(4), 444–452. [Google Scholar]
  48. Yang, X. , Fang, Y. , Chen, H. , Zhang, T. , Yin, X. , Man, J. , Yang, L. , & Lu, M. (2021). Global, regional and national burden of anxiety disorders from 1990 to 2019: Results from the Global Burden of Disease Study 2019. Epidemiology and Psychiatric Sciences, 30, e36. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supporting information.

Click here for additional data file. (32.5KB, docx)

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request. All code is publicly available on RPubs. Data are available upon request.

Articles from Depression and Anxiety are provided here courtesy of Wiley

RESOURCES