Acute suicidal ideation in context: highlighting sentiment-based markers through the diary entries of a clinically depressed sample

Abstract

Background

Despite major strides in conceptualizing and modeling the multifaceted nature of suicidal thought and behavior (STB) over the past few decades, the overall predictability of STB has not improved. This may be partly due to the dynamic nature of suicidal ideation (SI), which often fluctuates over hours, yet is largely overlooked in studies. Bolstered by the application and promise of natural language processing (NLP) across the mental health field, efforts toward richer operationalization of acute SI may include analyses on written data that occur alongside changes in SI, thus offering a better understanding of STB as it unfolds.

Methods

Ecological momentary assessment (EMA) data from 268 participants with major depressive disorder (MDD) were utilized to investigate acute changes in SI. Data consisted of thrice-daily SI severity scores measured through self-report responses to item 9 of the Patient Health Questionnaire mobile version (MPHQ-9) as well as free-form diary text. Using difference scores and probability of acute change thresholds, eleven acute SI phase trajectory types were defined to label change in SI over three consecutive EMAs. In total, 5,938 acute SI trajectories were paired with the temporally centered diary entries. The Sentiment Analysis and Cognition Engine (SEANCE) tool was applied to quantify the written content of each diary entry across eight established lexica. Entry results were grouped based on phase trajectory type, and the Kruskal-Wallis test was employed with post-hoc multiple hypothesis correction to statistically compare SEANCE features between all group pairs.

Results

There were 131 statistically significant (adjusted p-value < 0.05) pairwise differences between acute SI phase trajectory groups, implicating 31 NLP features. Consistent with the literature, results highlighted qualities of writing that are generally associated with heightened SI, including personal pronoun usage, passivity, and negative valence. Patterns of significance also uncovered novel contextual nuance in terms of how characteristics such as verbosity, hostility, anger, and pleasantness present in relation to SI over short change trajectories.

Conclusions

This work provides an accessible exploratory framework that capitalizes on the benefits of dense EMA sampling and NLP to profile and quantify acute SI trajectories. The use of the MPHQ’s item 9 to quantify SI is an important limitation as it is designed to also capture precursory SI, passive SI, and SI-adjacent behaviors, potentially overestimating the SI expressed by participants. Nonetheless, future research should continue to focus on short timeframes as there are likely important signals and interpretative nuances to SI expression that have yet to be fully detailed.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12888-025-07108-4.

Background

With 135 suicide deaths per day on average and approximately 1.6 million suicide attempts in 2022 alone, suicide is currently the eleventh leading cause of death in the United States [1]. Despite these numbers, 95% of Americans believe that suicide can be prevented and 75% believe that those who die by suicide provide signs of their intent beforehand [2]. Since Durkheim’s seminal study of suicide in 1897 [3], countless theories and models for suicidal thought and behavior (STB) have been proposed, incorporating biological [4], sociological [5], and psychological [6] mechanisms. In recent years, ideation-to-action-based frameworks have become among the most comprehensive and practical, drawing from over a century of ideas and empirical data [7]. These most prominently include the Interpersonal-Psychological Theory of Suicide (IPTS; [8]), the Integrated Motivational-Volitional model (IMV; [9]), and the Three-Step Theory of suicide (3ST; [10]) which have served as the basis for investigations that use cutting edge quantitative and analytical techniques to model STB [11–16].

Despite these theories providing utility to understand STB within research and clinical contexts [17–21], a recent meta-analysis has underlined an overall weaker-than-expected ability to detect and predict STB [22]. Importantly, this extends beyond automated models, where clinician assessments have been found to be slightly better than chance at predicting suicide attempts in at-risk individuals [23]. One reason for this finding may be due to the highly dynamic and heterogeneous nature of suicidal ideation (SI) [11]. While the field of suicidology acknowledges the temporal complexity of STB presentation, Fluid Vulnerability Theory (FVT; [24]) is the only theoretical model of STB that explicitly supports the conceptualization of SI as a dynamic change process with fluctuations over short time scales (i.e., hours or days). FVT also provides a strong precedent for study designs with shorter sampling intervals to be more aligned with the evolving phenomenology of STB. By extension, FVT highlights why so many longitudinal and sparsely sampled investigations have been unsuccessful in their ability to predict STB risk [25]. Indeed, research and modeling of SI is largely cross-sectional or consists of measures over coarse-grained periods of months or years, and it is only within the past decade that works have begun to focus on acute SI trajectories [11, 26]. Such efforts have documented within-person variability in SI over hours [11] and days [27], and have linked transitory SI with key constructs from IPTS, IMV, and 3ST [11]. Moreover, within clinical contexts, temporal patterns and acute fluctuations in SI presentation have been shown to better characterize vulnerability to recurrent attempts among those receiving outpatient treatment [28].

Notions of STB as a constantly evolving system motivate more precise work aimed at characterizing the array of different risk trajectories and qualitative states that may define acute SI as it unfolds. For example, an individual at the height of an SI escalation over the past several days may express themselves differently from an individual who has rapidly transitioned into a severe ideation state within the past few hours. Despite these two hypothetical individuals expressing high momentary SI, they may possess different risk profiles. In other words, an identical set of observable factors may ultimately lead to different capabilities or patterns in these factors to quantify risk—a result of capturing SI at phenomenologically and contextually different stages along its trajectory. Much of the literature supports this, as efforts in testing the utility of certain risk factors as predictive tools, including time-invariant operationalizations of SI, have been frequently met with inconsistent or contradictory results [29]. Where one behavioral proxy may reliably flag for short-lived bouts of heightened SI, another may be no better than chance in this regard, yet excel at predicting an impending decline in SI from a heightened and sustained state. A 50-year meta-analysis on STB modeling [25] offers insightful direction related to this—the application of advanced technology to more densely sample SI in real time will lead to a more complete understanding of how SI presents within finer temporal intervals.

The rates of smartphone ownership have sharply risen over the past few years with 90% of the United States population owning a smartphone [30]. As a result, smartphones have become a burgeoning method for the unobtrusive, dense, and ecologically valid collection of behavioral and mental health-related data, including SI and STB [11–13]. Smartphones allow for data collection within the context of day-to-day life rather than within the traditional laboratory setting, which tends to offer limited generalizability and temporal granularity. Prior work within the STB space has leveraged ecological momentary assessment (EMA)—the repeated sampling of current behaviors and experiences in real time and within a subject’s natural environment—to study passive SI [31–35], active SI [32–39], suicide desire [37], capability for suicide [40], intent to die by suicide [37, 39], and ability to keep oneself safe or resist the urge to die by suicide [37, 39]. More broadly, the dense and longitudinal nature of EMA data has provided novel insights for highly dynamic psychiatric constructs such as person-specific depression, including person-specific depression symptom profiles [41]. Given the acute nature of SI, further exploration and development of SI markers may benefit from novel and creative operationalizations of EMA-assessed SI that take full advantage of the dense and ecologically valid sampling approach. Doing so may ultimately support useful and novel models for SI which are powered to detect or predict temporally nuanced changes in SI.

One way to enhance the contextual utility of EMA data is to include prompts that invite free-form qualitative responses (e.g., diary entries) alongside more structured, quantitative assessments of state (e.g., SI severity). While both qualitative and quantitative responses are common among EMA-based studies, they are less frequently modeled in combination, especially within suicide research. The further development of, and ease of access to, the vast analytical toolkit of natural language processing (NLP) provides a rich array of means with which to extract and dimensionalize qualitative text for integration into downstream quantitative applications. Accordingly, the use of NLP in mental health research has grown exponentially over the past decade and has been applied in both clinical and non-clinical research [42, 43], including constructs and outcomes within STB [14, 44–48].

While there are many techniques that subsume NLP, sentiment analysis—the assessment of the affective qualities of free-form text—demonstrates strong utility for mental health research. Briefly, there are three overarching approaches to deploying sentiment analysis: (i) lexicon-based, (ii) machine/deep learning, and (iii) hybrid approaches. Where lexicon-based approaches typically rely on manual, expert curation to develop lists of mapped word-value assignments, automated approaches involve the training of machine learning models with labeled or unlabeled text data to automatically recognize key properties of words [49]. While lexicon-based approaches offer greater interpretability, customization, and relative simplicity of deployment/application, automated approaches provide scalability, an ability to detect nuanced patterns, and higher task-specific performance [49]. As the name would suggest, hybrid approaches combine elements of both lexicon-based and automated approaches, such as first preprocessing raw text through lexica to derive features that are then fed through a deep learning model architecture [50]. No approach is superior to another, and choice may depend on the priorities of the research endeavor (e.g., interpretability over performance, exploration over hypothesis testing, or domain independence over domain dependence).

The current work aimed to identify and interpret novel semantic markers and patterns in acute SI by methodologically focusing on the deployment of lexicon-based sentiment analysis, which has a broad suite of resource databases to explore [51, 52]. These include lexica related to emotions and emotional intensity [53, 54], affective norms [55], polarity [56], broad topical content [57], and the flow of communication [58]. Moreover, lexicon-based approaches have been frequently employed within mental health and STB-related research. For example, the Emotion Lexicon (EmoLex) [53] was used to model and predict emotional responses to public events on social media [59]; the Affective Norms for English Words (ANEW) lexicon [55] served to help predict the emergence of depression and post-traumatic stress disorder online [60]; and Valence Aware Dictionary and Sentiment Reasoner (VADER) [52] features have been applied to machine learning-based pipelines to detect suicide risk in Twitter posts [61]. In each of these applications, the models performed well, with areas under the receiver operating characteristic curve (AUC; a metric that reflects a model’s ability to discriminate between two classes and is derived from a plot of the trade-off between the true positive rate and false positive rate) ranging from 0.87 to 0.96.

Given the demonstrated need to develop more temporally appropriate and sensitive models of acute SI, the current work aimed to leverage densely collected EMA and free-text response data. Specifically, this research aimed to statistically explore how written language may be used to detect acute shifts in SI severity. Importantly, this work sought to address the current lack of temporal context within the risk factor literature by first developing a classification scheme for acute SI trajectories based on EMA. At baseline, the aim of the scheme was to provide an objective and easy-to-interpret framework for distinguishing among equivalently or similarly severe momentary SI states. Based on SI measures that reflect transitions among current, immediate past, and immediate future states (i.e., 4–8 h timeframes), this profiling of acute SI served to support the discovery of NLP-based features that (i) are associated with near-future increases or decreases in SI severity, (ii) signal for current heightened SI severity states, and (iii) empirically highlight the importance of understanding the acute context in which emotions co-occur with SI. In this manner, these efforts can be used to guide and support future research that aims to more fully appreciate the dynamics of SI, ultimately working toward improved detection, prediction, and prevention of STB-related outcomes.

Methods

Human subjects protection

The current study received approval from the Committee for the Protection of Human Subjects at Dartmouth College (STUDY00032081). Participants provided written informed consent prior to the collection of information and verbal informed consent prior to the start of the study. This consent procedure was approved by the Committee.

Participant recruitment and screening

Recruitment and screening

Participants across the United States were recruited online via Google Ads and Meta Ads from February 17, 2021 to December 1, 2023. The ads were directed at individuals with depression, and those interested in the study clicked on the ad to complete a screener questionnaire. An initial consent form was completed by 15,507 participants, and 4,625 of these participants completed the screener. Participants who were at least 18 years of age, owned an Android as their primary smartphone device, and self-reported elevated Major Depressive Disorder (MDD) symptoms on the Patient Health Questionnaire-9 (i.e., sum score ≥ 10) were invited to complete a more detailed screening procedure. Conducted on 1,305 participants, the procedure consisted of a more comprehensive battery of assessments and a 20-minute semi-structured screening interview via Zoom which reviewed participant depressive symptoms over the past 30 days. Participants who were still deemed as eligible due to self-report of an MDD-defined depressive episode over the past 30 days were invited to participate in a final clinical interview. This involved the Structured Clinical Interview for DSM-5 (SCID), which was administered via Zoom by either a psychiatrist or a postdoctoral research fellow in clinical psychology. Of the 431 participants interviewed, 312 met criteria for a current MDD episode and did not report a history of (i) mania, (ii) psychoses, or (iii) currently endorse moderate to high acute suicide risk and therefore were invited to participate in the main study.

Moderate to high suicide risk was determined based on self-reported endorsement of items 1 (a wish to be dead) through 5 (planning and intent to carry out a plan) on the screen version of the Columbia-Suicide Severity Rating Scale (C-SSRS) [62]. The C-SSRS was automatically administered as part of the initial screening process to any participant who provided a “non-zero” response to item 9 on the Patient Health Questionnaire-9 (PHQ-9) [63]. Items were presented within the context of the past month. If any participant endorsed item 3 (thoughts about how to attempt), item 4 (thoughts with some intention to act), or item 5, reflecting moderate or high risk, the participant was considered at-risk of indicating active suicidality during the study and was excluded from enrollment. The C-SSRS was developed across multiple institutions and with support from National Institute of Mental Health (NIMH) [62], and its validity in assessing suicide severity and immediacy has been demonstrated across a wide range of settings and populations [64].

Participants

The current investigation serves as a secondary analysis of Tracking Depression—an NIMH-funded research effort to monitor and study depression across 90 days [41, 65]. At the conclusion of recruitment (see above), 300 individuals met criteria for current MDD and were included in the 90-day study. Of these, N = 268 participants had sufficient ecological momentary assessment (EMA) and free-text diary entry data available for use in the current analysis (see “Data preprocessing and subselection” below). The overall research effort sought to recruit a nationally representative sample of individuals with MDD. Accordingly, participants in the present investigation were primarily White (n = 212; 79.1%), non-Hispanic (n = 236; 88.1%), and women (n = 225; 84%). In addition, 99.3% of participants (n = 266) endorsed some level of SI during the 90-day study. See Table 1 for additional information on cohort sociodemographics.

Table 1.

Cohort sociodemographic summary

Characteristic	Attribute	mean (sd) or count (%)	Characteristic	Attribute	count (%)
Age	—	40.72 (11.7)	Hispanic / Latino Ethnicity	No	236 (88.1)
Gender	Women	225 (84.0)		Yes	32 (11.9)
	Men	28 (10.4)	Highest Level of Education	Less than high school	2 (0.7)
	Non-binary	11 (4.1)		High school	15 (5.6)
	Other (Prefer to self-describe)	4 (1.5)		Trade/technical school	3 (1.1)
Transgender	No	258 (96.3)		Some college, no degree	64 (23.9)
	Yes	10 (3.7)		Associate degree	23 (8.6)
Sexual Orientation	Heterosexual	177 (66.0)		Bachelor’s degree	88 (32.8)
	Homosexual	14 (5.2)		Master’s degree	60 (22.4)
	Bisexual	35 (13.1)		Doctoral degree	13 (4.9)
	Bicurious	6 (2.2)	Current Student Status	Not a student	220 (82.1)
	Pansexual	18 (6.7)		Part-time	14 (5.2)
	Asexual	13 (4.9)		Full-time	34 (12.7)
	Other (Prefer to self-describe)	5 (1.9)	Income	Less than $20,000	46 (17.2)
Race	White	212 (79.1)		$20,000 to $39,999	49 (18.3)
	Black or African American	18 (6.7)		$40,000 to $59,999	49 (18.3)
	Asian	12 (4.5)		$60,000 to $79,999	39 (14.6)
	American Indian or Alaskan Native	1 (0.4)		$80,000 to $99,999	26 (9.7)
	More than one race	19 (7.1)		$100,000 to $149,999	38 (14.2)
	Other (Prefer to self-describe)	6 (2.2)		$150,000 or more	21 (7.8)

Note. N = 268

Data collection and procedure

Participants meeting eligibility requirements were onboarded on a rolling basis by a research coordinator. The onboarding process included downloading the Android-based MLife app [66]. Following onboarding, participants began the study and completed EMA prompts via the MLife app three times per day for 90 days. This resulted in up to 270 total possible daily EMAs per participant. Each EMA included the PHQ-9, an optional free-text response, as well as questions beyond the scope of the current investigation (e.g., in regards to anxiety and somatic symptoms). Prompts to complete the EMAs began four hours after a participant’s self-reported wake time (e.g., the first prompt of the day would be at 11 AM if the reported wake time was 7 AM). The next two prompts were delivered in four-hour intervals following the first prompt. Additionally, participants received a prompt every Monday morning to complete a weekly assessment, which included the PHQ-9, optional free-text response, and other questions not relevant to the current investigation. On average, it took participants 54.0 s (σ = 70.5) to complete an EMA. Participants were compensated $1 at the end of the study for every completed daily and weekly EMA. Participants additionally received a bonus of $50 for completing at least 90% of EMAs. Evaluation of compliance rates indicated a high level of compliance for participants, with a median of 256 (94.8%) EMAs completed per participant. Of those, 105 (45.4%) of EMAs (on average per participant), included an endorsement of SI. See Table  2 for more detailed statistics on participant EMA compliance.

Table 2.

Statistical summary of EMA data

Overall Cohort Statistics	Count (%)
Total participants	268
Participants with SI reported	266 (99.3)
Total EMAs	62,243
EMAs with SI reported	28,236 (45.4)
Total diary entries	9,586
Diary entries with representative EMA triplicates†	5,938 (61.9)
Per Participant Statistics	Mean [min, max]
EMA count	232.3 [6, 419*]
Diary entry count	35.8 [1, 236]
Percentage of EMAs with diary entry	15.9 [0.3, 86.4]
EMAs with SI reported	105.4 [0, 346]
EMAs with no SI reported	126.9 [0, 305]
Average SI severity	9.4 [0, 95.4]
Maximum SI severity	55.2 [0, 100]
Minimum SI severity	0.9 [0, 40]

Note. SI severity ranged on a sliding scale from 0 to 100. ^†See Data Preprocessing and Subselection within the Data Analytic Plan as well as Fig. 1 below for more information. *While the maximum possible number of EMAs (and free-text responses) for the 90-day study period was 270, participants could continue to provide EMA and free-text response data through the MLife app past this window. Only one participant opted to do so. Moreover, participants could self-initiate EMAs at any time; however, only the most recent and valid EMA completed within each time interval was used to represent that data point (thus, a maximum of three representative EMAs per day)

Measures

Patient Health Questionnaire-9

The PHQ-9 is a nine-item self-report measure that assesses the frequency of depressive symptoms over the past two weeks [63]. For the current study, we utilized a previously developed mobile-friendly version of the PHQ-9 (MPHQ-9) that prompted participants to respond with how they felt over the prior four hours [35, 67]. This version has demonstrated high correlations with the original version (r = 0.84) and good adherence rates (77.78%) [35]. In a recent analysis on the Tracking Depression study data, the MPHQ-9 demonstrated excellent internal consistency (ɑ = 0.91), good longitudinal stability (r = 0.69), and good convergent validity with the traditional PHQ-9 (r = 0.71) and the Inventory of Depression and Anxiety Symptoms – Expanded Version (r = 0.65) [68]. Moreover, it has been used to good effect in other investigations within the larger Tracking Depression study [41, 69–71]. Participants responded to the nine questions, each representing an individual depressive symptom, via a sliding scale from 0 (not at all) to 100 (constantly). To provide participants with more context and to minimize any potential floor or ceiling effects in what constitutes “experiencing a symptom,” participants were asked to reflect on the best and worst they had ever experienced each symptom in their lives when responding to the EMA prompt on frequency over the prior four hours. Importantly, participants did not have access to their prior EMAs and could not refer to their responses earlier in the day or week. The original version of the PHQ-9 has demonstrated consistently good sensitivity, specificity, internal reliability, and sensitivity to change [72, 73].

Free-Text responses

At the end of each EMA, participants were given the option to provide a free-text response to the prompt, “Please describe how you’ve been feeling over the past four hours and why.” Within a specified 2,500-character limit (increased from a 500-character limit approximately halfway through the study), participants were able to write about events that had transpired since their last prompt, their current thoughts or emotions, or anything else that they wished to write. Importantly, this was an optional component of the EMA with an option to skip this prompt each time if a participant did not wish to provide a text entry.

Data analytic plan

Data preprocessing and subselection

For each of the 300 participants, EMA responses that included a completed optional free-text response (hereafter, “diary entry”) were selected, resulting in 9,586 diary entries across N = 268 participants. Thirty-two participants did not engage with the diary entry component of the EMA and thus were excluded from analysis. Next, these 9,586 diary entries were further subset to only include those that had a completed EMA (themselves, either with or without an associated diary entry) immediately before and immediately after. For example, if a participant on a given day completed an afternoon diary entry, and it was followed by an evening EMA but not preceded by a morning EMA, then that diary entry was removed from the dataset. See Fig. 1 for an illustrated example of this rule. This procedure resulted in 5,938 diary entries with temporally adjacent EMAs for analysis. Of particular interest to the current study was item 9 of the PHQ-9 which asks, “In the past four hours, have you had thoughts that you would be better off dead, or of hurting yourself?” Thus, the final dataset for modeling consisted of 5,938 diary entry-SI score triplicates across the cohort. The text of each diary entry served to contextualize an acute SI trajectory for a given participant, herein quantified by the self-report response scores to item 9 of the PHQ-9 before, coincident, and after the diary entry.

It is important to note that item 9 is a broad construct that not only measures SI, but also may capture precursory SI through “thoughts of being better off dead” as well as SI-adjacent behaviors through (“thoughts of hurting yourself”). Generally speaking, the literature is mixed in terms of the PHQ-9’s usefulness as a measure of STB; however, it has the advantage in this exploratory work of being a wide net with which to capture subtle changes in SI and SI-related expressions against the backdrop of those suffering from clinical depression. While it would be preferable to use alternative measures of SI, the use of the PHQ-9 within the context of temporally dense EMA was not only minimally burdensome for participants, but was bolstered by previous work which has begun to establish its deployment within this setting [35, 68].

Fig. 1.

Hypothetical Example of Valid Participant EMA Triplicates Note. EMAs and diary entries completed over the course of the first five days for a hypothetical participant are shown. Data used for analysis was selected based on the requirement that a diary entry was completed as the second of three sequentially completed EMA. Each valid diary entry is illustrated (highlighted) alongside its respective EMA triplicate (gray bar that spans three consecutive EMAs). In the example shown, there are three valid data points (labeled “1”, “2”, and “3”)

Defining acute SI trajectories

Because self-report SI was quantified in terms of a sliding scale ranging from 0 to 100 (see “Measures” above), it was important to first define the limits of what would be considered “no significant change” between adjacent timepoints. Accordingly, this work utilized probability of acute change (PAC) to determine the threshold of stationarity [74, 75]. Following recommendations in previous work, the difference scores between all consecutive EMAs across the entire cohort were calculated, and the 90th percentile value of difference scores was used as the minimum required to define “change” [74, 75]. As a result, any difference below 12.0 units was not considered a change for the purposes of trajectory profiling as described in the following paragraph. To operationalize acute changes in SI, each of the score triplicates derived in “Data preprocessing and subselection” were assigned a trajectory class. Each trajectory class defines an acute “phase” of SI that is based on the context of endorsed SI severity surrounding a diary entry. Using the PAC-derived threshold of change, eleven phases were defined to capture an exhaustive list of every possible three-point trajectory of change:

1. Apex. Endorsed acute SI severity is higher at the time of the diary entry in comparison to both the EMA immediately before and after the diary entry.

2. Trough. Endorsed acute SI severity at the time of the diary entry is lower in comparison to both the EMA immediately before and after the diary entry.

3. Escalation. Endorsed acute SI severity at the time of the diary entry is higher in comparison to the EMA immediately before the diary entry and lower in comparison to the EMA immediately after the diary entry.

4. De-escalation. Endorsed acute SI severity at the time of the diary entry is lower in comparison to the EMA immediately before the diary entry and higher in comparison to the EMA immediately after the diary entry.

5. Plateau. Endorsed acute SI severity at the time of the diary entry is higher in comparison to the EMA immediately before the diary entry and equivalent to the EMA immediately after.

6. Valley. Endorsed acute SI severity at the time of the diary entry is lower in comparison to the EMA immediately before the diary entry and equivalent to the EMA immediately after.

7. Upturn. Endorsed acute SI severity at the time of the diary entry is equivalent to the EMA immediately before the diary entry and lower in comparison to the EMA immediately after.

8. Downturn. Endorsed acute SI severity at the time of the diary entry is equivalent to the EMA immediately before the diary entry and higher in comparison to the EMA immediately after.

9. Stability—Low. Endorsed acute SI severity at the time of the diary entry is equivalent to both the EMA immediately before and after the diary entry while also ranging in severity from 0 to 24.

10. Stability—Medium. Endorsed acute SI severity at the time of the diary entry is equivalent to both the EMA immediately before and after the diary entry while also ranging in severity from 25 to 75.

11. Stability—High. Endorsed acute SI severity at the time of the diary entry is equivalent to both the EMA immediately before and after the diary entry while also ranging in severity from 76 to 100.

A visualization of each phase class is provided in Fig. 2. Additionally, Fig. 2 provides the number of participants who had endorsed a given phase at least once, as well as the total number of diary entries (and their unique trajectories) defined by the phase across the dataset.

Fig. 2.

Summary of Acute SI Phase Trajectories Note. Each phase (panel) is illustrated and exemplified by the SI severity trajectories endorsed in the data. A dark line represents the average trajectory for a given phase type, while lighter gray lines represent the individual trajectories. Each panel also reports the total number (n) and percentage of participants with at least one instance of the respective trajectory as well as the total number and percentage of diary entries represented by the respective trajectory

Natural Language processing with SEANCE

To quantify the semantic and sentiment-based properties of each diary entry, this work utilized SEANCE (Sentiment Analysis and Cognition Engine), a highly cited and freely available natural language processing tool that leverages a rich suite of over 250 indices from eight well-known reference lexica [51]. SEANCE is provided as a stand-alone executable that takes in plain text files as input and outputs a comma-delimited file of NLP feature values that correspond with the content of the text. SEANCE allows for some customization at runtime, including which indices (lexica) to utilize, which types of words to analyze (i.e., nouns, verbs, adjectives, adverbs), and whether to turn on negation control. In the current study, all lexica were utilized, all words were considered, and negation control was enabled. Importantly, negation control operates by checking for negation terms (e.g., not) within a three-word window before and after each word in text. This is important for context as the word “happy” in “I’m not very happy” would be interpreted as positively-valenced in the absence of negation control. From the 271 NLP features generated across the 5,938 diary entries (see Supplemental File 1 for the complete dataset used in downstream analyses), 82 features were selected based on relevance and presence of variability across the data. For brevity, the following is a description of each lexicon along with example features and the total number of features utilized in the current work. A complete list of features along with their corresponding distributional statistics is provided in Supplemental File 2. For more details on the utilized lexica, features, and SEANCE, interested readers are encouraged to consult the excellent documentation and references provided by the SEANCE team [76].

1. General Inquirer (GI). A lexicon consisting of 119 lists with over 11,000 words originally developed for content analysis in politics, sociology, and psychology [57]. Example features: self, virtue, work. Total number: 31.

2. Lasswell. Drawing directly from the Dynamics of Culture, these 63 lists are a subset of the GI which distinguish between substantive goals and the elements and attributes of personal evaluation and social allocation [77]. Example features: power, rectitude, well-being. Total number: 4.

3. Geneva Affect Label Coder (GALC). Contains 38 short lists of emotion-based words [54]. Example features: boredom, envy, relief. Total number: 0.

4. Affective Norms for English Words (ANEW). Measures valence, pleasure, arousal, and dominance of words for around 1,000 English words [55]. Example features: valence, arousal, dominance. Total number: 6.

5. EmoLex. Consists of ten emotion-based word lists that were developed using crowdsourced rating efforts in combination with expert curation and references to other established lexica to label the emotional properties of words [53]. Example features: anger, joy, sadness. Total number: 10.

6. SenticNet. Measures four emotional dimensions using a semi-supervised algorithmic approach to serve as an extension of WordNet [78, 79]. Example features: pleasantness, attention, sensitivity. Total number: 5.

7. Valence Aware Dictionary for Sentiment Reasoning (VADER). Rule-based, crowd-sourced valence ratings developed for shorter texts that take syntactic and grammatical context into account [52]. Example features: negative, positive, neutral. Total number utilized: 4.

8. Hu-Liu polarity. Positive and negative word lists developed for product reviews and social texts [56]. Example features: proportion positive, number of negative words, proportion of positive to negative. Total number utilized: 5.

9. SEANCE components. Principal component analysis-based indices that cluster related features based on frequent co-occurrence [51]. Example features: failure component, certainty component, positive verbs component. Total number utilized: 16.

Group comparisons of natural Language features by acute SI phase

Stratifying the 5,938 diary entries and their associated NLP feature values into the 11 acute SI trajectory phase groups defined above (see Fig. 2), the Kruskal-Wallis test for multiple group comparisons (non-parametric equivalent to ANOVA) was performed on each of the 82 features (a total of 82 Kruskal-Wallis tests). Aligning with current recommendations [80], a minimum of five diary entries (each associated with three consecutive EMAs; 15 EMAs total) is required for each group to ensure robust approximation of the test statistic. As shown in Fig. 2, the smallest group (Escalation) exceeded this requirement (20 diary entries) and most groups (9 of 11) were more than an order of magnitude larger than the minimum recommended. The post-hoc Dunn test was then employed with the Benjamini-Hochberg correction procedure to control for multiple hypotheses. Given the exploratory nature of the work, control of the false discovery rate was selected over family-wise error control to ultimately highlight statistically significant feature differences between acute SI phase groups. This procedure was carried out using the sci-kit posthocs (v0.11.2) package in Python [81].

Results

There were 131 significant (adjusted p-value < 0.05) pairwise acute SI group differences across 31 NLP features. Most generally, these differences highlighted several semantic and sentiment-based differences in written content produced during heightened and diminished states of SI. These differences both recapitulate previous findings in the literature and offer promising support for further investigation into leveraging sentiment-based lexica (especially, VADER) in acute SI-based detection and prediction models. More specifically, findings drew attention to the contextual nuance of verbosity, hostility and anger (via the General Inquirer), as well as the emotional dimensions of pleasantness and aptitude (via SenticNet), that characterize the diary entries of individuals throughout various points in their SI trajectories. Additionally, features of writing such as increased personal pronoun usage, lower dominance, higher passivity, and a more negative outlook emerged as markers for more severe SI states. Throughout these group comparisons, the Apex contextual state of acute SI most frequently emerged as semantically unique and distinct from other states (78 of 131 significant associations; 60%). Figure 3 illustrates significant acute SI phase group differences across a selected group of NLP features. A complete account of statistics associated with these group differences is provided in Supplemental File 3.

Fig. 3.

Selected Significant Acute SI Phase Group Differences by Feature Note. Each network graphlet represents a SEANCE feature with statistically significant differences between acute SI phase trajectory groups. Each edge is a pairwise significant difference between two groups (nodes). Edge thickness is based on p-value significance threshold. APEX = Apex; TRGH = Trough; DTRN = Downturn; UTRN = Upturn; STBH = Stability—High; ESCA = Escalation; DESCA = De-escalation; STBM = Stability—Medium; PLAT = Plateau; VALL = Valley; STBL = Stability—Low

From the helpful suggestion of an anonymous reviewer, a sensitivity analysis was also conducted after the fact to determine if there were any differences in significance patterns when utilizing a subset of the data derived from participants who did not exceed the 500-character limit as imposed during the first half of the study (see “Free-Text responses” in “Measures”). Of the 5,938 total diary entries, 365 (6.15%) were greater than 500 characters in length. No participant had diary entries that were exclusively more than 500 characters, and 43 participants had at least one entry that was over 500 characters. After re-running statistical analyses on this excluded subset of the data, 135 significant group differences emerged (compared to the 131 significant group differences as reported). The additional group differences involved features from the EmoLex lexicon (i.e., negative emotion, anger, and sadness), and served to reinforce specific contextual patterns related to anger and more general, non-contextual signals of heightened SI as described in the Discussion. Moreover, the same 31 identical NLP features were implicated. All major patterns and associations as discussed and presented are supported through the results obtained on this cohort subset.

Discussion

The current work utilized EMA data across a 90-day study period to systematically profile the acute trajectories of self-report SI severity for N = 268 individuals with MDD. Focusing on a rich suite of NLP-based features, 11 classes of SI change were quantified and compared to explore statistical links between SI context and feature utility. From a starting set of 82 features, 31 were found to be statistically significant (p < 0.05), highlighting qualities of writing that were generally associated (and aligned in the literature) with heightened SI (e.g., personal pronoun usage, passivity, and negative valence). Importantly, some of these features were contextually nuanced in terms of how they present in relation to SI over short change trajectories (e.g., verbosity, hostility, and aptitude). Serving as an investigative and methodologically accessible template for future introspection and development within the acute SI space, this study offered a first look into signals and patterns of sentiment on relatively short (and more phenomenologically aligned) timescales of SI change. In doing so, the results draw preliminary attention to how the near-past and near-future trajectories of SI may influence the presentation and interpretation of in-the-moment, NLP-based signals of risk.

One feature that emerged as significantly different between multiple pairs of SI phase trajectory groups was word count (Fig. 3). Diary entries with significantly higher verbosity were found to belong to EMA triplets that reflected short-lived peaks in (APEX), and transitions to (ESCA), heightened states of SI severity. Moreover, these contrasted with states of both consistently low (STBL) and high (STBH) SI severity, where the number of words was significantly fewer. Where a previous linguistic analysis of suicide-related posts on Twitter (now X) generally found that the most strongly concerning tweets were characterized by a higher word count in relation to those that were non-suicide-related [82], there is no empirical work that more specifically distinguishes suicide-related content generated as a consequence of severe SI states with various gradations of permanency. To this end, the current finding suggests that there may in fact be a difference in the writing behavior of individuals with elevated SI that occurs within very short (e.g., 4–8 h) time scales, and that this difference in verbosity is more a function of SI’s transiency than its severity. Future work may benefit from investigating how relatively simple structural characteristics of writing such as number of words, sentence length, parallelism, and readability may differ when penned across varying spans of heightened ideation states, as these may be useful signals of near-future increases or decreases in risk.

A contextually interesting trend emerged in both hostility and anger. As shown in Fig. 3, higher hostility signaled for a rapid transition into and out of a more severe state of SI (APEX), while consistent levels of severe ideation had significantly lower hostility by comparison (STBH). This difference may suggest a capability for hostility to predict near-future decreases in heightened SI or serve as a marker of a short-lived, transient state of heightened SI. Higher anger, like hostility, was characterized by a rapid transition into and out of a severe SI state (APEX), yet unlike hostility, lower anger uniquely signaled for an ongoing or transitional state of SI decrease (DESC). In practical terms, a decrease or lack of anger in writing may serve as a marker for detecting remission of SI even if the individual is currently experiencing some level of appreciable ideation. In other words, a lack of anger need not be solely indicative of low or absent SI as would be expected, but may also define an active period of waning SI.

The literature strongly supports the importance of hostility and anger in profiling SI expression. For example, short-lived emotional hostility is consistent with how those with vulnerable narcissism—a trait for SI risk [83]—experience outbursts of this emotion that are followed in quick succession by shame and depression [84]. As such, the communication of hostility may be a useful marker for detecting spikes and predicting near-future decreases in SI severity. Additionally, broad conceptualizations of anger have been cross-sectionally associated with SI in adolescents [85], and aggression, along with its phenomenological ties to impulsivity, has been shown to be associated with increased suicidality and a likelihood of having a history of both SI and suicide attempts [86]. In support of acute temporal framing, irritability—a decreased threshold for anger—has been implicated as a proximal risk factor for elevated STB with differing escalating trajectories of STB across weeks influencing changes in baseline week-to-week irritability [87]. Moreover, in a study leveraging survey data from a large representative sample of adults in the United States, phasic irritability—episodic aggressive outbursts—was indicated to associate with SI, suicide planning, and suicide attempt [88]. Overall, the literature positively relates anger and associated emotional states with SI [89, 90] and the results herein complement these findings while providing additional temporal insight regarding the relative timing of how anger may decrease alongside SI.

Another interesting pair of significant features came from the SenticNet lexicon, where both higher and lower pleasantness in language signaled for a decrease in SI severity in the immediate future (Fig. 3). At first approximation, this result seems contradictory; however, higher pleasantness uniquely signaled for a drop in SI severity only when preceded by a more prolonged term of elevated SI (DTRN), while higher pleasantness was not indicative of decreased SI severity in the immediate future when the heightened state of SI was more transient (APEX). While speculative, expressions of pleasantness may serve as a regulatory or coping mechanism to decrease SI following habituation to a higher severity state. The same pattern was observed for aptitude. As the negative emotion of disgust underlies the construct of aptitude (see following paragraph), one speculative thought is that more severe states of SI may accompany or reflect an individual’s negative stable perceptions of themself. Thus, an increase in aptitude (a decrease in loathing or disgust) at the end of a prolonged state of heightened SI may indicate an imminent breakdown in this mental paradigm. More precise research with greater levels of temporal contextualization is needed to properly address the potential explanations for these observed patterns. Nevertheless, the differences observed between short-term (APEX) and longer-term (DTRN) states of heightened SI suggest that how momentary emotion in language may signal for future SI is not consistent because it is in part a function of the immediate past. By extension, this result highlights the limitations of developing markers for SI without proper temporal context of the individual’s SI history, a consideration that may be even more pertinent when dealing with predicting SI severity within relatively narrow windows of time. To the authors’ knowledge, no work has offered the resolution of temporal change in SI to properly interrogate this dynamic more rigorously. To develop effective risk models of SI in an acute setting, it is important that future research efforts are sensitive to the contexts of an individual’s SI state, thereby emphasizing temporal frameworks that quantify and model SI more completely.

Although pleasantness and aptitude were modeled discretely, they are tied to more complex and integrated conceptualizations of emotion. Indeed, SenticNet was developed using the Hourglass Theory of Emotion, which is an affective categorization model that organizes primary emotions around four independent and co-occurring dimensions (i.e., pleasantness, attention, aptitude, and sensitivity) with multiple levels of activation [91]. Thus, based on this framing, the overarching significance of lower pleasantness and lower aptitude dimensions as markers of heightened SI severity more precisely encapsulates associations with the negative emotions of grief, sadness, and pensiveness in the former, as well as loathing, disgust, and boredom in the latter. Most of these more nuanced emotions have been found in the literature to play key roles in STB progression and risk elevation [92–95], thus it may be fruitful to apply one or more components of SenticNet in future NLP-based studies to further explore the temporal qualities of these emotions in those with SI.

With 33 of 131 significant SI phase group differences attributed to VADER-based features of negative (17; Fig. 3), compound (15), and neutral (1) sentiment alone, VADER emerged among the leveraged lexica as most consistently discerning in terms of highlighting sentiment-based differences between high and low severity SI states. In contrast to the features of SenticNet, which demonstrated a contextual sensitivity to near-past and near-future SI, VADER’s features were more attuned to capture momentary associations between sentiment and SI. Further bolstered by previous works that have utilized VADER to promisingly probe the presentation of trauma-related [96], depression-related [97], and STB-related sentiment [98], the findings are nevertheless an endorsement for future work to explore the integration of VADER into the sentiment-based components of STB operationalization tasks.

Several other features, while not exhibiting patterns of significance between phase groups that can be used to inform acute SI context, nonetheless suggested a general utility in discriminating high from low SI severity. In terms of emotion, the current results implicate, lower positive emotion, joy and trust, as well as higher negative emotion and fear, as associated with heightened states of SI across the cohort; however, it is important to recognize that emotional experience in SI tends to be highly variable, dependent on the acute trajectories of co-occurring SI fluctuations, and at times unintuitive [47, 99]. In terms of semantics, writing that was more passive/less dominant and of a more negative outlook was indicative of higher SI. Passivity, through the lens of social problem solving, has been implicated as a contributor to suicidal vulnerability [100]. Additionally, echoing repeated findings in the literature [82, 101, 102], higher personal pronoun usage emerged as a clear marker (Fig. 3). Taken together, a majority of the observed patterns in written language and emotion echo previous findings in the literature, further bolstering the potential utility of these features and the convergent validity of this work.

Strengths and limitations

The current work offers a novel, accessible, and easily reproducible analytical framework for exploring the acute longitudinal contexts of SI written expression, leveraging a robust NLP tool that integrates a wide and rich array of prominent sentiment resources. Moreover, the data used to accomplish this was densely sampled over 90 continuous days, enabling a more temporally-nuanced investigation of acute SI dynamics beyond what has been typically seen in the literature. Despite these strengths, this study has important limitations to mention.

First, and most importantly, the use of item 9 of the MPHQ-9 to serve as a proxy for SI was broad and perhaps oversensitive to SI expression within the cohort (see “Data preprocessing and subselection” in “Data analytic plan“). While inappropriate for modeling within practical or clinical settings, item 9 was a means by which foundational NLP-based feature exploration could be logistically carried out, not only with minimal burden to participants, but within longitudinal and densely sampled timescales. Such efforts were carried out with the hopes of promoting future work that grounds theories and models in ecologically valid and empirically supported data paradigms. The measure is certainly imperfect, and its validity as a tool within mobile environments is still in the early days of assessment [35, 68]. Naturally, such nascency precludes robust and generalizable psychometric evidence, but there is precedence for further application and testing. For example, findings provide preliminary evidence that the MPHQ-9 is a valid EMA measure of depression and exhibits a strong internal consistency and stability that is superior to the PHQ-9 [68].

Second, the cohort was a representative sample of individuals with MDD, thus affecting the generalizability of the findings to the general population. For example, it is possible that the way in which depressed persons with SI express themselves in writing is different from those without MDD. Relatedly, the exclusion criterion of active suicidality established through the larger Tracking Depression study potentially led to a phenomenologically limited appreciation of NLP-based SI in the current investigation. While the results are still useful in directing future research emphases, it is possible that insights gleaned from these results on a well-educated, majority White, female, and heterosexual cohort are not generalizable to broader populations of suicidal individuals who may fall outside of these sociodemographic designations or may exist at the more extreme end of the risk severity spectrum.

Third, participant engagement with the optional diary entries as well as compliance with SI self-report through EMA completion were variable, thus impacting the total number of representative data points within and between participants in the analysis. As a consequence, some participants were more represented than others which could have affected the resulting patterns of feature significance. This fact also had an influence on how PAC was calculated—necessitating a cohort-level approach to cut-off determination rather than more robust and sensitive person-specific designations. Indeed, approximately 60% of the analyzed cohort had a small number of representative person-specific samples (e.g., less than 30), rendering estimations of within-person PAC unreliable and likely inaccurate. Variable engagement with EMA and diary entries also raises the question regarding whether a participant with more severe SI is less likely to complete a diary entry. Indeed, previous research into STB within a veteran population had shown that individuals who declined to endorse their STB were among a subset at heightened risk for suicide [103]. While this variable engagement with EMA in the current study does not invalidate the findings, it does make their generalizability less clear. The analytical approach outlined herein would thus ideally be repeated on cohorts with more consistently representative data.

Fourth, some of the acute SI phase trajectories were of low or imbalanced prevalence which may have hindered the ability to uncover novel SI contexts. For example, there were 195 instances of the Apex phase trajectory, but only 20 instances of Escalation (see Fig. 2), ultimately providing less of an opportunity to utilize the trend of SI escalation as an investigative piece of the contextual puzzle.

Last, given the novel methodological aspects of this work, the decision to define MPHQ-9 scores as “low” (0–24), “medium” (25–75), and “high” (76–100) for constructing the Stability phases, while based on a general correspondence with the 27-point PHQ-9 scale, has not been substantiated in the literature. The goal was to roughly capture extremes on the MPHQ-9 sliding scale for the purposes of highlighting heightened and dampened acute SI endorsement; however, future work is needed to examine how the MPHQ-9 may be most effectively translated from the traditional PHQ-9.

Conclusions

The prediction and prevention of STB-related outcomes will benefit from a more comprehensive appreciation of how SI acutely presents and fluctuates—a currently understudied and underappreciated component of the STB behavioral mosaic. The exploration of written expression through the lens of NLP is one approach that has the opportunity to provide useful quantitative markers for SI severity and change, yet such work often requires densely sampled data collection regimes to phenomenologically match the timescales of acute SI. Accordingly, the current research offered a broad and repeatable exploratory approach that married profiles of acute SI change with those of written sentiment, ultimately capitalizing on the affordances of frequent and continuous EMA data sampling—a ubiquitous technique within the behavioral and social sciences. While the findings implicated several “usual suspects” in terms of sentiment and SI associations, some qualities of writing related to verbosity, hostility, and pleasantness, among others, offered novel perspectives regarding how features of writing (and their associated interpretations with SI) may be dependent on the greater context of near-past and near-future SI states. Importantly, future efforts toward profiling acute SI via NLP- and sentiment-based features should strive to analyze data within shorter timescales as there are potentially important signals that have yet to be fully appreciated. To this end, the dynamics and shifting contexts of SI may be more robustly captured, quantified, and leveraged toward the increased sophistication and deployment of STB risk models within ecologically valid settings.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

3ST

Three Step Theory of suicide

ANEW

Affective Norms for English Words

ANOVA

Analysis of variance

AUC

Area under the curve

EMA

Ecological momentary assessment

GALC

Geneva Affect Label Coder

GI

General Inquirer

IMV

Integrated Motivational-Volitional (model of suicide)

IPTS

Interpersonal-Psychological Theory of Suicide

MDD

Major depressive disorder

NIMH

National Institute of Mental Health

NLP

Natural language processing

PAC

Probability of acute change

PHQ-9

Patient Health Questionnaire-9

SCID

Structured Clinical Interview for DSM-5

SEANCE

Sentiment Analysis and Cognition Engine

SI

Suicidal ideation

STB

Suicidal thought and behavior

VADER

Valence Aware Dictionary for Sentiment Reasoning

Author contributions

The following reflects individual contributions per the nomenclature of the Contributor Roles Taxonomy (CRediT): DL: Conceptualization, Data Curation, Methodology, Software, Investigation, Formal analysis, Visualization, Writing—original draft, Writing—review & editing. ACC: Investigation, Writing—original draft, Writing—review & editing. MVH: Investigation, Writing—review & editing. TZG: Investigation, Project administration, Writing—review & editing. AP: Software, Investigation, Writing—review & editing. SN: Software, Investigation, Writing—review & editing. DMM: Writing—review & editing. ATC: Software, Investigation, Writing—review & editing. NCJ: Investigation, Resources, Methodology, Writing—review & editing.

Funding

This work was funded by the National Institute of Mental Health (NIMH) and the National Institute of General Medical Sciences (NIGMS) grant (R01MH123482) as well as by an institutional grant from the National Institute on Drug Abuse (NIDA; 5P30DA02992610).

Data availability

The complete dataset used in analysis, the distributional statistics for all NLP features quantified with the data, as well as all statistical results are available in the supplemental files associated with this work. Supplemental materials are also available on Open Science Framework at https://osf.io/2uhkg/.

Declarations

Ethics approval and consent to participate

The current study received approval from the Committee for the Protection of Human Subjects at Dartmouth College (STUDY00032081) and was carried out in compliance with the Declaration of Helsinki. Participants provided written informed consent prior to the collection of information and verbal informed consent prior to the start of the study. This consent procedure was approved by the Committee.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.