Identification of maternal depression risk from natural language collected in a mobile health app

Abstract

Depression is one of the most common pregnancy complications, affecting approximately 15% of pregnant people. While valid psychometric measures of depression risk exist, they are not consistently administered at routine prenatal care, exacerbating the problem of adequate detection. The language we use in daily life offers a window into our psychological wellbeing. In this longitudinal observational cohort study of prenatal patients using a prenatal care mobile health app, we examine how features of app-entered natural language and other app-entered patient-reported data may be used as indicators for validated depression risk measures. Patient participants (n=1091) were prescribed a prenatal care app as part of a quality improvement initiative in the UPMC healthcare system from September 2019 – May 2022. Natural language from open-ended writing prompts in the app and self-reported daily mood, were entered by patients using the tool. Participants also completed a validated measure of depression risk - the Edinburgh Postnatal Depression Scale (EPDS) - at least once in their pregnancy. A variety of natural language processing tools were used to score sentiment, categorize topics, and capture other semantic and syntactic information from text entries. LASSO was used to model the relationship between the natural language features and depression risk. Open-ended text within a 30-day and 60-day timeframe of completing an EPDS was found to be moderately predictive of moderate to severe depression risk (AUROC=0.66 and 0.67, for each respective timeframe). When combined with average daily reported mood, open-ended text showed good predictive power (AUROC=0.87). Consistently predictive language features across all models included themes of “money” and “sadness.” The combination of natural language and other user-reported data collected through a mobile health app offers an opportunity for identifying depression risk among a pregnant population.

1. Introduction

Maternal depression is one of the most common complications of childbirth^1–2 and can have severe consequences for both mother and child, including preterm birth, poor infant attachment, failure to thrive, and developmental delays.^3–8 Best clinical practice guidelines recommend routine depression risk screening throughout pregnancy⁹, yet depression often goes undiagnosed and, subsequently, untreated.^9–10 Barriers to adequate screening include physician time, stigma, and sociocultural norms of motherhood.¹¹ Effective detection of depression risk during pregnancy requires more frequent risk assessment, in a manner supporting the natural expression of negative feelings.

Language offers insight into psychological wellbeing.¹² Studies of the relationships between language and psychosocial risk have largely focused on analysis of text from large social media forums, such as Twitter and Reddit.^13–14 A smaller number of observational journaling studies have shown that written language is correlated with individually experienced psychosocial risks, including depression.^15–17 Few have focused on language as it relates to maternal depression.^18–20 We further this prior work in two ways. First, we elicit journaling language from our population of interest by embedding open-ended text opportunities in prenatal care mobile health app. Second, we examine how the features of language, in conjunction with other patient-reported data, predict future depression risk scores. Here, we present our planned methodological approach to collecting and analyzing natural language data, highlighting preliminary results from a subset of the larger data collection.

2. Methods

2.1. Data Collection

Pregnant patients receiving prenatal care at the UPMC healthcare system were prescribed the MyHealthyPregnancy^™ (MHP) mobile health (mhealth) smartphone application²¹ from September 2019 to May 2022. MHP was generally prescribed during the first prenatal appointment, typically taking place between 7 and 10 weeks of pregnancy. Upon initiation of the app, participants selected an option to provide electronic consent for their de-identified app data to be used in research studies. The MHP app offered personal symptom and clinical risk monitoring (including logging their current daily mood on a 5-point Likert scale), pregnancy-related education, pregnancy tools (such as a kick-counter), and connection to relevant resources (e.g., hotlines, community organizations, and classes).

Participants were given the opportunity to document their pregnancy journey through voluntary open-ended journal responses paired with prompts (see Table 1 for sample prompts) and a dedicated section for notes, which did not contain specific prompts.

Table 1.

Open-ended text prompts

Open-ended text prompts
What had the biggest impact on your mood today, and why?
Healthy relationships are important during pregnancy and beyond. Are you feeling supported right now? Why or why not?
In the last month, what have you done to try to keep yourself safe from coronavirus?
Are you experiencing financial or other personal difficulties as a result of this pandemic?

In addition, participants were offered multiple opportunities to complete an Edinburgh Postnatal Depression Scale (EPDS)²² questionnaire to assess depression risk. The EPDS was prompted within the app once per trimester, and participants could opt to receive push notifications to remind them to complete the EPDS screening once a month. In June 2021, the app was updated so that the EPDS questionnaire could be always accessible in a dedicated section of the app for participants. Use of app data to specifically model depression risk from language was approved by the University of Pittsburgh’s Institutional Review Board (STUDY19100210). Participants could withdraw permission to use data at any time in the app’s settings. Participants typically took the EPDS questionnaire only once during their pregnancy with the typical gestational week being 13. Participants who completed EPDS more than once typically took it during gestational weeks 10 and 24.

2.2. Inclusions Criteria and Train-Test Split

Participants were eligible for study inclusion if they completed at least one app-administered EPDS score and one open-ended text entry during their pregnancy. To be able to predict depression risk from recent language, participants were included in one or more statistical analyses if they had at least one open-ended text entry less than 60 days before completing an EPDS assessment. Open-ended text entries needed to include at least one descriptive word that could be analyzed for sentiment, eliminating non-specific entries such as “yes” or “no”. Prior to all data analysis, participants were randomly assigned to either a testing group, a development group, or a training group, with a split of 15%, 15% and 70% of participants, respectively. This paper details preliminary results using the training and development sets only.

3. Analytical Approach

3.1. Open-ended text processing

Concatenated text entries were processed to remove less semantically rich terms, including the very commonly used today and yes, and stop-words like pronouns, prepositions, and other function words. Foreign language accent marks, capitalization, and all punctuation marks were also removed from text entries. Additionally, open-ended text entries that did not precede an EPDS score or that preceded EPDS scores by more than 60 days were eliminated from the study. Concatenated text entries ranged in length from 1 word (e.g., “lonely”) to 1795 words, with a median length of 21 words before removal of stop words.

3.2. Natural Language Processing (NLP) methods

Different NLP methods were selected specifically to create language outputs that quantified the topics, sentiment, and themes present in the natural language present in the open-text entries. Latent Dirichlet Allocation²³ (LDA) was used to produce topic model outputs. The number of LDA topics was selected to be 5. This was determined through optimization of depression prediction AUROC on a subset of 15% of the training set before generating that number of topics on the full dataset. The LDA topic output was then interpreted and labeled by the research team. SentiWordNet²⁴ (SWN) was used to quantify the presence of positive or negative sentiment, with each word scored as an average across all homonyms. Scores across an entire processed text entry were then combined to produce average positive sentiment, average negative sentiment, and the sum of the average positive and average negative, to give a total of 3 SWN features for each text entry. The Linguistic Inquiry and Word Count dictionary²⁵ (LIWC) was used to count the number of occurrences of 46 psychologically-informed themes, grammatical features, and positive and negative affect within each un-processed text entry. One set of pre-trained word embeddings, word2vec²⁶, was also used to capture words’ syntactic and semantic content as observed in a massive news corpus. Averaged word2vec features within a concatenated text entry yielded a 300-dimensional word embedding vector for each participant. SWN, LDA, and word2vec were all performed on processed text entries, while LIWC was performed on un-processed text entries and included the identification of stop-words and common words. This ensured that LIWC properly captured the use of pronouns, articles, and conjunctions.

3.3. Least absolutely shrinkage and selection operator (LASSO) approach

LASSO was used to predict EPDS scores from the language features extracted from open-ended text entries that occurred within two preceding timepoints (60 days and 30 days). This approach was selected due to its interpretability as a linear model. Additionally, LASSO drops out features found to be irrelevant in predicting depression, which allows for clear identification of language features that may be practically useful in a clinical context. A conservative 60-day time frame was selected to be reflective of a stable mental health state.²⁷ A 30-day time frame was also selected to examine language that may be more reflective of current mental health state. Both time points were selected in consultation with a clinical practitioner.

Five-fold cross validation was used to optimize the shrinkage penalty lambda, with values ranging from 0.001 to 1. The optimal lambda was chosen from the highest AUROC in cross-validation and used in a LASSO regression to predict EPDS scores in the development set. Entries with average EPDS scores of 14 or more were labeled as depressed, while entries with average scores less than 14 were labeled as non-depressed. The AUROC was calculated by comparing LASSO-predicted EPDS scores between the depressed and non-depressed groups in the development set. Each processed language input had a total of 354 natural language features obtained from these tools, which were then used in the modeling. Prior to modeling, all natural language features were standardized with a mean of 0 and a variance of 1. Figure 1 illustrates how individual language features were incorporated into the LASSO regression.

Fig. 1.

(a) Box 1 shows sample (modified for anonymity) open-ended text; (b) Box 2 shows the specific NLP techniques applied to the open-ended text; (c) Box 3 shows the regression model incorporating all NLP inputs, with K being the number of LDA topics (here optimized to 5)

4. Results

4.1. Participants

Participant demographics are show in Table 2. There were 1301 enrolled participants, with 210 participants withheld for the test set. An additional 425 participants were removed from analysis due to having no useable journal entries within 60 days prior to an EPDS score. The remaining 666 participants in the training and development sets had a total of 1007 entries among them, with participants having between 1 and 9 entries per person. In these preliminary results, we report on those 1007 entries. No significant demographic differences were found between those removed from analysis (due to incomplete entries or test set membership) and those retained in the development set analyses presented here.

Table 2.

Demographics

Demographic	Eligible participants (n=666)
Age, mean (SD)	30.3 (5.4)
Income, US dollars, thousands
<15	84 (12.6%)
15 to <50	171 (25.7%)
50 to 100	210 (31.5%)
>100	185 (27.8%)
Missing or preferred not to respond	16 (2.4%)
Race and Ethnicity
White/Caucasian	531 (79.7%)
Black or African American	73 (11.0%)
Hispanic/Latinx	10 (1.5%)
South Asian	10 (1.5%)
East Asian	10 (1.5%)
Native American	3 (0.5%)
Another ethnicity	24 (3.6%)
Preferred not to respond	5 (0.8%)
Educational Level
<High school	17 (2.6%)
High school or GED	186 (27.9%)
Associate degree	82 (12.3%)
Bachelor degree	184 (27.6%)
Postgraduate degree	191 (28.7%)
Missing or preferred not to respond	6 (0.9%)
Relationship Status
Partnered	630 (94.6%)
Not partnered	34 (5.1%)
Missing or preferred not to respond	2 (0.3%)

4.2. Depression prediction at two time points

Over the course of the study, 48.2% of participants included in the modeling had EPDS scores indicating no depression, 38.3% had scores indicating mild depression, 10.5% had scores indicating moderate depression, and 3.0% had scores indicating severe depression.

Language features from open-ended text entries were found to be moderately predictive of onset depression within both a 60-day and 30-day timeframe, shown in Table 3 below. Evaluation of the development sets yielded AUROC values of 0.67 and 0.66 respectively. The addition of average mood recorded for each 30-day window augmented the performance of our models. Open-ended text inputs were only included in the model if they were accompanied by reports of mood within the same 30-day window. This model yielded a development AUROC of 0.87, with only 8 natural language features found to be predictive of depression risk. Mood alone yielded a slightly lower development AUROC of 0.85, with both average mood and minimum mood having negative coefficients (−2.48 and −0.19 respectively).

Table 3.

Model performance

Model	Number of entries in the model (n)	AUROC on training set	AUROC on development set	R2 of development set
60-day timeframe	1007	0.73	0.67	0.30
30-day timeframe	938	0.73	0.66	0.16
30-day timeframe with mood	925	0.86	0.87	0.24
Mood alone	925	0.85	0.85	0.28

Table 4 shows the coefficients of each language feature from open-ended text features that were retained as predictive of new onset moderate to severe depression risk within a 60-day or 30-day timeframe. For a 60-day timeframe, there were 39 natural language features that were found to be predictive, including the average SWN negative sentiment score, 9 LIWC themes, and 29 word2vec features. For the 30-day timeframe, there were 45 natural language features that were found to be predictive, including the average negative sentiment SWN score, 1 LDA topic manually labeled as “social support,” 11 LIWC themes, and 19 word2vec features. When incorporating daily mood into the 30-day timeframe, only seven language features were found to be predictive of new onset depression. These included the single LDA topic of “social support,” 2 word2vec features, and 4 LIWC themes.

Table 4.

Coefficient ranges for each NLP feature type

Feature Type	60-day regression		30-day regression		30-day regression with mood
	No. of features	Coefficient range	No. of features	Coefficient range	No. of features	Coefficient range
SentiWordNet	1	0.20	1	0.22	0	N/A
LDA Topic Modeling	0	N/A	1	−0.29	1	−0.12
LIWC	9	(−0.20,0.68)	11	(−0.17, 0.43)	4	(0.00,0.27)
Word2Vec	29	(−0.32, 0.26)	19	(−0.35, 0.21)	2	(0.01, 0.08)
Average Mood	N/A	N/A	N/A	N/A	1	−2.27

In both the 60-day and 30-day models, LIWC and word2vec features performed above chance at predicting depression when not paired with any other NLP methods. LDA and SentiWordNet performed above chance for the 30-day model only. However, none of the NLP methods performed better on their own than when paired with other NLP methods.

Table 5 shows the coefficients for the LIWC themes that were associated with an increased likelihood of depression for each timepoint model. In both the 60-day and 30-day timeframe, themes of “money” and “sadness” were the most highly correlated with onset depression. Themes of “anxiety,” “discrepancy” (use of words such as oughta, wishing, regretful) and “causation” (e.g., independent, obedience, solution), in addition to use of “swearing” and “negation” words (e.g., mustn’t, shouldn’t, didn’t), were positively correlated with depression. Using first-person plural language (we will as opposed to I will or they will) and using language with a positive affect were both found to be negatively correlated with depression. In a 30-day timeframe, two additional themes emerged. “Anger” was positively associated with depression risk and reference to “body anatomy” was negatively associated with depression risk. When incorporating daily mood scores in the model, only the themes of “money,” “sadness,” and “discrepancy” were retained and a new theme of “tentativeness” (e.g., perhaps, depends, dunno) was associated with increased depression risk.

Table 5.

Coefficients for LIWC themes

LIWC Theme	Example words from theme	60-day regression	30-day regression	30-day regression with mood
Money	Bank, cheap, insurance, invest, spend	0.68	0.43	0.19
Sadness	Gloomy, lonely, tears, useless, pity	0.63	0.29	0.27
Positive affect	Beauty, calm, excited, fun	−0.20	−0.16	0
Causation	Changes, used, why, how, depends	0.18	0.09	0
First-person plural	Let’s, us, our, ourselves	−0.14	−0.17	0
Negation	Nothing, without, nope, neither, never	0.13	0.26	0
Discrepancy	Ideal, lacks, should, want, hoping	0.10	0.26	<0.01
Swearing	Hell, sucks, f**ck	0.10	0.14	0
Anxiety	Afraid, confused, distress, fear, panic	0.03	0.05	0
Anger	Aggressive, annoyed, fought, lying, temper	0	0.10	0
Body anatomy	Belly, fat, heart, ribs, sleep	0	−0.05	0
Tentativeness	Generally, hoped, sorta, sometime, unsure	0	0	0.01

5. Conclusions

Features of natural written language, collected through a prenatal care app, were found to be moderately predictive of depression risk onset among pregnant people, particularly when paired with self-reports of daily mood. While the additional predictive value of language features, beyond daily mood, is not large, the specific themes and use of types of language present in pregnant people’s written journal entries indicate topics that may precede or coincide with the identification of depressive symptoms.

Fewer language features were found to predict depression risk with a shorter timeframe. The language features that appear to consistently predict new onset depression across different time periods, even after accounting for fluctuations in daily mood, are feelings of “sadness” and concerns about “money.” These findings reflect greater incidence of depression among pregnant women experiencing financial stresses during the early months of the COVID-19 pandemic.²⁸ Other language features associated with depression risk may offer a window into the experiences of pregnant people with depressive symptoms. For example, the use of plural pronouns, such as “we’re excited,” which was negatively associated with depression risk could indicate feelings of belongingness or serve as an indicator of social support. Indeed, the LDA topic of “social support” itself was also negatively associated with moderate to severe depression risk. Use of discrepancy and negation words, both of which were positively associated with depression risk, may indicate an individual feeling like things aren’t going as they should. This could reflect both circumstances and personal evaluations of one’s mental health state.

These preliminary findings illustrate the potential of language and, consistent with other studies¹⁴, the use of topics, specifically, to help identify mental health status. While the predictive power of language features and mood alone are not great enough to be used as a form of depression risk screening, these findings suggest that the use of language elicited from mhealth tools, such as a prenatal support app, may serve as a complement to routine depression screening. Moreover, they may offer insight into topics and language structure that clinicians can use as cues for depression risk.

There are limitations to collecting this kind of language data within an mhealth tool. A sizeable number of participants were excluded due to incomplete, sparse, or inconsistent entries. High fidelity data depends on participants being continuously engaged, not only with the open-ended text features, but with the mhealth tool itself. Additionally, in this analysis, we did not have a baseline depression risk score. This prevents us from estimating the number of participants with pre-existing baseline depression risk. Moreover, while we are using validated measures of depression risk, depression risk is measured by voluntary self-report. This could lead to selection bias, whereby those with higher risk of depression are more likely to complete self-reports. Future full development set results will further explore combinative power of natural language inputs with other patient-report data.