Language as a Biomarker for Psychosis: A Natural Language Processing Approach

Abstract

Human ratings of conceptual disorganization, poverty of content, referential cohesion and illogical thinking have been shown to predict psychosis onset in prospective clinical high risk (CHR) cohort studies. The potential value of linguistic biomarkers has been significantly magnified, however, by recent advances in natural language processing (NLP) and machine learning (ML). Such methodologies allow for the rapid and objective measurement of language features, many of which are not easily recognized by human raters. Here we review the key findings on language production disturbance in psychosis. We also describe recent advances in the computational methods used to analyze language data, including methods for the automatic measurement of discourse coherence, syntactic complexity, poverty of content, referential coherence, and metaphorical language. Linguistic biomarkers of psychosis risk are now undergoing cross-validation, with attention to harmonization of methods. Future directions in extended CHR networks include studies of sources of variance, and combination with other promising biomarkers of psychosis risk, such as cognitive and sensory processing impairments likely to be related to language. Implications for the broader study of social communication, including reciprocal prosody, face expression and gesture, are discussed.

Introduction

Language and speech are the primary sources of data for clinicians to diagnose and treat mental disorders. They provide a rich source of information about the organization and content of thought, and they are easy and inexpensive to collect. Traditionally, language and speech have been analyzed through expert opinion, clinical ratings and manual linguistic analyses. While informative, these approaches have limitations. Expert opinion can be influenced by subjective appraisal. Clinical ratings can be restricted by incomplete response sets. Clinical judgments often lack precision because they are based on ordinal scales. Manual linguistic analyses can yield finer-grain distinctions than those afforded by clinical observations, but the effort required to conduct such studies is usually so high that they cannot be practically applied in large-scale studies, much less clinical settings. The close connection between language and higher-order thought processes entails that language and speech may offer one of the most informative collections of features for predicting mental illness (Elvevåg et al., 2016), but unless these features can be extracted quickly and reliably, the promise of this approach cannot be practically realized.

Language features are becoming more trackable. Computational methods from artificial intelligence and natural language processing (NLP) currently allow for the immediate and accurate extraction of linguistic features. Recent studies show how these features can be used to predict mental illness, even in the nascent stages of a disease (Foltz et al., 2016; Elvevåg et al., 2007; Bedi et al., 2015; Corcoran et al., 2018; Mota et al., 2017; Rezaii et al., 2019). Automated analyses of language may facilitate the transition from clinical practice based on clinical judgment alone to “measurement-based care” (Insel, 2017), opening up new ways of classifying psychopathology based on objective features. Such an approach is fully compatible with the goals of The National Institute of Mental Health’s Research Domain Criteria (RDoC). Language is emerging as a source of predictive features not only because the computational methods are making extraction relatively easy, but also because these methods are beginning to mine the kinds of features that are likely to be especially predictive of mental illness, including features relevant to the prediction of transition to psychosis in “clinical high risk” (CHR) individuals.

In applying NLP analytics to language and speech, replicated patterns may emerge that are characteristic for specific diagnoses or symptoms, prognostic for later outcomes, and/or markers of illness progression or treatment response, especially within psychiatric disorders. Therefore, these linguistic features or patterns may be treated as putative biomarkers that can be developed and validated, with standards of evidence established for their context of use in clinical trials (diagnostic, enrichment, stratification), in accordance with concept clearance by the National Institute of Mental Health (https://www.nimh.nih.gov/funding/grant-writing-and-application-process/concept-clearances/2014/biomarker-development-and-validation-establishing-standards-of-evidence-for-their-context-of-use-in-clinical-trials.shtml ). In this concept clearance, the psychosis prodrome is considered a priority area in respect to “unmet medical need, lack of objective endpoints, reasonable development path, and traction/feasibility.”

Herein, we will 1) review key findings on language production disturbance in psychosis and schizophrenia; 2) outline procedures for collecting and analyzing language in psychosis risk, including clinical ratings, manual analyses and automated methods, with attention to harmonization and risks; 3) describe the reasonable development path for linguistic biomarker development in schizophrenia and psychosis risk and consider combinations of linguistic biomarkers with other psychosis risk biomarkers across levels of analysis (genes, molecules, circuits, physiology, cognition/behavior); and 4) describe future plans to conduct analyses at the level of the dyad, and broaden data to include prosody and face expression.

Language production disturbance in psychosis/schizophrenia

Disorder in thought is evident as disorganization in communication. Thought disorder has long been recognized as characteristic of psychotic disorders such as schizophrenia (Roche et al., 2015). Kraepelin described “dream speech” (Kraepelin, 2010) and Bleuler described “loosening of associations” (Bleuler, 1950) as characteristic of schizophrenia specifically. Later investigators such as Harrow (Harrow & Quinlan, 1977) and Andreasen (Andreasen & Grove, 1986) found thought disorder existed in other psychotic disorders as well. Harrow solicited speech using the Rorschach, and applied the Thought Disorder Index (TDI), which comprises clinical ratings of observed language disturbances, rated by tiers of severity and frequency of occurrence. The TDI includes 1) “minor idiosyncrasies” such as flippant responses, vagueness, peculiar verbalizations, word-finding, clangs, perseveration and incongruous combinations; 2) “distinct oddness” items such as idiosyncratic symbolism, confusion, looseness, playful confabulation and fragmentation; 3) “psychotic disruption” items such as absurd responses, confabulations and autistic logic, and 4) complete break of “reality contact”, including contamination, incoherence and neologisms (Solovay et al., 1986). The application of the TDI to responses to the Rorschach has also been used to assess thought disorder in unaffected relatives of schizophrenia patients (Levy et al., 2010) and familial (Metsänen et al., 2006; Gooding et al., 2012) and clinical high risk cohorts (Kimhy et al., 2007; Kimhy et al., 2014), finding disparate results.

Andreasen, on the other hand, used natural language as the basis for study. She invited patients to talk without interruption for 10 minutes, and they were then asked about the personal and the abstract, for 30 to 45 minutes. Andreasen held that thought disorder could be assessed simply from observing a “patient’s speech and language behavior”, “without complicated experimental procedures” and “without any attempt to characterize the underlying cognitive processes”. Andreasen argued that in thought disorder, the speaker “violates the syntactical and semantic conventions which govern language usage” (Andreasen & Grove, 1986). Andreasen developed and validated the Scale for the Assessment of Thought, Language and Communication (TLC) (Andreasen, 1979a), which included 18 items (poverty of speech, illogicality, incoherence, clanging, neologisms, word approximations, poverty of content of speech, pressure of speech, distractible speech, tangentiality, derailment, stilted speech, echolalia, self-reference, circumstantiality, loss of goal, perseveration and blocking). The TLC did not discriminate among diagnoses of mania, depression and schizophrenia when applied to open-ended narratives but instead yielded two main domains of thought disorder, conceptualized as “positive” and “negative” thought disorder (Andreasen, 1979b). Positive thought disorder includes decreases in semantic or discourse coherence (e.g., tangentiality, derailment, and circumstantiality), whereas negative thought disorder includes poverty of speech and content. Overall, there was equivalent positive thought disorder among patients with mania or schizophrenia, whereas negative thought disorder was most severe in schizophrenia patients (Andreasen, 1979b). Further studies, including meta-analyses (Yalincetin et al., 2017), have largely confirmed Andreasen’s heuristic, showing that positive thought disorder is evident across diagnoses, with greater “negative” thought disorder in schizophrenia than in mood disorders. This is consistent with prognostic studies as well. The TLC was applied to videotaped semi-structured interviews with school-aged children of patients with schizophrenia or affective disorder, who were asked about family, friends, school and leisure activities (Gooding et al., 2013). While positive thought disorder ratings predicted psychosis, negative thought disorder was predictive specifically of schizophrenia, but not mood disorders with psychosis. Accuracy in prediction of diagnosis a decade later was as high as 92%, suggesting these are early core features of illness that predate psychosis onset.

In respect to this heuristic of positive and negative thought disorder, Barch and Berenbaum theorized that negative thought disorder is due to difficulty in generating a discourse plan, whereas positive thought disorder is due to difficulty in maintaining a discourse plan and monitoring ongoing content of speech. To test these hypotheses, they manipulated factors in eliciting speech in schizophrenia patients, including varying context before stories to influence generation of a discourse plan, and varying the question type to influence maintenance of a discourse plan (Barch & Berenbaum, 1997). They operationalized negative thought disorder as reduced verbosity (number of words) and syntactic complexity (mean number of dependent clauses per independent clause), and increased pause length. They used the TLC to count instances of positive disorder or disturbance in discourse coherence (e.g. “tangential responses”, “loss of goal”, “derailment”, “non-sequiturs” and “distractible speech”) in schizophrenia patients, adjusted for speech output. To further index discourse coherence, they included measures of referential cohesion, which refers to the use of language features that tie or link ideas between phrases or sentences (Halliday & Hasan, 2014). Referential cohesion can be pronominal (“Joe” is later referred to as “he”), demonstrative (“the girl” can be later referred to as this girl), comparative (“this” is contrasted with “that”). Overall, Barch and Berenbaum found that low context (fewer directions) yielded speech characterized by more negative thought disorder, whereas low structure of questions (e.g. vague topic) yielded speech characterized by more positive thought disorder, all within the same individuals. Their experimental manipulation showed that indications of thought disorder are context-dependent and more evident when auxiliary conversational structure by the interviewer is less present.

Positive thought disorder

Reduction in discourse or semantic coherence, as operationalized by the TLC as positive thought disorder, has been assessed in schizophrenia and related psychotic disorders through the NLP analytic of latent semantic analysis (LSA). LSA rests on the premise that word meaning is a function of the relationship of each word to every other word in the lexicon (Landauer & Dumais, 1997; Landauer et al., 1998). The key insight in LSA is that word meaning are implicit in distributions of frequencies across contexts. LSA begins with the construction of a term-document matrix. The rows in the matrix correspond to individual words and the columns to documents, or otherwise, contexts. Cells in the matrix are filled with frequency counts (the number of times a word appears in a given context) weighted by the relative importance of each frequency, as specified in the tf-idf algorithm (Robertson, 2004). To reduce noise and increase generalization, distribution of frequencies across contexts is projected into a lower (300–400) dimensionality space using single-value decomposition. The semantic space created by LSA can specify, for example, that the words “sofa” and “couch” are highly similar in meaning. Their high similarity stems from the tendency of the words to appear in the same contexts, even if they rarely appear together in the same context because doing so would be redundant. More formally, the relative similarity between any two words can be assessed in terms of the cosine of the angle between the vectors (or word embeddings) associated with each word. New approaches to constructing word embeddings have recently appeared, such as Word2Vec (Mikolov et al., 2013a; Mikolov et al., 2013b), GloVe (Pennington et al., 2014) and more recently BERT (Devlin et al., 2018). The principles behind these new approaches are similar to those of LSA given that word embeddings are derived from distributions over linguistic contexts. Once word meanings are available, they can be combined to create representations for sentences. Vectors for sentences are calculated by summing the vectors associated with each word in the sentence. Sentence vectors, in turn, can be used to measure semantic coherence at the discourse level by simply measuring the cosine between adjacent sentences.

In 2007, Elvevåg and colleagues were among the first to use LSA to compute discourse coherence in language elicited from schizophrenia patients and healthy volunteers using a variety of language tasks; patients were stratified based on TLC thought disorder ratings (Elvevåg et al., 2007). In schizophrenia patients with high TLC ratings, LSA detected more unusual word associations and less semantic similarity among animals successively named in a verbal fluency task. In interview, participants had prompts such as “Tell me the story of Cinderella/Romeo and Juliet” or “Why do some people believe in God?” or “What would someone need to do to do their laundry?” Using a “moving windows” method and computing successive cosines of phrases in respect to the initial prompt, patients with high TLC ratings lost coherence more quickly. They also had less discourse with other participants’ responses. In another study, Elvevåg and colleagues applied LSA to schizophrenia patients and their unaffected relatives, finding accuracy in discrimination of 86%; participants were asked to talk about whatever came to mind, perhaps what they did yesterday or what they would like to be doing (Elvevåg et al., 2010). Decreased LSA semantic coherence also characterizes older schizophrenia patients (Holshausen, et al., 2014), in whom it is related to poor adaptive functioning, independent of demographics and other symptoms.

Negative thought disorder

Negative thought disorder in schizophrenia and psychosis risk may plausibly be indexed through other NLP analytics, such as part-of-speech (POS) tagging (Santorini, 1990), by assessing semantic density as an index of poverty of content (Rezii et al., 2019) and also through the use of speech graph analysis (Mota et al., 2012; Mota et al., 2017). In respect to POS tagging, just as every word in a text can be ascribed a semantic vector using LSA, every word in a text can also be labeled or “tagged” in respect to its grammatical function (Bird, 2009; Santorini, 1990), again learning from a large corpus of text. Once words are tagged, indices of syntactic complexity can be determined, including sentence length determined using rules of grammar, and frequency of types of “complementizer” words such as “that” and “which”, which can be used to introduce dependent clauses. Reduced sentence length and “complementizer” usage comprised part of an NLP classifier that predicted psychosis onset in one CHR cohort, and which was correlated with negative symptom severity (Bedi et al., 2015), Poverty of content is a feature of negative thought disorder characteristic of schizophrenia (Andreasen, 1979b), and which is predictive of psychosis onset (Rezaii et al., 2019; van Rooijen et al., 2017). Rezaii et al., (2019) showed how this indicator of psychosis, which they describe as low semantic density, could be identified through the computational technique of vector unpacking. The technique of vector unpacking starts with a sentence vector, which is simply a vector created by adding together and normalizing the vectors associated with the words in a sentence. It also begins with a large inventory of vectors for most of the words used in a given language. The technique uses an optimization algorithm known as “gradient descent” to discover the linear combination of weighted word vectors from this inventory that best approximates the observed sentence vector. When there is minimal semantic overlap among words in a sentence, all the words in the sentence vector are usually recovered. However, when the semantics of the words in a sentence overlap in meaning, the number of meaning vectors needed to create the sentence is less than the number of content words, resulting in a reduction in semantic density. Rezaii et al., (2019) showed how this technique, in combination with analysis of the speaker’s content, could be used to predict psychosis onset among CHR individuals with high accuracy.

Patterns in language connectedness, that is the proximity in the discursive order of words regardless of content and syntax, offer yet another predictor of psychosis. Language connectedness, in particular its complexity, can be assessed using graph theory (Sigman & Cecchi, 2002). Graphs can be created from language by treating the words as nodes and the connections between successive words in a narrative as edges (Mota et al., 2012; Mota et al., 2017). Indices include the size of the strongly connected sub-graphs or components within the speech graph. Such sub-graphs can be used to discriminate the sparse speech of schizophrenia from that of manic psychosis. It can also be used to predict the emergence of first-episode psychosis, as well as account for the variance in negative symptoms within six months of onset (Mota et al., 2017). A normative developmental trajectory has been identified for these indices of complexity, showing early deviation for patients with psychosis (Mota et al., 2018). Further, these speech graph features have been correlated in psychosis with cortical gyrification, degree centrality in resting state functional connectivity, processing speed and clinical ratings of thought disorder (Palaniyappan et al., 2018). Speech graph methods hold promise for understanding language disturbance in CHR patients.

Procedures for collecting and analyzing language in psychosis risk

Clinical ratings

In schizophrenia and other psychotic disorders, language has been assessed in clinical interviews using the Positive and Negative Syndrome Scale (PANSS) (Kay & Opler, 1987), and in many psychosis risk cohorts, its derivative, the Structured Interview for Prodromal Syndromes/ Scale of Prodromal Symptoms (SIPS/SOPS) (Miller et al., 1999), as the primary way to evaluate “conceptual disorganization”. Similarly, for psychosis risk, the Comprehensive Assessment of At-Risk Mental States (CAARMS) (Yung et al., 2005) assesses “disorganised speech” through both subjective review and objective rating. These items primarily assess circumstantiality and tangentiality, akin to Andreasen’s “positive thought disorder” rubric, with the PANSS and SIPS/SOPS capturing Andreasen’s “negative thought disorder” through negative symptom items such as “emotional expression” and “ideational richness”.

Interestingly, the SIPS/SOPS “disorganized communication” item (e.g. P5) has consistently predicted psychosis onset in psychosis risk cohorts (DeVylder et al., 2014; Nelson et al., 2013; Demjaha et al., 2012; Addington et al., 2015; Cornblatt et al., 2015) including as a stable elevated trajectory over time in one medium sized cohort (N= 100, 26 converters), with a hazard of >2.2 (DeVylder et al., 2014). The predictive power was subsequently confirmed in the North American Prodrome Longitudinal Study (NAPLS) consortium (N = 764) (Addington, et al., 2015), with an increased hazard of 8.0 for the same cutoff (SIPS rating >2) at one NAPLS site, (N = 92, 25 converters), carrying the greatest weight in their predictive model (Cornblatt et al., 2015).

Manual linguistic analyses

Beyond clinical ratings, manual linguistic methods are used to assess disorder in thought. In an assessment of language abnormalities in speech transcripts, Bearden and colleagues showed that later transition to psychosis in CHR individuals was predicted by increased frequency of illogical thinking, with accuracy of 71%, compared with 35% for clinical ratings (Bearden et al., 2011). Poverty of content and decreased referential cohesion also predicted psychosis onset. In this study, the Story Game was used to elicit speech samples. The Story Game entails listening to two brief audiotaped stories, and then retelling each story, also answering sets of open-ended questions about the stories, such as what the participant liked about the story; it also entails creating a new story about one of four topics (e.g. “an unhappy child”). The Story Game was designed to be “an ecologically valid assessment of natural speech”, and has been validated and used across a number of conditions in children and adolescents, including autism and schizophrenia spectrum. The Story Game is rated using the Kiddie Formal Thought Disorder Rating Scale (K-FTDS), yielding frequency counts of instances of language disturbance adjusted for amount of speech produced (Caplan et al., 1989). Other than illogical thinking and poverty of content, other disturbances included looseness of associations and incoherence, which had low base rates in this risk cohort. For K-FTDS ratings, “illogical thinking” comprises a failure in reasoning or contradiction, and “poverty of content” describes a failure to elaborate, whereas “loose associations” were abrupt unpredictable topic changes, and “incoherence” was scrambled syntax (Bearden et al., 2011).

In this same study by Bearden and colleagues, transcripts were evaluated for cohesion, which refers to language features that tie or link ideas between phrases or sentences (Halliday & Hasan, 2014). Referential cohesion can be pronominal (“Joe” is later referred to as “he”), demonstrative (“the girl” can be later referred to as this girl), comparative (“this” is contrasted with “that”). Reduction in referential cohesion can be indexed by the number of unclear or ambiguous references, adjusted for number of words; it can be elicited in schizophrenia by the use of unstructured (vs. structured) questions (Barch & Berenbaum, 1997). Decreased referential cohesion in response to the Story Game in CHR individuals predicted both later schizophrenia outcome and impairment in role function at follow-up. Likewise, poverty of content also predicted later schizophrenia outcome, as well as impairment in social function at follow-up (Bearden et al., 2011).

Natural language processing in psychosis risk cohorts

Numerous studies have shown how manual analyses of natural language can be used to identify thought disorder. In practice, however, manual analyses are difficult to implement. Such challenges have led to the use of automated NLP methods in studying language patterns related to psychosis risk: a partial inventory of such methods is shown in Table 1. These approaches are often used in combination. For example, LSA has been paired with POS tagging to evaluate discourse coherence and syntactic complexity, respectively and together, broadly following the heuristic established by Andreasen and used by investigators such as Barch and Berenbaum. Semantic density, as an index of poverty of content, has been paired with actual semantic content (the meaning of the words themselves) to predict transition to psychosis. The use of more than one technique suggests that language-based assessments of thought disorder might be most effective when the different techniques are combined.

Table 1.

Natural Language Processing (NLP) techniques used in the assessment of psychosis.

Property of language	NLP technique	Outcome
Discourse coherence	Latent semantic analysis, word2vec, GLoVE	Predicts psychosis onset (Elvevåg et al., 2007; Bedi et al., 2015; Corcoran et al., 2018)
	Automatically represent each sentence as a vector and compare similarity of neighboring sentences using cosine similarity
Syntactic complexity	Syntactic parsing and Part-of-speech (POS) tagging	Predicts psychosis onset
	Automatically measures phrase structure, sentence length and frequency of part-of-speech classes (e.g. nouns, verbs, pronouns)	(Bedi et al., 2015; Corcoran et al., 2018)
Poverty of content	Vector unpacking	Predicts psychosis onset
	Automatically measure semantic density: number of vectors needed to reconstruct sentence meaning	(Rezaii, Walker, & Wolff, 2019)
Referential coherence	Coh-Metrix	CHR vs. HC group difference
	Tool that applies POS-tagging and compares number of morphological roots shared across sentences	(Gupta et al., 2018)
Metaphorical language	Use word2vec and neural network to automatically tag each word as literal or metaphorical	Discriminated first episode psychosis from HC (Gutierrez et al., 2017)
Language connectedness	Speech graph analysis	Discriminates mania from schizophrenia and predicts schizophrenia diagnosis at 6 months Mota et al., 2012; Mota et al., 2017

In a small proof-of-principle study, NLP was used with machine learning to determine baseline patterns that might predict later psychosis onset among CHR individuals (Bedi et al., 2015). In this small study, LSA and POS tagging analytics were applied to open-ended narrative of 30–45 minutes elicited using qualitative interviewing techniques. A machine learning classifier with high predictive power for psychosis onset was identified that comprised minimum semantic coherence from one phrase to the next, phrase length and usage of “determiners” such as “which” and “that” as “complementizers”, which introduce dependent clauses (Bedi et al., 2015). This classifier was correlated with positive and negative SIPS symptoms but outperformed them in classification accuracy. “Minimum semantic coherence” was validated by using it to index induced parametric scrambling of classic literary texts. Further, in a post hoc analysis of the classifier in an independent sample collected by Mota and colleagues (Mota et al., 2012), the classifier distinguished the language of schizophrenia patients from that of healthy individuals in a Brazilian cohort, after Portuguese transcripts were translated to English, suggesting the classifier might be robust across illness stages and across languages.

A similar approach, using LSA and POS tagging, with machine learning, was applied to the Story Game transcripts that Bearden and colleagues used to show that illogical thinking, poverty of content and decreased referential cohesion were predictive of psychosis onset in a CHR cohort (Corcoran et al., 2018). As speech was elicited using a more structured paradigm, and responses were briefer (< 20 mean words per response at UCLA vs. > 150 words per response in NYC), there was insufficient free speech for sentence-level analysis of coherence. Hence, semantic coherence was measured using k-level or “skip-gram” measures, which computes word-to-word variability at “k” inter-word distances. Five semantic and nine syntactic features were used for machine learning classification, with singular value decomposition (SVD) used for reduction of dimensions in the training set. A four-factor solution was found for the classifier, with the top three weighted toward semantic features, and the fourth weighted for syntax, specifically possessive pronouns (“complementizers” did not add to the model). The receiver operating characteristic (ROC) for the classifier had a within-set area under the curve (AUC) of 0.88, consistent with high accuracy (Corcoran et al., 2018).

Further, to show cross-validation across sites this machine learning classifier derived from the Story Game CHR dataset was applied verbatim to the dataset from the small proof-of-principle CHR study, after first applying a Procrustean global transform (rigid translation and rotation in Euclidean space) to minimize distortions, given difference in length of responses. In cross-validation, the classifier had an AUC of 0.71 in predicting psychosis in the second independent CHR longitudinal dataset (Corcoran et al., 2018).

A third NLP study of psychosis prediction in a CHR cohort used a different approach that examined semantic content and also used an innovative approach to evaluate “poverty of content” through the measure of “semantic density”, which reflects the number of core ideas within a sentence (Rezaii et al., 2019). A skip-gram version of Word2Vec was used. Like LSA, Word2Vec learns the meaning of words from scanning a large corpus of text (in this study the New York Times corpus), but does so using moving windows, so that neural networks are trained to predict and learn words within the context of other words within a moving window. The skip-gram version of Word2Vec predicts the surrounding words based on the central word in the window. Vector unpacking, as described, was used to calculate the number of meaning vectors needed to reconstruct the meaning of a sentence, or “semantic density.” These procedures were applied to transcripts of a standardized clinical interview, along with a measure of participants’ semantic content during the interview (e.g. what they tended to talk about). Overall, lower semantic density in speech, along with greater use of words related to sounds and voices, was predictive of psychosis transition with an accuracy of 90%. Work is being done to determine the cross-site validation of this machine learning classifier and its components of semantic density and content.

Additional studies have focused on group differences in language between CHR and healthy individuals (Gupta et al., 2018). One study evaluated referential cohesion, which as described earlier refers to language features that tie or link ideas between phrases or sentences (Halliday & Hasan, 2014), and which was found by Bearden and colleagues to predict later psychosis onset in CHR individuals, using manual linguistic analyses. In this study, the Coh-Metrix tool was used to assess referential cohesion, and was applied to written narrative descriptions elicited by a visual prompt. Coh-Metrix first applies part-of-speech (POS) tagging, and then identifies roots and morphological forms to identify relational connections (e.g. referential cohesion) across the text. CHR individuals showed less referential cohesion, which was associated with severity of positive and disorganization symptoms, and lower verbal learning scores (Gupta et al., 2018).

Yet another study used an NLP approach to evaluate the use of token- or word-level “metaphor” across stages of psychotic illness (Gutiérrez et al., 2017). Patients with schizophrenia have long been known to use words in an idiosyncratic or bizarre manner, with Andreasen noting examples of “watches” being referred to as “time vessels” and “gloves” as “hand shoes” (Andreasen, 1986). In the 1990’s, Billow and colleagues noted increased frequency of deviant (but not coherent) metaphorical speech in schizophrenia patients (Billow et al., 1997). Similar to other studies, skip-gram Word2Vec was used with a neural network to tag each word or token as literal or metaphorical, in respect to a large metaphor corpus. This was complemented by automated sentiment analysis, which rates words from 1 (very negative ) to 5 (very positive), computing sentiment (and its coherence) at the word and phrase level. Speech was elicited using qualitative interviewing methods. A classifier that used all of these features, plus gender and age, discriminated first episode psychosis from healthy controls had an accuracy of 84% (beyond 75% accuracy for metaphor usage alone); this best classifier tagged 85% of all CHR individuals within a dataset, including all future converters and most CHR non-converters (Gutiérrez et al., 2017). This suggests that this approach of NLP assessment of metaphor and sentiment may be useful for screening, if replicated.

Linguistic biomarkers: reasonable development path and mechanistic studies, in tandem with other CHR biomarkers

Reasonable development path

A reasonable development path for a risk biomarker consists of initial validation and identification of sources of variability, tests of reproducibility and reliability, and mechanistic studies, and then for next steps, standardization of protocols for use as a prognostic marker and target in clinical trials, with attention to sensitivity/specificity, traction/feasibility, acceptability, cost, utility and regulatory “context of use” determined by field trials.

Among the linguistic biomarkers of psychosis risk evaluated thus far, semantic coherence reduction has been cross-validated across risk cohorts. It also may have among the most traction for consideration with other biomarkers, as it has been evaluated during the last decade in schizophrenia cohorts with genetic and circuit-level units/levels of the RDoC matrix. A preliminary study suggested associations of LSA semantic coherence with SNPs in the Disrupted in schizophrenia 1 (DISC-1) gene (Nicodemus, et al., 2014). In respect to circuits, semantic coherence measured from free discourse on “religious belief” was associated during a word monitoring task with increased modality-specific activation in auditory and visual regions, and in superior/middle temporal regions in schizophrenia patients; by contrast, semantic coherence in healthy individuals was associated only with activation in executive regions during this same task (Tagamets et al., 2014). These findings suggest normal reliance on prefrontal regions for fluency and coherence, with potential compensation from more sensory regions in schizophrenia. This is consistent with the finding that abnormal activation of superior temporal gyrus during discourse processing predicts psychosis transition among CHR individuals (Sabb et al., 2010).

The optimal parameters for the solicitation of speech are not yet known and are necessary for harmonization across studies. Both speech graph and latent semantic analyses have been applied in schizophrenia to brief narratives of several sentences over fewer than five minutes, including recall of dreams and memories (Mota et al., 2012), and descriptions of free will or how to do laundry (Elvevåg et al., 2007). More subtle differences in CHR individuals may require longer transcripts (Corcoran et al., 2018). Some investigators capitalize on analyzing diagnostic interviews, which can provide the opportunity to evaluate symptom content (Rezaii et al., 2019).

A further limitation is that most NLP studies in schizophrenia and CHR have focused on transcripts of English, except for Mota’s speech graph analyses in Portuguese, such that generalization to other languages is not yet known. However, studies are underway to collect and analyze speech from Schizophrenia (SZ) and CHR cohorts who variably speak Dutch, Chinese and Spanish.

Mechanisms

Language production and comprehension rely on canonical circuits that involve superior temporal (and inferior frontal) regions. Reductions in discourse coherence and complexity may be related to pathology in the language circuit. In one study of schizophrenia patients, abnormal activation in superior temporal regions during a word-monitoring task was associated with decreased LSA coherence (Tagamets et al., 2014). In another study of CHR patients, increased activation during a naturalistic discourse processing paradigm was observed in a network of language-associated brain regions, with increased activation in superior temporal and inferior temporal gyri specifically predictive of later psychosis transition (Sabb et al., 2010).

A strategy for understanding the circuit dysfunction that underlies language disturbance in schizophrenia and CHR may be neuroimaging during the production and comprehension of natural language itself as has been done by Uri Hasson and colleagues, finding in normal individuals the synchronized recruitment of an extensive bilateral network (Silbert et al., 2014). Scrambling of stories heard at the word (1 +/− 0.5 sec), sentence (8 +/− 3 sec) and paragraph (38 +/− 17 sec) levels shows a normative expansion in topography of intersubject synchronization over longer time windows of intact speech (Lerner et al., 2011). Hasson has postulated a hierarchy of temporal receptive windows for language that reflect a topography from basic unimodal sensory regions (shorter windows) to higher-order processing cortical areas (longer windows). He has found that a temporal receptive window of ~10 seconds (i.e. sentence length in English) is needed for reliable activation in middle and superior temporal regions when listening to a narrative. It may be that schizophrenia and CHR patients will have a disruption in this topography, especially in superior temporal regions, which may be correlated with NLP indices. For example, if information processing breaks down over the 8–10 second time frame of a sentence, it is plausible that an individual may go off track (reduced coherence) or pause without elaboration (reduced complexity). Kuperberg has theorized that in schizophrenia, there is a breakdown in hierarchical generative framework of language, in which normally, higher-level inferences constrain interpretation of sensory information and are updated based on prediction error (Brown and Kuperberg, 2015): novel computational approaches will be needed to test this.

Language disturbances similar to those seen in SZ can be readily induced by NMDA receptor antagonists such as ketamine, suggesting the language disturbance seen in SZ and CHR may reflect underlying glutamatergic dysfunction. Language production requires “chunking” or grouping of contextually related stimuli, and the formation of “nested tree structures”, processes that involve superior temporal and inferior frontal regions (Dehaene et al., 2015). Related to this, deficits in perceptual grouping of visual stimuli (contour integration, visual closure) are also associated in schizophrenia with thought disorder (Uhlhaas, et al., 2006), and likewise can be disrupted by ketamine.

Language disturbance may be related to information processing deficits in hub-like regions such as the superior temporal gyrus (Collin et al., 2018). However, it is plausible that it is related also to deficits in sensory processing, specifically auditory mismatch, as well as to deficits in basic cognitive functions such as processing speed, working memory and verbal fluency. These sensory processing and cognitive deficits have been documented in both schizophrenia and CHR individuals, in whom they are associated with an increase in risk for psychosis. The proposal to study language together with them is discussed in the next section.

Combinations with other biomarkers

Linguistic biomarkers of psychosis risk can also be assessed in combination with other known replicated biomarkers of psychosis risk, including deficits in cognitive and sensory processing, and potentially genetics and imaging markers. They can also be evaluated alone and in combination to predict other outcomes in the pluripotent CHR population, poor functional outcome, onset of other disorders, and remission and recovery. For example, in psychosis risk studies, replicated predictors of psychosis onset include slowing of processing speed, and reductions in verbal fluency and verbal memory (Seidman et al., 2010; Fusar-Poli et al., 2012); these cognitive domains also are part of the NAPLS risk calculator for psychosis onset (Cannon et al., 2016). While impairments in these cognitive domains would be expected to be related to language impairments in psychosis risk states, the associations of these with reduction in semantic coherence and other language features (e.g., semantic density, referential cohesion, use of complementizers in dependent clauses) remain unknown.

Among psychosis risk biomarkers in CHR cohorts, physiological measures of auditory processing, including auditory P300s and mismatch negativity (MMN), are among the most replicated predictors of later psychosis onset (Bodatsch, et al., 2011; Perez et al., 2014; Van Tricht et al., 2015; Shaikh et al., 2012; Hamilton et al., 2019), and also have predictive power for functional outcomes (Hamilton et al., 2018). Auditory MMN is the event-related potential that occurs in response to a tone deviant from a series of tones, typically in duration, frequency or intensity. Thus far, NLP linguistic biomarkers have not yet been evaluated in respect to auditory processing in psychosis and its risk states, though we would expect these to be associated. By contrast, specific language impairment, a heterogeneous disorder observed in children, has been consistently associated with reductions in auditory mismatch negativity (MMN) potentials (Kujala & Leminen, 2017). In specific language impairment, auditory MMN is normalized by language exercise and remediation (Dacewicz et al., 2018), which suggests a remediation strategy for abnormal auditory processing and language in psychosis and its risk states.

Other identified risk biomarkers for psychosis include polygenic risk scores (Perkins et al., 2020), progressive reduction in gray matter (Cannon et al., 2015), and abnormal patterns of resting state functional connectivity (Anticevic et al., 2015; Colibazzi et al., 2017; Cao et al., 2018). The association of these with linguistic risk biomarkers is not yet known, nor their combined predictive power for various outcomes, including psychosis, functional impairment and remission.

Overall, a research agenda for large-scale CHR networks is to evaluate combinations of promising risk biomarkers for varied outcomes, including the onset of psychotic and other disorders, for functional outcome, and for remission and recovery. These biomarker studies can be used to refine practical biomarker use in the context of precision medicine, enable stratification and case enhancement for clinical trials, and elucidate mechanisms to provide targets for preventive intervention.

Future plans to conduct analyses at the level of the dyad, and broaden data to include prosody and face expression

This review has focused on the analysis of language structure and content in psychosis. But spoken language is much more than the words that are said. Indeed, there have been several studies that indicate that acoustic features may yield important and distinct clues about etiology of psychosis as well as tools for early identification and treatment tracking. For example, computational work has shown that individuals with schizophrenia exhibit less variability in their pitch (i.e. fundamental frequency, F0) and first two formants (F1-F2), which are resonant frequencies that are determined by the shape of the vocal tract during speech (Covington et al., 2012; Bernadini et al., 2016). In addition, research indicates individuals with psychosis pause more, speak at a slower rate, and have reduced intensity and vowel space area compared to healthy controls (e.g. Martinez-Sanchez et al., 2015; Compton et al., 2018; Arevian et al., 2020). This is particularly relevant as one recent study showed that CHR individuals have increased pauses in speech, similar to what is seen in schizophrenia (Stanislawski et al., 2019). In addition to quantitatively measuring flat affect or aprosody, acoustics could also reflect motor control deficits, which are common in the prodrome (Dean et al., 2018), as successful speech production requires complex motor coordination. For example in patients with Parkinson’s Disease, acoustic measures such as the stability of syllable durations, rate of change of speech, inappropriate voicing of consonants, (e.g. pronouncing /p/ in a more /b/-like way) have shown particular promise (e.g. Karlsson et al., 2020), and could be useful for identifying individuals at high risk for psychosis as well. Moreover, the experimental evidence indicates that Parkinson’s motor deficits are accompanied by syntactic and morphological effects besides prosody (García, 2016), which may even be present in the pre-manifest stage (García, 2017). Indeed, research from our team is currently underway to examine how acoustic features might tie into cerebellar and basal ganglia circuit pathology and predict course of illness. There is further reason for optimism about the potential for clinical translation. In particular, recent work has shown the possibility of remote monitoring treatment by combining content and prosody features from short speech samples in PD patients ingesting levopoda, as well as other psychoactive drugs (Horel, et al., 2020; Agurto et al., 2020). Although some of these measures have involved manual annotation in the past, recent advances in speech technologies allow for these measures to be automatically and objectively measured (Segal et al., 2019; Shrem et al., 2019), and to be used on a wider scale.

Importantly, impairment in social function is a key feature of psychosis and its risk states that may account for most of the morbidity of these syndromes. This impairment in social function is likely multifactorial in etiology, though may be largely accounted for by impairments in social communication. Thus, we must look at language and speech within dyads, and in the context of gesture and face emotion expression. Beginning work in this area shows that individuals at risk for psychosis have abnormal turn-taking (Sichlinger et al., 2019) as well as blunted facial affect during interview (Gupta et al., 2019). This may also be expanded to including the gestures that accompany speech, which are abnormal in psychosis risk individuals as well (Mittal et al., 2006; Millman et al., 2014; Bernard et al., 2015; Osborne et al., 2018).

Conclusions

Computational analysis of ecological language and communication behavior, both in vivo and digitally via smartphones and social media, are promising avenues to pursue to understand psychosis risk and emergence, evaluated in tandem with biomarkers across genetic, physiological, circuit-based and cognitive levels of analysis. Given the close ties with other core phenomenology, links with distinct mechanisms, and ease of ascertainment and analysis, it is clear that assessment of natural language processing will be an invaluable domain for understanding and treating individuals at CHR for psychosis.