Crisis text‐line interventions: Evaluation of texters' perceptions of effectiveness

Abstract

Objective

Crisis Text Line (CTL), the largest provider of text‐based crisis intervention services in the U.S., has answered nearly 7 million conversations since its inception in 2013. The study's objective was to assess texter's perceptions of the effectiveness of CTL crisis interventions.

Method

Survey data completed by 85,877 texters linked to volunteer crisis counselor (CC) reports from October 12, 2017, to October 11, 2018 were analyzed. The relationship of several effectiveness measures with texters' demographic and psychosocial characteristics, frequency of CTL usage, and texters' perceptions of engagement with their CCs was examined using a series of logistic regression analyses.

Results

By the end of the text‐based conversation, nearly 90% of suicidal texters reported that the conversation was helpful, and nearly half reported being less suicidal.

Conclusions

Our study offers evidence for CTL's perceived effectiveness. These findings are of critical importance in light of the launch of a nationwide three‐digit number (988) for suicide prevention and mental health crisis supports in the U.S., which will include texting.

Suicide prevention and mental health crisis intervention services in the U.S. are receiving unprecedented attention as evidenced by the Department of Health and Human Services' mid‐2022 launch of the 988 nationwide mental health crisis response line (Federal Communications Commission, 2022). The Federal Communications Commission mandated that the national crisis line offer texting alongside voice to access the service (Federal Communications Commission, 2021). This is important because online and texting crisis interventions are more likely than telephonic options to be used by young people (Haner & Pepler, 2016; Mokkenstorm et al., 2017). Young people are also more likely to discuss “weighty problems,” like mental health problems and suicide, on an online platform than by telephone (Haner & Pepler, 2016; Fukkink & Hermanns, 2009). Calls, texts, or chats to 988 will connect distressed individuals to trained crisis counselors of the National Suicide Prevention Lifeline (Lifeline) network of over 200 certified crisis centers in 50 U.S. states and the District of Columbia. The positive impact of 988 depends upon the effectiveness of the services provided by the U.S. mental health and suicide crisis response system (Gould & Lake, 2021).

The nation's largest provider of crisis counseling via text is Crisis Text Line (CTL) (https://www.crisistextline.org/). CTL is a non‐profit organization that provides free, 24/7 crisis counseling services via text messaging across the country; their volunteer crisis counselors (CCs) have engaged in nearly 7 million text‐based conversations since its 2013 inception (Crisis Text Line, 2022). CCs receive 30 h of interactive online training in building rapport using de‐escalation and good contact techniques; risk assessment; collaborative problem‐solving; and action planning. Once on the proprietary platform, they receive close supervision from salaried mental health professionals monitoring the text conversations. Research using CTL anonymized data has contributed to knowledge about texters' presenting psychosocial issues (Szlyk et al., 2020), help‐seeking behavior by rural adolescents (Thompson et al., 2018), and suicide contagion following Netflix's release of 13 Reasons Why (Sugg et al., 2019). However, no study has yet focused on texters' perceptions of effectiveness of CTL's service.

Numerous pertinent evaluations have been conducted of the Lifeline (see Gould & Lake, 2021 for a summary of this evidence). These evaluations bolstered support for establishing the national three‐digit crisis number and provided substantial evidence regarding effectiveness for suicidal individuals. Findings include: significant reductions in intent to die, hopelessness, and psychological pain over the course of their crisis call (Gould et al., 2007); post‐call follow‐ups with callers have a positive impact—80% of interviewed callers indicated that the follow‐up intervention stopped them from killing themselves, and 91% reported that it kept them safe (Gould et al., 2018); crisis counselors secure caller collaboration on an intervention on over 75% of imminent risk calls (Gould et al., 2016); and emergency services are involved on 43% of imminent risk calls from the person at risk (Gould et al., 2016), and on 58% of calls from concerned third parties (e.g., friends or family members) (Gould, Lake et al., 2022). In February 2022, CTL officially became a Lifeline network center.

The Lifeline Crisis Chat (LCC) network, which serves all ages, has grown extensively since its formal establishment in 2013, answering over 200,000 chats in 2020 (Gould, Chowdhury et al., 2021). A study assessing the effectiveness of LCC employed 13,130 linked pre‐ and post‐chat surveys (Gould, Chowdhury et al., 2021). Chatters were significantly less distressed at the end of the chat intervention than the beginning. This is important because the vast majority seeking help from LCC were highly distressed. Moreover, almost 84% of LCC chatters endorsed either current or recent suicidal ideation on the pre‐chat survey‐markedly higher than the estimated 23% of Lifeline callers identified by silent monitors as suicidal on the day of or day before their calls. Findings aligned with research reporting that crisis chatters reveal higher rates of suicidal ideation than crisis callers (Fukkink & Hermanns, 2009; Haner & Pepler, 2016; Lake et al., 2022). By the chat's conclusion, two‐thirds of suicidal chatters reported the chat as helpful and nearly half reported being less suicidal, offering empirical evidence for the effectiveness of Lifeline's crisis chat services.

Lifeline added crisis text services in 2021; this service has not yet been evaluated. Thus, the current evaluation of CTL is both novel and critically needed. We designed this evaluation to parallel our earlier evaluation of LCC (Gould, Chowdhury et al., 2021), employing similar measures and methods where feasible, to provide comparable effectiveness information. This study's overarching goal is examining texters' perceptions of the effectiveness of CTL's intervention for the general population of texters and particularly suicidal texters. Specific aims focus on (i) the extent to which texters feel their CCs are genuinely concerned about them; (ii) texters' perceptions of CTL's effectiveness, as assessed by the extent to which texters feel the conversation was helpful and whether they reported feeling more hopeful, less depressed, less overwhelmed, and less suicidal at the text intervention's conclusion; and (iii) the relationship between texters' perceptions of CTL effectiveness and texters' personal characteristics, number of CTL conversations during a 12‐month period, and perceptions of CCs' genuine concern. With the imminent transition to 988 as the nationwide number for suicide prevention and mental health crisis response, the role of text‐based crisis interventions in the U.S. will increase greatly. The current evaluation of CTL is thus more important and timelier than ever.

METHODS

Immediately following each conversation, CTL asks its volunteer CCs to complete a report on each conversation and a brief, optional survey is automatically sent to all texters except those requiring an “active rescue” (intervention of emergency services). CTL provided the research team with anonymized and de‐identified CC reports and voluntary texter surveys gathered from all crisis text conversations with 10 or more messages initiated by texters during a 12‐month period (October 12, 2017–October 11, 2018). CTL did not provide surveys for conversations with fewer than 10 messages as these are expected to not be meaningfully engaged with the service.

Sample

To prepare our sample we first eliminated conversations labeled as “prank,” (texter sent lewd or absurd messages), “third party” (texter was contacting CTL for help for someone else), and “testing” (texter was testing the service before referring it to others). Of 380,850 unique, valid texters who texted on behalf of themselves during the study period, 85,877 (22.5%) responded to at least the texter survey's first item: “Did you find this conversation helpful?” We selected each texter's first conversation in which they completed the first item, yielding a dataset wherein one conversation represented each unique texter. The sample's demographic characteristics are described in detail in our companion article (Pisani et al., 2022). In brief, 65.6% of the 81,056 texters reporting their age were aged 14–24; the remainder were 13 or younger (10.3%); 25–44 (19.8%); 45–64 (4.1%) and 65 or older (0.2%). The 71,252 texters who responded to the question about their race identified themselves as: White (53.3%); Hispanic/Latinx (13.1%); Black/African American (8.2%); Asian (3.5%); American Indian/Alaska Native (1.1%); Middle Eastern/North African/Arab (0.6%); Native Hawaiian/Other Pacific Islander only (0.4%). An additional 11.9% identified as “more than one,” 2.4% responded “other” and 5.6% preferred not to answer. Of the 72,310 texters who reported their gender, 77.2% identified as female, 13.4% as male and 7.7% as a gender minority (trans male (2.3%), trans female (0.4%), genderqueer (2.0%), agender (1.0%), “other” (2.0%)). An additional 1.7% preferred not to answer.

Measures

Crisis counselor report

Ladder‐up suicide risk assessment

Crisis counselors (CCs) rated the texter's place on a Suicide Risk “ladder” using information from the crisis conversation: Thoughts of Suicide, Plan, Accessible Means, and 24‐h Timeframe of Plan. These items form a Guttman scale (Perez, 2005), with Thoughts of Suicide alone forming the first step on the ladder and each additional step including all preceding steps (e.g., someone who answers positively to 24‐h Timeframe must also have answered positively to Thoughts of Suicide, Plan, and Accessible Means). The analyses employed the texter's highest score (i.e., most serious suicide risk).

Topics discussed in conversation

Crisis counselors identified issues discussed in the text conversation from a list of topics (“issue tags”). A comprehensive list of all CC‐assigned issue tags is provided in Szlyk et al. (2020). The issue tags employed in current analyses related to mental health (depression, eating problems, and substance problems), and interpersonal issues (e.g., physical and sexual abuse), paralleling our evaluation of LCC (Gould, Chowdhury et al., 2021).

Texter survey

At the conversation's conclusion, texters received an automated invitation to complete an optional survey, which was expanded by CTL with assistance from the authors: “Would you help us better help others by giving anonymous feedback about your experience today?” Texters who opted‐in reported the following information employed in the current analyses: demographics, mental health status, perception of the CC's genuine concern for them, whether they thought the conversation helped, and whether they felt more/less/the same hopeful, depressed, overwhelmed, or suicidal at the end of their text. Our reliance on the texter's perspective is crucial for person‐centered care (Agency for Healthcare Research and Quality, 2016).

Demographics

Texters reported their (i) age (“How old are you?”: Prefer not to answer, 13 or younger, 14–24, 25–44, 45–64, 65+”); (ii) gender (“Do you consider yourself to be: Prefer not to answer, agender, female, genderqueer, male, trans female, trans male, other – write in”); (iii) race (“What is your race or origin? [Select all that apply] Prefer not to answer, American Indian/Alaska Native, Asian, Black or African American, Hispanic/Latinx/of Spanish origin, Middle Eastern/North African or Arab, Native Hawaiian or Other Pacific Islander, White, other – write in”); and (iv) sexual orientation (“Do you consider yourself to be: Prefer not to answer, asexual, gay or lesbian, heterosexual or straight, pansexual, bisexual, questioning/unsure, other – write in”).

Mental health

The PHQ‐2 measured depressive symptoms (“Little interest in doing things”, “Feeling down, depressed or hopeless”) (Kroenke et al., 2003). Anxiety symptoms were measured with the GAD‐2 (“Feeling nervous, anxious or on edge,” “Not being able to stop or control worrying” (Kroenke et al., 2009). Both measures have established clinical cutoffs indicating a high likelihood of clinical conditions (Plummer et al., 2016; Staples et al., 2019). Two items adapted from the PROMIS measured social isolation (“I feel left out,” “I feel isolated from others” with response options ranging from “not at all” to “nearly every day”) (Cella et al., 2010; Riley et al., 2011). Social isolation has no clinical cutoff. For this evaluation, we considered texters “high” on social isolation if they answered “most days” or “nearly every day” on either item.

Texter's perception of crisis counselor's genuine concern

One question assessed texters' perceptions of their CCs' concern for them: “I feel my counselor was genuinely concerned for my well‐being” (responses on a 5‐point Likert scale of “strongly disagree” “somewhat disagree” “neither agree or disagree” “somewhat agree” “strongly agree”). An earlier LCC evaluation utilized a similar measure (Gould, Chowdhury et al., 2021).

Effectiveness outcomes

Five questions assessed texters' perceptions of the effectiveness of CTL's intervention: (1) “Did you find this conversation helpful?” (yes/no); (2) “Consider the feelings below, and let us know if they changed after you texted with a crisis counselor today?” (responses on a 3‐point Likert scale of “less,” “same” or “more” for each of: “hopeful,” “depressed,” “overwhelmed,” “suicidal”). An earlier evaluation of LCC utilized similar questions (Gould, Chowdhury et al., 2021).

Text metadata

Conversation frequency

The number of conversations each texter had with CTL during the study period was derived from their metadata containing timestamps of incoming and outgoing messages (Pisani et al., 2022), and grouped as: 1 conversation, 2–10, ≥10.

Statistical analyses

Data preparation and statistical analyses were conducted using SAS 9.4 (Inc., 2013) and R version 3.6.3 (R Core Team, 2020). A detailed description of data preparation and the comparison of the texters with a survey to those without are described in Pisani et al. (2022). We explored an inverse probability weighting procedure using 237 independent variables from the CC reports, metadata (e.g., time of day, number of messages in a conversation), and counts of word sequences from text messages to adjust for differences. The weighted‐ and unweighted‐point estimates of the covariates used in our analyses were very close, signaling that weights did not add meaningfully to our understanding of the population. Statistics unweighted for texter survey completion were thus generated and reported for all analyses.

Descriptive statistics were calculated for texters' perceptions of their CCs' genuine concern for them, and for all effectiveness outcomes. Descriptive statistics of the suicidal outcome only included responses from texters for whom CCs indicated at least Thoughts of Suicide on the Suicide Risk Ladder. Associations were tested between texters' perceptions of their CCs' genuine concern and the following covariates: age, race, gender, sexual orientation, depression symptoms, anxiety symptoms, social isolation, suicide risk, topics discussed in the conversation (“issue tags”), and conversation frequency. The five texter survey effectiveness outcomes' associations with all aforementioned covariates, including texters' perceptions of their CCs' genuine concern, were tested.

In the data, missing values occurred in both outcomes and covariates. We handled both types of missingness under the missing at random assumption. Missing values in covariates were adjusted for by including a binary missingness indicator variable corresponding to each covariate with the prevalence of missingness >5%. We deleted observations corresponding to variables with <5% missingness prevalence. The missing values in outcomes were adjusted for using inverse probability of weighting approach. We first fit logistic regressions to predict the missingness in the outcomes, including all covariates and their corresponding missing‐indicators as independent variables. We then performed our analyses using individuals with observed outcomes where each individual was weighted by the inverse of their probability of being a complete case (i.e., the outcome being observed). In the inverse probability weighted analyses, weights rebalanced the set of complete cases, such that it represents the whole sample. Weighted proportional odds logistic regression (POLR) or binary logistic regression models were estimated depending on the nature of the dependent variable, with all covariates entered at the same time.

Odds ratios with 95% confidence intervals were calculated for each outcome and covariate, adjusting for all the other covariates. Reference groups used in the analyses were either the covariates' most prevalent categories or the categories representing the absence or lowest values. Given the large sample size, many statistically significant odds ratios were expected; we, therefore, highlighted statistically significant odds ratios that we considered to be “clinically meaningful” using an a priori threshold of 20% higher (or lower) odds ratio (i.e., ≥1.20 [and for lower odds ratios we used the reciprocal, ≤0.83]).

The study's protocol involving secondary analysis of de‐identified data without access to links was considered to meet Federal and University criteria for exemption by the University of Rochester's Institutional Review Board (IRB) and not to meet the definition of Human Subjects research requiring review by the New York State Psychiatric Institute/Columbia University Department of Psychiatry's IRB.

RESULTS

Texters' perceptions of crisis counselor's genuine concern

Approximately, two‐thirds (65.2%) of texters (83.6% of those who responded to this question) agreed that their CCs were genuinely concerned for their well‐being (Table 1) (strongly agree: N = 38,834 (45.2% of all texters); somewhat agree: N = 17,125 (19.9%); neither agree or disagree: N = 5806 (6.8%); somewhat disagree: N = 3001 (3.5%); strongly disagree: N = 2146 (2.5%); response missing: N = 18,965 (22.1%)).

TABLE 1.

Relationship between texter's perception of crisis counselor's genuine concern and their personal characteristics

	Texter’s perception of counselor’s genuine concern? a
	n (of yes)	Row % b	OR (95% Cl) c
Total (N = 66,912/85,877) d	55,959	83.6/65.2 b	–
Age
13 or younger (N = 5792)	5145	88.8	1.49 (1.38, 1.63) i
14–24 (N = 42,606) e	35,938	84.3
25–44 (N = 13,463)	10,860	80.7	0.76 (0.72, 0.80) i
45–64 (N = 2517)	2066	82.1	0.78 (0.70, 0.86)
65+ (N = 104)	90	86.5	1.11 (0.64, 1.92)
Gender
Female (N = 44,499) e	39,260	84.4
Male (N = 7831)	6580	84.0	1.01 (0.959, 1.08)
Gender minority (N = 4601) f	3858	83.9	0.99 (0.91, 1.08)
Missing (N = 7981)	6261	78.4	0.76 (0.66, 0.87) i
Race
Asian (N = 2123)	1739	81.9	0.78 (0.70, 0.88) i
Black or African American (4975)	4258	85.6	1.08 (1.00, 1.19)
Hispanic, Latinx, or Spanish origin (N = 7792)	6736	86.4	1.14 (1.05, 1.21)
White (N = 31,814) e	26,802	84.2
Middle Eastern, North African, or Arab + American Indian/Alaska Native + Native Hawaiian or Other Pacific Islander (N = 1215) g	1026	84.4	0.98 (0.84, 1.14)
More than one (N = 7025)	5993	85.3	1.05 (0.97, 1.12)
Other‐write in (N = 1258)	1049	83.4	0.91 (0.78, 1.05)
Prefer not to answer (N = 3192)	2396	75.1	0.61 (0.56, 0.67) i
Missing (N = 14,625)	5960	79.3	0.63 (0.50, 0.78) i
Sexual orientation
Heterosexual (N = 31,189) e	26,354	84.5
All other sexual identities (N = 28,198) h	23,607	83.7	0.96 (0.91, 1.00)
Missing (N = 7525)	5998	79.7	1.47 (1.17, 1.85) i
Depression symptoms
Below cutoff (N = 13,523) e	11,871	87.8
Above cutoff (N = 51,759)	42,715	82.5	0.78 (0.73, 0.83) i
Anxiety symptoms
Below cutoff (N = 12,636) e	10,955	86.7
Above cutoff (N = 52,666)	43,659	82.9	0.92 (0.87, 0.98)
Social isolation
Below cutoff (N = 17,311) e	15,207	87.8
Above cutoff (N = 47,938)	39,355	82.1	0.70 (0.66, 0.74) i
Suicide risk ladder
None (N = 51,490) e	43,158	83.8
Suicidal thoughts (N = 8955)	7402	82.7	0.94 (0.88, 0.99)
Plan (N = 2174)	1795	82.6	0.96 (0.86, 1.07)
Accessible means (N = 3387)	2818	83.2	0.97 (0.89, 1.06)
Timeframe withing 24 h (N = 906)	786	86.8	1.37 (1.13, 1.66) i
Topics discussed in conversation
Depression – No (N = 38,616) e	32,238	83.5
Depression – Yes (N = 28,296)	23,721	83.8	1.10 (1.05, 1.14)
Eating problems – No (N = 64,565) e	53,986	83.6
Eating problems – Yes (N = 2347)	1973	84.1	0.96 (0.86, 1.07)
Substance problems – No (N = 65,519) e	54,786	83.6
Substance problems – Yes (N = 1393)	1173	84.1	1.10 (0.95, 1.26)
Physical abuse – No (N = 65,680) e	54,898	83.6
Physical abuse – Yes (N = 1232)	1061	86.1	1.20 (1.02, 1.41) i
Sexual abuse – No (N = 65,120) e	54,413	83.6
Sexual abuse – Yes (N = 1792)	1546	86.3	1.21 (1.06, 1.38) i
Conversation frequency
1 Conversation (N = 42,960) e	35,856	83.5
2 to 10 Conversations (N = 22,742)	19,100	84.0	1.01 (0.97, 1.06)
More than 10 Conversations (N = 1210)	1003	83.0	0.95 (0.82, 1.12)

Responses to “I feel my counselor was genuinely concerned for my well‐being” were dichotomized so that “somewhat agree” and “strongly agree” were combined (“agree”) and “strongly disagree” “somewhat disagree,” and “neither agree or disagree” were combined (“did not agree”) in the analysis.

These are unweighted percentages.

The ORs are based on weighted analyses, with all covariates included simultaneously in the logistic regression models.

The total number of texters is 85,877, of which 66,912 had non‐missing responses on the outcome, “Texter's Perception of Counselor's Genuine Concern.” Unlike the Total row, the percentages of “yes” responses for each covariate value are based on the N of non‐missing responses as the denominator, not the total N. The percentages based on the non‐missing responses compared to the total N better reflect the pattern of ORs, which weight for missingness in the outcome.

Reference category.

The gender minority categories (agender, genderqueer, trans female, trans male, and other) were combined in the analyses.

The three racial groups with a prevalence of <2% (Middle Eastern/North African/Arab; American Indian/Alaska Native; Native Hawaiian/Other Pacific Islander) were combined in the analyses.

Asexual, gay/lesbian, pansexual, bisexual, questioning/unsure, other were combined in the analyses.

Bolded odds ratios are those that were statistically significant and considered to be “clinically meaningful” using an a priori threshold of 20% higher (or lower) odds ratio (i.e., ≥1.20 [and for lower odds ratios we used the reciprocal, ≤0.83]).

Compared to 14–24‐year‐old texters (who comprised the majority), those aged 13 or younger had significantly higher odds of agreeing that their CC was genuinely concerned for their well‐being (OR = 1.49; CI = 1.38, 1.63). Texters aged 25–44 had significantly lower odds of providing positive feedback about their CCs' genuine concern (OR = 0.76; CI = 0.72, 0.80). Compared to White texters, Asian texters had significantly lower odds of agreeing that their CC was genuinely concerned (OR = 0.78; CI = 0.70, 0.88). No other significant racial differences emerged; however, texters who preferred not to answer the question about their race or whose responses were missing had significantly lower odds of providing positive feedback. Texters with depression scores above the clinical cutoff had significantly lower odds of providing positive feedback about their CC than texters scoring below (OR = 0.78; CI = 0.73, 0.83). Similarly, texters reporting high on social isolation had significantly lower odds of reporting their CC as genuinely concerned for their well‐being than texters reporting low social isolation scores (OR = 0.70; CI = 0.66, 0.74). Texters with highest suicide risk had significantly higher odds of reporting that their CC had genuine concern for them compared to those with lowest suicide risk (OR = 1.37; CI = 1.13, 1.66). Texters who discussed physical or sexual abuse with their CC had significantly higher odds of providing positive feedback about their CC's genuine concern than texters who did not (OR = 1.20; CI = 1.02, 1.41, OR = 1.21; CI = 1.06, 1.38, respectively). Gender had no significant impact on the odds of providing positive feedback, with the exception of texters whose response on the gender question was missing; these texters had significantly lower odds of providing positive feedback (OR = 0.76; CI = 0.66, 0.87).

Texters' perceptions of CTL's effectiveness

Overall, 86.5% (n = 74,289) of texters found the conversation helpful (Table 2). Approximately, one‐third of texters reported feeling more hopeful at the end of the conversation (37.8%); 27.6% indicated feeling less depressed, 39.5% reported feeling less overwhelmed, and 46.1% reported feeling less suicidal. Few texters reported that their emotional states worsened (6.4% less hopeful, 5.9% more depressed, 6.3% more overwhelmed, 5.1% more suicidal).

TABLE 2.

Texter's perceptions of text's effectiveness

Effectiveness outcomes	N	% a
Did you find this conversation helpful?
Yes	74,289	86.5
No	11,597	13.5
Missing	0	0
Consider the feelings below and let us know if they changed after you texted with a crisis counselor today
Hopeful
Less	5503	6.4
Same	27,683	32.2
More	32,440	37.8
Missing	20,251	23.6
Depressed
Less	23,702	27.6
Same	36,795	42.8
More	5088	5.9
Missing	20,292	23.6
Overwhelmed
Less	33,922	39.5
Same	26,229	30.5
More	5442	6.3
Missing	20,283	23.6
Suicidal b
Less	9088	46.1
Same	4983	25.3
More	1010	5.1
Missing	4637	23.5

These are unweighted percentages based on the total sample.

Non‐suicidal texters omitted.

Relationship between texters' perceptions of CTL effectiveness and their characteristics and perceived genuine concern by crisis counselors

Compared to texters aged 14–24 years, older texters consistently had less favorable perceptions of the effectiveness of their conversations, indicated by the significance and direction of the odds ratios for all outcomes. Texters 25 or older had significantly and “clinically meaningful” lower odds than 14–24‐year‐olds of reporting positive outcomes (i.e., finding the conversation helpful or reporting feeling more hopeful) and significantly higher odds of reporting being more depressed, overwhelmed, or suicidal at the end of the conversation (Table 3). For example, 42.5% of 14–24‐year‐olds reported being less overwhelmed at the end of the conversation, while only 29.4% of 45–64‐year‐olds reported this outcome (Table S4). (See Tables S1–S5 for full tabulation of percentages for each covariate/outcome pair.) Similarly, 49.3% of 14–24‐year‐olds reported being less suicidal at the end of the conversation, whereas only 15.4% of texters 65 or older indicated this (Table S5). Conversely, texters younger than 14 years reported several significantly and clinically meaningful better outcomes than 14–24‐year‐olds. The youngest group had significantly higher odds than 14–24‐year‐olds of finding the text helpful (OR = 1.47; CI = 1.34, 1.61) and reporting being more hopeful at the end of the conversation (OR = 1.26; CI = 1.19, 1.33). However, among the subset of suicidal texters, those younger than 14 also had significantly higher odds than 14–24‐year‐olds of reporting themselves as more suicidal at the conversation's end (OR = 1.56; CI = 1.39, 1.75).

TABLE 3.

Relationship between texters' post‐text perceptions of text's effectiveness and their personal characteristics and perception of crisis counselor's genuine concern

	“Did you find this conversation helpful?”	“Consider the feelings below and let us know if they changed after you texted with a crisis counselor today”
	Yes	Hopeful?	Depressed?	Overwhelmed?	Suicidal?
	OR (95% Cl) a	OR (95% Cl) a	OR (95% Cl) a	OR (95% Cl) a	OR (95% Cl) a , b
N in analysis	84,133	64,370	64,375	64,385	14,875
Age
13 or younger	1.47 (1.34, 1.61) i	1.26 (1.19, 1.33) i	0.93 (0.88, 0.99)	1.15 (1.08, 1.21)	1.56 (1.39, 1.75) i
14–24 c
25–44	0.78 (0.73, 0.83) i	0.78 (0.75, 0.81) i	1.25 (1.20, 1.30) i	1.25 (1.20, 1.30) i	1.14 (1.04, 1.25)
45–64	0.83 (0.74, 0.93) i	0.70 (0.65, 0.76) i	1.42 (1.30, 1.55) i	1.57 (1.45, 1.70) i	1.39 (1.15, 1.67) i
65+	0.59 (0.37, 0.96) i	0.59 (0.40, 0.88) i	1.40 (0.92, 2.14) i	2.01 (1.36, 2.96) i	4.84 (1.74, 13.5) i
Gender
Female c
Male	1.03 (0.96, 1.12)	1.05 (1.00, 1.12)	0.92 (0.88, 0.97)	1.19 (1.14, 1.25)	1.13 (1.02, 1.25)
Gender minority d	1.17 (1.05, 1.30)	1.03 (0.97, 1.10)	0.95 (0.89, 1.01)	0.81 (0.76, 0.86)	0.89 (0.78, 1.00)
Missing	0.75 (0.65, 0.87) i	0.96 (0.85, 1.08)	0.98 (0.86, 1.11)	1.03 (0.91, 1.16)	1.15 (0.89, 1.47)
Race
Asian	1.01 (0.87, 1.17)	0.96 (0.88, 1.05)	0.97 (0.88, 1.06)	1.06 (0.97, 1.16)	1.15 (0.94, 1.40)
Black or African American	0.92 (0.84, 1.02)	0.98 (0.92, 1.04)	1.02 (0.95, 1.08)	1.20 (1.13, 1.28) i	1.23 (1.08, 1.41) i
Hispanic, Latinx, or Spanish origin	1.28 (1.18, 1.40) i	1.06 (1.01, 1.12)	0.90 (0.85, 0.94)	1.03 (0.98, 1.08)	1.07 (0.95, 1.20)
White c
Middle Eastern, North African, or Arab + American Indian/Alaska Native + Native Hawaiian or Other Pacific Islander e	1.05 (0.87, 1.26)	1.06 (0.94, 1.187)	1.16 (1.03, 1.30)	1.21 (1.08, 1.36) i	1.02 (0.78, 1.32)
More than one	1.14 (1.04, 1.24)	1.10 (1.04, 1.16)	0.93 (0.88, 0.98)	0.95 (0.91, 1.01)	1.12 (1.00, 1.26)
Other‐write in	1.05 (0.88, 1.25)	0.98 (0.87, 1.09)	0.96 (0.85, 1.07)	1.06 (0.95, 1.19)	1.21 (0.96, 1.54)
Prefer not to answer	0.71 (0.64, 0.80) i	0.86 (0.79, 0.93)	1.13 (1.04, 1.22)	1.17 (1.08, 1.26)	1.21 (1.02, 1.44) i
Missing	0.93 (0.76, 1.14)	1.07 (0.89, 1.28)	0.97 (0.80, 1.16)	0.91 (0.76, 1.08)	1.21 (0.82, 1.79)
Sexual orientation
Heterosexual c
All other sexual identities f	0.99 (0.94, 1.05)	1.00 (0.97, 1.04)	1.06 (1.02, 1.10)	0.90 (0.87, 0.93)	1.09 (1.01, 1.18)
Missing	0.89 (0.72, 1.10)	0.88 (0.73, 1.05)	1.12 (0.93, 1.34)	1.22 (1.02, 1.46) i	0.92 (0.62, 1.37)
Depression symptoms
Below cutoff c
Above cutoff	0.70 (0.64, 0.76) i	0.71 (0.68, 0.75) i	2.24 (2.14, 2.35) i	1.40 (1.33, 1.46) i	1.87 (1.63, 2.14) i
Anxiety symptoms
Below cutoff c
Above cutoff	0.97 (0.90, 1.04)	0.91 (0.87, 0.95)	1.40 (1.34, 1.46) id	1.37 (1.31, 1.43) i	1.22 (1.09, 1.36) i
Social isolation
Below cutoff c
Above cutoff	0.73 (0.68, 0.79) i	0.79 (0.76, 0.82) i	1.46 (1.41, 1.52) i	1.23 (1.18, 1.28) i	1.28 (1.15, 1.42) i
Suicide risk ladder
None c	–	–	–	–	b
Suicidal thoughts g	0.91 (0.85, 0.97)	0.86 (0.82, 0.90)	1.21 (1.16, 1.27) i	1.06 (1.01, 1.11)	–
Plan	0.74 (0.65, 0.83) i	0.75 (0.69, 0.82) i	1.44 (1.32, 1.58) i	1.18 (1.08, 1.28)	1.30 (1.18, 1.44) i
Accessible means	0.90 (0.81, 0.99)	0.75 (0.70, 0.80) i	1.42 (1.32, 1.53) i	1.17 (1.09, 1.26)	1.25 (1.15, 1.36) i
Timeframe withing 24 h	0.83 (0.69, 1.01)	0.82 (0.72, 0.93) i	1.23 (1.07, 1.41) i	1.21 (1.06, 1.39) i	1.03 (0.89, 1.20)
Topics discussed in conversation
Depression	1.02 (0.97, 1.07)	0.99 (0.96, 1.03)	1.11 (1.08, 1.14)	1.04 (1.01, 1.07)	1.00 (0.94, 1.07)
Eating problems	1.01 (0.89, 1.15)	1.09 (1.00, 1.18)	0.96 (0.88, 1.04)	1.01 (0.93, 1.10)	0.97 (0.77, 1.17)
Substance problems	1.03 (0.87, 1.20)	1.08 (0.97, 1.20)	0.95 (0.85, 1.06)	0.98 (0.89, 1.10)	1.07 (0.86, 1.34)
Physical abuse	1.26 (1.04, 1.52) i	1.21 (1.08, 1.36) i	0.95 (0.84, 1.06)	1.10 (0.98, 1.24)	0.75 (0.58, 0.96) i
Sexual abuse	1.18 (1.01, 1.38)	1.09 (0.99, 1.20)	0.94 (0.85, 1.03)	0.95 (0.86, 1.04)	0.98 (0.80, 1.19)
Texter's perception of counselor's
Genuine concern h
Disagree c
Agree	23.93 (22.62, 25.32) i	7.20 (6.89, 7.54) i	0.18 (0.17, 0.18) i	0.22 (0.21, 0.23) i	0.16 (0.14, 0.17) i
Conversation frequency
1 Conversation c
2–10 Conversations	0.89 (0.85, 0.93)	0.86 (0.84, 0.89)	1.14 (1.11, 1.18)	1.14 (1.10, 1.17)	1.60 (1.49, 1.72) b
More than 10 conversations	0.83 (0.71, 0.98) i	0.75 (0.67, 0.84) i	1.33 (1.17, 1.50) i	1.26 (1.12, 1.42) i	2.72 (2.16, 3.41) i

The ORs are based on weighted analyses, with all covariates included simultaneously in the logistic regression models.

Non‐suicidal texters omitted from analyses.

Reference category.

The gender minority categories (agender, genderqueer, trans female, trans male, other) were combined in the analyses.

The three racial groups with a prevalence of <2% (Middle Eastern/North African/Arab; American Indian/Alaska Native; Native Hawaiian/Other Pacific Islander) were combined in the analyses.

Asexual, gay/lesbian, pansexual, bisexual, questioning/unsure, other were combined in the analyses.

“Suicide thoughts” was the reference group for the analysis of the suicide outcome.

Responses to “I feel my counselor was genuinely concerned for my well‐being” were dichotomized so that “somewhat agree” and “strongly agree” were combined (“agree”) and “strongly disagree” “somewhat disagree,” and “neither agree or disagree” were combined (“did not agree”) in the analysis.

Bolded odds ratios are those that were statistically significant and considered to be “clinically meaningful” using an a priori threshold of 20% higher (or lower) odds ratio (i.e., ≥1.20 [and for lower odds ratios we used the reciprocal, ≤0.83]).

No associations of gender with outcomes met our criterion of a “clinically meaningful” odds ratio; only texters with missing responses had significantly lower odds than female texters of reporting the conversation as helpful (OR = 0.75; CI = 0.65, 0.87). Similarly, regarding sexual orientation, only texters with missing responses had significantly higher odds than heterosexual texters of reporting feeling more overwhelmed at the conversation's conclusion (OR = 1.22; CI = 1.02, 1.46).

Few significant and clinically meaningful associations between race and outcomes emerged: Black/African American texters had higher odds than White texters of reporting being more overwhelmed (OR = 1.21; CI = 1.13, 1.28) or suicidal (OR = 1.23; CI = 1.08, 1.41) at the conversation's conclusion. Hispanic texters had higher odds than White texters of reporting their conversations as helpful (OR = 1.28; CI = 1.18, 1.40). However, absolute differences in the percentages of reported outcomes between White and Black/African or Hispanic texters were small. For example, 8.3% of Black/African texters reported being more overwhelmed at the conversation's conclusion compared to 6.1% of White texters (Table S4). Similarly, 6.8% of Black/African American texters reported being more suicidal at the end of the conversation compared to 4.7% of White texters (Table S5). Of Hispanic texters, 9.4% reported that the conversation was not helpful compared to 12.5% of White texters (Table S1).

Texters who scored above the clinical cutoff on the depression, anxiety, and social isolation scales had significantly lower odds than their less symptomatic counterparts of reporting positive outcomes (i.e., finding the conversation helpful or reporting feeling more hopeful after the text) and significantly higher odds of reporting being more depressed, overwhelmed, or suicidal at the conversation's conclusion (Table 3). For example, 49.7% of suicidal texters who also scored above the cutoff for depression reported feeling less suicidal, and 6.1% indicated being more suicidal at the end of the conversation, whereas 64.3% of suicidal texters who did not score above the depression threshold reported being less suicidal and 2.4% reported being more suicidal (Table S5).

Crisis counselors' ratings of texters on the Suicide Risk Ladder were statistically significantly associated with outcomes. Suicidal texters had statistically lower odds than non‐suicidal texters of reporting the conversation as helpful, or of reporting feeling more hopeful at the end (Table 3). They also had higher odds of reporting feeling more depressed, overwhelmed, or suicidal. For example, 9.5% of texters with a suicide plan or accessible means reported being more depressed at the end of the conversation, and approximately 21% of each group reported being less depressed. Among non‐suicidal texters, 5.3% reported feeling more depressed and 29% indicated they were less depressed at the end of the conversation (Table S3).

Topics that texters discussed with CCs (depression, eating problems, substance problems, physical abuse, sexual abuse) were not associated with texters' perceptions of their conversations' effectiveness, with the exception of texters who discussed physical abuse. These texters had higher odds of reporting finding conversations helpful (OR = 1.26; CI = 1.04, 1.52), or that they were more hopeful (OR = 1.21; CI = 1.08, 1.36) or less suicidal (OR = 0.75; CI = 0.58, 0.96) at the conversation's conclusion.

Texters' perception of CCs' concern was significantly and meaningfully associated with their perceptions of the text's effectiveness, yielding the largest associations with all outcomes compared to any other covariate. More positive feedback about CCs was found to yield nearly 24 times higher odds of texters reporting their conversations were helpful, and 7.2 times higher odds of feeling more hopeful at the end of the conversation. Moreover, texters who perceived their CCs as having genuine concern for them had lower odds of being more depressed (OR = 0.18; CI = 0.17, 0.18), overwhelmed (OR = 0.22; CI = 0.21, 0.23), or suicidal (OR = 0.16; CI = 0.14, 0.17) at the end of the conversation (Table 3). The percentage of texters who reported their conversations to be helpful was 14.5% if they strongly disagreed that their CC was genuinely concerned for their well‐being, 34.2% if they somewhat disagreed, 61.3% if they neither agreed or disagreed, 88.4% if they somewhat agreed, and 98.1% if they strongly agreed (Table S1). Similarly, the percentage of texters who reported being more hopeful by the conversation's conclusion was 8.9% if they strongly disagreed that their CC was genuinely concerned for their well‐being, 11.5% if they somewhat disagreed, 18.9% if they neither agreed or disagreed, 38.4% if they somewhat agreed, and 62.0% if they strongly agreed (Table S2).

Conversation frequency was significantly associated with texters' perceptions of effectiveness, as indicated by the significance and direction of odds ratios for all outcomes. Compared to texters with one conversation, those with 2–10 conversations or >10 had 1.6 and 2.7 times the odds of being more suicidal at the end of the conversation, respectively. Similarly, texters with >10 conversations had higher odds of reporting being more depressed (OR = 1.33; CI = 1.17, 1.50), overwhelmed (OR = 1.26; CI = 1.12, 1.42), or suicidal (OR = 2.72; CI = 2.16, 3.42) at the conversation's conclusion. Conversely, they had lower odds of reporting the conversation as helpful (OR = 0.83; CI = 0.71, 0.98) or feeling more hopeful (OR = 0.75; CI = 0.67, 0.84) (Table 3) at the end. Among suicidal texters with >10 conversations, 10.3% reported being more suicidal at the end of the conversation, compared to 6.8% of those with 2–10 conversations and 3.9% of those with only one (Table S5).

DISCUSSION

Our findings from 85,877 texters indicated that most people who text CTL perceive their CC as having genuine concern for them. Most CTL texters (86.5%) found their conversations with a CC helpful, with a substantial minority reporting feeling more hopeful (37.8%), less depressed (27.6%), less overwhelmed (39.5%), and suicidal texters reporting feeling less suicidal (46.1%). A broad theme across our findings was their similarity to results of the evaluation of Lifeline's chat crisis services (Gould, Chowdhury et al., 2021), which provides a key benchmark due to similarity in measures and setting, something found rarely in this field. In comparison to current CTL findings, that evaluation found that 66.8% of Lifeline chatters reported finding their conversation helpful, 31.5% were more hopeful, 26.8% were less depressed, 37.5% were less overwhelmed, and 45% of suicidal chatters were less suicidal. When interpreting comparisons with the chat evaluation, it should be noted that the CTL sample was 23% suicidal, whereas the Lifeline chat sample was almost 84% suicidal. As the current findings note, suicidal texters had somewhat less positive outcomes than non‐suicidal texters, which might partially explain the slightly less positive perceptions among Lifeline chatters, most of whom were suicidal, compared to CTL texters—most of whom were not.

Texter perceptions of CCs' genuine concern for them had the strongest associations with perceived effectiveness of the text conversation in the current study. This association is highlighted in earlier studies of crisis interventions. Mishara et al. (2007) found that counselors' behaviors underlying callers' engagement with counselors were significantly related to positive outcomes for crisis callers. Similarly, an earlier Lifeline evaluation (Gould et al., 2016) found that higher levels of engagement enhanced counselors' chances of mitigating imminent suicide risk through collaborative interventions and reduced needing to involve emergency services. Lifeline chatters' engagement with counselors—measured by the feeling that their counselors were genuinely concerned for their well‐being and understood them, or that they agreed with the counselor on plans—had the strongest associations with chatters' perceptions that the chat was effective. Counselors' success in engaging callers, chatters, and texters may facilitate “comforting communication” to reduce emotional distress (Caplan & Turner, 2007). These findings support CTL's emphasis on training volunteer CCs in building rapport using “good contact” techniques (Crisis Text Line, 2017) and Lifeline's policy on good contact and active engagement with all callers (Draper et al., 2015).

Younger texters were more likely to find the conversation helpful than older texters, despite controlling for younger texters' more favorable perception of the CC's genuine concern and the remaining covariates. Older texters were more likely to be less hopeful, and more depressed, overwhelmed, and suicidal at the end of the conversation. While CTL appears to address the needs of individuals aged 24 years or younger, who represent three‐quarters of CTL texters, more attention is needed to appeal to and more effectively intervene with texters across the lifespan. Lifeline's chat services were similarly more effective for minors than older age groups (Gould, Chowdhury et al., 2021).

To our knowledge, this is the first study examining the association of a crisis service's effectiveness with a client's race (although limited information is available regarding association with ethnicity). Thus, the current study addresses a critical knowledge gap about disparities in health care quality in the U.S. (Fiscella & Sanders, 2016). Texters who identified as Black or African American had higher odds than White texters of reporting being more overwhelmed or suicidal at the end of the conversation, even after controlling for all covariates. However, the magnitude of the differences in outcomes reported by Black or White texters was relatively small and, importantly, no differences emerged concerning perceptions of CCs' genuine concern, the conversation's overall helpfulness, or their hopefulness or depression at the conversation's conclusion. Texters who identified as Hispanic/Latinx/of Spanish origin provided more favorable feedback than White texters about the helpfulness of CTL's conversations. This is consistent with findings from an evaluation of Lifeline's crisis center follow‐up initiative showing that Hispanic callers had a more positive perception of the intervention—specifically, they had higher odds than non‐Hispanic callers of saying that follow‐up calls stopped them from killing themselves and kept them safe (Gould et al., 2018).

Given the high prevalence of mental health symptoms among CTL texters and the small proportion who reported receiving professional help (Pisani et al., 2022), our findings underscore a need to focus within and beyond CTL on long‐term needs of texters with the most severe mental health concerns. Several of the current findings highlight that the most distressed texters (those with high depression, anxiety, social isolation and frequent texting) felt less relieved at the end of their crisis conversation than asymptomatic texters or those who texted only once during the study period. Depressed, anxious and socially isolated texters had lower odds than less symptomatic texters of reporting the conversation as helpful or feeling more hopeful afterward and had significantly higher odds of reporting feeling more depressed, overwhelmed, or suicidal at the end of the conversation. Individuals who frequently texted (i.e., 10 or more times in the one‐year study period) had a similar pattern of results as symptomatic texters. Providing relevant and accessible referrals to texters is a key component of CTL's training (Crisis Text Line, 2017); its enhancement is recommended to build more effective ways of connecting the most distressed texters with various kinds of follow‐up care and evidence‐based interventions.

Crisis Text Line's ability to reduce suicidality in nearly half of suicidal texters can be considered a clinically meaningful accomplishment. This reduction in risk at a moment of crisis can provide texters with an opportunity to seek further help and engage in additional interventions that could save lives and have prolonged impact. Our finding that nearly half of suicidal texters reported feeling less suicidal after the text conversation is comparable to the reduction in suicidal ideation among chatters at the end of Lifeline crisis chat interventions (Gould, Chowdhury et al., 2021). This level of improvement mirrors the range of improvements reported after psychotherapy (Strauss et al., 2021). In contrast to CTL's positive outcomes, yet consistent with Lifeline crisis chat outcomes (Gould, Chowdhury et al., 2021), a small minority of all texters did not feel their conversations were helpful (13.5%) or felt less hopeful (6.4%), more depressed (5.9%), more overwhelmed (6.3%) or more suicidal (5.1%) after their conversations. While it would be ideal to have all or nearly all texters experiencing an improvement at the end of conversations, this is an unrealistic expectation given this population's distress levels (Pisani et al., 2022) and similar findings of worsening symptoms for a small portion of patients following psychotherapy (Strauss et al., 2021).

This study has several limitations. Firstly, only 22.5% of texters completed the voluntary post‐conversation survey. The survey weighting strategy we explored did not add meaningfully to our understanding of the population (Pisani et al., 2022). However, there are likely to be other factors, not measured in this study, that are associated with the completion of the survey and could yield some unknown bias. Second, we defined our sample for the current evaluation based on whether texters answered the first item in the survey. While this choice produced the largest sample size, it introduced a great deal of item‐level missingness. This was the rationale for conducting weighted analyses to account for missingness in both covariates and outcomes. Third, this evaluation focused on post‐conversation surveys from texters who exchanged at least 10 messages in their conversation. Thus, we do not know anything about individuals who disengaged before reaching a CC or abandoned the conversation before exchanging 10 messages. Finally, this evaluation covers short‐term self‐reported outcomes, and includes no follow‐up beyond the immediate post‐conversation period.

Notwithstanding these limitations, the study has numerous strengths. This is the first study to estimate the effectiveness of CTL, the largest provider of crisis text‐based services in the U.S. Furthermore, the size and diversity of the sample provided enough statistical power to assess the differential effectiveness of the intervention by race—the first such assessment for any crisis intervention. The large sample size also allowed us to examine the relationship of the outcomes with several texter characteristics and their perceived engagement with their CCs, crucial elements to person‐centered care.

In conclusion, our study offers novel empirical evidence for the short‐term effectiveness of CTL's services. The vast majority of texters reported that the conversation was helpful, and a sizeable minority reported feeling more hopeful, less depressed, less overwhelmed, and less suicidal after their conversation. The finding that nearly half of suicidal texters reported feeling less suicidal after the intervention aligns with Lifeline crisis chat intervention's reduction of suicidal risk and may represent the level of accomplishment that is feasible in one crisis session. Our findings also provide some empirical evidence for CTL's success in addressing unique needs of racially and ethnically diverse populations. Overall, texters across different races and ethnicities perceived their CTL CCs as having genuine concern for them and the effectiveness of the service was largely similar for all texters. This is a step toward overcoming racial and ethnic disparities in mental health care in the U.S. (Substance Abuse and Mental Health Services Administration, 2015).

CONFLICT OF INTEREST

MSG is an uncompensated member of Crisis Text Lines' Clinical Advisory Board. AP and CG are uncompensated members of Crisis Text Lines' Data Advisory Board. AE, DH, and CK have no conflicts to declare. SG is employed by the Crisis Text Line.

Supporting information

Tables S1–S5

Gould, M. S. , Pisani, A. , Gallo, C. , Ertefaie, A. , Harrington, D. , Kelberman, C. , Green, S. (2022). Crisis text‐line interventions: Evaluation of texters' perceptions of effectiveness. Suicide and Life‐Threatening Behavior, 52, 583–595. 10.1111/sltb.12873