Abstract
Objective
As part of the National Suicide Prevention Lifeline’s crisis response system, the Lifeline Crisis Chat Network (LCC) answers chats from hundreds of thousands of at‐risk individuals yearly. The study’s objective was to assess the effectiveness of these online crisis interventions.
Method
Data from 39,911 pre‐chat surveys and 13,130 linked pre‐ and post‐chat surveys completed by LCC chatters from October 2017–June 2018 were analyzed. The relationship of several effectiveness measures with chatter demographics, pre‐chat distress, suicidal ideation, and chatters’ perceptions of engagement with their counselors was examined using a series of logistic regression analyses.
Results
Chatters were significantly and substantially less distressed at the end of the chat intervention than at the beginning. By the end of the chat, two‐thirds of suicidal chatters reported that the chat had been helpful, while just under half reported being less suicidal.
Conclusions
Our study offers empirical evidence for the Lifeline's online crisis chat services’ effectiveness, but also highlights areas for improvement. This is of critical import in light of the recent designation of 988 as the nationwide number for the Lifeline beginning in 2022, which will increase the Lifeline's prominence in providing suicide prevention and mental health crisis interventions in the United States.
Keywords: crisis chat, effectiveness, lifeline, suicide
INTRODUCTION
The National Suicide Prevention Lifeline (“Lifeline”) is increasingly recognized as a critical component of the mental health and suicide crisis response system in the United States (e.g., U.S. Department of Health and Human Services, 2012, pp. 54, 57, 99). Now that the National Suicide Hotline Designation Act of 2020 has been signed into law (https://www.congress.gov/116/plaws/publ172/PLAW‐116publ172.pdf), designating 988 as the nationwide number for the Lifeline beginning in 2022, the Lifeline will have an ever‐increasing role in providing suicide prevention and mental health crisis interventions in the U.S. Assessment of the Lifeline’s effectiveness is more important than ever. Empirical evidence to support the effectiveness of the Lifeline’s telephone interventions has steadily grown since its inception (Gould et al., 2007; 2012; 2013; 2016; 2018). For example, caller’s emotional distress, and level of suicidality generally improves over the course of the Lifeline call (Gould et al., 2007, 2013; Ramchand et al., 2017).
To increase access to crisis services, Lifeline's service format has evolved to include not only telephone but also synchronous (i.e., in real time) chat crisis interventions. This is particularly critical for young people who are more likely to choose online rather than telephone crisis services (Haner & Pepler, 2016; Mokkenstorm et al., 2017), and who are also more likely to discuss “weighty problems,” such as mental health problems and suicide on an online crisis service than by telephone (e.g., Fukkink & Hermanns, 2009a; Haner & Pepler, 2016). The LCC is one of several online crisis services that has emerged in the past 15 years (e.g., Kids Help Line in Australia (King, Bambling, Reid, et al., 2006); Kids Help Phone in Canada (Haner & Pepler, 2016); SAHAR (Barak & Bloch, 2006) and ERAN (Gilat & Shahar, 2007) in Israel; Kindertelefoon (Fukkink & Hermanns, 2009a, 2009b) and 113Online (Mokkenstorm et al., 2017) in the Netherlands; Nevada Crisis Call Center (Evans et al., 2013) and the Veterans Crisis Line (Predmore et al., 2017) in the United States. Many of these specifically focus on chat crisis services for youth (e.g., Evans et al., 2013; Fukkink & Hermanns, 2009a; Haner & Pepler, 2016; King, Bambling, Reid, et al., 2006). The Lifeline Crisis Chat (LCC) network, which serves all ages, has grown extensively since its formal establishment in 2013, answering 231,335 chats in 2020.
With the rapid and extensive growth of online crisis services, there has been a parallel increase in research examining these services. Most studies have focused on the counselors’ interpretations and ratings of the intervention (e.g., Bambling et al., 2008; Predmore et al., 2017), the chatters’ motivations for using the services (King, Bambling, Lloyd, et al., 2006; Fukkink & Hermanns, 2009b; Predmore et al., 2017), and the content and clinical framework of chat interventions (e.g., Barak & Bloch, 2006; Chardon et al., 2011; Fukkink, 2011). However, there is a scarcity of research examining the effectiveness of these services. Fukkink and Hermanns (2009a, 2009b), in two studies of the Dutch Kindertelefoon youth helpline service, assessed repeated measures of over 300 children’s ratings of their well‐being and perceived burden of their problems before and after the chats. They found significant improvements in well‐being and decreases in perceived burden of problems. Similarly, King, Bambling, Lloyd, et al. (2006) examined repeated measures of emotional distress among 100 youth using the online counseling provided by the Kids Help Line in Australia and found that the youth were significantly less distressed at the end of the chat than they were at the beginning. These studies of young chatters are the only ones that assessed the effectiveness of online chat services directly from the perspective of the chatters. None of the studies examined the effectiveness of the chats specifically among suicidal chatters. One additional study of effectiveness examined the 113Online crisis chat service in the Netherlands (Mokkenstorm et al., 2017) by coding 526 chat transcripts, using methods and measures that had been developed for silent monitoring of telephone helplines (Mishara et al., 2007a, 2007b). The 113 Online service is not specifically targeted to youth, although it serves a predominantly young population. Several emotional states and suicidality of the chatters were coded in the first and last 10 min of the chat. Key findings included an improvement in several different emotional states among one‐third to nearly half of chatters. No change in emotional states was observed more frequently (43.2%–64.2%), and deterioration in chatters’ emotional states was relatively rare. The coding of suicidal ambivalence was missing for nearly two‐thirds of the chatters, so it is difficult to conclude anything about this outcome.
Formal evaluations of LCC services have only recently become the focus of research reports. Examining 1034 transcripts from chat sessions in 2015, Lake et al. (2021) found that crisis chat counselors adhered to Lifeline’s crisis intervention model—building rapport and collaborating with chatters to identify coping strategies and solutions to the chatter’s problems. However, the study also found that suicide risk assessments were not conducted on all chats, as recommended by the Lifeline. The current study complements this earlier LCC evaluation by directly assessing the chatter’s perspective of the chat’s effectiveness. To our knowledge, this is the first study to employ the chatters’ self‐reports to assess the effectiveness of an online service that targets all age groups. In addition to assessing the effectiveness of the chat intervention from the perspective of the chatter, we also aimed to assess whether chatter characteristics, including their suicidality, were associated with their perceptions of the chat’s effectiveness, and whether the chatter’s perceived engagement with the crisis counselor impacted their perceived effectiveness of the chat. These findings will provide critically important information to enhance the Lifeline’s chat services specifically, and online crisis services more generally.
METHODS
Sample
The seven centers that provided the core infrastructure of the LCC nationwide network, accounting for 72% of the chat interventions in the LCC during the study period, October 2017—June 2018, were recruited for participation in the current study. However, one of the centers was unable to implement the Lifeline's expanded post‐chat survey (described in a following section), which was a criterion for participation. Therefore, six of the seven core centers participated in the current study, accounting for 65% of the chat interventions across the LCC network, which had an additional 18 centers during the study period. A total of N = 41,061 chats were connected to chat counselors at the six centers during the study period. All of the chatters completed a pre‐chat survey. Based on the chatter's description of their main concern on the pre‐chat survey, 1083 third‐party chats (from individuals seeking help on behalf of others), 51 informational chats, and 16 prank chats were excluded from our sample, leaving a total of 39,911 chats which to the best of our knowledge are from chatters seeking help on their own behalf. These chats are the focus of the present study. In addition to the pre‐chat survey, 13,130 (32.9%) of these chats are associated with a completed post‐chat survey. The assessments are described below. The data were de‐identified in such a way as to make it impossible to identify multiple chats from the same chatter, which means our sample consists of chat surveys rather than unique chatters. Nevertheless, for ease of communication, we use the term “chatter” rather than “chat survey” throughout the paper.
Procedures
Lifeline Chat is accessed through a central portal located on the Lifeline website (https://suicidepreventionlifeline.org/chat). Before chatters are connected to a counselor, they are required to complete a short pre‐chat survey and agree to the terms of service. Following the chat session, chatters are prompted to complete another survey via an external link that gets uploaded to a data warehouse along with pre‐chat data. There is no mention of the post‐chat survey prior to reaching the end of the chat. Completion of the post‐chat survey is voluntary. Chatters have to opt in by clicking on the link before the survey will open. The message that introduces the survey is “Thank you for sharing with us today. If you're comfortable, we'd love your feedback on this conversation so we can continue to improve the service we provide. You can leave your feedback here: [link].” Our study protocol involved a secondary data analysis of linked, de‐identified pre‐ and post‐chat survey data provided to the study team by the Lifeline.
Measures
The Pre‐Chat Survey designed by the Lifeline included demographic questions about the chatter’s age (open field, categorized into four groups for analyses: ≤17, 18–24, 25–34, and ≥35) and gender (male, female, transgender, questioning, genderqueer, and other). These gender categories were developed in conjunction with the Trevor Project, the leading national organization providing crisis intervention services to LGBTQ youth (https://www.thetrevorproject.org/). In addition, the pre‐chat survey included the following questions: (1) “What is your main concern?” (The chatter was instructed to choose one from a list of 13 including “other”. They also had the opportunity to describe their main concern in a free text field.) (2) “Do you have thoughts of suicide?” (“yes – current (within the past 24 h)”; “yes – recent (within the past few days)”; “no”) and (3) “How upset are you?” (responses on a 5‐point Likert scale: 1 = I’m doing OK; 2 = a little upset; 3 = moderately upset; 4 = very upset; and 5 = extremely upset).
A Post‐Chat Survey, which was expanded by the Lifeline with the assistance of the study team, included the following questions: (1) “Now that you have finished your chat session, how upset are you?” (responses using same 5‐point Likert scale as in the pre‐chat survey); (2) “Did you find this chat service helpful?” (yes/no); (3) “After the conversation, how do you feel?” (responses on a 3‐point Likert scale of “less” or “same” or “more” for each of the following: “hopeful,” “depressed,” “overwhelmed,” and “suicidal”); and (4) three questions that assessed the chatter's perceptions of engagement with their counselor: (i) “I feel my chat counselor was genuinely concerned for my well‐being”; (ii) “I feel my chat counselor understood me.” (Each of these statements was responded to on a 4‐point scale from “strongly agree,” “agree,” “disagree,” and “strongly disagree”); and (iii) “Did you and your chat counselor agree on any plans for after the call (like things you can do or someone you can contact)?” (yes/no). These questions were based on assessments employed in our earlier Lifeline evaluations (e.g., Gould et al., 2007, 2013).
The project's protocol was approved by the Institutional Review Board of the New York State Psychiatric Institute and the Department of Psychiatry of Columbia University.
Statistical analyses
The statistical analyses were conducted using SAS version 9.4 (SAS Institute Inc.) and R version 4.0.0 (2020‐04‐24).
Descriptive statistics were calculated for all pre‐chat variables using the entire dataset to identify any inconsistent values. Chats with a post‐chat survey were compared to those without on all pre‐chat variables using multi‐predictor binary logistic regression to determine whether those answering the post‐chat survey differed from those who did not. The area under the receiver operator curve (AUC or C‐Statistics) was estimated to measure the model’s discrimination of chats with and without post‐chat survey, at different cutoff values of the predicted probability of answering the post‐chat survey. This estimate was used to decide whether the post‐chat analyses needed to be weighted to balance out under‐representation in the post‐chat survey of any particular types of pre‐chat records.
The remaining analyses focused on the subset of linked pre‐ and post‐chat survey data. First, records missing more than 50% of the pre‐ and post‐chat variables were excluded (1.3%, remaining n = 12,957). Next, multiple imputation was used to impute all missing data for the remaining subset, which uses information from other variables in the incomplete record to appropriately estimate the missing value, based on its associations with other variables in the dataset on the non‐missing records. Multiple Imputation with Chained Equations (MICE) package in the R (van Buuren & Groothuis‐Oudshoorn, 2011) was used to perform multiple imputation with proportional odds logistic regression (POLR), polytomous regression, binary logistic regression, and predictive mean matching to impute ordinal, categorical, binary, and continuous variables, respectively. All pre‐chat and post‐chat variables were included in the imputation model. Five imputed datasets were generated, analyzed as described below, and the results were pooled among the versions using the functions provided in the same package.
Associations were tested in a pairwise fashion between six post‐chat variables assessing effectiveness (how upset the chatter was after the chat; whether the chatter thought the chat intervention was helpful; the extent to which the chatter was more or less “hopeful,” “depressed,” “overwhelmed,” and “suicidal”) as dependent variables and the following independent variables: age, gender, main concern, pre‐chat suicidal thoughts, and the three post‐chat questions assessing the chatter's perceptions of engagement with the counselor. Analyses of the suicidal outcome only included the chatters who endorsed current or recent suicidal thoughts on the pre‐chat survey. Binary and proportional odds logistic regression models were estimated, depending on the nature of the dependent variable. All models were adjusted for center as a fixed effect because the low number of centers precluded the use of a random effect (hierarchical or nested) model. Odds ratios with 95% confidence intervals were calculated for each outcome and predictor combination. The Benjamini–Hochberg (BH) method was used to adjust the p‐values from these models for multiple testing. This method assures the preservation of the false discovery rate for the whole experiment at 5%. Furthermore, given the large sample size, many statistically significant odds ratios would be expected; therefore, we chose to focus on statistically significant associations with a 25% change in the odds for 5‐point Likert scale variables and 50% change for 3‐point Likert scale and binary variables. Lastly, chatters’ answers before and after the chat to the question “how upset are you?” (using the same 5‐point Likert scale response) were compared using Wilcoxon matched pairs signed rank test.
RESULTS
Description of crisis chatters
Two‐thirds of crisis chatters were female, and almost 8% of the sample identified themselves as gender minorities (see Table 1). The chatters were young—nearly 40% were minors, and over 70% were younger than 24 years. Over half reported depression as the main issue for which they contacted the LCC, with non‐suicidal self‐harm or anxiety each reported by 1 in 11 chatters. Current or recent suicidal thoughts were endorsed by over 80% of the chatters. Almost 60% of chatters reported that they were either very or extremely upset, with another 30% indicating that they were moderately upset.
TABLE 1.
Chatters’ pre‐chat characteristics: demographics, main concerns, suicidal ideation, and level of distress (N = 39,911)
| N | % | |
|---|---|---|
| Gender | ||
| Male | 10,190 | 25.5 |
| Female | 25,972 | 65.1 |
| Transgender | 1158 | 2.9 |
| Questioning | 826 | 2.1 |
| Genderqueer | 509 | 1.3 |
| Other | 475 | 1.2 |
| Missing | 781 | 1.9 |
| Age (years) | ||
| <17 | 15,576 | 39.0 |
| 18–24 | 12,922 | 32.4 |
| 25–34 | 6795 | 17.0 |
| >35 | 4278 | 10.7 |
| Missing | 340 | 0.9 |
| Main concerns | ||
| Depression | 19,642 | 49.2 |
| Non‐Suicidal Self‐Harm | 3454 | 8.7 |
| Anxiety | 3468 | 8.7 |
| Family issues | 2620 | 6.6 |
| Relationship issues | 2049 | 5.1 |
| Bullying/Problems in school | 957 | 2.4 |
| Physical, Sexual and/or Emotional abuse | 962 | 2.4 |
| Financial issues | 497 | 1.3 |
| Eating disorder | 438 | 1.1 |
| Physical health | 369 | 0.9 |
| Sexuality | 284 | 0.7 |
| Addictions | 232 | 0.6 |
| Other | 3324 | 8.3 |
| Missing (any main concern) | 1615 | 4.1 |
| Suicidal thoughts | ||
| Current (within the past 24 hours) | 21,869 | 54.8 |
| Recent (within the past few days) | 11,293 | 28.3 |
| No | 6625 | 16.6 |
| Missing | 124 | 0.3 |
| “How upset are you? | ||
| Doing OK | 1496 | 3.8 |
| A little upset | 2534 | 6.4 |
| Moderately upset | 12,012 | 30.1 |
| Very upset | 14,800 | 37.1 |
| Extremely upset | 8948 | 22.4 |
| Missing | 121 | 0.3 |
The logistic regression analysis to predict whether or not chatters completed a post‐chat survey found that the model’s discrimination power was low (AUC = 0.59, 95% CI = 0.59, 0.60). This indicates that weighting the post‐chat analyses based on the predictive likelihood of answering a post‐chat survey would introduce unnecessary noise as pre‐chat responses carried little information about post‐chat missingness. Thus, the following findings are based on unweighted analyses of the 13,130 chats with a post‐chat survey.
Chatters’ perceptions of engagement with chat counselors
Almost two‐thirds of chatters agreed that their counselors were genuinely concerned for their well‐being (32.6% strongly agree, 31.8% agree, 8.5% disagree, 11.4 strongly disagree, and 15.7% response missing). Approximately 59% agreed that their counselor understood them (28.6% strongly agree, 30.4% agree, 11.4% disagree, 12.7% strongly disagree, and 16.9% response missing). Nearly 51% reported that they and their chat counselor agreed on some plans for after the call (e.g., things they could do or someone they could contact).
Males had significantly lower odds than females of providing positive feedback about their counselors. They had lower odds of reporting that their chat counselors were genuinely concerned for their well‐being (OR = 0.87; 95% CI = 0.78–0.96); lower odds of agreeing that their chat counselor understood them (OR = 0.86; 95% CI = 0.78–0.95); and lower odds of reporting that they and their chat counselor agreed on a plan for after the call (OR = 0.81; 95% CI = 0.74–0.89).
Compared to minors, older callers had significantly lower odds of providing positive feedback about their counselors. Callers 35 years of age or older had lower odds of agreeing that their chat counselor was genuinely concerned for their well‐being (OR = 0.72; 95% CI = 0.61–0.85); had lower odds of feeling that their chat counselor understood them (OR = 0.77; 95% CI = 0.66–0.90); and lower odds of reporting that they and their chat counselor agreed on a plan for after the call (OR = 0.70; 95% CI = 0.61–0.82). A similar pattern of results emerged for the comparison of the other age groups with the minors.
Chatters’ perceptions of crisis chat's effectiveness
Overall, chatters were significantly less upset by the end of the chat, as assessed by changes in chatters’ responses to the question “how upset are you?” before and after the chat (Wilcoxon matched pairs signed rank test, Z = −66.8, p < .0001). Over half of the chatters (55.3%) who completed the post‐chat survey identified themselves as “OK” or “a little upset” by the end of the chat (see Table 2); whereas only 9.9% (1296/13,130) of these chatters had rated themselves as “OK” or “a little upset” at the beginning of the chat. Of the 2470 chatters who were extremely upset at the onset of the chat, 63.3% (n = 1564) rated themselves as less upset by the end, with 36.0% rating themselves as “OK” or “a little upset.” Similarly, among those who had rated themselves as very upset at the beginning of the chat (n = 5099), 69% (n = 3515) rated themselves as less upset, with 50.8% identifying themselves as “OK” or “a little upset.”
TABLE 2.
Chatters’ post‐chat perceptions of crisis chat effectiveness (N = 13,130)
| Post‐chat Survey Questions | N | % |
|---|---|---|
| “Now that you have finished your chat session, how upset are you?” | ||
| Doing OK | 4658 | 35.5 |
| A little upset | 2597 | 19.8 |
| Moderately upset | 2095 | 16.0 |
| Very upset | 1394 | 10.6 |
| Extremely upset | 2238 | 17.0 |
| Missing | 148 | 1.1 |
| “Did you find this chat service helpful?” | ||
| Yes | 8773 | 66.8 |
| No | 3884 | 29.6 |
| Missing | 473 | 3.6 |
| “After the conversation how do you feel?” | ||
| Hopeful? | ||
| Less | 2442 | 18.6 |
| Same | 4897 | 37.3 |
| More | 4136 | 31.5 |
| Missing | 1655 | 12.6 |
| Depressed? | ||
| Less | 3524 | 26.8 |
| Same | 5733 | 43.7 |
| More | 2160 | 16.5 |
| Missing | 1713 | 13.1 |
| Overwhelmed? | ||
| Less | 4925 | 37.5 |
| Same | 4294 | 32.7 |
| More | 2111 | 16.1 |
| Missing | 1800 | 13.7 |
| Suicidal? a | ||
| Less | 4944 | 45.0 |
| Same | 3351 | 30.5 |
| More | 1344 | 12.2 |
| Missing | 1354 | 12.3 |
Analysis of suicidal outcome only includes 10,993 chatters who endorsed current or recent suicidal thoughts on the pre‐chat survey.
Approximately two‐thirds of chatters who completed the post‐chat survey reported that they found the chat service to be helpful. With regard to changes in specific psychological states, nearly one‐third of chatters reported that they felt more hopeful after the chat; approximately one quarter indicated that they felt less depressed; over one‐third reported feeling less overwhelmed; and 45.0% of the suicidal chatters reported feeling less suicidal. However, there was a substantial minority who reported worsening of their emotional states, were still distressed at the end of the chat, or did not think the chat was helpful.
Relationship between chatters’ perceptions of crisis chat's effectiveness and their pre‐chat characteristics and perceived engagement with chat counselors
Males’ perceptions of the effectiveness of the chat were less favorable than females’ perceptions, as indicated by the significance and direction of the odds ratios for all the outcomes (see Table 3). Males had significantly lower odds of reporting positive outcomes (i.e., finding the chat services was helpful or reporting that they felt more hopeful after the chat) and had significantly higher odds than females of reporting that they were more upset, depressed, overwhelmed, or more suicidal at the end of the chat. A similar pattern of results, with even larger percent changes in the odds, was found for chatters whose gender was classified as “Other” as compared to females, but compared to females there were no significant differences in outcomes for the chatters who identified as transgender, questioning or genderqueer.
TABLE 3.
Relationship between chatters’ post‐chat perceptions of chat's effectiveness and their pre‐chat characteristics and feedback about their Counselors (N = 12,957 a )
| Post‐chat perceptions of effectiveness | “Now that you have finished your chat session, how upset are you?” | “Did you find this chat service helpful?” | “After the conversation how do you feel?” | |||
|---|---|---|---|---|---|---|
| Hopeful? | Depressed? | Overwhelmed? | Suicidal?b | |||
| OR (95% CI) | OR (95% CI) | OR (95% CI) | OR (95% CI) | OR (95% CI) | OR (95% CI) | |
| Pre‐chat characteristics | ||||||
| Gender | ||||||
| Female (reference) | – | – | – | – | – | – |
| Male | 1.29 * (1.20, 1.39) | 0.85* (0.77, 0.93) | 0.90* (0.83, 0.97) | 1.09 (1.00, 1.18) | 1.17* (1.07, 1.28) | 1.23* (1.12, 1.34) |
| Transgender | 1.20 (1.01, 1.44) | 0.81 (0.65, 1.00) | 0.81 (0.66, 0.99) | 1.19 (0.99, 1.44) | 1.09 (0.90, 1.31) | 1.13 (0.92, 1.39) |
| Questioning | 1.03 (0.85, 1.25) | 1.03 (0.80, 1.32) | 0.95 (0.77, 1.16) | 1.03 (0.82, 1.29) | 1.04 (0.85, 1.29) | 1.02 (0.80, 1.30) |
| Genderqueer | 1.18 (0.92, 1.53) | 0.77 (0.57, 1.05) | 0.93 (0.72, 1.21) | 1.16 (0.87, 1.55) | 0.86 (0.65, 1.13) | 1.10 (0.81, 1.49) |
| Other | 1.46 * (1.11, 1.91) | 0.63 * (0.46, 0.86) | 0.64* (0.48, 0.85) | 1.59 * (1.19, 2.11) | 1.47* (1.11, 1.95) | 1.67 * (1.22, 2.27) |
| Age (years) | ||||||
| ≤17 (reference) | – | – | – | – | – | – |
| 18–24 | 1.21* (1.12,1.30) | 0.79* (0.73, 0.87) | 0.81* (0.74, 0.87) | 1.15* (1.07, 1.24) | 1.10* (1.02, 1.19) | 0.99 (0.91, 1.08) |
| 25–34 | 1.66 * (1.51, 1.82) | 0.63 * (0.56, 0.70) | 0.62 * (0.56, 0.69) | 1.58 * (1.42, 1.75) | 1.50* (1.35, 1.68) | 1.30* (1.16, 1.45) |
| ≥35 | 2.32 * (2.06, 2.62) | 0.48 * (0.41, 0.55) | 0.48 * (0.42, 0.55) | 2.17 * (1.87, 2.53) | 2.20 * (1.92, 2.51) | 1.86 * (1.60, 2.17) |
| Main concerns | ||||||
| Depression (reference) | – | – | – | – | – | – |
| Non‐suicidal self‐harm | 1.15* (1.02, 1.29) | 0.92 (0.79, 1.07) | 0.84* (0.73, 0.95) | 1.12 (0.98, 1.28) | 1.14 (1.01, 1.30) | 1.45* (1.27, 1.67) |
| Anxiety | 0.82* (0.73, 0.92) | 1.15 (1.00, 1.34) | 1.24* (1.08, 1.41) | 0.73* (0.65, 0.83) | 0.97 (0.86, 1.10) | 0.75* (0.64, 0.88) |
| Family issues | 0.75 * (0.66, 0.86) | 1.14 (0.97, 1.33) | 1.21* (1.06, 1.38) | 0.73* (0.63, 0.83) | 0.92 (0.81, 1.05) | 0.76* (0.65, 0.90) |
| Relationship Issues | 1.18* (1.02, 1.36) | 0.88 (0.73, 1.05) | 0.95 (0.81, 1.10) | 0.88 (0.75, 1.02) | 0.96 (0.82, 1.13) | 1.01 (0.83, 1.24) |
| Bullying/Problems in school | 0.82 (0.65, 1.03) | 1.17 (0.89, 1.54) | 1.39* (1.09, 1.78) | 0.65* (0.51, 0.82) | 0.90 (0.70, 1.16) | 0.93 (0.71, 1.22) |
| Physical, sexual and/or emotional abuse | 1.54 * (1.24, 1.93) | 0.81 (0.63, 1.05) | 0.82 (0.64, 1.07) | 1.19 (0.94, 1.50) | 1.52* (1.18, 1.96) | 1.47* (1.14 1.91) |
| Financial issues | 1.29 (0.95, 1.75) | 0.72 (0.50, 1.03) | 0.81 (0.58, 1.12) | 1.06 (0.77, 1.45) | 1.56* (1.15, 2.14) | 1.22 (0.87, 1.73) |
| Eating disorder | 1.03 (0.75, 1.41) | 0.88 (0.59, 1.31) | 1.14 (0.80, 1.64) | 1.07 (0.76, 1.52) | 1.31 (0.94, 1.84) | 1.72 * (1.13, 2.62) |
| Physical health | 1.87 * (1.26, 2.77) | 0.51 * (0.33, 0.80) | 0.66 (0.44, 0.98) | 1.53 (1.00, 2.34) | 1.57 (0.99, 2.49) | 2.04 * (1.16, 3.59) |
| Sexuality | 0.57 * (0.37, 0.86) | 1.34 (0.77, 2.31) | 1.45 (0.95, 2.20) | 0.61 * (0.38, 0.97) | 0.64 (0.41, 1.00) | 0.79 (0.42, 1.49) |
| Addictions | 1.38 (0.84, 2.28) | 0.44 * (0.25, 0.77) | 0.65 (0.39, 1.09) | 1.43 (0.86, 2.38) | 2.16 * (1.27, 3.67) | 1.51 (0.82, 2.78) |
| Suicidal thoughts | ||||||
| No (reference) | – | – | – | – | – | N/A |
| Current | 1.71 * (1.56, 1.87) | 0.85* (0.76, 0.96) | 0.73* (0.66, 0.81) | 1.79 * (1.62, 1.98) | 1.21* (1.09, 1.34) | 1.52* (1.40, 1.64) |
| Recent (within past few days) | 1.11 (1.00, 1.22) | 1.04 (0.92, 1.18) | 0.90 (0.81, 1.00) | 1.41* (1.26, 1.57) | 1.05 (0.93, 1.18) | – |
| Chatter's perceptions of engagement with counselor | ||||||
| I feel my chat counselor was genuinely concerned for my well‐being | ||||||
| Disagree (reference c ) | ‐– | – | – | – | – | – |
| Agree (n = 8666) | 0.06 * (0.05, 0.06) | 45.65 * (40.05, 52.04) | 15.66 * (14.06, 17.44) | 0.07 * (0.06, 0.08) | 0.09 * (0.08, 0.10) | 0.07 * (0.06, 0.08) |
| I feel my chat counselor understood me | ||||||
| Disagree (reference c ) | – | – | – | – | – | – |
| Agree | 0.06 * (0.05, 0.06) | 50.45 * (43.15, 58.99) | 15.77 * (14.34, 17.35) | 0.07 * (0.06, 0.07) | 0.09 * (0.09, 0.10) | 0.07 * (0.06, 0.08) |
| Did you and your chat counselor agree on any plans for after the call (like things you can do or someone you can contact)? | ||||||
| No (reference) | – | – | – | – | – | – |
| Yes | 0.12 * (0.11, 0.13) | 22.12 * (19.53, 25.06) | 8.28 * (7.63, 8.99) | 0.15 * (0.13, 0.16) | 0.16 * (0.15, 0.18) | 0.13 * (0.12, 0.14) |
N is smaller than total sample of chatters with post‐chat survey because of missing values.
Analysis of suicidal outcome only includes 10,922 chatters who endorsed current or recent suicidal thoughts on the pre‐chat survey.
4‐point scale was dichotomized for analysis.
Odds ratio was significant at p < .05 after applying the Benjamini–Hochberg method for multiple testing.
Bolded values denote statistically significant associations with a 25% change in the odds for 5‐point Likert scale variables and 50% change for 3‐point Likert scale and binary variables.
Overall, minors reported significantly better outcomes than older chatters. The minors had significantly and substantially higher odds of finding the chat helpful and reporting that they were more hopeful at the end of the chat. Conversely, minors had significantly lower odds of reporting that they were upset, depressed, overwhelmed, or more suicidal at the end of the chat. The largest percentage change in the odds was found for the oldest age group, as compared to the minors.
The chatters’ main concerns for chatting were significantly related to their perceptions of the effectiveness of the chat. Compared to depression, chatters’ concerns relating to addictions, abuse, physical health problems, or eating disorders were associated with significantly less favorable outcomes. For example, chatters whose main concerns related to either addiction or physical health problems had a 56% and 49% decrease in their odds, respectively, of reporting the chat service to be helpful. This is reflected in the findings that among chatters whose main concern was depression, 67.3% reported that they found the chat service helpful, compared to 46.3% and 53.8% of the chatters with addiction or physical health concerns, respectively. (See Tables S1a–f for full tabulation of percentages for each predictor/outcome pair.) Furthermore, chatters whose main concern related to eating disorders and physical health had significantly and substantially higher odds of being more suicidal at the end of the chat than those who were mainly concerned about their depression. Main concerns that were associated with more favorable outcomes compared to depression were those relating to family issues and sexuality. For example, at the end of the chat, those whose main concerns were about family issues or their sexuality had significantly and substantially lower odds of being upset than chatters concerned about depression. To clarify the seeming inconsistency between the direction of the findings for those who identified their gender as “other” and those who selected sexuality as their main concern, it should be noted that only a small percentage of the chatters who selected sexuality as a main concern identified their gender as “other” (2.3%), with the remaining identifying as female (49.4%), male (23%), questioning (11.5%), transgender (10.3%), or genderqueer (1.2%).
Of chatters who had reported they were currently or recently suicidal when they started the chat, 42.7% and 49.3%, respectively, indicated that they were less suicidal at the end of the chat. Moreover, 65.0% and 69.5% of currently or recently suicidal chatters, respectively, found the chat to be helpful. Overall, suicidal chatters were significantly less upset by the end of the chat, as assessed by changes in their responses to the question “how upset are you?” before and after the chat (Wilcoxon matched pairs signed rank test, Z = −61.1, p < .0001). By the end of the chat, nearly half (49.6%) of the chatters who were currently suicidal at the start of the chat and 61.4% of recently suicidal chatters reported that they were either feeling OK or just a little upset.
The chatters’ greater perceived engagement with their counselors—as indicated by their feeling that their counselors were genuinely concerned for their well‐being, understood them, and had developed an action plan for after the call—had the strongest associations with the chatters’ perceptions of the chat's effectiveness. For example, positive feedback about their counselors was found to yield between 22 and 50 times the odds of finding the chat to be helpful. Of chatters who felt their counselors were genuinely concerned for their well‐being, felt their chat counselor understood them, or agreed with their counselors on a plan for after the call, 89.4%, 92.5%, and 92.2%, respectively, reported that the chat was helpful. Among chatters who did not provide this positive feedback about their counselors, only 16.0%, 22.3%, and 40.5%, respectively, reported finding the chat to be helpful.
DISCUSSION
Our findings from 13,130 linked pre‐ and post‐chat surveys completed by recipients of Lifeline's online crisis chat services indicated that chatters were significantly and substantially less distressed at the end of the chat intervention than they were at the beginning. This is of particular import because the vast majority of individuals seeking help from the LCC were highly distressed when they contacted the service. Moreover, almost 84% of LCC chatters endorsed either current or recent suicidal ideation on the pre‐chat survey, which is markedly higher than the estimated 23% of Lifeline callers who are suicidal on the day of or the day before their calls (Gould et al., 2013). Our findings are consistent with earlier research that also reported that crisis chatters reveal higher rates of suicidal ideation than crisis callers (Fukkink & Hermanns, Lake et al., 2009a; Haner & Pepler, 2016; 2021). By the end of the chat, two‐thirds of suicidal chatters reported that the chat had been helpful and nearly half reported they were less suicidal.
Our finding that nearly half of suicidal chatters reported that they were feeling less suicidal after the chat intervention can be compared to the reduction in suicidal ideation found after more time‐intensive interventions, such as the Collaborative Assessment and Management of Suicidality (Jobes et al., 2017). Jobes et al. (2017) reported that 27% of participants with moderate to severe suicidal ideation at baseline had no suicidal ideation after one month of treatment. Thus, for a single‐session chat intervention to achieve a reduction in suicidality in nearly half of chatters can be considered a significant accomplishment. While a single chat session could not be expected to eradicate suicide risk permanently, any reduction in risk at a moment of crisis for these individuals could open a window of opportunity for further help‐seeking and for engagement in additional interventions that could have a more sustained impact.
The LCC is not specifically targeted to youth, as are many other online chat services (e.g., Evans et al., 2013; Fukkink & Hermanns, 2009a; Haner & Pepler, 2016; King, Bambling, Reid, et al., 2006). Yet, those seeking services from the LCC were largely young, with nearly 40% being minors, and over 70% younger than 24 years. The appeal of online crisis services to youth has consistently been reported by others (Haner & Pepler, 2016; Mokkenstorm et al., 2017). In light of telephone hotlines’ underuse by youths (Gould & Kalafat, 2009), the Lifeline's complementary online crisis service is providing a critical service to reach an underserved population. Moreover, our findings indicated that the LCC services were particularly effective for minors as compared to older age groups. Minors were more likely to provide positive feedback about their counselors and were more likely to be less distressed, depressed, overwhelmed, or suicidal at the end of the chat than older chatters.
Unfortunately, the LCC was not as effective at reaching males—another underserved population—or at meeting their needs. Nearly two‐thirds of crisis chatters were female, and males’ perceptions of the chat's effectiveness were substantially less favorable than females’ perceptions. Males are consistently found to be less likely than females to seek help from a myriad of resources (e.g. Lubman et al., 2017; Milner et al., 2019; Wang, et al., 2007). Early studies of seeking help on the Internet speculated that this “gender divide” might be mitigated by offering online resources (Gould et al., 2002); however, recent studies (e.g., Mokkenstorm et al., 2017), as well as our current findings, have not borne this out. Developing suicide prevention strategies to reach males continues to be a challenge.
Chatters’ perceived engagement with counselors had the strongest associations with chatters’ perceived effectiveness of the chat in the current study. Counselors’ success in engaging chatters may facilitate “online comforting communication” which is postulated as necessary to reduce emotional distress (Caplan & Turner, 2007). Mishara et al. (2007b) found a similar relationship among callers to crisis hotlines; counselors’ supportive approach, good contact, collaborative problem‐solving together with empathy and respect—behaviors underlying callers’ engagement with counselors—were significantly related to positive outcomes for crisis callers. Similarly, an earlier Lifeline evaluation (Gould et al., 2016) found that the caller’s level of engagement was associated with positive caller outcomes; specifically, higher levels of engagement enhanced the counselors’ chances of mitigating imminent suicide risk through collaborative interventions and reduced the need to involve emergency services. These findings support Lifeline’s policy’s emphasis on developing “good contact” and promoting active engagement with all callers (Draper et al., 2015). Contrary to our current findings, King, Bambling, Reid, et al. (2006) found no significant effect of the two subscales of his “therapeutic alliance” measure most comparable to our engagement indices on outcomes of young people using online chat services in Australia.
Our study revealed some deficiencies in crisis chat counselors’ interventions. Crisis chat counselors appeared to be well‐equipped to intervene with chatters for whom depression was a main concern, as reflected by two‐thirds of these chatters reporting that the chat service was helpful. However, when the chatters’ concerns related to addictions, abuse, physical health problems, or eating disorders significantly less favorable outcomes emerged. This suggests the need to enhance crisis counselors’ training in these areas. The National Suicide Hotline Designation Act of 2020 (https://www.congress.gov/116/plaws/publ172/PLAW‐116publ172.pdf) has designated 988 as the national suicide prevention and mental health crisis hotline, to be implemented by summer of 2022. As such, Lifeline crisis counselors answering these calls and chats will need to provide effective services for chatters with a wide range of mental concerns, including addictions, abuse, physical health problems, and financial issues—serious problems in their own right and as risks for suicidal thoughts and behaviors.
There are several limitations of the current study. Firstly, only 33% of all chats had a post‐chat survey. We explicitly compared chats with a post‐chat survey to those without on their linked pre‐chat survey variables (age, gender, main concern, presence of suicidal ideation, and level of distress) and found that these chatter characteristics did not predict whether or not a post‐chat survey had been completed, indicating that our analyses using unweighted post‐chat survey data are likely to be generalizable to all Lifeline chats. However, there may be other factors, not measured in this study, that are associated with the completion of the post‐chat survey and could yield some unknown bias. Secondly, the pre‐ and post‐chat data were de‐identified so we were unable to determine whether our data contained multiple surveys from the same chatter. Thus, our unit of analysis is the chat survey not the chatter. Thirdly, our measurement instruments were simple and short with little available psychometric data. This allows the Lifeline to use these measures an ongoing basis for clinical not research purposes. Nevertheless, the Lifeline allowed us to assist them in modifying the post‐chat survey based on measures that had shown meaningful clinical associations in our earlier Lifeline evaluation studies (e.g., Gould et al., 2007, 2013). Therefore, the post‐chat survey, even though short, included several empirically based effectiveness measures and assessments of the chatter’s perceptions of engagement with the counselor.
Notwithstanding the limitations of this research, the study has numerous strengths. To our knowledge, this is the first study to provide estimates of the effectiveness of crisis chat interventions among suicidal chatters. Ours is the largest sample employed to examine the effectiveness of crisis chat. While our chatters were predominantly young, we had substantial numbers of chatters across the age span, which provided us with the statistical power to assess the differential effectiveness of the intervention by age. The large sample size also allowed us to examine the relationship of the outcomes with several chatter characteristics and measures of the chatters’ perceived engagement with their counselors. Furthermore, our reliance on the chatter's perspective complements earlier research that employed counselors’ perspectives or coding of chat transcripts, and is a key perspective for person‐centered care (U.S. Department of Health and Human Services, 2011).
In conclusion, our study offers some empirical evidence for the effectiveness of Lifeline's online crisis chat services. Emotional distress was significantly and substantially reduced by the end of the chat and most chatters reported that the chat was helpful. While the finding that nearly half of suicidal chatters reported that they were feeling less suicidal after the chat intervention is encouraging, nevertheless, half of the suicidal chatters reported that they did not feel less suicidal after their crisis chat. Moreover, a sizeable minority of all chatters did not feel the chat was helpful or were very or extremely upset at the end of the chat—reported most notably by those whose main concerns related to addiction, abuse, physical health problems, and eating disorders. Lastly, a challenge remains as to how to engage males in suicide preventive interventions, including Lifeline's crisis chat services. These findings can inform the Lifeline's efforts to increase the effectiveness of its interventions as they transition to 988 to meet the ever‐increasing needs of individuals in suicidal and mental health crises in the United States.
Supporting information
Table S1
Gould, M. S. , Chowdhury, S. , Lake, A. M. , Galfalvy, H. , Kleinman, M. , Kuchuk, M. , & McKeon, R. (2021). National Suicide Prevention Lifeline crisis chat interventions: Evaluation of chatters’ perceptions of effectiveness. Suicide and Life‐Threatening Behavior, 51, 1126–1137. 10.1111/sltb.12795
Funding information
This project was funded by the Substance Abuse and Mental Health Services Administration (SAMHSA) through a subcontract from ICF Macro, Inc
REFERENCES
- Bambling, M. , King, R. , Reid, W. , & Wegner, K. (2008). Online counseling experience of counsellors providing synchronous, single session counselling to young people. Counselling and Psychotherapy Research, 8, 110–116. [Google Scholar]
- Barak, A. , & Bloch, N. (2006). Factors related to perceived helpfulness in supporting highly distressed individuals through an online support chat. CyberPsychology & Behavior, 9, 60–68. 10.1089/cpb.2006.9.60 [DOI] [PubMed] [Google Scholar]
- Caplan, S. E. , & Turner, J. S. (2007). Bringing theory to research on computer‐mediated comforting communication. Computers in Human Behavior, 23, 985–998. 10.1016/j.chb.2005.08.003. [DOI] [Google Scholar]
- Chardon, L. , Bagraith, K. S. , & King, R. J. (2011). Counselling activity in single‐session online counseling with adolescents: An adherence study. Psychotherapy Research, 21, 583–592. [DOI] [PubMed] [Google Scholar]
- Draper, J. , Murphy, G. , Vega, E. , Covington, D. W. , & McKeon, R. (2015). Helping callers to the National Suicide Prevention Lifeline who are at imminent risk of suicide: The importance of active engagement, active rescue, and collaboration between crisis and emergency services. Suicide and Life‐Threatening Behavior, 46, 261–270. 10.1111/sltb.12128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Evans, W. P. , Davidson, L. , & Sicafuse, L. (2013). Someone to listen: Increasing youth help‐seeking behaviour through a text‐based crisis line for youth. Journal of Community Psychology, 41, 471–487. [Google Scholar]
- Fukkink, R. (2011). Peer counseling in an online chat service: A content analysis of social support. Cyberpsychology, Behavior, and Social Networking, 14(4), 247–251. 10.1089/cyber.2010.0163. [DOI] [PubMed] [Google Scholar]
- Fukkink, R. G. , & Hermanns, J. M. A. (2009a). Children’s experiences with chat support and telephone support. Journal of Child Psychology and Psychiatry, 50(6), 759–766. 10.1111/j.1469-7610.2008.02024.x. [DOI] [PubMed] [Google Scholar]
- Fukkink, R. , & Hermanns, J. (2009b). Counseling children at a helpline: Chatting or calling? Journal of Community Psychology, 37(8), 939–948. 10.1002/jcop.20340. [DOI] [Google Scholar]
- Gilat, I. , & Shahar, G. (2007). Emotional first aid for a suicide crisis: Comparison between telephonic hotline and internet. Psychiatry, 70(1), 12–18. [DOI] [PubMed] [Google Scholar]
- Gould, M. S. , Cross, W. , Pisani, A. R. , Munfakh, J. L. H. , & Kleinman, M. (2013). Impact of applied suicide intervention skills training on the National Suicide Prevention Lifeline. Suicide and Life‐Threatening Behavior, 43(6), 676–691. 10.1111/sltb.12049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gould, M. S. , & Kalafat, J. (2009). Role of crisis hotlines in suicide prevention. In Wasserman D., & Wasserman C. (Eds.), The Oxford textbook of suicidology: The five continents perspective (pp. 459–462). Oxford University Press. [Google Scholar]
- Gould, M. S. , Kalafat, J. , Munfakh, J. L. H. , & Kleinman, M. (2007). An evaluation of crisis hotline outcomes, Part II: Suicidal callers. Suicide and Life‐Threatening Behavior, 37(3), 338–352. [DOI] [PubMed] [Google Scholar]
- Gould, M. S. , Lake, A. M. , Galfalvy, H. , Kleinman, M. , Munfakh, J. L. H. , Wright, J. , & McKeon, R. (2018). Follow‐up with callers to the National Suicide Prevention Lifeline: Evaluation of callers’ perceptions of care. Suicide and Life‐Threatening Behavior, 48(1), 75–86. 10.1111/sltb.12339. [DOI] [PubMed] [Google Scholar]
- Gould, M. S. , Lake, A. M. , Munfakh, J. L. , Galfalvy, H. , Kleinman, M. , Williams, C. , Glass, A. , & McKeon, R. (2016). Helping callers to the National Suicide Prevention Lifeline who are at imminent risk of suicide: Evaluation of caller risk profiles and interventions implemented. Suicide and Life‐Threatening Behavior, 46(2), 172–190. 10.1111/sltb.12182. [DOI] [PubMed] [Google Scholar]
- Gould, M. S. , Munfakh, J. L. H. , Kleinman, M. , & Lake, A. M. (2012). National Suicide Prevention Lifeline: Enhancing mental health care for suicidal individuals and other people in crisis. Suicide and Life‐Threatening Behavior, 42(1), 22–35. 10.1111/j.1943-278X.2011.00068.x. [DOI] [PubMed] [Google Scholar]
- Gould, M. S. , Munfakh, J. L. H. , Lubell, K. , Kleinman, M. , & Parker, S. (2002). Seeking help from the internet during adolescence. Journal of the American Academy of Child and Adolescent Psychiatry, 41(10), 1182–1189. 10.1097/00004583-200210000-00007. [DOI] [PubMed] [Google Scholar]
- Haner, D. , & Pepler, D. (2016). “Live Chat” clients at kids help phone: individual characteristics and problem topics. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 25, 138–144. [PMC free article] [PubMed] [Google Scholar]
- Jobes, D. A. , Comtois, K. A. , Gutierrez, P. M. , Brenner, L. A. , Huh, D. , Chalker, S. A. , Ruhe, G. , Kerbrat, A. H. , Atkins, D. C. , Jennings, K. , Crumlish, J. , Corona, C. D. , Connor, S. O’. , Hendricks, K. E. , Schembari, B. , Singer, B. , & Crow, B. (2017). A randomized controlled trial of the Collaborative Assessment and management of suicidality versus enhanced care as usual with suicidal soldiers. Psychiatry, 80, 339–356. 10.1080/00332747.2017.1354607. [DOI] [PubMed] [Google Scholar]
- King, R. , Bambling, M. , Lloyd, C. , Gomurra, R. , Smith, S. , Reid, W. , & Wegner, K. (2006). Online counselling: The motives and experiences of young people who choose the Internet instead of face to face or telephone counselling. Counselling and Psychotherapy Research, 6, 169–174. 10.1080/14733140600848179. [DOI] [Google Scholar]
- King, R. , Bambling, M. , Reid, W. , & Thomas, I. (2006). Telephone and online counselling for young people: A naturalistic comparison of session outcome, session impact and therapeutic alliance. Counselling and Psychotherapy Research, 6, 175–181. 10.1080/14733140600874084. [DOI] [Google Scholar]
- Lake, A. M. , Niederkrotenthaler, T. , Aspden, R. , Kleinman, M. , Hoyte, A. & Gould, M. S. (2021). Lifeline Crisis Chat: Coding form development and findings on chatters’ risk status and counselor behaviors. [DOI] [PMC free article] [PubMed]
- Lubman, D. I. , Cheetham, A. , Jorm, A. F. , Berridge, B. J. , Wilson, C. , Blee, F. , Mckay‐Brown, L. , Allen, N. , & Proimos, J. (2017). Australian adolescents’ beliefs and help‐seeking intentions towards peers experiencing symptoms of depression and alcohol misuse. BioMed Central Public Health, 17, 658. 10.1186/s12889-017-4655-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Milner, A. , Scovelle, A. J. , & King, T. (2019). Treatment‐seeking differences for mental health problems in male‐ and nonmale‐dominated occupations: Evidence from the HILDA cohort. Epidemiology and Psychiatric Sciences, 28, 630–637. 10.1017/S2045796018000367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mishara, B. L. , Chagnon, F. , Daigle, M. , Balan, M. , Raymond, S. , Marcoux, I. , Bardon, C. , Campbell, J. K. , & Berman, A. (2007a). Comparing models of helper behavior to actual practice in telephone crisis intervention: A silent monitoring study of calls to the U.S. 1‐800‐SUICIDE network. Suicide and Life‐Threatening Behavior, 37(3), 308–321. [DOI] [PubMed] [Google Scholar]
- Mishara, B. L. , Chagnon, F. , Daigle, M. , Balan, M. , Raymond, S. , Marcoux, I. , Bardon, C. , Campbell, J. K. , & Berman, A. (2007b). Which helper behaviors and intervention styles are related to better short term outcomes in telephone crisis intervention? Results from a silent monitoring study of calls to the U.S. 1‐800‐SUICIDE Network. Suicide and Life Threatening Behavior, 37(3), 291–307. [DOI] [PubMed] [Google Scholar]
- Mokkenstorm, J. K. , Eikelenboom, M. , Huisman, A. , Wiebenga, J. , Glissen, R. , Kerkhof, J. F. M. , & Smit, J. H. (2017). Evaluation of the 113Online suicide prevention crisis chat service: Outcomes, helper behaviors and comparison to telephone hotlines. Suicide and Life‐Threatening Behavior, 47, 282–296. 10.1111/sltb.12286. [DOI] [PubMed] [Google Scholar]
- Predmore, Z. , Ramchand, R. , Ayer, L. , Kotzias, V. , Engel, C. , Ebener, P. , Kemp, J. E. , Karras, E. , & Haas, G. L. (2017). Expanding suicide crisis services to text and chat. The Journal of Crisis Intervention and Suicide Prevention, 38, 255–260. 10.1027/0227-5910/a000460. [DOI] [PubMed] [Google Scholar]
- Ramchand, R. , Jaycox, L. H. , Ebener, P. , Barnes‐Proby, D. , Gilbert, M. L. , & Goutam, P. (2017). Characteristics and proximal outcomes of calls made to suicide crisis hotlines in California: Variability across centers. Crisis, 38(1), 26–35. 10.1027/0227-5910/a000401. [DOI] [PubMed] [Google Scholar]
- U.S. Department of Health and Human Services (HHS) (2011). National Strategy for Quality Improvement in Health Care. Agency for Healthcare and Quality. https://www.ahrq.gov/workingforquality/about/nqs‐fact‐sheets/fact‐sheet.html. Retrieved January 20, 2021.
- U.S. Department of Health and Human Services (HHS) Office of the Surgeon General and National Action Alliance for Suicide Prevention (September 2012). 2012 National Strategy for Suicide Prevention: Goals and Objectives for Action. HHS. [PubMed] [Google Scholar]
- van Buuren, S. , & Groothuis‐Oudshoorn, K. (2011). mice: Multivariate imputation by chained Equations in R. Journal of Statistical Software, 45(3), 1–67. [Google Scholar]
- Wang, P. S. , Aguilar‐Gaxiola, S. , Alonso, J. , Angermeyer, M. C. , Borges, G. , Bromet, E. J. , Bruffaerts, R. , de Girolamo, G. , de Graaf, R. , Gureje, O. , Haro, J. M. , Karam, E. G. , Kessler, R. C. , Kovess, V. , Lane, M. C. , Lee, S. , Levinson, D. , Ono, Y. , Petukhova, M. , … Wells, J. E. (2007). Use of mental health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental health surveys. The Lancet, 379, 841–850. 10.1016/S0140-6736(07)61414-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Table S1