Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2026 Oct 28.
Published in final edited form as: J Am Coll Health. 2026 Aug 27;74(9):2898–2907. doi: 10.1080/07448481.2026.2717318

Relationships between Counselor Statements and Student Connection to Mental Health Care: A Cross-sectional Analysis of Digital Screening Platform Conversations

Susanna Lewis 1, Ashley M Witmer 2, Susan Han 1,*, Mark Dredze 3, Holly C Wilcox 2,4, James Aluri 1,4
PMCID: PMC13711413  NIHMSID: NIHMS2215853  PMID: 42658094

Abstract

Objective:

To examine the relationship between counselor communication techniques and students seeking referrals to care within the American Foundation for Suicide Prevention’s (AFSP’s) Interactive Screening Program’s (ISP’s) text-based chats.

Participants:

Data included 14,931 text-based chats between students and counselors from 58 U.S. colleges that used the ISP between 2009 and 2024.

Methods:

Natural Language Processing was used to analyze ISP counselor communication techniques within the online chats.

Results:

Counselor informing statements (aOR: 1.98, 95% CI: 1.78-2.21), emotional reactions (aOR: 1.11, 95% CI: 1.02-1.20), and longer dialogue length (aOR: 1.30, 95% CI: 1.28-1.32) were positively associated with seeking a referral. Counselor interpretations (aOR: 0.32, 95% CI: 0.29-0.35), explorations (aOR: 0.32, 95% CI: 0.28-0.36), and normalizing statements (aOR: 0.69, 95% CI: 0.61-0.78) were negatively associated with seeking a referral.

Conclusions:

Findings advance understanding of how counselors might prioritize using specific language in text-based chats to support students seeking a referral to care.

Keywords: College Student Mental Health, Screening, Help-Seeking, Counselor Empathy, Suicide Prevention

Introduction

Increasing numbers of college students are experiencing mental health symptoms, including suicidal thoughts and behaviors (STB). The 2024-2025 Healthy Minds Study (HMS) found that 37% of college students reported moderate to severe depression symptoms and 32% reported moderate anxiety symptoms on standardized questionnaires (Healthy Minds Network, 2025). Comparing the Healthy Minds Study Surveys from 2010 to 2025, the percentage of students reporting suicidal ideation (6.7% in 2010 to 11.3% in 2025), suicidal plans (1.7% to 4.4%), and suicide attempts (0.6% to 1.4%) in the past year all significantly increased (Healthy Minds Network, 2010 and 2025).

Students enrolled in higher education are vulnerable to mental health crises due to several factors: their age and developmental life stage (adolescence and young adulthood is a typical time for first onset of mental illness), as well as common stressors associated with college such as academic pressures, relationship difficulties, lack of financial support, and adjustment challenges related to being in a new academic and social environment (Riba, et al., 2015; Hurst, et al., 2013).

Alongside trends of increased mental health diagnoses and STBs among young adults are indications that more college students are utilizing mental health services. Students’ use of counseling and pharmacological treatment has increased in recent years (Lipson, et al., 2019; Morris, et al., 2021; Oswalt, et al., 2020). Despite these positive trends in help-seeking, most college students who need mental health treatment are not connected to care. Only 25% of college students who experience moderate or major depression symptoms receive treatment, and between 25-36% of college students experiencing suicidal thoughts in the past year engaged in mental health care (Jalene, et al., 2021; Barnett, et al., 2024).

This gap in treatment is caused by many factors. Lack of awareness of campus-based resources, assumptions about the costs of treatment, and time constraints can delay or prevent timely treatment (Córdova, et al, 2023; Peel & Ward, 2022; Hunt & Eisenberg, 2010). Seeking and engaging in treatment can be especially difficult for students experiencing STBs. Financial concerns, a lack of time, and concerns about privacy and stigma are commonly reported barriers to care for students at higher risk of suicide (Horwitz et al., 2020).

The persistent underutilization of treatment despite the rise in mental illness and help-seeking among college students emphasizes the need for innovative outreach approaches that engage with students experiencing mental health difficulties, including STBs, and support their connection to care (Abelson, et al., 2022).

The Interactive Screening Program (ISP)

One digital screening program that aims to address this need is the American Foundation for Suicide Prevention’s (AFSP’s) Interactive Screening Program (ISP). The ISP is meant to reduce barriers to seeking care, provide information about resources, and to connect students to traditional mental health treatment. The ISP has been implemented at nearly 60 colleges and universities in the United States and is a way for students to confidentially learn more about their mental health concerns and available resources, and to connect to services. Students are invited to complete a brief stress and depression questionnaire, which includes the Patient Health Questionnaire-9 (PHQ-9), and then receive personalized feedback to their questionnaire from a mental health counselor employed by that school (referred to as an “ISP counselor” in this paper). The feedback aims to address their concerns and provide information about available mental health resources. After completing their questionnaire, students have the option to exchange asynchronous online messages with an ISP counselor to address any barriers to seeking care that they may experience, to get support, and to request an appointment with their college counseling center or other off-campus mental health services (Haas et al., 2008; Bernanke et al., 2017).

ISP counselors are mental health professionals (social workers, counselors, and psychologists) employed independently by each college or university, typically housed within campus-based mental health services (AFSP, 2026; Haas et al., 2008). While the ISP counselors are trained mental health professionals and they do often offer support, validation, and psychoeducation to students over the platform, the ISP is not intended to be a text-based therapy platform. Instead, its primary goal is to connect students to campus mental health services.

Prior studies have explored the ISP’s cost-effectiveness and ability to reach students experiencing mental health symptoms, as well as its ability to connect students who are experiencing more severe mental health symptoms—including suicidal thoughts and behaviors—to treatment (Witmer, et al., 2026; Aluri et al., 2022; Davidson et al., 2020; Davidson et al., 2018; Deng et al., 2024; Downs et al., 2014; Garlow et al., 2008; Haas et al., 2008; Moffitt et al., 2014; Mortali & Moutier, 2018; Moutier et al., 2012; Zisook et al., 2022). While these studies establish the ISP’s effectiveness at reaching students experiencing moderate to severe distress, how ISP counselors’ communication techniques facilitate referrals to care remains unstudied.

ISP Counselor Interventions and Student Outcomes

ISP counselors exchanging online messages with participants respond to a range of presenting concerns, questions, thoughts and feelings expressed by students. AFSP training materials encourage ISP counselors to be empathetic, supportive, and affirming in the chats and to facilitate students seeking an in-person or telehealth connection to mental health services (L. Hoffman, personal communication, November 13, 2018). Even though the intention of the ISP is to connect students to more traditional mental health care, ISP counselors sometimes engage in brief, supportive interactions with students over the platform in certain situations, such as when a student does not want to be connected to traditional mental health treatment or expresses significant barriers to connecting to care. In such situations, the counselor might offer supportive conversation by validating the student’s experiences, exploring a student’s thoughts or feelings, or suggesting tools and techniques the student can use. Counselors responding to the ISP chats have individual communication styles, and unlike traditional therapy, where therapists can also use body language and volume and tone of voice to communicate, ISP counselors are completely reliant on written language to be able to convey information and support the student.

The text-based format of the ISP offers an opportunity to study written therapeutic components of empathy—such as exploring a client’s thoughts and feelings, expressing understanding, or communicating feelings related to what the client is saying (Howick et al., 2025). Within the context of traditional psychotherapy, empathy has been shown to have a positive effect on client outcomes (Elliott et al., 2018). However, the relationship between components of written empathy and outcomes in contexts such as a chat-based platform intended to connect students to care has not been investigated.

Filling this gap can clarify how empathy might be utilized by counselors in chat-based digital contexts intended to connect individuals to care. Motivational Interviewing (MI) is an evidence-based framework for understanding behavior change. The Transtheoretical Model of Change, which theorizes that people move through discrete stages of change before adopting new behaviors, informs MI interventions; it has also been suggested that MI could be a useful tool for changing behaviors associated with elevated suicide risk (Hoy, et al., 2016). Within the MI framework, empathic statements strengthen the client-counselor partnership that grounds the therapeutic work (Lord, et al., 2015). Empathic statements have been broken down into emotional reactions (expressions of emotion), interpretations (synthesizing and reflecting another’s experience), and explorations (asking questions) (Sharma, et al., 2020) categories that map onto MI’s emphasis on an empathic partnership, reflecting on aspects of the client’s experience, and asking questions (Frey, et al., 2024).

Additional communication strategies might be helpful. Normalizing experiences and providing information about services can shape subjective norms and attitudes, which are key predictors of help-seeking in the Theory of Planned Behavior (Adams, et al., 2022). Understanding the relationship between counselor statements and student behaviors can inform the training and communication strategies of counselors who interact with students on digital platforms such as the ISP. Optimizing these strategies is critical to strengthening innovative programs like the ISP that seek to reduce the treatment gap among a population of students with increasingly severe mental illnesses.

This study examined the relationship, within ISP text dialogues, between counselor’s written statements and students’ likelihood of seeking a referral to care. Our primary aim was to understand the relationship between different types of empathic statements (i.e., emotional reactions, interpretations, and explorations) and referral seeking. Our secondary aim was to investigate the relationships between other types of counselor statements (e.g., informing or normalizing statements) and referral seeking.

Methods

Study Design and Ethical Approval

This cross-sectional study analyzed all available data from institutions that used the ISP. Data were de-identified before being analyzed. The Institutional Review Board of Johns Hopkins University School of Medicine acknowledged this secondary data analysis as exempt from review and did not require informed consent.

Sample and Data Source

The ISP dataset contained data from 58 campuses across the United States (one institution opted out) and spanned fifteen years, from 2009 to 2024. ISP implementation and outreach strategies vary across institutions. Some institutions actively recruit participants by emailing invitations to their entire student body, some invite specific student populations, some employ a more passive approach by posting a link on their counseling or wellness center website, and others blend these outreach strategies (Haas et al., 2008; Moutier et al., 2012; Aluri et al., 2022). Despite these differences in recruitment, a core set of survey questions, the user interface, the counselor training, and the ability to dialogue with a counselor are consistent across sites.

First, participating students create an account and a custom username (that can preserve their anonymity if they choose) and then complete a Stress and Depression questionnaire. The Stress and Depression Questionnaire includes the PHQ-9, a valid and reliable scale for measuring depression severity (Kroenke et al., 2001); questions assessing current mental health symptoms; questions about sociodemographic characteristics, and a free-text field where students can describe any recent stressors they have experienced (AFSP, n.d.). After completing the questionnaire, students receive personalized feedback about their responses from an ISP counselor at their campus and an invitation to exchange asynchronous, private text-based messages with an that counselor through the “dialogue” feature.

Response rates for the ISP are unavailable because the total number of invited participants is not reported to AFSP. AFSP program data suggests that the number of students who participate in the ISP per year at each college or university ranges from <1% to 22% of each school’s enrolled population. Median participation was 1.3% of enrolled students. Differences in outreach strategies likely explain some of the variability in participation rates across campuses. In the dataset, 63,407 students had completed the ISP’s Stress and Depression Questionnaire. From this pool of eligible participants, students were included in the analytic sample if they participated in a dialogue in which both the student and the counselor sent a message and if they had complete data for age and gender.

Measures

NLP Methods

The ISP dialogue dataset, which includes free-text exchanges between students and counselors, contains tens of thousands of free text entries. Human analysis of this data would have required thousands of hours. Natural language processing methods (NLP) present an alternative to human coding to analyze large amounts of unstructured text (Jerfy et al., 2024). We used generative pre-trained transformer (GPT) models, which can be trained with prompt-based examples to identify features of new data (Xiao et al., 2023; Chen et al., 2025; Lyu et al., 2023; Fukuda et al., 2025). These models can identify and label elements from free-text clinical reports with stellar accuracy, sensitivity, and specificity (all above 90%, Chen et al., 2025), making them a suitable candidate to identify text-based constructs in our analyses of unstructured text between a clinician and a student.

Primary Outcome: Seeking a Referral to Care

The ISP dialogues were analyzed with NLP methods to assess whether a student sought a referral to care using a four-step process (Witmer et al., 2026). First, a definition for the binary variable Seeks a Referral was developed as part of a larger codebook that included descriptions for other variables. Between 4 and 6 human coders iteratively annotated random sets of 10 dialogues for all variables defined in the codebook. Any disagreements in coding were discussed and resolved, and definitions in the codebook were updated. This process was repeated until achieving satisfactory inter-rater agreement, which was defined as a Randolph’s Kappa score of > 0.6 12. This standardization process was stopped after collaboratively coding 70 dialogues, at which point the Seeks Referral variable had a Randolph’s Kappa score of κ = 0.93 for all 70 dialogues.

Second, 200 randomly sampled dialogues were coded by three humans. Each dialogue was labeled as Yes or No for Seeks a Referral by the majority vote of three human coders. Randolph’s Kappa score for Seeks a Referral (κ = 0.78) remained above the threshold of 0.6 (Witmer et al., 2026).

In the third step, the random subset of 200 tripled-human-coded dialogues was used to evaluate the performance of Large Language Models (LLMs). One-quarter of the random subset (50 dialogues) were randomly selected to serve as a ‘training’ set to evaluate different model settings (see Supplement 1), such as the prompt (the instructions given to the LLM) or temperature (a measure of the randomness in the LLM’s selection of responding words). Human-LLM reliability scores (Cohen’s Kappa) and accuracy (percent of answers matching the human codes) were calculated to evaluate model performance. The three highest performing models were applied to the other 150 dialogues (the ‘test’ set). GPT-4o achieved an accuracy score of 90.2% and a Cohen’s κ = 0.80 on the 200-dialogue random subset (Witmer et al., 2026). This 200-dialogue set includes 50 training set dialogues (κ = 0.84) and 150 test set dialogues (κ = 0.78).

In the fourth and final step, GPT-4o classified the remaining 14,931 dialogue entries for Yes or No for Seeks a Referral. NLP methods for determining the outcome of whether a student sought a referral to care, along with additional information about model parameters and settings, are described in further detail in the Supplemental Materials (Supplement 1).

Independent Variables: Counselor Empathy, Normalizing, and Informing Measurements, and Dialogue Length

The study’s codebook adapted Sharma et al’s (2020) conceptualization of empathy for text-based mental health support, which outlines three components: emotional reactions, interpretations, and explorations. Emotional reactions were coded as Yes if the counselor expressed emotions such as warmth, compassion, and concern for the student using “I” or “we” statements. Interpretations were coded as Yes if the counselor expressed an understanding of the student’s feelings and emotionally toned experiences in the dialogue. Explorations were coded as Yes if the counselor asked a question that intended to explore the student’s feelings or emotionally toned experiences in the dialogue.

Per the codebook, normalizing was coded as Yes if the counselor made a statement in the dialogue indicating that the student’s thoughts, feelings, or actions are expected, understandable, or common for a person in their situation, or if they acknowledged that their experience is shared by other people. Informing was coded as Yes if the counselor included information about location, pricing, scheduling, contact information, or availability of counseling services, or any on-campus or community resources, including general wellness resources, or if the counselor explained the process of starting care, privacy and confidentiality of care, mandated reporting, or what therapy or psychiatric care is like in the dialogue.

An analogous four-step process to that used to code dialogues for Seeks Referral was applied to determine whether each dialogue contained empathic, informing, or normalizing statements from the ISP counselors. First, 10 dialogues were iteratively coded by a team of human coders who discussed discrepancies and clarified codebook definitions. This process was repeated until, after 70 dialogues, all five independent variables hit the satisfactory threshold of > 0.60. Second, a team of six human coders tripled-coded the 200-dialogue test set for these five variables.

Third, multiple LLMs were evaluated on the 200-dialogue, triple-human-coded random subset. Prompts were developed from the codebook, and—for some variables—multiple prompts were tested until the best performing prompt was identified. GPT-4o was selected as the best performing for Emotional Reactions and was used to identify the other independent variables. GPT-4o achieved an accuracy score of 84.0% for emotional reactions, 87.0% for interpretations, 95.0% for explorations, 92.0% for informing statements, and an accuracy score of 91.0% for normalizing statements. See Supplement 2 for the final prompts used. Fourth and finally, the highest performing prompts were used to classify the rest of the dialogues.

Dialogue length was measured by the number of entries or ‘turns’ within each dialogue. For example, if a student messages asking for information about counseling resources (one entry), and the counselor responds (one entry), the dialogue length would be two.

Sociodemographic Characteristics

Sociodemographic measures were self-reported by ISP participants within the Stress and Depression Questionnaire. Each institution selected which sociodemographic variables to ask participants, leading to variation in some data availability. Adopting a cutoff used in prior work with ISP data, sociodemographic variables with more than 30% missing data in our dataset were excluded as covariates in the regression models.13 Age (6.3% missing data) and gender (1.8% missing data) were the covariates. Age was obtained as a numerical entry. Gender was collected from an open-ended question with the categorical response options of female, male, transgender, genderqueer, and other (which included individuals who selected self-describe, non-binary, gender agnostic, agender, third-gender, other or more than one response).

Analysis

All analyses were conducted using RStudio (version 2023.06.2+561) (Posit team, 2023). Descriptive statistics, including absolute numbers and percentages, were calculated for all ISP participants who engaged in the dialogue feature. The mean dialogue length and corresponding standard deviation were calculated and stratified by each empathy, normalizing, and informing variable. Differences in proportions were assessed using chi-square tests, and differences in means were assessed using two-sample t tests. Two-sided P < 0.05 indicated significance.

Logistic regression models were used to examine the relationships between the outcome of seeking a referral and the predictors of empathic, normalizing, and informing statements, and dialogue length among ISP participants who engaged in the dialogue feature. Participants with missing data for any of the included covariates—emotional reactions, interpretations, explorations, normalizing statements, informing statements, age, race and ethnicity, and gender—were excluded from the regression models. Six bivariate models were developed to examine the unadjusted odds ratios and 95% confidence intervals for seeking a referral by each of the predictors (three empathy subtypes, normalizing statements, informing statements, and dialogue length). A single multivariable logistic regression model then incorporated the included covariates to calculate adjusted odds ratios and 95% confidence intervals for these associations. Multicollinearity was evaluated using the generalized variance-inflation factor (GVIF). For each covariate in the multivariable model, we calculated GVIF^(1/2 x df), where df denotes the degrees of freedom. Values of GVIF^(1/2 x df) greater than or equal to two were considered significant for multicollinearity, consistent with established thresholds (Harris, 2021). GVIFs for all covariates were less than two, so multicollinearity was not detected.

Seven post-hoc analyses were conducted to understand how sample selection and missing data might affect the results of the multivariable regression analysis. Two ‘attrition’ analyses focused on sample selection. One attrition analysis compared students included in the descriptive sample to those excluded to identify who did not participate in a dialogue. To evaluate potential selection bias due to missing data, a second attrition analysis compared characteristics of participants included in the complete-case analytic sample to those who were excluded.

To evaluate the impact of missing data, we examined whether missingness in age, race and ethnicity, and gender was associated with referral-seeking. To assess the robustness of the logistic regression results, we conducted two sensitivity analyses in which the primary adjusted model was re-estimated excluding age and gender, respectively, to evaluate whether the associations between counselor communication statements and seeking a referral were influenced by the exclusion of students who were missing sociodemographic data. Finally, we conducted two sensitivity analyses to evaluate whether multivariate regression findings would change with the addition of year of response or race and ethnicity, which is associated with service utilization (Sheehan et al., 2024).

Results

Of the 63,407 students who completed a Stress and Depression Questionnaire, 17,091 had at least one dialogue message. Of these, 2,160 dialogues were excluded because they comprised of messages from the counselor only (n=2,037) or messages from the student only (n=123).An additional 1,117 dialogues were excluded due to missing data on age or gender. Our final sample comprised 13,814 ISP participants.

Among these participants, 64.1% were 18-24 years old (see Table 1). Of participants who responded to the race and ethnicity question, 55.1% identified as White, 15.6% identified as Asian/Pacific Islander, 9.1% identified as Hispanic/Latin(x), and 8.5% identified as African American/Black. Female participants outnumbered male participants by a large margin (70.2% to 27.1%). Of participants who reported their sexual orientation, 71.9% identified as heterosexual/straight and 11.5% identified as bisexual. Among ISP dialogue participants who responded to additional sociodemographic questions, 11.5% identified as international students, 27.8% as first-generation college students, 3.8% as student athletes, and 1.7% as U.S. Veterans.

Table 1.

Sociodemographic characteristics of ISP dialogue participants (N=13,814).

n % (including missing data) % (not including missing data)
Age
 18-24 8,858 64.1 N/A
 25+ 4,956 35.9 N/A
Race and Ethnicity
 African American/Black 810 5.9 8.5
 American Indian/Alaska Native 53 0.4 0.6
 Asian/Pacific Islander 1,484 10.7 15.6
 Hispanic/Latin(x) 869 6.3 9.1
 Multiethnic 516 3.7 5.4
 White 5,250 38.0 55.1
 Other 235 1.7 2.5
 Prefer not to answer 304 2.2 3.2
 Not available 4,293 31.1 --
Gender
 Female 9,698 70.2 N/A
 Male 3,744 27.1 N/A
 Transgenderb 86 0.6 N/A
 Genderqueerc 150 1.1 N/A
 Otherd 136 1.0 N/A
Sexual Orientation
 Heterosexual/Straight 5,131 37.1 71.9
 Bisexual 823 6.0 11.5
 Gay/Lesbian 353 2.6 4.9
 Othere 455 3.3 6.4
 Prefer not to Answer 370 2.7 5.2
 Not available 6,682 48.4 --
International Student
 Yes 750 5.4 11.5
 No 5,758 41.7 88.5
 Not available 7,306 52.9 --
First Generation College Student
 Yes 877 6.3 27.8
 No 2,281 16.5 72.2
 Not available 10,656 77.1 --
Student Athlete
 Yes 192 1.4 3.8
 No 4,849 35.1 96.2
 Not available 8,773 63.5 --
U.S. Veteran
 Yes 124 0.9 1.7
 No 7,156 51.8 98.3
 Not available 6,534 47.3 --
Open in a new tab

Sample is comprised of ISP participants with complete data on age and gender.

a

Includes Arab/Middle Eastern and “I do not identify as any of these options.”

b

Includes transgender, trans female/trans woman, and trans male/trans man.

c

Includes genderqueer and gender non-conforming.

d

Includes nonbinary, agender, gender agnostic, third gender, self-describe, multiple gender selections, none of the options, and prefer not to answer.

e

Includes questioning, asexual, panromantic, pansexual, and “I do not identify as any of these options”.

Abbreviations: ISP, interactive screening program; U.S., United States.

Slightly more than half of the participants, 7,098 (51.4%), sought a referral and 6,716 (48.6%) did not seek a referral (Table 2). Compared to dialogues in which students did not seek a referral, dialogues in which participants sought a referral had lower prevalence of counselors’ emotional reactions (52.4% vs. 56.7%), interpretations (29.3% vs. 58.4%), explorations (10.5% vs. 25.2%), and normalizing statements (10.4% vs. 20.1%) but had higher prevalence of informing statements (90.0% vs. 78.3%) with all P values <0.001. The mean number of dialogue entries was larger for students who sought a referral compared to students who did not seek a referral (5.6 vs. 3.3, P<0.001). The mean number of dialogue entries was also larger in conversations that included emotional reactions (4.7 vs 4.2 entries), interpretations (4.8 vs 4.2), explorations (6.1 vs 4.1), normalizing statements (5.7 vs 4.3), and informing statements (4.8 vs 3.0) with all P values <0.001 (See Table 2).

Table 2.

Empathy, normalizing, and informing statements among dialogues, by seeks referral.

Seeks referral (N=7,098) Does not seek referral (N=6,716) Total dialogue participants (N=13,814) Dialogue length
n % n % p-valuea n % mean SD p-valueb
Empathy
 Emotional reaction < 0.001 < 0.001
  Yes 3,721 52.4 3,810 56.7 7,531 54.5 4.7 5.1
  No 3,377 47.6 2,906 43.3 6,283 45.5 4.2 3.5
 Interpretation < 0.001 < 0.001
  Yes 2,083 29.3 3,922 58.4 6,005 43.5 4.8 5.4
  No 5,015 70.7 2,794 41.6 7,809 56.5 4.2 3.5
 Explorations < 0.001 < 0.001
  Yes 747 10.5 1,693 25.2 2,440 17.7 6.1 6.9
  No 6,351 89.5 5,023 74.8 11,374 82.3 4.1 3.6
Normalizing < 0.001 < 0.001
 Yes 738 10.4 1,353 20.1 2,091 15.1 5.7 6.8
 No 6,360 89.6 5,363 79.9 11,723 84.9 4.3 3.8
Informing < 0.001 < 0.001
 Yes 6,389 90.0 5,257 78.3 11,646 84.3 4.8 4.7
 No 709 10.0 1,459 21.7 2,168 15.7 3.0 2.2
Open in a new tab
a

The p-value for a chi-squared test comparing seeks referral vs. non-seeks referral among ISP participants who participated in a two-way dialogue with complete data on age and gender.

b

The p-value for a two-sample t-test comparing dialogue length by the presence vs. absence of empathy, normalizing, and informing statements among ISP participants who participated in the dialogue feature with complete data on age and gender.

In the multivariable analyses adjusted for emotional reactions, interpretations, explorations, normalizing, informing, age, gender, and dialogue length (Table 3), seeking a referral to care was positively associated with counselor emotional reactions (aOR: 1.11, 95% CI: 1.02-1.20) and with the counselor making informing statements (aOR: 1.98, 95% CI: 1.78-2.21). Seeking a referral was negatively associated with interpretations (aOR: 0.32, 95% CI: 0.29-0.35), explorations (aOR: 0.32, 95% CI: 0.28-0.36), and normalizing statements from the counselor (aOR: 0.69, 95% CI: 0.61-0.78). Seeking a referral to care was also associated with dialogue length. Each additional dialogue entry was associated with increased likelihood of seeking a referral (aOR: 1.30, 95% CI: 1.28-1.32).

Table 3.

Unadjusted and adjusted odds ratios for seeking a referral among dialogues, by empathy, normalizing, and informing statements.

OR (95% CI) aOR (95% CI)a
Empathy
 Emotional reaction
  No REF REF
  Yes 0.84 (0.79-0.90)*** 1.11 (1.02-1.20)*
 Interpretation
  No REF REF
  Yes 0.30 (0.28-0.32)*** 0.32 (0.29-0.35)***
 Explorations
  No REF REF
  Yes 0.35 (0.32-0.38)*** 0.32 (0.28-0.36)***
Normalizing
 No REF REF
 Yes 0.46(0.42-0.51)*** 0.69 (0.61-0.78)***
Informing
 No REF REF
 Yes 2.50 (2.27-2.76)*** 1.98 (1.78-2.21)***
Dialogue length 1.23 (1.22-1.25)*** 1.30 (1.28-1.32)***
Open in a new tab
a

Models are adjusted for all other empathy, normalizing, and informing covariates, age, gender, and dialogue length.

Signif. codes:

*

0.05,

***

0.001

Abbreviations: OR, odds ratio; aOR, adjusted odds ratio; CI, confidence interval.

Regarding our post-hoc analyses, we first assessed how the 48,476 students who did not participate in the dialogue differed from the 14,931 who participated in the two-way dialogue and found that these groups differed in terms of age, race and ethnicity, gender, sexual orientation, and international student status (all p < 0.05, see Supplemental Material 3). The 13,814 students who had complete data on all model covariates (and thus were included in the multivariable regression model) differed from the 1,117 who were not included in the multivariable regression due to missingness on at least one variable and found that they differed in terms of age and gender (all p < 0.001, see Supplemental Material 4). Chi-square tests indicated that missingness in age and gender was significantly associated with seeking a referral to care (see Supplemental Material 5). In the sensitivity analyses excluding age and gender from the adjusted logistic regression models, the associations between counselor communication statements and seeking a referral to care remained almost identical across all models (see Supplemental Material 6). Finally, the two sensitivity analyses adding covariates of race and ethnicity and year did not substantively change the relationships either (See Supplemental Material 7).

Discussion

In our primary analysis, counselor emotional reactions, counselor informing statements, and longer dialogues were associated with higher odds of a student seeking a referral to care. Counselor explorations, interpretations, and normalizing statements were associated with lower odds of seeking a referral.

Dialogue chats that included counselor emotional reactions (i.e., expressions of emotion toward the student such as, “I’m concerned about you”) were more likely to include a student request for a referral. While our cross-sectional analysis cannot disentangle temporality or causality, this association provides preliminary evidence that counselors’ empathic expressions might improve student outcomes in these text-based formats. In the context of traditional psychotherapy, therapist empathy, compassion, and interpersonal skills have positive effects on client engagement, making it more likely that the client will attend follow up appointments and participate in the therapeutic process (Elliott et al., 2018; Holdsworth et al., 2014; Rickwood et al., 2005). While ISP interventions are not wholly analogous to traditional therapy treatment, ISP counselors use language-based interventions similar to those used in traditional therapy to build rapport, and to express understanding and support. Our study preliminarily suggests that usage of specific language in the context of the ISP appears to have a positive influence on ISP user outcomes.

These online, text-based platforms, which can feel impersonal, might specifically benefit from the humanizing tone of the counselor using “I” and “We” statements. In a study looking at the effectiveness of another online college mental health screening program, eBridge, the authors noted that several students asked within the dialogue chats whether they were speaking with a ‘bot’ (King et al., 2022). The ISP counselor using emotional reactions might let the student know and feel that this is a human counselor they are speaking with, and not an automated messaging system.

Importantly, in a study looking at counselors’ use of MI techniques within ISP dialogues in workplace (not higher education) settings, use of counselor empathy was associated with lower odds of seeking a referral (Frey et al., 2024). The ISP is used in both workplace and college settings, which represent distinct participant populations that differ in developmental stage, help-seeking behaviors and norms, exposures to mental health stigma, and access to available services (Mackenzie et al., 2019; Mackenzie et al., 2006). MI aims to move clients experiencing ambivalence towards behavior change (Miller & Rollnick, 2012), an approach that fits well within the ISP (ISP counselors often interact with students experiencing ambivalence about seeking care). These studies can both help to inform recommendations for implementation of MI approaches by ISP counselors depending on participant population characteristics and implementation setting.

In our analysis, dialogues in which counselors used the other two types of text-based empathy—interpreting and exploring a student’s thoughts, feelings, or experiences—were less likely to include a student request for a referral to care. In contrast, the analysis of ISP dialogues in workplace settings (Frey et al., 2024) found that when ISP counselors asked questions in the dialogues, it was associated with higher odds of the employee seeking a referral. These differences re-emphasize the importance of adapting communication approaches to a specific audience. We are not aware of studies that explore counselors’ use of interpretation statements within an online, asynchronous, text-based format, and student outcomes; therefore, our investigation makes an important preliminary contribution to understanding their role.

The positive association between counselor informing statements and seeking a referral suggests that students might be encouraged to seek care if the ISP counselor provides information about services such as counseling center location, hours, eligibility, the scope of available mental health resources and supports, and the process of scheduling an appointment. As with all of our results, the cross-sectional nature of the study prevents us from establishing the temporal relationship of these statements. The sequence of statements is needed to understand how informing statements might causally encourage students to connect to care. If informing statements do encourage students to connect to care, it is likely because they help clarify misunderstandings about access to and the nature of these services. A lack of knowledge about available mental health services, how those services are delivered and accessed, and who is eligible for treatment, is a commonly cited barrier to seeking care for undergraduate and graduate students, as well as for students who are at elevated risk for suicide (Eisenberg et al., 2007; Wildey et al., 2022; Czyz et al., 2013). The ISP can help students to overcome these types of barriers by providing a mechanism to confidentially ask questions about resources, express concerns about the financial costs of seeking treatment, share uncertainty related to how therapy works including confidentiality and mandated reporting, or inquire about scheduling an appointment with campus mental health services.

In our study, counselor normalizing statements were inversely associated with students’ seeking a referral to care. Conceptually, normalizing statements could have both positive and negative influences on help-seeking. Normalizing statements that reduce stigma and emphasize the common experience of struggling with one’s mental health and needing outside help can be helpful in moving the needle for people who hesitate to seek professional mental health support (Clement et al., 2015). The Theory of Planned Behavior, which says that subjective norms (what you think other people think you should do) can influence behavior (Adams, et al., 2022), supports the possibility that when a counselor tells a student that someone in their situation frequently benefits from counseling, that they would be more likely to then seek a referral. The normalizing of suicidal thoughts and behaviors (STB), however, can be counterproductive to help-seeking. When counselors, family, and friends normalize STB, it makes it less likely that the person experiencing those thoughts will seek help (Oexle et al., 2022). Our analysis of ISP counselors’ normalizing statements did not differentiate between these different types of normalizing statements and could not disentangle temporality and causality. Future research could explore the effect that different types of normalizing statements have on student outcomes and compare those outcomes for students experiencing STBs and those who experience other types of mental health related distress.

Students were more likely to seek a referral if they engaged in longer dialogues. Each additional entry was associated with higher odds of seeking a referral to care. Students who were more engaged might be more committed and open to discussing their problems with a counselor—a disposition that might influence a student to seek a referral to care. Even though longer dialogues were associated with greater odds of referral seeking, some long dialogues did not result in a request for a referral. Many students experiencing entrenched barriers to seeking care, or who request anonymous support over the ISP and do not want to or are unable to schedule an appointment at the college counseling center may also engage in longer dialogue chats.

While the interactive format of the ISP is uncommon among mental health screening tools, eBridge used the ISP as a template when designing the program (King et al., 2022). Students who engaged with eBridge were found to be much more likely to seek a referral to mental health services than those students who did not participate (King et al., 2022). There are no eBridge studies that look at the relationship between counselor statements and student referral seeking or connection to care, which again highlights the novel contribution of this study.

Our study does have limitations. First, our analysis was not designed to determine the sequence of each type of counselor statement and the student statement about seeking a referral. The lack of sequential understanding opens the possibility that the negatively associated types of empathy (interpretations and explorations) occurred after the student decided to not seek a referral to care. A plausible example might be that the student states early in the dialogue that they are not interested in connecting to care and requests to talk over the ISP platform instead. In such a scenario, the dialogue might emulate virtual, text-based therapy—in which case, we would expect to see more types of empathic statements. Future analyses examining the sequence of these statements could strengthen our causal understanding of these relationships and generate recommendations for ISP counselors’ communication strategies.

Second, our study is likely not generalizable to the broader student body. Due to the nature of our research questions, our study was necessarily limited to those who participated in the ISP and engaged in a two-way dialogue. Previous studies have shown that ISP users are more likely to have elevated PHQ scores than the general college student population (Aluri et al., 2026), and our attrition analyses showed that dialogue participants had different sociodemographic characteristics from non-participants (see Supplemental Material 3). Broadening the scope of ISP participants to larger populations might uncover different relationships between counselor statements and student help-seeking behaviors.

Third, missingness in the data limits the generalizability of our findings. Missingness in age and gender was significantly associated with seeking a referral to care, raising the possibility that the complete-case analysis could introduce bias. To explore this possibility, we conducted sensitivity analyses and found that the associations between counselor communication statements and seeking a referral remained substantively unchanged when different sociodemographic covariates were excluded from the adjusted models. The primary findings were thus robust to covariate selection. Race and ethnicity, which are known to be associated with service utilization (Sheehan et al., 2024), were not included in the primary analysis due to significant missingness. Though a sensitivity analysis of participants with race and ethnicity data found similar relationships, this analysis and the other sensitivity analyses described in this paragraph cannot be generalized to inferences about those with missing data. Ultimately, we chose to exclude the 7.5% of students who were missing age or gender data, and our results cannot be generalized to those students.

Fourth, our only outcome was seeking a referral to care. Additional outcomes warrant consideration such as decreased emotional distress or depression symptoms. Another study found that asynchronous text-based therapy is an effective treatment for depression, anxiety, and other functional impairments (Song et al., 2023), raising the possibility that the ISP dialogues might confer similar benefits. Finally, our study was unable to account for many societal and communal factors that influence help-seeking, including financial concerns, accessibility of services (including services being offered in one’s preferred language), insurance status, and rural versus urban location (Huff et al, 2024).

Strengths of our study include high reliability scores among a large coding team, the use of a large dataset from nearly 60 colleges and universities, and the harnessing of NLP methods to broaden the scope of the analysis.

Further research is needed to evaluate the sequence of counselor and student statements to confirm the implications of this work. Establishing temporality can support causal claims about the relationship between counselor statements and students’ help-seeking behaviors. Our research provides only preliminary evidence that counselor empathic reactions and informing statements might catalyze students’ help-seeking behaviors. Ultimately, research that can better establish temporality and causality could provide critical guidance to counselors who engage with high-risk students through programs like the ISP or eBridge and help facilitate their connection to professional mental health services.

Conclusion

ISP counselors play a crucial role in providing support and information about available mental health resources to help students connect to care. The Theory of Planned Behavior and the Transtheoretical Model of Change both underpin conceptualizations for investigating relationships between counselor language and participants’ help-seeking behaviors within the ISP, in particular for those at higher suicide risk (Adams, et al., 2022; Hoy, et al., 2016). Students seeking a referral to care is associated with the counselor providing information in the dialogue chats about availability, access, and scope of services. Counselors’ use of emotional reactions is also associated with higher odds of a student requesting a referral for mental health care. While counselors’ use of explorations, interpretations, and normalizing statements in the ISP is not associated with a student seeking a referral, examination of other student outcomes – including student self-report of feeling better – may indicate other associations with these counselor techniques. This study highlights the potential positive association between seeking a referral and counselors’ use of informing statements and expressions of emotional reactions in certain dialogues. Should these findings be confirmed in future analyses that account for the temporal sequence of statements, this work can provide guidance for training counselors who provide outreach to students and engage with individuals participating in online, asynchronous, text-based mental health platforms.

Supplementary Material

Supplemental Material

Funding:

SL, AW, SH, Dr. MD, Dr. HW, and Dr. JA received funding from the American Foundation for Suicide Prevention (AFSP) to conduct a program evaluation of the ISP under a Data Contract (C-0-001-22 – Wilcox). The AFSP reviewed the manuscript for factual accuracy of the ISP program description only, and had no role in the study design, analysis, interpretation of data, drafting of the report, or the decision to submit the article for publication. AW receives funding from the NIMH Psychiatric Epidemiology Training Program (T32MH014592).

Conflicts of Interest:

Dr. MD receives consulting fees from Good Analytics and Bloomberg LP, Dr. HW serves on the board of the AFSP and AFSP’s Scientific Council, Dr. JA receives research funding from the AFSP (ECR-1-004-24 – Aluri). The other authors declare no competing interests.

Footnotes

Ethics approval: The authors confirm that the research presented in this article met the ethical guidelines, including adherence to the legal requirements, of the United States and underwent review from the Institutional Review Board of Johns Hopkins University School of Medicine. The IRB deemed this secondary data analysis as exempt from human subjects research review and therefore waived the requirement for written or verbal consent.

Generative AI Use: As a component of the analytic approach, the authors trained OpenAI GPT-4o to analyze ISP dialogue text and assess the primary outcome of a student seeking a referral to care as well as counselor use of empathic, normalizing, and informing statements. After using this tool, the authors reviewed and edited the content as needed and assume full responsibility for the content of the published article. All uses of AI tools are fully described in the manuscript. None of the manuscript was written using generative AI.

References

  1. Abelson S, Lipson SK, & Eisenberg D. (2022). Mental Health in College Populations: A Multidisciplinary Review of What Works, Evidence Gaps, and Paths Forward. Higher Education: Handbook of Theory and Research, 37, 133–238. 10.1007/978-3-030-76660-3_6 [DOI] [Google Scholar]
  2. Adams C, Gringart E, Strobel N. Explaining adults’ mental health help-seeking through the lens of the theory of planned behavior: A scoping review. Systematic reviews. 2022;11(1):160. doi: 10.1186/s13643-022-02034-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aluri J, Lewis S, Witmer AM, Aguirre C, Howland Z, King L, Han S, Dredze M, Wilcox HC. Mental Health Screening before and after the COVID-19 Campus Closures: A Cross-Sectional Comparison using Natural Language Processing. Public Health Reports. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aluri J, Witmer AM, Lewis S, Aguirre C, Howland Z, King L, … Wilcox HC. (2026). Sociodemographic and mental health characteristics associated with participating in a digital mental health screening platform on college campuses. Journal of American College Health, 1–7. 10.1080/07448481.2026.2629343 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Aluri J, Lewis S, Torres M, & Wilcox HC. (2022). A suicide prevention initiative across a multi-campus university before and during the COVID-19 pandemic. Journal of American College Health, 72(5), 1341–1345. 10.1080/07448481.2022.2076563 [DOI] [PubMed] [Google Scholar]
  6. American Foundation for Suicide Prevention. (2026). Interactive Screening Program. https://afsp.org/interactive-screening-program/
  7. Barnett P, Tickell A, Osborn T, Delamain H, Fonagy P, Pilling S, & Gibbon L. (2024). Help-Seeking and Disclosure in University Students with Suicidal Thoughts and Self-Harm: A Systematic Review. International Journal of Mental Health and Addiction. 10.1007/s11469-024-01416-8 [DOI] [Google Scholar]
  8. Bernanke J, Galfalvy HC, Mortali MG, Hoffman LA, Moutier C, Nemeroff CB, Stanley BH, Clayton P, Harkavy-Friedman J, & Oquendo MA. (2017). Suicidal ideation and behavior in institutions of higher learning: A latent class analysis. Journal of Psychiatric Research, 95, 253–259. 10.1016/j.jpsychires.2017.09.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chen Y, Dong M, Sun J, Meng Z, Yang Y, Muhetaier A, Li C, & Qin J. (2025). Leveraging GPT-4o for Automated Extraction and Categorization of CAD-RADS Features From Free-Text Coronary CT Angiography Reports: Diagnostic Study. JMIR Medical Informatics, 13, e70967–e70967. 10.2196/70967 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Clement S, Schauman O, Graham T, Maggioni F, Evans-Lacko S, Bezborodovs N, Morgan C, Rüsch N, Brown JSL, & Thornicroft G. (2015). What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychological Medicine, 45(1), 11–27. 10.1017/s0033291714000129 [DOI] [PubMed] [Google Scholar]
  11. Córdova P, Gordillo PG, Mejía HN, La I, Chacón AG, & Unzueta AS. (2023). Academic stress as a predictor of mental health in university students. Cogent Education, 10(2). 10.1080/2331186x.2023.2232686 [DOI] [Google Scholar]
  12. Czyz EK, Horwitz AG, Eisenberg D, Kramer A, & King CA. (2013). Self-reported Barriers to Professional Help Seeking Among College Students at Elevated Risk for Suicide. Journal of American College Health, 61(7), 398–406. 10.1080/07448481.2013.820731 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Davidson JE, Accardi R, Sanchez C, Zisook S, & Hoffman LA. (2020). Sustainability and Outcomes of a Suicide Prevention Program for Nurses. Worldviews on Evidence-Based Nursing, 17(1), 24–31. 10.1111/wvn.12418 [DOI] [PubMed] [Google Scholar]
  14. Davidson JE, Zisook S, Kirby B, DeMichele G, & Norcross W. (2018). Suicide Prevention: A Healer Education and Referral Program for Nurses. J Nurs Adm, 48(2), 85–92. 10.1097/NNA.0000000000000582 [DOI] [PubMed] [Google Scholar]
  15. Deng Y, Frey JJ, Osteen PJ, Mosby A, Imboden R, Ware OD, & Bazell A. (2024). Engaging Law Enforcement Employees in Mental Health Help-Seeking: Examining the Utilization of Interactive Screening Program and Motivational Interviewing Techniques. Administration and Policy in Mental Health and Mental Health Services Research. 10.1007/s10488-024-01384-0 [DOI] [PubMed] [Google Scholar]
  16. Downs N, Feng W, Kirby B, McGuire T, Moutier C, Norcross W, Norman M, Young I, & Zisook S. (2014). Listening to Depression and Suicide Risk in Medical Students: the Healer Education Assessment and Referral (HEAR) Program. Academic Psychiatry, 38(5), 547–553. 10.1007/s40596-014-0115-x [DOI] [PubMed] [Google Scholar]
  17. Eisenberg D, Golberstein E, & Gollust SE. (2007). Help-Seeking and Access to Mental Health Care in a University Student Population. Medical Care, 45(7), 594–601. 10.1097/mlr.0b013e31803bb4c1 [DOI] [PubMed] [Google Scholar]
  18. Elliott R, Bohart AC, Watson JC, & Murphy D. (2018). Therapist empathy and client outcome: An updated meta-analysis. Psychotherapy, 55(4), 399–410. 10.1037/pst0000175 [DOI] [PubMed] [Google Scholar]
  19. Frey JJ, Deng Y, Osteen PJ, Imboden R, Mosby A, Ware O, Bazell A, & Phillips D. (2024). How are motivational interviewing techniques associated with US healthcare employees’ mental health help-seeking behaviours when using the interactive screening programme. BMJ Public Health, 2(2), e001293–e001293. 10.1136/bmjph-2024-001293 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Fukuda H, Morishita M, Muraoka K, Yamaguchi S, Nakamura T, Habu M, Yoshioka I, Awano S, & Ono K. (2025). Evaluating the Accuracy and Performance of ChatGPT-4o in Solving Japanese National Dental Technician Examination. International Dental Journal, 75(4), 100847. 10.1016/j.identj.2025.100847 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Garlow SJ, Rosenberg J, Moore JD, Haas AP, Koestner B, Hendin H, & Nemeroff CB. (2008). Depression, desperation, and suicidal ideation in college students: results from the American Foundation for Suicide Prevention College Screening Project at Emory University. Depression and Anxiety, 25(6), 482–488. 10.1002/da.20321 [DOI] [PubMed] [Google Scholar]
  22. Haas A, Koestner B, Rosenberg J, Moore D, Garlow SJ, Sedway J, Nicholas L, Hendin H, Mann JJ, & Nemeroff CB. (2008). An Interactive Web-Based Method of Outreach to College Students at Risk for Suicide. Journal of American College Health, 57(1), 15–22. 10.3200/jach.57.1.15-22 [DOI] [PubMed] [Google Scholar]
  23. Harris JK. (2021). Primer on binary logistic regression. Family Medicine and Community Health, 9(Suppl 1). 10.1136/fmch-2021-001290 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Healthy Minds Network, Healthy Minds Study Healthy Minds Network. (2010). Healthy Minds Study 2009-2010: Data Report [Report]. University of Michigan, University of California Los Angeles, Boston University, and Wayne State University. https://healthymindsnetwork.org/ [Google Scholar]
  25. Healthy Minds Network, Healthy Minds Study Healthy Minds Network. (2025). Healthy Minds Study 2024-2025: Data Report [Report]. University of Michigan, University of California Los Angeles, Boston University, and Wayne State University. https://healthymindsnetwork.org/ [Google Scholar]
  26. Holdsworth E, Bowen E, Brown S, & Howat D. (2014). Client engagement in psychotherapeutic treatment and associations with client characteristics, therapist characteristics, and treatment factors. Clinical Psychology Review, 34(5), 428–450. 10.1016/j.cpr.2014.06.004 [DOI] [PubMed] [Google Scholar]
  27. Horwitz AG, McGuire T, Busby DR, Eisenberg D, Zheng K, Pistorello J, Albucher R, Coryell W, & King CA. (2020). Sociodemographic differences in barriers to mental health care among college students at elevated suicide risk. Journal of Affective Disorders, 271, 123–130. 10.1016/j.jad.2020.03.115 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Howick J, Bennett-Weston A, Dudko M, & Eva K. (2025). Uncovering the components of therapeutic empathy through thematic analysis of existing definitions. Patient Education and Counseling, 108596–108596. 10.1016/j.pec.2024.108596 [DOI] [PubMed] [Google Scholar]
  29. Hoy J, Natarajan A & Petra MM. Motivational Interviewing and the Transtheoretical Model of Change: Under-Explored Resources for Suicide Intervention. Community Ment Health J 52, 559–567 (2016). 10.1007/s10597-016-9997-2). [DOI] [PubMed] [Google Scholar]
  30. Huff NR, Dunderdale L, Kellogg AJ, & Isbell LM. (2024). Factors related to help-seeking and service utilization for professional mental healthcare among young people: An umbrella review. Clinical Psychology Review, 102504–102504. 10.1016/j.cpr.2024.102504 [DOI] [PubMed] [Google Scholar]
  31. Hunt J, & Eisenberg D. (2010). Mental health problems and help-seeking behavior among college students. Journal of Adolescent Health, 46(1), 3–10. 10.1016/j.jadohealth.2009.08.008 [DOI] [PubMed] [Google Scholar]
  32. Hurst CS, Baranik LE, & Daniel F. (2013). College student stressors: A review of the qualitative research. Stress and Health, 29(4), 275–285. 10.1002/smi.2465 [DOI] [PubMed] [Google Scholar]
  33. Jalene S, Pharr J, Sharma M, & Poston B. (2021). Depression, fitness, and student willingness to pursue university counseling and alternative antidepressant options. Journal of Education and Health Promotion, 10, 480. 10.4103/jehp.jehp_1421_20 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Jerfy A, Selden O, & Balkrishnan R. (2024). The growing impact of natural language processing in healthcare and public health. INQUIRY the Journal of Health Care Organization Provision and Financing, 61. 10.1177/00469580241290095 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. King CA, Eisenberg D, Pistorello J, Coryell W, Albucher RC, Favorite T, Horwitz A, Bonar EE, Epstein D, & Zheng K. (2022). Electronic bridge to mental health for college students: A randomized controlled intervention trial. Journal of Consulting and Clinical Psychology. 10.1037/ccp0000709 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kroenke K, Spitzer RL, & Williams JB. (2001). The PHQ-9: validity of a brief depression severity measure. Journal of general internal medicine, 16(9), 606–613. 10.1046/j.1525-1497.2001.016009606 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lipson SK, et al. (2019). “Increased Rates of Mental Health Service Utilization by U.S. College Students: 10-Year Population-Level Trends (2007-2017).” Psychiatric Services (Washington, D.C.) 70(1): 60–63. 10.1176/appi.ps.201800332 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lord SP, Sheng E, Imel ZE, Baer J, Atkins DC. More than reflections: Empathy in motivational interviewing includes language style synchrony between therapist and client. Behavior therapy. 2015;46(3):296–303. 10.1016/j.beth.2014.11.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Lyu Q, Tan J, Zapadka ME, Ponnatapura Janardhana, Niu C, Myers KJ, Wang G, & Whitlow CT. (2023). Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential. Visual Computing for Industry, Biomedicine, and Art, 6(1). 10.1186/s42492-023-00136-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Mackenzie CS, Heath PJ, Vogel DL, & Chekay R. (2019). Age differences in public stigma, self-stigma, and attitudes toward seeking help: A moderated mediation model. Journal of Clinical Psychology, 75(12), 2259–2272. 10.1002/jclp.22845 [DOI] [PubMed] [Google Scholar]
  41. Mackenzie CS, Gekoski WL, & Knox VJ. (2006). Age, gender, and the underutilization of mental health services: The influence of help-seeking attitudes. Aging & Mental Health, 10(6), 574–582. 10.1080/13607860600641200 [DOI] [PubMed] [Google Scholar]
  42. Miller WR, & Rollnick S. (2012). Motivational interviewing: Helping people change. Guilford press. [Google Scholar]
  43. Morris MR, et al. (2021). “Use of psychiatric medication by college students: A decade of data.” Pharmacotherapy 41(4): 350–358. 10.1002/phar.2513 [DOI] [PubMed] [Google Scholar]
  44. Mortali M, & Moutier C. (2018). Facilitating Help-Seeking Behavior Among Medical Trainees and Physicians Using the Interactive Screening Program. Journal of Medical Regulation, 104(2), 27–36. 10.30770/2572-1852-104.2.27 [DOI] [Google Scholar]
  45. Moutier C, Norcross W, Jong P, Norman M, Kirby B, McGuire T, & Zisook S. (2012). The Suicide Prevention and Depression Awareness Program at the University of California, San Diego School of Medicine. Academic Medicine, 87(3), 320–326. 10.1097/acm.0b013e31824451ad [DOI] [PubMed] [Google Scholar]
  46. Oexle N, Valacchi D, Grübel P, Becker T, & Rüsch N. (2022). Two sides of the same coin? The association between suicide stigma and suicide normalisation. Epidemiology and Psychiatric Sciences, 31. 10.1017/s2045796022000610 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Oswalt SB, et al. (2020). “Trends in college students’ mental health diagnoses and utilization of services, 2009–2015.” Journal of American College Health 68(1): 41–51. 10.1080/07448481.2018.1515748 [DOI] [PubMed] [Google Scholar]
  48. Peel R, & Ward N. (2022). The Impact of Relationship Issues on the Mental Health of Students in Higher Education. Mental Health and Higher Education in Australia, 241–256. 10.1007/978-981-16-8040-3_14 [DOI] [Google Scholar]
  49. Posit team. RStudio: Integrated Development Environment for R. Boston, MA: Posit Software, PBC; 2023. [Google Scholar]
  50. Randolph JJ. (2005). Free-Marginal Multirater Kappa (multirater K [free]): An Alternative to Fleiss’ Fixed-Marginal Multirater Kappa. Online submission. [Google Scholar]
  51. Riba M, Kirsch D, Martel A, & Goldsmith M. (2015). Preparing and Training the College Mental Health Workforce. Academic Psychiatry, 39(5), 498–502. 10.1007/s40596-015-0406-x [DOI] [PubMed] [Google Scholar]
  52. Rickwood D, Deane FP, Wilson CJ, & Ciarrochi J. (2005). Young people’s help-seeking for mental health problems. Australian E-Journal for the Advancement of Mental Health, 4(3), 218–251. 10.5172/jamh.4.3.218 [DOI] [Google Scholar]
  53. Sharma A, Miner A, Atkins D, & Althoff T. (2020, November 1). A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support. ACLWeb; Association for Computational Linguistics. 10.18653/v1/2020.emnlp-main.425 [DOI] [Google Scholar]
  54. Sheehan A, Walsh R, & Liu R. (2024). Racial and ethnic trends in mental health service utilisation and perceived unmet need in the USA. Journal of Epidemiology & Community Health, 78(4), 228–234. 10.1136/jech-2023-220683 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Song J, Litvin B, Allred R, Chen S, Hull TD, & Areán PA. (2023). Comparing Message-Based Psychotherapy to Once-Weekly, Video-Based Psychotherapy for Moderate Depression: Randomized Controlled Trial. 25, e46052–e46052. 10.2196/46052 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Wildey MN, Fox ME, Machnik KA, & Ronk D. (2022). Exploring graduate student mental health and service utilization by gender, race, and year in school. Journal of American College Health, 1–9. 10.1080/07448481.2022.2145898 [DOI] [PubMed] [Google Scholar]
  57. Witmer AM, Aguirre C, Lewis S, Howland Z, King L, Han S, Dredze M, Wilcox HC, & Aluri J. (2026). Higher-risk psychiatric and sociodemographic characteristics predicting referral-seeking among college students using the interactive screening program. Journal of Affective Disorders, 121449. 10.1016/j.jad.2026.121449 [DOI] [PubMed] [Google Scholar]
  58. Xiao Z, Yuan X, Liao QV, Abdelghani R, & Oudeyer P-Y. (2023). Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding. 28th International Conference on Intelligent User Interfaces. 10.1145/3581754.3584136 [DOI] [Google Scholar]
  59. Zisook S, Doran N, Mortali M, Hoffman L, Downs N, Davidson J, Fergerson B, Rubanovich CK, Shapiro D, Tai-Seale M, Iglewicz A, Nestsiarovich A, & Moutier CY. (2022). Relationship between burnout and Major Depressive Disorder in health professionals: A HEAR report. Journal of Affective Disorders, 312, 259–267. 10.1016/j.jad.2022.06.047 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material
NIHMS2215853-supplement-Supplemental_Material.docx (51.7KB, docx)

RESOURCES