Treating Individuals with Suicidal Ideation in Primary Care: Patient-level Characteristics Associated with Follow-up in the Collaborative Care Model

Abstract

Introduction:

The Collaborative Care Model (CoCM) is an evidence-based approach which embeds behavioral health providers (BHPs) into primary care. Whether patients with suicidal ideation (SI) are willing to engage in CoCM is unclear.

Methods:

Using Patient Health Questionnaire-9 (PHQ-9) administrative data from primary care practices within an urban academic health system, we identified patients with and without SI who were referred to a CoCM BHP. We compared engagement, defined as attendance at ≥1 CoCM visit, across groups.

Results:

Between 2018 and 2022, 7,391 primary care patients were referred to a CoCM BHP. 892 of these patients reported SI on the PHQ-9 (754 on “several days” during the previous two weeks and 138 on “more than half or most days”). Across groups, most patients engaged in CoCM. Patients reporting SI on several days engaged at a lower rate (61.4%) than those reporting SI on more than half or most days (65.9%). Both SI groups engaged at a lower rate than the 6,499 patients who did not report SI (67.5%).

Conclusion:

Most patients referred to a CoCM BHP engaged in ≥1 visit. Rates were lower for patients with SI, with the lowest rate among those reporting SI on several days.

Introduction

Over the last three decades, rates of suicide have increased significantly among adults in the U.S.¹ Despite these trends, the use of behavioral health services among individuals at risk of suicide has plateaued: roughly 40% of individuals with a suicide attempt who reported a need for services did not receive them in 2008 through 2019.² This finding aligns with a larger body of research which demonstrates that most individuals who die by suicide do not actively engage with behavioral health services in the time leading up to their death.^3,4

A key barrier in access to behavioral health services is limited proximity to behavioral health professionals. In the U.S., half of counties do not have a psychiatrist, two-thirds of counties do not have a psychiatric nurse practitioner, and over one-third of counties do not have a psychologist.⁵ As of March 2023, the Health Resources and Services Administration (HRSA) estimates that nearly 160 million Americans reside in a mental health shortage area.⁶ Issues in access are compounded by inadequate insurance reimbursement, which has translated into relatively low rates of network participation and new patient acceptance among psychiatrists and other behavioral health professionals.^7,8,9 More difficulty accessing behavioral health services has been associated with higher suicide risk.¹⁰

Historically high rates of suicide, as well as the more recent increases in depression and anxiety disorders during the COVID-19 pandemic, indicate a growing demand for behavioral health services and new modes of care delivery.¹¹ Primary care has been singled out for its potential to address this unmet need—indeed, most individuals who die by suicide had a primary care visit in the previous year.^12,13 The Collaborative Care Model (CoCM), a model of care that is typically delivered in primary care settings by a coordinated team of primary care and behavioral health professionals, could help millions of Americans access behavioral health services, including those at risk of suicide.^14,15,16 The evidence base for CoCM is robust, with scores of studies reporting its effectiveness in hard-to-reach and hard-to-treat populations, such as the elderly and individuals with comorbid chronic conditions.^17,18 In addition to depression and anxiety disorders, recent systematic reviews have demonstrated that the CoCM and similar models of care can effectively treat suicidal ideation.^19,20

To ensure that the CoCM can support individuals at risk of suicide, a better understanding of who engages is needed. In this study, we describe the characteristics of patients who completed an initial assessment for CoCM provided by primary care practices in a large, urban, academic health system. For patients who reported suicidal ideation during the initial assessment and were referred back to the behavioral health care provider in the primary care practice (rather than to specialty behavioral health services outside of primary care), we assessed whether they successfully engaged in behavioral health services in their primary care practice. The breadth of the CoCM assessment allows us to compare follow-up rates across a host of patient-level characteristics, including variables that are rarely included in studies using administrative data, such as whether the patient felt financially comfortable.

Methods

This study was approved by the Institutional Review Board of the University of Pennsylvania. A waiver of informed consent was granted for the use of administrative data on the grounds that (a) the research involves no more than minimal risk to participants; (b) the waiver will not adversely affect the rights and welfare of participants; and (c) this research, which was retrospective and used administrative data, could not be practically carried out without the waiver. All methods were performed in accordance with the relevant guidelines and regulations.

Context

In 2018, Penn Medicine launched the Penn Integrated Care (PIC) program in eight primary care practices, and has since expanded it to more than 20 practices.¹⁵ PIC is a novel application of CoCM that includes a Resource Center responsible for centralized telephonic assessment, triage, and referral management. After an initial referral from one of the participating Penn Medicine primary care practices, Resource Center staff, who are bachelor’s level behavioral health intake coordinators, call patients to assess eligibility for either behavioral health services provided in the primary care practice or referral management for patients with more complex care needs, who are assisted in finding specialty care in the community. The Resource Center staff attempt to reach each referred patient at least three times.

Assessments comprise questions about sociodemographic characteristics and standardized behavioral health screening tools, including the Patient Health Questionnaire-9 (PHQ-9), a depression screener with an item focused on suicidal ideation (item 9), and the Generalized Anxiety Disorder (GAD-7) scale.^21,22 An algorithm based on clinical severity suggests the most appropriate level of care from the following options: behavioral health services in the primary care practice, specialty care in the community, or provision of self-directed resources.²³ The algorithm assigns patients with more severe or complex presentations (i.e., psychosis, manic symptoms, addiction) to be referred to specialty care. The presence of suicidal ideation did not automatically result in a referral to specialty care.

For those patients referred to the primary care arm of PIC, behavioral health services were coordinated by a behavioral health care provider embedded in the primary care practice. Providers are trained in progress monitoring, brief evidence-based interventions like safety planning, and monitoring of pharmacotherapy, in collaboration with the primary care provider and a psychiatric consultant.

Data

To capture attendance following referral to the primary care arm of PIC, we utilize two datasets in this retrospective analysis. First, we use assessment data from the Resource Center collected between 2018 and 2022 to identify patients with suicidal ideation per the PHQ-9 item 9.¹³ Second, we use episode reports that were created for any patient with at least one follow-up visit for behavioral health services at their primary care practice following a referral from the Resource Center. The two datasets can be linked using a unique patient identifier generated by the health system. We focus on patients who were referred to the primary care arm because specialty care referrals are most commonly to behavioral health professionals outside of the health system, making attendance at those appointments difficult to track.

The cohort of interest in the present study includes patients who, during their Resource Center assessment, report suicidal ideation, defined as an elevated score on item 9 of the PHQ-9: “Over the last two weeks, how often have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way?” We categorized groups based on the following response options: moderate or severe suicidal ideation indicated that patients reported thoughts of suicide on “more than half the days” (2 points) or “nearly every day” (3 points), while mild suicidal ideation indicated that patients selected “several days” (1 point). For comparison, we also tracked patients who selected “no days” (0 points). This item has been shown to be predictive of the risk of repeat suicide attempts and of suicide death.^24,25

The assessment also collected sociodemographic characteristics using items developed by the Department of Veteran Affairs Integrated Care program, including gender, age, race/ethnicity, as well as information that is not routinely captured in administrative data, such as marital status, employment, self-reported financial comfort, and self-reported health status (i.e., excellent, very good, good, fair, and poor).

Statistical Analysis

First, we measure the demographic, economic, and clinical characteristics of patients with suicidal ideation who were referred by the Resource Center to the primary care arm of PIC, stratifying by whether they reported moderate-to-severe, mild, or no suicidal ideation. Demographic characteristics include age, gender, race/ethnicity (Asian, Black, Hispanic, White), and marital status (single, married, separated/divorced, widowed); economic characteristics include whether the patient was employed and whether the patient felt comfortable with their current financial situation; and clinical characteristics include PHQ-9 (minus the PHQ-9 item 9) and GAD-7 scores. We determine whether characteristics differed across the three cohorts using group ANOVAs.

Next, we measure follow-up rates overall and by demographic, economic, and clinical characteristics (to compare patients with higher and lower depression and anxiety symptoms, we construct binary indicators for whether the PHQ-9 or GAD-7 was 10 or more). Follow up is defined by whether patients who were referred to the primary care arm of PIC attend at least one visit following their referral. The primary study sample includes those patients who reported moderate or severe suicidal ideation (i.e., on more than half/most days) or mild suicidal ideation (i.e., on several days), but we also analyzed engagement for patients who did not report suicidal ideation. Differences in follow-up rates across the three cohorts were assessed using group ANOVAs and overlapping confidence intervals; differences between two groups are considered significant if confidence intervals do not overlap. Analyses are performed in Stata 17.0. All tests are two-sided and statistical significance is defined using the 95% level of significance.

Results

Between 2018 and 2022, nearly 18,445 unique patients completed an assessment with the Resource Center (see supplemental appendix). Of these, 7,391 were referred back to primary care for behavioral health services, 138 (1.9%) of whom had moderate or severe suicidal ideation (i.e., they reported a PHQ-9 item 9 score of “more than half the days” or “most days”). Another 754 patients (10.2%) reported mild suicidal ideation (i.e., they reported a PHQ-9 item 9 on “several days”).

Among all patients referred to the primary care arm, including those who did not report suicidal ideation, nearly three-quarters were female and the average age was approximately 45 years. In terms of race and ethnicity, 3.5% were Asian, 46% were Black (non-Hispanic), 6.9% were Hispanic, and 40% were White (non-Hispanic), while 36% were married, 12% were separated or divorced, 5.1% were widowed, and 47.3% were single. Nearly two-thirds of patients were employed, but fewer than half reported feeling financially comfortable. The average GAD-7 score was 9.5, the average PHQ-9 score (less item 9) was 9.0, and nearly 35% reported being in fair or poor health.

There were some notable differences in characteristics for patients who reported moderate or severe suicidal ideation, patients who reported mild suicidal ideation, and patients who did not report suicidal ideation, which were assessed using group ANOVAs (Table 1). Notably, Black patients were more likely to report any suicidal ideation (p<0.001) and married patients were less likely to report any suicidal ideation (p<0.001). Patients with any suicidal ideation were less likely to report feeling financially comfortable (p<0.001) and were substantially more likely to report being in fair or poor health (p<0.001). Patients with any suicidal ideation also reported higher depression and anxiety scores (both p<0.001).

Table 1.

Summary statistics

	Moderate to severe suicidal ideation	Mild suicidal ideation	No suicidal ideation	Difference?
Number of patients	138	754	6,499	Prob>F
Demographics
Sex
Female	74.6%	69.0%	74.1%	0.009
Male	25.4%	31.0%	25.9%	0.009
Age in years	45.01	43.28	45.10	0.021
Race/ethnicity
Asian	0.7%	3.7%	3.5%	0.195
Black	63.8%	49.3%	45.3%	<0.001
Hispanic	6.5%	5.8%	7.0%	<0.001
White	22.5%	35.1%	41.5%	<0.001
Marital Status
Married	23.9%	31.4%	36.6%	<0.001
Separated/divorced	12.3%	11.3%	11.8%	0.897
Widowed	8.0%	3.8%	5.2%	0.087
Single	55.8%	53.4%	46.4%	<0.001
Economic Characteristics
Working	52.2%	62.9%	65.3%	0.003
Financially comfortable	27.5%	34.4%	48.4%	<0.001
Clinical Characteristics
Fair or poor health	55.1%	43.1%	33.3%	<0.001
GAD-7 (anxiety)	11.81	10.84	9.34	<0.001
PHQ-9 (less item 9)	13.11	11.20	8.62	<0.001

The follow-up rate after referral to the primary care arm, as defined by at least one visit coordinated by the behavioral health care provider, was 66.9% for all patients (Figure 1). Follow-up rates were slightly lower for patients who reported moderate or severe suicidal ideation (65.9%) than for patients who did not report suicidal ideation (67.5%). The lowest follow-up rate was among patients who reported mild suicidal ideation (61.4%). A group ANOVA determined that follow-up rates across the three cohorts, overall and by patient-level characteristics, were statistically significant at the 95% level of confidence (Table 2).

Figure 1.

Follow up rates overall and stratified by whether patients reported suicidal ideation

Note. Follow up defined by at least one visit following referral to the behavioral health care provider in a primary care practice. Between 2018 and 2022, 138 unique patients reported moderate to severe suicidal ideation, 754 reported mild suicidal ideation, and another 6,499 did not report suicidal ideation.

Table 2.

Difference in follow-up rates for patients with moderate to severe suicidal ideation, mild suicidal ideation, and no suicidal ideation by demographic, economic, and clinical characteristics

	Moderate to severe suicidal ideation	Mild suicidal ideation	No suicidal ideation	Difference?
Number of patients	138	754	6,499	Prob>F
Demographic Characteristics
Female	65.0%	60.0%	67.5%	0.002
Male	68.6%	64.5%	67.6%	0.639
Under age 25	81.3%	57.3%	66.0%	0.111
Aged 25–44	70.7%	62.1%	68.7%	0.037
Aged 45–64	63.4%	60.2%	65.6%	0.318
Over age 64	47.8%	64.6%	68.5%	0.083
Asian	-	46.4%	73.5%	0.004
Black	68.2%	59.7%	63.0%	0.259
Hispanic	77.8%	54.5%	63.3%	0.338
White	64.5%	66.4%	73.4%	0.032
Married	78.8%	67.5%	72.5%	0.181
Separated/Divorced	52.9%	63.5%	64.6%	0.604
Widowed	36.4%	51.7%	65.0%	0.064
Single	67.5%	58.1%	64.7%	0.028
Economic Characteristics
Working	76.4%	65.2%	70.0%	0.046
Not working	54.5%	55.0%	63.1%	0.015
Financially comfortable	76.3%	66.0%	73.5%	0.030
Not financially comfortable	62.0%	59.0%	62.0%	0.441
Clinical Characteristics
Fair/poor health	63.2%	58.2%	62.4%	0.334
Good/excellent health	69.4%	63.9%	70.1%	0.027
PHQ-9: over 10	65.0%	60.3%	64.9%	0.124
PHQ-9: less than 10	71.4%	64.4%	69.3%	0.322
GAD-7: over 10	65.2%	61.6%	68.1%	0.020
GAD-7: less than 10	67.3%	61.1%	67.0%	0.112

Note. Follow up defined by at least one visit following referral to the behavioral health care provider in a primary care practice. Between 2018 and 2021, 138 unique patients reported suicidal ideation for more than half or most days, 754 reported suicidal ideation for several days, and another 6,499 did not report suicidal ideation. We excluded the subgroup identifying as Asian who reported suicidal ideation on more than half or most days due to small sample size.

Among patients with moderate to severe suicidal ideation, there were no statistically significant differences in follow-up rates by patient-level characteristics, as demonstrated by overlapping confidence intervals (see supplemental appendix). Among patients without reported suicidal ideation, which was adequately powered to find smaller effect sizes, we found that follow-up rates were higher for White patients than for Black patients (73.4% vs. 63.0%). We also found higher follow-up rates among married patients compared to single patients (72.5% vs. 64.7%), and among patients who were financially comfortable compared to patients who were not financially comfortable (73.5% vs. 62.0%). While we did not find meaningful differences in follow-up rates by sex or age, we did find that patients with good/excellent health were more likely to engage than patients with fair or poor health, and that patients who were working were more likely to engage than patients who are not working. Many of these patterns emerged among patients who reported suicidal ideation, but the differences were not statistically significant.

Discussion

In 2020, over 45,000 Americans died by suicide and over one million Americans had a suicide attempt.^26,27 Identifying effective ways to reach individuals at risk of suicide is critical to reducing the number of lives lost to suicide. The Collaborative Care Model (CoCM), which embeds behavioral health professionals in non-specialty health settings like primary care, has been shown to effectively treat individuals at risk of suicide.²⁰ While CoCM remains relatively rare despite its robust evidence base,²⁸ the nationwide need for proximate behavioral health services, coupled with the introduction of new payment codes that directly reimburse primary care practices for delivering behavioral health services, may increase the adoption and delivery of collaborative care.^29,30,31 It is unclear whether this will benefit individuals who experience suicidal ideation.

Here, we used administrative data from a large, urban health care system to measure follow-up rates among patients who reported suicidal ideation during a CoCM assessment and were referred back to the behavioral health care provider at the primary care practice. We also examined the demographic, economic, and clinical characteristics of patients with reported suicidal ideation who did and did not engage after referral. A limitation of studying engagement using medical records is that we are unable to follow patients who utilized care outside of the health system.

Overall, we found that 12% of patients who were referred to the primary care arm of CoCM reported suicidal ideation for at least several days in the past two weeks during their initial behavioral health assessment. This is lower than other estimates; for example, a study by Rossom et al. found that 20% of patients who were treated for psychiatric conditions in a large sample of outpatient encounters had an elevated PHQ-9.³² This difference is likely attributable to the fact that many patients who reported suicidal ideation were referred to specialty care settings, which frequently occurred outside of the health system, making it difficult to track. Differences could also relate to individuals’ willingness to disclose suicidal symptoms during the initial assessment; an important question is whether patients feel comfortable disclosing these symptoms in phone-based assessments with individuals who are not their providers.

A second key finding was that patients with suicidal ideation were marginally less likely to attend a visit following a referral to the behavioral health care provider in the primary care practice, particularly those patients that scored a 1 on item 9, indicating that they experienced suicidal ideation on several days. We also found lower follow-up rates among some groups, including Black patients, single patients, and patients who were not financially comfortable, although these differences were statistically significant only for the cohort of patients without suicidal ideation, which was adequately powered to detect smaller effect sizes.

The difference in follow up rates across the three groups was modest: patients who did not report SI had a follow-up rate that was 2.4 percentage points higher than patients who reported SI on half or most days, and 6.1 percentage points higher than patients who reported SI on a few days. However, as described by Ozer and Funder, the consequences of small effect sizes, which can “accumulate into large ones in at least some, and probably many, but certainly not all circumstances.”³³ Given our focus on suicide, we argue that small differences in follow-up care is a circumstance in which small effect sizes can result in large and meaningful outcomes.

In addition to the shortcomings associated with studying engagement using medical records, we are limited by our focus on a single health system and shorter-term outcomes. While some studies have examined longer-term outcomes associated with CoCM using administrative data, these studies have not focused on patients with elevated rates of suicidality.³⁴ Another limitation is our inability to examine other relevant characteristics that may determine engagement, such as whether patients identified as LGBTQ+. While behavioral health services delivered in primary care settings can be successfully adapted to meet the needs of LGBTQ+ patients, more research is needed to assess whether engagement and follow-up rates differ.³⁵

A final limitation is our reliance on item 9 of the PHQ-9. While item 9 is predictive of suicide risk and is a brief and efficient screen in primary care settings—an important consideration given our focus on CoCM—some studies suggest that other screening tools, such as the Columbia Suicide Severity Rating Scale, are more accurate.^36,37 To enhance future research on treating suicidality in CoCM, item 9 of the PHQ-9 could be augmented with a more comprehensive suicide risk assessment.

Despite these limitations, this study contributes to the literature by demonstrating that most patients who reported suicidal ideation were willing to engage in behavioral health services delivered in a primary care setting, but at slightly lower rates than those without suicidal ideation. To our knowledge, it is the first to use administrative data to measure engagement in CoCM among individuals with suicidal ideation. We also identified several demographic, economics, and clinical characteristics that were associated with patients’ attendance at an initial visit following referral to a behavioral health care provider in primary care settings. This suggests that barriers to care may differ in meaningful ways across groups of patients and that strategies that aim to increase initiation and retention in CoCM need to be tailored to address multi-faceted drivers of engagement. Our team is conducting an ongoing chart review to better examine interventions provided by behavioral health care providers in primary care for patients at risk for suicide. This work will be important in elucidating opportunities to improve quality of services for individuals at risk for suicide and training for behavioral health care providers.

Data availability statement:

Our data source is medical records, which are not publicly accessible. The Stata code used to generate the findings is available upon request to the corresponding author at candon@upenn.edu, 3535 Market Street, 3rd Floor, Philadelphia, PA 19104, U.S.