Predicting Suicide Attempts for Racial and Ethnic Groups of Patients During Routine Clinical Care

Abstract

Objective

To examine variation in suicidal ideation and its relationship to risk of suicide attempt in the subsequent 90 days by race and ethnicity.

Method

Participants were all adults who completed a patient health questionnaire (PHQ9) used to assess depression symptoms and suicidal ideation during an outpatient encounter between 1/1/10 and 12/31/12 (n = 509,945 patients; n = 1,228,308 completed PHQ9s). The main outcome of the study was suicide attempt up to 90 days following the administration of a PHQ9. Data came from the Virtual Data Warehouse (VDW) from four healthcare systems in the Mental Health Research Network (MHRN).

Results

The sample was majority female (73.7%), primarily 30 – 64 years old (60.1%) and healthy (64.5% comorbidity index = 0), and over half were non-Hispanic white (52.9%). Only Asian patients (OR: 1.31; 95% CI: 1.24,1.39) had higher odds of reporting suicidal ideation when compared to non-Hispanic whites. All racial and ethnic groups had increased risk for suicide attempt with increased frequency of suicidal ideation.

Conclusions

The PHQ9 item 9 can be used as an indicator of suicidal ideation and risk for suicide attempt up to 90 days after the reported ideation in racial and ethnic minority patients during routine clinical care.

Introduction

A recent report from the National Center for Health Statistics found that death from suicide increased by 24% across all U.S. populations studied from 1999 to 2014 (Curtin, Warner, & Hedegaard, 2016). It is estimated that for every suicide death there are 25 attempts (“American Foundation of Suicide Prevention. Suicide Statistics,”), which clearly indicates there are multiple opportunities for prevention. We have also shown in large healthcare systems that almost all patients who attempted suicide (including racial and ethnic minorities) had some healthcare utilization before these attempts providing an opportunity for identification and intervention (Simon et al., 2016).

Increasing suicide rates have resulted in many national organizations working to improve identification and intervention strategies for those at highest risk (“The Zero Suicide Initiative,”). The Joint Commission issued a Sentinel Event Alert in February 2016, recommending that healthcare systems screen all patients across healthcare settings for suicide risk using a brief, standardized, evidence-based screening tool (“Sentinel Event Alert 56: Detecting and treating suicide ideation in all settings,” 2016). We have previously reported that response to item 9 of the commonly used Patient Health Questionnaire depression module (PHQ9) is a strong predictor of subsequent suicide attempt and suicide death among patients treated in large healthcare systems³ and this finding was replicated in patients treated in the Department of Veterans Affairs (VA) (Louzon, Bossarte, McCarthy, & Katz, 2016).

Rates of suicide attempt and death in the U.S. vary widely across racial and ethnic groups with the highest rates in non-Hispanic whites and Native Americans/Alaskan Natives (“American Foundation of Suicide Prevention. Suicide Statistics,” ; Curtin et al., 2016). To date, no studies have examined how well suicidal ideation, as measured with the PHQ9 item 9 response, predicts subsequent suicide attempt for different racial and ethnic groups. Understanding this relationship has important implications for the practical design and implementation of successful suicide screening and prevention strategies to meet national guidelines and quality metrics for depression care including the National Center for Quality Assurance measures for identification and treatment outcomes for depression (“National Committee for Quality Assurance (NCQA). HEDIS Depression Measures Specified for Electronic Clinical Data Systems,”).

Our study is designed to address gaps in the literature by examining the variation in suicidal ideation and its relationship to subsequent risk of suicide attempt 90 days following this reported ideation by race and ethnicity while controlling for factors that have been shown to impact suicidal ideation and attempts (Louzon et al., 2016; Simon et al., 2016). These have included gender, age, treatment for mental health conditions, comorbidity burden, and severity of depression symptoms.

Methods

Settings and Data Sources

Data were obtained from four healthcare systems in the Mental Health Research Network (MHRN): Kaiser Permanente of Washington (formerly known as Group Health Cooperative), HealthPartners, Kaiser Permanente of Colorado, and Kaiser Permanente of Southern California. These healthcare systems routinely assessed depression symptoms and suicidal ideation in 2010–2012 using the PHQ9 and provided comprehensive medical care, including specialty mental healthcare, to approximately 5 million members and/or patients in the states of California, Colorado, Idaho, Minnesota, Washington, and Wisconsin.

Members are enrolled through employer-sponsored insurance plans, individual insurance plans, and capitated Medicaid and Medicare programs, and are generally representative of each system’s regional population. All participating healthcare systems are mixed-models that provided care through both internal and external contract providers.

Data from electronic medical records, insurance claims and other administrative data systems from each organization are organized into a Virtual Data Warehouse (VDW). The VDW is a federated data system that facilitates sharing of de-identified data by using common data definitions and formats while keeping protected health information at each healthcare system (Ross et al., 2014). The local Institutional Review Board for Human Subjects at each site approved all study procedures and granted waivers of consent to use de-identified retrospective data for research.

Study Population

The study sample included PHQ9 responses recorded in electronic medical records between 1/1/2007 and 12/31/2012 from patients who were members at the time of the assessment. Health plan membership was defined as being enrolled for 12 months before the assessment. Patients could contribute multiple PHQ9s to data analyses if these were administered in separate encounters during the study period.

Measures

Depression severity and suicidal ideation

All four healthcare systems used the PHQ9 for assessment of patient depression severity as well as suicidal ideation at initial and follow-up visits for depression diagnoses and monitoring of treatment, but procedures for its use varied between healthcare systems and between clinics within healthcare systems. The first eight items of the PHQ9 ask about frequency of symptoms of depression, such as anhedonia, feeling depressed, or trouble with sleep. Item 9 of the PHQ9 asks, “over the last two weeks, how often have you been bothered by thoughts that you would be better off dead, or of hurting yourself in some way?” Response options for all items of the PHQ9 are “not at all” (0), “several days” (1), “more than half the days” (2) or “nearly every day” (3).

We used the total score of the first eight items on the PHQ9 as our indicator of depression severity, commonly referred to as the PHQ8 (Kroenke, Spitzer, & Williams, 2001). The PHQ8 is scored in the same way as the total score for all nine items and ranges from 0 to 27, with 0 – 4 indicating no or minimal depression symptoms, 5 – 9 mild, 10 – 14 mild to moderate, 15 – 19 moderate to severe, and 20 – 27 severe depression symptoms (Kroenke et al., 2001; Kroenke et al., 2009). We also abstracted the department in which the PHQ9 was administered (primary care, specialty mental health, other).

Suicide attempt

Nonfatal suicide attempts following completion of a PHQ9 were identified using electronic medical records (for services provided at healthcare system facilities) and insurance claims (for services provided by external providers or facilities). Among all encounters following completion of a PHQ9, three criteria were used to identify non-fatal suicide attempts: 1) ICD-9 diagnosis of self-inflicted injury or poisoning (E950 through E958), 2) ICD-9 diagnosis of injury or poisoning considered possibly self-inflicted (E980 through E988), and 3) ICD-9 diagnosis of suicidal ideation (V62.84) accompanied by a diagnosis of either poisoning (960 through 989) or open wound (870 through 897). We have previously reported that validation by clinician review of 200 full-text medical record notes found a weighted average positive predictive value to be 94.6% (Simon et al., 2013; Simon & Savarino, 2007; Simon, Savarino, Operskalski, & Wang, 2006). Electronic medical records and claims for identification of non-fatal suicide attempts were available through 12/31/12. At the time of the study, the transition to ICD-10 diagnoses in these healthcare systems had not occurred.

Race and ethnicity

All healthcare systems were implementing meaningful use requirements (Blumenthal & Tavenner, 2010) to collect self-reported race and ethnicity from their members at the time of the study. Typically, new and current members were asked to complete a self-report form that included separate questions for both their race and ethnicity. These forms were included in both membership applications, and at inpatient and outpatient visits. Responses were entered into the electronic medical record by healthcare system staff.

Racial and ethnic categories used for the current study were standardized across healthcare systems and followed national recommendations for mutually exclusive race categories (Institute of Medicine Subcommittee on Standardized Collection of Race/Ethnicity Data for Healthcare Quality, 2009; Taylor, Lopez, Martínez, & Velasco, 2012). Briefly this logic consisted of assigning Hispanic ethnicity even if a patient also reported a different race category. This follows the recommendations from a national survey of Hispanics living in the U.S. which found that Hispanic people considered themselves a race of people and not an ethnicity (Taylor et al., 2012). If a patient’s records contained two or more race categories (rather than a single category of “mixed race”), they were assigned the least prevalent race category in the U.S. population. For example, if a patient indicated they were both Native Hawaiian/Pacific Islander and non-Hispanic black, they were categorized as Native Hawaiian/Pacific Islander in our analyses. This was done to maximize our ability to understand differences for the least represented racial and ethnic minority patients. This is a convention used for analyses using the VDW (Institute of Medicine Subcommittee on Standardized Collection of Race/Ethnicity Data for Healthcare Quality, 2009).

Analyses

Descriptive statistics are provided for depression severity, suicidal ideation (no ideation [item 9 = 0] and ideation [item 9 score ≥ 0]), and suicide attempt (yes/no) 90 days after PHQ9 administration by race and ethnicity. As with our previous work (Simon et al., 2016), the analysis of the risk of suicide attempt 90 days (outcome) following a PHQ9 accounted for multiple PHQ observations per person and for censoring of availability of outcome data because individuals were no longer members of their health plan.

Logistic regression models were used to estimate the relative odds of reporting suicidal ideation at any visit across race/ethnicity groups, accounting for other patient characteristics. Partly conditional Cox proportional hazards regression models were used to estimate the relative hazards of suicide attempt following any visit across racial and ethnic groups of patients, accounting for other patient characteristics. Area under the curve (AUC) and standard error (SE) findings were also provided for each racial and ethnic minority group of patients to evaluate the predictive value of the PHQ9 item 9 for suicide attempts.

Each new questionnaire defined a new period at risk, and a patient could contribute multiple overlapping risk periods by completing multiple PHQs. Each suicide attempt could be linked to multiple prior PHQ results from a single patient. This approach examines risk based on data available when the PHQ was completed, regardless of subsequent questionnaires. It avoids informative censoring that would occur if the likelihood of completing a later PHQ was related to risk of a subsequent suicide attempt. Each suicide attempt risk period was censored at the time of disenrollment from the healthcare system, death, or last availability of suicide attempt data. Because predictors of repeat suicide attempts may differ from first attempts, PHQ9 observations following a suicide attempt were excluded from analyses of suicide attempts.

Because each individual patient could contribute multiple PHQ responses, the robust sandwich estimator (Taylor et al., 2012) was used to calculate confidence limits for hazard ratio estimates. All models adjusted for age, gender, co-morbidity burden, mental health treatment status at the time of the PHQ9, visit type where the PHQ9 was recorded, depression severity as measured by the PHQ8 total score, and healthcare system. We report adjusted odds ratios (aORs) and adjusted hazard ratios (aHRs) with corresponding 95% confidence intervals (CIs) using Non-Hispanic white as the referent population. Analyses were performed using SAS version 9.3.

Results

Population

There were 509,945 patients with 1,228,308 completed PHQ9s included in the study (see Table 1). The sample was 73.7% female, primarily 30 – 64 years old (60.1%) and healthy (64.5% comorbidity index = 0). Distribution of patients across racial and ethnic categories was as follows: 52.9% non-Hispanic white, 24.3% Hispanic, 7.8% non-Hispanic black, 6.9% Asian, 1.4% Hawaiian/Pacific Islander or Native American/Alaskan Native, and 6.6% other/mixed/unknown race/ethnicity. Many patients had received no mental health treatment in the past two years (44.9%) or had ongoing mental health treatment (40.5%) at the time of the PHQ9. PHQ9s were administered mostly in mental health visits (40.8%) followed by primary care (36%) and other types of visits (23.2%).

Table 1.

Descriptive characteristics for patients (n = 509,945) and patient health questionnaire (PHQ9) responses (n = 1,228,308) included in the study between 1/1/2010 and 12/31/2012. Data are presented by suicidal ideation in the past two weeks. Positive suicidal ideation corresponded to PHQ item 9 response categories “several days”, “more than half the days”, or “nearly every day” and was self-reported in 16.6% of the PHQ9 responses.

	Individuals (n = 509,945)		PHQs (n = 1,228,308)		Positive Ideation 16.6%; (n = 204,405)
Female	73.7%	375,624	72.1%	885,110	14.9%	132,267
Age Category (years)
18–29	25.1%	128,059	20.2%	247,883	14.6%	36,275
30–44	31.1%	158,545	28.3%	347,728	14.9%	51,813
45–64	29.0%	147,645	34.9%	429,029	20.2%	86,661
65+	18.6%	94,881	16.6%	203,668	14.6%	29,656
Race/Ethnicity
Non-Hispanic white	52.9%	269,796	65.3%	801,715	17.9%	143,229
Hispanic	24.3%	123,996	15.9%	195,292	10.6%	20,707
Non-Hispanic black	7.8%	39,959	6.6%	81,159	16.5%	13,395
Asian	6.9%	35,273	4.9%	60,513	13.0%	7,866
Hawaiian/Pacific Islander	0.6%	2,925	0.5%	5,948	17.3%	1,028
Native American/Alaskan Native	0.8%	4,189	1.2%	14,400	21.3%	3,061
Other/Mixed/Unknown	6.6%	33,807	5.6%	69,281	21.8%	15,119
Visit Type Where PHQ9 Was Collected
Specialty Mental Health	22.6%	115,389	40.8%	500,751	24.0%	120,296
Primary Care	55.9%	284,970	36.0%	442,509	10.8%	47,654
Other	35.3%	179,776	23.2%	285,048	12.8%	36,455
Healthcare Site
1	22.3%	113,519	31.8%	390,484	20.9%	81,794
2	11.5%	58,682	22.6%	277,834	18.8%	52,213
3	12.4%	63,330	13.7%	168,636	23.6%	39,727
4	53.8%	274,414	31.9%	391,354	7.8%	30,671
Mental Health Treatment Status at Time of PHQ9
No Treatment	44.9%	228,723	23.5%	289,040	4.6%	13,177
Past Treatment	10.2%	51,908	5.5%	68,157	8.5%	5,823
New Current Treatment	24.7%	126,081	13.2%	161,531	21.8%	35,267
Ongoing Treatment	40.5%	206,605	57.8%	709,580	21.2%	150,138
Comorbidity Burden
None	64.5%	329,152	61.9%	760,312	15.7%	119,400
One	17.1%	87,343	15.7%	192,499	18.4%	35,370
Two or More	16.2%	82,484	15.3%	188,448	18.9%	35,547

Racial and Ethnic Differences in Suicidal Ideation

Table 2 provides the results for the adjusted logistic regression estimates for the odds of expressing any level of suicidal ideation (1 – 3 for item 9) in different racial and ethnic groups compared to non-Hispanic whites. The odds of reporting suicidal ideation were strongly associated with increasing severity of depression symptoms (aOR range 6.32 – 20.90) and with ongoing mental health treatment (aOR: 1.98; 95% CI: 1.92,2.04). After accounting for these factors and other patient characteristics, only patients who reported Asian (aOR: 1.31; 95% CI: 1.24,1.39) or other/mixed/unknown race (aOR: 1.09; 95% CI: 1.04,1.13) had higher odds of self-reporting suicidal ideation when compared to non-Hispanic whites.

Table 2.

Adjusted odds ratios (aOR) and 95% confidence intervals (CI) for suicidal ideation (no ideation, any ideation) by race/ethnicity while controlling for age, gender, co-morbidity burden, mental health treatment status, visit type where PHQ9 was recorded, healthcare site, and depression severity as measured by the PHQ8. Any suicidal ideation was counted as responses to item 9 of the Patient Health Questionnaire (PHQ9) of “several days”, “more than half the days”, or “nearly every day” in the last two weeks.

	aOR	95% CI		p
Race/Ethnicity
Non-Hispanic white	[Reference]
Hispanic	1.03	0.99	1.07	0.10
Non-Hispanic black	1.02	0.97	1.07	0.48
Asian	1.31	1.24	1.39	< 0.001
Native Hawaiian/Pacific Islander	1.21	1.00	1.47	0.05
Native American/Alaskan Native	0.98	0.87	1.10	0.74
Other/Mixed/Unknown	1.09	1.04	1.13	< 0.001
Comorbidity Burden
None	[Reference]
One	1.09	1.05	1.12	< 0.001
Two or More	1.23	1.18	1.27	< 0.001
Gender
Male	[Reference]
Female	0.70	0.68	0.72	< 0.001
Age Category (years)
18–29	[Reference]
30–44	0.87	0.84	0.90	< 0.001
45–64	1.01	0.98	1.05	0.46
65+	1.00	0.95	1.04	0.87
Visit Type Where PHQ9 Was Collected
Specialty Mental Health	[Reference]
Primary Care	0.55	0.53	0.56	< 0.001
Other	0.81	0.79	0.84	< 0.001
Mental Health Treatment Status at Time of PHQ9
No Treatment	[Reference]
Past Treatment	1.33	1.28	1.39	< 0.001
New Current Treatment	1.86	1.80	1.92	< 0.001
Ongoing Treatment	1.98	1.92	2.04	< 0.001
Healthcare Site
Site 1	[Reference]
Site 2	0.91	0.87	0.95	< 0.001
Site 3	1.09	1.05	1.13	< 0.001
Site 4	0.80	0.77	0.83	< 0.001
Depression Severity (Total Score on PHQ8)
None to Mild (0 – 9)	[Reference]
Moderate to Moderately Severe (10 – 19)	6.32	6.18	6.46	< 0.001
Severe (≥ 20)	20.90	20.25	21.58	< 0.001

Prediction of Suicide Attempts for Different Racial and Ethnic Groups

Table 3 presents unadjusted results for the relationship between levels of suicidal ideation and subsequent suicide attempt within 90 days of this ideation by race and ethnicity. In general, for all racial and ethnic groups of patients, the unadjusted rates of 90-day subsequent suicide attempt were similar within each frequency category of ideation. However, non-Hispanic white, Hawaiian/Pacific Island, Native American/Alaskan Native patients, and those patients with other/mixed/unknown race and ethnicity had higher rates of suicide attempt with no evidence of ideation 90 days prior to the attempt (range 0.12% to 0.15%) than non-Hispanic black, Asian, and Hispanic patients (range 0.05% to 0.06%).

Table 3.

Unadjusted frequency of suicide attempt by levels of suicidal ideation for different racial and ethnic groups of patients. Frequency of suicidal ideation in the past two weeks is measured using the Patient Health Questionnaire (PHQ9) item 9 response categories of “not at all”, “several days”, “more than half the days”, or “nearly every day”. Data are presented as frequency (n).

	Not at All		Several Days		More than Half the Days		Nearly Every Day
	Responses	Attempts	Responses	Attempts	Responses	Attempts	Responses	Attempts
Non-Hispanic white	658,475	0.14% (920)	93,891	0.50% (473)	29,734	0.95% (281)	19,596	1.73% (339)
Hispanic	174,585	0.05% (86)	13,013	0.35% (46)	4,407	0.36% (16)	3,287	0.58% (19)
Non-Hispanic black	67,763	0.06% (72)	7,773	0.30% (23)	3,333	0.51% (17)	2,289	1.00% (23)
Asian	52,647	0.06% (33)	5,073	0.59% (30)	1,783	0.70% (13)	1,010	0.40% (4)
Hawaiian/Pacific Islander	11,336	0.15% (17)	1,821	0.71% (13)	699	0.43% (3)	541	0.55% (3)
Native American/Alaskan Native	4,920	0.12% (6)	582	0.17% (1)	227	0.44% (1)	219	2.28% (5)
Other/Mixed/Unknown	54,162	0.14% (78)	9,613	0.30% (30)	3,310	0.63% (21)	2,196	0.77% (17)

Table 4 shows the relative hazard of suicide attempt 90 days after reported ideation by race and ethnicity for patients who reported any level of suicidal ideation compared to those who did not report any ideation while controlling for factors associated with suicide attempt. A test for interaction or effect modification indicated that the relationship between suicidal ideation and hazard of subsequent suicide attempt varied significantly across racial and ethnic groups (p=.035). Although the hazard of suicide attempt generally increased with increasing frequency of ideation for all racial and ethnic groups of patients, the pattern of increase differed among racial and ethnic groups. The relative hazard of suicide attempt varied more across racial and ethnic groups when considering thoughts of suicide nearly every day (PHQ9 item 9 score = 3) with Native Hawaiian/Pacific Islander patients having the highest (aHR: 9.88; 95% CI: 1.85,52.74) and non-Hispanic black patients having the lowest (aHR: 3.34; 95% CI: 1.95,5.71) cumulative hazard of suicide attempt. The power for the PHQ9 item 9 to discriminate between those who would attempt suicide within 90 days and those who would not, as measured by the area under the curve (AUC), ranged from 0.63 (.03) in patients with other/mixed/unknown race and ethnicity to 0.73 (.03) in Asian patients (see Table 4).

Table 4.

Adjusted hazard ratios (aHR) with 95% confidence intervals (CI), and area under the curve (AUC) with standard error for suicide attempts 90 days after completion of the PHQ9 by race and ethnicity.^* Data are presented by item 9 responses. This question asks about thoughts of being better off dead or of hurting oneself in some way over the last two weeks. Responses are: “not at all” (0), “several days” (1), “more than half the days” (2), or “nearly every day” (3).

	PHQ9 Item 9 Score	aHR	95% CI
			Lower Limit	Upper Limit
Non-Hispanic white AUC = 0.69 (.01)	0	[Reference]
	1	2.16	1.94	2.40
	2	3.29	2.84	3.80
	3	5.43	4.41	6.67
Hispanic AUC = 0.69 (.02)	0	[Reference]
	1	2.44	1.78	3.34
	2	3.22	2.13	4.89
	3	4.57	3.10	6.73
Non-Hispanic black AUC = 0.66 (.03)	0	[Reference]
	1	1.47	1.02	2.12
	2	1.44	0.89	2.35
	3	2.66	1.55	4.55
Asian AUC = 0.73 (.03)	0	[Reference]
	1	2.41	1.47	3.97
	2	3.48	1.79	6.74
	3	3.18	1.73	5.82
Native Hawaiian/Pacific Islander AUC = 0.65 (.05)	0	[Reference]
	1	2.62	0.55	12.37
	2	3.83	0.77	18.96
	3	7.69	1.43	41.35
Native American/Alaskan Native AUC = 0.71 (.09)	0	[Reference]
	1	2.04	1.20	3.47
	2	3.14	1.26	7.83
	3	5.38	1.77	16.31
Other/Mixed/Unknown AUC = 0.63 (.03)	0	[Reference]
	1	2.04	1.56	2.66
	2	2.86	1.99	4.11
	3	4.67	2.92	7.49

^*Models controlled for age, gender, co-morbidity burden, mental health treatment status, visit type where PHQ9 was recorded, healthcare site, and depression severity as measured by the PHQ8; AUC = area under the curve; CI = confidence interval.

Discussion

The frequency of suicidal ideation as measured by the PHQ9 was associated with risk for suicide attempt in the subsequent 90 days for patients in all racial and ethnic groups, after controlling for patient demographics, mental health treatment, healthcare site, and depression severity. Taken together with our previous findings (Simon et al., 2016) and the findings of our colleagues in the VA (Louzon et al., 2016), this supports the use of the PHQ9 item 9 as a tool for the identification of risk for suicide attempt during routine clinical practice in all racial and ethnic groups of patients. The cumulative hazard of suicide attempt did vary among racial and ethnic groups when suicidal ideation was nearly every day (aHR 3.34 to 9.88).

It should be noted that if we had not accounted for other patient demographics, mental health treatment, healthcare site, and depression severity, our conclusions regarding frequency of suicidal ideation would have been different (see Table 1). In this case, we would have concluded that all racial and ethnic groups except Native Americans/Alaskan Natives and patients of other/mixed race/unknown race/ethnicity reported lower rates of suicidal ideation when compared to non-Hispanic whites. This emphasizes the importance of accounting for the severity of depression as well as demographic and treatment factors in comparing suicidal ideation among different racial and ethnic groups. We have also seen the importance of adjustment in models predicting depression medication adherence for racial and ethnic minority patients (Rossom et al., 2016).

Although the item 9 response of the PHQ9 showed sensitivity in predicting higher risk of suicide attempt with increasing frequency of ideation for most patients in our sample, the pattern of increase varied across racial and ethnic groups (see Table 3). For non-Hispanic black and Native Hawaiian/Pacific Islander patients there was not a clear relationship between ideation and suicide attempt until thoughts of suicide were nearly every day in the past two weeks. For Asian patients, there did not seem to be increasing risk of suicide attempt beyond reporting thoughts of suicide more than half the days in the past two weeks. The findings in non-Hispanic black patients support the national suicide trends (Curtin et al., 2016), and the differences when compared to non-Hispanic white patients may reflect cultural beliefs and practices that protect against suicide attempts (Wang, Joel Wong, Tran, Nyutu, & Spears, 2013). However, the sample size for some racial and ethnic groups in our population, especially Native Hawaiian/Pacific Islanders and Native Americans/Alaskan Natives was very small, resulting in large confidence intervals for our estimates. Findings for these groups of patients should be treated with caution until larger studies are done with this unique group of patients specific to depression severity and suicidal ideation and behavior.

There are several limitations to these study conclusions. One of the most significant is that even in this large sample, we were not able to estimate prediction models for suicide deaths because the rates of suicide were so low in some ethnic and minority groups. Future studies in this area should use substantially more healthcare systems to study the predictive value of the PHQ9 item 9 for suicide deaths in racial and ethnic minority patients. In addition, PHQ9 utilization rates in the healthcare systems studied were still low during the study period. Patients may have been given a PHQ9 for specific reasons such as having a high comorbidity burden, having a high risk for self-harm, or of having a mental health condition (34% of our population had at least one comorbid mental health condition). Although we controlled for comorbidity burden and mental health treatment in our analyses, findings for suicidal ideation and subsequent attempt in racial and ethnic minorities might be very different if the PHQ9 had been done in the general patient population served by the healthcare systems in the study. The newest screening recommendations from the National Center for Quality Assurance (“National Committee for Quality Assurance (NCQA). HEDIS Depression Measures Specified for Electronic Clinical Data Systems,”), where every patient with a depression diagnosis is to be evaluated periodically with the PHQ9, will provide an opportunity to re-examine our findings for a larger population of patients in the next few years.

Another limitation of our study, also related to generalizability, is that our patients had healthcare insurance through an employer group. This limits our conclusions for low-income, indigent patients who might be comprised of a higher proportion of racial and ethnic minority groups. Finally, there was a small proportion of patients (see Table 3), more in some racial and ethnic groups than others, who attempted suicide without any self-reported thoughts of suicide 90 days before the attempt. We have reported this finding in our previous work (Simon et al., 2016) as has the Veteran’s Administration (Louzon et al., 2016). As suggested by investigators at the VA (Louzon et al., 2016), it might be wise to include depression severity (as measured by the PHQ8) as another indicator of possible distress in addition to item 9 of the PHQ9.

Finally, the power of the PHQ9 item 9 to discriminate between patients who would and would not attempt suicide 90 days after administration was modest for all racial and ethnic groups of patients in our study (overall AUC = 0.70 [.03]), meeting minimal standards for any predictive test (Simundic, 2009). These findings are supported by a recent meta-analysis by Franklin and colleagues (2017) which found that after 50 years of research into accurate predictors of suicide, most instruments currently available had limited predictive ability and as such, finding effects for moderating variables such as race and ethnicity was also be limited. Their suggestion was to move beyond the standard regression predictor approaches to complex, machine learning algorithm development with a broader range of variables and real time, repeated monitoring of symptomology and behavior.

In conclusion, we found no large differences among races and ethnicities in the rates of suicidal ideation and subsequent 90-day suicide attempt if depression severity at the time of the ideation, treatment for mental health conditions, healthcare site, and other patient demographics were considered. The relationship between increasing frequency of suicidal ideation and subsequent 90-day suicide attempts was not proportional for all racial and ethnic groups of patients; however, reporting thoughts of suicide nearly every day in the last two weeks was a significant predictor of suicide attempt for all racial and ethnic groups of patients, regardless of depression severity, healthcare system, or other patient factors. The PHQ9 item 9 questionnaire has acceptable predictive accuracy for suicidal ideation and can be used to monitor risk of suicide attempt in routine clinical care for all racial and ethnic groups of patients.