Abstract
Objectives:
To describe the implementation process and assess results of a large-scale universal depression screening program with pathways to suicide risk screening in a pediatric integrated delivery network.
Study design:
This retrospective study analyzes depression and suicide risk screening data for 95,613 patients ages 12 to 17 years.
Results:
Of the 95,613 adolescent patients who were screened for depression, 2.4% (2,266) screened positive for risk for moderate-severe depression (>10 Patient Health Questionnaire; PHQ; 9-item version) and 4.1% (3,942) endorsed elevated suicide risk (≥1 Columbia Suicide Severity Rating Scale; C-SSRS). Overall, 51% of screened patients who present with a primary psychiatric concern screened positive for elevated risk of suicide (2,132). Two percent of screened patients who presented with a primary medical concern screened positive for elevated risk of suicide. Nearly half (45.9%) of all elevated suicide risk screenings were from patients with a primary medical concern.
Conclusions:
A large-scale universal depression screening program with a pathway to identify elevated suicide risk was implemented in a pediatric healthcare system using the PHQ and the C-SSRS. This screening program identified youth with moderate-severe depression and elevated risk for suicide with and without presenting psychiatric concerns across service settings.
Keywords: prevention, implementation
Adolescent depression is associated with a greater risk of substance use disorders, early pregnancy, poor academic performance, impaired social functioning, and suicide1,2,3. Suicide was the second leading cause of death for children, youth, and young adults ages 10–24 in the US in 2017 (19.2% of deaths)4. US children’s hospital emergency departments (ED) and inpatient units confronted a two-fold increase in suicidal ideation and attempts between 2008 and 20155,6. These trends became more acute with the Coronavirus pandemic as the proportion of psychiatry-related visits among pediatric EDs began to increase in April 2020 and overall ED visits declined. Compared with the same period in 2019, psychiatry-related ED visits in 2020 for children aged 5-11 increased 24% and for youth aged 12 to 17 increased 31%7.
As the rates of suicide have increased nationally8, medical settings often serve as the initial point of contact for individuals with psychiatric needs, particularly those with urgent or critical needs and few other supportive resources9,10,11,12. In an analysis of 40 studies of adults with healthcare contact prior to suicide, 45% of those who died by suicide had contact with a primary care professional within one month prior to death, far exceeding the rate of contact with behavioral health professionals (approximately 19%)13. This highlights the need to embed depression and suicide awareness and prevention measures within pediatric primary care settings9,10,14,15.
Behavioral health services have been increasingly integrating with other types of healthcare and the role of pediatric health care professionals has similarly expanded to include identifying elevated risk for suicide and delivering interventions to prevent suicide among youth16,17. The growing literature has highlighted the feasibility, acceptability, and benefits of screening programs in hospitals18, primary care, and ED settings20. The Joint Commission promotes the dissemination of suicide risk screening in healthcare organizations through the National Patient Safety Goal requiring suicide screening among all patients who are being “evaluated or treated for behavioral health conditions as their primary reason for care…” (NPSG Requirement 15.01.01)21. However, non-universal screening, focused only on those children presenting to urgent care clinics or EDs with a primary psychiatric concern, poses challenges for accomplishing the goal of preventing suicides as many at-risk children may be missed22,20.
A health care system detected suicide risk in approximately 3% of pediatric health care encounters, with the highest rates among adolescents presenting with psychiatric problems in emergency departments23. Tiered clinical pathways for suicide risk detection standardize essential care provision24.
Implementation of Universal Depression Screening and Suicide Risk Screening Pathway
In 2016, Rady Children’s Hospital, San Diego (RCHSD) launched an initiative to universally screen all youth ages 12 to 17 for depression with a pathway for suicide risk screening, regardless of the primary reason for a patient’s visit. With increasing rates of behavioral health crisis identified among RCHSD patients, the screening project was developed with an emphasis on depression screening to identify patients who might benefit from intervention as early as possible, while still developing a pathway to address suicide risk.
Senior leadership committed to the effort and identified key leaders for the initiative. Formalizing roles and responsibilities helped to promote sustained engagement in the implementation effort and steady progress. The system-wide initiative included coordination from a multi-disciplinary team including physicians, registered nurses, medical social workers, industrial engineers, and others with expertise in information technology, clinical informatics, quality management, and psychiatry. The team met biweekly to develop and implement screening procedures.
To select screening instruments stakeholders consulted with colleagues at other pediatric hospitals, reviewed empirical support for the scores provided by the tools to identify adolescents with depression or at risk for suicide, balanced with tool feasibility based on time to complete the tool, response burden, ease of scoring, cost of implementation, and expertise required to administer the tools. Equally important to selecting the specific tools for screening were the subsequent actions and interventions provided based on screening results.
Developing process maps helped define key actions, roles, responsibilities, and timelines, which helped to promote stakeholder buy-in and participation in the process25. For example, the medical social work team could develop strategies for consistent safety planning interventions, discharge practices, and referral pathways in collaboration with physicians and nurses in different service settings. The customer service team helped define an information management pathway to support care connection with families of patients who were provided referrals to outpatient behavioral health services and discharged home.
The goals of the universal depression screening program with pathways to suicide risk were to screen all patients ages 12 years and older for depression at each encounter (emergency department, primary care, urgent and acute care, inpatient specialty care), or within every 30 days of each outpatient specialty encounter. In addition, identify those at risk for moderate to severe depression (score ≥10 Patient Health Questionnaire; PHQ-9), and identify those endorsing suicidal ideation (PHQ item 9) and further assess suicidal ideation or behaviors (Columbia Suicide Severity Rating Scale; C-SSRS). We also developed a safety plan with the patient and caregiver for those at risk, including transfer to acute psychiatric services or discharge with caregiver support and appropriate follow-up behavioral health services. We also confirmed that patients at elevated risk for severe depression and suicide are connected with a behavioral health provider after discharge.
The initial March 2016 launch began in the endocrinology specialty clinic, followed in April by. a staged implementation with compliance monitoring in specialty clinic areas with chronically ill patients (e.g., neurology, nephrology, pulmonology). Operational adjustments within service units and across the organization have occurred gradually after the rollout (Figure 1). This program has required ongoing iterations to workflow based on direct-service-provider feedback and suggestions, adjustments to electronic medical record tools to automate screening steps and interventions, and real-time reporting tools for managers to quickly intervene when steps were missed or other appropriate actions were not taken. A multidisciplinary taskforce was convened for ongoing biweekly meetings to review workflow, policy, compliance and quality metrics, and outcomes of the screening program.
Figure 1.
Items were administered verbally by a medical assistant without caregivers present and responses concurrently entered into the electronic health record. The training for medical assistants to administer the screening tools included guidance on how to describe the screening if patients are hesitant, what to do if responses seem incongruent to the patient’s presentation (i.e., consulting with a social worker or nurse), and emphasizing the screening questions are focused on specific timeframes (e.g., the past two weeks for the PHQ). Training for staff also included instruction on reading the questions verbatim and not offering an interpretation of the item (i.e., if a patient struggles with their response staff responds with, “Pick the answer that you feel is best.”). The screening was available for all patients regardless of language spoken, with interpreter services when needed. Patients determined to be unable to provide valid responses to the screening tools based on intellectual disabilities, impaired consciousness, or impaired functioning were not screened.
Of note, the screening program was developed without the addition of behavioral health resources. Workflow automation and decision support tools were used to ensure that patients were connected to providers based on their screening responses.
Methods
Statistical analyses are based on retrospective medical record reviews of adolescent patients (12-17 years old) who completed a hospital encounter within RCHSD from April 11, 2016 to August 31, 2020. The scientific and institutional review board at the University of California, San Diego approved this study. RCHSD is a quaternary care, trauma, teaching, and research institution affiliated with the University of California, San Diego. Rady Children’s Health Network (RCHN) is a Southern California based pediatric integrated delivery network comprised of RCHSD and pediatric primary and specialty care providers all using the same electronic medical record. RCHSD is the primary provider of health care to the more than 700,000 children and youth in San Diego County26 and some children and youth living in Imperial and Southern Riverside counties. San Diego County is ethnically and racially diverse with children and youth ages 0-18 who are Hispanic or Latinx (41.5%), White (37.1%), Asian (10.8%) Black (4.8%), American Indian (0.5%), Pacific Islander (0.4%), and two or more races (5.0%)27.
Universal Depression Screening Protocol and Suicide Risk Screening Pathway
Universal depression screening was conducted with the 2-item and 9-item versions of the Patient Health Questionnaire (PHQ-2; PHQ-9)28,29. The PHQ-2 is a two-item nonproprietary, self-report instrument used to identify risk for major depressive disorder. The PHQ-2 items comprise the first two items of the PHQ-9, assessing anhedonia and hopelessness in the past two weeks, two diagnostic markers for major depressive disorder. The PHQ-9 includes seven additional depressive symptoms, including appetite, sleep, psychomotor behaviors, and suicidal ideation. Respondents indicate the frequency with which they had that problem within the last 2 weeks on a scale from 0 (“Not at all”) to 3 (“Nearly every day”). Patients were first administered the PHQ-2; those screening positive for depressive symptomatology on the PHQ-2 (score ≥ 3) were then administered the remaining questions on the PHQ-9. All adolescents who endorsed moderate-severe depression symptomology (PHQ-9 score of 10-19) but not elevated risk for suicide met with a behavioral health professional, typically a licensed clinical social worker, and were provided behavioral health-related educational materials and service referrals.
The suicide risk screening pathway included patients who endorsed PHQ item 9 (thoughts of being better off dead or thoughts of self-harm), who were then administered the 6-item screener version of the C-SSRS30. The 6-item C-SSRS screener is commonly used in pediatric healthcare settings with youth ages 12 to 18 to detect elevated suicide risk and has been translated into numerous languages31. Item content is focused on either suicidal ideation or suicidal behaviors (i.e., actions taken to prepare for suicide or previous suicide attempts). Adolescents then met with a behavioral health professional for safety assessment and safety planning32, typically with caregivers/parents. Information gathered from the safety assessment informed the discharge disposition. Discharge disposition options included a referral to the emergency department or inpatient psychiatric hospitalization, discharge home with caregiver/parent monitoring with access to suicide means assessed and restricted, an appointment made within 24-hours with a behavioral health practitioner at the RCHSD Behavioral Health Urgent Care walk-in clinic, and continued services with an existing behavioral health provider. Patients discharged after being identified as at risk for moderate-severe depression or at elevated risk for suicide were subsequently contacted by RCHSD customer service staff or behavioral health care coordinators to ensure that the patient had successfully transitioned to follow-up services or to provide additional care coordination as needed.
Measures
Depression and Suicide Risk Screening Outcome Categories.
Based on adolescent responses to the PHQ-9 and C-SSRS, screening encounters were classified into 3 outcome categories for this study: 1) negative depression (PHQ-2 < 3 or PHQ-9 score < 10) and negative suicide risk (CSSRS = 0), 2) positive depression (PHQ-9 score ≥ 10) and negative suicide risk (CSSRS = 0), and 3) positive suicide risk (CSSRS > 0). The PHQ-2 and PHQ-9 have been evaluated for psychometric performance in healthcare settings among youth and performed well33. Richardson et al suggested screening with the PHQ-2 and administering the remaining PHQ-9 items if the youth PHQ-2 score exceeds 228. Total PHQ-9 scores of ≥10 are categorized as moderate, with scores of ≥20 categorized as severe29. Others have found item 9 from the PHQ-9 offers predictive information about suicide attempts and suicide death over the subsequent two years34 and has been used to prompt further assessment of suicidal ideation and behaviors34. However, researchers have also demonstrated poor suicide risk sensitivity through depression screening alone (i.e., nearly one-third of youth at risk for suicide were not detected with depression screening alone)36. Psychometric evidence for scores produced by the C-SSRS has been demonstrated among pediatric populations in the US, Turkey, and Denmark, including internal consistency, predictive validity, sensitivity, and specificity30,37,38,39,40,41.
Patient Classification by Encounter Department.
Based on the encounter department for where the screening occurred, patients were grouped into five categories: 1) Outpatient Specialty Care (n = 105,901; 51.50%), 2) Emergency Department (n = 61,336; 29.83%),3) Inpatient Medical Unit (n = 7,220; 3.51%), 4) Primary Care (n = 25,816; 2.47%), and 5) Urgent Care (n = 5,075; 2.47%).
Diagnostic Classification Systems.
We relied on the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM)42 coding system to determine classification to the medical versus psychiatric health group status.
Demographics and Clinical Covariates.
Demographic and clinical characteristics from the electronic health record have been found to influence depression and suicidal risk in recent research. Such variables include patient age43,15,2,44, sex43,15,2,44, race44,45 and existing psychiatric conditions46,47.
Data Analysis
During the study timeframe there were 205,650 screening encounters with patients ages 12 to 17. Encounters were excluded if screening was administered or documented incorrectly (eg, missing data; negative PHQ-2 responses but remaining PHQ-9 items administered; ‘no’ response to PHQ-9 item 9 but CSSRS administered; n = 497), or if there were duplicate encounters when patients transitioned between service units during their encounter (n = 134). The total number of unique patients who were screened for depression with pathways for suicide risk screening (≥1 encounter) was 95,613 youth (Figure 2). We examined the distributions of demographics and clinical characteristics based on classification into the three depression and suicide risk screening outcome categories (negative depression, negative suicide risk; positive depression, negative suicide risk; positive depression, positive suicide risk; Table 1), and ICD classification as primary psychiatric or medical classification based on ICD diagnostic codes; Table 2. Lastly, we examined PHQ-9 and C-SSRS sum scores by outcome categories of depression and suicide risk screening and medical versus psychiatric ICD categorization. In addition to screening categories, common sum scores were calculated based on summing PHQ-9 rating scale scores from 0-3 for each item (sum score range 0-27) or endorsed individual C-SSRS items; ‘yes’=1, ‘no’=0 (sum score range 0-6). Between-group differences (negative depression, negative suicide risk versus positive depression, negative suicide risk versus positive depression, positive suicide risk; and primary medical concern versus primary psychiatric concern) were assessed using t-test for age, the number of encounters (treated as a continuous variable), and chi-square tests or Fisher exact tests for categorical variables (sex/gender, race/ethnicity, and admitting department). Data preparation and analyses were performed in R, version 1.3.109348.
Figure 2.
Table 1.
Patient depression and suicidality screening groups with admitting department and demographics for N=95,613
Characteristics (n, %/mean, SD) | Total Number of Patients Screened (N=95,613) |
Negative Depression, Negative Suicide Risk (n=89,405) |
Positive Depression, Negative Suicide Risk (n=2,266) |
Positive Depression, Positive Suicide Risk (n=3,942) |
P Value |
---|---|---|---|---|---|
Admitting Department | |||||
Emergency Department | 31,610 (33.1%) | 28,070 (31.4%) | 666 (29.4%) | 2,874 (72.9%) | <.001 |
Inpatient Medical Floor | 2,675 (2.8%) | 2,454 (2.7%) | 53 (2.3%) | 168 (4.3%) | |
Outpatient Specialty Care | 38,668 (40.4%) | 37,168 (41.6%) | 973 (42.9%) | 527 (13.4%) | |
Primary Care | 19,645 (20.5%) | 18,842 (21.1%) | 484 (21.4%) | 319 (8.1%) | |
Urgent Care | 3,015 (3.2%) | 2,871 (3.2%) | 90 (4%) | 54 (1.4%) | |
Age in years | 14.5 (1.9) | 14.4 (1.9) | 15.0 (1.8) | 14.8 (1.7) | <.001 |
Sex | |||||
Female | 46,963 (49.1%) | 42,643 (47.7%) | 1,478 (65.2%) | 2,842 (72.1%) | <.001 |
Male | 48,644 (50.9%) | 46,756 (52.3%) | 788 (34.8%) | 1,100 (27.9%) | |
Unknown | 6 (0%) | 6 (0%) | 0 (0%) | 0 (0%) | |
Race/Ethnicity | |||||
White | 29,802 (31.2%) | 27,697 (31%) | 739 (32.6%) | 1,366 (34.7%) | <.001 |
Black | 3,909 (4.1%) | 3,591 (4%) | 115 (5.1%) | 203 (5.1%) | |
Hispanic/Latino | 42,898 (44.9%) | 40,248 (45%) | 1,001 (44.2%) | 1,649 (41.8%) | |
Asian/Pacific Islander | 5,639 (5.9%) | 5,323 (6%) | 106 (4.7%) | 210 (5.3%) | |
American Indian/Alaska Native | 5,639 (5.9%) | 164 (0.2%) | 10 (0.4%) | 8 (0.2%) | |
Unknown | 12,112 (12.7%) | 11,366 (12.7%) | 278 (12.3%) | 468 (11.9%) | |
Refused | 1,071 (1.1%) | 1016,(1.1%) | 17 (0.8%) | 38 (1%) |
Table 2.
Patients by visit type, encounter department, screening outcome categories, depression and suicide risk symptomatology
(n,%/mean,sd) | Total Number of Patients Screened (N=95,613) |
Total Number of Active Screening Months (to 8/31/2020) |
Presented with Primary Medical Concern n= 91,436 |
Presented with Primary Psychiatric Concern n= 4,177 |
P value |
---|---|---|---|---|---|
Encounter Department | |||||
Specialty Clinic | 38,668 | 53 | 38,160 (98.7%) | 508 (1.3%) | <.001 |
ED | 31,610 | 39 | 28,153 (89.1%) | 3,457 (10.9%) | |
Primary Care | 19,645 | 16 | 19,590 (99.7%) | 55 (0.3%) | |
Urgent Care | 3,015 | 9 | 3,002 (99.6%) | 13 (0.4%) | |
Inpatient Unit | 2,675 | 39 | 2,531 (94.6%) | 144 (5.4%) | |
Screening Outcome Category | |||||
Negative Depression | 89,405 | 87,640 (98.0%) | 1,765 (2.0%) | <.001 | |
Positive Depression | 2,266 | 1,986 (87.6%) | 280 (12.4%) | ||
Elevated Suicide Risk | 3,942 | 1,810 (45.9%) | 2,132 (54.1%) | ||
PHQ-9 Sum Score | 13.0 (6.4) | 11.4 (6.0) | 16.8 (5.8) | <.001 | |
C-SSRS Sum Score | 4.1 (1.5) | 3.9 (1.7) | 4.2 (1.5) | <.001 |
There were significant differences in encounter department (p= <.001), screening category (p= <.001), PHQ-9 sum score (p= <.001), and C-SSRS sum score (p= <.001).
Results
Table 1 describes the demographic characteristics (age, sex, race/ethnicity, admitting department) of patients (n=95,613) who were screened throughout the implementation of the universal depression with a pathway for suicide risk screening program from April 11, 2016, to August 31, 2020. The mean age was 14.5 (SD 1.9) years, and approximately half (50.9%) were male. The group was racially diverse, with 45% identifying as Hispanic/Latino, 31% as White, 6% as Asian/Pacific Islander, and 4% as Black. The majority were evaluated in outpatient specialty clinics (~40%) or the emergency department (~30%). About 13% were seen in primary care, and the remainder were admitted to the inpatient units (~4%) or seen in the urgent care department (~3%).
There were significant differences in age (p <.001), sex (p <.001), race/ethnicity (p<.001), encounter department and number of encounters by screening outcome category with a total of 93.5% (89,405) screening negative for moderate-severe depression with no suicide risk, 2.4% (2,266) screening positive for moderate-severe depression with no suicide risk, and 4.1% (3,942) screening positive for elevated suicide risk. Compared with adolescents who screened negative for moderate-severe depression with no risk for suicide, adolescents who screened positive for moderate-severe depression and for elevated risk of suicide were more likely to be older in age and female. The average number of encounters for each patient (an indicator of patient service utilization) was highest within the elevated suicide risk outcome group (average number of encounters was 2.5 per patient), and lowest with the moderate-severe depression outcome group (average number of encounters was 2.0 per patient; overall average 2.2, SD 2.5).
Table 2 describes patients by visit type, encounter department, screening outcome categories, PHQ-9, and C-SSRS sum scores. Of the 95,613 who were screened for depression and suicide risk, 4.4% (4,177) were patients with a primary psychiatric concern, and 95.6% (91,436) presented with a primary medical concern. The majority (82.8%/3457) of patients presenting with a primary psychiatric concern were seen in the emergency department, and only one-third (30.8%/28,153) of patients presenting with a primary medical condition were seen in the emergency department. Sixty-three percent (57,750/91,436) of the patients presenting with a primary medical condition were treated in non-acute outpatient settings (outpatient specialty care, primary care), and 13.5% (563/4177) of the patients who presented with a primary psychiatric concern were treated in non-acute outpatient settings. Fifty-one percent (2,132) of patients who presented with a primary psychiatric concern screened positive for moderate-severe depression and elevated suicide risk, and 2% of patients who presented with a primary medical condition screened positive for moderate-severe depression and elevated suicide risk. Almost 7% of patients who presented with a primary psychiatric concern screened positive for risk of moderate-severe depression and no suicide risk, and about 2% of patients who presented with a primary medical condition screened positive for risk of moderate-severe depression and no suicide risk. The average PHQ-9 and C-SSRS sum scores were highest among patients with a primary psychiatric concern (16.8 for primary psychiatric concern versus 11.4 among patients with a primary medical concern).
Discussion
In a large pediatric integrated delivery network, youth ages 12 to 17 were universally screened for depression in the ED, on inpatient medical floors, in urgent care, primary care, and specialty clinic settings, regardless of primary presenting concern (medical versus psychiatric) and with a pathway for subsequent suicide screening. Universal depression screening resulted in 2.4% of screened patients reporting moderate-severe depression without suicide risk. The rate of positive suicide risk screens (4.1%) was higher than that of previous studies on universal suicide screening with a similar pediatric healthcare sample (2.9% positive suicide risk screens)23.
Targeted depression with a pathway to elevated risk of suicide screening designed only for those with a primary psychiatric concern will identify only a portion of the youth experiencing depression or suicidality symptoms. In this case, targeted screening of youth presenting with a primary psychiatric concern would have missed 87.6% (1,986) of the total youth reporting depression (but not suicide risk) and 45.9% (1,810) of the total number of youth endorsing suicide risk. Youth who endorsed suicide risk also had significantly more hospital encounters than other patients, averaging 2.5 encounters per patient, suggesting patients with suicide-related needs utilize more hospital services than patients with moderate-severe depression or no depression.
About 83% of pediatric patients presenting with a primary psychiatric concern were evaluated in the emergency department. These findings have several clinically relevant implications. First, the transportability of brief (one-session) depression and suicide risk interventions with safety planning are critical for delivery in emergency health care settings, with a focus on developing stable linkages for patients and families to initiate intensive outpatient and outpatient behavioral health treatment. Second, care coordination teams in non-acute settings, such as primary care and outpatient specialty clinics should accompany depression and suicide screening programs. Care coordination teams can properly identify and support patients who utilize the emergency department as their behavioral health provider and better link those patients to outpatient and intensive outpatient services by assessing barriers and facilitators to care.
The age for screening in this example was 12 to 17 years; however, emergent research has highlighted the prevalence of depression and suicide risk among children younger than 12 years old43,23. Also, in this study depression screening item endorsement was an eligibility criterion for administering the suicide risk screening tool. However, there is research that suicide-related behaviors are not necessarily connected to depression and, in many cases, can be connected with non-depressed anxiety36,49. Although our system is aligned with Joint Commission National Patient Safety Goal for suicide risk screening21, we are exploring ways of including universal suicide screening rather than a pathway activated through universal depression screening and lowering the age of screening to include 10- and 11-year-old patients.
The screening system included best efforts to reduce self-reporting bias with depression and suicidality, however self-report data for psychiatric symptomology may be limited due to stigma, fear of intervention, repeated administration fatigue, or other respondent biases50,51,52. This study was cross-sectional and did not include measures of treatment engagement, so we are unable to determine how many of the adolescents identified by the screening were offered referrals for or engaged in treatment. This study was also conducted in a single pediatric healthcare system, which may limit the generalizability of the screening results.
Universal screening of depression with pathways for suicide risk screening in pediatric healthcare systems provided early opportunities to identify youth in need of support, which can be leveraged to increase access to treatment. Given the rising rates of depression and suicide among youth and the frequency of ED utilization for psychiatric emergencies, depression and suicide screening in pediatric healthcare systems provides a safety net to recognize when youth are at-risk and could benefit from intervention. Depression and suicide screening programs should include patients presenting with psychiatric concerns as well as those presenting with medical concerns. These findings contribute to the increasingly evident need for future longitudinal implementation and intervention research to equip providers with tools to address the psychiatric needs of youth seeking health care.
Acknowledgments:
We acknowledge Margareta (Meg) Norton, R.N. Strategic Consultant to the Rady Children’s Hospital, San Diego President and CEO for her visionary leadership in support of this program, and the many other RCHSD leaders and staff who worked so diligently to resolve the barriers required to implement this initiative.
Supported by the National Institute on Drug Abuse grants U01 DA041089 (to L.A.). The content is solely the view of the authors and does not necessarily represent the official view of the NIH.
Abbreviations:
- PHQ
Patient Health Questionnaire
- C-SSRS
Columbia Suicide Severity Rating Scale
- RCHSD
Rady Children’s Hospital-San Diego
- ED
Emergency Department
Footnotes
The authors declare no conflicts of interest.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
6. References
- 1.Siu AL. Screening for depression in children and adolescents: US Preventive Services Task Force recommendation statement. Annals of internal medicine. 2016. Mar 1;164(5):360–6. [DOI] [PubMed] [Google Scholar]
- 2.Brent DA, Baugher M, Bridge J, Chen T, Chiappetta L. Age-and sex-related risk factors for adolescent suicide. Journal of the American Academy of Child & Adolescent Psychiatry. 1999. Dec 1;38(12):1497–1505. [DOI] [PubMed] [Google Scholar]
- 3.Rohde P, Lewinsohn PM, Seeley JR. Comorbidity of unipolar depression: II. Comorbidity with other mental disorders in adolescents and adults. Journal of abnormal psychology. 1991. May;100(2):214. [PubMed] [Google Scholar]
- 4.Heron MP. Deaths: leading causes for 2017. [Google Scholar]
- 5.Murphy SL, Xu J, Kochanek KD, Arias E. Mortality in the united states, 2017. [PubMed] [Google Scholar]
- 6.Plemmons G, Hall M, Doupnik S, Gay J, Brown C, Browning W, Casey R, Freundlich K, Johnson DP, Lind C, Rehm K. Hospitalization for suicide ideation or attempt: 2008–2015. Pediatrics. 2018. Jun 1;141. [DOI] [PubMed] [Google Scholar]
- 7.Leeb RT, Bitsko RH, Radhakrishnan L, Martinez P, Njai R, Holland KM. Mental health-related emergency department visits among children aged< 18 years during the COVID-19 pandemic—United States, January 1-October 17, 2020. Morbidity and Mortality Weekly Report. 2020. Nov 13;69(45):1675. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hedegaard H, Curtin SC, Warner M. Increase in suicide mortality in the United States, 1999–2018. [PubMed] [Google Scholar]
- 9.Committee on Psychosocial Aspects of Child and Family Health. The future of pediatrics: mental health competencies for pediatric primary care. Pediatrics. 2009. Jul 1;124(1 ):410–21. [DOI] [PubMed] [Google Scholar]
- 10.Wintersteen MB, Diamond GS. Youth suicide prevention in primary care: A model program and its impact on psychiatric emergency referrals. Clinical Practice in Pediatric Psychology. 2013. Sep;1(3):295. [Google Scholar]
- 11.Farberow NL. Suicide prevention in the hospital. Psychiatric Services. 1981. Feb;32(2):99–104. [DOI] [PubMed] [Google Scholar]
- 12.Cheung AH, Dewa CS, Levitt AJ, Zuckerbrot RA. Pediatric depressive disorders: management priorities in primary care. Current opinion in pediatrics. 2008. Oct 1;20(5):551–9. [DOI] [PubMed] [Google Scholar]
- 13.Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. American Journal of Psychiatry. 2002. Jun 1;159(6):909–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Davis M, Rio V, Farley AM, Bush ML, Beidas RS, Young JF. Identifying adolescent suicide risk via depression screening in pediatric primary care: an electronic health record review. Psychiatric services. 2021. Feb 1;72(2):163–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Aguinaldo LD, Sullivant S, Lanzillo EC, Ross A, He JP, Bradley-Ewing A, Bridge JA, Horowitz LM, Wharff EA. Validation of the ask suicide-screening questions (ASQ) with youth in outpatient specialty and primary care clinics. General hospital psychiatry. 2021. Jan 1;68:52–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Burkhart K, Asogwa K, Muzaffar N, Gabriel M. Pediatric integrated care models: A systematic review. Clinical pediatrics. 2020. Feb;59(2):148–53. [DOI] [PubMed] [Google Scholar]
- 17.Duong MT, Bruns EJ, Lee K, Cox S, Coifman J, Mayworm A, Lyon AR. Rates of mental health service utilization by children and adolescents in schools and other common service settings: A systematic review and meta-analysis. Administration and Policy in Mental Health and Mental Health Services Research. 2020. Sep 17:1–20. [DOI] [PubMed] [Google Scholar]
- 18.Roaten K, Johnson C, Genzel R, Khan F, North CS. Development and implementation of a universal suicide risk screening program in a safety-net hospital system. The Joint Commission Journal on Quality and Patient Safety. 2018. Jan 1;44(1):4–11. [DOI] [PubMed] [Google Scholar]
- 19.Zuckerbrot RA, Maxon L, Pagar D, Davies M, Fisher PW, Shaffer D. Adolescent depression screening in primary care: feasibility and acceptability. Pediatrics. 2007. Jan 1;119(1):101–8. [DOI] [PubMed] [Google Scholar]
- 20.Ballard ED, Cwik M, Van Eck K, Goldstein M, Alfes C, Wilson ME, Virden JM, Horowitz LM, Wilcox HC. Identification of at-risk youth by suicide screening in a pediatric emergency department. Prevention science. 2017. Feb 1;18(2):174–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Joint Commission. National patient safety goal for suicide prevention. R3 Report: Requirement, Rationale, Reference. R2 Report. 2019. [Google Scholar]
- 22.Patel A, Watts C, Shiddell S, Couch K, Smith AM, Moran MJ, Conners GP. Universal adolescent suicide screening in a pediatric urgent care center. Archives of suicide research. 2018. Jan 2;22(1):118–27. [DOI] [PubMed] [Google Scholar]
- 23.Roaten K, Horowitz LM, Bridge JA, Goans CR, McKintosh C, Genzel R, Johnson C, North CS. Universal pediatric suicide risk screening in a health care system: 90,000 patient encounters. Journal of the Academy of Consultation-Liaison Psychiatry. 2021. Mar 9. [DOI] [PubMed] [Google Scholar]
- 24.Brahmbhatt K, Kurtz BP, Afzal KI, Giles LL, Kowal ED, Johnson KP, Lanzillo E, Pao M, Plioplys S, Horowitz LM, Workgroup P. Suicide risk screening in pediatric hospitals: clinical pathways to address a global health crisis. Psychosomatics. 2019. Jan 1;60(1):1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Trebble TM, Hansi N, Hydes T, Smith MA, Baker M. Process mapping the patient journey: an introduction. Bmj. 2010. Aug 13;341. [DOI] [PubMed] [Google Scholar]
- 26.U.S. Census Bureau QuickFacts [Internet]. 2021. [cited 2021 July 31]. Available from: https://www.census.gov/quickfacts/fact/table/sandiegocountycalifornia,CA/PST045219.
- 27.Kidsdata.org Demographics of California’s Children [Internet]. 2021. [cited 2021 July 31] Available from https://www.kidsdata.org/topic/33/child-population-race/.
- 28.Richardson LP, Rockhill C, Russo JE, Grossman DC, Richards J, McCarty C, McCauley E, Katon W. Evaluation of the PHQ-2 as a brief screen for detecting major depression among adolescents. Pediatrics. 2010. May 1;125(5):e1097–103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kroenke K, Spitzer RL, Williams JB. The PHQ- 9: validity of a brief depression severity measure. Journal of general internal medicine. 2001. Sep;16(9):606–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, Currier GW, Melvin GA, Greenhill L, Shen S, Mann JJ. The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. American journal of psychiatry. 2011. Dec; 168(12):1266–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.The Columbia Lighthouse Project [Internet]. The Columbia Lighthouse Project, [cited 2021Aug16]. Available from: https://cssrs.columbia.edu/ [Google Scholar]
- 32.Stanley B, Brown GK. Safety planning intervention: a brief intervention to mitigate suicide risk. Cognitive and behavioral practice. 2012. May 1;19(2):256–64. [Google Scholar]
- 33.Allgaier AK, Pietsch K, Frühe B, Sigl- Glöckner J, Schulte- Körne G. Screening for depression in adolescents: validity of the patient health questionnaire in pediatric care. Depression and anxiety. 2012. Oct;29(10):906–13. [DOI] [PubMed] [Google Scholar]
- 34.Simon GE, Coleman KJ, Rossom RC, Beck A, Oliver M, Johnson E, Whiteside U, Operskalski B, Penfold RB, Shortreed SM. Risk of suicide attempt and suicide death following completion of the Patient Health Questionnaire depression module in community practice. The Journal of clinical psychiatry. 2016. Feb 24;77(2):0-. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Rossom RC, Simon GE, Beck A, Ahmedani BK, Steinfeld B, Trangle M, Solberg L. Facilitating action for suicide prevention by learning health care systems. Psychiatric services. 2016. Aug 1;67(8):830–2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Horowitz LM, Mournet AM, Lanzillo E, He JP, Powell DS, Ross AM, Wharff EA, Bridge JA, Pao M. Screening pediatric medical patients for suicide risk: is depression screening enough? Journal of Adolescent Health. 2021. Jun 1;68(6):1183–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Brent DA, Greenhill LL, Compton S, Emslie G, Wells K, Walkup JT, Vitiello B, Bukstein O, Stanley B, Posner K, Kennard BD. The Treatment of Adolescent Suicide Attempters study (TASA): predictors of suicidal events in an open treatment trial. Journal of the American Academy of Child & Adolescent Psychiatry. 2009. Oct 1;48(10):987–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Conway PM, Erlangsen A, Teasdale TW, Jakobsen IS, Larsen KJ. Predictive validity of the Columbia-Suicide Severity Rating Scale for short-term suicidal behavior: a Danish study of adolescents at a high risk of suicide. Archives of suicide research. 2017. Jul 3;21(3):455–69. [DOI] [PubMed] [Google Scholar]
- 39.Gipson PY, Agarwala P, Opperman KJ, Horwitz A, King CA. Columbia-suicide severity rating scale: predictive validity with adolescent psychiatric emergency patients. Pediatric emergency care. 2015. Feb;31(2):88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Horwitz AG, Czyz EK, King CA. Predicting future suicide attempts among adolescent and emerging adult psychiatric emergency patients. Journal of Clinical Child & Adolescent Psychology. 2015. Sep 3;44(5):751–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Kilincaslan A, Gunes A, Eskin M, Madan A. Linguistic adaptation and psychometric properties of the Columbia-Suicide Severity Rating Scale among a heterogeneous sample of adolescents in Turkey. The International Journal of Psychiatry in Medicine. 2019. Mar;54(2):115–32. [DOI] [PubMed] [Google Scholar]
- 42.World Health Organization. The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. World Health Organization; 1992. [Google Scholar]
- 43.Aguinaldo LD, Goldstone A, Hasler BP, Brent DA, Coronado C, Jacobus J. Preliminary analysis of low-level alcohol use and suicidality with children in the adolescent brain and cognitive development (ABCD) baseline cohort. Psychiatry research. 2021. May 1;299:113825. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Gould MS, Fisher P, Parides M, Flory M, Shaffer D. Psychosocial risk factors of child and adolescent completed suicide. Archives of general psychiatry. 1996. Dec 1;53(12):1155–62. [DOI] [PubMed] [Google Scholar]
- 45.Bridge JA, Asti L, Horowitz LM, Greenhouse JB, Fontanella CA, Sheftall AH, Kelleher KJ, Campo JV. Suicide trends among elementary school-aged children in the United States from 1993 to 2012. JAMA pediatrics. 2015. Jul 1;169(7):673–7. [DOI] [PubMed] [Google Scholar]
- 46.Gould MS, Greenberg TE, Velting DM, Shaffer D. Youth suicide risk and preventive interventions: a review of the past 10 years. Journal of the American Academy of Child & Adolescent Psychiatry. 2003. Apr 1;42(4):386–405. [DOI] [PubMed] [Google Scholar]
- 47.Beautrais AL. Serious suicide attempts in young people: a case control study (Doctoral dissertation, University of Otago; ). [Google Scholar]
- 48.Team RC. R: A language and environment for statistical computing.
- 49.Hill RM, Castellanos D, Pettit JW. Suicide-related behaviors and anxiety in children and adolescents: a review. Clinical psychology review. 2011. Nov 1;31(7):1133–44. [DOI] [PubMed] [Google Scholar]
- 50.Johnson T, Fendrich M. Modeling sources of self-report bias in a survey of drug use epidemiology. Annals of epidemiology. 2005. May 1;15(5):381–9. [DOI] [PubMed] [Google Scholar]
- 51.Hunt M, Auriemma J, Cashaw AC. Self-report bias and underreporting of depression on the BDI-II. Journal of personality assessment. 2003. Feb 1;80(1):26–30. [DOI] [PubMed] [Google Scholar]
- 52.Stone R. Pregnant women and substance use: fear, stigma, and barriers to care. Health & Justice. 2015. Dec;3(1):1–5. [Google Scholar]