Abstract
Objective
Suicide is a leading cause of death for adolescents in the United States. Alternative settings to treat suicidal ideation (SI) are needed. Study primary objectives include evaluating the safety and effectiveness of telehealth crisis intervention services (CIS) and in-person outpatient crisis intervention clinics (OCIC) relative to the current standard of care of inpatient psychiatric hospitalization. A secondary aim seeks to assess changes in suicidal ideation, and patient and parent treatment satisfaction.
Method
The study team, consisting of study staff at 4 sites, conducted an observational longitudinal study of patients (12-18 years of age) who were seen in the Emergency Department (ED) for suicidality and were referred to inpatient treatment, in-person OCIC, or telehealth CIS. Primary outcome data, including recurrent ED visits and hospitalizations because of SI, suicide attempts, and life satisfaction, were collected for 24 weeks. All analyses were adjusted for age, sex, and baseline suicidality severity scores. A total of 249 patients were enrolled.
Results
There were no statistically significant differences in suicide attempts, time to first suicide attempt, ED visits, hospitalizations, and life satisfaction among the 3 treatment arms. There was no statistically significant difference in outcomes for treatment satisfaction among the treatment groups.
Conclusion
In this observational study, in-person OCIC and telehealth CIS did not have significantly different outcomes from the current standard of care of inpatient psychiatric hospitalization. This broadens the scope of services that appear to be safe and effective for adolescents experiencing moderate suicidal thoughts. Future research using randomized controlled trials to clarify the causal effect of different interventions is warranted.
Clinical trial registration information
Observational Study to Compare Outcomes of Different Psychiatric Treatment of Suicidal Adolescents (Pre-START); https://clinicaltrials.gov/study/NCT04625686.
Diversity Inclusion Statement
We worked to ensure that the study questionnaires were prepared in an inclusive way. The author list of this paper includes contributors from the location and/or community where the research was conducted who participated in the data collection, design, analysis, and/or interpretation of the work.
Key words: suicide, adolescent, treatment
Plain language summary
The PreSTART study sought to explore treatment outcomes for adolescents experiencing suicidal thoughts. Inpatient treatment, outpatient crisis intervention centers, and telehealth crisis interventions were explored and compared to evaluate safety and effectiveness over a 24-week period. Results found no statistical difference in recurrent suicidal events or treatment satisfaction among the different treatments.
Graphical abstract
Suicide is the second leading cause of death for adolescents in the United States.1, 2, 3, 4, 5 Mental health needs have increased in recent years due to the COVID-19 pandemic. According to the Centers for Disease Control and Prevention (CDC), during 2020, the proportion of mental health–related ED visits among young people 12 to 17 years of age increased 31% compared with that in 2019.2 Inpatient psychiatric hospitalization, a frequent treatment setting for adolescents with suicidality,6, 7, 8, 9 is problematic because it can lead to a financial and logistical burden on families,10, 11, 12, 13 disruption in school,14 financial cost to the health care system,15,16 a substantial increase in the risk of suicide attempts after short psychiatric hospitalizations,17, 18, 19, 20, 21, 22, 23 assault by other patients,24, 25, 26 and poor self-esteem from the stigma associated with mental health challenges and needed treatment.27, 28, 29 Alternatives to inpatient psychiatric treatment need to be explored because of bed shortages,8,30 access issues,31, 32, 33, 34, 35 and overflow boarding of adolescent psychiatric patients in medical units and emergency departments (EDs).36, 37, 38, 39
Outpatient crisis intervention clinic (OCIC) is an alternative treatment option to provide intensive short-term (1-6 weeks) support with multiple therapeutic visits for patients and families each week immediately after the crises (ED visit). OCIC provides these services for adolescents and their families who are waiting for long-term outpatient services without exposing adolescents to the above-mentioned risks of inpatient psychiatric hospitalization.40, 41, 42, 43 Another treatment option that has seen an increase in use is telehealth crisis intervention services (CIS). In 2023, there was an unprecedented use of telehealth across health care systems in the United States, yet there is a deficiency of research indicating whether telehealth for adolescents with suicidal thoughts is both safe and effective. This information is paramount, given the current trends in the use of telehealth and its potential for further implementation for mental health services.
The Pandemic Restart of Suicide Treatment Alternatives for Teens (PreSTART) study (restarted during COVID restrictions) aimed to explore the safety and effectiveness of in-person OCIC and telehealth CIS in comparison to the current standard of care for adolescents with suicidal thoughts, namely, inpatient psychiatric hospitalization. The goal of this pragmatic study was to identify treatment settings and approaches that led to a lower risk of a suicidal event (primary outcome) as well as higher treatment satisfaction and life satisfaction (secondary outcomes) for both the legal guardians/parents and the patients.11,44,45
Method
Study Design
The research team conducted an observational longitudinal study between October 2020 and June 2021 evaluating the outcomes of patients referred to inpatient psychiatric treatment, in-person OCIC, or telehealth CIS. The study was reviewed and approved by the Cincinnati Children’s Hospital Institutional Review Board. Participants younger than 18 years of age provided written informed assent and had their legal guardian present at the ED to provide written informed consent. Participants 18 years old provided written informed consent. Non-study clinicians evaluated and referred the patients who presented to the ED for suicidal ideation. Participants were enrolled in-person during the ED visit or remotely within 72 hours of discharge. Baseline suicidality severity scores were collected via the Concise Health Risk Tracking Self-Report Tool (CHRT-SR). We collected outcomes every 2 weeks from participants and guardians via remote surveys. Surveys collected information on severity of suicidality, severity of recurrent suicidal events, follow-up care, life satisfaction, and treatment satisfaction. Study staff additionally completed weekly medical chart reviews to record recurrent suicide-related events. Follow-up care occasionally occurred outside of the enrolling institution’s health system. To address this situation, guardians were questioned via follow-up surveys to identify follow-up care. Study staff stayed connected with families throughout study enrollment and followed up via phone, email, or text messaging to increase study retention and data collection.
Participants
Study staff recruited a sample of 249 youth and their legal guardians across 4 research sites (Figure 1). All youth presented to 1 of the 4 participating EDs (2 in Ohio, 1 in Texas, and 1 in New York) with suicidal thoughts and were discharged to 1 of the 3 higher level of care treatment arms after an evaluation by an ED clinician. The ED clinician was a psychiatrist, psychologist, or clinical social worker. Youth were 12 to 18 years of age. All patients and guardians/parents were able to speak English with sufficient fluency to complete study surveys. Adolescents who presented with clinically determined imminent risk of suicide or who had made a suicide attempt within the last 24 hours were excluded. Adolescents who had prior in-person OCIC treatment in the past 12 months were excluded. Participants were included in the study only if they were sent to 1 of the 3 treatments from the ED. Disposition was decided by the ED clinician after evaluation with the patient and family to determine treatment needs. The telehealth CIS group could include treatment from a pre-existing mental health provider if the appointment was scheduled directly after and in relation to the baseline emergency department visit. Participants were enrolled during their ED visit or remotely within 48 hours of their ED visit.
Figure 1.
Participant Recruitment Diagram
Note:CCHMC = Cincinnati Children’s Hospital; CIS =crisisinterventionservice; NCH = Nationwide Children’s Hospital; OCIC =outpatient crisis intervention clinic.
Interventions
Inpatient Hospitalization
Inpatient psychiatric hospitalization treatment included supportive individual therapy, improving patient coping skills, family meetings about safety planning, and medication management. Inpatient services could include psychiatric evaluation, laboratory orders, assessments by nursing and other disciplines, milieu group and individual therapy, and after-care planning. Specific services provided to each patient were not collected by the study team.
In-Person Outpatient Crisis Intervention Clinic
In-person OCIC services included crisis psychotherapy/intervention, evidence-based screening/risk assessments, coordination of care/referral to an ongoing provider, short-term therapy/crisis treatment, safety planning, and/or referral and linkage to follow-up care. Treatment in OCIC could include group therapy, individual therapy, and/or family therapy, depending on the site. Not all sites provided group therapy services. Because of the pragmatic approach to the research design, interventions provided were site specific, but all sites included the crisis intervention component, that is, crisis stabilization (therapeutic intervention) as well as safety planning and assessment. In addition, patients were scheduled within 7 business days of the ED visit for follow-up care for in-person OCIC services.
Telehealth Crisis Intervention Services
Telehealth CIS was defined as psychiatric services provided via telehealth services that were scheduled within 7 business days of the crisis assessment. Interventions could be delivered by a social worker, therapist, or psychiatrist. Services included individual and family therapy, referrals for medication management, crisis stabilization/safety assessments, and safety planning. Services were not in-person and occurred via a computer video/phone system, using institutional conferencing software and technical support.
All treatment groups provided referrals for ongoing outpatient care as part of the treatment plan. Referrals were made; however, it was up to the families to follow through with treatment. Duration of hospitalization was not collected.
Outcomes
The primary outcome measure was suicidality defined by the number of suicidal event recurrences (ie, return to ED for SI, hospitalization for SI, or recurrent suicide attempt). The secondary outcome measures included changes in severity of suicidal ideation, life satisfaction, and treatment satisfaction.
Surveys were completed with both the legal guardian and patient at all intervals. Time to first recurrence of a suicidal event and number of suicidal events were measured using a Suicidal Event Form, to characterize these events at baseline and every 2 weeks. The Suicide Event Form was created by study staff and requested information on the number of suicidal events and the event dates, and allowed the study participant to share specific information on events in narrative responses.
Severity of suicidal ideation was measured with the Concise Health Rating Tool–Self Report scale (CHRT-SR) at baseline and every 2 weeks. The CHRT-SR is a 14-item validated scale for measuring suicide propensity and suicidal thoughts. The scoring on the CHRT-SR ranges from 0 to 56. Based on prior studies, the tool was found to have an 80% sensitivity and a 50% specificity in suicidal adolescents, with scores in the 30s indicating “moderate” suicidality.46
The Suicidal Treatment AlteRnatives for Teens (START)–Clinical Features (CF) form, a short supplemental form with 13 questions (developed by our team), was completed at baseline and every 2 weeks to monitor suicide-related events and substance use based on information from the patient, legal guardian, and/or electronic health record. The patient’s medical record was reviewed by study staff.
Life satisfaction was measured with validated self-report scales including the Patient-Reported Outcomes Measurement Information System (PROMIS) Short Form v 1.0, General Life Satisfaction–Short Form 5a (to assess the legal guardian’s well-being), PROMIS Parent-Proxy Life Satisfaction–Short Form 8a, and PROMIS Pediatric Life Satisfaction–Short Form 4a every 2 weeks.47
Treatment satisfaction data about telehealth CIS, in-person OCIC, and inpatient psychiatric hospitalization, completed by legal guardians and child participants, was collected only at the 2-week survey milestone. Treatment satisfaction was measured once with the Client Satisfaction Questionnaire (CSQ-8) after completion of telehealth CIS, OCIC, or inpatient treatment, which was typically completed by week 3 (but could be completed by week 2 to week 6).
Statistical Methods
Descriptive Analysis of Baseline Features
We used descriptive statistics for continuous variables using means with standard deviations and frequency counts with percentages for categorical variables. One-way analysis of variance (ANOVA) with the Tukey post hoc and Pearson χ2 tests were conducted to test for significant between-group differences in means for continuous variables and proportions for categorical measures, respectively, among the 3 treatment arms, corresponding to each variable at baseline.
Comparison of Primary/Secondary Outcomes Among Treatment Arms
For the primary outcome, we used the Cox proportional hazard model to compare the effects of the 3 treatments. To calculate the hazard (ie, number of incidents in each specified period of time of observation), we used the first recurrent suicidal event following enrollment as the “failure” event. We studied 3 types of recurrent suicidal events: a self-reported suicide attempt, a hospitalization secondary to suicidal ideation or an attempt, or an ED visit secondary to suicidal ideation or an attempt. We assessed hazard ratios between the 2 treatment arms (ie, in-person OCIC vs inpatient treatment, and telehealth CIS vs inpatient treatment) while adjusting for age, sex, and baseline CHRT-SR score. We used the threshold at the 2-sided p value of .004, which is equivalent to 0.05 divided by 12 (to correct to multiple tests due to 4 outcomes and 3 timepoints) to determine whether the treatment effect was statistically and significantly different between the 2 treatment programs.
For secondary outcomes such as the CHRT-SR score, life satisfaction, and treatment satisfaction, we used the general linear model repeated measures (GLM-RM) to determine the effect of treatment on the CHRT-SR scores. Poisson regression models were used to analyze the impact of the treatment on the number of suicide attempts, ED visits, and hospitalizations. Bonferroni adjustment was applied to all post hoc analyses to correct for multiple comparisons. The GLM-RM included time (baseline/week 2 to week 4; baseline/week 2 to week 12; baseline/week 2 to week 24) as a repeated measure, and treatment arm (inpatient psychiatric treatment, in-person OCIC, telehealth CIS) and time-by–treatment arm as fixed effects while adjusting for baseline CHRT-SR scores. During the GLM-RM procedure, we tested the homogeneous distribution of the variances with the Mauchly test of sphericity.48
As an additional approach, we also used the “difference in differences” (DID) method to evaluate the treatment effect on outcomes that are continuous variables such as CHRT-SR scores before and after treatment. We regarded the inpatient treatment arm as the reference group and compared it with in-person OCIC and telehealth CIS groups, separately.
Heterogeneous Treatment Effects
To further explore whether the treatment effect was modified by socioeconomic status or gender identity, we implemented a 2-step approach: (1) selecting the non-treatment predictors for the treatment outcome; and (2) testing the interaction between the selected non-treatment predictor in the multivariable model.
Outcomes were collected every 2 weeks for 6 months. Primary outcomes focused on weeks 4, 12, and 24. Secondary outcomes focused on changes from baseline and weeks 4, 12, and 24.
Results
Table 1 shows the breakdown of patients by subgroups including inpatient, OCIC (likely referring to an Outpatient Clinic Intensive Care program), and telehealth groups. There were no statistically significant differences in age (mean, 15.0 years), or sex assigned at birth (75.5% female) across the subgroups. Race was also fairly evenly distributed, with White (66.3%), African American (13.3%), Others (12.4%), and Mixed (8.0%) represented. In terms of household characteristics, close to half of the patients lived with both parents (42.2%) and annual household income was spread fairly evenly across the categories provided. Private insurance was the most common form of child insurance (62.7%), followed by public insurance (36.1%), with very few participants having no insurance (1.2%). Baseline CHRT-SR scores (measuring suicidality risk) were statistically significantly lower in the OCIC group (32.1) compared to the other groups (approximately 34.6). There were no statistically significant differences in treatment satisfaction scores between the groups.
Table 1.
Demographic and Clinical Characteristics of Patients
| Characteristic | Overall (n = 249) | Inpatient (n = 87) | OCIC group (n = 81) | Telehealth group (n = 81) | p |
|---|---|---|---|---|---|
| Age, y, mean (SD) | 15.0 (1.6) | 15.3 (1.5) | 14.9 (1.6) | 14.9 (1.7) | .191 |
| Sex assigned at birth, n (%) | |||||
| Male | 61 (24.5) | 20 (23.0) | 22 (27.2) | 19 (23.5) | .793 |
| Female | 188 (75.5) | 67 (77.0) | 59 (72.8) | 62 (76.5) | |
| Gender identity, n (%)∗ | |||||
| Male | 58 (23.3) | 20 (23.0) | 20 (24.7) | 20 (22.2) | .607 |
| Female | 157 (63.1) | 60 (69.0) | 48 (59.3) | 49 (60.5) | |
| Transgender or non-binary | 28 (11.2) | 5 (5.7) | 11 (13.6) | 12 (14.8) | |
| Missing/undisclosed | 6 (2.4) | 2 (2.3) | 2 (2.5) | 2 (2.5) | |
| Race, n (%) | |||||
| White | 165 (66.3) | 57 (65.5) | 53 (65.4) | 55 (67.9) | .457 |
| African American | 33 (13.3) | 10 (11.5) | 11 (13.6) | 12 (14.8) | |
| Others | 31 (12.4) | 14 (16.1) | 12 (14.8) | 5 (6.2) | |
| Mixed | 20 (8.0) | 6 (6.9) | 5 (6.2) | 9 (11.1) | |
| Guardians in household, n (%) | |||||
| Both parents | 105 (42.2) | 41 (47.1) | 30 (37.0) | 34 (42.0) | .483 |
| Single parent | 118 (47.4) | 35 (40.2) | 44 (54.3) | 39 (48.1) | |
| Other | 26 (10.4) | 11 (12.6) | 7 (8.6) | 8 (9.9) | |
| Missing | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) | |
| Annual household income, n (%) | |||||
| <$20,000 | 22 (8.8) | 7 (8.0) | 9 (11.1) | 6 (7.4) | .114 |
| $20,001-$40,000 | 38 (15.3) | 10 (11.5) | 11 (13.6) | 17 (21.0) | |
| $40,001-$60,000 | 30 (12.0) | 7 (8.0) | 13 (16.0) | 10 (12.3) | |
| $60,001-$90,000 | 54 (21.7) | 18 (20.7) | 23 (28.4) | 13 (16.0) | |
| >$90,000 | 105 (42.2) | 45 (51.7) | 25 (30.9) | 35 (43.2) | |
| Child’s insurance status, n (%) | |||||
| Private insurance | 156 (62.7) | 51 (58.6) | 49 (60.5) | 56 (69.1) | .416 |
| Public insurance | 90 (36.1) | 35 (40.2) | 30 (37.0) | 25 (30.9) | |
| No insurance | 3 (1.2) | 1 (1.1) | 2 (2.5) | 0 (0.0) | |
| Baseline CHRT-SR score, mean (SD) | 34.6 (10.0) | 38.0 (10.4) | 32.1 (10.3) | 33.5 (8.2) | <.001 |
| Participant CSQ Treatment Satisfaction scores, mean (SD) | 21.9 (6.3) | 23.1 (6.5) | 21.8 (5.9) | 20.9 (6.3) | .126 |
| Legal Guardian CSQ Treatment Satisfaction scores, mean (SD) | 23.7 (6.0) | 24.6 (6.3) | 23.0 (5.9) | 23.5 (5.7) | .283 |
Note: p Values address the probability that means or proportions differ across arms. p Values in boldface type indicate statistically significant results. CCHMC = Cincinnati Children’s Hospital and Medical Center; CHRT-SR = Concise Health Rating Tool Self-Report; CSQ = Client Satisfaction Questionnaire; OCIC = outpatient crisis intervention clinic; NCH = Nationwide Children’s Hospital.
There were no statistically significant differences in primary outcomes, recurrent ED visits, hospitalizations, or suicide attempts among the inpatient, OCIC, and telehealth groups, as none of the 95% confidence intervals overlapped with zero (Tables S6-S8, available online). Table 2, Table 3 show that the change of counts of the suicidal events and hospitalizations did not vary by treatment group after adjusting for age, sex at birth, baseline CHRT-SR scores and site. Outcomes were adjusted for age, sex at birth, baseline CHRT-SR (suicidality severity), and site. The regression coefficient (OCIC vs inpatient) was estimated to be 0.45 within 12 weeks and 0.3 within 24 weeks. Therefore, the power estimates based on the survival analysis assuming an exponential distribution were 0.87 and 0.53 for the 12-week follow-up period and the 24-week follow-up period, respectively.
Table 2.
Comparative Treatment Effects on the Number of Suicide Attempts From Poisson Regression
| Week | Coefficient | Standard error | z | p | 95% CI |
|---|---|---|---|---|---|
| 0-4 | 0.17a | 0.10 | 1.71 | .088 | –0.02 to 0.36 |
| 0.45b | 0.29 | 1.53 | .126 | –0.13 to 1.02 | |
| 0-12 | 0.12a | 0.28 | 0.43 | .670 | –0.42 to 0.66 |
| 0.01b | 0.24 | 0.02 | .983 | –0.47 to 0.48 | |
| 0-24 | –0.02a | 0.30 | –0.07 | .940 | –0.62 to 0.57 |
| –0.46b | 0.29 | –1.59 | .113 | –1.02 to 0.11 |
Note: The coefficient indicates the difference in the logs of expected counts (ie, number of events). The covariates that were adjusted include age, sex assigned at birth, baseline Concise Health Rating Tool Self-Report (CHRT-SR) score, and site.
In-person OCIC vs inpatient.
Telehealth crisis intervention services vs inpatient.
Table 3.
Comparative Treatment Effects on the Frequency of Subsequent Hospitalization Using Poisson Regression
| Week | Coefficient | Standard error | z | p | 95% CI |
|---|---|---|---|---|---|
| 0-4 | 0.04a | 0.13 | 0.31 | .760 | –0.22 to 0.30 |
| 0.13b | 0.13 | 1.04 | .296 | –0.12 to 0.38 | |
| 0-12 | 0.34a | 0.22 | 1.51 | .132 | –0.10 to 0.78 |
| 0.001b | 0.21 | 0.00 | .997 | –0.40 to 0.40 | |
| 0-24 | 0.03a | 0.26 | 0.11 | .911 | –0.48 to 0.54 |
| –0.10b | 0.21 | –0.48 | .634 | –0.52 to 0.31 |
Note: The coefficient indicates the difference in the logs of expected counts (ie, number of events). The covariates that were adjusted include age, sex assigned at birth, baseline Concise Health Rating Tool Self-Report (CHRT-SR) score, and site.
In-person outpatient crisis intervention clinics (OCIC) group compared with inpatient group.
Telehealth crisis intervention services group compared with inpatient group.
Table 4 illustrates the results of comparing the numbers of ED visits across the 3 treatment settings. Among all comparisons, we found a greater number of visits to ED among the telehealth group as compared to inpatient group at weeks 0 to 4 (Poisson regression coefficient: 0.45, adjusted p = .005), the difference remained statistically significant after multi-testing correction. Note that no statistical difference in the number of any suicidal events was found between the OCIC and telehealth CIS groups.
Table 4.
Comparative Treatment Effects on the Frequency of Emergency Department Visits From Zero-Inflated Poisson Regression Analysis Results
| Week | Coefficient | Standard error | z | Unadjusted p | 95% CI |
|---|---|---|---|---|---|
| 0-4 | 0.24a | 0.14 | 1.75 | .079 | –0.03 to 0.52 |
| 0.45b | 0.16 | 2.80 | .005 | 0.13 to 0.76 | |
| 0-12 | 0.05a | 0.19 | 0.25 | .804 | –0.33 to 0.43 |
| 0.18b | 0.15 | 1.20 | .231 | –0.11 to 0.46 | |
| 0-24 | –0.08a | 0.26 | –0.30 | .768 | –0.58 to 0.43 |
| 0.03b | 0.18 | 0.14 | .885 | –0.32 to 0.37 |
Note: The coefficient indicates the difference in the logs of expected counts (ie, number of events). The covariates that were adjusted include age, sex assigned at birth, baseline Concise Health Rating Tool Self-Report (CHRT-SR) score, and site. An unadjusted p value <.004 was used as the significance threshold to correct for multiple tests.
In-person outpatient crisis intervention clinics (OCIC) group compared with the inpatient group.
Telehealth crisis intervention services group compared with the inpatient group.
The DID analysis results indicate that there were no statistically significant interactions at any time periods between time and treatment arms for the secondary outcomes such as CHRT-SR scores, general life satisfaction, pediatric item life satisfaction, legal guardian item life satisfaction scores, and treatment satisfaction (Tables S1-S5, available online). In addition, no significant effects of target individual attributes on the primary treatment outcomes were detected. Furthermore, no statistical difference was found between the OCIC and telehealth CIS groups.
Discussion
The results of this study provide initial evidence that may have important implications for the treatment of adolescents experiencing suicidal crises. Historically, inpatient treatment has been considered the standard of care for adolescents experiencing suicidal thoughts. Despite this fact, many ED and outpatient mental health providers are faced with the reality that inpatient treatment is not always available or might have unanticipated adverse effects. In addition, there is the assumption that patients assigned to inpatient care are more ill (ie, have higher levels of suicidal ideation) than patients assigned to receive outpatient referrals. To take the confounding effect of baseline suicidal ideation, we adjusted for it in every statistical model, and further used the DID method to compare before-and-after differences regardless of the baseline difference. Therefore, our findings are unlikely to be biased because of differences in baseline severity of suicidality.
The results of PreSTART show that in-person OCIC and telehealth CIS services have no statistically significant differences in outcomes for suicidal events. The findings suggest that none of the treatment settings yielded significantly better outcomes over the 6-month follow-up period. The primary outcome for this study was the recurrence of suicidal events, identified as ED visits, hospitalizations, and suicide attempts. We found that in-person OCIC and telehealth CIS services trended toward a decrease in suicidal events at months 1 and 3. At month 6, there were no statistically significant group differences in outcomes. Providing this information to mental health professionals, including psychiatrists, psychologists, and social workers, may improve quality of care by broadening the evidence base to support treatment options that are more available than inpatient psychiatric care in many parts of the country. Inpatient treatment has historically been known as the “safest option” by providers. However, our findings suggest that there are safe and effective alternatives to inpatient hospitalization for adolescents with suicidal ideation.
The secondary aims for this study were to explore suicidality severity scores (ie, CHRT-SR score) and the life satisfaction and treatment satisfaction among the treatment groups. Results showed that patients have no statistically significant differences in CHRT-SR scores (Cohen d = 0.08) and life satisfaction and treatment satisfaction at month 6 (Cohen d = 0.22 and 0.03, respectively). We found that life satisfaction scores did not differ significantly across the 3 treatments groups, although all groups showed improvement in scores over time. This finding could indicate the general nature of suicidal youth and perspectives on life unrelated to the treatment that they receive. In terms of treatment satisfaction, we did not find a significant difference among the 3 treatment groups when factoring in suicidality severity. It is important to note that treatment satisfaction data were collected only once, at week 2. Long-term impressions of treatment may change over time.
Although the sample size was moderate, the multi-site enrollment allowed for broad geographic and demographic representations. The inclusion and exclusion criteria were few, thus maximizing enrollment and generalizability. The focus of enrollment was to capture participants who were experiencing a mental health crisis due to suicidality. Enrollment and treatment occurred in large urban areas; however, individual participants lived in urban and rural areas.
Because of the COVID-19 pandemic and restriction of research personnel in the ED, PreSTART had an observational rather than randomized design, as randomization would have required in-person recruitment during the mental health assessment. PreSTART was conducted remotely, after the ED visit. Therefore, sample bias could exist in the participants whom the study team were able to contact for study inclusion. Since this study was observational, unmeasured confounding variables could exist. Adolescents who presented in the ED and were determined to be clinically at imminent risk for suicide or who made a suicide attempt were excluded; therefore, clinical judgment was involved in the selection of study participant inclusion. Despite the efforts to adjust for baseline differences in the severity of suicide risk, selection biases (such as clinician and parental preference for certain dispositions, availability of inpatient beds at the time of the ED visit, and insurance limitations) might still limit the interpretation of the current findings. Therefore, this observational study cannot establish a causal association between the study outcomes and treatment conditions. Another limitation is that the study did not collect length-of-treatment data for inpatient or OCIC care and was not able to collect information from outside health systems; therefore, the time to first suicidal event could have been shorter for inpatients. The study team plans to collect and analyze this information in future work exploring this subject. Potential bias may be found in the number of participants lost to follow-up in the inpatient group, highlighting possible systematic differences from those participants retained in the study. In addition, treatment sites within a condition might have driven the outcomes for that condition; however, the sample size precluded analyses examining this possibility. Overall, the study findings provide initial evidence to support the use of alternative treatment settings for suicidal youth, while also underscoring the necessity of scrutinizing and validating the results in a larger-scale, randomized controlled study with a greater statistical power that could reduce biases such as those arising in an observational study.
CRediT authorship contribution statement
Jennifer Combs: Writing – original draft, Project administration, Data curation, Investigation, Writing – review & editing. Ping-I Lin: Writing – review & editing, Formal analysis, Data curation. Melissa P. DelBello: Writing – review & editing, Conceptualization. Adam C. Carle: Writing – review & editing, Methodology. Jeffrey A. Bridge: Supervision, Resources, Project administration. David A. Axelson: Supervision, Resources, Project administration. Victor Fornari: Supervision, Resources, Project administration. Vera Feuer: Supervision, Resources, Project administration. Graham J. Emslie: Supervision, Resources, Project administration. Betsy D. Kennard: Supervision, Resources, Project administration. Stephen C. Porter: Supervision, Resources. Michael T. Sorter: Resources. Drew Barzman: Writing – review & editing, Supervision, Project administration, Methodology, Investigation, Conceptualization.
Footnotes
This article was reviewed under and accepted by Robert L. Findling, MD, MBA.
This project was funded by the Patient-Centered Outcomes Research Institute (PCORI) PCS-2018C1-11111.
This article is part of a special series devoted to the subject of suicide in children and adolescents, with a focus on the need for improvement to current approaches to prediction, prevention, and treatment. This special series is edited by Guest Editor Lynsay Ayer, PhD, Deputy Editor Daniel P. Dickstein, MD, and Editor Manpreet K. Singh, MD, MS.
The research was performed with permission from the Institutional Review Boards of Cincinnati Children's, Nationwide Children's, and UT Southwestern.
Consent has been provided for descriptions of specific patient information.
This study was presented as a poster at the American Academy of Child and Adolescent Psychiatry Annual Meeting; October 17-22, 2022; Toronto, Ontario, Canada, and October 23-28, 2023; New York, New York.
Data Sharing: Deidentified participant data and Data dictionary available with publication at Figshare, an open-access data repository; Figshare.com of the study results. The data will be made available to researchers with proper approval for any purpose without investigator support.
Ping-I Lin served as the statistical expert for this research.
Dr. Michael T. Sorter passed away in February 2025. We are grateful for his mentorship and visionary leadership in advancing pediatric mental health in Cincinnati and beyond. It was a privilege to learn from and work alongside him.
The authors thank the youth and families who participated in the study.
Disclosure: Ping-I Lin has received research grant support from Rural Health Research Funding, Australian Rotary Health Grant, Neuroscience Mental Health and Addiction Theme and Clinical Academic Group (CAG), and Frontier Health and Medical Research Initiative. Melissa P. DelBello has received research support from the National Institute of Health, PCORI, AbbVie, Alkermes, Eli Lilly, Intracellular, Janssen, Johnson and Johnson, Lundbeck, Myriad, Novartis, Otsuka, Pfizer, Sage, Shire, Sunovion, Supernus, and Vanda and has provided consultation or advisory board services for Alkermes, Allergan, Janssen, Johnson and Johnson, Lundbeck, Medscape, Myriad, Neuronetics, Pfizer, Sunovion, and Sage. Adam C. Carle has received grant support from the National Institute of Health, the Food and Drug Administration, the American Board of Family Medicine, the Agency of Healthcare Research and Quality, and the Cure START Now Foundation. Jeffrey A. Bridge has received research grant support from the National Institute of Mental Health and the Centers for Disease Control and Prevention; he is also a member of the Scientific Advisory Board of Clarigent Health. David A. Axelson has received royalties from Wolters/Kluwer and research grant support from the National Institute of Mental Health. Graham J. Emslie has received research support from the American Foundation for Suicide Prevention (AFSP), Janssen Research Development, the National Institutes of Health, and the State of Texas. Betsy D. Kennard has received research support from the American Foundation for Suicide Prevention, the National Institutes of Health, and the State of Texas. Betsy D. Kennard has received royalties from Guilford Press and is on the board of the Jerry M. Lewis, MD, Research Foundation and the George G. and Alva Hudson Smith Foundation. Drew Barzman has received support from the National Institutes of Health. Jennifer Combs, Victor Fornari, Vera Feuer, Stephen C. Porter, and Michael T. Sorter have reported no biomedical financial interests or potential conflicts of interest.
Supplemental Material
References
- 1.Curtin S.C., Warner M., Hedegaard H. Increase in suicide in the United States, 1999-2014. NCHS Data Brief. 2016;241:1–8. [PubMed] [Google Scholar]
- 2.Leeb R.T., Bitsko R.H., Radhakrishnan L., Martinez P., Njai R., Holland K.M. Mental health–related emergency department visits among children aged <18 years during the COVID-19 pandemic — United States, January 1–October 17, 2020. MMWR Morb Mortal Wkly Rep. 2020;69:1675–1680. doi: 10.15585/mmwr.mm6945a3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Simon T. Suicide rates for teens aged 15–19 years, by sex—United States, 1975–2015. CDC Weekly. 2017;66:816. doi: 10.15585/mmwr.mm6630a6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Centers for Disease Control and Prevention, National Center for Injury Prevention and Control . 2015. Web-Based Injury Statistics Query and Reporting System (WISQARS)https://www.cdc.gov/injury/wisqars.pdf/leadingcausesofdeathbyagegroup2015-a.pdf (Accessed 25 November 2024) [Google Scholar]
- 5.Centers for Disease Control and Prevention, National Center for Injury Prevention and Control . 2016. Web-Based Injury Statistics Query and Reporting System (WISQARS)https://www.cdc.gov/injury/images/lc-charts/leading_causes_death_highlighting_violence_2016_1030w800h.gif (Accessed 25 November 2024) [Google Scholar]
- 6.Blanz B., Schmidt M.H. Practitioner review: preconditions and outcomes of inpatient treatment in child and adolescent psychiatry. J Child Psychol Psychiatry. 2000;41(6):703–712. [PubMed] [Google Scholar]
- 7.Green J., Jacobs B., Beecham J., et al. Inpatient treatment in child and adolescent psychiatry—a prospective study of health gain and cost. J Child Psychol Psychiatry. 2007;48(12):1259–1267. doi: 10.1111/j.1469-7610.2007.01802.x. [DOI] [PubMed] [Google Scholar]
- 8.Blader J.C. Acute inpatient care for psychiatric disorders in the United States, 1996 through 2007. Arch Gen Psychiatry. 2011;68(12):1276–1283. doi: 10.1001/archgenpsychiatry.2011.84. [DOI] [PubMed] [Google Scholar]
- 9.Gliatto M.F., Rai A.K. Evaluation and treatment of patients with suicidal ideation. Am Fam Physician. 1999;59(6):1500–1506. [PubMed] [Google Scholar]
- 10.Perlick D.A., Rosenheck R.A., Clarkin J.F., et al. Impact of family burden and affective response on clinical outcome among patients with bipolar disorder. Psychiatr Serv. 2004;55(9):1029–1035. doi: 10.1176/appi.ps.55.9.1029. [DOI] [PubMed] [Google Scholar]
- 11.Weller B.E., Faulkner M., Doyle O., Daniel S.S., Goldston D.B. Impact of patients’ psychiatric hospitalization on caregivers: a systematic review. Psychiatr Serv. 2015;66(5):527–535. doi: 10.1176/appi.ps.201400135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Busch S.H., Barry C.L. Mental health disorders in childhood: assessing the burden on families. Health Aff. 2007;26(4):1088–1095. doi: 10.1377/hlthaff.26.4.1088. [DOI] [PubMed] [Google Scholar]
- 13.Puotiniemi T., Kyngäs H., Nikkonen M. Factors associated with the coping of parents with a child in psychiatric inpatient care. Int J Nurs Pract. 2001;7(5):298–305. doi: 10.1046/j.1440-172x.2001.00272.x. [DOI] [PubMed] [Google Scholar]
- 14.Preyde M., Parekh S., Heintzman J. Youths’ experiences of school re-integration following psychiatric hospitalization. J Can Acad Child Adolesc Psychiatry. 2018;27(1):22. [PMC free article] [PubMed] [Google Scholar]
- 15.Bardach N.S., Coker T.R., Zima B.T., et al. Common and costly hospitalizations for pediatric mental health disorders. Pediatrics. 2014;133(4):602–609. doi: 10.1542/peds.2013-3165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Stensland M., Watson P.R., Grazier K.L. An examination of costs, charges, and payments for inpatient psychiatric treatment in community hospitals. Psychiatr Serv. 2012;63(7):666–671. doi: 10.1176/appi.ps.201100402. [DOI] [PubMed] [Google Scholar]
- 17.Brent D.A., McMakin D.L., Kennard B.D., Goldstein T.R., Mayes T.L., Douaihy A.B. Protecting adolescents from self-harm: a critical review of intervention studies. J Am Acad Child Adolesc Psychiatry. 2013;52(12):1260–1271. doi: 10.1016/j.jaac.2013.09.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hunt I.M., Kapur N., Webb R., et al. Suicide in recently discharged psychiatric patients: a case-control study. Psychol Med. 2009;39(3):443–449. doi: 10.1017/S0033291708003644. [DOI] [PubMed] [Google Scholar]
- 19.Knesper D.J., American Association of Suicidology, and Suicide Prevention Resource Center . Continuity of Care for Suicide Prevention and Research: Suicide Attempts and Suicide Deaths Subsequent to Discharge from the Emergency Department or Psychiatric Inpatient Unit. Education Development Center Inc; Newton, MA: 2010. [Google Scholar]
- 20.Groholt B., Ekeberg Ø., Haldorsen T. Adolescent suicide attempters: what predicts future suicidal acts? Suicide Life-Threat. Behav. 2006;36(6):638–650. doi: 10.1521/suli.2006.36.6.638. [DOI] [PubMed] [Google Scholar]
- 21.Goldston D.B., Daniel S.S., Erkanli A., et al. Psychiatric diagnoses as contemporaneous risk factors for suicide attempts among adolescents and young adults: developmental changes. J Consult Clin Psychol. 2009;77(2):281. doi: 10.1037/a0014732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Levine L.J., Schwarz D.F., Argon J., Mandell D.S., Feudtner C. Discharge disposition of adolescents admitted to medical hospitals after attempting suicide. Arch Pediatr Adolesc Med. 2005;159(9):860–866. doi: 10.1001/archpedi.159.9.860. [DOI] [PubMed] [Google Scholar]
- 23.Chung D.T., Ryan C.J., Hadzi-Pavlovic D., Singh S.P., Stanton C., Large M.M. Suicide rates after discharge from psychiatric facilities: a systematic review and meta-analysis. JAMA Psychiatry. 2017;74(7):694–702. doi: 10.1001/jamapsychiatry.2017.1044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Barton G., Rey J.M., Simpson P., Denshire E. Patterns of critical incidents and their effect on outcome in an adolescent inpatient service. Australian New Zealand Journal of Psychiatry. 2001;35(2):155–159. doi: 10.1046/j.1440-1614.2001.00872.x. [DOI] [PubMed] [Google Scholar]
- 25.Ryan E.P., Hart V.S., Messick D.L., Aaron J., Burnette M. A prospective study of assault against staff by youths in a state psychiatric hospital. Psychiatr Serv. 2004;55(6):665–670. doi: 10.1176/appi.ps.55.6.665. [DOI] [PubMed] [Google Scholar]
- 26.Sukhodolsky D.G., Cardona L., Martin A. Characterizing aggressive and noncompliant behaviors in a children’s psychiatric inpatient setting. Child Psychiatry Hum Dev. 2005;36(2):177–193. doi: 10.1007/s10578-005-3494-0. [DOI] [PubMed] [Google Scholar]
- 27.Choi C., Ferro M.A. Comparing self-concept among youth currently receiving inpatient versus outpatient mental health services. J Can Acad Child Adolesc Psychiatry. 2018;27(1):69. [PMC free article] [PubMed] [Google Scholar]
- 28.Talbot F., Harris G.E., French D.J. Treatment outcome in psychiatric inpatients: the discriminative value of self-esteem. Int J Psychiatry Med. 2009;39(3):227–241. doi: 10.2190/PM.39.3.b. [DOI] [PubMed] [Google Scholar]
- 29.Moses T. Being treated differently: stigma experiences with family, peers, and school staff among adolescents with mental health disorders. Soc Sci Med. 2010;70(7):985–993. doi: 10.1016/j.socscimed.2009.12.022. [DOI] [PubMed] [Google Scholar]
- 30.Geller J.L., Biebel K. The premature demise of public child and adolescent inpatient psychiatric beds. Psychiatr Q. 2006;77(3):251–271. doi: 10.1007/s11126-006-9012-0. [DOI] [PubMed] [Google Scholar]
- 31.National Research Council and Institute of Medicine . National Academies Press; 2009. Adolescent Health Services: Missing Opportunities (Committee on Adolescent Health Services and Models of Care for the Treatment, Prevention, and Healthy Development) [PubMed] [Google Scholar]
- 32.Kataoka S.H., Zhang L., Wells K.B. Unmet need for mental health care among US children: variation by ethnicity and insurance status. Am J Psychiatry. 2002;159(9):1548–1555. doi: 10.1176/appi.ajp.159.9.1548. [DOI] [PubMed] [Google Scholar]
- 33.Owens P.L., Hoagwood K., Horwitz S.M., et al. Barriers to children’s mental health services. J Am Acad Child Adolesc Psychiatry. 2002;41(6):731–738. doi: 10.1097/00004583-200206000-00013. [DOI] [PubMed] [Google Scholar]
- 34.Larson J., Stewart M., Kushner R., Frosch E., Solomon B. Barriers to mental health care for urban, lower income families referred from pediatric primary care. Adm Policy Ment Health Ment Health Serv Res. 2013;40(3):159–167. doi: 10.1007/s10488-011-0389-1. [DOI] [PubMed] [Google Scholar]
- 35.Semansky R.M., Koyanagi C. Child adolescent psychiatry: obtaining child mental health services through Medicaid: the experience of parents in two states. Psychiatr Serv. 2004;55(1):24–25. doi: 10.1176/appi.ps.55.1.24. [DOI] [PubMed] [Google Scholar]
- 36.Claudius I., Donofrio J.J., Lam C.N., Santillanes G. Impact of boarding pediatric psychiatric patients on a medical ward. Hosp Pediatr. 2014;4(3):125–132. doi: 10.1542/hpeds.2013-0079. [DOI] [PubMed] [Google Scholar]
- 37.Hazen E.P., Prager L.M. A quiet crisis: pediatric patients waiting for inpatient psychiatric care. J Am Acad Child Adolesc Psychiatry. 2017;56(8):631–633. doi: 10.1016/j.jaac.2017.03.022. [DOI] [PubMed] [Google Scholar]
- 38.Gallagher K.A., Bujoreanu I.S., Cheung P., et al. Psychiatric boarding in the pediatric inpatient medical setting: a retrospective analysis. Hosp Pediatr. 2017;7(8):444–450. doi: 10.1542/hpeds.2017-0005. [DOI] [PubMed] [Google Scholar]
- 39.Misek R.K., DeBarba A.E., Brill A. Predictors of psychiatric boarding in the emergency department. West J Emerg Med. 2015;16(1):71. doi: 10.5811/westjem.2014.10.23011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Blumberg S.H. Crisis intervention program: an alternative to inpatient psychiatric treatment for children. Ment Health Serv Res. 2002;4(1):1–6. doi: 10.1023/a:1014071507949. [DOI] [PubMed] [Google Scholar]
- 41.Greenfield B., Larson C., Hechtman L., Rousseau C., Platt R. A rapid-response outpatient model for reducing hospitalization rates among suicidal adolescents. Psychiatr Serv. 2002;53(12):1574–1579. doi: 10.1176/appi.ps.53.12.1574. [DOI] [PubMed] [Google Scholar]
- 42.Gutstein S.E., Rudd M.D. An outpatient treatment alternative for suicidal youth. J Adolesc. 1990;13(3):265–277. doi: 10.1016/0140-1971(90)90018-3. [DOI] [PubMed] [Google Scholar]
- 43.Beautrais A.L., Joyce P.R., Mulder R.T. Youth suicide attempts: a social and demographic profile. Aust N Z J Psychiatry. 1998;32(3):349–357. doi: 10.3109/00048679809065527. [DOI] [PubMed] [Google Scholar]
- 44.Mathur V.K., Freeman D.G. A theoretical model of adolescent suicide and some evidence from US data. Health Econ. 2002;11(8):695–708. doi: 10.1002/hec.704. [DOI] [PubMed] [Google Scholar]
- 45.Sunderji N., de Bibiana J.T., Stergiopoulos V. Urgent psychiatric services: a scoping review. Can J Psychiatry. 2015;60(9):393–402. doi: 10.1177/070674371506000904. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Mayes Taryn, et al. Predicting future suicidal events in adolescents using the Concise Health Risk Tracking Self-Report (CHRT-SR) J Psychiatr Res. 2020;126:19–25. doi: 10.1016/j.jpsychires.2020.04.008. [DOI] [PubMed] [Google Scholar]
- 47.Boß L., Lehr D., Reis D., et al. Reliability and validity of assessing user satisfaction with Web-based health interventions. J Med Internet Res. 2016;18(8) doi: 10.2196/jmir.5952. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Grieve A. Tests of sphericity of normal distributions and the analysis of repeated measures designs. Psychometrika. 1984;49(2):257–267. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.