Abstract
This pilot open trial examined the feasibility, acceptability, and preliminary outcomes of the Collaborative Assessment and Management of Suicidality for teens (CAMS-4Teens) who presented to outpatient care with suicidal thoughts and behaviors. Participants were 22 adolescents (13–17; 59% identified as female) with clinical elevations (≥7) on the Suicidal Behaviors Questionnaire-Revised (SBQ-R). Primary outcomes were feasibility and acceptability. We also explored outcomes of suicidal thoughts and behaviors, as well as mixed effects modeling for weekly assessments of the Suicide Status Form (SSF) Core Assessment constructs. Our main implementation outcomes suggest that the intervention is acceptable, appropriate, and feasible to deliver. Clinicians were adherent to the model with high ratings of adherence. In addition, preliminary evaluation of suicidal thoughts and behaviors found a large effect size for reduction in suicidal thoughts. Benchmarking to other adolescent suicide specific interventions and the Collaborative Assessment of Management of Suicidality with adult populations provide promise that suicidal adolescent responses may be on par with established interventions. Findings from the study are preliminary in nature and intended to inform if CAMS with adolescents is a promising approach to engage and treat patient-defined “drivers” of suicide. The results suggest that a future investigation with power to detect significant change over another active intervention is warranted.
Keywords: Suicide prevention, suicide-specific treatment, adolescent suicide prevention
Suicide is a significant and preventable public health problem that requires additional treatment strategies to reduce death rates. Despite efforts to address the devastating impact of death by suicide, rates increased from 1999 to 2017, for all age groups under 75 years old (Curtin et al., 2016; Hedegaard et al., 2018). Young tween and teen girls (aged 10–14) demonstrated the largest increase (200%) in death by suicide in the 15-year period observed (Curtin et al., 2016; Hedegaard et al., 2018). In addition to those who die by suicide, a significant portion of adolescents report suicidal thoughts and behaviors (STB). Youth Risk Behavior Survey data suggest that 17.7% of US high school aged adolescents seriously consider suicide, and nearly 9% will make an attempt in a given year (Lowry et al., 2014). Considering the full spectrum of STB, from suicidal ideation to suicide attempts and death by suicide, the scope of the problem during adolescence is significant. Because the onset of STB is typically during adolescence, suicide prevention policies and strategies need to focus on adolescent cohorts and investigation of promising suicide-specific interventions for adolescents is a national public health priority.
There is a clear need for developing evidence-based strategies in treating adolescent STB. The last two decades has seen tremendous growth in the evaluation of interventions strategies for adolescent STB, with a recent meta-analysis highlighting 25 randomized controlled trials (RCTs) targeting youth 12–19 (Kothgassner et al., 2020). Kothgassner’s meta-analysis suggested that treatment interventions demonstrated small effects above active control conditions. To date the only psychotherapy approach with replicated effects for reducing STB is full-model Dialectical Behavior Therapy adapted for adolescents (i.e., DBT-A with individual, group, phone coaching, and consultation) (Mehlum et al., 2014; Mehlum et al., 2016; McCauley et al., 2018). DBT-A demonstrated moderate effects on STB in the recent meta-analysis (Kothgassner et al., 2020). Despite the encouraging success of DBT with suicidal teens, it is nevertheless a labor-intensive and costly intervention to adherently deliver, contributing to access limitations. Although efforts have been made to improve access to DBT, the fact remains that many youths drop out of mental health treatments, and report ongoing self-injurious thoughts and behaviors even after an initial course of treatment. Additional options to sequence care and triage to the appropriate level of care would advance the ability to provide access and engagement (McCauley et al., 2018; Ougrin et al., 2015; Pistorello et al., 2020). Evidence in support of other promising treatment approaches including CBT, Mentalization, and family centered treatments like Attachment-Based Family Therapy (ABFT; Diamond et al., 2019) and SAFETY (Asarnow et al., 2017) is emerging. A meta-analysis, conducted by Ougrin and colleagues (2015), suggested that interventions with a strong family component and with multiple treatment sessions were associated with significant reduction of non-suicidal self-injury (NSSI) and suicide attempts, as opposed to studies with weak family involvement and/or single session interventions. However, this meta-analysis finding is complicated by the fact that a number of family therapy trials (e.g., Diamond et al., 2019; Cottrell et al., 2018) have not demonstrated reductions in key suicide risk-related outcomes compared to treatment as usual or nondirective supportive therapy. Support favoring interventions with a strong family involvement was found for suicidal ideation but not NSSI and suicidal behavior in the family-centered studies included in the Kothgassner and colleagues (2020) analysis. The studies reviewed also recruited potentially different populations with some studies recruiting youth with ideation only (Diamond et al, 2019), some either NSSI or suicide attempts (Asarnow et al., 2017), and others requiring both suicidal thoughts and behaviors (McCauley et al., 2018). These methodological differences plus the lack of clarity around the differential impact of suicidal thoughts and behaviors on suicide risk and treatment outcomes (Large et al., 2021) highlight the need for increased suicide-specific treatment research. Additional intervention options that demonstrate acceptability, feasibility, and readiness for scalability are clearly needed to create a range of effective clinical responses to save more lives and decrease suicide-related suffering. Accordingly, there is a pressing need to have scientific evaluation of common elements that characterize quality improvement for suicide-specific healthcare including brief interventions which provide the following: (a) direct prioritization of STB in therapy (independent of diagnosis), (b) continuous risk assessment and patient monitoring on suicide risk indicators, and (c) suicide-specific stabilization planning that help manage acute crises (Zima et al., 2013).
The Collaborative Assessment and Management of Suicide (CAMS) is an effective, well-established, suicide-focused intervention framework that has been demonstrated to reduce suicidal risk in five randomized controlled trials with adult clinical samples (Andreasson et al., 2016; Comtois et al., 2011; Jobes et al., 2017; Pistorello et al., 2020; Ryberg et al., 2019; Swift et al., n.d.). CAMS is further supported by ten published correlation/open clinical trials with adults (see Jobes et al. 2017 for a review and Swift et al., in press for a meta-analysis). CAMS-guided care helps clinicians to effectively engage, assess, and ultimately treat suicidal risk through the use of a multi-purpose clinical tool called the Suicide Status Form (SSF). CAMS is designed to enhance the therapeutic alliance and increase motivation in the patient in a joint effort to effectively engage the patient to collaboratively target and treat the problems that the patient articulates compel them to consider suicide (i.e., called suicidal “drivers” within CAMS). The structure of the CAMS framework has broad appeal and scalability as clinicians can continue to use their own theoretical approach to treat the drivers of suicidality identified through the therapeutic process, making the approach more amenable to uptake by a wider range of clinicians. Replicated trials demonstrate that CAMS used with adults is reliably associated with improvements in overall symptom distress, rapid reduction in suicidal ideation, and changes in the “suicidal mode” (i.e., suicide cognitions; ratio of wish to live vs. wish to die ratings; implicit associations with suicide) over the course of clinical care (Comtois et al., 2011; Ellis et al., 2012; Jobes, 2012; Swift et al., in press). Further, individuals treated with CAMS report higher patient satisfaction, better treatment retention, increased hope, decreased hopelessness, and reduced medical healthcare utilization compared to enhanced treatment as usual (Comtois et al., 2011; Swift et al., in press) and the assessment experience embedded in the CAMS experience functions as a therapeutic assessment as highlighted in a metanalysis of different clinical assessment approaches (Poston & Hanson, 2010).
Prior work outlines the theoretical framework and application of clinical practice perspectives of CAMS with youth populations highlighting the consistency of the framework with developmental goals of adolescence (O’Connor et al., 2014) and modifications for use with young children (Ridge-Anderson et al., 2016). Further, the psychometric validation of the SSF Core Assessment (which is completed at the start of every CAMS session) indicates that it is valid and reliable for use with adolescents and does not need to be adapted or modified for this age group (Brausch et al., 2020). The 2-factor structure of the measure accurately discriminated between youth whose suicidality resolved and those with persistent SI. The SSF overall suicide risk item was correlated with both past history of self-injury as well as implicit bias toward suicide-related stimuli indicating that the SSF captures suicidality among adolescents well. The support of clinical trial data and relative ease of CAMS training and integration into clinical practice, have generated interest in its use with adolescent populations (Corona et al., 2019). However, we are not aware of any published pilot work to explore its feasibility and utility with this population. CAMS may be the ideal frontline intervention due to its ease of training, and core components structuring the quality improvement practices in suicide-specific treatment. Indeed, there is promising preliminary evidence among suicidal colleges students that CAMS might be an effective initial response for many suicidal patients with DBT being offered to more complex cases of patients with multiple suicide attempt histories, more baseline dysregulation, and borderline personality features (Pistorello et al., 2020), which creates the promise of matching different evidence-based treatments to different patient populations instead of using a “one size fits all” approach to clinical care (Jobes & Chalker, 2019).
To address the need for more data on promising suicide-specific care for adolescents, the goals of the present study are to report on feasibility, acceptability and appropriateness of the CAMS intervention with suicidal adolescents to examine the proximal outcomes including suicidal thoughts and behaviors and the SSF Core Assessment constructs that are evaluated over the entire course of CAMS-guided care (i.e., psychological pain, stress, agitation, hopelessness, and self-reported overall behavioral risk of suicide. In order to understand the relative impact of CAMS, compared to other outpatient interventions for suicidality, a series of benchmarking analyses were conducted.
Method
Setting and Recruitment
Participants were recruited from Seattle Children’s Hospital (SCH), a tertiary medical center serving youth in a five-state WWAMI region. SCH mental health services provide short and longer-term community mental health programs including inpatient psychiatric, medical consult-liaison, and outpatient crisis and psychotherapy services. Flyers and emails were sent to providers in emergency, inpatient and outpatient settings in the hospital. Clinicians involved in the evaluation or treatment of potential participants provided a description of the study procedures to patients who may be eligible. When the patient agreed to obtain more information, a study staff screened the potential participant and conducted consent procedures.
Participants
As a preliminary exploratory study of using CAMS with suicidal adolescents, both patients and clinicians are considered participants as both reported on their own perspectives and experience in the trial.
Adolescent patients.
Potential participants were recruited from psychiatric emergency, consult liaison psychiatric services, inpatient psychiatry and outpatient psychotherapy services at SCH. Participants were included in the study if they were: 13–17 years of age, exceeded the clinical cut-off (score 7) on the Suicidal Behavior Questionnaire-Revised (SBQ-R), provided consent and parental permission to participate in study procedures. To maximize study generalizability, we excluded only youth for whom participation in research was not possible or study treatment was inappropriate. As such, adolescents were excluded if they presented with diagnoses of psychosis, life threatening eating disorder, or autism spectrum diagnosis/intellectual disability as these diagnoses may warrant a different treatment pathway and limited English proficiency that would interfere with ability to completing assessments.
Clinicians.
Study providers included licensed master’s and doctoral-level clinicians and postdoctoral trainees. All recruited clinicians identified as female (100%) and White, Non-Hispanic/Latinx (100%). Clinical experience ranged from one to 10+ years.
Treatment
Patient participants were assigned to CAMS for up to 16 60-minute sessions, with treatment length determined by CAMS clinical response criteria. The clinical response criteria were, as defined in prior trials and the CAMS manual (Jobes, 2016) three consecutive sessions of low ratings of overall risk along with management of suicidal thoughts and feelings and no suicidal behaviors. CAMS in this study was implemented as presented in the published manual in its second edition, and as has been evaluated with adult populations (Jobes, 2016). As noted earlier, the treatment has an assessment framework embedded in SSF that helps a clinician to collaboratively identify, engage, conceptualize, along with the patient the nature of their suicidal struggle so that driver-focused outpatient treatment can be “co-authored” by the clinician and patient. Treatment strategies that target and treat patient-defined drivers of suicidality (e.g., being bullied at school, conflicts with parents, or a history of sexual abuse by a family member) allow clinicians flexibility and do not require a clinician to shift their theoretical orientation beyond adopting ongoing measurement feedback via the SSF and driver-focused care.
Parent participation was not restricted, and parents participated in at least two of the CAMS sessions at the first and last session. The first session allowed parents an opportunity to a) provide the therapist with their perspective on the adolescent’s functioning; and b) discuss and problem solve in relation to the CAMS Stabilization Plan that is developed in the first session. Within CAMS the final outcome/disposition affords a summary of the work done to date, discussion of the adolescent’s drivers, and any additional care needed now or in the future.
Assessments and Measures
Assessments were completed at baseline, end of treatment, and a follow-up assessment completed six months after study enrollment. Assessments examined suicidality outcomes, depressive symptoms, hopelessness, therapeutic alliance, and service use.
Suicide Risk Outcomes
Suicidal Behavior.
The Suicide Behaviors Questionnaire-Revised (SBQ-R; Osman et al., 2001) is a four-item screening measure that assesses suicidal ideating, suicide attempts, threats of suicide, and self-reported likelihood of suicide. Higher scores represent greater suicide risk. The SBQ-R cut off of 7 has been used in college populations to identify those at risk for suicide with high sensitivity and specificity (Osman et al., 2001).
Suicidal Ideation.
The Suicidal Ideation Questionnaire Junior (SIQ-Jr; Reynolds, 1987) was used to assess the severity of suicidal ideation (Reynolds, 1987). Fifteen items were rated on a Likert scale from I’ve never had this thought (0) to I have this thought every day (6), with higher scores indicating higher severity of suicidal ideation. The internal consistency was strong (α=.90), similar to prior research (Reynolds, 1987; Reynolds & Mazza, 1999).
Suicide Risk Severity.
Clinicians reported on suicidal behaviors at the end of treatment. The Columbia Suicide Severity Rating Scale (C-SSRS; Posner et al., 2008) is a structured interview indicating suicidal ideation, suicide attempts, and NSSI. The scale has good convergent and divergent validity, and the intensity of ideation subscale has moderate to strong internal consistency (α=.73).
Suicide Status Form Core Assessment.
The Suicide Status Form (SSF; (Jobes et al., 1997a) is a multipurpose clinical tool that is used to assess, treatment plan, track, and document clinical outcomes. It broken into three main phases four pages for the 1st session SSF, two pages of an interim-tracking version of the SSF, and finally a two-page outcome-disposition version of the SSF. The SSF Core Assessment constructs of psychological processes related to suicide risk- psychological pain, stress, agitation, hopelessness, and self-hate on a five-point Likert scale are used across every phase of CAMS-guided care. In the first CAMS session, the SSF Core Assessment ratings have prompts for written descriptions of construct (e.g., what they find most painful or what makes them hopeless). The youth also reports on overall self-reported risk of suicide (Jobes, 2016). Youth rank order the relative importance of the five SSF Core Assessment constructs and youth further describe the nature of their suicidality in terms of their respective reasons for living and dying. Prior demonstrations of the quantitative and qualitative aspects of the SSF have been demonstrated with good reliability and validity for youth (Brausch et al., 2020), college students (Jobes et al., 1997), and suicidal inpatients (Conrad et al., 2009). The first session CAMS SSF assessment ends with a review of key warning signs and risk factors (e.g., suicidal thoughts, past attempts, substance abuse, etc.).
Depression Severity.
The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001) is a self-report questionnaire of depression about the frequency and severity of depressive symptoms in the past two weeks. The 9 items map onto the DSM-IV criteria for major depressive disorder (American Psychiatric Association, 1994). The PHQ-9 is widely used and shows good validity, and high sensitivity and specificity. Internal consistency is high (α = .86) (Richardson et al., 2014).
Implementation Outcomes
Acceptability.
The 4-item Acceptability of Intervention Measure (Weiner et al., 2017) was used to assess clinicians’, adolescents’ and parents’ perception of acceptability including liking, approving, and welcoming use of CAMS intervention. Items were rated on a 5-point Likert scale from completely disagree (1) to completely agree (5). Prior psychometric evaluation suggest acceptable measurement model fit and high reliability (Weiner et al., 2017) and internal consistency in this study was strong (α= .93).
Appropriateness.
The 4-item Intervention Appropriateness Scale (Weiner et al., 2017) was used to assess clinicians’ adolescents’ and parents’ perception of fit with items related to fit for the setting, applicability to their work and a good match for the needs of the users. Items were rated on a 5-point Likert scale from completely disagree (1) to completely agree (5). Prior psychometric evaluation suggest acceptable measurement model fit and high reliability (Weiner et al., 2017) and internal consistency in this study was excellent (α=.97).
Feasibility.
The 4-item Feasibility of Intervention Measure (Weiner et al., 2017) used to assess clinicians’ adolescents’ and parents’ perception of acceptability including possible, doable, and easy of using CAMS. Items were rated on a 5-point Likert scale from completely disagree (1) to completely agree (5). Internal consistency in this study was strong (α=.91).
Adherence.
The CAMS Rating Scale, version 3 (CRS.3; (Jobes, 2016) was used to assess adherence to the CAMS framework. CMR.3 has 14 total items, each of which is rated on a 7-point Likert scale ranging from 0 (poor) to 6 (excellent), with 3 representing satisfactory performance. The first 11 items are organized into domains that capture key components of the CAMS framework. These domains are Collaboration, Suicide Focus, Risk Assessment, Treatment Planning, and Intervention. Prior work demonstrates the CRS.3 as reliable and valid (Corona & Jobes, 2015; Corona et al., 2019). All sessions were audiotaped and coded for adherence to the key elements of the CAMS intervention. The first author coded sessions and was trained by Dr. Jobes in using the CRS.3.
Qualitative Ratings.
Clinicians were asked about their experience with the intervention with two open ended questions about CAMS with the following prompts: (1) Please tell us what you like about using CAMS as a therapeutic framework. (2) Please tell us about changes that you would like to see when treating youth with suicide risk in a CAMS framework.
Data Analyses
Descriptive statistics are provided to describe study recruitment, participation, and follow-up rates. Means and standard deviations are presented for acceptability, feasibility, and appropriateness measures. To explore the effect of CAMS on resolving suicide risk, CAMS remission criteria (three sessions of low self-reported risk, management of suicide urges, and no self-injury) was used. In addition, to explore the impact of CAMS on significant proximal outcomes of theoretically linked risk factors for suicide of psychological pain, agitation, hopelessness, self-hate, and self-reported suicide risk, generalized linear mixed models were used. Finally, benchmarking analyses were conducted to compare the relative effectiveness of CAMS to another outpatient intervention for adolescent suicidality. A priori definition of equivalence is an effect size within .2 of the benchmarked trials (Minami et al., 2007). All analyses were intent to treat.
Procedures
The study was approved by the Institutional Review Board (IRB), and all participants including a parent/guardian completed and signed IRB approved consent forms before beginning the initial evaluation process. Youth who were eligible and gave consent, completed a clinic-based baseline assessment to confirm eligibility and obtain baseline characteristics. Ineligible or declining youth were offered treatment referrals outside the study.
All participants received CAMS until they met CAMS response criteria for a period of up to 16 sessions. After patients were identified for the study, a research assistant did a brief screening to determine eligibility and schedule a baseline assessment. Treatment sessions were held in an outpatient psychiatry clinic at a children’s hospital. Weekly clinical assessments were administered by the clinician including the PHQ-9 and the Suicide Status Form. Follow-up assessments were conducted at end of treatment and 24 weeks from baseline. All adolescent and parent follow-up assessments were conducted on online platform Qualtrics. For parents and youth who did not complete measures through the online platform, the option of completing the measures in person or by phone was given.
Results
Baseline Characteristics
Figure 1 summarizes patient flow through study procedures. Out of the 45 adolescents screened, 23 were excluded due to ineligibility (N=6), no interest in participating (N=11), and lost to follow-up (N=6). Of the adolescents eligible to participate, 22 provided parental and teen consent. At follow-up, 16 youth completed a follow-up assessment. Figure 1 demonstrates the participant flow through the study. Eighteen youth described themselves as White, two as Pacific Islander, two as Latinx, one Asian, and one preferred not to respond. At baseline, the mean SBQ-R score was 13.45 (SD = 2.64; range 8–17) and SIQ mean score was 56.05 (SD = 18.92; range 15–88) indicating a suicidal sample exceeding clinical cut-offs on both measures. Seven out of 22 youth (31.8%) endorsed a lifetime suicide attempt at baseline. Eighteen out of 22 (81.8%) of the patients in the study were prescribed psychiatric medications. For those prescribed medications, 8 youth were prescribed 1 medication, nine youth were prescribed 2 medications, and 1 youth was prescribed three medications. The most common medication class prescribed was an SSRI (89%), followed by an alpha-2 agonist (16.7%) and antihistamine (16.7%). Table 1 summarizes the demographic and clinical characteristics of the adolescent participants and Table 2 summarizes the clinician characteristics.
Figure 1.
CAMS-4Teens Consort Diagram
Table 1.
Summary of demographics and clinical characteristics for adolescent participants.
Characteristics | Adolescents (N=22) |
---|---|
| |
n (%) | |
Sex at birth | |
Male | 7 (31.8%) |
Female | 15 (68.2%) |
Intersex | 0 |
Total | 22 (100%) |
Gender | |
Male | 8 (36.4%) |
Female | 13 (59.1%) |
Transgender Male | 1 (4.5%) |
Transgender Female | 0 |
Total | 22 (100%) |
Age (years) | |
Mean ± sd | 15.41 ± 1.26 |
Ethnicity (Hispanic or Latino) | |
Not Hispanic or Latino | 18 (81.82%) |
Hispanic, of Spanish Origin or Latino | 4 (18.18%) |
Prefer not to answer | 0 (0%) |
Total | 22 (100%) |
Race | |
White | 18 (81.82%) |
Black or African American | 0 |
American Indian or Alaska Native | 0 |
Asian | 1 (4.55%) |
Native Hawaiian or Other Pacific Islander | 2 (9.09%) |
Other, not specified above | 1 (4.55%) |
Unknown or prefer not to answer | 0 |
Multiracial | 0 |
Total | 22 (100%) |
Suicide Attempt History | |
Suicidal Ideation | 3 (13.6%) |
Suicide Plan | 12 (54.5%) |
Suicide Attempt | 7 (31.8%) |
Total | 22 (100%) |
Note. Youth could elect multiple responses for their racial identity.
Table 2.
Summary of demographics for clinicians.
Characteristics | Clinicians (N=5) |
---|---|
| |
n (%) | |
Gender | |
Female | 5 (100%) |
Total | 5 (100%) |
Age (years) | |
25 to 34 | 2 (40%) |
35 to 44 | 2 (40%) |
55 to 64 | 1 (20%) |
Total | 5 (100%) |
Ethnicity (Hispanic or Latino) | |
Not Hispanic or Latino | 5 (100%) |
Hispanic, of Spanish Origin or Latino | 0 |
Total | 5 (100%) |
Race | |
White | 5 (100%) |
Black or African American | 0 |
American Indian or Alaska Native | 0 |
Asian | 0 |
Native Hawaiian or Other Pacific Islander | 0 |
Other, not specified above | 0 |
Multiracial | 0 |
Total | 5 (100%) |
Degree | |
Master’s | 2 (40%) |
PhD | 3 (60%) |
Total | 5 (100%) |
Professional Role | |
Psychologist | 3 (60%) |
Mental Health Counselor | 2 (40%) |
Total | 5 (100%) |
Years of Experience | |
1–3 | 2 (40%) |
4–6 | 1 (20%) |
7–9 | 1 (20%) |
10 or more | 1 (20%) |
Total | 5 (100%) |
Note.
Feasibility, Acceptability, and Appropriateness Outcomes
Adolescents participated in an average of 5.22 sessions (SD = 4.29, range = 1–16). Table 3 shows the core implementation outcomes. Adolescents rated CAMS, on average, as moderately acceptable (M = 3.93, SD = .91), appropriate (M = 3.61, SD=1.16), and feasible (M = 3.97, SD = .79); whereas clinicians who completed measures of acceptability, feasibility, and appropriateness indicated a high degree of satisfaction with the CAMS framework: acceptability (M = 4.60 SD = .51), appropriateness (M = 4.73, SD = .46), and feasibility (M = 4.27, SD = .59). Fidelity to the intervention was evaluated using audiotaped sessions. Clinicians demonstrated high levels of adherence with a mean score of 5.27 (out of 6).
Table 3.
Adolescent and Clinician Outcomes
Outcome | Adolescent Report | Clinician Report | ||
---|---|---|---|---|
| ||||
Mean | SD | Mean | SD | |
| ||||
Acceptability | 3.93 | .91 | 4.60 | .51 |
Appropriateness | 3.61 | 1.16 | 4.73 | .46 |
Feasibility | 3.97 | .79 | 4.27 | .59 |
Fidelity | -- | -- | 5.27 | .78 |
Note. Abbreviations. SD=Standard Deviation
Clinician’s qualitative comments augmented our understanding of the acceptability and appropriateness of the CAMS intervention. For example, one clinician noted:
I really like that CAMS gives me a structure to follow during what can be very painful conversations. It helps me feel like I’m staying on track with what will be most helpful for families. I also really appreciate that it gives kids/parents the chance to actually complete their own information. I think it’s helpful to be transparent with clients about what exactly you’re doing in session, as opposed to them telling me things and me writing it on my pad. Maybe that sounds a little funny, but I think it adds to their comfort in knowing that everything that’s happening in the session is right in front of them.”
Outcome Descriptives
In evaluation of CAMS clinical response, 54.5% of youth resolved their suicidality during their course of treatment as per CAMS resolution criteria. Youth suicidal ideation, as measured by the SIQ-Jr, significantly decreased over the course of treatment (t = 3.33, p =.005, d =.67). The mean SIQ-Jr score at the end of treatment was 40 (SD = 19.85), indicating an average 10-point decrease. Clinicians reported that 22.7% (N=5) youth endorsed NSSI, 13.6% (N=3) of youth had a suicide attempt and 9.1% (N=2) had an aborted suicide attempt. The two youth who an aborted suicide attempt also reported a suicide attempt. There was also a statistically significant decrease in depression severity (t (20) = 5.24, p <.001, d = 1.14), such that on average youths’ depression severity ratings decreased by 6.09 points (see Table 4).
Table 4.
Suicide Risk Descriptive Outcomes
Outcome | Pretreatment | End of Treatment | 6 months Follow up | |||
---|---|---|---|---|---|---|
| ||||||
Mean | SD | Mean | SD | Mean | SD | |
| ||||||
SIQ-JR total score | 56.05 | 18.92 | 40.00 | 19.85 | 45.85 | 25.71 |
SBQ-R total score | 13.45 | 2.64 | 12.25 | 2.91 | 13.12 | 2.07 |
PHQ9 | 20.33 | 5.18 | 14.32 | 6.39 | 15.23 | 7.97 |
Note. Pre-treatment (N = 22); End of treatment (N = 16); 6-month follow up (N = 16)
Generalized Mixed Models
We estimated unconditional multilevel models that included days in treatment as a Level-1 variable in order to further evaluate the impact of treatment on theoretically meaningful suicidal risk factors as measured on the SSF (Table 5). Results indicated significant improvement in psychological pain (β = −.0075, SE = .002, p <.001), agitation (β = −.007, SE = .002, p <.001), hopelessness (β = −.01, SE = .003, p =.001), and self-hate (β = −.009, SE= .002, p <.001). Furthermore, self-reported risk of suicide evidenced a significant decrease over the course of treatment (β = −.008, SE= .002, p <.001). There was not a significant effect on stress (β = −.004, SE = .003, p = .17).
Table 5.
GLMM Outcomes for Core SSF Assessment Constructs
Outcome | Coefficient (SE) | p | B |
---|---|---|---|
| |||
Psychological Pain | −.007546 (.002) | .001* | .42 |
Agitation | −.007 (.002) | .005* | .41 |
Stress | −.004(.003) | .17 | .15 |
Hopelessness | −.01(.003) | .001* | 1.10 |
Self-hate | −.009(.002) | <.001** | .61 |
SSF Suicide Risk | −.008(.002) | <.001** | .54 |
p < .01
p < .001 (N=22)
Benchmarking Outcomes Using Prior Studies
In order to evaluate the relative effectiveness of CAMS in resolving adolescent STB, we compared our outcomes from the current open trial to outcomes from a randomized control trial evaluating a brief, family-focused outpatient intervention for suicidality (Resourceful Adolescent Parent program; RAP-P; Pineda & Dadds, 2013), a more intensive outpatient randomized controlled trial (i.e., DBT; McCauley et al., 2018) and CAMS in adult population (Ellis et al., 2017). RAP-P demonstrated a large effect size on suicidality (d =.76) as assessed via the Ask Suicide Questionnaire, (encompassing both suicidal ideation and behavior). Our findings similarly indicate a large effect size for CAMS on suicidal ideation (d =.67). Accordingly, our benchmarking analyses suggest that CAMS performs comparably to an established outpatient suicide prevention program (i.e., less than a .2 difference in effect sizes) (Minami et al., 2007).When we benchmark the impact of CAMS on suicidal ideation for adolescent populations compared to adult populations, the observed effect size in our study (d=.67) is diminished in comparison to more intensive psychotherapies (i.e., DBT; d=1.62; McCauley et al., 2018) and adult trials (d=.98–1.89; Comtois, 2011; Jobes et al., 2017). The prior DBT and CAMS work also reported suicide attempt outcomes and therefore two chi square tests of independence were performed to examine the relationship between treatment and suicide attempt. There were no significant differences between proportion of reporting suicide attempts in this study compared to the adolescent DBT study (13.6% CAMS4TEEN vs. 9.7% in DBT; χ2 (1, N= 94) = .27, p =.60) or the adult CAMS study (11%; χ2 (1, N= 95) = .11, p =.73).
Discussion
The results of this open pilot trial demonstrate that CAMS was feasible to deliver with a suicidal adolescent population, acceptable and appropriate to adolescents, their parents, and clinicians to treat suicidal risk. CAMS, an intervention supported by replicated trials with adults, may offer another approach for adolescents given its effectiveness with adults and its lower cost (c.f., Jobes et al 2015; McCutchen, 2020) and burden compared to other interventions. Core components of CAMS includes Stabilization Planning while focusing on identification and intervention with idiographic problems that make suicide appealing to the patient (i.e., the “patient-defined drivers”). Adolescent and clinician stakeholders gave feedback on key implementation outcomes of feasibility, acceptability, and appropriateness suggesting that the intervention could be conducted safely, was well matched to youth’s needs, and youth and clinicians were satisfied with the treatment approach. It is important to note, however, that clinicians reported higher ratings on these metrics compared to adolescents. It is possible that clinicians like this modality as it provides a structure for determining specific and unique factors contributing to risk for each adolescent presenting with suicidal thoughts and behaviors (STB) as there are few interventions to support clinicians in delivering the core components of suicide-specific care (Jobes, 2016).
In addition to implementation outcomes, there were promising outcomes related to STB. The majority of youth resolved, according to CAMS criteria, demonstrating four consecutive weeks of no STB, management of urges, and low self-reported suicide risk, similar to the response rates obtained in adult populations (Pistorello et al., 2018) as well as other well-established interventions for self-injury (i.e., in the CARES DBT trial 54.2% of youth had no injury at the end of the intervention; McCauley et al., 2018). When the severity of suicidal ideation was used as the primary outcome, a moderate effect size was observed in reduction of suicidal ideation, on par with other brief programs but smaller in impact than the CAMS trial with adults (Ellis et al., 2017; Jobes et al., 2017) and other more comprehensive programs for youth (i.e., DBT; McCauley et al., 2018). Finally, 13.6% of youth reported suicide attempts during the follow up period, a proportion comparable to the reported number of suicide attempts during treatment for the adolescent DBT study (McCauley et al., 2018), CAMS trial in adult populations (Jobes et al., 2017), and the superiority trial comparing DBT to CAMS (Andreasson et al., 2016). The Andreasson study suggested that 10 weekly sessions of CAMS was equally impactful as twice weekly DBT for 16 weeks. There are several important directions and implications of this work. Developing a number of scalable and accessible suicide-specific interventions is needed to reduce this pressing public health problem. CAMS-4Teens suggests an opportunity for this framework to be evaluated further and integrated with other promising or established interventions. For example, several trials have recently completed or are underway that underscore the importance of matching patient complexity to intervention (Jobes & Chalker, 2019; Kessler et al., 2020) or evaluating adaptive interventions in order to have decision rules based on response to intervention (Pistorello et al., 2020). Our findings suggesting that significant subset of youth resolve their suicidality in a weekly outpatient setting in a few sessions provides promise that we could identify treatment pathways with less (CAMS) and more intensive (DBT) treatment pathways based on initial presentation. In our setting, we have implemented CAMS-4Teens as soon as suicidal ideation and behaviors are identified and patients who do not respond can “step up” to more complex and intensive interventions like DBT. Future work focused on predictors of treatment response and remission would be an important step forward to efficient resource allocation of suicide-specific care.
Findings also suggest promising change on important risk factors for suicide that were measured by the weekly SSF administrations. Though preliminary, significant within group improvements were observed across psychological pain, agitation, hopelessness, self-hate, and self-reported risk of suicide over the course of treatment. Stress ratings did not change reliably across treatment.
There are several limitations to consider in contextualizing these initial findings. First, given the exploratory nature of the work, youth were not randomized to the CAMS intervention and therefore there was no comparison condition to ground the improvements. In addition, the sample is small and limited to those within the metropolitan area of Seattle and may not generalize to other populations. The sample is very limited with respect to representation of Black youth. The combination of our recruitment which was predominately conducted from within the hospital setting, and the barriers and stigma associated with accessing psychiatric care may account for lack of enrollment of Black youth in this small trial. Given the growing understanding of younger Black youth’s risk (Ruch et al., 2019) and racial disparity in mental health service utilization (Assari & Caldwell, 2017), culturally responsive suicide prevention efforts for Black youth ought to be integrated into community and religious settings (Molock et al., 2008) and explicitly address impact and risk associated with racism (Robinson et al., 2021).
In summary, this is the first study to evaluate the impact of CAMS guided care in order to reduce adolescent STB. The small pilot indicates high ratings of feasibility, acceptability, and appropriateness, particularly as reported by clinicians and, to a lesser extent, adolescents. With regards to STB outcome, there was promising data regarding impact of key suicide outcomes including a sizable proportion of youth meeting response criteria, a large effect size on suicidal ideation, and few suicide attempts during the follow up period. These findings warrant a larger trial powered to detect effects between other standard interventions for suicidal youth. As little is known about which adolescents will benefit the most from available treatments or how to match treatment decisions to a youth’s specific suicide risk and history, large trials evaluating the heterogeneity of treatment effects on suicide risk outcomes as well as determining if there are differences in youth preferences based on key implementation outcomes (i.e., the feasibility, acceptability, and appropriateness) that ultimately affect who will receive and benefit from the intervention is critically needed.
Clinical Impact Statement.
This open pilot trial demonstrated that the Collaborative Assessment and Management of Suicidality framework was feasible and acceptable to adolescents and clinicians. It is an important public health priority to increase access to and evidence for suicide-specific intervention strategies that can improve suicide prevention efforts with adolescent populations.
Acknowledgments
This research was supported by the Agency for Healthcare Research and Quality (K12HS022982; Adrian).
References
- American Psychiatric Association. (1994). DSM-IV: Diagnostic and Statistical Manual of Mental Disorders. JAMA: The Journal of the American Medical Association, 272(10), 828. 10.1001/jama.1994.03520100096046 [DOI] [Google Scholar]
- Andreasson K, Krogh J, Wenneberg C, Jessen HKL, Krakauer K, Gluud C, Thomsen RR, Randers L, & Nordentoft M. (2016). Effectiveness of dialectical behavior therapy versus collaborative assessment and management of suicidality treatment for reduction of self-harm in adults with borderline personality traits and disorder- a randomized observer-blinded clinical trial. Depression and Anxiety, 33(6), 520–530. 10.1002/da.22472 [DOI] [PubMed] [Google Scholar]
- Asarnow JR, Hughes JL, Babeva KN, & Sugar CA. (2017). Cognitive-behavioral family treatment for suicide attempt prevention: A randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 56(6), 506–514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Assari S, & Caldwell C. (2017). Mental Health Service Utilization among Black Youth; Psychosocial Determinants in a National Sample. Children, 4(5), 40. 10.3390/children4050040 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brausch AM, O’Connor SS, Powers JT, McClay MM, Gregory JA, & Jobes DA (2020). Validating the Suicide Status Form for the Collaborative Assessment and Management of Suicidality in a Psychiatric Adolescent Sample. Suicide and Life-Threatening Behavior, 50(1), 263–276. 10.1111/sltb.12587 [DOI] [PubMed] [Google Scholar]
- Comtois KA, Jobes DAS, O’Connor S, Atkins DC, Janis K, C E. Chessen, Landes SJ, Holen A, & Yuodelis-Flores C. (2011). Collaborative assessment and management of suicidality (CAMS): Feasibility trial for next-day appointment services. Depression and Anxiety, 28(11), 963–972. 10.1002/da.20895 [DOI] [PubMed] [Google Scholar]
- Conrad R, Walz F, Geiser F, Imbierowicz K, Liedtke R, & Wegener I. (2009). Temperament and character personality profile in relation to suicidal ideation and suicide attempts in major depressed patients. Psychiatry Research, 170(2–3), 212–217. 10.1016/j.psychres.2008.09.008 [DOI] [PubMed] [Google Scholar]
- Corona CD, Gutierrez PM, Wagner BM, & Jobes DA (2019). Assessing the Reliability of the CAMS Rating Scale Using a Generalizability Study. Crisis, 40(4), 273–279. 10.1027/0227-5910/a000565 [DOI] [PubMed] [Google Scholar]
- Corona CD, & Jobes DA (2015). The psychometric properties of the CAMS rating scale: A preliminary evaluation. 48th Annual Conference of the American Association of Suicidology, Atlanta, Georgia. [Google Scholar]
- Cottrell DJ, Wright-Hughes A, Collinson M, Boston P, Eisler I, Fortune S, Graham EH, Green J, House AO, Kerfoot M, Owens DW, Saloniki E-C, Simic M, Lambert F, Rothwell J, Tubeuf S, & Farrin AJ (2018). Effectiveness of systemic family therapy versus treatment as usual for young people after self-harm: A pragmatic, phase 3, multicentre, randomised controlled trial. The Lancet Psychiatry, 5(3), 203–216. 10.1016/S2215-0366(18)30058–0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Curtin S, Warner M, & Hedegaard H. (2016). Increase in Suicide in the United States, 1999–2014. NCHS Data Brief, 241, 1–8. [PubMed] [Google Scholar]
- Diamond GS, Kobak RR, Krauthamer Ewing ES, Levy SA, Herres JL, Russon JM, & Gallop RJ (2019). A Randomized Controlled Trial: Attachment-Based Family and Nondirective Supportive Treatments for Youth Who Are Suicidal. Journal of the American Academy of Child & Adolescent Psychiatry, 58(7), 721–731. 10.1016/j.jaac.2018.10.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ellis TE, Green KL, Allen JG, Jobes DA, & Nadorff MR (2012). Collaborative Assessment and Management of Suicidality in an Inpatient Setting: Results of a Pilot Study. Psychotherapy (Chicago, Ill.), 49(1), 72–80. 10.1037/a0026746 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ellis TE, Rufino KA, & Allen JG (2017). A controlled comparison trial of the Collaborative Assessment and Management of Suicidality (CAMS) in an inpatient setting: Outcomes at discharge and six-month follow-up. Psychiatry Research, 249, 252–260. 10.1016/j.psychres.2017.01.032 [DOI] [PubMed] [Google Scholar]
- Jobes DA (2012). The Collaborative Assessment and Management of Suicidality (CAMS):An evolving evidence-based clinical approach to suicidal risk. Suicide and Life-Threatening Behavior, 42(6), 640–653. 10.1111/j.1943-278X.2012.00119.x [DOI] [PubMed] [Google Scholar]
- Jobes DA (2016). Managing suicidal risk: A collaborative approach (Second edition). The Guilford Press. [Google Scholar]
- Jobes DA, & Chalker SA (2019). One size does not fit all: A comprehensive clinical approach to reducing suicidal ideation, attempts, and deaths. International Journal of Environmental Research and Public Health, 16(19), 3606. 10.3390/ijerph16193606 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jobes DA, Comtois KA, Gutierrez PM, Brenner LA, Huh D, Chalker SA, Ruhe G, Kerbrat AH, Atkins DC, Jennings K, Crumlish J, Corona CD, Connor SO, Hendricks KE, Schembari B, Singer B, & Crow B. (2017). A Randomized Controlled Trial of the Collaborative Assessment and Management of Suicidality versus Enhanced Care as Usual with Suicidal Soldiers. Psychiatry, 80(4), 339–356. 10.1080/00332747.2017.1354607 [DOI] [PubMed] [Google Scholar]
- Jobes DA, Jacoby AM, Cimbolic P, & Hustead LAT (1997). Assessment and treatment of suicidal clients in a university counseling center. Journal of Counseling Psychology, 44(4), 368–377. 10.1037/0022-0167.44.4.368 [DOI] [Google Scholar]
- Kessler RC, Chalker SA, Luedtke AR, Sadikova E, & Jobes DA (2020). A preliminary precision treatment rule for remission of suicide ideation. Suicide and Life-Threatening Behavior, 50(2), 558–572. 10.1111/sltb.12609 [DOI] [PubMed] [Google Scholar]
- Kothgassner OD, Robinson K, Goreis A, Ougrin D, & Plener PL (2020). Does treatment method matter? A meta-analysis of the past 20 years of research on therapeutic interventions for self-harm and suicidal ideation in adolescents. Borderline Personality Disorder and Emotion Dysregulation, 7(1), 9. 10.1186/s40479-020-00123-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kroenke K, Spitzer RL, & Williams JBW (2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Large M, Corderoy A, & McHugh C. (2021). Is suicidal behaviour a stronger predictor of later suicide than suicidal ideation? A systematic review and meta-analysis. Australian & New Zealand Journal of Psychiatry, 55(3), 254–267. 10.1177/0004867420931161 [DOI] [PubMed] [Google Scholar]
- Lowry R, Crosby AE, Brener ND, & Kann L. (2014). Suicidal thoughts and attempts among U.S. high school students: Trends and associated health-risk behaviors, 1991–2011. Journal of Adolescent Health, 54(1), 100–108. 10.1016/j.jadohealth.2013.07.024 [DOI] [PubMed] [Google Scholar]
- McCauley E, Berk MS, Asarnow JR, Adrian M, Cohen J, Korslund K, Avina C, Hughes J, Harned M, Gallop R, & Linehan MM (2018). Efficacy of Dialectical Behavior Therapy for Adolescents at High Risk for Suicide: A Randomized Clinical Trial. JAMA Psychiatry, 75(8), 777. 10.1001/jamapsychiatry.2018.1109 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mehlum L, Ramberg M, Tørmoen AJ, Haga E, Diep LM, Stanley BH, Miller AL, Sund AM, & Grøholt B. (2016). Dialectical Behavior Therapy Compared with Enhanced Usual Care for Adolescents with Repeated Suicidal and Self-Harming Behavior: Outcomes Over a One-Year Follow-Up. Journal of the American Academy of Child & Adolescent Psychiatry, 55(4), 295–300. 10.1016/j.jaac.2016.01.005 [DOI] [PubMed] [Google Scholar]
- Mehlum L, Tørmoen AJ, Ramberg M, Haga E, Diep LM, Laberg S, Larsson BS, Stanley BH, Miller AL, Sund AM, & Grøholt B. (2014). Dialectical Behavior Therapy for Adolescents with Repeated Suicidal and Self-harming Behavior: A Randomized Trial. Journal of the American Academy of Child & Adolescent Psychiatry, 53(10), 1082–1091. 10.1016/j.jaac.2014.07.003 [DOI] [PubMed] [Google Scholar]
- Minami T, Wampold BE, Serlin RC, Kircher JC, & Brown GS (Jeb). (2007). Benchmarks for psychotherapy efficacy in adult major depression. Journal of Consulting and Clinical Psychology, 75(2), 232–243. 10.1037/0022-006X.75.2.232 [DOI] [PubMed] [Google Scholar]
- Molock SD, Matlin S, Barksdale C, Puri R, & Lyles J. (2008). Developing Suicide Prevention Programs for African American Youth in African American Churches. Suicide and Life-Threatening Behavior, 38(3), 323–333. 10.1521/suli.2008.38.3.323 [DOI] [PubMed] [Google Scholar]
- O’Connor SS, Brausch A, Ridge Anderson A, & Jobes DA (2014). Applying the collaborative assessment and management of suicidality (CAMS) to suicidal adolescents. International Journal of Behavioral Consultation and Therapy, 9(3), 53–58. 10.1037/h0101641 [DOI] [Google Scholar]
- Osman A, Bagge CL, Gutierrez PM, Konick LC, Kopper BA, & Barrios FX (2001). The Suicidal Behaviors Questionnaire-Revised (SBQ-R): Validation with Clinical and Nonclinical Samples. Assessment, 8(4), 443–454. 10.1177/107319110100800409 [DOI] [PubMed] [Google Scholar]
- Ougrin D, Tranah T, Stahl D, Moran P, & Asarnow JR (2015). Therapeutic interventions for suicide attempts and self-harm in adolescents: Systematic review and meta-analysis. Journal of the American Academy of Child & Adolescent Psychiatry, 54(2), 97–107.e2. 10.1016/j.jaac.2014.10.009 [DOI] [PubMed] [Google Scholar]
- Pineda J, & Dadds MR (2013). Family Intervention for Adolescents with Suicidal Behavior: A Randomized Controlled Trial and Mediation Analysis. Journal of the American Academy of Child & Adolescent Psychiatry, 52(8), 851–862. 10.1016/j.jaac.2013.05.015 [DOI] [PubMed] [Google Scholar]
- Pistorello J, Jobes DA, Compton SN, Locey NS, Walloch JC, Gallop R, Au JS, Noose SK, Young M, Johnson J, Dickens Y, Chatham P, Jeffcoat T, Dalto G, & Goswami S. (2018). Developing Adaptive Treatment Strategies to Address Suicidal Risk in College Students: A Pilot Sequential, Multiple Assignment, Randomized Trial (SMART). Archives of Suicide Research, 22(4), 644–664. 10.1080/13811118.2017.1392915 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pistorello J, Jobes DA, Gallop R, Compton SN, Locey NS, Au JS, Noose SK, Walloch JC, Johnson J, Young M, Dickens Y, Chatham P, & Jeffcoat T. (2020). A Randomized Controlled Trial of the Collaborative Assessment and Management of Suicidality (CAMS) Versus Treatment as Usual (TAU) for Suicidal College Students. Archives of Suicide Research, 1–25. 10.1080/13811118.2020.1749742 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Posner K, Brent D, Lucas C, Gould M, Stanley B, Brown G, Fisher P, Zelazny J, Burke A, Oquendo M, & Mann J. (2008). Columbia-suicide severity rating scale (C-SSRS). New York, NY: Columbia University Medical Center, 3. [Google Scholar]
- Poston JM, & Hanson WE (2010). Meta-analysis of psychological assessment as a therapeutic intervention. Psychological Assessment, 22(2), 203–212. 10.1037/a0018679 [DOI] [PubMed] [Google Scholar]
- Reynolds WM (1987). Suicidal ideation questionnaire (SIQ). Odessa, FL: Psychological Assessment Resources. [Google Scholar]
- Reynolds WM, & Mazza JJ (1999). Assessment of Suicidal Ideation in Inner-City Children and Young Adolescents: Reliability and Validity of the Suicidal Ideation Questionnaire-JR. School Psychology Review, 28(1), 17–30. 10.1080/02796015.1999.12085945 [DOI] [Google Scholar]
- Richardson LP, Ludman E, McCauley E, Lindenbaum J, Larison C, Zhou C, Clarke G, Brent D, & Katon W. (2014). Collaborative care for adolescents with depression in primary care: A randomized clinical trial. JAMA, 312(8), 809–816. 10.1001/jama.2014.9259 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ridge Anderson A, Keyes GM, & Jobes DA (2016). Understanding and treating suicidal risk in young children. Practice Innovations, 1(1), 3–19. 10.1037/pri0000018 [DOI] [Google Scholar]
- Robinson WL, Whipple CR, Jason LA, & Flack CE (2021). African American adolescent suicidal ideation and behavior: The role of racism and prevention. Journal of Community Psychology, jcop.22543. 10.1002/jcop.22543 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ruch DA, Sheftall AH, Schlagbaum P, Rausch J, Campo JV, & Bridge JA (2019). Trends in suicide among youth aged 10 to 19 years in the United States, 1975 to 2016. JAMA Network Open, 2(5), e193886–e193886. 10.1001/jamanetworkopen.2019.3886 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ryberg W, Diep LM, Landrø NI, & Fosse R. (2019). Effects of the Collaborative Assessment and Management of Suicidality (CAMS) Model: A secondary analysis of moderation and influencing factors. Archives of Suicide Research, 1–20. 10.1080/13811118.2019.1650143 [DOI] [PubMed] [Google Scholar]
- Swift JK, Trusty WT, & Penix EA (in press). The Effectiveness of the Collaborative Assessment and Management of Suicidality (CAMS) Compared to Alternative Treatment Conditions: A Meta-Analysis. Suicide and Life-threatening Behaviors. [DOI] [PubMed] [Google Scholar]
- Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, & Halko H. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12, 108. 10.1186/s13012-017-0635-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zima BT, Murphy JM, Scholle SH, Hoagwood KE, Sachdeva RC, Mangione-Smith R, Woods D, Kamin HS, & Jellinek M. (2013). National quality measures for child mental health care: Background, progress, and next steps. Pediatrics, 131(Supplement 1), S38–S49. [DOI] [PMC free article] [PubMed] [Google Scholar]