Project Kealahou: Improving Hawai‘i's System of Care for At-Risk Girls and Young Women through Gender-Responsive, Trauma-Informed Care

Abstract

Project Kealahou (PK) is a six-year, federally-funded program aimed at improving services and outcomes for Hawai‘i's female youth who are at risk for running away, truancy, abuse, suicide, arrest and incarceration. PK builds upon two decades of sustained cross-agency efforts among the state's mental health, juvenile justice, education, and child welfare systems to promote system-of-care (SOC) principles of community-based, individualized, culturally and linguistically competent, family driven, youth-guided, and evidence-based services. In addition, PK emphasizes trauma-informed and gender-responsive care in serving its target population of females ages 11–18 years who have experienced psychological trauma.

Results from the first four years of the implementation of PK in the Department of Health's (DOH) Child and Adolescent Mental Health Division (CAMHD) highlight the serious familial, socioeconomic, functional, and interpersonal challenges faced by the young women who receive services in Hawai‘i's SOC. Despite the challenges faced by PK youth and their families, preliminary results of the evaluation of PK show significant improvements across multiple clinical and functional domains of service recipients. A financial analysis indicates that these outcomes were obtained with a minimal overall increase in costs when compared to standard care alone. Overall, these results suggest that PK may offer a cost effective way to improve access, care, and outcomes for at-risk youth and their families in Hawai‘i.

Introduction

It is estimated that about two-thirds of children and youth with mental health challenges in the United States do not receive the mental health services they need.¹ In many communities, services for youth with mental health challenges are unavailable, unaffordable, or insufficient, leaving them at risk for difficulties in school and the community.² Hawai‘i, under the auspices of two decades of collaborative state and federal initiatives, is making sustained efforts to improve its mental health services and the overall system of care (SOC) for youth and their families. See Table 1 for an historical timeline of such efforts in Hawai‘i.

Table 1.

Timeline of Hawai‘i's System of Care Projects

Name (Translation)	Duration	Target Population/Location
‘Ohana (“Family”) Project	1994–2000	Ages 13–21/Leeward O‘ahu
Project Ho‘omohala (“Evolving Towards Maturity”)	2005–2010	Ages 15–21/Kalihi-Palama, O‘ahu
Project Kealahou (“A New Pathway”)	2009–2015	Females ages 11–18/Central, Honolulu, and Windward O‘ahu
Project Laulima (“Working Together”)	2012–2017	CAMHD youth with developmental disabilities/Statewide

The foundation for these collaborative efforts—the Community Mental Health Initiative (CMHI)—is funded nationwide by the Substance Abuse and Mental Health Services Administration (SAMHSA) and is one of the most concentrated and sustained federal-state government mental health care partnerships. The CMHI has greatly advanced the SOC model for child and adolescent mental health nationwide,^3,4 and recognizes the importance of family, school, and community contexts in seeking to promote the potential of every child, regardless of mental health challenge. SOC principles aim to ensure that services for youth and their families are family-driven, youth-guided, individualized, culturally and linguistically competent, accessible, community-based, least-restrictive; and provided through interagency, collaborative, and coordinated efforts.⁴

The goal of the SOC is to help families keep the youth at home, in school, out of trouble, and leading balanced, connected, responsible lives in the community. This requires not only extensive coordination and collaboration among disparate and often disconnected service sectors, such as mental health, education, juvenile justice, and child welfare—each a complicated system unto itself—but also requires strong trust and collaboration among youth, families, and their service providers. Continuously eliciting the trust and active participation of youth and their families in individualizing and completing mental health services is a key engagement strategy in Hawai‘i's SOC project. Local treatment engagement efforts are grounded in and supported by the research literature which has found one of the most robust predictors of positive outcomes for CMHI youth to be greater participation in mental health treatment by youth and their caregivers.^5,6

Project Kealahou (PK) intensively engages youth and families through a trauma-informed, culturally-resonant engagement process that includes time to “talk story” before committing to treatment as well as opportunities to make real choices about their service involvement. PK elicits active participation from youth and families in coordinated service planning and project activities throughout the program. A Public Health Insights column in the September 2013 issue of this journal provides more information about the rationale and design of PK.⁷

Once engaged in services, PK girls and their families receive gender-responsive, trauma-informed, culturally-responsive, community-based services, including: intensive case management; community supports by paraprofessionals (ie, peer support for youth and caregivers); structured group activities; and evidence-based treatments (eg, Trauma-Focused Cognitive Behavioral Therapy and Girls Circle psychoeducational support groups). PK seeks to help girls who have experienced psychological trauma find “a new pathway” (kealahou) to a better future by healing past hurts and taking constructive steps toward a more hopeful future.

After two years of planning (2009–2011), and two-and-a-half years of services implementation (2011–2014), PK recently entered its fifth year of a six-year collaborative effort among the mental health, education, juvenile justice, and child welfare service sectors to enhance Hawai‘i's SOC for youth with complex needs. This report describes the basic demographic and clinical features at intake of the youth and families served to date (2011–2014). Preliminary results are presented regarding clinical and functional outcomes as well as participant satisfaction with various aspects of their treatment. Also, a cost analysis compares the level of mental health expenditures for PK participants prior to and after the implementation of PK services.

Methods

Participants

Youth were referred to PK primarily from the public education, juvenile justice, and mental health systems; details of programmatic structure are published elsewhere.⁷ Inclusion criteria were: age less than 18 years; meeting diagnostic criteria for an Axis I disorder as defined in the Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision (DSM-IV-TR); and a history of trauma. Of 304 referrals to PK between 2011 and 2014, eligibility criteria for PK services were met by 234. Of these, 144 chose to enroll in PK services. Once enrolled into PK services, one PK youth and/or one caregiver per family were eligible to participate in the project's evaluation, which consisted of a baseline and four semiannual follow-up interviews. Of the 144 youth who chose to enroll in PK services, 75 declined to participate in the evaluation. After completing a University of Hawai‘i IRB-approved informed consent process, 69 youth and 31 caregivers (providing information on their youth's behalf), for a total of 100 participants, agreed to participate in the evaluation. Twenty-four of these participants were youth who also had a caregiver participate in the evaluation; 45 were youth participating without a caregiver informant; and 7 were caregivers participating without a youth informant. These 100 informants provided information on 76 youth by completing a baseline interview upon enrollment into the evaluation. Twenty-eight youth and 16 caregivers completed both a baseline and a six-month follow-up interview, providing the data on which outcome analyses presented herein are based. Of the 41 youth enrolled in the evaluation without six-month follow-up data, 36 missed their six-month follow-up interview and 5 have not yet reached the six-month interview timeframe.

Variables and Measures

Data were collected through one- to two-hour-long structured interviews with youth and/or their caregivers at intake and at six-month intervals during the first two and a half years of PK services (September 2011 – April 2014). Data collected from respondents included demographic information, including age, race/ethnicity, gender, income, family/child history, presenting problems, clinical outcomes, and participant satisfaction, that was measured on a 5-point scale with “5” representing the highest level of satisfaction. Table 2 details measurement instruments and the participant characteristics derived from them.

Table 2.

Measures and Their Corresponding Variables

Measure	Variables
Behavioral and Emotional Rating Scale, 2nd Edition (BERS-2C/2Y).8	Youth Social and Emotional Strengths
Caregiver Information Questionnaire, Revised: Caregiver-Intake (CIQ-RC-I).9	Caregiver Custody Status, Family/Child History, and Household Income
Caregiver Strain Questionnaire (CGSQ).10	Caregiver Perceived Stress
Child Behavior Checklist (CBCL 6-18).11	Behavioral Problems, Emotional Problems, and Social Competence
Columbia Impairment Scale (CIS).12	Relationship, Behavioral, and Emotional Impairments
Education Questionnaire-Revision 2 (EQ-R2).13	School Performance, Referral Agency, and Agency Involvement
Enrollment and Demographic Information Form (EDIF).14	Age, Ethnicity, Youth Diagnosis, Presenting Problems, and Psychosocial/Environmental Problems
Revised Children's Manifest Anxiety Scale, Second Edition (RCMAS-2).15	Ratings of Youth Anxiety
Reynolds Adolescent Depression Scale, Second Edition (RADS-2).16	Ratings of Youth Depression
Youth Services Survey (YSS); YSS for Families (YSS-F).17	Ratings of Participant Satisfaction with Services

For the Services and Cost Study, service types, durations, and costs were gathered on the 72 participants who were enrolled in the evaluation as of September 2013. For 41 of these participants, cost data was available for both the six months prior to and the six months after the start of PK services, allowing a comparison of the cost of PK and the cost of the standard care received prior to PK enrollment.

Analyses

Pearson chi-squares, paired T-tests and ANOVA were calculated using IBM SPSS Statistics for Windows, Version 22.0 (Armonk, NY) to compare differences in youth and caregiver reports at baseline and six-month follow-up, as well as differences between PK youth enrolled in the evaluation and PK youth not enrolled in the evaluation.

Results

Youth who participated (n = 76) in the evaluation did not differ demographically from youth who did not participate in the evaluation (n = 68). There were no significant differences between these two groups in terms of age (ANOVA; P = .398), racial/ethnic identity (Pearson's chi square; P > .175) or diagnosis (Pearson's chi-square; P > .243). Evaluation participants, however, reported more suicide attempts (Pearson's chi-square; P = .034), greater persistent noncompliance with adults (Pearson's chi-square; P = .031) and more problematic life domains, including social, legal, educational and primary support problems (ANOVA, P = .030) than those not in the evaluation.

Referral Sources and System of Care Involvement

During the first two-and-a-half years of PK, 304 youth were referred from SOC partners, primarily from the education, juvenile justice, and mental health systems. See Table 3 for details. Youth enrolled in PK services (N = 144) were typically receiving intensive and/or specialized services from several state agencies concurrently at intake, with 68% of PK youth reporting involvement with mental health, 52% school-based behavioral health or special education, 35% family court, 30% probation, 20% girls' court, 15% corrections, 15% child welfare, 9% substance abuse, 4% other, and 2% physical health services agencies. Because youth could identify multiple agency involvement, the percentages of these categories sum to more than 100%.

Table 3.

Sources for Project Kealahou Referrals (N=304)

Referral Source	Percentage of Total Referrals
Mental Health	40%
Juvenile Justice	25%
Education	23%
Homeless Shelter	5%
Child Welfare	3%
Self-Referred	2%

PK Youth and Family Characteristics at Intake

Ethnicity and Age

The racial/ethnic backgrounds reported by PK girls (N = 144) at intake were: Native Hawaiian (57%), White (55%), Chinese (38%), Filipino (36%), Japanese (20%), Samoan (16%), Native American/Alaska Native (14%), Puerto Rican (12%), Other Pacific Islander (10%), African American (9%), Mexican (7%), Other Asian (6%) and Other Hispanic (2%). Because participants could choose more than one race/ethnicity, the percentages of these categories sum to more than 100%. The average age of youth at intake (N = 144) was 15.4 years (SD=1.7), with a range from 11 to 18 years.

Diagnoses

The most common diagnoses of PK youth at intake (N = 144) were mood disorders (42%), substance use disorders (32%), conduct disorders (29%), post-traumatic stress disorder (PTSD, 21%), adjustment disorder (14%), attention deficit hyperactivity disorder (ADHD, 13%), oppositional defiant disorder (13%), and disruptive behavior disorder (11%).

Family Circumstances and History

Caregivers for PK youth were primarily single biological mothers (57%), with only 17% of PK youth living in two-parent households, according to caregivers (n = 30; one caregiver participant did not complete a baseline interview) at study baseline. Median annual household income for PK youth at intake was in the $20,000 – $24,999 range (n = 27; 3 caregivers interviewed at baseline did not provide this information). Caregiver (n = 30) reports on the family and child history of PK youth at baseline are detailed in Table 4.

Table 4.

Family and Youth History at Baseline: Project Kealahou (n=30)^a Versus Community Mental Health Initiative (CMHI) Sites^b (n=633)

History of…	Project Kealahou Youth	CMHI Youth
Depression in Family	77%	75%
Substance Abuse in Family	77%	61%
Domestic Violence in Family	63%	44%
Runaway Youth	60%	46%
Substance Abuse by Youth	53%	27%
Household Member Convicted of Crime	53%	35%
Survivor of Physical Assault	45%	25%
Survivor of Sexual Assault	35%	23%
Suicide Attempt by Youth	30%	28%

^aAccording to caregiver (n=30) responses to the Caregiver Information Questionnaire (CIQ-RC-I).¹⁰

^bIncludes data only from Community Mental Health Initiative sites serving female youth ages 11–21.¹⁸

Legal and School Problems

At intake, PK youth (n=69) reported extensive juvenile justice involvement, with 64% having a history of arrest and 44% having been convicted of a crime. PK caregivers (n=30) at intake reported that 33% of PK youth had Individualized Educational Plans, 48% were in Special Education classes, and 26% had been suspended from school in the past six months.

Outcomes at 6-Month Follow-up

Follow-up results, though preliminary, show significant improvement from baseline to six-month follow-up on measures of youth strengths (P = .024), competence (P = .027), depression (P = .009), impairment (P = .007), behavioral problems (P = .017), emotional problems (P = .007), as well as caregiver strain (P = .001), as seen in Table 5. These results are based on paired T-tests of data gathered from PK youth (n = 28) and caregivers (n = 16) who completed both baseline and 6-month follow-up interviews during the first two-and-a- half years of PK services.

Table 5.

Clinical Outcomes for Project Kealahou^a Participants at 6-Month Follow-up

Domain	Measure	n	Source	Baseline Score	Follow-Up Score	Outcome	Significance
Caregiver Strain	CGSQ	16	Caregiver	10.62	8.41	Improved	P = .001
Youth Impairment	CIS	16	Caregiver	29.19	21.56	Improved	P = .007
Youth Emotional Problems	CBCL (6-18)	16	Caregiver	71.63	64.38	Improved	P = .007
Youth Depression	RADS-2	28	Youth	55.46	52.14	Improved	P = .009
Youth Behavioral Problems	CBCL (6-18)	16	Caregiver	73.44	67.75	Improved	P = .017
Youth Strengths	BERS-2Y	28	Youth	86.29	92.36	Improved	P = .01
Youth Competence	CBCL (6-18)	16	Caregiver	30.69	34.88	Improved	P = .027
Youth Strengths	BERS-2C	16	Caregiver	73.19	77.88	Improved	P = .041
Youth Anxiety	RCMAS-2	26	Youth	55.31	54.12	Stable	P = .488

^aOutcomes reported for participants who completed both intake and 6-month follow-up interviews.

PK also received high marks from youth (n = 29) and caregivers (n = 17) who completed a six-month follow-up interview regarding their satisfaction with key aspects of PK. See Table 6 for details.

Table 6.

Caregiver (n=17) and Youth (n=29) Perspectives on Project Kealahou^a

Satisfied with…	Project Kealahou Caregivers	Youth
Access to Services	82%	75%
Participation in Treatment	88%	76%
Cultural Sensitivity	100%	90%
Satisfaction with Services	71%	83%
Outcomes	71%	79%
Functioning	77%	79%
Social Connectedness	82%	86%

^aThis table shows the percentage of respondents reporting positively (ie, a rating of 3.5 or more out of 5 possible points, with “5” representing that they “strongly agree”).

Services and Cost Study

In addition to its ongoing longitudinal outcome study, PK recently completed a study comparing the types and costs of services PK girls and their families received before and during PK. The Service Use and Cost Study is designed to examine PK's service usage pattern and its costs of services in comparison to standard care in the public mental health system. Findings show that overall, PK youth and their families enrolled in the evaluation as of September 2013 (n = 72) received more services (1,819 service events) during their first six months of enrollment in PK compared to the six months prior to enrollment in PK (1,680 service events).²⁰ For those participants for whom cost data was also available both before and after the onset of PK services (n = 41), the total cost for mental health services for the cohort during the first six months of PK enrollment ($365,803) was, however, only slightly higher ($21,662 more) than the total cost of mental health services for PK girls in standard care for the 6 months prior to PK enrollment ($344,141). Thus, the cost per service event was lower for PK ($201) compared to standard care ($205).²⁰ Furthermore, these figures do not account for expected cost savings from decreased service usage and costs for PK girls in juvenile justice, child welfare, and educational settings, which have yet to be determined.

Whereas the frequency of most service types (ie, community therapeutic, psychiatric inpatient, and residential treatment) did not change substantially after the shift from standard care to PK-enhanced care, the frequency of one service type did increase substantially. While enrolled in PK, there was a substantial increase in the level of community support services (eg, peer support for youth and caregivers). Specifically, only 2.4% of PK girls received community support services in the six months prior to enrolling in PK compared to 68.3% of PK girls receiving this service during the first six months of enrollment in PK.¹⁹ Moreover, PK accomplished this sizable increase in community support services for a negligible cost increase of $5,490 (over a 6-month time frame) compared to the cost of the community supports that were provided to the same 41 PK participants in CAMHD standard care for the six months prior to enrolling in PK. ¹⁹

Discussion

Analysis of data from the first two-and-a-half years of PK's ongoing services implementation shows that PK serves a diverse cohort of young women with serious personal, family, educational, and legal challenges that necessitate intensive and/or specialized services from multiple state agencies concurrently. Further analysis of outcome data demonstrates that PK youth and their caregivers achieved significant improvements in key functional, mood, and social domains. PK also ranks in the top 25% of all currently funded CMHI SOC sites nationwide on participant satisfaction with cultural sensitivity of services, quality of services, participation in services and overall satisfaction with services, and in the top 50% of CMHI sites on satisfaction with access to services and outcomes.²⁰ Moreover, the analysis of service usage and cost data shows that, despite vastly expanded access to community support services in the form of peer support for PK youth and caregivers, the overall cost increase to enhance CAMHD's standard care with PK services appears to be small.

Final results after the completion of PK in October 2015 are expected to confirm the findings to date of significant improvements for PK youth and caregivers. This expectation is based on significant positive outcomes seen as well as findings from other CMHI-funded SOCs nationally. The CMHI has shown that after completing such programs (ie, after 18–24 months of treatment), youth similar to those enrolled in PK evidence significant improvements in behavioral and emotional functioning, school performance, suicidal ideation/attempts, and juvenile justice involvement, all of which have been shown to lead to decreased costs for the SOC.⁴ In particular, given the established link between service involvement of youth and positive outcomes discussed earlier, ⁵ an especially promising finding from this preliminary evaluation of PK is the high rating by PK youth of their satisfaction with their level of participation in PK services. These findings are consistent with PK's emphasis on intensively engaging and involving participants in all aspects of services.

Strengths and Weaknesses of the Current Study

This evaluation benefits from participation in the larger CMHI study cohort, which offers a depth and breadth of data from a battery of standardized measures administered to thousands of participants nationwide. While there is no evidence of systematic selection bias into this evaluation, a weakness of this study is the relatively low proportion of participants—28 of 69 youth and 16 of 31 caregivers—who completed both baseline and six-month follow-up interviews. Overall, however, the attrition rate is low at only 7%. Youth who did not complete a six-month follow-up interview may still participate in 12-, 18- and 24-month follow-ups, which would allow for more in-depth analyses as well as analyses of longer-term outcomes among this cohort.

Another weakness of this study lies in its inability to determine whether and which elements of PK services are responsible for its successful outcomes. Further research is needed to better understand the impact of PK's unique components, with particular attention to the apparent cost-effectiveness of community supports provided by paraprofessional staff (ie, peer partners for youth and caregivers). In the future, a larger study cohort and the inclusion of a comparison group of matched CAMHD-only youth who are administered the same battery of tests at the same frequency as PK youth, could help distinguish any impact of PK above and beyond the impact of CAMHD standard care alone. Also, in the future, access to the national CMHI database could allow for direct comparisons of results from Hawai‘i and comparable mainland programs.

Lessons Learned

The apparent effectiveness of PK's model of intensive and peer-delivered community-based supports is likely to be the key lesson learned from this innovative service model implementation for at-risk youth in Hawai‘i. As such, the task ahead for the state's public mental health administration would be twofold: dissemination and sustainability. Regarding dissemination, the task for CAMHD is to translate this successful project into sustainable practices in its state-operated and/or contracted services. One particular challenge CAMHD faces in this regard is that peer support services for youth, though a key element of PK and other SOCs, are a new and untested practice throughout CAMHD standard care. Uptake of this element by CAMHD standard care will require substantial administrative support in the form of planning, resource allocation, and workforce development. The ultimate challenge for CAMHD, of course, will involve leveraging PK's current funding and past successes in order to achieve sustainability and possible statewide expansion once federal funding ends in October 2015.

Conclusion

Significant clinical improvements, high satisfaction levels among PK participants, and a relatively low cost for services offer support for PK's SOC enhancements for effectively serving youth with complex needs spanning multiple state agencies. By the conclusion of the evaluation, more outcome data derived from 6-, 12-, 18- and 24-month follow-up interviews will be available to help stakeholders and funders make data-based decisions about how to best serve this challenging at-risk population. At that time, the primary tasks for the State of Hawai‘i will be to sustain, integrate, and possibly expand on the system of care enhancements achieved by PK.

Conflict of Interest

None of the authors identify a conflict of interest.