Understanding the implementation of coordinated specialty Care for Early Psychosis in New York state: A guide using the RE-AIM framework

Abstract

Aim:

Consistent evidence shows that early interventions for individuals with psychosis lead to improvements in symptoms, social functioning and treatment satisfaction. These results, combined with the allocation of specific funds for early psychosis services, have contributed to the emergence and dissemination of coordinated specialty care for early psychosis in the United States. Despite the rapid growth of such services across the country over the last 5 years, implementation processes are not yet well understood. We employ the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework to describe processes, achievements and challenges of an early psychosis program called OnTrackNY that has been implemented in New York State.

Methods:

OnTrackNY is a coordinated specialty care program that delivers early intervention services that include both medications and psychosocial interventions to youths experiencing a first episode of non-affective psychosis. By drawing on outcome and care process data that are collected quarterly from all OnTrackNY sites, we describe the status of each RE-AIM dimension regarding OnTrackNY implementation followed by an evaluation of both achievements and shortcomings.

Results:

In general terms, OnTrackNY has shown to be a scalable and sustainable model for addressing early psychosis, reaching and providing recovery-oriented services to a large population in need.

Conclusion:

Despite its advancements, a series of limitations pose challenges to the implementation and maintenance of the model including, but not to, the lack of incentives for coordination of services, the fragmentation of child and adult services, and concerns about financial sustainability.

1 |. INTRODUCTION

A population-based approach to delivering early intervention for individuals with psychosis has become standard in many high-income countries (HICs) such as England, Australia, Denmark and Singapore (Csillag et al., 2017). Numerous well-controlled international studies tested early psychosis programs that provided evidence-based psychosocial interventions and medication, establishing that such approaches lead to improvements in symptoms, social functioning, treatment engagement, quality of life and treatment satisfaction (Correll et al., 2018; Dixon, Goldman, Srihari, & Kane, 2018). In the United States, the National Institute of Mental Health (NIMH) funded Recovery After an Initial Schizophrenia Episode (RAISE) studies developed and tested a multi-element team-based approach to early psychosis, now called coordinated specialty care (CSC) (Dixon et al., 2015; Kane et al., 2016). The evidence for the short-term benefits of these types of programs is now well established (Correll et al., 2018).

These promising results, combined with the decision by the U.S. Congress to allocate funds within the community mental health block to support early intervention services for psychosis, laid the groundwork to scale-up CSC programs throughout the U.S. In New York State (NYS), a CSC program called “OnTrackNY” was launched in 2013. As of 2018, there are 12 OnTrackNY sites in New York City and nine sites elsewhere in NYS, each serving 35 to 75 individuals. OnTrackNY team members receive initial training and ongoing technical assistance (TA) by OnTrackCentral at the Center for Practice Innovations, a training and TA centre at the New York State Psychiatric Institute (NYSPI) which is funded by the New York State Office of Mental Health (OMH). The OnTrackNY model is fully described elsewhere (Bello et al., 2017), and intervention manuals and additional materials are available at www.ontrackny.org/resources.

Despite the rapid growth of CSC across the United States over the last 5 years, the process of implementation of CSC is not yet well understood and characterized. Several features of the U.S. healthcare system such as limited federal oversight, state-specific regulatory standards, limited incentives for coordination of services, fragmentation of child and adult services, and concerns about financial sustainability, have posed challenges to widespread implementation and maintenance of the CSC model (Dixon, 2017). These limitations have also been reported in both HIC’s and middle-income countries when implementing early psychosis programs (Csillag et al., 2017). Without understanding and addressing these challenges, the deployment, widespread and ultimate sustainment of these services will be in jeopardy.

We use the RE-AIM (Reach, Effectiveness, Adoption, Implementation and Maintenance) framework (Glasgow, Vogt, & Boles, 1999) to describe processes, achievements and challenges faced in the state-wide implementation of OnTrackNY in NYS. We draw on outcome and care process data that are collected quarterly for each participant for quality improvement, effectiveness evaluation and fidelity monitoring. Data are collected at each site using standardized admission, follow-up and discharge forms, which are completed by clinicians through chart review and reports from participants and their families (Nossel et al., 2018a,b). We also obtain data from surveys in which providers assess the quality of OnTrackCentral training and technical assistance.

2 |. THE RE-AIM FRAMEWORK

The RE-AIM framework (Glasgow et al., 1999) is a useful model to assist with planning, evaluation, and reporting of research conducted in real-world complex settings. We chose RE-AIM due to its flexibility to describe both implementation and dissemination processes, as well as intervention design and evaluation, identifying challenges that need to be understood and addressed (Gaglio, Shoup, & Glasgow, 2013).

Each letter of the model represents one of the following dimensions: Reach, Effectiveness, Adoption, Implementation and Maintenance. Reach refers to the number or proportion of eligible individuals who are enrolled in a health program. Effectiveness is the impact of the program on outcomes at the individual level. Adoption is the number of institutions and/or health staff that are willing and able to deliver the program. Implementation refers to program fidelity, delivering the program as intended. Finally, Maintenance is defined at both the institutional (extent to which the program has become institutionalized)- and individual levels (long-term effects of a program on outcomes) (Glasgow et al., 1999).

For each RE-AIM dimension, we summarize the status of OnTrackNY deployment. We then briefly evaluate the performance of OnTrackNY for that dimension. We expect that identifying the accomplishments and deficits of the program implementation to date will contribute to understanding how to improve CSC implementation efforts in other locales.

2.1 |. Reach

We interpret Reach as the number of eligible people who are enrolled in OnTrackNY and the extent to which the program is serving the population in need. Eligible individuals are NYS residents between the ages of 16 to 30 with a primary non-affective psychotic disorder, and duration of psychosis of at least a week but less than 2 years. Individuals with the following conditions are excluded: Intellectual Disability (IQ < 70) or Autism Spectrum Disorder; primary diagnosis of substance-induced psychosis, psychotic mood disorder, or psychosis secondary to a general medical condition; and serious or chronic medical illness significantly impairing functioning independent of psychosis. Recruitment is led by an Outreach and Recruitment Coordinator (ORC), a licensed master’s or doctoral-level clinician, who coordinates and conducts outreach and recruitment activities including, but not limited to, the development of relationships with potential referrers, the provision of informational presentations about early psychosis and OnTrackNY and the use of promotional materials (eg, brochures, website, social media, recovery videos and newsletters). The ORC explains the program to referred individuals and families, completes an initial phone screening and then conducts an eligibility evaluation including clinical interview, collateral information and record review when needed.

From October 2013 to 2018, 1215 individuals were enrolled. The average time from the onset of psychosis to enrolment was 7.7 (median = 5.5) months. About 25% (N = 1157) of those referred (N = 4570) were eligible. Most individuals were referred from psychiatric inpatient units (43%), outpatient mental health providers (25%), by themselves or family members (19%) or from other settings (13%), including the school system, the legal system, and community organizations. Table 1 shows the demographic characteristics of enrolled individuals at admission.

TABLE 1.

Demographic characteristics at admission (n = 1215)

Characteristic	N	%
Age (years)
<18	158	13
18–24	863	71
25–30	182	15
>30	0	0
Gender
Female	899	74
Male	316	26
Race-ethnicity
Non-Hispanic white	316	26
Non-Hispanic black	425	35
Hispanic	316	26
Asian	97	8
Multiracial	24	2
Education
Enrolled in school	365	30
Not enrolled in school	851	70
Employment
Employed	194	16
Not employed	1021	84
Living situation
Lives alone	49	4
Lives with parent(s)	1001	83
Lives with other(s)	146	12
Insurance status
Uninsured	49	4
Public	644	53
Private	437	36
Other	73	6
Missing	37	3

2.1.1 |. Evaluation

The program’s policy of serving individuals who have access to services irrespective of income or health insurance enhances its “reach.” It is actually a requirement of receiving funding from the state that OnTrackNY sites enrol individuals regardless of their ability to pay for services. Examination of the racial and ethnic composition of the population served indicates that OnTrackNY includes diverse communities. However, the preponderance of referrals from psychiatric inpatient units reflects some challenges in reaching into the community before a first hospitalization. Further, the lack of available information or population registries limits the assessment of the extent to which OnTrackNY is reaching its target population. Published “treated incidence” estimates of non-affective psychosis range from 15 to 30 per 100 000 person-years (Kirkbride et al., 2012). Given the population of NYS (19.8 million), we would expect at least 3960 new cases per year (Humensky, Dixon, & Essock, 2013). With a total of 920 program slots at present, it is clear that the reach of OnTrackNY remains extremely limited. Modest increases in program capacity that are planned will not fix this problem. Truly serving the population, therefore, will require a more precise estimate of the incidence and a more robust integration of the program model into the care system across payors.

Additionally, the time from onset of psychosis to OnTrackNY enrolment, while shorter than some research studies conducted in the United States (Addington et al., 2015; Srihari et al., 2015), in part due to differences in eligibility criteria, is still long (almost 8 months) when compared to the international consensus that states early psychosis programs should start within 3 months of illness onset (Bartolome & McGorry, 2005). Minimizing the duration of untreated psychosis (DUP) is essential for achieving better outcomes (Oliver et al., 2018).

2.2 |. Effectiveness

Effectiveness refers to clinical, occupational and social outcomes. OnTrackNY clinicians provide data including demographic characteristics, diagnosis, substance use, antipsychotic use, hospitalization and suicidal ideation or behaviour. Clinicians also assess participants with the MIRECC Global Assessment of Functioning (GAF) symptom, occupational and social functioning scales (Niv, Cohen, Sullivan, et al., 2007).

We recently reported promising results on rates of hospitalization and enrolment in school and employment (Nossel et al., 2018a,b). Briefly, following enrollment in OnTrackNY, education/employment rates increased from 40% to 80% by 6 months, hospitalization rates decreased from 70% to 10% by 6 months, and improvement on Global Assessment of Functioning scores continued to 12 months (n = 325). Data are updated continuously; outcomes for an expanded sample of 634 are consistent with our published report (Nossel et al., 2018a,b).

2.2.1 |. Evaluation

The improved rates on education and employment reported above are at least comparable to those reported in other CSC studies (Nossel et al., 2018a,b). Reduced rates of hospitalization are also promising, though these are relatively expected given the high rates of recent hospitalization at enrolment. The lack of a comparison group, however, limits our ability to make causal inferences. Furthermore, we do not have data for those discharged from the program, and therefore we can only measure outcomes on those who remain enrolled. The use of administrative data and Medicaid claims may permit comparisons to individuals who do not receive OnTrackNY or receive other services.

2.3 |. Adoption

We understand adoption as the characteristics of OnTrackNY sites and the degree to which their clinicians are willing and able to deliver the program. OMH invites sites to participate in OnTrackNY based on geography and: (a) experience providing care to adolescents and young adults with early psychosis; (b) ability to do community outreach to identify individuals experiencing FEP; (c) a strong recovery orientation; (d) youth friendliness; (e) access to or relationship with an inpatient hospital; (f ) strong psychiatric supervision and clinical leadership to work with high-risk population and (g) ability to provide 24 h/d and 7 d/wk contact for individuals and families experiencing a crisis. By design, sites include rural, urban and suburban areas as well as outpatient clinics at academic and community hospitals, state-operated outpatient clinics, mental health agencies, and a federally qualified health centre.

Regular surveys have evaluated training and TA activities to assess whether team staff members are willing and able to employ OnTrackNY principles in their daily practices. In a recent evaluation (n = 68), most clinicians agreed (62%) or strongly agreed (36%) with the statement: “I feel comfortable implementing the practices/approach covered in OnTrackNY training.” Clinicians also reported agreeing (49%) or strongly agreeing (46%) with the item: “I feel I am able to implement my role(s) within the OnTrackNY model successfully to help participants and families.” Finally, providers found trainers “skillful and knowledgeable,” reported high satisfaction with “the quality of the training and technical assistance,” and most agreed or strongly agreed that “the material covered was useful to me for implementing my role(s) on the OnTrackNY team.”

2.3.1 |. Evaluation

The invitation process has been mostly successful in identifying sites that support the delivery of the model. However, the OnTrack model requirements have often pushed sites to the limits of flexibility and intensity (eg, service in the community, availability during crises). Frequent clinician turnover and lack of available qualified staff have been a challenge as well, requiring one site to close. Further, while urban sites have been relatively easy to identify and develop, implementing programs in rural and suburban areas has been more difficult. One approach has expanded inclusion criteria in order to leverage the resources of a clinically strong clinical team located in a less populated area that was struggling to build a full caseload. OnTrackCentral has also worked closely with OMH and county mental health directors in rural areas to explore options for adapting the model for rural areas, including considering the creation of satellite clinics and using telepsychiatry.

It is difficult to interpret the training evaluations because of the risk of desirability bias. However, feedback has largely been positive with clinicians requesting additional training in providing supported education; managing specific symptoms and co-morbid conditions such as obsessive-compulsive disorder, substance abuse and trauma; conducting more effective outreach; addressing metabolic conditions and other medication side effects; and addressing cognitive issues. They have sought increased opportunities for collaborative exchanges among regional providers, and in some cases, fewer supervision calls.

2.4 |. Implementation

This dimension refers to the fidelity of delivering OnTrackNY as planned. A Fidelity Scale has been developed which uses the regularly collected data as well as site visits; it was adapted for OnTrackNY from the First Episode Psychosis Fidelity Scale (FEPS-FS) (Addington et al., 2016). Site visits include observation of a team meeting, interviews with staff members and at least one client and a family member or other support person, and the review of random client charts and program records. A total of 24 domains are evaluated, including staffing, team communication, community outreach, flexibility of services, initial assessment and treatment planning, safety planning, prescribing practices, client preferences and shared decision making, working with families, and supported employment and education. An annual fidelity assessment is planned. Of the six conducted to date, all programs demonstrated high fidelity, meeting 20 to 24 of the 24 domains. The site visit proved to be a useful adjunct to regular summaries of data, particularly for domains related to crucial care processes for the model (eg, shared decision making). Reports were also reviewed with teams and OnTrackCentral trainers to inform future training.

2.4.1 |. Evaluation

On a positive note, the program has created a face valid fidelity scale which was built upon developed anchors and a set of procedures that are well described and reasonably practical and has begun to use it. It has been well received. However, the scale has not yet been demonstrated to be reliable or valid when measured against program outcomes. A larger critical issue for the field is the lack of knowledge regarding critical ingredients of CSC services.

2.5 |. Maintenance

Maintenance is defined at the institutional and individual levels. Institutional maintenance requires a viable long-term financing model. OMH has subsidized OnTrackNY programs with state and earmarked federal dollars from the Community Mental Health block grant. Clinics must bill Medicaid and commercial insurance for reimbursable services, with NYS making up the difference. A recent time study of a subset of teams in 2017 showed that most efforts were spent on providing a combination of mental health services that are billable under a typical Medicaid State Plan, as well as recovery and rehabilitation services including peer, employment, education and family supports, which are reimbursable under a Medicaid Home and Community Based Services waiver in NY State. The teams also spent time providing care management and participating in administrative activities (eg, supervision, case conferences, outreach and evaluation pre-intake), which are not typically reimbursable in public or commercial sectors. Of note, estimated revenue was expected to cover less than half of the teams’ average total costs.

Maintenance at the individual-level considers the long-term impact of OnTrackNY on outcomes and the way individuals are followed after discharge. At present, we do not have data on outcomes following discharge.

2.5.1 |. Evaluation

At the institutional level, the state-wide oversight and commitment to program viability have been essential in program creation and expansion. The use of state, federal and private commercial dollars, as well as some grant funds, have leveraged resources to create a large, vibrant program. Simultaneously, the lack of established and adequate payment mechanisms represents a significant threat to the sustainability of OnTrackNY. This underscores the importance of developing funding rules in both the public and private sectors that can support the cost of CSC programs. Also, we must note that NYS OMH subsidies will likely decline over time, and federal priorities related to Block Grants can also change, potentially jeopardizing the continuity of the program. Discussions with commercial insurers regarding the possibility of a bundled payment rate are preliminary.

The work of OnTrackNY has only minimally addressed maintenance at the individual level to date which could include consideration of the appropriate length of intervention, the provision of follow-up treatment, and the sustainability of treatment effects after discharge. To assess the durability of treatment effects following discharge, we are developing plans to contact participants 3 months following discharge to inquire about functioning; conducting a chart review of discharged clients’ subsequent outpatient records; and analysing claims data to assess service use following discharge.

3 |. CONCLUSION

This summary of our experience deploying OnTrackNY through the prism of the RE-AIM framework has revealed a number of key findings. In general terms, OnTrackNY has shown to be a scalable and sustainable model for addressing early psychosis in NYS, reaching and providing recovery-oriented services to a large population in need. Also, it is one of the first early psychosis programs using routine outcome measurement for quality improvement, effectiveness evaluation and fidelity (Addington et al., 2018), which has allowed us to present outcomes to policy makers, stakeholders and researchers in a straight-forward and appealing manner contributing to the rapid expansion of the program. The review underlines the value of the state-wide commitment to provide consistent financing and the technical assistance and supports needed to deploy this type of program across a range of agencies in different communities.

At the same time, we see the threats and significant limitations that potentially jeopardize the implementation of CSC services in spite of considerable investment. These include the lack of durable financing strategies and support for processes to examine and maintain fidelity. The need for greater availability for services to the population is also apparent. We expect that other CSC programs throughout the country and worldwide may experience similar challenges.

This analysis is limited by the generalizability of our findings, as the degree of state support in NYS (including state subsidies for CSC services and support of a TA centre that collects, analyses, and closely monitors data and provides training and TA) is not available in many states in the United States, and elsewhere in other HICs. Nonetheless, though local circumstances may differ, the RE-AIM framework may be a useful tool for other states and/or countries to plan and evaluate the implementation of CSC services.