“Real-world” first-episode psychosis care in Massachusetts: Lessons learned from a pilot implementation of harmonized data collection

Abstract

Aim:

Increasing evidence points to the value of coordinated specialty care (CSC) for early intervention in psychotic disorders. This report characterizes clinical and socio-demographic features of patients at CSC programs in Massachusetts (MA), assessed by a standardized battery incorporated into “real-world” clinical care.

Methods:

The MA psychosis network for early treatment developed a pilot battery to coordinate assessments across six CSC clinics. Programs reported baseline, 6-month, and 12-month data from a sample of 287 patients with intake dates ranging from April 2015 to December 2020.

Results:

Patients showed improvements in functioning, emergency service use and several symptom domains at 6 and 12 months. Missing data proved to be a limitation.

Conclusions:

Patients improved on several meaningful domains within the first year of CSC treatment. Future implementation efforts in cross-program data collection should consider strategies to circumvent limitations related to heterogeneity between clinics, patient discharge and clinics' capacity for data collection.

1 ∣. INTRODUCTION

Early intervention and treatment for psychotic disorders remains a critical goal across the United States and beyond. For individuals experiencing early-course psychosis, meta-analyses have shown that coordinated specialty care (CSC) models consisting of multi-element, team-based and evidence-based approaches including medication management, psychotherapy, family education and vocational rehabilitation are more effective at improving a variety of clinical outcomes compared to so-called treatment as usual (Correll et al., 2018).

Beginning in 2015, the United States federal government required funds to be set aside in each state for first episode psychosis (FEP) specialty care. The department of mental health (DMH) in Massachusetts (MA) was at this time funding two treatment programs for prevention & recovery in early psychosis (PREP) in Boston and Holyoke, respectively titled PREP East and PREP West. The DMH used these additional set-aside funds to develop a FEP technical assistance centre, the Massachusetts psychosis network for early treatment ("MAPNET"), and to support private hospital- and agency-based CSC programs through temporary grants. These temporary grants helped to offset program costs that would not be reimbursed by insurers such as staff training, team meetings and employment and education coaching.

A core clinical assessment battery was developed by the author group and piloted at the PREP East program in 2015–2017. This report describes the implementation effort of both this pilot initiative and subsequent expansion of the data collection initiative from the PREP East program to an cohort of community-based clinics that received FEP block grant funding in MA from 2017–2020. With recent efforts for the establishment of a national consortium of FEP services reporting harmonized outcomes in the United States (i.e., “EPINET”), we believe that the results of this pilot can inform the national implementation strategy. The analyses reported here aimed to characterize the clinical and socio-demographic features of “real-world” FEP patients and their outcomes, as assessed by a standard clinical assessment battery designed to be incorporated into regular clinical work in both academic and non-academic settings.

2 ∣. METHODS

2.1 ∣. Setting and procedures

Clinics included in the present analysis are PREP West, PREP East, Community HealthLink's screening and treatment of early psychosis, the collaborative pathway at advocates, Cambridge health alliance's recovery in shared experiences, and MA general hospital's first episode and early psychosis program.

Assessment procedures varied between programs based on available resources. At some sites, data were collected by a patient's primary clinician, whereas others relied upon clinical trainees or administrative staff. Several clinics also relied on medical record review for diagnostic and demographic information, whereas others had clinicians record diagnosis directly. De-identified data were submitted to MAPNET and the DMH annually by all programs. This project was integrated as part of a statewide quality improvement project directed by the MA DMH. Approval for secondary use of MAPNET's de-identified clinical data for research purposes, including the analyses in this paper, was obtained by McLean Hospital's LEAP centre from the Mass General Brigham Institutional Review Board.

2.2 ∣. Measures and analysis

This assessment battery underwent pilot and feasibility testing at PREP East from 2015 to 2017. During this period, assessments were administered by clinical trainees (master's and doctoral psychology students) and supervised by a licensed psychologist. Training in the battery was provided to all sites by a clinical psychologist and took about 3 h; administration varied from 20–40 min. All clinics incorporated this assessment into routine clinical care by administering the battery to patients within 1 month of admission and at six-month intervals following. All programs offered CSC services including psychotherapy, medication management, vocational counselling and family education.

The battery is comprised of clinician-rated scales and client-rated self-report measures. Information on demographics, housing, employment, education, service use and substance use was collected through questionnaires administered verbally by clinicians. Outcomes reported here include service utilization, symptom measures as rated on a modified version of the clinician-rated dimensions of psychopathology (Diagnostic and Statistical Manual of Mental Disorders, 2013, p. 5), the social functioning scale (Birchwood et al., 1990) and the Lehman's quality of life scale (Melle et al., 2005).

To assess the significance of changes in the demographic composition of the sample at the 6- and 12-month assessments, we performed two-tailed independent samples t-tests and chi-square tests of independence. Changes in intra-individual functioning, service use and clinical measures were assessed using two-tailed paired-samples t-tests and McNemar chi-square tests. Odds ratios and Cohen's d effect sizes were calculated to assess the magnitude of change from baseline to 6- and 12-months.

3 ∣. RESULTS

3.1 ∣. Program and sample characteristics

A total of 287 patients were included in the present analysis. Programs reported data for between 25 and 73 patients each with intake dates ranging from April 2015 to December 2020. The majority of this sample was male, white and non-Hispanic or Latino, with an average age of 22.73 (see Table 1). Demographic characteristics varied greatly between programs. Twelve-month follow-ups captured a significantly smaller proportion of data from Hispanic or Latino patients and a larger proportion of data from white patients (Table 2).

TABLE 1.

Characteristics of Massachusetts first episode psychosis programs (2015–2020)

Program	Insurance accepted	Ages served	Onset of psychosis	Total (n)	Age (m)	Male (%)	White (%)	Black (%)	Asian (%)	Multiracial (%)	Other race (%)	Hispanic or Latino (%)
Program 1	Masshealth, medicare and commercial insurance	18–40	<3 years	55	26.75	60.8%	64.7%	13.7%	15.7%	9.8%	5.9%	9.8%
Program 2	All forms of insurance	16–30	<3 years	46	22.02	61.4%	22.7%	52.3%	9.1%	18.2%	13.6%	20.5%
Program 3	All forms of insurance	16–30	<3 years	73	23.07	59.0%	50.0%	29.4%	11.8%	0.0%	0.0%	23.5%
Program 4	Masshealth, medicare, commercial insurance and managed care plans	13–35	<3 years	36	20.90	72.2%	58.3%	19.4%	8.3%	2.8%	5.6%	5.6%
Program 5	Masshealth, medicare, health safety net and commercial insurance	15–30	No limits	52	22.27	62.7%	21.7%	39.1%	10.9%	2.2%	8.7%	17.4%
Program 6	Masshealth, medicare and commercial insurance	14–35	No limits	25	23.68	56.0%	88.0%	4.0%	0.0%	0.0%	4.0%	4.0%
Total	—	—	—	287	22.73	62.2%	47.9%	28.0%	10.2%	6.4%	6.8%	14.0%
Missing data, n (%)	—	—	—	—	87 (30.3%)	41 (14.3%)	51 (17.8%)	51 (17.8%)	51 (17.8%)	51 (17.8%)	51 (17.8%)	51 (17.8%)

TABLE 2.

Changes in demographic characteristics, functioning, service use, and symptoms from intake to 6- and 12-month assessments

	Baseline		6 months					12 months
	n a	%/m	n a	%/m	Odds ratio/ Cohen's db	χ2/t c	p valued	n a	%/m	Odds ratio/ Cohen's db	χ2/t c	p valued
Total sample (n = 287)	256	89%	102	36%				59	21%
Demographics
Age (m)	199	22.7	58	22.7	d = 0.01	0.10	.92	37	23.3	d = −0.12	−0.78	.44
Male (%)	239	63%	71	63%	OR = 1.00	2.49	.29	43	74%	OR = 1.18	3.67	.16
White (%)	229	49%	67	58%	OR = 1.19	3.18	.07	42	64%	OR = 1.31	4.37	.04e
Black (%)	229	28%	67	28%	OR = 1.03	<0.001	1.00	42	21%	OR = 0.78	0.71	.40
Asian (%)	229	10%	67	3%	OR = 0.30	4.44	.04e	42	10%	OR = 0.95	<0.001	1.00
Hispanic/Latino (%)	229	14%	67	7%	OR = 0.53	2.79	.09	42	2%	OR = 0.17	4.47	.03e
Employment, education and legal (%)
Paid work (past month)	244	39%	89	44%	OR = 1.11	1.14	.29	55	47%	OR = 1.20	6.67	.01f
Enrolled in school (past month)	221	28%	90	36%	OR = 1.29	0.00	1.00	56	48%	OR = 1.75	0.56	.45
Taking at least one college, high school, technical, or GED class	221	25%	90	29%	OR = 1.14	0.44	.51	56	46%	OR = 1.83	1.07	.30
Living independently	221	10%	90	11%	OR = 1.12	0.00	1.00	56	9%	OR = 0.90	0.00	1.00
Living with family	221	84%	90	83%	OR = 1.00	0.00	1.00	56	88%	OR = 1.05	0.00	1.00
Legal involvement past month	238	13%	77	10%	OR = 0.78	0.00	1.00	56	9%	OR = 0.68	0.00	1.00
Service use (% past 6 months)
Emergency department (Psych)	239	59%	93	27%	OR = 0.46	12.90	<.001f	55	31%	OR = 0.52	6.86	.009f
Hospitalization	86	38%	40	13%	OR = 0.33	4.17	.04e	32	19%	OR = 0.49	0.57	.45
Clinical measures (m)
Hallucinations	245	0.96	90	0.89	d = 0.05	0.10	.92	53	0.77	d = 0.15	0.12	.90
Delusions	243	1.14	90	0.81	d = 0.27	2.61	.01e	54	0.70	d = 0.35	0.89	.38
Negative symptoms	243	1.43	90	1.13	d = 0.22	2.05	.04e	54	1.06	d = 0.29	2.36	.02e
Depression	244	1.53	90	1.37	d = 0.13	1.64	.10	53	1.60	d = −0.06	0.20	.84
Social functioning	240	5.69	84	6.31	d = −0.36	−2.33	.02e	51	6.27	d = −0.31	−1.07	.29
Quality of life	69	4.23	35	4.60	d = −0.26	−2.18	.04e	25	4.56	d = −0.26	−2.39	.05e

^aNumber of observations available for each measure at the indicated time point.

^bOdds ratio (for nominal measures) or effect size of change (for linear measures) compared to baseline assessment.

^cChi-square statistic (for nominal measures; test of Independence for demographics and McNemar for outcome measures) or t statistic (for linear measures; independent samples for demographics & paired-samples for outcome measures) compared to baseline assessment.

^dp value of chi-square test (for nominal measures) or paired samples t-test (for linear measures).

^ep < .05.

^fp < .01.

Fifty-two percent of patients (n = 121) had a primary diagnosis of schizophrenia, schizoaffective disorder, or schizophreniform. Thirty percent (n = 70) had a diagnosis of an unspecified or other psychotic disorder, 12% (n = 29) had an affective disorder diagnosis such as bipolar disorder or depression with psychotic features and 6% (n = 13) had a primary diagnosis of a non-psychotic disorder. The most common comorbidity was substance use disorder (n = 54, 23.2%), followed by anxiety disorders (n = 20, 8.6%), developmental or learning disabilities (n = 15, 6.4%) and trauma-related disorders (n = 15, 6.4%).

3.2 ∣. Change from baseline to 6- and 12-months

Table 2 identifies measures of functioning, service use and symptom severity at the baseline, 6-month, and 12-month assessments. Patients showed overall improvement in functioning, with a 20% increase in employment at the 12-month assessment (Table 2). Notable reductions in emergency service use were observed at both the 6- and 12-month assessments, including a 54% decrease in emergency department visits and a 67% decrease in hospitalizations at 6 months. Patients saw significant improvements in delusions, negative symptoms, social functioning and self-reported quality of life at 6 months, of which improvements in negative symptoms and quality of life remained significant at the 12-month assessment. Hallucinations and depression did not show significant changes over time. Scores and effect sizes for all domains are presented in Table 2.

4 ∣. DISCUSSION

Our findings illuminate both the potential successes and limitations of a real-world implementation strategy for coordinated assessments in a loose network of independent clinical programs. Patients treated in CSC programs saw clear benefits within the first 6–12 months of treatment on several meaningful domains. Without a control group, however, it is impossible to determine whether CSC treatment was responsible for these improvements, or to measure the extent to which outcomes might compare to untreated illness.

A notable outcome of this naturalistic approach to data collection is the significant attrition in patient outcomes data between baseline and follow-up assessments. Many factors may explain this attrition; some patients experience rapid improvement and discontinue CSC within 6 months, whereas others drop out without clinical improvement. Patients may also find the assessments to be burdensome or irrelevant to their treatment and thus decline to participate in follow-up assessments. Using patient stakeholder input to guide the battery selection might be a good strategy to enhance patient engagement in the assessment process. Missing data was a limitation for various clinical measures at all-time points, especially for self-report measures such as the quality of life scale. Programs reported that administering self-report scales was more challenging than completing the clinician-oriented measures, especially for patients seen remotely via teleconference. Clinics also recorded lower rates of secondary diagnoses than expected, likely due to a reliance on chart review related to a limited ability to prioritize formal assessment of comorbidities at intake. Additional barriers exist in clinics' capacity to administer follow-up assessments due to staff turnover or competing demands, particularly within clinics serving patients with greater risk or clinical complexity. Future iterations of this assessment will ask clinics to report the status of each enrolled patient (enrolled, discharged and/or declined assessment), a strategy we recommend to national stakeholders to increase the interpretability of outcomes data.

In MA, the majority of clinics serve individuals aged 15–30, and include a range of psychosis diagnoses with an onset within 3 years. This is similar to reports from clinics in California and New York, though global differences exist in program eligibility criteria (Anderson et al., 2019; Niendam et al., 2019). With the expansion of early psychosis services and standardization of assessments to evaluate their impact, it is important to be aware of these inconsistencies; for example, the inclusion of affective or at-risk patients, differing age ranges and varying insurance statuses of patients and healthcare contexts across countries. Beyond eligibility criteria, additional differences exist in the widely varying demographic characteristics of patients served. This heterogeneity may hamper the harmonization of data collection efforts and limit the interpretability of these data.

The next phase of this project will expand the focus of FEP service development to community mental health centres in underserved areas of the state. Our experience in this endeavour recently led to a strategic review of early intervention efforts in the state outlining several actionable next steps (Johnson et al., 2021). The MA DMH has awarded a new round of program grants to such centres with a goal of establishing new FEP clinics, differing from the initial strategy of expanding existing programs in the cohort described here. As new programs become established, we will also monitor the sustainability of this first cohort; in particular, assessment data will describe services provided, system-wide outcomes, and the extent to which clinics continue to serve the FEP population. Lastly, many of these clinics have joined the U.S. National Institute of Mental Health's “EPINET” initiative (https://nationalepinet.org/) to establish national standards for data collection in clinical outcomes for CSC. As the field moves forward, we hope that these pilot data can serve as a starting point for national improvements to this vital method of care.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.