Screening for Early Emerging Mental Experiences (SEE ME): Feasibility of psychosis screening in US integrated care

Abstract

Objective:

This study aimed to assess the feasibility of the screening for early emerging mental experiences model, which is designed to screen for psychosis in settings with integrated primary and mental health care.

Methods:

Psychosis screening, triage, and engagement (July 2021–June 2022) were implemented in four integrated care practices serving approximately 7,000 patients in the targeted age range (14–26 years). Practice and community stakeholders participated in the project’s design and development. Psychosis care specialists provided training and case consultation to general medical providers and behavioral health clinicians (BHCs). The BHCs screened all patients referred for selective screening. One practice conducted universal screening of patients (ages 14–26) attending well visits.

Results:

Training sessions were attended by 100% (N=6) of the BHCs and by 79% (N=27 of 34) of the primary care providers. The BHCs selectively screened and triaged 266 patients (89% of their new patients). Providers conducted universal screening of 606 patients (67% of that site’s well visits). The screening samples were >90% White and >55% rural, consistent with the clinics’ populations. Rates of positive selective screens were consistent with published rates in similar populations. Of the recorded screening-related activities, 92% (146 of 159) were completed within the billable intake time, and 11% (N=17) of these patients engaged in treatment. The providers reported that the project was important and positive.

Conclusions:

Systematic assessment of psychosis symptoms, followed by triage and engagement, appeared to be feasible and acceptable to patients and providers in integrated care settings.

INTRODUCTION

Psychosis is widely considered a harbinger and indicator of the most serious mental health conditions.^1,2 Characterized primarily as a disruption of perception and thought, it is often unrecognized for months to years, with devastating short- and long-term consequences.^3–5 Research outside of the U.S. suggests that training, consultation, and screening within primary care settings can improve early identification.^6–9 However, systematic mental health screening, particularly for psychosis and in primary care settings, has had little traction in the U.S.^10,11,12 Indicated screening, that is, screening individuals with atypical or unexplained changes in functioning or behavior, is advocated by early intervention scientists.^13–15 Yet even highly developed programs identify only a small proportion of people who are developing psychotic disorders prior to acute psychosis onset.¹⁶ With the growth of behavioral health integration (BHI, behavioral healthcare imbedded in primary care settings)¹⁷ and mental health screening, it is time to place early intervention in major mental ill-health squarely within the domain of preventive medicine.

We secured funding to develop and test the initial feasibility of an early intervention model, called screening for early emerging mental experiences (SEE ME).¹⁸ This model was designed to leverage the low stigma and general accessibility of primary care settings to improve identification of young people at risk for or in the early stages of a psychotic disorder. SEE ME moves beyond the current outreach and referral model to build capacity, not only for screening, but for triage and initial treatment engagement within integrated care. In this project, our objectives were to identify the volume and demographic characteristics of the patient population, the provider time, effort, and other needs required for implementation, interest in and use of peer support services, best data collection methods, and the initial acceptability of the screening to patients and providers.

This article reports on elements of the project’s development and feasibility. A subsequent article will report on the project’s outcomes (e.g., triage decisions and referrals to specialized care).

METHODS

2.1. Project Overview

This project was implemented in collaboration with the Portland Identification and Early Referral (PIER) program, a coordinated specialty care program for 14–35 year-olds with a recent onset of psychosis symptoms or a psychotic disorder. With foundation and institutional funding, the project was designed to include stakeholder engagement and protocol development (January-June, 2021) and implementation (July, 2021-June, 2022). The funding supported part-time efforts by the project staff, PIER program staff (i.e., a child and adolescent psychiatrist and a master’s-level counselor), primary care physician leads (practice leads were paid a stipend to champion the project among primary care providers [PCPs]), a behavioral health administrator, master’s-level behavioral health clinicians (BHCs), and PIER program peer and parent support specialists who were available remotely. The participating practices received stipends (lunch and salary offsets) to incentivize providers’ attendance at the training sessions. The project was implemented within the context of Behavioral Health Integration managed by Maine Behavioral Healthcare (MBH). Importantly, the project took place within the context of the COVID-19 pandemic and a new Health Resources and Services Administration grant for a state-wide Maine Pediatric and Behavioral Health Partnership (MPBHP) to provide child psychiatry consultation to PCPs. Because the psychiatrist for the PIER program (AMM), MPBHP, and SEE ME was the same person, recognition of psychosis symptoms was emphasized beyond the scope of the SEE ME project. Individualized data were not collected in order to maximize recruitment of a general population sample (and to avoid the need for parental consent). The MaineHealth Institutional Review Board (IRB) determined that the proposal did not constitute research requiring IRB oversight. The possibility of comorbid conditions, including substance use disorders, was assumed, but such assessment was outside the scope of this project.

2.2. Stakeholder Engagement

Input was obtained from community stakeholders via three video conference meetings. Maine’s Offices of Behavioral Health and Child and Family Services and other state-level contacts, as well as representatives from MaineHealth’s Departments of Pediatrics and Psychiatry and the Women and Children Service Line. Stakeholders influenced the scope of the project, coordination with other services, and the focus on individual practices’ needs. Stakeholders with lived experience drove changes in how remote peer support services were introduced by BHC and in the flyers (e.g., removal of medical terms such as disorder and diagnosis and an emphasis on connecting with others with similar experiences).

2.3. Participating Practices

Four MaineHealth practices in southern and western Maine agreed to participate. These practices were selected on the basis of their recognized leadership and capacity and to include pediatric services (three sites), an adolescent and young adult medicine clinic (embedded within one of the pediatric practices), and a family medicine clinic. Together they served roughly 7,000 adolescents and young adults in the target age range (14–26, > 90% White, > 55% rural). The project was integrated into each practice’s usual behavioral health procedures.

2.4. Screening Population

The target age range of 14–26 was selected because it has been described as the peak age range for psychotic symptom onset;^19–21 thus, practices serving adolescent and pediatric populations were chosen to maximize rapid identification of patients. The willingness of the practice sites to engage in selective or universal screening was considered a measure of the project’s feasibility. All of the practices implemented selective screening (i.e., screening patients who were at high risk for psychosis because of an identified mental health concern, as defined by a referral to a BHC). One practice also chose to implement universal screening at all well visits.

2.5. Measures

From an array of available psychosis screening tools, with varying psychometric properties in clinical and general population samples,^13,14,22,23 we chose to use the 16-item Prodromal Questionnaire (PQ-16) for selective screening. The PQ-16 is a short screening tool established for use with clinical populations. Scores on the PQ-16 range from 0 to 105. Higher scores indicate increased risk and are determined via the degree of item-related distress—a factor that is better associated with psychosis and psychosis risk than is item endorsement alone.^22,23

The nine-item Early Psychosis Screener (EPS-9) was selected for universal screening due to its brevity, readability (no higher than a third grade level), simplicity of scoring, and promising psychometric properties.^24–26 Scores on the EPS-9 items range from 1 to 5, with higher scores indicating higher frequency and thus risk. The survey taken by the providers after completion of the project (see the online supplement to this article) was informed by the theory of planned behavior and was designed to assess attitudes, subjective norms, and perceived behavioral control.²⁷

2.6. Procedures

Selective Screening.

Specific screening procedures were developed for each site by the project staff in conjunction with each clinic’s lead physician and BHC. Because of the COVID-19 pandemic, most of the selective screening was conducted during telehealth visits. The PQ-16 screening was completed, along with other mental health screening, at the first BHC visit—via self-report or by reading the screening questions to patients and recording their responses.

Universal Screening.

In conjunction with other general medical and mental health screenings, medical assistants administered the EPS-9 to universally screen patients in the waiting room prior to a well visit. The PCPs were asked to subsequently inquire about each patient’s family mental health history and about all positive screening responses.

Triage.

After a broad assessment of diagnoses, comorbid conditions, biopsychosocial factors, and responses to the screening questions, the BHCs triaged the patients into levels of care. Patients with no psychosis risk factors or symptoms were assigned to usual care, those with complex or ambiguous clinical presentations were assigned to extended assessment, those with familial risk or subthreshold psychosis risk indicators were assigned to usual care plus a plan for monitoring psychosis risk indicators, and those with suspected psychosis or psychosis risk syndromes were referred for specialized psychosis assessment and care.

Engagement.

The BHCs engaged patients selected to receive monitoring or specialized care (and their caregivers, when possible) in a developmentally appropriate and clinically relevant discussion of their experiences, with the goal of successful linkage to relevant care. As part of these discussions, BHCs provided psychoeducation, addressed stigma or other concerns, and/or facilitated a referral to psychiatric or specialized psychosis care.

2.7. Training

Training of the BHCs in selective screening, triage, and engagement. All BHCs had masters in social work. They completed four 4-hour trainings and had the primary role of screening, triage, and engagement. They were asked to attend two consultations per month for the first six months and one per month afterward. Consultation was offered four times per month to accommodate BHCs who worked part-time.

Training of the PCPs in general project support (all sites) and universal screening (one site). We invited all medical providers (PCPs, nurses, physician assistants, and nurse practitioners) to PCP training sessions although the scheduling and content were primarily geared toward PCPs (pediatricians and family medicine practitioners, including doctors of medicine and doctors of osteopathic medicine). Training commitments varied by practice and were considered a measure of feasibility given complex PCP schedules, staffing patterns, populations, and training priorities. They ranged from 10- or 15-minute sessions (in the context of existing staff meetings) to 50-minute lunchtime sessions. Multiple training times were offered to accommodate part-time schedules and vacations. Of the general medical providers, only PCPs from one practice were trained to universally screen patients for psychosis and to refer patients who screened positive to a BHC for triage and treatment engagement.

Contribution of individuals with lived experience. Individuals with lived experience had an important role in training, raising awareness of differing perspectives, values, and language preferences. Some identified medical language (e.g., deficits, illness, disorder, diagnoses) as hurtful or diminishing and discussed the impact of being dismissed, belittled, or traumatized by mental health and general medical providers. These individuals advocated for greater respect for patients’ autonomy and for greater social justice in mental health care. Others (particularly parents) identified medical labels and education as useful for connecting with relevant communities and resources and for making sense of baffling experiences and observations.

Topics covered by the training sessions are described in the online supplement. The training objectives were to increase or improve the BHCs’ and PCPs’ knowledge of the phenomenology and epidemiology of experiences along the psychosis spectrum, familiarity with people with firsthand experience of psychosis, comfort in asking initial and follow-up assessment questions, familiarity with online and printed resources, and comfort in engaging youths and families who might be reluctant to access mental health care.

2.8. Peer Support Services Offered

Remote peer support by a youth and a parent support partner trained in the the intentional peer support and family peer support models ²⁷ was offered, as appropriate, to patients and families and was promoted via flyers provided by the BHCs. Patients could contact the peer support partners directly or within a session, via their BHC. Staffing the clinics with in-person peer support staff was not feasible.

2.9. Data Collection and Analyses

Feasibility was measured via stakeholder input, numbers and percentages of screenings conducted, number of psychosis-related consultations sought, BHC and PCP attendance at training sessions, time reports, and BHCs’ and PCPs’ responses on the one-year postproject survey. Decisions and suggestions about the project’s implementation were logged by project staff (KAW, KME) at each meeting, were grouped and summarized via iterative review by the principal investigator and project staff (KAW, KME, DMW, RBJ), and were discussed with the BHCs and PCPs. Stakeholders’ comments were paraphrased for simplicity, unless enclosed in quotation marks.

Patients’ screening responses were recorded in an electronic health records system (Epic) via a study-specific template. Deidentified group-level data were retrieved from Epic for all medical records containing the project’s template. During the six months prior to implementation of Epic software at their practices, two of the BHCs recorded responses via a REDCap link. These data were combined with the group-level Epic data. The absence of individual-level data prevented statistical comparisons. The BHCs recorded the time and effort spent with each patient on a spreadsheet. Two of the BHCs were unable to complete their spreadsheets because of time constraints. The postproject survey data were collected anonymously via a REDCap survey link sent by the study staff and by the physician leaders of each practice. Psychiatric consultations were tracked through the MPBHP. Data on triage decisions were not analyzed and will be reported in a separate article.

RESULTS

3.1. Project Implementation and Provider Needs

All BHCs completed the 16 hours of training and attended monthly supervisions. Of the 34 PCPs, 27 (79%) attended at least one training session, as did 70% of all the other general medical providers (N=31of 44). Twenty (59%) of the PCPs attended two or more sessions. Four BHCs requested special case consultation. Eleven of the PCPs sought psychiatric consultation related to psychosis concerns for seven patients in the targeted age range. Consultation regarding psychosis-related concerns was also sought for three patients age <14. Lessons learned about the project’s implementation are summarized in the online supplement.

3.2. Volume and Demographic Characteristics of the Patients Screened

A total of 266 selective screenings (via the PQ-16 during a BHC appointment) were conducted, representing 89% of the 298 newly referred patients seen by the BHCs during the 1-year project (reasons for not screening the remaining 11% of patients were not collected). A total of 606 universal screenings (via the EPS-9 during a well visit) were documented during the same period, representing successful screening of 67% of the 900 well visits during this period (delayed start-up of the project and variable PCP follow-through reduced the rate). As displayed in Table 1, the demographic characteristics of the screening samples were nearly identical to those of the clinics’ overall patient populations. The screened patient population was >90% White and >55% rural. Although few patients belonged to racial-ethnic minority groups, most of these patients were included in the universal or selective screening.

TABLE 1.

Demographic characteristics of primary care patients screened (July 1, 2021-June 30, 2022)

		Universally Screened Patients N = 606 Age = 16.1±1.6		All Wellness Visit Patients N = 900 Age = 16.4±1.8		Selectively Screened Patients N = 266 Age = 16.8±2.6		All BHC Intakes N = 298 Age = 17.3±3.1
Sample Characteristics		N	%	N	%	N	%	N	%
RUCA Code
	1–3 Metropolitan	208	35	289	32	109	42	127	43
	4–6 Micropolitan	3	> 1	3	> 1	89	34	102	34
	7–10 Small Town/Rural	390	65	608	68	64	24	66	22
	4–10 Rural	393	65	611	68	153	59	168	57
Sex
	Male	275	45	434	48	77	29	84	28
	Female	331	55	466	52	187	71	214	72
Ethnicity
	Hispanic	11	2	11	< 1	10	4	10	3
	Non-Hispanic	593	98	887	99	252	95	284	95
	Unknown/Declined	2	< 1	2	< 1	2	1	4	1
Race
	Asian	4	< 1	6	< 1	1	< 1	2	1
	Black or African American	3	< 1	4	< 1	2	1	3	1
	White	584	96	871	97	246	93	274	92
	Multiracial	12	2	16	< 1	4	2	7	2
	Other	2	< 1	2	< 1	2	1	2	1
	Declined	1	0	1	< 1	10	4	10	3

Note. Data on sex and ethnicity were missing for two patients and on race for one patient in the selective screening sample. Rural-urban commuting area (RUCA) codes were unavailable for five patients in the universal screening sample, for four patients in the selective screening sample, and for three patients in the all-intake sample. “Rural” was calculated as the sum of all non-metropolitan RUCA codes (4–10).

3.3. BHC Time and Effort

Four (67%) of the BHCs, from three practices, logged hours spent on screening activities for nine of the 12 months of the project. Billable and nonbillable hours spent were documented for 159 patients (60% of the 266 patients screened). Of the screenings with such documentation, 92% (N=146) were completed within the first intake visit, and none extended beyond three sessions or 2 hours total. BHCs documented treatment engagement with 17 patients (11%), for an average of 1.00 billable hour per patient (range 0.25–2.88). Clinicians documented an average of 0.42 nonbillable hours per patient, which included the time spent completing the project’s documentation, communicating with relevant collateral contacts and with PCPs, preparing for sessions, and seeking consultation (i.e., typical nonbillable care).

3.4. Interest In and Use of Peer Support Services

No referrals between the ages of 14–26 were made to the project peer support partners. The BHCs indicated that the patients were generally not interested in talking to an additional unfamiliar person. Participants successfully referred to the PIER program were approached directly by peer support persons as part of that program’s engagement.

3.5. Initial Acceptability of Screening to Patients and Providers

Of 606 universal screens conducted, a quarter (N=152) of the patients endorsed at least one item at a threshold frequency (“sometimes”, on Likert scale from “never” to “always”; Brodey, B., personal communication, May 6, 2021). Nearly half (47%, N=125) of the 266 patients receiving selective screening met a published positive screen threshold (PQ-16 distress score ≥8) for a general mental health population.²³ According to the BHCs, a few patients were puzzled by the screening questions, but most answered them without any observable reaction.

On the one-year postproject survey (See Figure 1 and the online supplement), all six (100%) of the BHCs reported having seen patients who had experienced psychosis or a psychosis-like symptom, having comfort with screening patients for psychosis, and believing that the screening was important. They also reported that the screening resulted in better mental health care (online supplement). “Lack of time” was identified as the primary barrier to screening (Table 2).

Figure 1. One-year Post Project Survey Results.

Note. PCP: primary care provider; BHC: Behavioral Health Clinician; Figure 1 displays the results of select survey responses. Complete survey results can be found in the Online Supplement.

TABLE 2.

Barriers to psychosis screening reported by the primary care providers (PCPs) and behavioral health clinicians (BHCs) responding to the one-year postproject survey

Barriers	PCPs N = 12		BHCs N = 6
	N	%	N	%
Lack of time	9	75	4	67
Lack of appropriate screening tool	1	8	0	0
Lack of confidence in what to do after screening	2	17	0	0
Lack of appropriate resources for patients who screen positive	3	25	1	17
Lack of comfort discussing the screen with the patient or family	0	0	0	0
Other	2	17	1	17
No barriers	1	8	1	17

Note. “Other“ included inability to engage or lack of timely resources for patients.

Of the PCPs, 12 (35%) completed the one-year postproject survey, and most reported comfort in screening patients for psychosis (N=9, 75%) and having had patients who had experienced psychosis or psychosis-like symptoms (N=11, 92%) (online supplement). The PCPs reported that screening for psychosis was important and a majority indicated that the screening “likely” to “very likely” led to better mental health care. Barriers to screening reported included lack of time, lack of confidence in what to do after screening, and lack of appropriate resources for patients with positive screening results (Table 2). Although 75% (N=9) reported usually or almost always asking patients and families about family mental health history, only 25% (N=3) indicated usually asking about a family history of psychosis (online supplement).

DISCUSSION

Use of the SEE ME model appeared feasible within U.S. integrated care settings. Even amid the COVID-19 pandemic and a youth mental health crisis, 89% (N=266) of possible selective screenings and 67% (N=606) of one site’s possible universal screenings were completed. Given the start-up delays contributing to the lower universal screening rate, these rates suggested that a substantial amount of screening can occur in the primary care setting. The BHCs’ time spent on screening and triage was largely billable as part of a standard intake, and the BHCs and PCPs showed strong engagement in this project.

Aspects of the project that appeared to work well included BHCs taking the lead role with screen-triage-engagement activities. Given the role of administrative burden in physician burnout, the use of BHCs to conduct screenings is particularly important. Having PCPs conduct universal screening may not be feasible in most settings. In this project, engaging practice-based physician leaders; providing stipends; and offering training that emphasized lived experience, specific language for inquiring about psychosis, differential diagnosis and triage, and patient and family engagement strategies worked well.

The PCPs and BHCs indicated overall positive impressions of the project and a general impression that the project improved patient care. Because these providers often direct reluctant patients toward mental health care, their comfort in asking about potentially frightening and stigmatized experiences is important. The biggest barrier to screening identified was lack of time. Secondary barriers to screening included the lack of appropriate resources for patients who screened positive.

Peer support is increasingly recognized as a core component of comprehensive care after psychosis onset.^{29, 30} Despite substantial efforts to connect such youths (and their caregivers) with remote peer support, no connections to peer support were made directly via the BHCs. The patients, who often had disclosed their experiences for the first time, were not ready to talk with another unfamiliar person by text or telephone. One patient specifically wanted validation from a specialist. Stigma may also affect the desire to connect with others who have had similar experiences. Onsite peer support, or direct referral through electronic health records, would likely increase engagement.

The lessons learned from this project were limited to the current context. The participating practices were selected because of their strong physician leaders and interest in the mental health of adolescents and young adults. Feasibility must be established in a wider range of integrated care practices. The billable services related to screening, triage, extended assessments, and follow-up appointments varied by health care system and practice. Our conclusions were also limited by the lack of diagnostic information from structured interviews and the absence of individualized data. The generally White sample also limits the generality of the findings.

Further implementation of this project will require funding and practice-specific strategies for effective triage, engagement, screening, documentation, and follow-up. Efforts to improve general mental health screening and linkage to care will affect the capacity for addressing psychosis concerns, and SEE ME will likely improve general mental health treatment in turn. Regardless, outcomes will depend on the availability of high-quality care for the patients identified.

CONCLUSIONS

Early intervention in major mental health conditions relies on early detection. Current strategies that are based on help seeking identify only a small fraction of individuals in need within the time window associated with best outcomes. This project suggested that training integrated care providers to screen most adolescents and young adults for psychosis symptoms, and to conduct triage and care linkage as needed, may be possible within the context of an initial mental health appointment or even within a well visit. Next steps include assessing SEE ME’s effects on the timing and accuracy of psychosis detection and on linkage to care and qualitatively analyzing the project’s feasibility.

Highlights:

• The providers valued early detection of psychosis and were receptive to training (10–50 minutes for general medical staff; 16 hours for behavioral health clinicians).

• Only 11% of the patients screened required extended assessment or psychosis-specific engagement.

• Selective and universal screening for psychosis appears to be feasible within U.S. integrated care settings.