Acceptability of Psychosis Screening and Factors Affecting Its Implementation: Interviews with Community Health Care Providers

Abstract

Objective

Although screening for psychosis may reduce the duration of untreated psychosis, the barriers and facilitators associated with implementing such a procedure in various care settings have not been explored.

Methods

Investigators conducted in-depth, semistructured interviews with 17 members of school counseling service or community mental health staff at sites administer a psychosis-screening tool. Using an indictive approach to thematic analysis, they evaluated the acceptability of psychosis screening and barriers and facilitators of implementation.

Results

Participants reported few barriers to implementation. However, several service-, client-, and program-level factors were considered to significantly affect the implementation of screening. Most participants found that using the screening tool did not significantly affect their overall workload. Facilitators included the importance of leadership support, the novelty of using a technology-based screener, regular staff training, and the importance establishing an effective link between community services and specialty care, with these different factors considered significant at different stages of the process. Screening for psychosis was associated with significant advantages over referrals based on clinical judgement alone, including increased speed and accuracy of identification, increased confidence in diagnosis, and the provision of a clear pathway to specialty treatment.

Conclusions

The experiences of staff members working in school counselling and community mental health teams suggest that incorporating a technology-based screening procedure for early psychosis is feasible. Identifying barriers and facilitators at various stages of the screening procedure may reduce the dropout of clients potentially eligible for early psychosis care.

INTRODUCTION

The duration of untreated psychosis predicts future clinical and functional impairment (1, 2), affecting outcomes years later (3, 4). Starting treatment with three months of psychosis onset is recommended (5), given that outcomes are significantly poorer once this period is exceeded (6). However, initiation of specialist treatment in the United States is delayed, on average, between one and three years (7, 8). Early psychosis services have been developed with the aim of reducing the duration of untreated psychosis (5). However, they have not been found to significantly reduce delays (9). The impact of community education and awareness campaigns has been mixed (10).

Studies examining care pathways to early psychosis clinics suggest that delays within health services account for the greatest contribution of untreated psychosis (11), and are largely due to underdiagnosis in non-specialty care (12). Consequently, implementing screening for psychosis in nonspecialty settings, as is recommended practice for depression (13), may be one method to reduce the duration of untreated psychosis. In a recent trial (14), screening for psychosis risk detected a threefold higher prevalence of persons at high risk of psychosis in community mental health settings compared with standard care in addition to identifying a significant number of individuals whom they considered “fully psychotic” but who had not been previously identified as having psychosis. Therefore, screening for psychosis may represent a fast, effective solution to identification of early psychosis, without the need for additional expertise. However, the acceptability of psychosis screening, as well as barriers to and facilitators of its implementation, have not been explored.

In this study, we interviewed providers in school counseling services and community mental health clinics who screened for psychosis with a computer tablet–based screening tool as part of a research trial. The community mental health clinics adopted universal screening, and the school counseling services screened all individuals who were evaluated as part of special education assessments or who were seeking behavioral health services. These settings were selected because both have been identified as locations where a comprehensive early psychosis detection strategy could help reduce the duration of untreated psychosis (12, 15, 16).

Prior to starting the screening program, site staff received training on how to use the tablet, submit the referral, provide answers to commonly-asked questions, and introduce the tablet to clients or guardians. Training refreshers were offered on an annual basis. In the school setting, staff social work trainees provided the tablets to the clients. In the community mental health clinics, the tablets were provided by receptionists to clinical staff, dependent upon each sites preference.

On presenting at the referral site, all clients ages twelve to 30 were asked to provide consent prior to completing the Prodromal Questionnaire – Brief (PQ-B) (17) on a tablet. School sites obtained parental consent prior to presenting the individual with the tablet. Each screen was then assigned a unique study identifier, and all data were encrypted on the tablet. If a client’s total distress score was 20 or higher, the screening device instructed the provider to refer the individual to the local early psychosis service for a comprehensive assessment of psychosis-like symptoms. Following consent from the client (or a guardian if the client was a child), the provider faxed a referral to the University of California, Davis, Early Detection and Preventative Treatment early psychosis clinic, and a clinician from this service contacted the client via telephone to conduct an assessment. Clients who refused to continue at any stage of the screening or referral process continued to receive services with their current provider.

In this study, we evaluated staff experiences of implementing this screening procedure in order to determine acceptability of the intervention and to identify barriers to and facilitators of implementation.

METHODS

Design

We conducted qualitative, semi-structured interviews to examine providers’ experiences of implementing an electronic version of the PQ-B (17) as a screening tool for psychosis. Interviews were conducted at sites implementing a clinical trial evaluating psychosis screening as a method to reduce the duration of untreated psychosis and took place in July and August 2016. We analyzed participants’ responses by using an inductive approach to thematic analysis.

Two investigators were involved in each interview, one with significant experience of the main screening trial (HVS) and one who was more removed from the process (MS or HHG). HVS is the study coordinator for the screening program. MS is a research fellow with experience in running clinical trials evaluating complex interventions and in conducting qualitative interviews. HHG, a psychiatrist and social scientist, has extensive services research experience. Prior to conducting the study, all procedures were approved by the University of California, Davis, Institutional Review Board.

Participants

Investigators recruited participants from screening sites across the Sacramento area. Purposive sampling was adopted, recruiting staff members in a broad range of roles including clinical, management, administration and support staff. The only eligibility criterion was that they had to either be involved, or manage someone involved, in at least one aspect of the screening procedure. Consenting participants were interviewed at their respective clinical sites or schools.

Topic Guide and Data Analysis

Prior to the interviews, we developed a topic guide for conducting the interviews. After the preliminary interviews were completed, the guide was reviewed and refined, with greater focus given to emerging themes as more interviews were completed. [A copy of the staff interview topic guide is available as an online supplement to this article.] Interviews were recorded and transcribed, with identifiable data removed prior to transcription.

Through an iterative process, the investigators conducted the interviews and analyzed the data. After each batch of interviews was completed, the research team met to identify preliminary themes, generate hypotheses, and further develop the topic guide. This process was repeated until saturation of the main topic areas was reached. During analysis, a series of preliminary codes were developed to identify patterns in the data. From these codes, a set of emerging themes was identified and refined, before eventually being finalized. Four researchers were involved in developing and refining the coding framework.

RESULTS

The screening was implemented across two schools and five community health centers, with 584 screens completed between June 2015 and August 2016. Investigators interviewed 17 participants, including two clinical managers, one psychiatrist, six clinicians, one school psychologist, one coordinator of student support services, two senior office assistants, one mental health assistant, two chart room managers, and one AmeriCorps volunteer at a community mental health clinic.

Acceptability of screening

Participants were generally positive about the screening procedure. Reported advantages included faster client identification, increased confidence in clinical judgement, and a clear pathway to specialty services. Twelve participants reported that they felt implementing the procedure did not significantly affect their overall workload, whereas four reported that it had led to an increase in their workload.

Improved client identification

Seven participants indicated that the screener identified patients they would otherwise have missed, or else detected them faster than would be the case in standard practice.

“I feel like it definitely its picking up on it faster, and I feel like it does do a much more thorough job than maybe other clinicians may do during their assessment process.” (Participant_08, Intake Manager).

Participants reported that psychotic symptoms were previously not typically addressed during the initial assessment, unless they were mentioned in the referral or reported unprompted by the client themselves.

Increased confidence in clinical judgement

Three participants reported that a positive screen was helpful in validating the providers’ clinical judgement, supporting their decision to refer to specialty services. Some participants appreciated that the screener produced quantifiable data, which could be used in conjunction with clinical judgement to initiate referrals more quickly. Others highlighted that the screener provided a more comprehensive evaluation of more subtle symptoms that may have otherwise been missed. For example, one participant, a school psychologist, said:

“I think that the tablet is a useful tool in terms of data collection as part of a battery of assessments and just having clear numerical quantitative data to say, “Okay, this is what he’s saying. This is what she’s saying. Here’s the numbers to back it up. Let’s get the ball rolling.” (Participant_23, School Psychologist).

Clear pathway to specialist services

Three participants reported the process helped to establish a clear pathway to specialty care. Being involved in the screening program provided important information on the nature of the services available, the eligibility criteria, and the means to refer somebody when necessary. Such experiences highlight the importance of an ongoing partnership between screening sites and specialty care services that can accept referrals following a positive screen.

“I think it matches what it’s intended for: early detection and giving them that pathway over to you guys that they might not otherwise get.” (Participant_02, clinician).

Negative aspects of screening

Most participants reported that the screening and referral procedure was not difficult to manage alongside their current work schedule. However, three were concerned that the screener added to an already busy assessment process. These participants reported that although the assessments were short, any additional work was considered overwhelming. Finally, one school-based participant reported that one of their trainees was uncomfortable with administering the tool due to concerns that it may label individuals as “crazy”, and suggested that this may have been attributable to the trainee’s own lack of information about psychosis and the study.

“A few of them, had their own personal biases about the study without having a lot of… I don’t believe he had a lot of information. But it was kind of categorized as a way to identify students who are “crazy”. (Participant_18, Coordinator of Student Support Services).

Barriers and Facilitators to Implementation

Participants reported a number of barriers and facilitators relevant to different stages of the screening process. The various barriers and facilitators, characterized as service-, client-, and program-related factors, are summarized below. Some factors, such as the introduction of the screening tool, were considered to be relevant to only one stage of the procedure. Others, such as the impact of staff training, were considered significant at multiple stages. Figure 1 identifies the various stages of the screening procedure and the points in the process during which clients were more likely to drop out. Table 1 presents a number of solutions to barriers at each stage of the process, as proposed by participants, and quotations supporting the solution. [More supporting quotes on the themes described above are available in the online supplement.]

Figure 1.

Flow diagram depicting the screening procedure, and where client dropout occurs

Table 1.

Stages of dropout/low uptake, and proposed solutions to address potential barriers

Stages of low update/dropout	Facilitators/proposed solutions to address potential barriers	Supporting quotes
Client does not complete the screening procedure	Adopting more personal approach when introducing screener to increase initial client engagement and address anxiety, suspiciousness or ambivalence.	“I think the warmer approach works more effectively. I try to work it seamlessly into the intake process, like I said earlier, and I think that’s been received a lot better than just giving a tablet over and reading from the script.”
	Including screener as part of an intake checklist as both a reminder to clinicians, and to enable internal auditing	“We actually added the EDAPT screening to one of those checklists. That’s been really helpful. As a manager, I can make sure when that checklist is being turned in, is that EDAPT screening paper in there or not. That’s been a way that our agency has helped addressed that.”
	Leadership support in the implementation of procedures; enabling troubleshooting alongside those that administer the screening tool to address issues as they arise	“That no one participated; I thought that was really strange. I wrote some notes down about it and then talked to my boss afterwards and she agreed that maybe something should change, and we came up with the whole kind of more personal thing.”
	Staff member with experience of discussing mental health issues should introduce the screener. If not possible, consider providing training to address possible skill deficits	“I’ve done it and it’s not really a burden to me because I’m used to it. I’m used to dealing with the mental health side of things… but they’ve [the receptionists] never even done anything related to mental health or counseling, so they might not be so comfortable as I was.”
	Careful consideration of terms used to minimize issues related to stigma of psychosis	“If you do use the word psychosis, they’re going to freak out. I mean, if you use the word mental health, they’re going to freak out. I have to sometimes use behavioral health.”
	Provide information to the referral site to support internal auditing, aid the referral conversation with the client, and to increase staff motivation.	“Because I’m a data person, I would appreciate having numbers such as how many were referred at this site specifically… Because then, I can say this shows me either it’s being done, it’s not being done, what can we do to improve it.”
Client not referred to specialty services	Directly addressing clients’ ambivalence regarding changing of services	“They usually will go forward with the interview, as long as that’s explained to them – this isn’t, ‘No, we don’t wanna see you anymore.’ This is, ‘We’re gonna make sure we’re doing what’s best for the kid’.”
	Simplifying the process from screening to referral to minimize loss within system	“Sometimes I think there’s a kind of, you know, too many cooks in the kitchen kind of thing. Like, if it was just one-if the clinician or the doctor had the tablet and it was just a one-on-one thing between them, there wouldn’t be some kind of hand off, and then maybe that’s where parts get missed.”
	Regular staff training to improve implementation of procedures, particularly in situations where high staff turnover would be expected	“When we started, we trained everyone. And we had the meetings and stuff, and the PowerPoint. But that was probably five generations ago of staff. So, I think people being educated on actually how to do it might be a barrier in that they don’t know, then they’ll tend to not do it.”
Contact between client and specialty service not made	Assertive outreach to very low functioning clients, or those who experience severe anxiety using the telephone	“I think that’s going to be the only way you’re going to get to look at people that are that low-functioning is with outreach.”
	Consider including family/carer involvement as part of the initial engagement process	“More family involvement. I know that in the cases where we’ve had difficulty engaging those clients, I’ve had conversations about working with the family around that.”

Service-level factors

Support of leadership

Seven participants reported experiencing strong leadership support for the screening program, and most considered it important to its successful implementation. Examples of support by management included adding the screening procedure to assessment checklists to improve completion rates, managers meeting with senior administrative staff to work though implementation issues, tailoring referral forms and procedures at a site level to make it easier for staff to refer patients following a positive screen, and regular meetings with staff to ensure that the screening was being completed.

Organizational issues

Organizational factors were identified either as barriers to or facilitators of successful implementation by four participants. At one site, a different staff member was responsible for providing clients with the tablet, completing the referral form, and faxing the form to the specialty care service, and this diffusion of responsibility led to some referrals not being sent. Participants suggested that reducing the number of people involved may improve the likelihood of the referrals being submitted. Another participant reported that high staff turnover and structural changes made consistent implementation of any new procedures difficult. Regarding other potential facilitators, incorporating the screening procedure into existing checklists, which can be reviewed as part of ongoing procedures, was considered helpful.

Client-level factors

Symptoms, low functioning and ongoing life stressors

Seven participants reported that some clients refused to engage in the screening process because of suspiciousness, anxiety, and poor general functioning. In some cases, clients and families were experiencing a period of crisis, leaving them feeling too overwhelmed to take part. One participant reported significant challenges in trying to engage her clients in a telephone assessment because of their high disorganization and low functioning. In such situations, it was suggested that greater outreach from specialty services, more support from family, or both were necessary to engage clients.

Clients’ wish to not change services

Nine participants reported working with at least one client who expressed resistance about receiving care from a different service, sometimes leading to refusing a referral. Ambivalence was attributed to a preference for current services, convenience, hesitation about receiving any treatment, concern about the assessment length, and treatment delay. Some participants suggested that this barrier could be mitigated by carefully explaining the nature and reason for referral and ensuring rapid access to the specialty treatment provider.

Program-Level Factors

Introduction to screening

Twelve participants discussed issues related to how screening was introduced, or by whom. Although it was not necessary for trained clinicians to introduce screening, most participants suggested that the provider who dispenses the tablets should have enough experience in discussing mental health issues to answer the typical questions of clients and guardians. Adopting a personal approach was consistently recognized as an important facilitator for addressing client ambivalence, anxiety, and suspiciousness and was typically preferred over the scripts that were provided during training. Some participants avoided using “psychosis” and similar terms, believing that doing so could lead clients to disengage from the process because of stigma around the illness.

Use of technology-based device

Eight participants reported that clients appreciated using a tablet for screening and preferred this method to paper-based questionnaires. The main benefits cited were ease of use, speed of completion, and the novelty factor of using a tablet. One participant reported having difficulty in using the tablet because of a lack of experience with such technology.

Training

Tablet training and the user manual were well received. Four participants noted that the training and the manual helped with understanding the procedures, learning how to introduce screening, and answering potentially difficult questions. In school settings, role-playing of common client and family concerns during staff training was considered by one participant to be an important tool in learning to address difficult situations. Regarding barriers, in sites that typically experience high staff turnover, such as teaching hospitals, infrequent training was considered problematic, given that newer, untrained staff were less likely to adhere to screening protocols.

Communication and access to information

One participant in a managerial role reported that collecting data on the number of referrals faxed to the specialty care site by each staff member would help improve screening implementation. Another participant suggested that having more information about the outcome of the full evaluation would be a motivating factor to continuing the screening program.

DISCUSSION

Main findings

Staff who implemented a screening program for psychosis reported that the program had a number of advantages compared with standard care, including improved detection of psychosis and increased confidence in the clinical judgment of staff. The provision of a clearer pathway to specialty care was also considered important, highlighting the importance of an ongoing partnership between the screening sites and specialty care. Most participants reported that they were able to incorporate screening into existing practice without a significant increase in workload, although one-quarter were concerned about increased workload.

Regular training, leadership support, the novelty of using a tablet-based screening tool, and careful introduction of the screening improved client uptake in the program and reduced dropout at each stage of the process. These findings are consistent with other studies evaluating other screening programs for different conditions (18, 19). Some of the barriers identified were related specifically to screening for psychosis, including symptoms such as suspiciousness, stigma, and impairments in functioning, which affect the client’s ability to successfully engage. To address these barriers, it is likely that providers will require more knowledge and expertise in discussing mental health issues relative to screening programs for other, less severe behavioral health concerns.

Although the experiences reported by participants from community mental health clinics and school counseling services were broadly consistent, it is notable that school-based participants raised concerns about a lack of staff knowledge and the need to incorporate role-playing into training. That is perhaps unsurprising, given that school staff would be less likely to come into contact with persons with psychosis and that the school counselors who administered the screening were trainees with limited clinical experience and expertise. As a result, more training or supervision may be necessary for the successful implementation of screening programs for psychosis in school settings compared with sites such as community mental health clinics.

In another significant finding, different barriers and facilitators were implicated in breakdowns at various stages of the screening procedure. Barriers and facilitators considered important in one stage of the screening and referral process may be less important at a different stage. Consequently, these results have important implications for how health screening programs should be evaluated and implemented. Determining the effectiveness of each stage of the screening process separately (for example, completion of screening, agreeing to the referral being sent, and successful engagement with the new service) may be one way to better understand how various factors affect the screening program, allowing for more targeted solutions to improve screening implementation.

Strengths and limitations

Although studies have examined the barriers and facilitators of screening for conditions such as perinatal depression (18), trauma (19), developmental and behavioral delays (20), and substance abuse (21), to our knowledge this is the first study to examine the effectiveness of psychosis screening in community settings. We interviewed participants in various clinical, managerial and nonclinical roles, capturing a broad range of experiences related to screening implementation.

In regard to limitations, it is unclear whether the themes identified would generalize to other settings, such as primary care, to other psychosis screening tools, or to standard clinical practice (where additional research staff support would not be available). Given the interviewers’ association with the screening program, it is possible that participants felt less comfortable highlighting negative aspects. However, participants received no individual or organizational incentives for taking part in the screening project, and some agreed to be interviewed with the expressed aim of highlighting problems. Additionally, one aim of the investigation was to identify problems with implementation, potentially leading the interviewers to focus more on the more negative appraisals of the program. Finally, the clients themselves were not interviewed, meaning their perspectives could not be considered.

Conclusions

The experiences of staff members working in school counseling and community mental health teams suggest that incorporating a technology-based procedure for screening for early psychosis is feasible. Identifying barriers and facilitators at different stages of the screening procedure may help to improve the implementation of such programs by reducing the dropout of clients potentially eligible for early psychosis care. Making changes in who introduces the screening process and how, addressing client ambivalence about changing services, utilizing the support of family members, providing more assertive outreach, and emphasizing the importance of training were all identified as ways to improve screening effectiveness and facilitate successful linkage with specialty services.