How Do Early Psychosis Services Define and Operationalize the Duration of Untreated Psychosis?

Abstract

Reducing the duration of untreated psychosis (DUP) is a key aim of early psychosis (EP) care. However, substantial variability in how the start and end points of DUP are defined impact its utility in clinical decision making, and as an outcome measure. In this study, qualitative interviews were conducted with providers to assess how EP services and providers define, operationalize, and measure DUP. Twenty-five providers across 14 clinics were interviewed. Participants emphasized symptom frequency, conviction, distress caused, and impact when determining psychosis onset. DUP endpoint was typically identified as the first assessment in an episode of care that included an accurate diagnosis, leading to specialty EP treatment. Participants proposed a more structured operationalization of DUP, relative to those historically adopted in the literature. Integrating front-line provider perspectives could improve the accuracy of DUP measurement and address the heterogeneity in how the construct is operationalized across research and practice.

Introduction

A longer duration of untreated psychosis (DUP) has been consistently associated with poorer outcomes for individuals with early psychosis (EP).^1–3 DUP is a modifiable risk factor that can be reduced by early identification programs, community outreach, and addressing barriers to treatment.^4,5 This can in turn lead to improved outcomes.⁶ As a result, reducing DUP is recognized as an important goal for EP services, with standards set by the World Health organization (WHO).⁷ However, there is no standard or best practice method to measure DUP that can be applied across EP programs.⁸ In studies that have reported DUP, a large range has been reported, both within studies and across research groups. This is likely to be at least partly attributable to differences in the operationalization and assessment of the construct.⁹

In the literature, DUP has been broadly defined as the length of time between the onset of psychotic symptoms, and the initiation of treatment.¹⁰ Therefore, to measure DUP one must identify two separate timepoints: 1) the onset of psychosis and 2) the onset of treatment (see Figure 1). However, the operationalization of these timepoints have varied widely.^9–12 For example, DUP onset has been defined as the first experience of delusions or hallucinations;¹³ evidence of active illness, including positive, negative and affective symptoms;¹⁴ onset of any positive symptoms, or catatonic motor behavior;¹⁵ or first noticeable change in behavior.¹⁶ The frequency of the symptoms has been included as an additional qualifier in some studies, specifying that symptoms must have lasted for several days, be sustained, appear at least several times over a period of a week, and/or not be limited to brief moments.^17–20 The endpoint of DUP, meaning the onset of treatment, has been defined as the first contact with a mental health professional,²¹ initiation of treatment,¹⁹ initiation of first effective treatment,¹⁷ first treatment with antipsychotics,²³ first psychiatric hospitalization,^14,23,24 or study entry point.^15,25 Others have adopted more restrictive criteria such as antipsychotics given over a specified period of time, and at an appropriate dosage, leading to clinical response.^26–30

Figure 1.

Duration of Untreated Psychosis in the Timeline from Initial Onset to Treatment

Key: CHR: Clinical High-Risk.

While evidence suggests that there is no difference in reported DUP length between studies that use either first hospital admission or the initiation of antipsychotic medication as the end point,³¹ it is unclear whether other methods used to define either the start or end of DUP affect its reported length. This is a significant issue, as it is feasible that services and interventions designed to reduce DUP could be considered to have either succeeded or failed to achieve WHO recommendations based purely on how the construct is operationalized. Furthermore, this heterogeneity impacts the ability to compare outcomes across research groups or practices and affects the utility of amalgamating reported figures across practices. This is problematic in era of “big data” that is working towards developing learning healthcare systems for EP care.³²

In order to help harmonize how DUP is operationalized across both research groups and clinical services, one novel way forward may be to develop standardized definitions to determine what constitutes the start and endpoint of DUP based on the experiences of those who currently deliver EP care. Given that EP programs are exclusively designed to provide treatment during the early “critical period”,³³ service providers in these settings typically have extensive experience of evaluating the onset of psychosis as part of their intake assessments to determine client eligibility. As treatment providers, they are in a unique position to help determine what may be considered an appropriate start point of such treatment. Evaluating the experiences of such providers may provide new insights into how DUP is currently defined and operationalized in EP services. In turn, these experiences could provide novel approaches to standardizing these terms, both in research and clinical practice. Operationalizing DUP based upon the experiences of clinicians may increase the likelihood that such a method may be more intuitive and acceptable to practitioners, increasing the feasibility of it being implemented in current clinical practice.

In this study, a stakeholder group was held with providers, family members, clients and countylevel mental health representatives to develop preliminary definitions of the start and end point of DUP, incorporating different perspectives on what features and challenges may be most significant to measuring this construct. Our main intention was to ensure that the starting definitions were not too heavily influenced by a research perspective which may not translate to clinical practice, or not be seen as feasible by clients and family members. Following this meeting, qualitative interviews with EP providers were then conducted to assess how they define, operationalize and measure DUP, using the definitions proposed during the stakeholder meeting.

Method

Design

Qualitative, in-depth, semi-structured interviews were used to access EP providers’ experiences of defining, operationalizing and measuring DUP in a clinical setting. Prior to conducting the interviews, a stakeholder meeting was convened at [BLINDED] with the aim of developing preliminary definitions of the start and end point of DUP which would be the basis for the interview guide. The study was completed as part of a [BLINDED] project to standardize the measurement of DUP across state services [BLINDED].

Participants

In order to ensure a broad range of responses, county-level policy makers, EP service providers, service users, and family members of individuals receiving EP care were recruited for a stakeholder meeting. Prospective participants were identified and approached on the basis of their ongoing involvement in a [BLINDED] project examining the implementation of early psychosis services across the state.

Following the stakeholder meeting, 30 active EP programs across the state of California were identified through a review of Mental Health Block Grant applications, and feedback from stakeholders conducted as part of an earlier study evaluating the structure of EP programs across California [BLINDED]. The contact person of each identified program was then approached, via email, and EP staff members were invited to participate in a qualitative semistructured telephone interview. Across consenting sites, staff members with managerial and clinical roles at each site were then selectively recruited for their insight and comprehensive knowledge of DUP measurement at the program level. Of the 14 sites that agreed to take part, nine were community-based EP programs and five were university-based EP clinics and research programs.

Data collection and analysis

An interview guide was developed by the research team based on input from the stakeholder meeting (see supplementary material). Topics included defining the start and endpoint of DUP, determining how these concepts can be operationalized in clinical practice, and evaluating the barriers and facilitators to determining an accurate assessment. Participants were provided a copy of interview questions at the time of scheduling.

Interviews were conducted via telephone to best accommodate participant availability and were conducted by one of two postdoctoral-level researchers; one a medical sociologist specializing in qualitative methods, and one a services researcher specializing in early psychosis. All interviews were recorded, transcribed, and hand-coded. The method of enquiry used was directed content analysis.³⁴ In using this approach, the analysis drew both on the existing literature and the theories proposed during the stakeholder meeting as a starting point, from which the participants either endorsed or refined the proposed definitions based on their clinical experiences, or else proposed new definitions which were then further explored.

Through an iterative process, the investigators conducted the interviews and analyzed the data. After conducting several interviews, the research team met to identify preliminary themes, generate hypotheses, and further develop the interview guide. One area that required significant revision related to how to define the end point of DUP (i.e. the start of treatment). Most sites did not typically use any one definition to determine the start of treatment and could be recorded as different for different purposes (i.e. for billing, meeting service inclusion criteria, and then what they considered the most clinically relevant). Given this investigation focused on the impact of DUP on clinical outcomes, we encouraged participants to come to a definition they considered most clinically meaningful. As definitions and operationalizations were then proposed during the interviews, these were presented at subsequent interviews for consideration. Such a process was repeated until data saturation was reached. Manifest and latent content were assigned codes based on existing theory, concepts emerging from the data (data driven codes), and the research questions (structural codes).³⁵ One such manifest code was the challenge of determining an accurate diagnosis and date of onset in clients that reported co-occurring substance abuse. A latent code included the challenge of disentangling co-occurring phenomena when aiming to diagnose and determine psychosis onset. These could be related to factors such as substance abuse, culturally normative experiences, other mental health comorbidities, or features inherent to the clients’ developmental stage. From such codes, a set of categories were identified and a set of emerging themes were established.

Results

Stakeholder meeting

Eighteen participants were invited to participate in the stakeholder meeting, of which 12 attended (response rate 66.7%). Attendee details are presented in Table 1. The stakeholder group was used with the intention of developing preliminary definitions of DUP in a manner that considered the perspectives of all relevant stakeholders. This was to ensure that the definitions used in the interviews were not based purely on the perspectives of researchers. In doing so, the aim was to minimize the risk of developing definitions that might not be considered intuitive, appropriate or realistic to those who would be involved in the assessment process in a typical clinical setting, be it as the provider, client, or family member providing collateral information.

Table 1.

Details of Stakeholder Meeting Attendees and Interview Participants

Participant Details
Stakeholder meeting attendees (n, %)	12
Providers	4	33.3%
County Representatives	2	16.7%
Family Members	4	33.3%
Consumer Advocates	2	16.7%
Interview Attendees (n, %)	25
Program Manager/Director	9	36%
Psychologist	3	12%
Psychiatrist	3	12%
County Representative	3	12%
Clinical Supervisor	2	8%
Clinical/Program Coordinator	2	8%
Program Supervisor	1	4%
Assistant Director	1	4%
Marriage and Family Therapist	1	4%
Interview group size (median, IQR)	2	1 – 3
Clinic type (n, %)	14
University affiliated	4	28.6%
Community based	10	71.4%

Key: IQR: Inter-quartile range.

The group developed the following definition of DUP to be used as a starting point for discussion in the interviews:

The first point at which psychotic symptoms (such as hallucinations, delusions and/or disorganized speech or behavior) become recurrent or persistent AND cause significant distress, impact behavior, or are dangerous.

Participants identified co-occurring substance abuse and other comorbidities, developmental delay, a longer duration of psychosis, and younger age of the client as challenges to accurately determining the onset of psychosis.

Multiple definitions of treatment onset were proposed, including first entry into a specialist EP program, first contact with any type of mental health treatment program, and first contact with any psychosocial treatment. Due to the lack of consensus regarding treatment onset, no provisional definition was proposed for the qualitative interviews.

Qualitative interviews

Of the 30 EP programs active across California, 14 were interviewed. Four clinics were university-affiliated practices and 10 were community-based. No consistent differences in the answers given across clinic types were identified. Across these sites, 25 participants were interviewed over a series of 12 meetings. Participant details are presented in Table 1.

Determining psychosis onset (the start point of DUP):

The definition of psychosis onset developed during the stakeholder meeting was presented to participants. While some considered this to be an appropriate definition consistent with the criteria they use in clinical practice, most thought that additional caveats were necessary to improve the consistency of interpretation between programs. Some participants felt that the definition proposed was too subjective, and required either more concrete examples of functional impairment, such as an inability to adhere to routine or hygiene, or more clarification on what criteria should be used to determine what constitutes “recurrent and persistent”.

“…those probably need to be spelled out more specifically because I think, otherwise, it’s going to be fairly subjective. Like at what point do people think this impairment is significant?”

Other participants suggested that it was important to make explicit any exclusion criteria, such as symptoms not being attributable to developmental disorder, or temporary effects of substance abuse.

“The way it’s defined right here, it is describing someone that might be under the influence of any illegal drugs, or maybe someone that might be having a stroke or having other issues. So, maybe, be very specific that it’s related to mental health.”

Finally, some participants suggested that it was necessary to determine whether doubt in the individuals’ psychotic experience could be induced, given the importance of conviction in distinguishing between full psychosis and attenuated positive symptoms. This distinction is a focus of diagnosing the clinical high risk (CHR) syndrome for psychosis. Many of these programs serve the CHR population and have had formal training and supervision around CHR diagnosis.

“Well, the other piece that we think about – which is sort of implied in this definition but may be worth stating clearly – is the level of conviction. Particularly around unusual thoughts and delusions.”

In order to assess the onset of psychosis, most participants reported using a comprehensive, multi-stage model of assessment. Some used a preliminary screen by phone, prior to an indepth face-to-face meeting that all sites completed. These in-person interviews were conducted using either an unstructured clinical interview, or a structured clinical assessment tool such as the Structured Clinical Interview for DSM disorders (SCID),³⁶ or the Structured Interview for Prodromal States (SIPS) which determines the presence of the CHR syndrome and a threshold psychosis syndrome.³⁷ Following this assessment, some sites reported holding a consensus meeting in order to confirm diagnosis and illness onset. At one site, the participants reported their program has external supervision from an expert diagnostician to provide support in more complex cases.

“I think it requires a thorough evaluation, I believe, by multiple people in different settings.”

Participants identified a number of challenges to identifying an accurate onset date of psychosis. Many of the challenges related either to the difficulties in obtaining reliable collateral information, or the challenge of disentangling co-occurring phenomena. Consistent with the stakeholder meeting, disentangling psychosis onset from comorbid substance use was recognized as the biggest and most common difficulty. Some participants noted the challenges of differentiating full psychosis from attenuated symptoms, and from other disorders such as Autism.

“The other pieces that I think make cases really complicated are drug use. If you've got somebody who's a substance user and trying to tease apart the symptoms related to substance use versus the symptoms related to psychosis, I think that can be really hard.”

Other challenges included longer duration of psychosis, which can make a timeline more difficult to reconstruct due to memory impairment and a more complicated history, and the younger age of patients when they have difficulty describing their experiences verbally.

“If somebody comes into the program really close in time to when the onset was, we’re usually able to get really accurate information. If it’s been several years, it’s very hard to pin down exactly the month and time unless they’ve had a hospitalization or something that was attached to it.”

In addition, disentangling psychotic experiences from culturally normative experiences, particularly around spiritual/religious beliefs, adaptive vigilance, and beliefs held within the individuals’ peer group was considered challenging. It was suggested that in different situations this can lead to both an under- and over-reporting of psychosis.

“You do have to take culture into account. And I think one of two things happens in the untrained evaluation. They either take it into account too much, where they're like, "Oh, it's just a teenager, or it's just a child." Or they don't take it into account at all. And so, you either get underrating or overrating.”

In order to mitigate the challenges identified, seeking collateral information was consistently identified as the most important method to improve accuracy when determining psychosis onset. Some participants suggested that information from family members was particularly helpful in identifying when functional changes had occurred. Others reported that it is important to provide psychoeducation about psychosis first, in order to explain what the signs and symptoms of psychosis are, and to address any possible stigma.

“It’s amazing how much information loved ones actually have, that they often don’t realize they have.”

“The caveat is that needs to come with a great deal of psychoeducation. Again, explaining what the symptoms and signs of psychosis are to your audience.”

In addition to seeking collateral information, one participant suggested using a timeline such as the UCLA Diagnostic Timeline tool where the provider, and client and/or family member together plot the emergence of symptoms, periods of substance abuse, major life events, and hospitalizations over time to help disentangle co-occurring phenomena and help better identify psychosis onset.^12,37 Finally, using life events as anchor points to help identify when particular symptoms emerge, and using a team-based approach to evaluation was considered important. In most cases, this meant presenting an individual’s assessment to the multidisciplinary team where both the diagnosis and date of onset could be debated and confirmed.

“I don’t understand why we’re not trained on this in graduate school […]. This timeline is just such a valuable way to look at diagnoses”

“We also discuss as a team, the whole team together. […] If there's still grey areas and a consensus can't be reached, then it's brought to the entire team, and we try to get to the bottom of it via the entire team.”

Determining treatment onset (the endpoint of DUP):

In terms of defining treatment onset, participants reported one of two different perspectives. Some reported that they would consider the patient receiving any form of mental health treatment as being the point in which DUP should be considered ended:

“I think for me somebody had come from another type of treatment and had gotten at least some services, I would consider that already being beginning in treatment.”

However, the majority of participants appeared to endorse a more structured process, where treatment onset is defined by the completion of three discreet steps. These included 1) an accurate diagnosis, 2) the offer of treatment appropriate for the diagnosis, and then 3) engagement in the treatment, which in most cases was considered to be in the context of the EP specialty care program.

“I think it comes back to that comment we made earlier about they need to be accurately diagnosed and then given appropriate treatment for that diagnosis.”

In situations where these steps were not completed, most participants did not consider DUP to have ended. When all steps were completed, then the initiation of that particular treatment episode - i.e. when the first diagnostic assessment took place - was proposed as the most appropriate point at which DUP could be considered ended.

“In the example of the person that comes in and does the evaluation, comes back two weeks later, does their welcome session, and then fully engages; I would include the intake [assessment] as treatment.”

Participants reported that these steps should occur within one episode of treatment, but do not necessarily need to occur within the same service. For example, if a patient is given an accurate diagnosis in one service and appropriate treatment is initiated before being referred on to a specialty EP program, some participants considered that this intervention should be included as part of the single episode of care. For most sites, the full package of care delivered by EP services was considered necessary to define the end of DUP. In the few cases that antipsychotic medication alone was mentioned by participants, all but one participant reported that this should not be considered as sufficient.

“For me, it’s entering our program. A lot of the literature out there will say that initiating antipsychotic regimen is treatment, but we wouldn't have early psychosis programs if that was fully effective. And so, the actual entering of the early psychosis program for treatment is the correct level of care.”

Discussion

Participants provided a number of insights relevant to standardizing the measurement of DUP in early psychosis. In order to improve the consistency of how psychosis onset is determined between sites, concrete thresholds regarding the frequency of psychotic symptoms, their impact on everyday functioning, and the conviction in which they are held were recommended. In the measurement of psychosis onset, a comprehensive, multi-step method of assessment was considered necessary. Participants identified substance abuse and other comorbidities, chronicity, client age, cultural norms, and the presence of sub-threshold symptoms as challenges to accurately determining the point of onset. The importance of collateral information, using a timeline as a visual aid, anchoring changes in symptoms and functioning to co-occurring life events, and adopting a multidisciplinary team-based approach to assessment were all identified as methods in which to mitigate the impact of these factors.

In regards to defining the endpoint of DUP, most participants emphasized the importance of an accurate diagnosis to enable the provision of appropriate treatment. Contrary to much of the current research literature, patients receiving an antipsychotic alone, without the full package of EP care, was not considered sufficient to conclude DUP had ended.

Defining psychosis onset:

Based on the responses from the stakeholder meeting and subsequent interviews with EP program staff and leadership, the following definition of the startpoint of DUP was proposed:

“The first point at which psychotic symptoms (such as hallucinations or delusions experienced with full conviction, and/or disorganized speech or behavior) are 1) present, on average, at least four times a week for an hour a day for one month; 2) are not better accounted for by another cause (e.g. substance abuse, neurological disorders); and 3) cause significant distress, are dangerous, or impair the person’s ability to engage in standard daily tasks.”

The frequency and duration of symptoms are based on the Presence Of Psychotic Syndrome criteria of the SIPS,³⁷ while the phrase “not better accounted for by another cause” is based on a term adapted from the SCID-V.³⁶

Definition of treatment onset:

Most participants endorsed a model to determine the point of treatment onset consistent with Figure 2. This comprises of three discreet steps, including 1) accurate diagnosis, 2) the offer of evidenced-based treatment appropriate for the diagnosis, and then 3) engagement in that treatment. In this context, treatment was defined as engagement in the full package of EP care. Following the successful completion of these three steps, the first date that the assessment took place was considered an appropriate start point for treatment. In most cases, this would take place as part of the EP program intake assessment but could start in prior to the EP referral if an accurate diagnosis was made, and the appropriate treatment then initiated as part of a single episode of care.

Figure 2.

Process in Determining the Endpoint of Duration of Untreated Psychosis.

Key: DUP: Duration of Untreated Psychosis.

Strengths and Limitations

One strength of the study is that participants were recruited from EP programs throughout the state of California. As a result, the authors were able to record the experiences of a range of clinicians and managers working in rural and urban locations, both in university and community sites, treating a diverse range of clients. In addition, the interview schedule developed through the stakeholder process allowed for input from multiple sources, including researchers, providers, county representatives, and family and consumer advocates, allowing for many perspectives to be considered throughout the process.

Regarding limitations, in early program interviews there appeared to be some hesitation to answer honestly for fear of providing an incorrect answer, or an attempt to provide the ideal answer. This issue was identified early in the process, and the interview guide was amended to clarify that there were no correct answers, and that the authors were only asking for the participants’ ideas and suggestions for this process based on their own experiences. This new approach seemed to increase the candor of the participants. Second, it is important to note that the interviews were all conducted in small groups. The study was conducted in this manner as the interviews were part of a larger investigation where different topics of interest required the knowledge and expertise of different providers. However, it is possible this may have led to the interview being impacted by internal agency politics, where less senior participants may have been less likely to present a dissenting view, or else not provide what they believed to be a model answer. Third, the ideas and opinions gathered during the stakeholder and EP program interviews were based solely upon information from individuals from California, which is significant given the state has operated under a service-led approach to EP program development. As a result, it is possible that interviewing participants from states that have developed services utilizing a more centralized model may have yielded additional information. Fourth, of the 30 EP programs that were approached to take part in the study, participants from only 14 different clinics were interviewed (46%). This relatively low response rate may have impacted the range of experiences reported, particularly given a disproportionate number of community-based services elected not take part compared to university-affiliated programs. However, the responses across sites types appeared to be broadly consistent, suggesting both that this did not significantly affect the results, and that saturation of the main themes had been reached.

Implications for Behavioral Health

The definition of DUP onset proposed in this study appears to be more stringent than those typically used in the current research literature.^9–11 Historical definitions that do not fully consider the duration, frequency, conviction, or impact of symptoms may inflate the reported length of DUP due to more transient positive symptoms indicative of CHR being incorrectly characterized as psychosis.¹⁰ By defining the endpoint of DUP as the first assessment in a single episode of care that leads to treatment at a specialty psychosis service, it is possible that this date would be significantly later than historical definitions that use either the date of first administration of antipsychotic medication, or first hospitalization. At present, it is unknown whether these changes to the operationalization of DUP lead to a change in the reported DUP lengths in clinical or research samples or impact the noted association to poorer treatment response.²

Participants reported that key to determining both the onset and endpoint of DUP was the accurate diagnosis of the illness. Accurate diagnosis can be complex, and typically requires significant training and resources. As a result, the experiences of clinical staff interviewed in this study suggest that clinical services and research teams may be required to invest considerable resources into determining the length of DUP in cases where the accuracy of this outcome is considered of critical importance. As a result, it is likely that investing in such an exhaustive approach to determining DUP may not be feasible for services not adequately resourced to do so. Furthermore, it is unclear whether such an approach would be clinically meaningful in alternative settings where both the diagnosis and duration of illness is not so critical to determining treatment eligibility or outcome. If less intensive methods are adopted, the subsequent effect that this may have on DUP assessment accuracy is currently unclear.

With the proposed definitions based on the experiences of a small number of participants, the next steps will be to conduct a larger survey of relevant providers, observe how DUP is assessed to understand how the process occurs in real-time, refine and validate the proposed process though a consensus conference, and then test the feasibility of implementing this process in clinical practice. Following this, developing specific training materials and guidelines for assessing DUP that can be incorporated into existing training programs for providers specializing in EP care, such as those delivered by the Early Psychosis Prevention and Intervention Centre,³⁸ may be necessary to ensure the process is fully incorporated into the EP care model, and to maximize future uptake in community clinics.