Receipt and targeting of evidence-based psychosocial interventions for people living with psychoses: findings from the second Australian national survey of psychosis

Abstract

Aims.

Clinical Practice Guidelines (CPGs) recommend evidence-based psychosocial interventions (EBPIs) to improve consumer recovery; however, availability appears limited. We describe receipt of six EBPIs, reported by people with psychoses, and associations with service and consumer characteristics, including indicators of need (eligibility) and benefit (suitability).

Methods.

Participants in the 2010 Australian national survey of psychosis (n = 1825) were interviewed to assess demographic, functional, mental and physical health characteristics and service use in the previous year. Six EBPIs (Cognitive Behaviour Therapy for psychosis; Family Psycho-Education (FPE); Relapse Prevention Planning (RPP); Skills Training; Supported Employment; and Assertive Community Treatment) were chosen, based on the strength and consistency of CPG recommendations. Associations between receipt of interventions and eligibility and suitability indicators were examined via correlations and χ². Logistic regression was used to predict receipt of one or more EBPIs and to identify predictors of each individual EBPI.

Results.

Less than one-quarter of the sample reported receipt of an evidence-based level of any intervention: rates ranged from 3.4% (FPE) to 21.1% (RPP). The model predicting receipt of one or more EBPIs was statistically significant (χ² (20, n = 1746) = 216.12, p < 0.01) and marginally useful. Nine variables contributed uniquely, of which six were service characteristics. The strongest predictors of receipt were being assigned a psychologist as a case manager (p < 0.01, OR(CI) = 2.36(1.50–3.72)) and accessing a non-clinical mental health support service in the past year (p < 0.01, OR(CI) = 2.01(1.60–2.51)).

Conclusions.

Prior reports of limited receipt of EBPIs are reinforced. There is patchy evidence for targeting of EBPIs to those who might benefit most. Service characteristics contribute more to the prediction of receipt than clinical characteristics. Greater implementation effort and better targeting are required to bridge evidence-practice gaps, including improved evidence-based practice literacy among professionals and needs-based service re-design to improve provision and optimise consumer outcomes.

Introduction

Clinical Practice Guidelines (CPGs) for schizophrenia (NICE, 2014; Galletly et al. 2016) recommend best available treatments and services. They include evidence-based psychosocial interventions (EBPIs) such as Cognitive Behaviour Therapy (CBT). EBPIs are particularly important for people living with schizophrenia and other psychotic disorders, since many experience sub-optimal clinical recovery even with recommended medication.

There is little international data about receipt of EBPIs. A systematic review of UK studies addressing implementation of national recommendations (Ince et al. 2016) found reported rates from 4% to 100% for CBT and 0% to 53% for Family Interventions, with heterogeneity too great to allow aggregation of results. Availability appears to be limited (Harris & Boyce, 2013; Haddock et al. 2014), except in mental health services that have systematic implementation programs (e.g. McHugo et al. 2007). Most studies audit conformance with CPG recommendations using clinical files (e.g. Lehman & Steinwachs, 1998; West et al. 2005) or through service provider reports (e.g.Magliano et al. 1998; Fadden et al. 2002; Kim & Salyer, 2008). We are not aware of studies seeking patient reports of EBPIs for psychoses, other than the Rethink survey in the UK, reported by Ince et al. (2016), which sought self-reports about receipt of CBT without specifying the definition of the treatment or duration. Further, we are unaware of systematic enquiries about the extent of receipt of EBPIs within larger population-based samples; information that is particularly helpful for national policy and funding.

No treatment is likely to be relevant to the needs of all patients. Judgements about the adequacy of observed rates of receipt of an intervention, or its targeting, should consider the numbers of people requiring the intervention. Therefore, we searched the literature for guidance on both eligibility and suitability indicators for EBPIs. We defined eligibility indicators as demographic, illness or disability-related characteristics that meet conditions for providing the intervention. For example, having regular contact with family members indicates eligibility for Family Psycho-Education (FPE). This eligibility threshold avoids including people for whom the intervention was not intended, whilst minimising exclusions. Further, the resource costs in providing EBPIs are significant; thus they are likely to be provided not just because a person meets eligibility criteria, but also because they are seen as likely to need, or benefit from, the intervention. Hence, suitability indicators were defined as intervention-specific demographic, illness or disability-related characteristics associated with better targeting, the likelihood of engagement (Fanning et al. 2012) or successful outcomes (van der Gaag et al. 2011).

Based on the strength and consistency of CPG recommendations we chose six EBPIs for study: CBT for psychosis (CBTp); FPE; Relapse Prevention Planning (RPP); Skills Training (ST); Supported Employment (SE); and Assertive Community Treatment (ACT) (See Table 1 for definitions).

Table 1.

Six evidence-based psychosocial interventions (EBPIs): definitions, corresponding SHIP survey questions and respective eligibility and suitability indicators according to peer-reviewed literature and available national psychosis survey variables

EBPI	Definition and aims (from literature and CPGs)	Corresponding 2010 national psychosis survey (SHIP) question	Evidence-based level adopted	‘Strong’ eligibility indicators from literature	Eligibility indicators available in SHIP	‘Strong’ suitability indicators from literature	Suitability indicators derived from SHIP variables
CBT for psychosis (CBTp)	Reduce distress and increase adaptive behaviour by working with thoughts and beliefs that mediate emotional and behavioural responses, and by enhancing adaptive coping	Did you receive any therapy where you and your therapist explored your thoughts, feelings and beliefs about your symptoms and illness and came up with new ways of understanding and coping with them?	Eight or more sessions	None	N/A	Persisting positive symptoms, fewer negative symptoms and absence of co-morbid substance abuse	identifying ‘uncontrolled symptoms of mental illness as a top three challenge in the next year’; ‘lack of relief from medication for mental health’; low Carpenter negative syndrome score in past year (less than 2); absence of ‘any substance use in past year’
Family Psycho-Education (FPE)	Reduce consumer relapse and carer burden by bringing together the consumer and their family or other carers to learn together about the disorder, and build skills such as communication, problem solving and crisis support	Have you and your family met together regularly with a mental health clinician to learn about mental illness and improve your communication and problem-solving skills?	Six or more sessions	Some contact with family	Has some ‘contact with family during past year’	Frequent contact with family	Composite indicator created: ‘living with any relative’ and ‘at least weekly face to face contact with any family’
Relapse Prevention Planning (RPP)	Pick up early warning signs of relapse in time for treatment and support changes that might prevent a relapse, treat it early or mitigate its full impact	Have you done some future planning with a mental health worker in which you discussed your past experiences of being unwell and identified warning signs that a relapse may occur? AND Have you made a written plan about what you and a significant other person in your life, and the mental health service could do if those warning signs return?	A written plan in addition to the discussion	None	N/A	Recent history of relapse and past history of relapse	‘any mental health admission in past year’; ‘multiple episodes or continuous illness in the course of their disorder’
Skills Training (ST)	Address functional skill difficulties through use of behavioural skill training or educational skill teaching models of change	Have you participated in a group or individual training program for improving social skills or independent living skills, that included assessment of your skills, teaching and practicing new skills, then trying them out in your daily life?	Six or more sessions	Functional skills deficits	Greater than ‘only mild difficulties’ on the Personal and Social Performance scale (i.e., PSP < 71)		‘illness interferes with desired relationships’; ‘dysfunction in overall socialising’; ‘relationships deteriorated due to illness’; MSIF overall global residential rating: moderate or lower role performance
Supported Employment (SE)	Assists the individual to find a job in the open labour market and provides support to keep them in that job, including advice, assistance with making necessary adjustments in the workplace (e.g. changing work schedules) and with managing their health and sources of stresses within the workplace	Did an employment support worker provide you with support, advice, counselling or speak with your case manager about your mental health needs?	Six or more sessions	Desire to work	‘Looking for work’ or ‘employed during past year’	Better prior work history and education beyond secondary level	‘wanted help to work or use one's time’; ‘educated beyond secondary level’
Assertive Community Treatment (ACT)	Service model defined by a discrete team with a small fixed caseload, enabling intensive practice characterised by in vivo and extended hours delivery of interventions through assertive outreach, medication supervision, problem resolution and individualised rehabilitation	Was there a period of time when your main mental health care was provided by a team from the mental health services who regularly visited you in your home, in the daytime and evenings, to help with your mental health?	N/A	Difficulties in providing treatment in the context of prolonged and complex illness	Exclude persons with ‘single episode with good recovery’	Difficulties with engagement, frequent hospital admissions, co-morbid substance use, poor course of disorder and poor functioning	‘poor functioning according to PSP (i.e. PSP < 31)’; ‘very unconfident that current medication is a good thing for their mental health’; ‘multiple episodes of illness with partial recovery or a continuous illness in the course of their disorder’; ‘two or more admissions for mental health in past year’; ‘any substance use in past year’; ‘very dissatisfied with help and support received from case manager’.

With regard to eligibility and suitability criteria, we identified no eligibility restriction for CBTp (NICE, 2014; Galletly et al. 2016). Three suitability indicators were evident: persisting positive symptoms (Sivec & Montesano, 2012); absence of comorbid substance abuse (Barrowclough et al. 2010); and fewer negative symptoms (Klingberg et al. 2011; Thomas et al. 2011), each associated with better outcome. We identified minimal literature about eligibility or suitability for FPE. It was assumed that current or recent family contact was an eligibility criterion and frequent family contact was a suitability criterion (e.g. Lehman et al. 2004; Dixon et al. 2010; NICE, 2014). For RPP, some CPGs (Lehman et al. 2004; NICE, 2014) link eligibility with recurrent episodes of disorder; however, this has not been universally adopted (e.g. Agius et al. 2007), and would exclude some patients with significant relapse risk (Alvarez-Jimenez et al. 2012). Hence we adopted no eligibility criterion. Past history (Herz et al. 2000; Gumley et al. 2003) and recency (Garety et al. 2008) of relapse have driven selection in trials, thus were included as suitability indicators.

Functional disability is a clear eligibility criterion for ST (Lehman et al. 2004; Dixon et al. 2010). Although better cognition (e.g. Ucok et al. 2006), learning skills (Silverstein et al. 2005) and neuropsychological functioning (Granholm et al. 2008) are associated with improved outcomes, ST is intended for consumers with deficits that may be associated with these factors so we did not consider them as suitability variables. Instead, we opted for indicators related to greater need: social skills difficulties and poorer role functioning. The dominant model of SE, Individual Placement and Support, has one eligibility indicator – the individual's desire to work (Bond, 2004). For suitability indicators, good evidence exists that better prior work history and education beyond secondary level are associated with better vocational outcomes (e.g. Campbell et al. 2010; Major et al. 2010). The target population for ACT is variously described as patients with longer term and complex mental health problems, difficulties in engaging with community mental health services and/or experiencing frequent admissions (Dixon, 2000; Cuddeback et al. 2006; Aagaard & Muller-Nielsen, 2011; Brugha et al. 2012; Kortrijk et al. 2012; Killackey et al. 2015). We adopted absence of longer-term illness (i.e. single episode with good recovery) as the least exclusionary eligibility criterion and used the following suitability indicators: difficulties with engagement; frequent hospital admissions; co-morbid substance use; poor course of disorder; and poor functioning.

It is likely that demographic, clinical and service characteristics may be generally predictive of receipt of EBPIs – identifying these may indicate barriers and enablers for EBPIs. For example, having a first language other than English, symptom severity, and poorer cognitive functioning may impact engagement in, or benefit from, any EBPI. Service characteristics were also included as potential predictors since service provision is brokered by case managers and may be enabled by the involvement of an NGO (non-clinical mental health support service) (Harvey et al. 2016).

To better understand receipt of six EBPIs in Australia, and the characteristics of those who receive them, we used available variables from the dataset collected within the second Australian national psychosis survey (Morgan et al. 2012) to address the following questions:

• 1.How prevalent is receipt of one or more EBPIs within a 12-month period in a treated sample of people with psychotic disorders?
• 2.Are patient eligibility and suitability indicators for an EBPI associated with receipt of that intervention?
• 3.Are demographic and clinical characteristics of consumers, and service characteristics, associated with receipt of EBPIs?

Method

Materials and methods

Survey and participants

All participants took part in the Survey of High Impact Psychosis (SHIP). This second Australian national survey of psychosis covered seven catchment areas (total area 62 000 square kilometres) with a population of 1.5 million people aged 18-64 years. A two-phase design was used (Morgan et al. 2012). During the census month (March 2010), screening for psychosis occurred in public specialised mental health services and in non-government organisations supporting people with a mental illness. In addition, those with a recorded diagnosis of psychosis and in contact with clinical services during the previous 11 months were identified from administrative records. People who screened positive for psychosis were randomly selected, stratified by age group (18–34 years and 35–64 years), for interview and assessment. Of 7955 people who were screen positive and eligible, 1825 were interviewed (for full details, see Morgan et al. 2012). The study was approved by institutional human research ethics committees at each site. All participants provided written informed consent.

Measures and data handling

The interview schedule probed: demographic characteristics, education, employment, housing, symptomatology, substance use, functioning and disability, physical health, use of mental health services and medication. Externally developed instruments were used, along with questions from the previous national low prevalence disorders survey (Jablensky et al. 2000) and new questions developed specifically for the 2010 survey (Morgan et al. 2012). The diagnosis was based on the Diagnostic Interview for Psychosis (DIP-DM) (Castle et al. 2006). The number of negative symptoms was based on the Carpenter Deficits syndrome from the World Health Organization Schedules for Clinical Assessment in Neuropsychiatry, without taking attribution into account (Kirkpatrick et al. 1989; World Health Organisation, 1999). The Personal and Social Performance Scale (Morosini et al. 2000) assessed behavioural and social functioning and role performance over the previous year. The course of disorder was rated by the interviewers based on the entire interview. Premorbid and current cognitive ability were assessed using the National Adult Reading Test-Revised (Nelson & Willison, 1991) and the Digit Symbol Coding Test from the Repeatable Battery for the Assessment of Neuropsychological Status (Randolph et al. 1998), respectively.

Receipt of EBPIs

Survey questions about receipt of six EBPIs (CBTp, FPE, RPP, ST, SE and ACT) were crafted to describe each from a participant perspective. The formal names of the interventions were not used since they may not be known or correctly applied by participants. The questions, reproduced in Table 1, aimed to provide sufficient information to briefly capture the essence of the intervention whilst minimising information processing demands. An evidence-based level was specified for each intervention, except for ACT since the intervention question implied sufficient duration. These levels reflected the minimum specified in CPGs or meta-analyses, discounted by about one-third in order to minimise false negatives since some participants would not have completed a current intervention at the time of interview.

Eligibility and suitability indicators for EBPIs

Existing SHIP questions were examined for their utility as eligibility and suitability indicators for each EBPI; it was not possible to add further items to the already lengthy SHIP survey to reflect all the identified eligibility and suitability indicators. However, one eligibility item was identified for each of four EBPIs and 2–6 suitability items for each EBPI (see Table 1).

General predictors of receipt of interventions

Demographic, clinical and service characteristics collected by the survey and likely to be associated with receipt of any EBPI are listed in Table 2.

Table 2.

Profile of participants (n = 1825) on selected demographic, clinical and service characteristics, and suitability indicators for evidence-based psychosocial interventions (EBPIs)

Demographic characteristics	N reporting	n (%)
Age group	1825
18–34 years		773 (42.4)
35–65 yearsa		1052 (57.6)
Male	1825	1087 (59.6)
In a marital or de facto relationship	1825	312 (17.1)
English is first language	1825	1657 (90.8)
Completed 12 years of education	1802	574 (31.9)
Clinical characteristics	N reporting	n (%)
Personal and social performance (PSP) score	1825
Poor functioning (0–30)		90 (4.9)
Varying degrees of disability (31–70)		1412 (77.4)
Absence of disability or only mild difficulties (71–100)		323 (17.7)
Course of disorder	1825
Single episode		147 (8.1)
Multiple episodes with good recovery		542 (29.7)
Multiple episodes with partial recovery		580 (31.8)
Continuous chronic illness		376 (20.6)
Continuous chronic illness with deterioration		180 (9.9)
Insight present about disorder (lifetime)	1825	1389 (76.1)
Lifetime diagnosis of alcohol abuse/dependence (Yes)	1825	921 (50.5)
Lifetime diagnosis of other substance abuse/dependence (Yes)	1825	995 (54.5)
Non-Affective Psychosis	1825	1150 (63.0)
Positive symptoms (present state) (Yes)	1825	1015 (55.6)
Number of negative symptoms (past year)	1825
0–1		534 (29.3)
2–4		889 (48.7)
5–6		402 (22.0)
Digit Symbol Coding Test	1609
More than 1 s.d. below mean		223 (14.4)
Within 1 s.d. of mean		1124 (69.4)
Greater than 1 s.d. above mean		262 (16.2)
National Adult Reading Test Full Scale IQ estimate	1546
More than 1 s.d. below mean		267 (17.3)
Within 1 s.d. of mean		1000 (64.7)
Greater than 1 s.d. above mean		279 (18.0)
Service Characteristics	N reporting	n (%)
Accessed Non-Government Organisation (past year) (Yes)	1802	544 (30.2)
Community Treatment Order in past year (Yes)	1825	350 (19.2)
Case manager	1100
Medical registrar		77 (7)
Nurse		549 (49.9)
Psychologist		72 (6.5)
Social worker		310 (28.2)
Occupational therapist		92 (8.4)
Satisfaction with a case manager	1261
Very satisfied		806 (63.9)
Somewhat satisfied		279 (22.1)
Neither satisfied nor dissatisfied		84 (6.7)
Somewhat dissatisfied		49 (3.9)
Very dissatisfied		43 (3.4)
Suitability indicators	N reporting	n (%)
Poor functioning (PSP < 31)	1825	90 (4.9)
Two or more mental health admissions (past year)	1825	225 (12.3)
Uncontrolled symptoms of mental illness in top three challenges in next year (Yes)	1825	469 (25.7)
Substance use in past year (No)	1809	1139 (63.0)
Mental health admission in past year (Yes)	1809	626 (34.6)

Note: ^aThe sample was limited to persons aged 18–64, but one participant had turned 65 by the time of the interview.

s.d., standard deviation.

Analysis

Univariate relationships between receipt of individual EBPIs and their respective suitability indicators were examined via correlations for continuous variables or χ² for categorical variables, after checking that assumptions were met.

Receipt of one or more EBPIs was predicted using hierarchical logistic regression. Four blocks of independent variables were entered: demographic variables; lifetime clinical and substance use variables; mental health symptomatology; and service provision variables. Since ACT differs in being a service model rather than a single intervention, we re-ran the model excluding ACT.

Separate hierarchical logistic regressions were run for each EBPI. Where eligibility indicators were identified, the participant sample was restricted to those who met these. The same four blocks of independent variables were used as for the first regression analysis; a fifth block was added to include specific suitability indicators if identified in univariate analyses.

Variables in blocks one to four (demographic; lifetime clinical and substance use; mental health symptomatology; and service provision) were chosen on a priori grounds. Given that our earlier univariate suitability analyses were designed to identify suitability indicators amongst a larger pool of putative suitability indicators, we included only those which were significantly related to the relevant EBPI in block 5.

Most variables were binary (e.g. no/yes) or continuous (e.g. 0–100); ordinal variables (course of disorder and satisfaction with case manager) were treated as continuous in regression analyses.

Due to reduced sample sizes for satisfaction with a case manager and cognitive functioning variables, these predictors were only used in secondary analyses to avoid limiting the sample for the main analyses.

Each variable was inspected individually for outliers with no differences at the 5% trimmed mean level beyond the criterion of 0.2. Hierarchical logistic regression assumptions were met with >40 cases per predictor and absence of multi-collinearity (all correlations <0.5; tolerance scores >0.1). In the final model, predictors were identified using p < 0.05 as a cut-off, with odds ratios and confidence intervals examined. Our criterion for acceptance of a model as useful was set at 25% better than the chance prediction.

Data analysis was conducted using IBM SPSS for Windows Version 21 (SPSS., 2009).

Results

Profile of people with psychosis in the Survey of High Impact Psychosis

Survey participants were predominantly single (61.2%) and male (59.6%). Less than one third (31.5%) had completed the final year of schooling and almost one third (32.7%) were in paid employment at some time during the year prior to the interview. Most (63.2%) had obvious or severe dysfunction in their capacity to socialise over the past year. Almost half had a diagnosis of schizophrenia (47.0%) and most had experienced multiple episodes of psychosis with periods of good or partial recovery between (61.5%) (for a detailed description of the overall sample: Morgan et al. 2012). Table 2 presents demographic, clinical and service characteristics, and suitability indicators for participants.

Receipt of individual EBPIs

Table 3 presents receipt and eligibility data. Receipt of each intervention was reported by between 12 and 41% of participants in the past year: up to about one fifth reported an evidence-based level. RPP was the most frequently reported intervention (41.3%; 21.1% at evidence-based level) and FPE the least (11.6% and 3.4%, respectively). With the exception of ST and RPP, an evidence-based level accounted for less than half of the reported receipt. Applying eligibility indicators (where applicable) made little difference to rates of reported receipt.

Table 3.

Receipt of any, and evidence-based levels of, psychosocial interventions, including by eligibility

Evidence-based psychosocial intervention (EBPI)	Receipt of any level		Receipt of evidence-based levela		EBPI receipt by those eligible
	n (%)	Total	n (%)	Total	n (%)	Total
CBT for psychosis	407 (22.9)	1776	180 (10.1)	1776	N/Ab	N/A
Family Psycho-Education	208 (11.6)	1783	61 (3.4)	1783	61 (3.6)	1753
Relapse Prevention Planning	725 (41.3)	1755	369 (21.1)	1746	N/Ab	N/A
Skills Training	268 (15.1)	1777	168 (9.5)	1777	151 (10.1)	1502
Supported Employment	88 (15.0)	584c	41 (7.0)	584	41 (7.0)	584c
Assertive Community Treatment	N/A	N/A	161 (8.9)	1802	148 (8.8)	1678

^aSee Table 1 for criteria for levels of evidence-based provision.

^bNo eligibility criteria applied for this intervention.

^cReceipt of employment support was only asked of those actively looking for work and/or employed during the past year. This matched our identified eligibility criterion of having an employment goal.

Relationships between receipt of individual EBPIs and their putative suitability indicators

Table 4 presents the suitability indicators for each intervention and the proportion of eligible participants who met each. With the exception of ACT, the proportion meeting each suitability indicator was relatively high, ranging from 18% with deteriorated relationships due to illness (ST) to 92% with multiple episodes or continuous illness (RPP). Each EBPI had at least one suitability indicator met by a majority of participants. Associations between each suitability indicator and receipt of the corresponding intervention are reported: three of the six interventions showed associations (CBTp, RPP, ACT). Of 19 suitability indicators tested, five were significantly associated with receipt of the respective EBPI (see Table 4).

Table 4.

Suitability indicators for psychosocial interventions and associations with receipt

	Suitability indicator met		Association with receipt
Suitability indicators for each intervention	n (%)	Total	χ2a	p	Total
CBT for psychosis
Top three challenge of uncontrolled symptoms	469 (25.7)	1825	16.39	0.000	1776
Lack of relief from medication	635 (39.8)	1594	NS
Low Carpenter negative syndrome score	534 (29.3)	1825	NS
Absence of substance use	1139 (63.0)	1809	5.146	0.023	1764
Family Psycho-Education
Living with any relative	851 (48.8)	1753	N/A
At least weekly face to face family contact	1343 (76.6)	1753	N/A
Composite indicatorb	839 (47.9)	1753	NS
Relapse Prevention Planning
Any mental health admission past year	626 (34.6)	1809	17.63	0.000	1745
Multiple episodes or continuous illnessc	1678 (91.9)	1825	NS
Skills Training
Illness interferes with desired relationships	574 (39.5)	1452	NS
Dysfunction in overall socialising	1031 (68.7)	1501	NS
Relationships deteriorated due to illness	261 (17.6)	1484	NS
Poorer independent living role performance	592 (32.4)	1825	3.02	0.082	1777
Supported Employment
Wanted help to work or use time	175 (22.2)	787	NS
Post-school qualification	443 (56.3)	787	NS
Assertive Community Treatment
Poor functioning (PSP<31)	89 (5.3)	1678	4.28	0.039	1658
Very unconfident medication a good thing	87 (5.4)	1612	2.94	0.086	1515
Multiple episodes with partial recovery or continuous illnessc	1136 (62.2)	1678	NS
Two or more admissions in past year	213 (12.7)	1678	16.935	0.000	1658
Any substance use in past year	616 (36.7)	1678	NS
Very dissatisfied with case manager support	38 (3.6)	1058	3.75	0.053	1058

^aχ² results reported where p < 0.10 (df = 2).

^bmet ‘living with any relative’ and ‘at least weekly face to face contact’.

^cThese aspects of course of disorder were rated by the interviewers based on the entire interview.

Prediction of receipt of one or more EBPIs

Hierarchical logistic regression was performed to assess the impact of demographic, clinical and service provision characteristics on the likelihood that participants would be in receipt of one or more EBPIs. The full model containing all predictors was statistically significant (χ² (20, n = 1746) = 216.12, p < 0.01), indicating that it could differentiate between participants who were, and were not, in receipt of one or more EBPIs. The model explained between 11.6% (Cox & Snell R square) and 15.7% (Nagelkerke R square) of the variance in group status, correctly classifying 66% of cases. Entry of demographic variables at block 1 and service provision variables at block 4 improved the model fit (block 1: 1.5% (Cox and Snell R square) and 2.0% (Nagelkerke R square) of variance explained, 59.8% correct classification; block 2: 2.1% (Cox and Snell R square) and 2.9% (Nagelkerke R square) of variance explained, 60.8% correct classification; block 3: 2.2% (Cox and Snell R square) and 3.0% (Nagelkerke R square) of variance explained, 60.9% correct classification).

The proportional by chance accuracy rate was 52%. Using the criterion of a 25% or better prediction beyond chance for model usefulness (i.e. 1.25 × 52 = 65%), the prediction of 66% of cases at the final step can be considered useful, albeit marginally.

As shown in Table 5, nine variables made a unique contribution to the model. The strongest predictor of receipt of one or more EBPIs was being assigned a psychologist as a case manager, OR(CI) = 2.36(1.50–3.72). Accessing an NGO in the past year was also a moderately strong predictor: those who had accessed an NGO were twice as likely to have received one or more EBPIs (OR(CI) = 2.01(1.60–2.51)).

Table 5.

Hierarchical logistic regression: Receipt of one or more evidence-based psychosocial interventions v. no evidence-based psychosocial interventions received (n = 1746)

Predictor	B	s.e.	p	OR	CI (L,U)
Block 0
Constant	−0.40	0.05	<0.01	0.67
Block 1
Sex (male/female)	0.44	0.13	<0.01	1.55	1.27–1.90
Age (18–65)	−0.01	0.01	<0.01	0.99	0.98–0.99
In a marital/de facto relationship (no/yes)	−0.08	0.13	0.53	0.92	0.71–1.20
Completed Year 12 (no/yes)	0.05	0.11	0.62	1.01	0.86–1.30
First Language English (no/yes)	−0.17	0.18	0.35	0.81	0.58–1.15
Constant	−0.09	0.19	0.63	0.91
Block 2
Personal and Social Performance (PSP) score (0–100)	0.00	0.00	0.48	0.00	0.99–1.00
Course of disordera	0.08	0.05	0.10	0.08	0.99–1.19
Insight about disorder (no/yes)	−0.26	0.12	0.03	1.30	1.03–1.65
Lifetime alcohol abuse/dependence (no/yes)	−0.02	0.11	0.87	0.98	0.79–1.23
Lifetime other substance abuse/dependence (no/yes)	−0.18	0.12	0.13	0.84	0.67–1.06
Type of psychosis (non-affective/affective)	0.10	0.11	0.33	0.11	0.90–1.37
Constant	0.13	0.36	0.71	0.14
Block 3
Any positive symptoms in past 4–6 weeks (no/yes)	0.02	0.11	0.87	1.02	0.82–1.26
Carpenter negative syndrome score (0–6)	0.03	0.03	0.33	1.03	0.97–1.09
Constant	0.00	0.38	0.99	1.00
Block 4 (final)
Accessed NGO in past year (no/yes)	0.70	0.11	<0.01	2.01	1.60–2.51
Community treatment order in past year (no/yes)	0.31	0.14	0.02	1.37	1.04–1.79
Case manager – medical registrar (no/yes)	−0.76	0.24	<0.01	0.47	0.29–0.74
Case manager – nurse (no/yes)	0.22	0.11	0.04	1.25	1.01–1.54
Case manager – psychologist (no/yes)	0.86	0.23	<0.01	2.36	1.50–3.72
Case manager – social worker (no/yes)	0.16	0.13	0.21	1.17	0.92–1.50
Case manager – occupational therapist (no/yes)	0.47	0.20	0.02	1.59	1.08–2.34
Constant	−0.14	0.41	0.73	0.87

Evidence-based psychosocial interventions: CBT for psychosis, Family Psycho-Education, Relapse Prevention Planning, Skills Training, Supported Employment, Assertive Community Treatment.

Bold indicates a significant p value of <0.05.

^asingle episode with good recovery; multiple episodes with good recovery; multiple episodes with partial recovery; continuous, chronic illness; continuous, chronic illness with deterioration.

Participants with occupational therapists (OR(CI) = 1.59(1.08–2.34)) and nurses (OR(CI) = 1.25(1.01–1.54)) as case managers were also more likely to have received one or more EBPIs whereas participants with medical registrars as case managers were less likely to have done so (OR(CI) = 0.47(0.29–0.74)).

Despite comprising less than half the sample (40.5%), women were 55% more likely than men to receive an EBPI (OR(CI) = 1.55(1.27–1.90)). Participants who had insight about their disorder were more likely to have received one or more EBPIs (OR(CI) = 1.30(1.03–1.65)) as were younger aged clients, compared with older clients (OR(CI) = 0.99 (0.98–0.99)), albeit very slightly – the chance of receiving an EBPI decreased 1% for each additional year of age.

Participants who had received treatment via a community treatment order in the past year were slightly more likely to have received one or more EBPIs (OR(CI) = 1.37(1.04–1.79)).

Secondary analyses showed that satisfaction with case manager was a significant predictor; however, including this variable in the model decreased overall prediction because of the associated reduction in sample size. The cognitive variables (DSCT and NART) were also examined in an additional analysis, however, were not found to be significant predictors.

Given the nature of ACT as a service type rather than a discrete psychosocial intervention as were the other EBPIs, we re-ran the model excluding ACT. The overall level of prediction improved slightly (67.8% of cases compared with 66%) although R² estimates reduced. The predictors remained unchanged except for ‘case manager – nurse’ and ‘community treatment order in the past year’, which were no longer statistically significant.

Prediction of individual EBPIs

Although the equations for all but FPE were statistically significant, none met the 25% greater than chance criterion and the amount of variance explained was low (Table 6). Depending on the method, the variance explained ranged from 1.7% for FPE (Cox and Snell R square) to 17.6% for SE (Nagelkerke R square) (Table 6). Significant predictors varied across interventions. For example, predictors of receipt of ST were: not being in a marital/de facto relationship; the presence of insight; accessed NGO in past year; treatment via CTO in past year; and having an OT as a case manager. Whereas, for receipt of SE, predictors were: completed Year 12 education; the better course of disorder; and, accessed NGO in past year (see Table 6). Apart from insight, which was associated with a greater likelihood of receipt of CBTp, RPP and ST and a lesser likelihood of receipt of ACT, no demographic, clinical or service characteristic was significant for more than three of the six EBPIs. Each of the suitability indicators identified previously as being associated with an intervention was confirmed as a significant predictor when added in the last block (that is, uncontrolled symptoms as a top three challenge in the next year as a predictor of receipt of CBTp, any mental health admission in the past year for RPP, and two or more admissions in the past year for ACT). The strongest service predictor, and the strongest identified predictor overall was having a psychologist as case manager: this was associated with four times the likelihood of receiving CBTp.

Table 6.

Hierarchical logistic regression models for receipt of each evidence-based psychosocial intervention with odds ratios for predictors significant at p < 0.05, and model statistics

Predictor	CBT for psychosis (n = 1403)	Family Psycho-Education (n = 1660)	Relapse Prevention Planning (n = 1691)	Skills Training (n = 1412)	Supported Employment (n = 748)	Assertive Community Treatment (n = 1636)
Demographic characteristics
Sex (male/female)	2.21 (1.60–3.05)	–	1.43 (1.12–1.82)	–	–	–
Age (18–65)	–	–	0.98 (0.97–0.99)	–	–	–
In a marital/de facto relationship (n/y)	–	–	–	0.47 (0.25–0.89)	–	–
Completed Year 12 (n/y)	1.60 (1.16–2.21)	–	–	–	3.26 (1.64–6.49)	–
First Language English (n/y)	–	–	–	–	–	–
Clinical characteristics
Personal and Social Performance (PSP) Score (0–100)a	1.01 (1.00–1.03)	–	–	–	–	0.97 (0.96–0.99)
Course of disorderb	1.27 (1.08–1.50)	–	–	–	1.45 (1.04–2.02)	0.79 (0.65–0.95)
Insight about disorder (n/y)	2.27 (1.31–3.95)	–	1.44 (1.07–1.96)	1.81 (1.16–2.82)	–	0.65 (0.44–0.95)
Lifetime alcohol abuse/dependence (n/y)	1.47 (1.01–2.13)	–	–	–	–	–
Lifetime other substance abuse/dependence (n/y)	0.55 (0.38–0.81)	–	–	–	–	–
Type of psychosis (non-affective/affective)	2.51 (1.79–3.52)	–	–	–	–	–
Any positive symptoms past 4–6 weeks (n/y)	–	–	–	–	–	–
Carpenter negative syndrome score (0–6)	–	–	0.91 (0.84–0.98)	–	–	1.14 (1.02–1.26)
Service characteristics
Accessed Non-Government Organisation (NGO) in past year (n/y)	–	–	1.69 (1.30–2.20)	2.16 (1.51–3.09)	3.03 (1.49–6.18)	–
Community treatment order in past year (n/y)	–	–	1.40 (1.03–1.90)	2.58 (1.42–4.70)	–	2.64 (1.79–3.87)
Case manager – medical registrar (n/y)	0.36 (0.13–0.99)	–	0.54 (0.29–0.98)	–	–	n/ac
Case manager – nurse (n/y)	–	2.03 (1.11–3.72)	1.36 (1.06–1.74)	–	–	n/ac
Case manager – psychologist (n/y)	4.39 (2.65–7.26)	–	1.85 (1.18–2.90)	–	–	n/ac
Case manager – social worker (n/y)	–	2.06 (1.07–3.97)	1.43 (1.08–1.88)	–	–	n/ac
Case manager – occupational therapist (n/y)	–	–	–	1.69 (1.01–2.81)	–	n/ac
Suitability indicatorsa
Two or more admissions past year (n/y)	n/a	n/a	n/a	n/a	n/a	1.78 (1.14–2.77)
Top 3 challenge of uncontrolled symptoms (n/y)	1.48 (1.03–2.13)	n/a	n/a	n/a	n/a	n/a
Any admission past year (n/y)	n/a	n/a	1.40 (1.07–1.82)	n/a	n/a	n/a
Model statistics
χ2 (df, p)	145.82 (22, <0.001)	29.10 (20, 0.09)	135.07 (21, <0.001)	80.38 (20, <0.001)	45.54 (20, 0.001)	79.44 (16, <0.001)
Variance explained	8.2%, 16.9%	1.7%, 6.5%	7.7%, 12.0%	5.5%, 11.2%	5.9%, 17.6%	4.7%, 10.5%

^aPoor functioning (PSP) was a statistically significant univariate suitability indicator for Assertive Community Treatment, but PSP score was included in each of these models in step 2 (clinical characteristics) rather than step 4 as it was hypothesised to be a broad clinical predictor.

^bSingle episode with good recovery; multiple episodes with good recovery; multiple episodes with partial recovery; continuous, chronic illness; continuous, chronic illness with deterioration.

^cCurrent case manager cannot be a predictor of entry to current service.

Discussion

People living with psychosis reported limited receipt of EBPIs during a year of treatment in specialist mental health services: fewer than half received each intervention and less than one-fifth received an evidence-based level. This includes RPP, which might be assumed to be frequently indicated and routinely available in this setting, and FPE which, although recommended by CPGs in Australia since 1984, was reported at an evidence-based level by fewer than 4%. Ineligibility did not explain this limited receipt. We found only patchy evidence for targeting of psychosocial interventions to those who might benefit most; less than one-quarter of suitability indicators were associated with receipt of the relevant EBPI, suggesting that targeting or uptake was largely due to other factors. Our model predicted receipt of one or more EBPIs, correctly classifying 66% of participants. Interestingly, clinical predictors, such as symptomatology, had little predictive value, whereas service provision characteristics – accessing a non-clinical mental health support service (NGO), being on an involuntary community treatment order and the case manager's profession – made the greatest contribution to prediction. This reinforces other research demonstrating the influence of service characteristics on the delivery of EBPIs (Magliano et al. 2006). This suggests that the evidence for clinical factors predicting outcomes (Granholm et al. 2008; Thomas et al. 2011; Kortrijk et al. 2012) may be overlooked in decisions about providing EBPIs to people with psychosis.

Previous reports of limited receipt of EBPIs by people with psychosis are reinforced (Bond et al. 2000; Killackey et al. 2008; Prytys et al. 2011; Harris & Boyce, 2013; Haddock et al. 2014). Interventions to improve relationships (FPE, ST) were rarely reported and apparently poorly targeted, suggesting multiple barriers to their implementation should be considered, including organisational factors and negative staff attitudes (Haddock et al. 2014; Ince et al. 2016; Magliano et al. 2017). Reported receipt of SE was also low, perhaps reflecting the poor integration of employment and mental health supports in Australia (Killackey & Waghorn, 2008).

In contrast, receipt of ACT (8.9%) was similar to that reported in a US study of veterans (7%, McCarthy et al. 2009), with evidence of targeting to those likely to be at risk for hospitalisation. Several explanations are possible: ACT is a well-defined intervention with clear fidelity criteria (Monroe-DeVita et al. 2011); team-based interventions, such as ACT, may be easier to implement and sustain; and, ACT has been subject to more systematic implementation efforts in Australia (Harvey et al. 2012; Monroe-DeVita et al. 2012). Thus, ACT implementation may provide useful lessons for the implementation of other EBPIs.

Suitability is a complex notion. Current recommendations, such as for FPE (Galletly et al. 2016) and CBTp (NICE, 2014), are widely inclusive; yet it seems likely that some subgroups may not accept, and not all may benefit from, these interventions (Barrowclough et al. 2010; Klingberg et al. 2011; Thomas et al. 2011). Research has not addressed the ideal proportion of people with psychosis who should receive each intervention, making evidence-practice gaps difficult to quantify. Further, the RCTs generating efficacy data are typically underpowered to examine outcome predictors (Menon et al. 2015). Future studies predicting outcome and suitability may prompt better targeting of interventions such as tailoring CBTp for ethnic minorities (Rathod et al. 2013).

Having a psychologist as a case manager was the strongest predictor of receipt of one or more EBPIs, and also strongly predicted receipt of CBTp and RPP in the individual regressions. Psychologists are trained as ‘scientist-practitioners’ (McDermott et al. 2012), likely resulting in awareness of, and skills to deliver, EBPIs. Although negative attitudes to the implementation of psychosocial interventions amongst practitioners, including psychologists, have been reported (Ince et al. 2016; Magliano et al. 2017), this is not a uniform finding. It is possible that the relatively few psychologists who seek to work with people experiencing psychosis in Australia represent a subgroup with more positive attitudes. Linked with this, psychologists may be especially likely to seek to work with, or be allocated to, patients who might be more easily engaged in EBPIs in services where allocations take account of professional interests and skills. It is also possible that the choice of case manager reflects the fact that the patient is already in receipt of therapy delivered by a particular professional, although we were unable to further explore any of these possibilities. By contrast, patients perceived to have few psychosocial needs may be allocated to medical case managers in services where the medical role has a narrower bio-medical focus. A less generous interpretation is that although the role of case manager includes ensuring that EBPIs are offered on the basis of consumer needs, case managers’ comfort or familiarity with EBPIs may lead to favouring interventions related to their own profession. Profession-specific biases potentially detract from optimal treatment.

Our regression models predicting receipt of individual EBPIs were statistically significant for all but FPE; however, the variance explained was small. Nonetheless, they confirmed the relevance of the selected suitability indicators for individual EBPIs and may provide clues to factors that influence receipt. Receipt of RPP, ST and SE were each associated with having accessed an NGO. This appears consistent with these services’ focus on wellness promotion, practical support for recovery and community re-integration. Better insight was positively associated with CBTp, RPP and ST suggesting that a degree of insight may be required for interventions fostering self-management and skills.

Strengths and limitations of the study

To our knowledge, this is the first study using epidemiological data to examine receipt of EBPIs by people living with psychosis. Strengths include the use of the large representative sample from the Australian national survey and its suite of variables for modelling purposes. Nonetheless, the study has a number of limitations. Our choice of a ‘least exclusionary’ eligibility criterion for each EBPI was intended to prioritise awareness of lack of receipt of each EBPI for all who could benefit. However, this may have been at the expense of over-estimating the number eligible. Some suitability indicators noted in the relatively sparse literature were not available for these analyses or were approximations. The study should be regarded as exploratory, therefore, especially since data are cross-sectional. There are also ‘common elements’ shared by some of these psychosocial interventions (e.g. identification and monitoring of individual Early Warning Signs of relapse is included in some FPE models, as well as being central to RPP) (Chorpita et al. 2007). Therefore, we cannot rule out the possibility of partial overlap between those psychosocial interventions in data collection. The study design also did not allow us to explore whether the psychosocial interventions were considered, offered and refused or not offered by case managers. Building on Haddock et al.’s (2014) small study would be valuable. This limitation is also being addressed in a small sub-study currently being prepared for submission. Similar to other studies relying on reports of provision (e.g. West et al. 2005), our data were reliant on participant self-report and therefore subject to recall bias. This may have led to underestimating receipt if the description of the intervention was not recognised by the participant; an over-estimate of evidence-based receipt is also plausible, considering we were unable to assess intervention quality.

Conclusions

Australians living with psychoses report receipt of EBPIs, but with the exception of the ACT, at lower levels than recommended by evidence-based CPGs, and relatively unrelated to eligibility and suitability indicators. Greater implementation effort and better targeting are likely required. Recognised implementation strategies might be usefully drawn upon (Monroe-DeVita et al. 2012). These include policies which define program standards and build in contingencies and incentives for implementation of psychosocial interventions. Education, training and consultation are also essential to strengthen evidence-based practice literacy in the professions, particularly those other than psychologists, to guide individual service planning (Harris & Boyce, 2013; Magliano et al. 2017). Needs-based system re-design may also be needed (Harris & Boyce, 2013; Ince et al. 2016), drawing upon characteristics of mental health service systems that effectively implement evidence-based interventions. The study highlights how little we know about receipt, targeting and benefits of EBPI provision. Future research should model a better-targeted service delivery system taking account of patient needs over time (McGorry et al. 2006), and the relative benefit and opportunity costs from EBPI provision (Pandiani et al. 2004).

Financial support

The study was funded by the Australian Government Department of Health and Ageing.

Conflicts of interest

None.

Ethical standards

The study was approved by institutional human research ethics committees at each site. All participants provided written informed consent.

Availability of data and materials

Access to the data and materials on which this study is based is authorised through the SHIP Access and Publication Committee. The Committee Convenor is Professor Vera Morgan (vera.morgan@uwa.edu.au) and Executive Officer is Assistant Professor Anna Waterreus (anna.waterreus@uwa.edu.au).