Lack of evidence to favor specific preventive interventions in psychosis: a network meta‐analysis

Abstract

Preventing psychosis in patients at clinical high risk may be a promising avenue for pre‐emptively ameliorating outcomes of the most severe psychiatric disorder. However, information on how each preventive intervention fares against other currently available treatment options remains unavailable. The aim of the current study was to quantify the consistency and magnitude of effects of specific preventive interventions for psychosis, comparing different treatments in a network meta‐analysis. PsycINFO, Web of Science, Cochrane Central Register of Controlled Trials, and unpublished/grey literature were searched up to July 18, 2017, to identify randomized controlled trials conducted in individuals at clinical high risk for psychosis, comparing different types of intervention and reporting transition to psychosis. Two reviewers independently extracted data. Data were synthesized using network meta‐analyses. The primary outcome was transition to psychosis at different time points and the secondary outcome was treatment acceptability (dropout due to any cause). Effect sizes were reported as odds ratios and 95% confidence intervals (CIs). Sixteen studies (2,035 patients, 57% male, mean age 20.1 years) reported on risk of transition. The treatments tested were needs‐based interventions (NBI); omega‐3 + NBI; ziprasidone + NBI; olanzapine + NBI; aripiprazole + NBI; integrated psychological interventions; family therapy + NBI; D‐serine + NBI; cognitive behavioural therapy, French & Morrison protocol (CBT‐F) + NBI; CBT‐F + risperidone + NBI; and cognitive behavioural therapy, van der Gaag protocol (CBT‐V) + CBT‐F + NBI. The network meta‐analysis showed no evidence of significantly superior efficacy of any one intervention over the others at 6 and 12 months (insufficient data were available after 12 months). Similarly, there was no evidence for intervention differences in acceptability at either time point. Tests for inconsistency were non‐significant and sensitivity analyses controlling for different clustering of interventions and biases did not materially affect the interpretation of the results. In summary, this study indicates that, to date, there is no evidence that any specific intervention is particularly effective over the others in preventing transition to psychosis. Further experimental research is needed.

Individuals at clinical high risk for psychosis (CHR‐P)1 present with attenuated psychotic symptoms, impairments of social, emotional and cognitive functioning2, and help‐seeking behaviour3. They have around 20% risk of developing psychosis (but not any other non‐psychotic disorder4, 5) over a two‐year period6.

Primary indicated prevention in CHR‐P individuals has the unique potential to alter the course of the disorder7 and improve clinical outcomes8. Current international guidelines – such as those of the National Institute for Health and Care Excellence (NICE) and the European Psychiatric Association (EPA) – recommend that CHR‐P individuals be primarily offered cognitive behavioural therapy (CBT) with or without family interventions9, 10. However, while prophylactic treatment with antipsychotics is altogether prohibited by NICE guidelines9, the EPA allows its use in the case of severe and progressive symptomatology10.

The evidence supporting these partially conflicting recommendations is relatively unclear11, despite several pairwise meta‐analyses having been published to date10, 12, 13, 14, 15, 16, 17, 18. For example, earlier meta‐analyses concluded that no reliable recommendations with respect to specific interventions could be made, because studies were too heterogeneous12, with comparable efficacy across different treatments16 or no effects at all17. The most recent meta‐analysis concluded that both CBT and antipsychotics are effective13. The other meta‐analyses were affected by mistakes19 or methodological limitations, such as the use of overall effect sizes computed across heterogeneous interventions of questionable clinical interpretability10, 12, 18, inclusion of patients not assessed with standard CHR‐P instruments (e.g., with schizotypal disorders20)12, 13, 15, 18, inclusion of non‐randomized and uncontrolled trials10, pooling of time‐dependent outcomes21 in the same group (e.g., 6 and 12 months18) or no time stratification at all13, or poor meta‐analytical approaches13. Meta‐analyses have acquired a major influence on clinical practice and guidelines22, so they can be particularly harmful if they are of suboptimal quality.

Another problem is that the included trials involved a variety of specific interventions12, which were inconsistently clustered in pairwise comparisons. For example, although CBT is an umbrella term for a plethora of heterogeneous strategies23, different CBT protocols have been lumped together, and the specific efficacy of each defining element or specific protocol remains unclear24.

The objective of this network meta‐analysis (NMA) was to summarize the available evidence about the specific efficacy of different preventive interventions in CHR‐P individuals. NMA offers additional benefits over standard pairwise analyses in that the comparative efficacy of specific interventions can be estimated and ranked, even when two treatments have never been compared directly head‐to‐head25. Furthermore, since NMA can improve the precision of estimates by allowing integration of both direct and indirect treatment effect estimates26, it is recommended over pairwise meta‐analyses by the World Health Organization as a basis for clinical guidelines27. Therefore, NMA should be considered the highest level of evidence in CHR‐P treatment guidelines28.

METHODS

The protocol for this study was registered on PROSPERO (CRD42017069550). The study was conducted in accordance with the PRISMA statement29.

Interventions included

We included all randomized controlled trials (RCTs) of pharmacological and/or non‐pharmacological interventions for CHR‐P individuals. We were a priori interested in the following non‐pharmacological interventions: CBT (various protocols), psychoeducation, family therapy, supportive counselling, needs‐based interventions (NBI), and integrated psychological therapies. We were also interested in the following pharmacological interventions: antipsychotics (olanzapine, risperidone, ziprasidone, aripiprazole) and novel/experimental pharmacotherapies (omega‐3 fatty acids and D‐serine). As indicated in the protocol, additional interventions emerging from the literature search were also considered (e.g., glycine and cognitive remediation).

The definition of the exact types of interventions is essential to reduce heterogeneity and produce robust informative results of direct clinical significance. As such, we first took each trial and carefully identified the treatment components that were characterizing each specific intervention, as detailed below.

Needs‐based interventions (NBI)

Since CHR‐P patients recruited in clinical trials are help‐seeking adolescents and young adults accessing clinical services, randomizing them to no treatment is not considered a reasonable or ethical option30. Defining “treatment as usual” in these samples is also challenging, because treatment is not standardized and largely depends on local service configurations and the availability of specific resources or competences.

We therefore used the most established and original definition of NBI employed by the founders of the CHR‐P paradigm, which focuses on the presenting symptoms and problems already manifest31. In accordance with this definition32, NBI may include any of the following components: a) supportive psychotherapy primarily focusing on pertinent issues such as social relationships and vocational or family problems; b) case management, providing psychosocial assistance with accommodation, education or employment; c) brief family psychoeducation and support; d) medications other than antipsychotics; and e) clinical monitoring and crisis management31, 33.

Cognitive behavioural therapy, French & Morrison protocol (CBT‐F)

The CBT‐F protocol34 is based on the principles developed by Beck35. The intervention is formulation‐driven, problem‐focused and time‐limited, with manualized strategies selected on the basis of the patient's prioritized problem. The key components include building engagement, collaborative goal‐setting and formulation, normalizing experiences, evaluating appraisals and core beliefs, and behavioural experiments34, 36.

Cognitive behavioural therapy, van der Gaag protocol (CBT‐V)

The protocol developed by van der Gaag et al37 essentially includes the French & Morrison protocol34, but with two additional components. These comprise psychoeducation about dopamine system supersensitivity and training/behavioural experiments on cognitive biases that may contribute to paranoia38. Further behavioural goals include sustaining school and work attendance, enhancing social relationships, and reducing cannabis use37.

Integrated psychological interventions, Bechdolf protocol (IPI)

The protocol developed by Bechdolf et al39 contains a number of components, including individual CBT‐F34, manualized group social skills training, computerized cognitive remediation to address thought and perception deficits, and manualized psychoeducational multi‐family group sessions39, 40.

Family‐focused therapy, Miklowitz protocol (FFT)

A family‐focused therapy (FFT) protocol, initially designed for those with or at risk of bipolar disorder, was adapted by Miklowitz et al41 for the CHR‐P population. The key components include psychoeducation and development of a prevention plan with the patient and family, sessions where the patient and family practice skills for better communication, and sessions focusing on enhancing problem solving skills41.

Pharmacological interventions

Pharmacological interventions included currently licensed medications, novel or experimental pharmacotherapies, and nutritional supplements.

Placebo

The placebo designation was reserved for placebo pills administered as pharmacological control conditions. Placebos were designed to match the active drug intervention in appearance but without the pharmacological compound of interest.

Nodes for the network meta‐analysis

The specific interventions listed above were pooled into “nodes” for the network meta‐analysis. Nodes were defined by the linear combination of any of the above specific interventions. Each individual pharmacological treatment was assigned to its own node. As indicated in the protocol, different dosages of the same drug/molecule were classed under the same node. Placebo was initially considered as a separate node from NBI. However, in line with the protocol, sensitivity analyses investigated the effect of alternate clustering of nodes (see statistical analysis).

Search strategy and selection criteria

We performed a multi‐step literature search using the following keywords: (risk OR prodromal OR prodrom* OR ultra high risk OR clinical high risk OR high risk OR genetic high risk OR at risk mental state OR risk of progression OR progression to first‐episode OR prodromally symptomatic OR basic symptoms) AND (psychosis) AND (RCT OR randomized controlled trial OR placebo controlled trial OR trial).

First, systematic searches were conducted in the Web of Science (which includes Web of Science Core Collection, BIOSIS Citation Index, KCI ‐ Korean Journal Database, MEDLINE, Russian Science Citation Index, and SciELO Citation Index), the Cochrane Central Register of Controlled Trials, and Ovid/PsychINFO databases, until July 18, 2017, with no restrictions on language or publication date.

Second, we used Scopus/Web of Science to search reference lists of retrieved articles and previously conducted systematic reviews and meta‐analyses. We manually searched for published and unpublished data in relevant conference proceedings, trial registries and drug‐approval agencies. In addition, we contacted study authors for supplemental data and searched the OpenGrey database for grey literature.

Abstracts identified by this process were then screened, and full‐text articles were retrieved for further inspection against the inclusion and exclusion criteria (as detailed a priori in the protocol). The literature search, study selection and data extraction were conducted by two authors (CD, UP) independently. During all stages, in the case of disagreement, consensus was reached through discussion with a third author (PFP).

Studies were eligible for inclusion when the following criteria were fulfilled: a) original articles, abstracts or pilot studies; b) RCTs (including cluster randomized trials, but excluding cross‐over studies); c) designed as blinded (either single‐ or double‐blind); d) conducted in CHR‐P individuals as established by validated assessments, i.e. Comprehensive Assessment of At‐Risk Mental States (CAARMS)42, Structured Interview for Psychosis‐risk Syndromes (SIPS)43, 44, Positive and Negative Syndrome Scale (PANSS)45, Brief Psychiatric Rating Scale (BPRS)46, or Early Recognition Inventory (ERIraos)47; e) comparing specific preventive interventions as defined above; and f) sample size of 10 or greater48.

The exclusion criteria were: a) reviews/non‐original data; b) studies lacking at least two compared groups; c) studies of first‐episode psychosis or other non‐CHR‐P groups; d) lack of data needed for meta‐analytical computation of the primary (transition) outcome (authors were contacted and asked to provide summary data); e) lack of proper randomization (quasi‐randomization, observational naturalistic studies); f) samples size < 10; and g) articles presenting overlapping, redundant data (for a particular outcome at the same time point). Specifically, in the case of overlapping samples, we used the largest one. Studies that were designed as blinded but could not maintain blinding during follow‐up (e.g., for psychological interventions) were not excluded.

Outcome measures and data extraction

The primary outcome was transition to psychosis. Due to the variable effect of time on transition risk6, 21, we stratified outcomes and analyses into 6 and 12 month follow‐up time points. Sample sizes were based on the numbers randomized to each arm, to prevent artificial inflation of transition risk6, 49. Participants who dropped out of individual studies after randomization were classified as non‐transitions6, 10, 14, 50.

Where studies did not report sufficient data to extract the primary outcome, we contacted the relevant authors. In the case of non‐response or where studies presented data graphically, numerical data were digitally extracted from the Kaplan‐Meier plots using a previously validated procedure51, 52, as defined in the protocol.

The secondary outcome was the acceptability of interventions (discontinuation due to any cause), indexed as the number of participants who dropped out of each arm for any reason following randomization, over the number randomized53, 54, 55.

In addition, we extracted the following information for each study: first author and year of publication, country, types of outcomes, intervention and control descriptions, study design, quality assessment (see below), intervention period and follow‐up duration, study arm details (sample size, mean age, percent male), and diagnostic tools used for CHR‐P diagnosis and determining transition to psychosis.

Quality of the evidence

Risk of bias

The Cochrane Risk of Bias tool56 was used to assess and classify the risk of bias in each of the included studies, as per criteria defined a priori. A judgement was made about whether each study had a high, low or unclear risk of bias in each of the following six domains: random sequence generation, allocation concealment, blinding of participants and study personnel, blinding of outcome assessments, incomplete outcome data, and selective outcome reporting.

The overall risk of bias was classified as low if none of the above domains was rated as high risk and three or less were rated as unclear risk. It was classified as moderate if one domain was rated as high risk, or none rated as high risk but four or more rated as unclear risk. All other studies were classified as having a high risk of bias57.

To represent the quality of evidence associated with comparisons in the network meta‐analysis, we used coloured edges in the network plots, as recommended58.

GRADE

We assessed the certainty of evidence contributing to network estimates of the primary outcome using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework59. The GRADE method characterizes the quality of a body of evidence on the basis of six factors: study limitations, imprecision, heterogeneity, inconsistency, indirectness, and publication bias59.

We tabulated the findings for the above six factors to aid in the decision‐making process for the downgrading of evidence. If one of the factors was present for a comparison, then the overall confidence rating for that comparison was considered for downgrading by one or two levels (as appropriate). Each comparison started as high quality/confidence (as based on RCTs), and was downgraded to moderate, low or very low, depending on the presence, severity and potential impact of the aforementioned factors. These represented the final judgements about the certainty of the evidence59, 60.

Statistical analysis

Frequentist NMAs were conducted for transition and acceptability outcomes using the network package in STATA (version SE 14.2; StataCorp). First, a network plot was constructed for each outcome61 to ensure that nodes of the network were sufficiently connected58. We then performed a NMA assuming consistency and a common heterogeneity across all comparisons in the network. This allowed us to derive a single summary treatment effect (odds ratio, OR) for every possible pairwise comparison of treatments, which takes account of all evidence from the network of trials, including both direct and indirect comparisons. Correlations in effect sizes induced by multi‐arm trials62 were accounted for58, 63. The resulting relative ORs with 95% confidence intervals (CIs) for each pair of treatments were reported in league tables64.

The interventions were then ranked by the surface under the cumulative ranking curve (SUCRA), which accounts for the location as well as the variance of all relative treatment effects65. SUCRA is a numeric presentation of the overall ranking and provides a single number (from 0 to 100%) associated with each intervention66. The higher the SUCRA value, and the closer to 100%, the higher the likelihood that an intervention is in the top rank, and vice versa66. Cluster ranking methods58, 65 – using both transition and acceptability SUCRA values – were used to order the treatments in league tables, in line with recent guidance which requires interpretation of SUCRA only in the context of NMA uncertainty, rather than at face value66. Statistical significance was set at p < 0.05.

We assessed the assumption of consistency by calculating, for each closed loop, an inconsistency factor (differences between direct and indirect evidence) along with 95% CIs and associated p values. We plotted the results graphically as the ratio of ORs (RORs) and 95% CIs for each loop64. Inconsistency was defined as disagreement between direct and indirect evidence, with 95% CIs for RORs excluding 1.

Given the low power of the loop‐specific approach and its focus on local inconsistency (between direct and indirect evidence), we also tested a full design‐by‐treatment model62 for the primary outcome to evaluate inconsistency more globally, including between trials with different designs (e.g., two‐arm vs. multi‐arm). A NMA under the inconsistency model was applied and a χ² test was used to infer about the statistical significance of all possible inconsistencies in the networks67.

The transitivity assumption was examined by assessing the distributions of potential effect modifiers for every comparison in the network, including percentage of males68, age69, percentage exposed to antipsychotic medications at baseline70, type of blinding and publication year6. The presence of small‐study effects was assessed by visual inspection of comparison‐adjusted funnel plots59.

To evaluate the impact of study quality and our data analysis procedures, we conducted sensitivity analyses for the primary outcome restricted to: a) studies with a low risk of bias for the blinding of outcome assessments; b) studies whose data were not digitally extracted (e.g., from Kaplan‐Meier plots); and c) published data only. We also repeated the analyses after applying alternate clustering of the following nodes: a) pooling NBI and placebo; b) pooling different CBT protocols; c) pooling different types of antipsychotic molecules, and d) separating the different NBI components (i.e., supportive therapy vs. clinical monitoring vs. other). Finally, network meta‐regressions were planned in the case of substantial heterogeneity and at least ten studies71 to test the impact of different CHR‐P diagnostic instruments/criteria.

RESULTS

Characteristics of the included studies

1,556 references were found in the literature search, most of which were not reporting RCTs in CHR‐P individuals; 49 were fully screened for the inclusion and exclusion criteria, resulting in a final sample of 16 studies (Figure 1). There were only five, four, two and three studies reporting data for 18, 24, 36 and >36 month time points, respectively, and therefore all results reported hereafter are for 6 and 12 months only.

Figure 1.

PRISMA flow chart of the study selection process. CHR‐P – clinical high risk for psychosis, RCT – randomized controlled trial

The 16 studies used in the analyses of the primary outcome contributed data on 2,035 patients, with a mean age of 20.1 ± 2.8 years, and 57% were male (Table 1). The mean sample size was 127 (range 44‐304). Six studies were conducted in North America, six in Europe, three in Australia and one was multi‐national. Two studies were three‐arm and the rest were two‐arm trials. Two studies had a treatment duration of <6 months, ten of 6 months, and four of 12 months. Of the 14 studies with available information on sponsorship/funding, three31, 75, 81, 82 acknowledged pharmaceutical company grants. The CAARMS and the SIPS were the most common CHR‐P diagnostic instruments44 (six and seven studies, respectively).

Table 1.

Details of included studies

Study	Study arms (N)	Network inclusion	Treatment duration (months)	Follow‐up time points (months)	% male	Mean age	CHR‐P criteria	Study design	Country	% exposed to antipsychotics at baseline
Addington et al30	CBT‐F + NBI (27) NBI (24)	6, 12	6	6,12,18	71	20.9	SIPS	SB‐RCT	Canada	0
Amminger et al33	Omega‐3 + NBI (41) NBI (40)	6, 12	3	6,12,84	33	16.4	PANSS	DB‐RCT	Austria	0
Bechdolf et al39	IPI (63) NBI (65)	6, 12	12	6,12,18, 24	63	26.0	ERIraos	SB‐RCT	Germany	0
Bechdolf et al72	NBI + ARI (96) NBI (55) CBT‐F + NBI (129)	6, 12	12	6,12	66	24.4	SIPS + BS	SB‐RCT	Germany	3.4
Cadenhead et al73	Omega‐3 + NBI (65) NBI (62)	6, 12	6	6,12,18, 24	56	18.8	SIPS	DB‐RCT	US, Canada	0
Kantrowitz et al74	D‐serine + NBI (20) NBI (24)	6	4	4	66	19.4	SIPS	DB‐RCT	US	11.4
McGlashan et al75	NBI + OLA (31) NBI (29)	6, 12	12	12, 24	65	17.7	SIPS	DB‐RCT	US, Canada	10
McGorry et al31	CBT‐F + RIS + NBI (31) NBI (28)	6, 12	6	6,12,36‐48	58	20.0	BPRS	SB‐RCT	Australia	0
McGorry et al76	Omega‐3 + NBI (153) NBI (151)	6, 12	6	6,12	46	19.2	CAARMS	DB‐RCT	Multi‐national	0
Miklowitz et al41	FFT + NBI (66) NBI (63)	6	6	6	57	17.4	SIPS	SB‐RCT	US, Canada	20.9
Morrison et al36	CBT‐F + NBI (37) NBI (23)	6, 12	6	6,12,36	69	22.0	CAARMS	SB‐RCT	UK	0
Morrison et al77	CBT‐F + NBI (144) NBI (144)	6, 12	6	6,12,18,24	63	20.7	CAARMS	SB‐RCT	UK	0
Stain et al78	CBT‐F + NBI (30) NBI (27)	6, 12	6	6,12	40	16.3	CAARMS	SB‐RCT	Australia	0
van der Gaag et al37	CBT‐F + CBT‐V + NBI (98) CBT‐F + NBI (103)	6, 12	6	6,12,18,48	49	22.7	CAARMS	SB‐RCT	The Netherlands	1.5
Woods et al79	NBI + ZIP (24)	6	6	6	64	22.3	SIPS	DB‐RCT	US	0
Woods80	NBI (27)
Yung et al81	CBT‐F + NBI (44)	6, 12	12	6,12	39	18.1	CAARMS	SB‐RCT	Australia	0
McGorry et al82	CBT‐F + RIS + NBI (43) NBI (28)

CBT – cognitive behavioural therapy, CBT‐F – French & Morrison CBT protocol, NBI – needs‐based interventions (including placebo), IPI – integrated psychological interventions, ARI – aripiprazole, OLA – olanzapine, RIS – risperidone, FFT – family‐focused therapy, CBT‐V – van der Gaag CBT protocol, ZIP – ziprasidone, SIPS – Structured Interview for Psychosis‐risk Syndromes, PANSS – Positive and Negative Syndrome Scale, ERIraos – Early Recognition Inventory, BS – basic symptoms, BPRS – Brief Psychiatric Rating Scale, CAARMS – Comprehensive Assessment of At‐Risk Mental States, SB‐RCT – single‐blind randomized controlled trial, DB‐RCT – double‐blind randomized controlled trial

For the 6‐month analysis of the primary outcome, these 16 studies provided data on 20 direct comparisons between 11 different treatment nodes (Figure 2). Three studies provided follow‐up data only for the 6‐month analysis, and therefore the 12‐month analysis consisted of 13 studies (N=1,811), providing data on 17 direct comparisons between 8 different treatment nodes (Figure 2). The network plots for the acceptability outcome were the same at 12 months and similar at 6 months (integrated psychological interventions was missing).

Figure 2.

Network plots of direct comparisons in the network meta‐analysis for transition outcome at 6 (on the left) and 12 months (on the right). The width of the lines is proportional to the number of trials comparing each pair of treatments and the size of each node is proportional to the number of studies testing the specific treatment. The color of the lines represents the comparison‐specific bias level for the blinding of outcome assessments in the majority of trials (black = low risk, dark grey = unclear risk, light grey = high risk). NBI – needs‐based interventions (including placebo), IPI – integrated psychological interventions, FFT – family‐focused therapy, Dser – D‐serine, CBT – cognitive behavioural therapy, CBT‐F – French & Morrison CBT protocol, CBT‐V – van der Gaag CBT protocol, RIS – risperidone, Om3 – omega‐3 fatty acids, ZIP – ziprasidone, OLA – olanzapine, ARI – aripiprazole.

Primary outcome: transition

Results of the NMA showed a lack of evidence for clearly superior efficacy of specific treatments in preventing transition, with no significant effects of any one intervention over any others at 6 or 12 month time points (Figures 3 and 4).

Figure 3.

Relative effect sizes for transition to psychosis and acceptability (dropout for any reason) at 6 months, odds ratios (95% CI). Comparisons between treatments should be read from left to right, and the estimate is in the cell in common between the column‐defining treatment and the row‐defining treatment. Treatments are reported in descending order (from top left to bottom right) as per the cluster ranking for transition and acceptability. For transition, an OR less than 1 favors the column‐defined treatment. For acceptability, an OR less than 1 favors the row‐defined treatment. All 95% CIs include the null hypothesis OR = 1. Dashes (‐) indicate no available NMA estimate. CBT – cognitive behavioural therapy, CBT‐F – French & Morrison CBT protocol, CBT‐V – van der Gaag CBT protocol, NBI – needs‐based interventions (including placebo), RIS – risperidone, FFT – family‐focused therapy, IPI – integrated psychological interventions, ARI – aripiprazole, OLA – olanzapine, ZIP – ziprasidone, Om3 – omega‐3 fatty acids.

Figure 4.

Relative effect sizes for transition to psychosis and acceptability (dropout for any reason) at 12 months, odds ratios (95% CI). Comparisons between treatments should be read from left to right, and the estimate is in the cell in common between the column‐defining treatment and the row‐defining treatment. Treatments are reported in descending order (from top left to bottom right) as per the cluster ranking for transition and acceptability. For transition, an OR less than 1 favors the column‐defined treatment. For acceptability, an OR less than 1 favors the row‐defined treatment. All 95% CIs include the null hypothesis OR = 1. CBT – cognitive behavioural therapy, CBT‐F – French & Morrison CBT protocol, CBT‐V – van der Gaag CBT protocol, NBI – needs‐based interventions (including placebo), RIS – risperidone, IPI – integrated psychological interventions, ARI – aripiprazole, OLA – olanzapine, Om3 – omega‐3 fatty acids.

Using NBI as a comparator, the OR and 95% CI for each treatment (all OR < 1 favor the given treatment) at 6 months were: 0.06 (0.00‐1.90) for integrated psychological interventions; 0.17 (0.01‐2.69) for family‐focused therapy + NBI; 0.22 (0.02‐2.17) for CBT‐F + CBT‐V + NBI; 0.29 (0.03‐2.57) for olanzapine + NBI; 0.21 (0.04‐1.08) for CBT‐F + risperidone + NBI; 0.52 (0.03‐10.72) for ziprasidone + NBI; 0.56 (0.03‐11.51) for D‐serine + NBI; 0.64 (0.15‐2.68) for omega‐3 + NBI; 0.73 (0.27‐2.01) for CBT‐F + NBI; and 0.94 (0.15‐5.73) for aripiprazole + NBI.

At 12 months, ORs against the NBI comparator were: 0.04 (0.00‐1.06) for integrated psychological interventions; 0.15 (0.02‐1.25) for olanzapine + NBI; 0.21 (0.03‐1.60) for CBT‐F + CBT‐V + NBI; 0.43 (0.11‐1.68) for CBT‐F + risperidone + NBI; 0.58 (0.23‐1.47) for CBT‐F + NBI; 0.64 (0.18‐2.26) for omega‐3 + NBI; and 1.39 (0.26‐7.28) for aripiprazole + NBI.

While almost all the interventions at both time points had estimates favoring them over NBI, the differences were not beyond chance, and the 95% CIs for the NMA estimates were often very large, indicating substantial imprecision. The cluster ranking (based on SUCRA values for transition and acceptability) at 6 and 12 months is illustrated by the ordering of treatments in Tables 2 and 3.

No statistically significant inconsistency was evident at any time point, with 95% CIs for all RORs compatible with zero inconsistency (ROR=1). However, only two loops were available. Using the design‐by‐treatment interaction test62, we found no evidence for significant inconsistency for 6 month (p=0.90) and 12 month (p=0.93) networks.

Only two studies had an overall low risk of bias33, 79; five had unclear risk72, 73, 74, 75, 76, and nine had high risk30, 31, 36, 37, 39, 41, 77, 78, 81. The edges (lines) in Figure 2 reflect the Cochrane risk of bias for the blinding of outcome assessments, estimated as the level of bias in the majority of trials and weighted according to the number of studies in each comparison58. The GRADE assessment highlighted low or very low confidence in almost all estimates, primarily due to study limitations (high risks of bias) and imprecision.

The numbers of studies remaining (at 6 and 12 months, respectively) after exclusion of those with a high or unclear risk of bias for the blinding of outcome assessments were 10 and 8; after exclusion of those whose data were extracted by digitizing Kaplan‐Meier plots were 13 and 12; after exclusion of unpublished studies were 13 and 11. The NMA model was refitted accordingly and no differences in conclusions were observed for any OR at any time point.

Repeating the analyses treating NBI + placebo as a separate node to NBI, or separating the different NBI components, had no effect on the NMA estimates, and therefore we used the pooled NBI + placebo in the main analysis (Table 1, Figures 2, 3, 4). Similarly, pooling together different CBT protocols or different antipsychotic molecules in the same node produced no significant results. There were not enough studies to allow robust meta‐regression analyses on the type of CHR‐P instruments. Visual inspection of funnel plots revealed no substantive evidence of small‐study effects.

Secondary outcome: acceptability

Acceptability data were available for 14 of 16 studies at 6 months (N=1,848), and 12 of 13 studies at 12 months (N=1,752). There were no significant differences in acceptability between any treatment comparisons at 6 or 12 months (Figures 3 and 4). The SUCRA cluster ranking (for transition and acceptability) is illustrated in those figures.

DISCUSSION

This is the first network meta‐analysis exploring the efficacy of specific interventions for the prevention of psychosis in CHR‐P individuals. Adopting strict inclusion criteria, a total of 16 RCTs, with 2,035 patients, were included in the analyses. There were not enough studies to analyze data with a NMA approach beyond 6 and 12 month follow‐ups. Two networks were established at 6 and 12 months, including 11 and 8 nodes, respectively. Network meta‐analyses showed no clear evidence of superior efficacy for any specific intervention at any time point. The results were not affected by biases, inconsistency or small‐study effects.

The main finding of the current study is that there is a lack of evidence to favor specific effective interventions to prevent psychosis in CHR‐P individuals. Our analyses were based on a detailed protocol, which defined the exact type of interventions and nodes a priori. This was done with the aim of providing robust informative results of direct clinical significance. For example, deconstructing the efficacy of different types of CBT that are based on different protocols83 seems necessary to inform accurate and evidence‐based clinical guidelines for patients, clinicians and policy makers. Our NMA comparing the different CBT protocols is also timely, since authors have recently claimed that the “black box” of CBT should be unpacked into its specific therapeutic components23, 24, 84, 85, 86.

In a similar fashion, our NMA represents the first attempt at deconstructing – through sensitivity analyses – the effect of different components (including placebo) that characterize NBI, which is usually employed as the control condition in this field. We also restricted our literature search to include only RCTs designed to be blinded, and studies that strictly used CHR‐P assessment instruments, to minimize selection biases. Therefore, to date, our study represents the most fine‐grained analysis that has deconstructed the specific effect of preventive interventions for psychosis.

Negative (non‐significant) results are rarely published in psychiatric literature87, which is affected by excess of statistical significance88, 89, 90, 91, 92. In fact, interpreting negative findings is particularly challenging, because absence of evidence is not evidence of absence93. In particular, when large CIs are observed (as in Figures 3 and 4), some sizeable effects may still have been missed. Nevertheless, our work represents the most powered data synthesis in this field. For example, the meta‐analysis by Stafford et al15 – on which current clinical guidelines are based – analyzed 11 studies, but one of them included an open‐label trial (N=124)94 and another did not assess participants against standard CHR‐P criteria (N=79)20, leaving nine studies (N=1,043) that are in common with the current NMA. Since that meta‐analysis, seven new trials involving 992 new CHR‐P participants (an increase of more than 50%) have been published, all of which reported non‐significant effects41, 72, 73, 74, 76, 78, 79, 80. Since our NMA included these new data, it is more powered than previous pairwise analyses.

In the context of power considerations, indirect evidence, when combined with direct evidence through NMA, increases the power and precision of treatment effect estimates compared to pairwise analyses26. Furthermore, when we pooled different CBT protocols or antipsychotic molecules in the same node – thus increasing the statistical power – no significant results were still observed. Overall, the core result of our NMA is more congruent with the evidence emerging from the most recent trials, compared to previous evidence syntheses.

The current lack of evidence to support specific preventive treatments is also consistent with the fact that the three largest interventional studies in this field have all produced negative findings95. Earlier studies that dominated the conclusions of some previous meta‐analyses (e.g., the omega‐3 trial33) were likely false positives. There is also converging lack of significant benefits on other clinical outcomes besides transition to psychosis, such as attenuated symptom severity14, 15, 96, functioning10, 14, 18, depressive comorbidities15, distress14, and quality of life14, 15.

These findings, taken together, are particularly problematic given the conceptual concerns over the clinical validity and significance of the dichotomous concept of transition within the CHR‐P paradigm97, 98. More to the point, it is not clear whether the currently tested treatments are only delaying the onset of psychosis as opposed to altering the course of the disorder7. Long‐term outcome trials are scarce and the results are conflicting.

The additional caveat is that the exact mechanism of action of the tested preventive treatments is – at best – poorly defined, due to lack of an established and validated pathophysiological model underlying the onset of psychosis in CHR‐P samples. A lack of mechanistic models forces researchers to proceed with empirical attempts that may eventually prove unsuccessful, as has ultimately been the case for omega‐3 fatty acids76. However, as our ability to stratify CHR‐P individuals into more homogenous subtypes improves, so may our success in testing specific treatments targeted to underlying biological and psychological mechanisms99.

Our findings may have an impact on research and clinical practice. In times of scarce resources, our NMA can help to focus the next generation of research on the most promising interventions. Although our ranking analysis should be interpreted cautiously66, 100 in the context of non‐superiority of any intervention compared to any other, it suggests that CBT‐F, which currently represents the most widely adopted intervention, may not be the best candidate (of relevance, the largest CBT‐F trial to date provided non‐significant results77). On the other hand, the apparent promising profile of integrated psychological approaches could be the target of future replications.

Future research in this area will need to test novel interventions that may act on underlying psychological or neurobiological processes associated with the onset of psychosis. Although there are no clinically valid CHR‐P biomarkers yet available101, several international consortia are ongoing (PRONIA102, NAPLS103, PSYSCAN104) with the aim of developing them. At the same time, it seems warranted to address the clinical heterogeneity1, 6, 49, 105, 106 that may prevent the discovery of reliable preventive treatments, and to improve the design of the next generation of trials. For example, it is apparent that unstructured recruitment processes and risk enrichment procedures in samples undergoing CHR‐P assessment have a substantial role in determining the actual level of risk for psychosis in these individuals107, 108, 109, leading to underpowered and non‐significant trials95. On a clinical side, individuals meeting CHR‐P criteria may be informed that, at present, there is no evidence for specific treatments being more effective than any others, and current options should be carefully weighted on a personal basis depending on an individual's needs.

This study has some limitations. First, only 16 RCTs were included, reflecting the paucity of high‐quality studies available in the CHR‐P field. However, capitalizing on the increased power and precision of NMA26, the Cochrane group has conducted such analysis in even smaller databases, including as few as three to seven studies110, 111, 112, 113. Furthermore, sufficient data were available for 6 and 12 month networks only, which precluded insight into whether treatments may have some effectiveness in the longer term. As a result of the sparse literature, many nodes were not well connected, with the corollary of limited ability to check for inconsistency, more imprecise estimates and wide 95% CIs.

In addition, the quality of NMA rests on the quality of included studies, many of which were found to be at high or unclear risk of bias, with GRADE confidence estimates predominantly low or very low – suggesting that true effects may be substantially different from the estimates. This is particularly the case for trials including any psychological interventions. We addressed this issue through a strict and detailed assessment of biases and sensitivity analyses. Going forward, given that all comparisons in the NMA were downgraded due to study limitations (risk of bias) and imprecision, the addition of high‐quality studies with adequate sample sizes is needed to improve these confidence ratings.

A final limitation is that, whilst dropout due to any cause was available from the majority of trials, this is a rather crude measure of treatment acceptability, and a more proximal index, such as specific adverse effects, may have revealed significant differences between treatments, in particular for trials of antipsychotic molecules. However, these outcomes are rarely reported in the CHR‐P literature.

In conclusion, there is currently no evidence to favor specific interventions for the prevention of psychosis. Further experimental research in this field is needed.