Has cognitive behavioural therapy for psychosis been oversold?

Peter McKenna—Yes

After the discovery of chlorpromazine in 1952 and the subsequent rise of biological psychiatry, the mainstream view of schizophrenia became that it was amenable only to drug treatment. But everything changed in 1997 when a pioneering trial found that drug resistant patients improved when given a form of cognitive therapy adapted to target delusions and hallucinations, two of the core “positive” symptoms of schizophrenia.¹ Many other trials— more than 50 to date—have since followed.

Problems with the evidence

Promising results in early studies is one thing; demonstrating that a treatment is effective in large, well controlled trials is another. And it is here that cognitive behavioural therapy for schizophrenia has run into difficulties. There have been around nine moderately sized or large (35-257 participants in the cognitive therapy group), blind trials comparing cognitive behavioural therapy with usual treatment or a control psychological intervention (trials of psychotherapy are invariably carried out under single blind rather than double-blind conditions). Only two of these nine methodologically rigorous trials have had positive results on their primary outcomes of overall symptoms, positive symptoms, or relapse at the end of the treatment period,^{2 3} and in one of these the authors noted that the blinding became compromised as the trial went on.³

Notwithstanding the mixed signals from these and many other smaller or less rigorous studies, meta-analyses over the years have generally been supportive. The leading example here is unquestionably the meta-analysis, or rather series of meta-analyses, carried out for the 2009 National Institute for Health and Care Excellence’s schizophrenia guideline.⁴ On the basis of these, NICE concluded that cognitive therapy was effective in reducing rates of readmission to hospital and duration of admission. It was also judged to be effective in reducing overall symptom severity, both at the end of treatment and after up to 12 months’ follow-up. Effectiveness against positive symptoms was more limited, but evidence was marshalled for benefits on hallucinations.

However, this meta-analysis was flawed. It examined a large number of outcome measures in two main comparisons—cognitive behavioural therapy versus standard care and cognitive behavioural therapy versus other active (psychological) treatments. These included not just readmission rates and symptoms but also relapse, social and occupational functioning, mortality, suicide, and insight; many of the measures were non-independent—for example, symptoms at six, 12, and 18 months’ follow-up, and summed positive symptoms as well as separate ratings for delusions and hallucinations, the latter with various subsidiary measures such as command hallucinations, malevolence, and omniscience. Slightly under half of the 110 individual meta-analyses ((excluding sensitivity analyses, re-analyses with outliers removed, etc) carried out for these two main comparisons contained only one or two studies. In these circumstances, there is a risk that some of the positive findings will have been down to chance. In fact, correcting for multiple comparisons using false discovery rate, a method appropriate for correlated variables,⁵ we found that the individual meta-analyses for only six out of 65 measures remained significant in the comparison of cognitive therapy with standard care, mainly related to aspects of hallucinations, and none of the 45 in the comparison with other active treatments (data available from the authors on request).

Negative results

Subsequent meta-analyses have had more sobering findings. In 2012, the Cochrane Collaboration compared cognitive therapy with other psychological therapies, both active and inactive, and concluded that there was no clear and convincing evidence of benefits for relapse, readmission to hospital, or a range of mental state measures.⁶ Another similar meta-analysis found an effect size for symptom scores of just 0.16 compared with other psychological interventions.⁷

We recently comprehensively reviewed all randomised trials carried out worldwide to date.⁸ The crude pooled effect size was 0.33 for overall symptoms in 34 studies and 0.25 for positive symptoms in 33 studies, both values being in the small range. We also examined the influence of sources of bias, something that previous meta-analyses had done to only a limited extent. Use of blind evaluations was found to exert a highly significant effect, with the effect size falling to 0.15 in 20 blind studies of overall symptoms and to 0.08 in 20 blind studies of positive symptoms; the last result was non-significant. We found no compelling evidence for an effect on hallucinations.

For the time being, the cognitive therapy for schizophrenia ship sails on. The treatment is officially mandated by the English government for all patients with schizophrenia, and its implementation is being monitored. Large grants to explore ever wider applications in psychotic patients continue to be applied for and awarded. A recent News Focus article in Science was scrupulously balanced but nevertheless came down firmly in favour of cognitive behavioural therapy and other psychological treatments being effective in schizophrenia.⁹ But behind all the fanfare, it has been evident for some time that this form of treatment is being kept afloat only by efforts to play up weak or equivocal findings and to discredit the increasingly ominous results from meta-analysis.

David Kingdon—No

If psychosis is biological in origin, how can a psychological therapy be expected to have any effect on it? Psychological therapies in psychosis have done harm in the past, with families being falsely accused of causing psychosis. It is entirely reasonable therefore to question whether the development of psychological interventions could be harmful or ineffective and so displace valuable resources from areas of need.¹⁰

But non-biological approaches—for example, occupational therapy, physiotherapy, and psychological approaches—are accepted as having a place in unequivocally biological conditions such as stroke. Is there anything intrinsically wrong with them having a place in management of psychosis? A substantial body of work now shows the relevance of psychological¹¹ and social influences.¹²

Evidence shows benefit

Meta-analyses of cognitive therapy for psychosis have consistently shown an effect size of around 0.3.⁷ But not everyone agrees that this translates into clinical effectiveness. McKenna and colleagues have consistently concluded that, despite recently finding a similar effect size to other meta-analyses, no publication bias, and no significant effect of non-specific therapy controls on differential outcome, cognitive behavioural therapy for psychosis has been “oversold.”⁸ They suggest that there have been few successful studies and that meta-analysis has been necessary to demonstrate any effect. This is an extraordinary interpretation of their own forest plots, which show that in nearly all the studies selected cognitive behavioural therapy is favoured over controls. They list many studies that have achieved significant results for their primary outcome measures. Among the studies that they describe as failures, they include a comparison of cognitive behavioural therapy with befriending published by Archives of General Psychiatry that shows clinically significant effects at the end of the trial period¹³ and a continuing significant effect at five years.¹⁴

They remove studies from their final analysis that they describe as methodologically unsound. This is not because of flaws in randomisation procedure or incompleteness of outcome data, but because they are deemed to be unmasked—that is, treatment allocation was not confirmed as concealed from assessors. On this basis, they removed 10 studies from the 30 analysed (already reduced from over 50), thereby reducing the overall effect size to 0.15, but in several instances, such removal is disputable. Many of the studies described as “unclear” indicate that independence or masking from allocation occurred and in others the authors have confirmed that it occurred.

There are, of course, difficulties with meta-analyses. Studies use different methods, types of therapy, inclusion and exclusion criteria, and outcome measures, making comparison and meta-analysis difficult. Direct replications in research into cognitive behavioural therapy for psychosis are rare as public funding bodies are continually seeking innovation (and for studies to be methodologically sound). Rather than examining the same techniques and populations, studies have used brief interventions, examined therapy modifications, or been targeted at specific patient groups. Some have therefore not shown benefit for cognitive behavioural therapy because of the choice of participants—for example, coexistence of substance misuse¹⁵—or because of treatment problems, such as too brief treatment periods.¹⁶ These studies continue to be included in meta-analyses despite the fact that the treatment is not effective when delivered in this way. This makes it all the more significant that positive effect sizes exist.

Pharmaceutical standards

Would we be having this argument if cognitive behavioural therapy was a drug? If an effect size of about 0.3 is accepted, this is similar to that of clozapine compared with conventional antipsychotics. ¹⁷ Clozapine is universally accepted as an important additional treatment in psychosis but it does cause substantial side effects. Cognitive behavioural therapy has fewer side effects and those that do exist would generally be considered less harmful. Clinical guidelines internationally recommend the use of both clozapine and cognitive behavioural therapy, but non-drug treatment has regrettably not benefited from the communications and advertising budgets that commercial support brings. If comparison is made with promotion of the second generation antipsychotics, whose effect size compared to the first generation is modest, then the idea that cognitive behavioural therapy has been oversold becomes even more difficult to sustain.

So who could be accused of overselling cognitive behavioural therapy? International clinical guideline committees seem the chief culprits, including the National Institute for Health and Care Excellence (three committees have supported its use), the Schizophrenia Patient Outcomes Research Team (two committees), and national psychiatric associations in America, Australasia, Germany, and many other countries.¹⁸ In the UK, it is supported by the Schizophrenia Commission, the Royal College of Psychiatrists, mental health charities, carers and, finally, those who use services themselves. Can they all have got it so wrong?

Cognitive behavioural therapy complements the use of medication and family work within the context of mental health services. It even seems to be acceptable to many people who refuse to take medication, with early evidence suggesting that it may reduce symptoms.¹⁹ Treatment is evolving as a thriving research community investigates new approaches—for example, mindfulness, focusing on worry, acceptance and commitment therapy, and competitive memory training—and use in distinct populations, such as those who have persistent symptoms despite taking clozapine.²⁰ Psychosis remains a challenge: psychological approaches do not hold all the answers, but the evidence suggests that they can have an important and acceptable role in helping people cope with very distressing and disabling experiences.

Cite this as: BMJ 2014;348:g2295