An Updated Meta-Analysis of Randomized Controlled Evidence for the Effectiveness of Community Treatment Orders

Abstract

Objectives:

It is unclear whether community treatment orders (CTOs) for people with severe mental illnesses can reduce health service use, or improve clinical and social outcomes. Randomized controlled trials of CTOs are rare because of ethical and logistical concerns. This meta-analysis updates available evidence.

Method:

A systematic literature search was performed of the Cochrane Schizophrenia Group Register, Science Citation Index, PubMed, MEDLINE, and Embase to November 2013. Inclusion criteria were studies comparing CTOs with standard care including those where control subjects received voluntary care, for most of the trial.

Results:

Three studies provided 749 subjects for the meta-analysis. Two compared compulsory treatment with entirely voluntary care, while the third had control subjects receiving voluntary treatment for the bulk of the time. Compared with control subjects, CTOs did not reduce readmissions (risk ratio 0.98, 95% CI 0.82 to 1.16) or bed days (mean difference [MD] −16.36; 95% CI −40.8 to 8.05) in the subsequent 12 months (n = 749). Moreover, there were no significant differences in psychiatric symptoms (standardized MD −0.03; 95% CI −0.25 to 0.19; n = 331) or the Global Assessment of Functioning (MD −1.36; 95% CI −4.07 to 1.35; n = 335). Only including the 2 studies that compared compulsory treatment with entirely voluntary care made no difference to the results.

Conclusions:

CTOs may not lead to significant differences in readmission, social functioning, or symptomatology, compared with standard care. Their use should be kept under review.

There is controversy over whether CTOs can reduce health service use, or improve clinical outcome and social functioning. RCTs of CTOs are rare because of ethical and logistical concerns. To date, there have only been 3. Two from the United States compared court-ordered compulsory treatment with entirely voluntary care. Both were published 15 years ago, 1 from New York and the other from North Carolina.^1–4 These 2 studies have been the subject of considerable debate. Both excluded patients with a history of dangerousness from randomization. Although understandable from a pragmatic and ethical standpoint, this limited generalizability as these patients may be the most likely candidates for CTOs. In the case of the New York study,^1,2 a smaller than expected sample size and high attrition compromised study power. There were also concerns about adherence to the research protocol and apparent confusion among staff and patients that some in the control arm were on CTOs.⁵ A further issue was that both clinicians and police officers were reluctant to enforce compulsory community treatment.² All these factors may therefore have led to an underestimation of the intervention’s effect.² In the case of the North Carolina study, the RCT results were sometimes combined with those of an additional nonrandomized group of violent patients who were also placed on CTOs.^3,4,6 In addition, the significant effects on bed days in nonrandomized analyses of extended outpatient committal received greater prominence than those of the primary RCT, where there were no differences in outcomes. Attrition was also higher for some of the secondary outcomes.

In a situation of limited studies and participants, meta-analyses can help address the issue of insufficient power. A Cochrane review of the 2 RCTs published at the time, both from the United States (n = 416), found no differences between CTO patients and control subjects in health service use, psychiatric morbidity, or social functioning.⁵

A subsequent study from England was known as the OCTET.⁷ OCTET was a study of clinician-initiated, rather than court-ordered, treatment studied in the American RCTs and thus more relevant to other jurisdictions, such as Australia, Canada, and New Zealand. The study randomized patients discharged from hospital to an experimental group (CTO) or a control group (leave under Section 17 of the Mental Health Act), comparing their outcomes at 12 months. Section 17 allows patients to leave the hospital for days or even months while still subject to recall.⁸ Although the length of initial compulsory outpatient treatment differed significantly between the 2 groups (medians of 183 days, compared with 8 days), Section 17 patients averaged 4 months on some form of compulsory treatment over the course of the study. Unlike the US studies, patients with a history of violence were not excluded. However, patients were only included if they were thought to be equally suitable for a relatively short Section 17 or a CTO. Patients who may have especially benefited from a CTO may therefore have been excluded.⁸ Further, around 20% of eligible patients lacked capacity to consent or refused to participate. Finally, treating clinicians were able to make decisions independent of randomization, thus around one-fifth of patients in either arm swapped treatments.

Uncertainty about whether OCTET was a comparison of CTOs with voluntary care, or 2 types of compulsory treatment, meant that the results of this trial were not combined with the other 2 in the update of the Cochrane review currently under progress.

Clinical Implications

• RCTs of CTOs are rare because of ethical and logistical concerns. In such a situation of limited studies and participants, meta-analyses can help address the issue of insufficient power. This brief communication indicated that compulsory CTOs did not reduce hospital admissions or bed days in the subsequent 12 months, compared with RCT subjects. Moreover, there were no significant differences in psychiatric symptoms or the GAF.

• Any extension of the use of compulsory CTOs should be accompanied by further evaluation of effectiveness.

Limitations

• Only 3 studies were included; the generalizability of results to other jurisdictions is therefore unclear.

• All 3 studies in our review may have been subject to selection bias.

• There is debate as to whether 1 of the studies was a comparison of CTOs with voluntary care, or 2 types of compulsory treatment. However, excluding this study made no difference to the results.

However, an alternative view is that Section 17 patients were on voluntary treatment for the bulk of follow-up and results could be included with those of the other 2 studies. Therefore, our paper presents the effect of combining the OCTET study with the relevant outcomes from the other 2 studies.

Method

We undertook a systematic literature search of the Cochrane Schizophrenia Group Register, Science Citation Index, PubMed, MEDLINE, and Embase to November 2013. Inclusion criteria were RCTs that compared compulsory CTOs with standard care, including studies where control subjects received voluntary care for most of the trial. The primary outcomes were readmission and bed days during 12 months of follow-up. Secondary ones were psychiatric morbidity and social functioning during the same period. Data extraction was conducted by 2 independent researchers.

We assessed the quality of included studies using the following criteria of the risk of bias assessment tool, developed by the Cochrane Collaboration: selection bias; blinding; attrition bias; and, reporting bias.⁹

We calculated the mean differences for continuous data where studies used the same scale for each outcome, and the standardized mean difference for data that used different scales. We assessed heterogeneity using the I² statistic, a measure that does not depend on the number of studies in the meta-analysis and hence has greater power to detect heterogeneity when the number of studies is small. We used the random effects model given the heterogeneity of the data.

Results

Figure 1 shows the results of the systematic review. Five papers from 3 studies provided 749 subjects for the meta-analysis. Two from the United States compared compulsory treatment with entirely voluntary care,^1–4 while the third, from England, had control subjects who received voluntary treatment for an average of around 8 out of the 12 months of follow-up.⁷

Figure 1.

Selection of randomized controlled trials (RCTs) for inclusion in the systematic review and meta-analysis

The quality of the included studies was not optimal, reflecting the pragmatic nature of such effectiveness trials with only 1 study meeting at least 3 of the 4 quality criteria.⁷ Attrition was low for the 2 primary outcomes (15%), but higher for secondary ones (30%).

All 3 studies reported on hospital readmission and bed days (n = 749). We combined these results using the risk ratio and MD, respectively. Two studies included the results of psychiatric rating scales, the Positive and Negative Syndrome Scale and the Brief Psychiatric Rating Scale (n = 331), which we combined using SMD.^1,7 They also reported results from the GAF scale (n = 335), which were combined using the MD.^1,7 In 3 analyses, the I² was 0%. However, for bed days, it was 73%.

Compared to control subjects, CTOs did not reduce readmissions (Figure 2) or bed days (MD −16.36; 95% CI −40.8 to 8.05) in the subsequent 12 months (n = 749), and there were no significant differences in psychiatric symptoms (SMD −0.03; 95% CI −0.25 to 0.19; n = 331) or the GAF (MD −1.36; 95% CI −4.07 to 1.35; n = 335).

Figure 2.

Readmission in the 12 months of follow-up

Discussion

Given the logistical and ethical difficulties of undertaking RCTs in this area, it is important to maximize the use of existing RCT evidence. Inclusion of the OCTET study almost doubled the number of participants available for meta-analyses, compared with the earlier Cochrane review.⁵ In addition, the form of compulsory CTOs in the OCTET study was more similar to that found in Canada, compared with the 2 studies from the United States.

The most obvious limitation is the small number of studies; the generalizability of results to other jurisdictions is, therefore, unclear. Another is uncertainty about the voluntary or compulsory status of the control subjects in the OCTET study, which will doubtless be the subject of ongoing debate. However, meta-analyses restricted to the 2 studies comparing compulsory treatment with entirely voluntary care gave similar results.^5,10 A further issue in all 3 RCTs was selection bias. Depending on the study, patients with a history of dangerousness were excluded,^1–4 and around 20% of eligible patients lacked capacity to consent to the study, or refused to take part.⁷ In addition, attrition was higher for the secondary outcomes of psychiatric morbidity and social functioning. Finally, the bed days analysis should be interpreted with caution, given significant statistical heterogeneity.

Bearing these limitations in mind, our results suggest that CTOs may not lead to significant differences in readmission, social functioning, or symptomatology, compared with standard care. Accordingly, there remains sufficient uncertainty about CTOs to warrant further large RCTs that would compare CTO patients with voluntary control subjects. Ideally, these should have multiple outcomes and sufficient numbers for adequate statistical power, and be undertaken in jurisdictions with well-established CTO programs.

Abbreviations

CTO

community treatment order

GAF

Global Assessment of Functioning

MD

mean difference

OCTET

Oxford Community Treatment Order Evaluation Trial

RCT

randomized controlled trial

SMD

standardized mean difference