Canadian Studies on the Effectiveness of Community Treatment Orders

Abstract

Objectives:

Community treatment orders (CTOs) for people with severe mental illnesses are used across most of Canada. It is unclear if they can reduce health service use, or improve clinical and social outcomes. This review summarizes the evidence from studies conducted in Canada.

Method:

A systematic literature search of PubMed and MEDLINE to March 2015 was conducted. Inclusion criteria were quantitative and qualitative studies undertaken in Canada that presented data on the effect of CTOs on outcomes.

Results:

Nine papers from 8 studies were included in the review. Four studies compared health service use before and after compulsory treatment as well as engagement with psychosocial supports. Three were qualitative evaluations of patients, family, or staff and the last was a postal survey of psychiatrists. Hospital readmission rates and days spent in hospital were all reduced following CTO placement, while outpatient attendance and participation in psychiatric services and housing all improved. Family members and clinicians were generally positive about the effect of CTOs but patients were ambivalent. However, the strength of the evidence was limited as many of the studies were small, only one included control subjects, and there was no adjustment for potential confounders using either matching or multivariate analyses. Only 2 qualitative studies included the views of patients and their families.

Conclusions:

The evidence base for the use of CTOs in Canada is limited and this lack of Canadian research is in marked contrast to other countries where there have been large studies that have used randomized or matched control subjects. Their use should be kept under review.

Community treatment orders (CTOs) in Canada require people with mental illness to comply with a treatment plan that generally requires them to attend regular, specified appointments to receive medication, counselling, and education. Canadian CTOs are clinician-initiated, rather than court-ordered as in the United States, and thus research conducted in Canada may be more generalizable to jurisdictions, such as Great Britain, Australia, and New Zealand.¹ The use of CTOs has expanded across Canada over the last 20 years, starting in Saskatchewan in the mid-1990s, followed by Ontario in 2000.² More recently, provisions for CTOs have been included in amendments to provincial mental health legislation in Nova Scotia, Newfoundland and Labrador, and Alberta.³ In at least 2 jurisdictions, the immediate impetus was a high profile murder of a sports commentator in Ontario and of a police officer in Alberta.³ All Canadian provinces, with the exception of New Brunswick, now have legislative provisions that allow the use of CTOs, or alternatives, such as extended leave (as opposed to shorter spells of conditional leave),^4,5 and in the case of New Brunswick, their introduction is under active consideration. There are several methodological issues when assessing the effectiveness of compulsory community treatment.^6,7 The first is specifying both the purpose and desired outcomes of compulsory community treatment. Contact with mental health services is often used, but while this is necessary to receive treatment, it can be argued that contact is a process variable and not the ultimate desired outcome. Providing a less restrictive alternative to hospitalization is another goal but the meaning of success can be similarly unclear. Do CTOs reduce admission rates, allowing people to remain in their communities during treatment, or are rates increased as a result of earlier identification of relapse? Brief admissions to prevent further deterioration may be seen as part of the workings of a CTO, as opposed to longer admissions that occur as a result of contravening the order.⁷ Would length of stay be a more appropriate measure on the basis that increased admission would still be the least restrictive alternative if people spent less time in hospital?

Conversely, if the purpose of CTOs is to reduce crime, forensic outcomes such as arrests, convictions, and custodial sentences may be more appropriate measures.⁸ Finally, from a patient’s perspective, a wider range of outcomes may reflect an improved quality of life. These include mortality, physical health, severity of psychiatric symptoms, self-harm, arrest and incarceration rates, freedom from victimization, ability to work, quality of life, and stability and independence of type of residence.

Several provincial governments have commissioned mandated reviews of CTO legislation.^9,10 These included views of patients, consumer groups, families, and health care staff gathered through interviews and (or) focus groups rather than the analysis of quantitative data. In all of these, families and health care staff have generally been positive about the effect of CTOs in stabilizing patients’ lives. However, patients and consumer groups express more ambivalence. The Nova Scotian review also presented some quantitative results from a small mirror image study that showed 31 out of 44 patients had reduced admissions, compared with before CTO placement, and that survival in the community was longer.¹⁰ Importantly, none of these reports were systematic reviews or peer reviewed and their findings are at odds with 3 more quantitatively orientated systematic reviews from elsewhere that showed little clinical advantage to CTOs.^11–13 Possible explanations for the discrepancies include methodological differences, or that differences between legislation in Canada and elsewhere mean Canadian CTOs are more effective. This is therefore a systematic review of both the quantitative and qualitative scientific literature on CTOs conducted in Canada.

Method

A comprehensive search of PubMed and MEDLINE was undertaken in March 2015 using combinations of the following free text and MeSH terms: Canada, community treatment order, compulsory community treatment, outpatient commitment, and CTO. Inclusion criteria were quantitative and qualitative studies undertaken in Canada that presented data on the outcomes of CTOs. This included clinical features, psychosocial functioning, quality of life, and contact with the criminal justice system, as well as views of interested parties, such as clinicians, patients, and families. Review articles were excluded although bibliographies were searched for potential articles

Results

Figure 1 shows the results of the systematic search. Nine papers from 8 studies were included in the review (Table 1). Because of the wide range of methodologies and outcomes, no attempt was made to meta-analyze the data.

Figure 1.

Selection of studies for inclusion in the review.

Table 1.

Summary of included studies.

Author		Year	Design	n	Mean age	Outcome
O’Brien14		2005	UBA	25 60% male	42	↓admissions and bed days ↑ contact with community services ↑ supportive housing arrangements
O’Brien15		2009	UBA	84 60% male	45	↑ contact with community services in 80% of the sample ↑ supportive housing arrangements
Same study
Frank16		2005	UBA	42 52% male	48.4	↓ time to readmission
Nakhost17		2012	UBA	72 52% male	46.3
Hunt19		2007	CBA	224 CTOs 92 control subjects 45% male	51% aged between 31 and 40 years	↓admissions 0 to 12 month follow-up ↓ bed days 6 to 12 month follow-up
O’Reilly22		2000	Survey of psychiatrists	50	—	62% satisfied or extremely satisfied with the effect of CTOs
Orr18		2012	Qualitative study of MHS staff	55 +	—	CTO cases had ↑time in the community, continuity of care, and QoL
O’Reilly23		2006	Qualitative study of patients, family and MHS staff	14 CTOs 14 family members + staff total N=78	44 — —	Family and clinicians positive Patients ambivalent
Mfoafo-M’Carthy24		2014	Qualitative study of ethnic minority patients	24	18 to 59	Patients ambivalent

— = age data was not included in the study; CBA = controlled before-and-after study; CTO = community treatment order; MHS = mental health services; QoL = quality of life; UBA = uncontrolled before-and-after study.

Characteristics of People on Community Treatment Orders

Surveys and other studies of consecutive patients placed on CTOs indicate that Canadian patients on these orders are generally similar to those in other jurisdictions.^9,10,14–18 They are typically single males with a median age of 40 years, a long history of schizophrenia-like or serious affective illness, and previous admissions or supervised discharge.

Mirror Image Studies of Outcomes

Although CTOs have been in use in Canada since 1995, the first published empirical studies of patient outcomes did not appear until 2005. One was a before-and-after study of 25 patients from Ottawa.¹⁴ This report showed a significant decrease in admissions to hospital and significant increases in the use of support services, such as community-based crisis, mental health, or medical services (Table 1). The focus of care also changed from being primarily hospital-based outpatients to greater use of community-based services, such that more than 75% of patients said they were in contact with 2 or more agencies. In addition, CTO placement was associated with increased involvement in supportive housing. Improved living arrangements following CTO placement were echoed in a later, larger study (n = 84) by the same authors (Table 1).¹⁵ All patients placed on CTOs from the Royal Ottawa Mental Health Centre and the Montfort Hospital during a 3-year period were included. They found that 80% of patients used a greater number of community-based services following CTO placement, the most significant being in the use of case management. However, as noted previously, it is unclear whether this marks a process or outcome measure given that CTOs mandate contact with such services. The authors did report a significant shift to supportive housing following a CTO. A third paper¹⁶ published in 2005 was another small study (n = 42) from Quebec (Table 1). Frank et al¹⁶ studied all patients discharged to a CTO from a Montreal hospital affiliated with McGill University from July 1, 1998, to June 30, 2000. The last follow-up of the entire sample was on April 1, 2002. Entry into the study was on discharge from the index admission, and data collected from hospital records both before, and after, the index admission. The period ranged from 2 to 10 years and was divided into 4 sections (Figure 2): the index period following CTO placement (this ended with the first readmission [postindex admission], or censorship [the end of the study]); the postindex period following the postindex admission; the pre-index period which ran from the time of the admission (pre-index admission) prior to the index admission; and the early period, defined as the 5 years before the beginning of the pre-index admission.

Figure 2.

Time course of the community treatment order studies from Quebec.

Patients in the index period following CTO placement had the longest median time to readmission (662 days, n = 42), closely followed by the postindex period (253 days, n = 8). These survival times in the community were longer than before the CTO in both the pre-index (128 days, n = 33), and early periods (199 days, n = 25). Pairwise comparisons indicated that the time-to-readmission for the index period was significantly longer than both the early and pre-index periods, but not the postindex period. However, a significant limitation is that data were missing from between 22% and 66% of the sample depending on the comparison being made leading to very small study numbers.

The same team expanded their study using a larger sample size and a longer follow-up period (Table 1).¹⁷ They also included an additional Montreal hospital that was affiliated with McGill University. As before, entry into the study was on discharge from the index admission but recruitment was extended to August 10, 2007. Data were collected over the same 4 periods: early, pre-index, index, and postindex (Figure 2). All patients who were placed on a CTO at the 2 hospitals in Montreal in the period covered by the study were eligible for inclusion. However, of the 90 possibly eligible patients, 28 (31%) could not be included because of missing data at various time points primarily from the early and pre-index periods, leaving a sample of 72. As in the earlier, smaller study patients in the index period following CTO placement had the longest median time to readmission (662 days, n = 72), closely followed by the postindex period (420 days, n = 57). Survival in the community was longer than before the CTO in both the pre-index (193 days, n = 67), and early periods (164.5 days, n = 60). Again, as before, pairwise comparisons indicated that the time-to-readmission for the index period was significantly longer than both the early and pre-index periods, but not the postindex period. Although these results seem promising, a significant limitation of this study was that 31% of eligible patients could not be included because of missing data. Even among patients who could be included, data on follow-up were missing in 21%.

In conclusion, before-and-after studies suggest that hospital readmission rates and days spent in hospital are all reduced following CTO placement, while outpatient attendance and participation in psychiatric services and housing are all improved. Strengths of these studies include the use of existing clinical files or databases making refusal to participate not an issue. Further, the use of patients as their own control may have addressed problems of matching for impaired insight or overt treatment refusal that can confound case-matched approaches. However, weaknesses include that routinely collected data were not always available for all time points. Comparing outcomes during the time period before and after the initiation of a CTO may have also introduced the potential confounding effect of service change, such as bed closures or improved outpatient services that may have decreased hospital use independent of a CTO. It may therefore be impossible to say if improved outcomes were due to the CTO or enhancements to other services. Finally, results could have been affected by regression to the mean, whereby patients who had the highest hospital use were most likely to be placed on a CTO and therefore to show lower rates of subsequent use by chance alone.

Controlled Studies of Outcomes

Unlike the United States or Great Britain, there have been no randomized trials and only one controlled before-and-after study of existing hospital data.¹⁹ This was a study of case-managed patients in Toronto, some of whom were on a CTO and some who were not (Table 1). An advantage of this method was that both groups were theoretically offered services of the same type and intensity through one group of case managers, and in theory, it should have been possible to establish the additive effect of CTOs on patient outcomes. However, no data on the intensity of community contacts during the 12 months following the CTO were presented, therefore it is impossible to know if this was indeed the case. The main findings were that the CTO group had significantly greater reductions in cumulative bed days per hospital admission within both the first and second 6-month period of follow-up. They also had significantly greater reductions in hospital admissions during the second 6-month period although the absolute reduction was small (1.3, compared with 0.9 admissions). The CTO group also had a significantly higher portion of patients who were well enough to leave case management within these first two 6-month periods. Because of the naturalistic nature of the study, matching was not possible. The authors did check on how similar the CTO and control groups were at baseline and although there were no significant differences on most demographic or clinical characteristics, there were 2 areas where there were differences.¹⁹ First, a significantly higher percentage of patients placed on CTOs were living with family members in privately owned residences, while a larger proportion of control subjects were living with people other than their family, such as in boarding homes. It is therefore possible that greater family support might have contributed to their improved outcomes. Second, patients placed on CTOs patients had more admissions and cumulative bed days than the control subjects prior to entry into the study. This could have contributed to the greater reductions in subsequent hospitalization after CTO placement, raising the possibility that their scores were regressing to the mean. In conclusion, the one controlled before-and-after study to be carried out in Canada showed similar reductions in inpatient use that were reported in the mirror image designs. The association between increasing benefit and longer CTO placement could be explained as either a long-term benefit of the intervention, or that those on the CTO long term were kept on them because things were going well clinically and the CTO was presumed to be responsible.⁷ Unfortunately, the nonrandomized design makes it difficult to tell. In addition, there are methodological concerns about the lack of adjustment for potential confounders, and the possibility of regression to the mean.

Epidemiological Studies

Unlike other jurisdictions, such as the United States or Australia, there have been no epidemiological studies of the effect of CTOs in Canada using province-wide administrative data. This is surprising given that administrative data are collected Canada-wide for the purpose of managing health care services and, in particular, for providing payment for services.²⁰ There was one study that used Nova Scotian databases but this was before CTOs were introduced to the province, and the Canadian patients were control subjects for a study of the effectiveness of CTOs in Western Australia.²¹

Surveys of Clinicians

A survey that was mailed to all Saskatchewan psychiatrists 3 years after the introduction of CTOs in the province achieved a response rate of 72% (n = 50).²² There was widespread support for the measure (Table 1). Sixty-two per cent of psychiatrists indicated that they were either satisfied or extremely satisfied with the effect of CTOs on patient care, while 10% were dissatisfied or extremely dissatisfied. Fifty-five per cent of psychiatrists were satisfied or extremely satisfied with the support provided by the province in the application and enforcement of CTOs, while 16% were dissatisfied or extremely dissatisfied.

Qualitative Studies

There have been 3 studies (Table 1). One looked at the experience of health service staff in Alberta through a focus group of CTO leads and coordinators.¹⁸ Like the survey of psychiatrists in Saskatchewan, views were generally positive. The focus group participants felt that patients on CTOs remained in the community for longer periods without readmission with greater participation in activities and (or) support groups. Discussion groups involving 55 physicians came to a similar view. Unfortunately, the paper gave no details about the qualitative methodology that was used and relied heavily on anecdotal evidence. No patients’ views were directly canvassed.¹⁸

A further study examined the opinions of patients who had been placed on a CTO (n = 14), their relatives (n = 14), mental health clinicians, and representatives of community agencies about the use of CTOs in Saskatchewan (total n = 78) (Table 1).²³ Patients were assessed using in-depth interviews, while their relatives, mental health professionals, and representatives of community agencies participated in facilitated focus groups. As in the earlier survey of psychiatrists in Saskatchewan,²² most mental health clinicians felt that the orders were helpful for specific patients.²³ However, psychiatrists and case managers criticized the administrative burden associated with CTOs. Relatives were also very positive about CTOs and some reported that they had been essential in bringing stability to patients’ lives. However, patients themselves were much more ambivalent (Table 1). Most experienced some degree of coercion but many also believed that CTOs provided necessary structure to their lives.

Unlike the study from Alberta, our paper gave extensive details of the methodology and analysis. Another major strength was that the study assessed the view of a wide range of interested parties, and, in particular, included patients on CTOs

However, only 42% of patients placed on CTOs agreed to take part in the study and a further 3 patients with a past history of CTO placement were added to increase the number of subjects. The ones who agreed may have had a better prognosis and greater insight resulting in more positive views on treatment orders.²³ By contrast, those who refused to participate may have had less insight and so resented CTO placement.²³ The authors tried to reduce this potential bias by including relatives of patients who refused to be interviewed although it is important to realize that the views of patients and their carers may not always agree.²³

Another study to incorporate the patient’s perspective was one on the effect of CTOs on people from an ethnic minority and how they perceived the intervention especially in comparison with other experiences in the mental health system (Table 1). Twenty-four people who had been on CTOs for periods from a few months to more than 3 years participated.²⁴ Most were Black Canadians, African Canadians or Caribbean Canadians, the rest being from Asia or the Middle East. As in the Saskatchewan study, patients had contradictory feelings and perceived both positive and negative impacts of CTOs (Table 1). The former included improved rapport with the case management and clinical team, increased medication compliance, and greater feelings of self-efficacy.²⁴ Negative impacts included feelings of being coerced and the associated stigma.²⁴

Discussion

A systematic search of PubMed and MEDLINE yielded only 9 papers from 8 studies that had been conducted in Canada. In the case of the quantitative research, all the studies were small and limited to a single population; only one included control subjects (Table 1). In the case where control subjects were included, there was no adjustment for potential confounders using either matching or multivariate analyses.¹⁹ Outcomes in the quantitative studies were also largely restricted to health service use, such as hospital readmissions and length of stay with little information on other more important patient level outcomes, such as mental state, quality of life, and satisfaction (Table 1). Only 2 qualitative studies including the patient’s perspective (Table 1), and transferability of findings of one may be limited given it was focused on the experience of a subgroup, those from a visible minority.²⁴ Existing data suggest that the views of patients, and those of their families and clinicians, are at odds (Table 1). It is not surprising that families support CTOs because, given current funding levels, they are desperate for anything that promises assistance, however illusory.²⁵ Patients can take a very different view. One has likened CTOs to house arrest in home-based institutions.²⁵ The lack of research is in marked contrast to the United States, Great Britain, and Australia where there have been large studies that used randomized or matched control subjects.^7,11–13 Given the wide-spread introduction of CTOs into Canada, it is surprising how little research has been conducted into their effectiveness. The uncertainty concerning the effectiveness of CTOs is not entirely reflected by the Canadian Psychiatric Association’s Position Paper on mandatory outpatient treatment that broadly endorses continued use.²⁶

Randomized controlled trials (RCTs) will always be difficult to conduct in this area because of practical and ethical concerns but there is sufficient uncertainty about the effectiveness of CTOs to justify a study that randomizes patients to a CTO or treatment as usual. At the very least, larger controlled before-and-after studies are required. Given the availability of province-wide administrative data, these studies could cover entire jurisdictions. Epidemiological studies using such province-wide administrative databases have several advantages over other methodologies.²⁷ For instance, all patients who are placed on CTOs can be studied, including patients with a history of dangerousness who are commonly excluded from other research designs. It is also possible to follow up almost all the intervention group and matched control subjects, other than those who die or leave the jurisdiction. Therefore, these studies are less vulnerable to selection and follow-up bias. In addition, some important outcomes, such as mortality, cannot easily be assessed by an RCT.

One potentially fruitful approach would be the comparison of adjacent jurisdictions with and without CTOs or other forms of extended leave. Maritime Canada offers such an opportunity as it contains 2 provinces with similar characteristics, one of which has CTOs (Nova Scotia) and the other which has no such legislation or other forms of extended leave (New Brunswick).²⁷ This would therefore be a perfect laboratory for a comparison of provinces that would not have the limitations of previous work comparing jurisdictions from different countries.²¹

Government-commissioned reports are no substitute for investigator-initiated, peer-reviewed research. It is unlikely that any other group of patients would be subjected to an equally invasive intervention in the absence of an adequate evidence base.