How did a Housing First intervention improve health and social outcomes among homeless adults with mental illness in Toronto? Two-year outcomes from a randomised trial

Abstract

Objectives

We studied the impact of a Housing First (HF) intervention on housing, contact with the justice system, healthcare usage and health outcomes among At Home/Chez Soi randomised trial participants in Toronto, a city with an extensive service network for social and health services for individuals who are experiencing homelessness and mental illness.

Methods

Participants identified as high needs were randomised to receive either the intervention which provided them with housing and supports by an assertive community treatment team (HF+ACT) or treatment as usual (TAU). Participants (N=197) had in-person interviews every 3 months for 2 years.

Results

The HF+ACT group spent more time stably housed compared to the TAU group with the mean difference between the groups of 45.8% (95% CI 37.1% to 54.4%, p<0.0001). Accounting for baseline differences, HF+ACT group showed significant improvements over TAU group for community functioning, selected quality-of-life subscales and arrests at some time points during follow-up. No differences between HF+ACT and TAU groups over the follow-up were observed for health service usage, community integration and substance use.

Conclusions

HF for individuals with high levels of need increased housing stability and selected health and justice outcomes over 2 years in a city with many social and health services.

Trial registration number

ISRCTN42520374.

Strengths and limitations of this study.

• The Housing First strategy resulted in immediate housing stability and observer-rated mental health which remained high over the full follow-up period among those in the treatment arm.

• Participant functioning psychiatric hospitalisations and number of arrests improved among participants in the treatment compared to the control arm.

• In a well-resourced urban setting with myriad services to individuals who are homeless, those in the control arm were also able to achieve gains over the follow-up, though not as great as those in Housing First arm.

• Interviewers were not blinded to treatment status of participants.

• The intensity of observations may have facilitated change in treatment and control groups.

Introduction

Innovations in interventions serving individuals who experience long-term homelessness are gaining ground.¹ The shift away from a reliance on a cycle of shelters and transitional housing towards providing permanent supportive housing has been documented in North America, Europe and Australia.^2–4 ‘Treatment first’ models that seek to stabilise individuals with regard to mental illness and substance use prior to providing permanent housing are giving way to ‘Housing First’ (HF) models that are built on principles of consumer choice and mental health recovery while providing immediate access to housing and optional community supports. HF has been documented to improve a range of outcomes, including housing stability, quality of life, vocational opportunities and community integration, over treatment first or usual care models^5–12 but less so for substance abuse outcomes.¹³

The At Home/Chez Soi (AH/CS) randomised trial was conducted in five cities across Canada and enrolled individuals with serious mental illness who were experiencing chronic homelessness in the largest trial to date of the HF model.¹⁴ AH/CS consisted of scattered-site housing and off-site community supports promoting participant choice, personalised goals and fostering of resilience and empowerment. These supports were tailored to those who demonstrated high or moderate needs as assessed at baseline identified by severity of mental health conditions and levels of mental healthcare use and social problems experienced in the year prior to enrolment. Participants randomised to receive the intervention were provided with housing and supports, while those randomised to treatment as usual (TAU) were not provided with any active intervention or support from the trial. AH/CS was intentionally implemented in a variety of settings across Canada, with Toronto being a site where a large proportion of those enrolled were born outside of Canada or considered themselves to belong to an ethnoracial minority group. Moreover, Toronto has a broad network of mental health services—inpatient and outpatient services, assertive community treatment (ACT), court support services, crisis programmes (eg, mobile crisis team and the distress line) and ethnoracial-focused agencies—which serve individuals who are homeless and were available to those in TAU.

The AH/CS intervention tailors services to a level of need during initial assessment (high and moderate needs). Our HF intervention provided intensive training of AH/CS staff on the HF model and principles of mental health recovery, and implemented fidelity assessments at two points in time. Follow-up rates over time were high. While results for selected outcomes for the high needs portion of AH/CS for the cross-site sample has been published,¹⁵ examining site-specific findings can yield important information about how the intervention works locally and can inform needed improvements to local services. Findings from the cross-site analysis may differ from the site-specific results because the TAU conditions vary across cities. Of the five intervention cities, Toronto had one of the richest service settings at the start of the trial, all of which would have been available to the TAU group: 13 ACT teams were operating in Toronto, some with a focus on serving individuals who are homeless; >20 Community Health Centres many with specialised services for individuals experiencing homelessness; over 40 shelters that serve individuals living with homelessness with over 60% providing some type of health service; over 7000 supportive housing units designed for individuals with persistent mental illness; and a city run programme, Streets to Homes, that links individuals living with homelessness to temporary or permanent housing.

Sites adapted core HF strategies to suit their own contexts and populations while still being adherent to the basic principles of HF and ACT.^16–18 Close to half of the population in Toronto was born outside of Canada; thus, the Toronto site focused on those participants who identify as ethnoracial minorities. Toronto's high needs group was 40% ethnoracial compared to 27% for the five-city ACT sample.¹⁹ Prior large scale social interventions have noted the importance of including a focus on adaptation and context in understanding how and why programmes are or are not successful.²⁰ There were also demographic differences at baseline in Toronto's sample compared to the full cross-site sample such as the proportion of women (26% for Toronto vs 32% for the full sample) and substance use problems (60% for Toronto vs 73% for the full sample).²¹ Thus, an examination of site-specific findings yields further insights into whether and how HF strategies work in service-rich settings and can inform changes at the local level to improve existing services.

Methods

Sampling and recruitment

Participants for this study were from the Toronto site of the AH/CS study. For inclusion in the study, at the time of enrolment, participants had to be ≥18 years of age, absolutely homeless or precariously housed, have a confirmed major mental disorder and not be served by an ACT programme.^{14 22} Participants were recruited from October 2009 to July 2011 using a targeted recruitment where local service providers followed a structured referral strategy and those referred were further screened for eligibility by the study team to ensure that the study sample approximated those experiencing homelessness in Toronto^{22 23} (see figure 1). Referrals came from shelters, drop-in centres, hospitals, outreach programmes, and service providers and were reviewed by an intake coordinator for eligibility and assessment. A more detailed description of recruitment and exclusions at baseline can be found elsewhere.²²

Figure 1.

AH/CS Toronto site, participant flow through the study. ACT, assertive community treatment; AH/CS, At Home/Chez Soi; HF, Housing First.

At baseline, participants were assessed for disability and severity of mental health problems and then stratified into a high or moderate needs group. Participants were randomised into the high needs group based on a Multnomah Community Ability Scale (MCAS) score of <62, indicating at least moderate disability, a diagnosis of a psychotic or bipolar disorder and meeting one of the following criteria: being hospitalised for a mental illness at least twice in 1 year over the past 5 years, indication of comorbid substance use and recent arrest or incarceration. High needs participants were randomised after their baseline interview into TAU or in the HF+ACT arm of the trial. HF+ACT participants were provided with a rent allowance of $600. The ACT mental health team had a participant-to-staff ratio of 10:1, available 7 days/week; 12 hours/day²² and provided a number of relevant services which included case management, initial and ongoing assessment, psychiatric care, employment and housing assistance, family support and education, substance use services and intensive support to allow participants to live successfully in the community.²²

Data collection

Data were collected by in-person interviews every 3 months for a 2-year period and entered into a secure web-based database system. Participants were financially compensated when interviewed. For our primary, secondary and exploratory outcomes described below, data were collected and analysed at baseline and 6-month, 12-month, 18-month and 24-month time points.

Housing stability, the proportion of days spent in long-term housing while enrolled in the study, was the primary outcome of interest and was assessed using the Residential Timeline Follow-back (RTLFB) questionnaires at each interview which captured participants' 3-month residential history (ie, move in/out dates for each type of residence such as street, temporary housing, permanent housing, emergency/crisis and institution).²⁴ Among participants with at least one follow-up interview, the percentage of days stably housed was calculated as the total number of days stably housed divided by the total number of days for which any type of residence data was provided by the participant.

Two secondary outcomes were assessed. Quality of life was measured via the Euro Qol EQ-5D visual analogue that enables respondents to rate their health on a vertical scale capturing the ‘worst imaginable health state’ (score of 0) to ‘best imaginable health state’ (score of 100).¹⁴ Functional ability was assessed by trained interviewers at the end of the interview using the MCAS, a 17-item scale capturing degree of functional ability among individuals living with mental illness in the areas of health, adaptation, social skills and behaviour. Higher scores indicate little to no impairment.^{25 26}

The remaining outcomes examined were considered exploratory and were examined to generate rather than test hypotheses about the impact of HF. Lehman Quality of Life Interview 20 captured disease-specific quality of life through seven subscales (family, finances, leisure, living situation, safety, social and global). Higher scores, ranging from 7 to 140, are indicative of better quality of life.²⁷ The Colorado Symptom Index (CSI) was used to assess Mental Health symptom severity through 14 questions asking how often participants experienced specific psychiatric symptoms ‘at least every day’ to ‘not at all’ (range 1–5) generating an overall summary score ranging from 14 to 40 with lower scores reflecting fewer symptoms.^{28 14} The number of days spent in a hospital (psychiatric, general and psychiatric unit, separately) was calculated using the data obtained from the RTLFB questionnaire. The RTLFB assesses the days spent in each hospital type at each interview point and the outcome considered was the sum of days over all observation periods.¹⁴

Substance use severity was measured via the 5-item GAIN Substance Use Disorder Scale Short Screener (GAIN SS) which assesses the number of substance-related problems. Severity was derived by counting the number of problems in the past month with a range from 0 to 5 and higher values corresponding to a greater problem severity.

Community integration was measured using the Community Integration Scale (CIS) which assesses psychological integration based on levels of satisfaction with home and neighbourhood ranging from 4 to 20, and physical integration from counts of social activities such as going to a concert, meeting people for coffee and participating in sports on a scale of 0–7 with higher scores reflecting greater integration.¹⁴ Health service and justice use were also measured using two items from the Health Service and Justice Service Use questionnaire: the number of emergency department visits and the number of arrests, in the past 6 months.¹⁴

Statistical power and analyses

Site-specific sample sizes were set to 100 participants per group at enrolment and at 65 per group at 24 months to account for an attrition rate of 35%. These numbers were based on the assumption of an α of 0.05, a β of 0.20 and a moderate effect size of 0.5,^{14 15} which was anticipated for the primary and secondary outcomes.

Descriptive statistics were first generated for baseline variables among the HF+ACT and TAU groups. Data for each participant were included up until the end of the 2-year follow-up or until his or her point of death. Days stably housed over the 2-year period was examined using a two sample t-test which compares the average days stably housed for the HF+ACT and TAU groups. For the number of days hospitalised over the 2-year period, a zero-inflated negative binomial model was used to account for the likelihood of excessive zeros and overdispersion (ie, variance larger than the mean) among participants who had at least 1 day of hospitalisation⁶ (PROC GENMOD in SAS (SAS Institute) V.9.4). Some participants did not provide residence days for the full 2 years follow-up (due to missing data, attrition or death), and therefore, this model also included an offset as the natural log of total reported residence years. The mean number of days hospitalised, among those admitted during the 24 months period, rate ratios and 95% CIs were also estimated. We generated figures to display the data for all secondary and exploratory outcomes for HF+ACT and TAU groups over 24-month follow-up period.

For secondary and exploratory outcomes, which were measured longitudinally from baseline, linear and generalised linear mixed models were used for continuous and count outcomes, respectively, with a random intercept to estimate subject-level variation. An indicator for the treatment arm was included in the models, with the TAU group as the reference group and also a categorical time variable (0, 6, 12, 18 and 24) with baseline as the reference category. By including the baseline outcomes in the model, changes between arms over time were examined through the use of a treatment by time interaction. For continuous outcomes, this value may be interpreted as the difference in mean change from baseline, whereas for count outcomes, these values correspond to the ratio of the average number of events between arms (a ratio between groups) and the baseline ratio (ie, a ratio of ratios) at the 6, 12, 18 and 24 months measurement periods.

Multiple imputation with chained equations using the ‘mi impute chained’ command in STATA (StataCorp) V.13 was conducted to impute missing data due to loss to follow-up, withdrawal, skipped interviews, non-response on specific items and the lack of interviewer confidence in participant responses. Forty imputed data sets were generated and analysed using SAS PROC MIXED for continuous outcomes and PROC GLIMMIX for count outcomes, assuming an unstructured correlation matrix for the repeated measures. Results were combined across the 40 data sets using PROC MIANALYZE.⁷

All tests were two sided and statistical significance was set at a p value of <0.05.

Results

Of the 575 participants recruited into the study in Toronto, 197 participants met criteria for the ‘high needs’ group. Of these 197 ‘high needs’ participants, 97 were randomly allocated to the HF+ACT group and 100 participants were randomly allocated to the TAU group. By 24 months, 13 (17%) ‘high needs’ participants were lost to follow-up, 3 (2%) died and 2 (1%) were withdrawn (see figure 1). Demographic and mental health characteristics of the participants at baseline are presented in table 1.

Table 1.

AH/CS Toronto site, sample demographic, health and social characteristics at baseline for the total sample of high needs participants, those receiving HF+ACT and those in the TAU, 2009–2013

Characteristics	ACT (N=97)	TAU (N=100)	Total (N=197)
Age—years, mean (SD)	38.18 (11.04)	41.90 (11.81)	40.07 (11.56)
Gender, N (%)
Female	33 (34.0)	19 (19)	52 (26.4)
Male	64 (66.0)	79 (79)	143 (72.59)
Other	0 (0)	2 (2)	2 (1.02)
Marital status, N (%)
Married/partnered	3 (3.26)	4 (4)	7 (3.55)
Divorced/separated/widowed	19 (20)	25 (25)	44 (22.34)
Single/never married	73 (76.8)	71 (71)	144 (73.1)
Country of birth, N (%)
Canada	61 (62.9)	60 (60)	121 (61.42)
Other	36 (37.1)	40 (40)	76 (38.58)
Ethnic or cultural identity, N (%)
Aboriginal	7 (7.2)	3 (3)	10 (5.08)
Ethnoracial	47 (48.4)	54 (54)	101 (51.27)
White	43 (44.3)	43 (43)	86 (43.65)
Housing status, N (%)
Absolutely homeless	91 (93.8)	97 (97)	188 (95.43)
Precariously housed	6 (6.2)	3 (3)	9 (4.57)
Lifetime duration of homelessness—years, mean (SD)	5.57 (6.42)	7.13 (7.34)	76.31 (83.14)
Education, N (%)
Less than high school	48 (49.5)	53 (53.54)	101 (51.27)
Completed high school	16 (16.5)	22 (22.22)	38 (19.29)
Some postsecondary school	33 (34.0)	24 (24.24)	57 (28.93)
MCAS score, mean (SD)	54.47 (5.78)	54.97 (6.65)	54.73 (6.23)
MINI diagnostic categories, N (%)
Depressive episode	17 (17.5)	18 (18)	35 (17.77)
Manic or hypomanic episode	13 (13.4)	7 (7)	20 (10.15)
Post-traumatic stress disorder	14 (14.4)	11 (11)	25 (12.69)
Panic disorder	6 (6.2)	3 (3)	9 (4.57)
Mood disorder with psychotic features	23 (23.7)	25 (25)	48 (24.37)
Psychotic disorder	56 (57.73)	60 (60)	116 (58.88)
Substance use related problems	61 (62.89)	59 (59)	120 (60.91)
Suicidality level
No/low	28 (28.9)	38 (38)	66 (33.5)
Moderate	15 (15.5)	13 (13)	28 (14.21)
High	43 (44.3)	40 (40)	83 (42.13)

ACT, assertive community treatment; AH/CS, At Home/Chez Soi; HF, Housing First; MCAS, Multnomah Community Ability Scale; TAU, treatment as usual.

For our primary outcome, over the 24-month period, participants in the HF+ACT group spent more time stably housed compared to the TAU group (mean difference=45.8% (95% CI 37.1% to 54.4%, p<0.0001); group means: 73.6% (95% CI 67.5% to 79.6%) and 27.8% (95% CI 21.6% to 34.1%) for HF+ACT and TAU groups, respectively). We also break this down by year in table 2 which illustrates the proportion of participants stably housed by year of follow-up. For the EQ-5D quality of life, one of our secondary outcomes, there were improvements over the 24-month period for HF+ACT and TAU groups, but there was no significant treatment by time interaction (see table 2). The mean change from baseline for the MCAS score, another secondary outcome, was found to be significant for the HF+ACT group over the TAU group at 6 months, with a treatment difference of 3.08 (95% CI 0.26 to 5.91, p=0.0324) and at 24 months, with a difference of 3.18 (95% CI 0.54 to 5.84, p=0.0185). These mean changes in the MCAS score over time were indicative of improvement over baseline for the HF+ACT group compared to the TAU group (table 3).

Table 2.

Aggregated proportion (95% CI) of participants stably housed by period of follow-up and trial arm

Period of follow-up	HF+ACT	TAU
3–12 months	95.7% (91.6% to 99.8%)	37.5% (27.4% to 47.6%)
15–24 months	88.6% (82.0% to 95.3%)	61.0% (50.2% to 71.9%)

ACT, assertive community treatment; HF, Housing First; TAU, treatment as usual.

Table 3.

AH/CS Toronto site treatment by time interactions across 24 months of follow-up

	HF+ACT vs TAU
Time point(s)	6 vs 0			12 vs 0			18 vs 0			24 vs 0
Continuous outcomes	Mean	95% CI	p Value	Mean	95% CI	p Value	Mean	95% CI	p Value	Mean	95% CI	p Value
EQ-5D	−3.01	−11.56 to 5.55	0.4901	3.91	−4.00 to 11.81	0.3326	3.52	−4.86 to 11.90	0.4092	6.66	−0.88 to 14.20	0.0833
CSI	−1.15	−4.64 to 2.34	0.5178	−1.76	−5.19 to 1.67	0.3156	−1.66	−5.29 to 1.97	0.3683	−0.80	−4.25 to 2.65	0.6494
QoL-I20 total score	−0.77	−6.72 to 5.19	0.8008	0.08	−5.81 to 5.97	0.9795	0.83	−4.97 to 6.63	0.7787	−0.11	−5.70 to 5.49	0.9706
QoL-I20 family	−0.90	−2.86 to 1.06	0.3664	0.26	−1.74 to 2.26	0.7979	−0.75	−2.63 to 1.13	0.4357	−0.46	−2.35 to 1.44	0.6347
QoL-I20 finance	−0.59	−1.71 to 0.53	0.3038	−0.16	−1.27 to 0.94	0.7709	−0.57	−1.70 to 0.55	0.3144	0.56	−0.50 to 1.63	0.3007
QoL-I20 leisure	−0.84	−2.89 to 1.22	0.4248	−0.20	−2.31 to 1.90	0.8512	0.07	−2.02 to 2.15	0.9505	0.21	−1.82 to 2.23	0.8422
QoL-I20 living*	0.94	0.20 to 1.68	0.0127	0.56	−0.15 to 1.27	0.1240	0.63	−0.10 to 1.37	0.0920	0.14	−0.57 to 0.85	0.6952
QoL-I20 social	−0.19	−1.53 to 1.15	0.7816	−0.94	−2.29 to 0.41	0.1706	−0.19	−1.58 to 1.20	0.7890	−1.09	−2.40 to 0.21	0.1002
QoL-I20 safety	0.70	−1.20 to 2.61	0.5676	0.57	−1.39 to 2.53	0.5676	1.02	−0.90 to 2.94	0.2981	0.53	−1.30 to 2.37	0.5689
QoL-I20 global	0.11	−0.55 to 0.77	0.7412	0.00	−0.63 to 0.63	0.9905	0.61	−0.01 to 1.23	0.0553	0.01	−0.60 to 0.61	0.9855
MCAS	3.08	0.26 to 5.91	0.0324	1.42	−1.33 to 4.17	0.3099	2.00	−0.83 to 4.83	0.1651	3.19	0.54 to 5.84	0.0185
CIS psychological	−0.94	−2.26 to 0.39	0.1645	−0.82	−2.18 to 0.54	0.2349	−1.08	−2.44 to 0.28	0.1194	−0.71	−2.05 to 0.63	0.2965
Count outcomes	Rate	95% CI	p Value	Rate	95% CI	p Value	Rate	95% CI	p Value	Rate	95% CI	p Value
GAIN	0.89	0.61 to 1.29	0.5324	0.86	0.60 to 1.25	0.4374	0.71	0.48 to 1.06	0.0957	0.91	0.61 to 1.34	0.6224
CIS physical	0.98	0.72 to 1.34	0.9055	0.94	0.70 to 1.26	0.6791	1.06	0.78 to 1.44	0.7228	0.92	0.68 to 1.24	0.5810
Emergency room visits	0.88	0.64 to 1.19	0.4007	0.93	0.67 to 1.30	0.6853	1.34	0.96 to 1.87	0.0842	1.28	0.93 to 1.77	0.1249
Arrests*	0.55	0.23 to 1.31	0.1792	0.65	0.26 to 1.58	0.3409	0.38	0.14 to 1.02	0.0538	0.36	0.14 to 0.97	0.0426

Bold typeface indicates p values <0.05.

AH/CS, At Home/Chez Soi; CIS, Community Integration Scale; MCAS, Multnomah Community Ability Scale; TAU, treatment as usual.

Results for our exploratory outcomes varied. Among those with at least one hospitalisation, participants in the HF+ACT group experienced significantly fewer days in a psychiatric hospital compared with TAU group (HF+ACT mean=65.8, 95% CI (40.9 to 105.8); TAU mean=168.4, 95% CI (98.1 to 289.0); rate ratio: 0.4, 95% CI (0.2 to 0.8), p=0.0104) (see bars in the middle of figure 2, dark bars are the treatment group and light bars are TAU group, vertical lines are 95% CIs). For stays in a psychiatric unit or in a general hospital, however, there were no significant differences observed between the HF+ACT and TAU groups (figure 2).

Figure 2.

Differences in days hospitalised between HF+ACT and TAU groups for the high needs groups over the 24-month period of follow-up among participants with at least one hospitalisation, Toronto's AH/CS study, 2009–2013. Dark coloured bars represent the treatment group, while light coloured bars represent TAU group. Lines on the bars represent 95% CI. ACT, assertive community treatment; AH/CS, At Home/Chez Soi; HF, Housing First; TAU, treatment as usual.

When changes were compared between groups taking into account a time by treatment interaction, the HF+ACT group showed greater improvements than the TAU group over the 2-year follow-up period for just a few outcomes (see table 3 and figures 3 and 4). There was a rate reduction for arrests at 24 months compared to baseline, with the HF+ACT group having a greater reduction (0.36, 95% CI 0.14 to 0.97, p=0.0426) (see table 2). For the QoL living subscale, a significant mean change from baseline was detected at 6 months (0.94, 95% CI (0.20 to 1.68), p=0.0127) and for QoL global at 18 months (0.61, 95% CI (−0.01 to 1.23), p=0.0553) but for the latter not confirmed at the p<0.05 level (see table 3).

Figure 3.

Secondary and explanatory outcomes for HF+ACT (solid line) versus TAU (dashed line) groups over the 24-month period for the high needs participants in Toronto's AH/CS study, 2009–2013. *Differences between HF+ACT and TAU groups significant at the p<0.05 level with baseline values taken into account. ACT, assertive community treatment; AH/CS, At Home/Chez Soi; CIS, Community Integration Scale; HF, Housing First; TAU, treatment as usual.

Figure 4.

Toronto AH/CS high needs group over 24 months of follow-up. Time by treatment interactions for outcomes that differed over the 2-year follow-up period. **Differences between HF+ACT and TAU groups significant at the p<0.05 level with baseline values taken into account. ACT, assertive community treatment; AH/CS, At Home/Chez Soi; HF, Housing First; MCAS, Multnomah Community Ability Scale; TAU, treatment as usual.

Discussion

For the primary outcome of housing stability, the HF+ACT arm showed a large and significant difference of 45.8% greater time stably housed over the 24-month follow-up period compared to the TAU arm (73.6% of the time stably housed HF+ACT group vs 27.8% for the TAU group). This finding confirms what has been demonstrated in other randomised controlled trials and quasi-experiments of the HF approach in the USA^{6 29 30} and more recently multiple sites in Europe.³¹ Despite variations in country and local contexts in terms of the generosity of the social safety net and rates of homelessness across these studies, similar rates of housing stability have been reported for those enrolled in a HF programme: at or around 80% at 1 or 2 years of follow-up.^{6 29–31} With regard to our secondary outcomes, for quality of life as measured by EQ5-D, both groups improved over the 24-month follow-up period with HF+ACT group showing greater improvements but these differences did not meet the cut-off of p<0.05. For functional ability (MCAS), another secondary outcome, both groups showed improvements over time and significant differences between HF+ACT and TAU groups were observed very early (6 months) and at the last follow-up points. Since it is assessed by trained observers, this measure compliments our other self-reported outcomes on mental health symptoms.

For exploratory outcomes, only one outcome showed no improvement over the 2-year follow-up period, community integration (CIS) which reflects participation in activities such as going to restaurants, places of worship, libraries or volunteering. For HF+ACT and TAU groups, CIS levels remained flat over time. A previous study with a pre–post design in the USA reported similar findings of no change over a 1-year follow-up in almost all of these community integration activities.³² For the remaining exploratory outcomes, HF+ACT and TAU groups showed improvements in the follow-up period, but the treatment by time interactions were not significant with the exception of three outcomes: psychiatric unit hospitalisations, arrests and QoL living. Our study showed a treatment effect for days spent in psychiatric hospitals, but not in general hospitals or psychiatric units, though time spent in the latter two was very small over the 24 months. Gulcur et al⁶ in a 2-year follow-up study examined psychiatric hospitalisations and among those recruited from psychiatric hospitals, HF reduced overall stays; other types of hospitalisation were not examined in this study.

While arrests were reduced among HF+ACT and TAU groups in our study, arrests in HF+ACT group decreased significantly more than that in TAU group over baseline levels at the 24-month follow-up point. In one of the only studies of jail time among HF participants, investigators in the USA used a pre–post design to demonstrate significant reductions in arrests and days spent in jail attributed to the intervention.³³ Clifasefi et al³³ who studied reasons for arrests before and after HF enrolment concluded that bookings were almost all misdemeanours and were due to issues of homelessness (eg, public order crimes, theft, court order violation) which were therefore significantly reduced once participants were housed.

We examined seven subscales (family, finances, leisure, living situation, safety, social and global) of the Lehman Quality of Life Interview and only for the living subscale did we see significant improvements immediately after enrolment between HF+ACT and TAU groups accounting for treatment by time interactions. This difference might reflect the significant change in perceptions of living conditions upon receiving housing in the HF+ACT arm which would not have been the case for the TAU arm as relatively few were housed within the first 6 months of the study, just over 10% for TAU and 80% for HF+ACT at the 6-month point.¹⁵ A prior pre–post study measuring the Lehman QoL subscales found that at 1-year postenrolment, significant improvements were seen for the living, family relations, financial situation and social subscales.³² While we did see improvements in our sample on all these same subscales of the QoL-20, the HF+ACT and TAU groups increased at similar rates over time.

We did not see greater improvements among the HF+ACT group for mental health as measured by the Colorado Symptom Index (CSI) compared to TAU group as we anticipated. This is surprising given that the interviewer-rated MCAS showed significant improvements over TAU group for early and later time points and also psychiatric hospitalisations were significantly reduced in the HF+ACT group over the follow-up period. Past studies of HF have also reported no improvements in mental health symptoms over time.³⁴ On the other hand, improvements in substance use (via the GAIN) were also not documented in our sample over the 24-month period which often goes hand in hand with mental health symptoms. It may also be that CSI is less sensitive to detecting improvements in mental well-being than the MCAS.

Overall, for our secondary and exploratory outcomes, our data supported fewer of the hypothesised changes than we originally anticipated would occur as a result of the HF+ACT. There are a number of possible reasons why we did not see more improvements in more of the outcomes. Longitudinal evaluations that enrol participants experiencing episodic crises (eg, with housing, mental health or substance use issues) often experience results that reflect ‘regression to the mean’ where participants naturally improve regardless of the treatments being administered. Given that we saw improvements in outcomes for participants in both groups over time for all but one of our outcomes, it is certainly possible that our findings reflect this phenomenon. It is also possible that given the intensity of the study activities and observations, even for those in the TAU group, many of whom were contacted monthly by study staff, that the Hawthorne effect, where participants positively change behaviours as a result of being part of a study, resulted in improvements in both groups. Even if participants did not change their behaviours, providers in the service community may have provided more assistance to those in TAU group. Given the service-rich context in Toronto, including the availability of housing and housing supports, many TAU participants had access to housing and similar mental health and addictions services as HF participants resulting in minimal differences between the two groups over the follow-up period time. While it is beyond the scope of this article to compare the intensity of service participation for HF+ACT and TAU groups as it requires additional data from across myriad sectors of the service system, we hope to investigate this further in the future. Sample size may have been an issue which we discuss further below.

A major strength of the AH/CS approach was the supports (eg, training, staffing resources, ability to innovate and adapt to local contexts) provided to ensure that sites adhered to the HF model with fidelity assessments occurring at two time points in the project. Fidelity to HF programme components has been associated with better housing and health outcomes in prior studies.⁵ Fidelity assessments in our study, conducted early to promote midcourse corrections and later to ascertain whether programme fidelity was maintained, confirmed that the AH/CS programme was highly adherent in all areas across the full follow-up period, including housing choice, service philosophy, service array and programme structure.^{35 36} The Toronto team benefitted from the training and the local communities of practice created to ensure that housing and support providers were adhering to and not deviating from the core models.^{35 36} Yet, despite the high ratings for fidelity assessments, Toronto had other advantages that may have contributed to the overall improvements over time for all participants and the small differences observed between the HF+ACT and TAU groups. Toronto has a number of active housing programmes with the largest one being Streets to Homes. It has been available prior to AH/CS beginning in February 2005, where outreach workers house ∼600 homeless people per year with 87% remaining housed after 1 year.³⁷ In addition, >50 organisations provide various mental health or supportive housing services in the city.³⁸ Thus, in Toronto, it is possible that the added benefit of the AH/CS implementation of HF+ACT was small at best, given the rich array of pre-existing ACT and HF programmes.

One limitation to our study is that our interviewers could not be blinded to the treatment status of participants. This might have affected our finding of more positive functional ability as measured by MCAS, for example. Our sample size calculations were based on a moderate effect size and some of our findings illustrated smaller impacts of the programme. Thus, our sample size may have been insufficient to detect effect sizes smaller than a moderate effect. However, analysing our data using longitudinal methods helped us to overcome the problem of low statistical power.

Our findings have implications for services provided to individuals living with homelessness and severe mental illness in Toronto. The good news is that this service-rich setting—including numerous ACT teams, housing and supports, shelters and health service agencies (eg, CHCs)—enabled a significant proportion of the TAU group to improve in health and social outcomes over time. Yet, HF participants performed significantly better than TAU participants in the area of housing outcomes, psychiatric hospitalisations and mental health (as rated by independent observers). These findings suggest that there is room for improvement in the system of services for this population in Toronto. One area requiring more attention is systems integration of these services within sectors (eg, health and housing) and across sectors to better serve clients facing multiple health and social challenges as well as to reduce duplication. HF+ACT group had to overcome the siloed nature of these two sectors as part of the intervention¹⁷ which partially explained why HF+ACT participants were able to experience greater improvements than those in TAU group. Thus, applying those lessons to the overall service environment in Toronto should benefit all clients being served by those systems.

In summary, our findings suggest that even in a city with extensive mental health and housing services like Toronto, for a handful of outcomes, we can do better than providing (uncoordinated) services to high-risk populations. This is supported by the few significant differences we saw in our data but, perhaps more importantly, by the improvements early in the programme, right after participants are housed, for some of our outcomes in the HF+ACT group compared to the TAU group. On the other hand, our data also suggest that it may take more time for outcomes such as arrests to show improvements and that longer follow-up periods for two or more years might yield additional information about how HF+ACT affects health and social outcomes for this population.