Abstract
Objective
This study used data from a multi‐site evaluation of six assisted outpatient treatment (AOT) programs to assess the effectiveness of AOT on treatment adherence, clinical functioning, and social functioning outcomes.
Methods
Sites provided primary data (e.g., structured client interviews across multiple timepoints) and subsets of secondary data on Medicaid/non‐Medicaid service encounters, local/state arrest records, and public/private hospitalizations. One of the six sites provided primary data for a comparison population. Pre‐post analyses pooled data across each of the six AOT sites and incorporated robustness checks using comparison and secondary data.
Results
Among 392 AOT clients, appointment and treatment adherence increased by more than 24% and 20%, respectively, in 6‐ and 12‐month follow‐up windows relative to baseline. Symptomology, perceived mental health ratings, life satisfaction, and therapeutic alliance scores all improved significantly. Violent behavior was reduced by more than 19%, and suicidal ideation decreased by more than 24%. There was more than a 40% reduction in the percentage of clients with any psychiatric inpatient episodes, and more than an 8‐night reduction in the number of nights spent in psychiatric inpatient care. The proportion of clients with arrests decreased by more than 19%, those with any illicit drug use decreased by more than 14%, and in the 6 months following entry into AOT, there was a 12% reduction in the proportion of clients experiencing any homelessness.
Conclusions
Findings point to significant improvements across a range of client outcomes upon receipt of AOT, spanning domains of treatment adherence, clinical functioning, and social functioning outcomes.
Highlights
This study used primary and secondary data from six assisted outpatient treatment (AOT) program sites to evaluate the effectiveness of AOT on treatment adherence, clinical functioning, and social functioning outcomes.
Clients demonstrated substantial and sustained improvements across multiple domains following receipt of AOT, including better treatment adherence, reduced psychiatric hospitalizations, decreased criminal justice involvement, and improved mental health and life satisfaction.
Clients who spent at least 6 months under an AOT order showed greater reductions in violence, suicidal ideation, psychiatric hospitalizations, and homelessness than those with a shorter order.
Assisted outpatient treatment (AOT) is a programmatic intervention in mental health (MH) law that is designed to improve treatment participation and long‐term clinical outcomes for certain adults with serious mental illnesses (SMI). Legally, AOT extends the state's civil commitment authority from hospital‐based care into the realm of community MH services (1). Mandated services under AOT typically include outpatient pharmacotherapy with medication management, some form of intensive case management such as Assertive Community Treatment (ACT) and, often, access to subsidized housing. AOT is enforced by the noncriminal sanctions of a time‐limited civil court order, informally “committing” the service system to the person as much as committing the individual to comply with treatment. The American Psychiatric Association has endorsed AOT as a tool to promote recovery and decrease likelihood of clinical deterioration among those with SMI (2).
At present, AOT is authorized in 48 states (all except Connecticut and Massachusetts). To be eligible for AOT an individual typically must be 18 years of age or older, have a diagnosis of SMI—usually a persistent psychotic illness or major mood disorder that impairs insight into need for treatment—and be considered able to live safely in the community with available supports. However, statutory variation and variability in site‐level determinations of AOT both within and across jurisdictions means that no two implementations of AOT in the United States look exactly alike (3). For example, programs may use a preventive, step‐up approach for people in the community, or they may use a step‐down approach, essentially taking on the form of a conditional release for those in inpatient psychiatric care.
In general, prior studies have shown that AOT programs can be effective at improving key outcomes across treatment engagement and clinical and social functioning (Table 1). Studies conducted in the United States comprise the bulk of research on AOT, and have reported increased receipt of medication, reduced hospitalizations, and reduced lengths of stay, all persisting post‐AOT.
TABLE 1.
Research summary and strength of findings. a
| Area and finding b | United States | ||
|---|---|---|---|
| North Carolina (4, 5, 6, 7, 8, 9, 10) | New York (11, 12, 13, 14, 15, 16) | Other domestic (17, 18, 19) | |
| Study characteristics | |||
| Analytic design | RCT (AOT vs. case management) | Pre–post + RCT (AOT vs. case management) | Pre–post (AOT vs. case management) |
| Sample size(s) c | N = 331 | N = 3576 | Ns = 19–231 |
| Data sources | Client interviews, administrative records | Medicaid claims, case manager reports | Medical records, court records |
| Client outcomes on AOT | |||
| Reduced hospitalization | ++ | +++ | +/− |
| Increased receipt of medication | ++ | +++ | – |
| Increased receipt of case management | ++ | +++ | – |
| Improved functioning | ++ | ++ | – |
| Decreased arrests | ++ | ++ | – |
| Client outcomes post‐AOT | |||
| Reduced hospitalization | – | +++ | – |
| Increased receipt of medication | – | +++ | – |
| Increased receipt of case management | – | +++ | +++ |
AOT, assisted outpatient treatment; RCT, randomized controlled trial.
Key: +++ very strong evidence; ++ moderately strong evidence; + some evidence; +/− equivocal findings; – not tested.
Sample size differed across analyses as a function of data availability; this table presents the total N for parent studies.
For example, the Duke Mental Health Study in North Carolina examined the effectiveness of AOT compared with that of case management services alone (4). The randomized controlled trial (RCT) found that AOT significantly reduced the monthly risk of hospitalization in repeated measures multivariate analyses but found no significant difference in the proportion of participants with any hospitalizations in a year, except for consumers who experienced longer court orders. When compared with those who underwent brief periods of AOT (i.e., <6 months), individuals with extended AOT (i.e., 6‐month to a year) had a lower probability of arrest, fewer inpatient hospitalizations, reduced costs, and improved treatment adherence outcomes. However, the length of court order was not randomized.
In contrast, the Bellevue Study, an RCT of a pilot AOT program in New York, found no significant differences between AOT and control (e.g., case management) participants in hospital readmission rates and lengths of stay, although the small sample size and a lack of enforcement of the court order in the case of noncompliance have been cited as limitations (11). However, a subsequent retrospective, quasi‐experimental evaluation in New York was conducted following the enactment of a statewide AOT statute known as Kendra's Law (12). This evaluation showed multiple improved outcomes for AOT recipients, including reduced hospitalizations and lengths of stay, and increased receipt of psychotropic medications and intensive case management services. Still, it was unclear whether the New York AOT study's findings would generalize to other places with fewer resources.
In response to the methodological limitations present in prior studies on AOT, as well as in recognition that AOT's effectiveness will likely vary depending on the ways in which it is implemented, researchers have advocated for multisite observational studies using an array of data to consider those varying contexts and outcomes (20, 21, 22, 23, 24). Following congressionally mandated grant funding for new AOT funding, the Office of the Assistant Secretary for Planning and Evaluation, in consultation with the Substance Abuse and Mental Health Services Administration (SAMHSA) and the National Institute of Mental Health, supported an evaluation of SAMHSA's AOT Grant Program for Individuals with Serious Mental Illness, a 4‐year pilot program that funded AOT grants set forth in the 2014 Protecting Access to Medicare Act. The purpose of the project carried out by RTI International, in partnership with Duke University and Policy Research Associates, was to conduct an in‐depth evaluation of six of the 18 SAMHSA‐funded AOT sites. Here, we report findings of the effectiveness of AOT on key treatment, clinical functioning, and social functioning outcomes.
METHODS
Data
Data were collected by case study sites between September 2017 and March 2021. The primary data source was a structured client interview, conducted at baseline and at 6‐month follow‐up. A total of 392 participants were interviewed. These interviews gathered information around multiple domains, including housing, perceived functioning and wellbeing, clinical symptoms, treatment history and service use, medication use, substance use, and criminal justice involvement. Typically, non‐primary treating clinicians collected these data, though in some cases sites used a local evaluator. All individuals administering the interview received training on how to reduce participant burden while ensuring quality/completeness of collected data. One AOT site additionally provided interview data for a comparison population not under the AOT program.
Three of six sites provided supplemental data on Medicaid/non‐Medicaid service encounters, local/state arrest records, and/or public/private hospitalization data. These data elements overlapped considerably with self‐reported information from the structured client interview, and were analyzed as a supplemental, confirmatory analysis. Each of these three sites provided secondary data for their AOT population, as well as an ACT or ACT‐like comparison population.
Data collection and de‐identification was conducted by participating sites under the SAMHSA grant. RTI's institutional review board deemed this resulting evaluation not human subjects research.
Measures
Measures from the structured client interview mapped onto constructs of treatment adherence, clinical functioning, and social functioning. Treatment adherence was measured across two dimensions: (1) adherence to appointments and (2) adherence to prescribed psychotropic medications. For both measures, adherent individuals were defined as those who reported that they never or rarely missed an appointment or taking a medication. Clinical functioning was captured with seven discrete, self‐report measures pertaining to clinical health: (1) modified Colorado Symptom Inventory score (25), (2) perceived MH status, (3) perceived functioning, (4) life satisfaction, (5) three therapeutic alliance scores, (6) violence/victimization score, and (7) suicidal ideation. Social functioning was measured with four discrete variables: (1) use of acute psychiatric health care, (2) arrests, (3) illicit drug use, and (4) homelessness. All measures were defined based on a 6‐month lookback period.
We additionally included several regression controls in our analyses: demographic characteristics (gender, age, race/ethnicity, marital status, children, education level, and employment status) and previous use of MH services. Moderators included an indicator for whether the client was “stepped‐up” from the community or “stepped‐down” from a psychiatric inpatient setting; whether the client was on an AOT order for at least 6 months; whether site staff considered the client to have successfully completed their AOT order; and whether they had any prior criminal justice involvement.
Statistical Analysis
One of the AOT sites provided structured client interview data for a comparison population of individuals who voluntarily enrolled in an ACT or intensive case management program at a sister site. However, post‐hoc power analyses showed that this study was not powered to detect effects for most of our outcomes using a comparison group strategy. Instead, we present results from a pre‐post analysis pooling data across each of the six AOT sites and incorporate multiple robustness checks to determine whether the pre‐post results potentially overstate the impacts of AOT on client‐level outcomes from baseline to 6‐ and 12‐month follow‐ups. Although we rely primarily on the pre‐post analyses in this study, we conducted a series of complementary comparison group analyses which paralleled the pre‐post analyses, controlling for the same set of confounders and using a propensity score analysis. The propensity score model was a logistic regression, and inverse propensity of treatment weights were produced to use as analytic weights in the outcome models.
The pre‐post analyses used a regression model to adjust for multiple confounding factors, and compared the regression adjusted means during one of two follow‐up periods to the regression adjusted means during the baseline period. Bayes factors were calculated for all analyses to qualify the strength of the evidence under a categorization scheme ranging from “barely worth more than a mention” to “decisive evidence” (26). The robustness checks included testing: (1) whether comparison group results had sufficiently large Bayes factors and were in the same direction as pre‐post analyses; (2) whether pre‐post analyses with overlapping secondary data were in the same direction/significance as the primary pre‐post analyses; and (3) whether comparison group analyses with overlapping secondary data were in the same direction/significance as primary pre‐post analyses.
RESULTS
Sample Characteristics at Baseline
Table 2 presents sociodemographic characteristics of the client sample (N = 392) at baseline, as well as treatment adherence, clinical functioning, and social functioning in the 6 months prior to AOT.
TABLE 2.
| Characteristic | Mean/% | SD |
|---|---|---|
| Sociodemographic characteristics | ||
| Female | 39.3% | |
| Black, non‐Hispanic | 39.2% | |
| White, non‐Hispanic | 53.4% | |
| Other/Hispanic | 7.5% | |
| Age in years | 40.3 | 13.3 |
| Married | 5.4% | |
| Any children | 42.8% | |
| High school graduate | 58.9% | |
| Some college | 17.5% | |
| Enrolled in school at baseline | 5.8% | |
| Employed at baseline | 13.7% | |
| Client stepped down from inpatient setting | 79.4% | |
| CJ involved at baseline | 24.1% | |
| Treatment adherence constructs | ||
| Percent deemed appointment adherent | 68.3% | |
| Percent deemed medication adherent | 69.2% | |
| Clinical functioning constructs | ||
| MCSI score | 3.7 | 0.9 |
| Perceived MH rating | 2.6 | 0.9 |
| Life satisfaction score | 4.8 | 1.3 |
| Working alliance inventory: Goal scale | 14.7 | 4.2 |
| Working alliance inventory: Task scale | 14.7 | 4.5 |
| Working alliance inventory: Bond scale | 15.9 | 4.5 |
| Any violence | 24.6% | |
| Any suicidal ideation | 29.7% | |
| Social functioning constructs | ||
| Any psychiatric ED visits | 14.0% | |
| Any psychiatric inpatient episodes | 61.8% | |
| Number of psychiatric inpatient nights | 12 | 23 |
| Any arrests | 27.6% | |
| Any illicit drug use | 28.1% | |
| Homelessness | 16.9% | |
AOT, assisted outpatient treatment; CJ, criminal justice; ED, emergency department; MCSI, modified Colorado Symptom Index; MH, mental health; SD, standard deviation.
N = 392.
Treatment Adherence, Clinical Functioning, and Social Functioning Outcomes
Table 3 shows that AOT was associated with statistically significant changes in treatment adherence, clinical functioning, and social functioning outcomes.
TABLE 3.
| Outcome/follow‐up period | Change | 95% CI |
|---|---|---|
| Treatment adherence outcomes | ||
| Percent deemed appointment adherent, % a | ||
| 6‐month follow‐up | 25.9*** | (15.9, 35.9) |
| 12‐month follow‐up | 24.2* | (3.1, 45.2) |
| Percent deemed medication adherent, % a , b | ||
| 6‐month follow‐up | 20.2** | (7.7, 32.8) |
| 12‐month follow‐up | 21.8*** | (10.9, 32.7) |
| Clinical functioning outcomes | ||
| MCSI score | ||
| 6‐month follow‐up | 0.7*** | (0.3, 1.0) |
| 12‐month follow‐up | 0.8*** | (0.4, 1.2) |
| Perceived MH rating | ||
| 6‐month follow‐up | 0.5*** | (0.2, 0.7) |
| 12‐month follow‐up | 0.5*** | (0.3, 1.0) |
| Life satisfaction | ||
| 6‐month follow‐up | 0.4** | (0.2, 0.7) |
| 12‐month follow‐up | 0.8** | (0.3, 1.3) |
| Therapeutic alliance, goals | ||
| 6‐month follow‐up | 2.3** | (0.9, 3.7) |
| 12‐month follow‐up | 2.6** | (0.6, 4.5) |
| Therapeutic alliance, tasks | ||
| 6‐month follow‐up | 2.2** | (0.6, 3.8) |
| 12‐month follow‐up | 3.0*** | (1.5, 4.6) |
| Therapeutic alliance, bond | ||
| 6‐month follow‐up | 1.7* | (0.3, 3.2) |
| 12‐month follow‐up | 2.2** | (0.7, 3.6) |
| Any violence, % | ||
| 6‐month follow‐up | −19.0** | (−33.4, −4.6) |
| 12‐month follow‐up | −22.5** | (−36.6, −8.4) |
| Any suicidal ideation, % | ||
| 6‐month follow‐up | −24.7** | (−40.0, −9.3) |
| 12‐month follow‐up | −26.7*** | (−41.6, −11.8) |
| Social functioning outcomes | ||
| Any psychiatric ED visits, % | ||
| 6‐month follow‐up | −12 | (−38.6, 14.7) |
| 12‐month follow‐up | −12.1 | (−39.0, 14.9) |
| Any psychiatric IP episodes, % | ||
| 6‐month follow‐up | −41.0* | (−75.0, −6.9) |
| 12‐month follow‐up | −46.9*** | (−73.2, −20.7) |
| Number of psychiatric inpatient nights | ||
| 6‐month follow‐up | −9.6* | (−17.7, −1.6) |
| 12‐month follow‐up | −8.9** | (−15.3, −2.6) |
| Any arrests, % | ||
| 6‐month follow‐up | −19.2*** | (−21.7, −16.6) |
| 12‐month follow‐up | −24.0*** | (−26.0, −22.1) |
| Any drug use, % | ||
| 6‐month follow‐up | −14.2*** | (−21.8, −6.6) |
| 12‐month follow‐up | −19.6*** | (−26.3, −12.9) |
| Homelessness, % | ||
| 6‐month follow‐up | −12.2** | (−21.0, −3.4) |
| 12‐month follow‐up | −8.7 | (−25.8, 8.3) |
AOT, assisted outpatient treatment; CI, confidence interval; ED, emergency department; IP, inpatient; MCSI, modified Colorado Symptom Index; MH, mental health.
The following outcomes used a logistic regression to model the probability of the outcome: any violence, suicidal ideation, any arrests, any illicit drug use, and homelessness. The remaining models used an ordinary least squares regression functional form. All models controlled for the following confounders: gender, age, race/ethnicity, marital status, parenthood, education, indicators for in school or employed, criminal justice involvement at baseline, length of AOT order, and an indicator for whether the client was stepped down from an institutional setting. Marginal effects were calculated so that all estimates are interpreted as the% change in the outcome observed at 6‐month follow‐up. N = 392.
*p < 0.05, **p < 0.01, ***p < 0.001.
Treatment Adherence
Appointment adherence increased by more than 24% in both 6‐ and 12‐month follow‐up windows relative to baseline appointment adherence. Medication adherence also increased by more than 20% in both 6‐ and 12‐month follow‐up windows relative to baseline.
Clinical Functioning
Modified Colorado Symptom Index (MCSI) scores, perceived MH ratings, life satisfaction, and therapeutic alliance scores all increased by a significant and meaningful amount. Violent behavior was reduced by more than 19% in 6‐ and 12‐month follow‐up windows as compared with the baseline period, and suicidal ideation decreased by more than 24% in 6‐ and 12‐month follow‐up windows as compared with the baseline period.
Social Functioning
There was not a statistically significant change in the percentage of clients with any psychiatric emergency department visits. However, there was more than a 40% reduction in the percentage of clients with any psychiatric inpatient episodes, and more than an 8‐night reduction in the number of nights spent in psychiatric inpatient care. The percentage of clients with arrests decreased by more than 19%, the percentage of clients with any illicit drug use decreased by more than 14%, and in the 6 months following entry into AOT, there was a 12% reduction in clients experiencing any homelessness.
Moderators of AOT Outcomes
Table 4 shows that step‐down status compared to step‐up status was not associated with a statistically significant difference in changes for any of the observed outcomes. However, criminal justice involvement at baseline potentially moderated changes in violent behavior and illicit drug use, such that those with criminal justice involvement at baseline had a 28.8% decrease in the likelihood of any violent behavior, whereas clients with no involvement had a 14.5% decrease. Clients with criminal justice involvement at baseline had a 23.7% decrease in the likelihood of illicit drug use, whereas clients with no involvement had a 9.5% decrease.
TABLE 4.
Potential moderators of observed changes in clinical and social functioning outcomes among AOT participants at 6‐month follow‐up. a , b
| Outcome/moderator | Change for moderated group | Change for unmoderated group | Difference in change | 95% CI |
|---|---|---|---|---|
| Clinical functioning outcomes | ||||
| MCSI score | ||||
| Client was stepped down | 0.7 | 0.5 | 0.1 | (−0.6, 0.9) |
| Criminal justice involvement at baseline | 0.9 | 0.5 | 0.3 | (−0.1, 0.7) |
| At least a 6‐month AOT order | 0.5 | 0.7 | −0.2 | (−0.5, 0.2) |
| Successful order completion | 0.8 | 0.3 | 0.4** | (0.2, 0.7) |
| Any violence, % | ||||
| Client was stepped down | −17.6 | −22.1 | 4.5 | (−3.7, 12.8) |
| Criminal justice involvement at baseline | −28.8 | −14.5 | −14.3*** | (−16.1, −12.6) |
| At least a 6‐month AOT order | −24.5 | −14.7 | −9.8** | (−14.4, −5.1) |
| Successful order completion | −19.5 | −16.7 | −2.9 | (−6.3, 0.6) |
| Suicidal ideation, % | ||||
| Client was stepped down | −25.3 | −16.3 | −9.0 | (−37.8, 19.7) |
| Criminal justice involvement at baseline | −16.6 | −26.2 | 9.6 | (−6.6, 25.8) |
| At least a 6‐month AOT order | −27.4 | −21.4 | −5.9** | (−9.7, −2.2) |
| Successful order completion | −22.1 | −26.8 | 4.7 | (−2.6, 12.1) |
| Social functioning outcomes | ||||
| Number of psychiatric IP nights | ||||
| Client was stepped down | −10.7 | −6.8 | −3.9 | (−20.0, 12.3) |
| Criminal justice involvement at baseline | −5.8 | −11.3 | 5.5 | (−0.7, 11.6) |
| At least a 6‐month AOT order | −2.5 | −14.5 | 12.0 | (5.9, 18.1) |
| Successful order completion | −9.8 | −10.3 | 0.5 | (−10.6, 11.6) |
| Any arrests in past 6‐month, % | ||||
| Client was stepped down | −18.4 | −14.5 | −3.9 | (−10.0, 2.1) |
| Criminal justice involvement at baseline | ‐ | ‐ | ‐ | ‐ |
| At least a 6‐month AOT order | −11.1 | −18.6 | 7.5 | (−0.5, 15.6) |
| Successful order completion | na | na | na | na |
| Any illicit drug use, % | ||||
| Client was stepped down | −14.0 | −10.1 | −3.9 | (−12.8, 5.0) |
| Criminal justice involvement at baseline | −23.7 | −9.5 | −14.2*** | (−22.0, −6.4) |
| At least a 6‐month AOT order | −13.6 | −12.9 | −0.6 | (−19.5, 18.2) |
| Successful order completion | −11.0 | −18.7 | 7.7 | (1.5, 13.9) |
| Any homelessness, % | ||||
| Client was stepped down | −10.7 | −17.5 | 6.8 | (−1.4, 15.0) |
| Criminal justice involvement at baseline | −9.2 | −13.0 | 3.9 | (−7.8, 15.5) |
| At least a 6‐month AOT order | −20.5 | −8.3 | −12.2*** | (−18.0, −6.4) |
| Successful order completion | −14.8 | −7.9 | −7.0* | (−12.9, −1.0) |
AOT, assisted outpatient treatment; CI, confidence interval; MCSI, modified Colorado Symptom Index; na, not available (results not available due to lack of convergence).
The following outcomes used a logistic regression to model the probability of the outcome: any violence, suicidal ideation, any arrests, any illicit drug use, and homelessness. The remaining models used an ordinary least squares regression functional form. All models controlled for the following confounders: gender, age, race/ethnicity, marital status, parenthood, education, indicators for in school or employed, criminal justice involvement at baseline, length of AOT order, and an indicator for whether the client was stepped down from an institutional setting. Marginal effects were calculated so that all estimates are interpreted as the% change in the outcome observed at 6‐month follow‐up. N = 392.
*p < 0.05, **p < 0.01, ***p < 0.001.
Length of the AOT order also potentially moderated changes in violent behavior, suicidal ideation, psychiatric inpatient utilization, and homelessness. The length of the initial order and subsequent renewals varied widely across clients, largely as a function of state statute and clinical staff judgment. The average length of an AOT order with renewals was 228.7 days (SD = 142.5). Among clients with an AOT order of at least 6 months, we observed a 24.5% decrease in the likelihood of any violence from baseline to 6‐month follow‐up, whereas among clients with an AOT order of <6 months, we observed a 14.7% decrease. Clients who had an AOT order of at least 6 months had a 27.4% decrease in the likelihood of suicidal ideation, whereas clients with an AOT order of <6 months had a 21.4% decrease. Clients with an AOT order of at least 6 months had 14.5 fewer nights, whereas clients with an AOT order of <6 months only had 2.5 nights fewer psychiatric inpatient nights. Clients with an AOT order of at least 6 months had a 20.5% decrease in the likelihood of experiencing any homelessness, whereas clients with an AOT order <6 months had a smaller, 8.3% decrease.
In addition, successful order completion per clinical staff judgment potentially moderated changes in MCSI scores, illicit drug use, and homelessness. Clients who successfully completed an AOT order had a 0.8‐point increase in their MCSI scores, whereas clients who did not successfully complete an AOT order had only a 0.3‐point increase. This finding suggests that symptoms improved more among those who successfully completed AOT orders than among those who did not. Although the likelihood of any illicit drug use decreased generally, clients who successfully completed orders had an 11.0% decrease in their likelihood of illicit drug use, whereas clients who did not successfully complete orders had a larger 18.7% decrease. It should be noted that some programs routinely administered drug testing in some form. Lastly, clients who successfully completed AOT orders had a 14.8% decrease in the likelihood of experiencing any homelessness, whereas clients who did not had a 7.9% decrease.
DISCUSSION
Prior research on the efficacy of AOT has shown variance in the strength and significance of findings across studies, leaving questions around the effectiveness of AOT on health and social outcomes (22). Moreover, the existing evidence base on AOT does not sufficiently account for differences in AOT program implementation or client characteristics in relation to client outcomes. As a result, there are gaps in our understanding of how and for whom AOT works (22, 24).
The present study used data from six AOT programs to assess effectiveness of AOT on key outcomes. Findings from pooled, within‐group analyses showed that AOT is associated with improvements in treatment adherence, clinical functioning, and social functioning outcomes from 6 months prior to AOT to 6‐ and 12‐month follow‐ups upon initiation of AOT. During and after AOT, clients exhibited improvements in appointment and medication adherence. We also found significant improvements in client symptomology, perceived MH, and life satisfaction. AOT clients reported an increase in therapeutic alliance and reductions in violent behaviors and suicidal thinking while on AOT and after the order. However, it should be noted that serious acts of violence were very rare and reductions in violent behavior mainly involved minor acts of violence. There was evidence of a substantial reduction in number of inpatient psychiatric hospitalizations and days in the hospital during and after AOT. We additionally found evidence of improved social functioning associated with AOT, including reductions in arrests, drug use, and homelessness that, apart from homelessness, all persist post‐AOT. Bayesian analyses showed that there was decisive evidence of improvements in nearly all of the client‐level outcomes assessed.
Results from comparisons to non‐AOT participants offer additional perspective regarding the strength of the evidence regarding AOT's effectiveness; however, it is important to note that these comparisons were generally underpowered. Among sufficiently powered analyses, improvements in perceived MH status were greater among AOT clients, as was the reduction in psychiatric inpatient encounters. Therapeutic alliance improvements were mixed, with only the working alliance inventory showing a greater change among AOT clients relative to non‐AOT clients. Other outcomes were underpowered in between‐group analyses. Despite this, point estimates suggested little clinical significance in the relative changes for most outcomes. Accordingly, these findings suggest that treatment provided under AOT delivers comparable outcomes to alternate treatment options, such as ACT or similar intensive case management.
As AOT does not purport to deliver “better” outcomes, but rather to increase engagement in the treatment needed to obtain such outcomes, the question remains as to whether these AOT clients would have seen the same improvements without the court order. Notably, compared with AOT clients at baseline, the clients in the comparison group were less likely to be appointment or medication adherent, had a similar probability of having at least one psychiatric inpatient episode, and had fewer average nights spent in a psychiatric inpatient setting. Although propensity scores were used to adjust for between‐group differences, our findings do not provide the same clarity around AOT versus non‐AOT outcomes as would be provided in a true experimental study, like an RCT. To that end, we tested potential moderators representing select differences in how AOT may be implemented, as well as key characteristics of the target population.
Findings on length of time on the order underscore those established previously (4). Specifically, spending at least 6 months on an AOT order was associated with a reduced likelihood of violent behavior, fewer suicidal thoughts, fewer psychiatric inpatient nights, and a reduced likelihood of homelessness. Those who successfully completed AOT orders showed better improvement in symptomology and reduced homelessness, albeit a smaller decrease in illicit drug use. There were no noted differences in outcomes between those who are referred from the community (step‐up) and from inpatient hospitalization or jail stay (step‐down). In contrast, criminal justice involvement at time of initiation of the AOT order was associated with subsequently greater reductions in violent behavior and arrests, suggesting that this population is within the scope of AOT's effectiveness.
One important limitation to these findings is that these AOT programs were in a start‐up phase, still developing clinical and administrative procedures and gearing up their associated ACT or ICM teams. Many prior AOT studies, with the exception of the Bellevue study, represented mature AOT programs. It is not certain that mature AOT programs would yield stronger improvements, but many of these findings may have been affected by being in a formative phase.
CONCLUSIONS
Previous observational studies and RCTs have shown that AOT can be effective at improving client engagement and functioning, although the strength of evidence for each outcome differs across studies. Our findings point to significant improvements across a range of client outcomes, including appointment and medication adherence, symptomology, perceived MH, life satisfaction, and therapeutic alliance during and after AOT. AOT clients also demonstrate reductions in violent behaviors, suicidal thinking, arrests, drug use, homelessness, and number of inpatient hospitalizations and days in the hospital.
The analyses for this study were performed under contract HHSP233201600021I with the Assistant Secretary for Planning and Evaluation. The contents are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Department of Health and Human Services or any of its agencies.
The authors report no conflicts of interest or financial relationships with commercial interests.
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1. Fitch L, Swanson J. Civil commitment and the mental health care continuum: historical trends and principles for law and practice. Rockville: Substance Abuse and Mental Health Services Administration, Office of the Chief Medical Officer. https://www.samhsa.gov/sites/default/files/civil‐commitment‐continuum‐of‐care.pdf (2019). Accessed 01 Dec 2024. [Google Scholar]
- 2. American Psychiatric Association . Position statement on involuntary outpatient commitment and related programs of assisted outpatient treatment. https://www.psychiatry.org/getattachment/d50db97b‐59aa‐4dd4‐a0ec‐d09b4e19112e/Position‐Involuntary‐Outpatient‐Commitment.pdf (2015). Accessed 01 Dec 2024.
- 3. Meldrum ML, Kelly EL, Calderon R, Brekke JS, Braslow JT. Implementation status of assisted outpatient treatment programs: a national survey. Psychiatr Serv. 2016;67(6):630–635. 10.1176/appi.ps.201500073 [DOI] [PubMed] [Google Scholar]
- 4. Swartz MS, Swanson JW, Hiday VA, Wagner HR, Burns BJ, Borum R. A randomized controlled trial of outpatient commitment in North Carolina. Psychiatr Serv. 2001;52(3):325–329. 10.1176/appi.ps.52.3.325 [DOI] [PubMed] [Google Scholar]
- 5. Swartz MS, Swanson JW. Economic grand rounds: can states implement involuntary outpatient commitment within existing state budgets? Psychiatr Serv. 2013;64(1):7–9. 10.1176/appi.ps.201200467 [DOI] [PubMed] [Google Scholar]
- 6. Swanson JW, Swartz MS, Wagner HR, Burns BJ, Borum R, Hiday VA. Involuntary out‐patient commitment and reduction of violent behaviour in persons with severe mental illness. Br J Psychiatry. 2000;176(4):324–331. 10.1192/bjp.176.4.324 [DOI] [PubMed] [Google Scholar]
- 7. Swartz MS, Swanson JW, Wagner HR, Burns BJ, Hiday VA, Borum R. Can involuntary outpatient commitment reduce hospital recidivism? Findings from a randomized trial with severely mentally ill individuals. Am J Psychiatr. 1999;156(12):1968–1975. 10.1176/ajp.156.12.1968 [DOI] [PubMed] [Google Scholar]
- 8. Swanson JW, Borum R, Swartz MS, Hiday VA, Wagner HR, Burns BJ. Can involuntary outpatient commitment reduce arrests among persons with severe mental illness? Crim Justice Behav. 2001;28(2):156–189. 10.1177/0093854801028002002 [DOI] [Google Scholar]
- 9. Swartz MS, Swanson JW, Wagner HR, Burns BJ, Hiday VA. Effects of involuntary outpatient commitment and depot antipsychotics on treatment adherence in persons with severe mental illness. J Nerv Ment Dis. 2001;189(9):583–592. 10.1097/00005053-200109000-00003 [DOI] [PubMed] [Google Scholar]
- 10. Swanson JW, Swartz MS, Elbogen EB, Wagner HR, Burns BJ. Effects of involuntary outpatient commitment on subjective quality of life in persons with severe mental illness. Behav Sci Law. 2003;21(4):473–491. 10.1002/bsl.548 [DOI] [PubMed] [Google Scholar]
- 11. Steadman HJ, Gounis K, Dennis D, Hopper K, Roche B, Swartz M, et al. Assessing the New York City involuntary outpatient commitment pilot program. Psychiatr Serv. 2001;52(3):330–336. 10.1176/appi.ps.52.3.330 [DOI] [PubMed] [Google Scholar]
- 12. Swartz MS, Wilder CM, Swanson JW, Van Dorn RA, Robbins PC, Steadman HJ, et al. Assessing outcomes for consumers in New York's assisted outpatient treatment program. Psychiatr Serv. 2010;61(10):976–981. 10.1176/appi.ps.61.10.976 [DOI] [PubMed] [Google Scholar]
- 13. Gilbert AR, Moser LL, Van Dorn RA, Swanson JW, Wilder CM, Robbins PC, et al. Reductions in arrest under assisted outpatient treatment in New York. Psychiatr Serv. 2010;61(10):996–999. 10.1176/appi.ps.61.10.996 [DOI] [PubMed] [Google Scholar]
- 14. Phelan JC, Sinkewicz M, Castille DM, Huz S, Link BG. Effectiveness and outcomes of assisted outpatient treatment in New York State. Psychiatr Serv. 2010;61(2):137–143. 10.1176/ps.2010.61.2.137 [DOI] [PubMed] [Google Scholar]
- 15. Van Dorn RA, Swanson JW, Swartz MS, Wilder CM, Moser LL, Gilbert AR, et al. Continuing medication and hospitalization outcomes after assisted outpatient treatment in New York. Psychiatr Serv. 2010;61(10):982–987. 10.1176/ps.2010.61.10.982 [DOI] [PubMed] [Google Scholar]
- 16. Schneeberger AR, Huber CG, Lang UE, Muenzenmaier KH, Castille D, Jaeger M, et al. Effects of assisted outpatient treatment and health care services on psychotic symptoms. Soc Sci Med. 2017;175:152–160. 10.1016/j.socscimed.2017.01.007 [DOI] [PubMed] [Google Scholar]
- 17. Bursten B. Posthospital mandatory outpatient treatment. Am J Psychiatr. 1986;143:1255–1258. [DOI] [PubMed] [Google Scholar]
- 18. Geller J, Grudzinskas AJ, McDermeit M, Fisher WH, Lawlor T. The efficacy of involuntary outpatient treatment in Massachusetts. Adm Policy Ment Health. 1998;25:271–285. [DOI] [PubMed] [Google Scholar]
- 19. Van Putten RA, Santiago JM, Berren MR. Involuntary outpatient commitment in Arizona: a retrospective study. Psychiatr Serv. 1988;39(9):953–958. 10.1176/ps.39.9.953 [DOI] [PubMed] [Google Scholar]
- 20. Morrissey JP, Desmarais SL, Domino ME. Outpatient commitment and its alternatives: questions yet to be answered. Psychiatr Serv. 2014;65(6):812–815. 10.1176/appi.ps.201400052 [DOI] [PubMed] [Google Scholar]
- 21. Swanson JW, Swartz MS. Why the evidence for outpatient commitment is good enough. Psychiatr Serv. 2014;65(6):808–811. 10.1176/appi.ps.201300424 [DOI] [PubMed] [Google Scholar]
- 22. Cripps SN, Swartz MS. Update on assisted outpatient treatment. Curr Psychiatry Rep. 2018;20(12):1–4. 10.1007/s11920-018-0982-z [DOI] [PubMed] [Google Scholar]
- 23. Swartz MS, Swanson JW. Randomised controlled trials and outpatient commitment. Lancet Psychiatry. 2017;4(12):e30. 10.1016/s2215-0366(17)30443-1 [DOI] [PubMed] [Google Scholar]
- 24. Hancq ES, Munetz M, Silver SC, Parker HA, Bonfine N. Critical gaps in assisted outpatient treatment research in the United States. Adm Policy Ment Health. 2024;51(6):1–4. 10.1007/s10488-024-01377-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Conrad KJ, Yagelka JR, Matters MD, Rich AR, Williams V, Buchanan M. Reliability and validity of a modified Colorado Symptom Index in a national homeless sample. Ment Health Serv Res. 2001;3:141–153. 10.1023/a:1011571531303 [DOI] [PubMed] [Google Scholar]
- 26. Lee MD, Wagenmakers EJ. Bayesian cognitive modeling: a practical course. Cambridge, United Kingdom: Cambridge University Press; 2014. [Google Scholar]