The Cost-Effectiveness of a Novel Online Social Therapy to Maintain Treatment Effects From First-Episode Psychosis Services: Results From the Horyzons Randomized Controlled Trial

Abstract

Background

Digital interventions have potential applications in promoting long-term recovery and improving outcomes in first-episode psychosis (FEP). This study aimed to evaluate the cost-effectiveness of Horyzons, a novel online social therapy to support young people aged 16–27 years following discharge from FEP services, compared with treatment as usual (TAU) from a healthcare sector and a societal perspective.

Study design

A cost-effectiveness analysis (CEA), based on the change in social functioning, and a cost-utility analysis (CUA) using quality-adjusted life years were undertaken alongside a randomized controlled trial. Intervention costs were determined from study records; resources used by patients were collected from a resource-use questionnaire and administrative data. Mean costs and outcomes were compared at 18 months and incremental cost-effectiveness ratios were calculated. Uncertainty analysis using bootstrapping and sensitivity analyses was conducted.

Study results

The sample included 170 participants: Horyzons intervention group (n = 86) and TAU (n = 84). Total costs were significantly lower in the Horyzons group compared with TAU from both the healthcare sector (–AU$4789.59; P < .001) and the societal perspective (–AU$5131.14; P < .001). In the CEA, Horyzons was dominant, meaning it was less costly and resulted in better social functioning. In the CUA, the Horyzons intervention resulted in fewer costs but also yielded fewer QALYs. However, group differences in outcomes were not statistically significant. When young people engaged more with the platform, costs were shown to decrease and outcomes improved.

Conclusions

The Horyzons intervention offers a cost-effective approach for improving social functioning in young people with FEP after discharge from early intervention services.

Introduction

Psychotic disorders are among the most devastating of all-mental disorders. They represent a major burden for patients, their families, and societies.¹ The first onset of psychosis usually occurs between the ages of 15 and 25 and involves severe psychotic symptoms, such as hallucinations, delusions, thought disorder, negative symptoms disorders, and significant comorbid symptomatology.² The early course of psychosis is characterized by recurrent relapses and up to 80% of first-episode psychosis (FEP) patients will experience a psychotic relapse within 5 years of remission from the initial episode.³ These symptoms severely disrupt a person’s ability to complete education, obtain employment, and develop intimate relationships.^4,5 As a result, lifelong social isolation,⁶ unemployment,⁶ social anxiety,⁷ chronic depression,^7,8 substance use, and suicide are high in people living with psychosis.^9,10 Compared to healthy controls, patients with FEP also experience significantly lower quality of life (QoL).¹¹

Since the early 2000s, early intervention services for psychosis have spread globally to support young patients achieve both symptomatic remission and long-term full functional recovery.¹² However, maintaining treatment effects remains a significant challenge.¹³ Specialist FEP services typically provide intensive support for two years, but evidence suggests that some important treatment benefits seen at the end of this period may not persist over time.¹⁴ Therefore, the goal of FEP services to improve functioning and prevent relapse over the long term remains unrealized for many young people. The recognition of these limitations has created an impetus for developing effective and sustainable treatment models focused on preventing relapse and improving long-term recovery from early psychosis. Providing lower-intensity care beyond discharge from specialized services has been proposed because of a promising and sustainable approach for sustaining the social recovery of FEP patients.¹⁵

Online, mobile, and social media interventions provide a promising and potentially cost-effective alternative to promote recovery and extend the benefits of specialized FEP services following the initial 2 years of specialized support.^16–19 Emerging evidence demonstrates that online and mobile-based interventions are promising and at least as effective as usual care.¹⁹ They are also feasible, acceptable, and engaging for young people with mental ill-health.²⁰ A recent online psychosocial intervention (Horyzons) aims to foster long term recovery in FEP and bridge the gap between specialized intervention and standard treatment.²¹ Horyzons integrates peer-to-peer online social networking with theory-driven, evidence-informed therapeutic interventions targeting social functioning, vocational recovery, and relapse prevention supported by peer workers, clinicians, and vocational professionals, with the aim of supporting young people with FEP following 2 years of specialized support. Recent findings demonstrated that the Horyzons intervention improved vocational or educational attainment and reduced young people’s use of hospital emergency services.¹⁸ Despite these promising outcomes, the cost-effectiveness of the Horyzons intervention remains unknown. Generally, there is a lack of evidence evaluating the cost-effectiveness of online interventions within the context of mental health.^19,22 Therefore, the aim of this study was to evaluate the cost-effectiveness of Horyzons from a healthcare sector and a societal perspective in comparison with treatment as usual (TAU) over 18 months following a young person’s discharge from a specialized FEP service.

Methods

The economic evaluation was conducted alongside the randomized controlled trial (RCT) of Horyzons. In reporting the design and findings of our economic evaluation, we adhere to the 2022 Consolidated Health Economic Evaluation Reporting Standards presented in Supplementary Table 1.²³

Study design and participants

The Horyzons study was an 18-month, parallel-group, single-blind, and phase 4 RCT. Research assistants collected data at baseline, 6-month, 12-month, and 18-month follow-up interviews. Participants aged 16–27 diagnosed with FEP, in remission and nearing discharge from the Early Psychosis Prevention and Intervention Centre (EPPIC), Melbourne, were eligible to participate. The study protocol was registered (ANZCTR; ACTRN12614000009617) and published elsewhere.²¹ The trial was approved by the Melbourne Health Human Research Ethics Committee (HREC/12/MH/151; ref. 2013.146). Written consent was obtained from all-participants or their parent/legal guardian for participants under the age of 18 years. Recruitment commenced in October 2013 and the last follow-up assessment was completed in July 2018. Following discharge from EPPIC, eligible participants were randomly assigned (1:1), based on a computer-generated randomization schedule, to either TAU plus Horyzons, or TAU alone.

Intervention

Horyzons was developed in partnership with young people and based on the Moderated Online Social Therapy (MOST) model.^24,25 The MOST model integrates: (1) interactive online therapy (“Pathways and Steps”), (2) peer-to-peer online social networking (“the café”), (3) peer moderation, and (4) expert support by mental health clinicians (eg, clinical psychologist, and social workers) and vocational workers trained in Individual Placement and Support. The online therapy pathways target key risk factors and salient domains in a person’s early recovery process. These pathways include content about: understanding psychosis, identifying and exercising personal strengths, promoting positive connections with others, fostering positive emotions, early warning signs and prevention of relapse, managing stress and anxiety, dealing with depression, and vocational skills. In addition, the content of pathways is tailored to individual clinical characteristics by online clinicians. The therapy objectives of pathways and steps are enhanced through discussions young people have with peer-workers in the “cafe” (purpose-built online social network) under the guidance of “coaches” (expert moderators). Further descriptions of the Horyzons features can be found elsewhere.^18,21

Comparator

The comparator was TAU following discharge from the Melbourne EPPIC clinic. Standard treatment consists of follow-up support options with either a general practitioner, private psychiatrist, primary care youth mental health services, or adult mental health service. Additionally, participants were provided with a booklet containing practical information on e-mental health resources (eg, Moodgym and Reach-out).

Outcome measures

Our primary outcome measure was a change in social functioning from baseline to 18-month follow-up, measured by the Personal and Social Performance Scale (PSP). The PSP provides a single, overall rating score ranging from 1 to 100, with higher scores representing better personal and social functioning.²⁶ The PSP is a reliable and validated tool and is recommended for measuring social functioning in psychosis.²⁷ The PSP formed the outcome measure of the cost-effectiveness analysis (CEA).

We also undertook a cost-utility analysis (CUA) based on Quality-Adjusted Life Years (QALYs), measured using the Assessment of Quality of Life 8-Dimensions (AQoL-8D) questionnaire. The AQoL-8D is a multi-attribute utility instrument, developed for the purpose of increasing sensitivity to psychosocial and mental health-related domains of QoL.²⁸ The AQoL-8D performs best in the mental health context^29,30 and comprises 35 items that assess eight domains of QoL, including independent living, relationships, mental health, coping, pain, senses, self-worth, and life satisfaction. A total utility score is calculated, ranging from 0 (denoting death) to 1 (denoting full health), based on preferences from the Australian general population.²⁸ QALYs were calculated using the “area under the curve” approach based on the utility scores at baseline, 6-, 12-, and 18-month follow-up. A 5% discount rate was applied to QALYs that occurred after 12 months.³¹

Service use and cost

Service use and associated costs were measured from a healthcare sector and a societal perspective, as recommended by The Second Panel on Cost-Effectiveness in Health and Medicine,³² using a Resource Use Questionnaire (RUQ) that was administered by a research assistant at baseline, 6, 12, and 18-month follow-up interviews. The baseline RUQ was relatively brief, capturing key resources used over the past month. Components of the resources used over 6 months in follow-up RUQ included: (i) headspace services, (ii) specialized mental health clinical services, (iii) private mental health services, (iv) medication and diagnostic tests, (v) inpatient services and emergency services, (vi) productivity impacts, and (vii) other nonhealth services (eg, financial advice and vocational guidance). In Australia, headspace centres are one-stop-shop youth-friendly early intervention mental health services for young people aged 12–25 years with access to a range of health professionals.^33,34

We calculated costs by multiplying volumes of health services by related unit costs (Supplementary Table 2). Where appropriate, we asked participants how long each consultation with the health professional was, their mode of transport to make a return journey from home to each health care provider, whether they were accompanied by someone, and the estimated out-of-pocket costs. We obtained unit costs for different resources from the Medicare Benefits Schedule (MBS) for health professional visits and diagnostic tests (using a weighted average cost paid by the government),³⁵ and the Pharmaceutical Benefits Scheme (PBS) for medication.³⁶ We accessed online Australian retail pharmacy sites to determine patient costs for other medications and supplements not covered by the PBS.³⁷ The National Hospital Cost Data Collection Cost Report (Round 18) was used for hospital stays and emergency department visits³⁸; the same report was sourced to cost specialized mental health services based on nonadmitted services (Tier 2) that included outpatient community services. For hospital stays and emergency department visits, we applied different unit costs depending on whether the reason for the visit was mental health-related (based on reasons provided in the RUQ). Unit costs for other services, such as financial advice or vocational guidance were obtained from the National Disability Insurance Scheme pricing agreement³⁹; some professionals costs were based on national employee earnings.⁴⁰ Travel costs were calculated according to the type of transport used (eg, car, public transport, and taxi) and a 20 km return trip was assumed. The use of ambulances was costed separately. Time costs were calculated based on the average length of each visit reported in the RUQ in addition to travel time, which was assumed to be 30 min. An assumption was made that visits to health professionals occurred outside the working hours for patients that engaged in formal employment. Time costs were valued at 25% of the average wage rate to represent the value of participants’ lost leisure time.⁴¹ The same unit cost was used to value the time cost of the person accompanying the participant to the visits, where applicable. We based productivity costs on the number of days participants were absent from work and valued those using the human capital approach, because recommended.³² Thereby, lost working hours were multiplied by the average hourly wage rate. Unpaid and volunteer work were valued at 25% of the average wage rate. All-costs are presented in Australian Dollars (AUD) for the 2014 reference year; unit costs reported for other years were adjusted for inflation using industry-specific indices.⁴² Discounting of 5% was applied to all-costs occurring beyond 1 year.^31,43 The impact inventory provided in Supplementary Table 3 outlines the details of cost components by the respective perspective adopted.³²

Participants were also asked to provide consent for accessing their MBS and PBS data over the 18-month trial period, and 1 month prior to the baseline assessment. PBS data provides the patient contribution and the amount paid by the government for each prescription medication through the scheme. The MBS data provides the total provider charge, the government benefit paid and the out-of-pocket costs. The MBS and PBS information was supplied by Services Australia. We used MBS and PBS data in the sensitivity analyses and replaced the relevant components in the RUQ with the administrative data (Supplementary Table 3). We obtained, from the Centre for Victorian Data Linkage, that manages the Victorian Admitted Episodes Dataset (VAED), administrative hospital admission data for all-participants. This is a minimum data set for each patient admission for all-Victorian public and private hospitals.

Intervention cost

We calculated the cost of delivering the Horyzons intervention using a microcosting approach, based on study records. Training costs included 2-day workshops, delivered to peer support workers by a peer support coordinator. We also cost the time spent by peer moderators to lead the peer-to-peer online social networking (“the café”). The cost for intervention delivery included the time of clinical moderators, who provided guidance to young people, monitored their clinical status, and ensured the safety of the social networking environment. The system was moderated daily during weekdays (ie, 2 h/d), twice daily on weekdays, and once daily over weekends. If a young person required vocational assistance, the vocational moderator provided individualized online support. We based personnel costs on real salaries paid during the Horyzons trial; further details are provided in Supplementary Table 4. Although costs associated with the initial research, design, and set-up of the intervention are considered “sunk” costs (unrecoverable past expenditures), a mean cost of $9.56 was added to all-participants in the intervention group. This was derived because the mean cost per user based on the eligible population likely to receive the intervention once rolled out and expected uptake rates, which have previously been applied.⁴⁴ The eligible population consisted of Australians aged 16–25 years⁴⁵ living with a psychotic disorder and in contact with specialized mental health services (0.31%).⁶ It was assumed that 50% would meet the eligibility criteria for Horyzons and 50% would agree to use the intervention. We used the same approach to the cost of the ongoing operation of the Horyzons platform over the course of the trial, which included IT resources for system maintenance. The final average cost per participant was estimated at $2004 (over an 18-mo period) and was added to each participant in the intervention group.

Statistical analyses

We conducted all-analyses in STATA 15 according to the intention-to-treat principle. We undertook a missing values analysis by group allocation (Supplementary Table 5 and Supplementary Figures 1 and 2). We applied multiple imputations by chained equations to impute missing values for cost and effects using the “ice” package in STATA.⁴⁶ In total, we imputed 50 datasets that reflected the percentage of cumulative missing values. The imputation model included all-cost categories (eg, hospitalisation, time cost), PSP and AQoL-8D scores at all-time points, administrative data, age, and sex. We then ran the costs and outcomes analysis models across the imputed data sets and combined them using Rubin’s rule.⁴⁷

For both arms, we calculated mean outcomes and costs from both a healthcare sector perspective and a societal perspective. Generalized Linear Models (GLM) were used to assess the mean differences in costs and outcomes between the two arms, adjusted for sex, age, and baseline values. For mean differences in total costs, we applied a gamma distribution and a log link whereas for the incremental effect in QALYs, a Gaussian distribution and identity link were used. We examined differences in PSP change scores using ordinary least squares regression analysis. Incremental cost-effectiveness ratios (ICERs) based on PSP change and QALYs were calculated by dividing the overall difference in mean total costs between the intervention group and the control group by the difference in mean outcomes between the two groups. The nonparametric “bootstrapping” technique was then used with 1000 iterations of the ICER so that we could plot sampling uncertainty on cost-effectiveness planes.⁴⁸

We assessed the budget impact associated with the introduction of the Horyzons intervention using the cost estimates from the trial, which we extrapolated to all-Australians aged 16–25 years using population data from the Australian Bureau of Statistics, along with prevalence, incidence, and mortality rates reported for people with psychosis aged 18–24 years.^6,49 In doing so, we explored two scenarios: (1) adopting an optimistic scenario that assumed participation of all-eligible patients with psychosis aged 16–25 years in Australia; and (2) a more pessimistic scenario that assumed inclusion criteria, participation rates, and dropout rates observed in the trial.

Sensitivity analyses

To examine the robustness of the results, we carried out some sensitivity analyses. First, we completed the analysis using participants with complete data. A second sensitivity analysis was conducted that included presenteeism cost (ie, working while ill), based on the World Health Organization’s Health and Work Performance Questionnaire (HPQ).⁵⁰ We assessed reduced work performance as a percentage below 100% and estimated the implied loss in hours of work by multiplying this estimate by the number of hours worked. Due to an omission in the RUQ asking participants how many days they felt their work performance was reduced, an assumption was made that performance reported applied to 50% of the working days in the past 6 months. Another sensitivity analysis was undertaken by replacing the relevant components in the RUQ with the administrative MBS, PBS, and VAED data. For CUA, we conducted a further sensitivity analysis to explore the robustness of the results by not discounting QALYs after 1 year.

We also explored costs and outcomes in the intervention group via user profiles. These profiles were determined in a previous study using the Horyzons dataset and included: (1) low use; (2) maintained use of social components; and (3) maintained use of both therapy and social components.⁵¹ These were explored descriptively only.

Results

The sample included 170 participants, who were randomized to the Horyzons intervention group (n = 86) or TAU (n = 84). The baseline sample characteristics in Table 1 indicate that the groups were well-matched at baseline. Although baseline AQoL-8D values were slightly lower and baseline mean costs slightly higher for the Horyzons intervention group, these differences were not statistically significant. A similar percentage of participants consented to the use of MBS or PBS data.

Table 1.

Baseline characteristics of study sample

	Horyzons (n = 86)	TAU (n = 84)	Total (n = 170)
Age, mean ± SD	21.01 ± 2.93	20.81 ± 2.83	20.91 ± 2.88
Gender, N (%)
Males	45 (52.33)	45 (53.57)	90 (52.94)
Females	41 (47.67)	39 (46.43)	80 (47.06)
Employment status, N (%)
Unemployed	32 (39.02)	24 (29.27)	56 (34.15)
Studying only	16 (19.51)	23 (28.05)	39 (23.78)
Paid work only	20 (24.39)	17 (20.73)	37 (22.56)
Concurrent study and paid work	14 (17.07)	18 (21.95)	32 (19.51)
Education status, N (%)
Not currently studying	54 (62.79)	39 (46.43)	93 (54.71)
Not currently studying but enrolled	2 (2.33)	4 (4.76)	6 (3.53)
Studying part-time	5 (5.81)	14 (16.67)	19 (11.18)
Studying full-time	25 (29.07)	27 (32.14)	52 (30.59)
Education level, N (%)
Y 8	1 (1.16)	2 (2.38)	3 (1.76)
Y 9	7 (8.14)	7 (8.33)	14 (8.24)
Y 10	16 (18.60)	19 (22.62)	35 (20.59)
Y 11	16 (18.60)	20 (23.81)	36 (21.18)
Y 12	45 (52.33)	36 (42.86)	81 (47.65)
PSP, mean ± SD	66.64 ± 13.66	66.27 ± 12.85	66.46 ± 13.22
AQoL-8D total, mean ± SD	.58 ± .23	0.60 ± .21	0.59 ± 0.22
AQoL-8D mental, mean ± SD	.28 ± .20	0.29 ± .18	0.28 ± 0.19
AQoL-8D physical, mean ± SD	.66 ± .21	0.69 ± .21	0.67 ± 0.21
Baseline costs, mean ± SD a	682.51 ± 1084.58	591.36 ± 1020.84	637.74 ± 1051.62
Consented to use of MBS, N (%)	63 (73.26)	62 (73.81)	125 (73.53)
Consented to use of PBS, N (%)	59 (68.60)	59 (70.23)	118 (69.41)

^aCost in 2014 AUD capturing resources used 1 month before the baseline assessment.

We provide a detailed analysis of the costs and outcomes by treatment allocation in Table 2. We found that mean costs for emergency department visits and time costs were statistically significantly lower in the Horyzons group compared with TAU. While lower costs in the Horyzons group were also detected across all-other cost components, they did not reach statistical significance. The total costs were significantly lower in the Horyzons groups compared with TAU from both the healthcare sector (mean difference –$4789.59; 95% CI –5724.70 to –3854.46; P < .001) and societal perspectives (mean difference -$5131.14; 95% CI –6116.98 to –4145.29; P < .001). In both groups, PSP scores slightly decreased from baseline to 18-month follow-up interviews but the decrease was smaller in the Horyzons group. While AQoL-8D scores were lower in the Horyzons group at baseline compared with TAU, the QoL improved in both groups. No statistically significant group differences were observed on the PSP or AQoL-8D at any timepoint, including total QALYs.

Table 2.

Mean cost and outcomes by treatment allocation

	Horyzons (n = 86)		Control (n = 84)		Mean difference	SE	95% CI	P-value
	Mean	SD	Mean	SD
Intervention cost	2004	0	0	0	2004	0	–	–
Headspace services	492.96	186.21	480.99	152.282	11.96	255.031	–507.97; 531.89	.985
Specialized mental health clinical services	6677.7	2271.3	8899.09	2339.15	–2221.38	3131.70	–8553.04; 4110.27	.467
Community/ Private mental health services	1453.03	250.12	2338.27	553.31	–885.24	580.05	–2061.02; 290.55	.092
Emergency department visits	144.86	57.73	403.73	106.45	–258.87	122.00	–501.65; –16.10	.033
Hospital admissions	5978.55	1668.93	9189.76	2261.35	–3211.21	2804.16	–8825.94; 2403.53	.252
Ambulance	270.56	99.04	522.84	157.47	–252.27	182.45	–617.28; 112.73	.172
Medication	1844.74	379.83	1668.14	358.46	176.60	502.75	–835.54; 1188.75	.722
Diagnostic tests	198.10	48.87	245.83	58.29	–47.73	73.39	–196.41; 100.95	.514
Other	31.92	15.13	38.38	13.64	–6.46	19.27	–45.92; 32.99	.682
Out-of-pocket	628.77	123.96	734.20	161.93	–105.44	219.30	–553.06; 342.19	.634
Accompanied by someone	168.00	34.11	293.07	59.84	–125.06	73.36	–272.61; 22.49	.075
Time cost	153.41	23.88	281.73	50.64	–128.32	54.81	–237.10; -19.55	.009
Travel cost	98.04	12.20	106.46	12.58	–8.41	18.25	–45.28; 28.46	.641
Absenteeism	1653.04	429.05	1726.35	871.22	–73.30	907.33	–1907.49; 1760.88	.977
Total health care costs a	19693.3	3199.95	24482.9	3711.48	–4789.58	477.04	–5724.70; –3854.46	<.001
Total societal costs a	21797.7	3316.39	26928.8	3995.55	–5131.14	502.91	–6116.98; –4145.29	<.001
PSP baseline	66.64	1.47	66.27	1.40	0.37	2.03	–3.65; 4.38	.858
PSP 6m	67.74	2.34	66.46	1.66	1.27	2.91	–4.51; 7.06	.663
PSP 12m	66.22	2.76	63.62	2.21	2.61	3.56	–4.53; 9.74	.468
PSP 18m	65.59	2.42	64.81	2.15	0.78	3.31	–5.86; 7.42	.815
PSP change (baseline to 18m)	–1.053	2.530	-1.467	2.300	0.414	3.495	–6.574; 7.400	.906
AQoL-8D baseline	0.582	0.027	0.602	0.024	–-0.020	0.036	–-0.091; 0.051	.575
AQoL-8D 6m	0.549	0.043	0.631	0.031	–-0.082	0.053	–-0.188; 0.024	.128
AQoL-8D 12m	0.643	0.042	0.645	0.036	–-0.002	0.058	–-0.118; 0.115	.975
AQoL-8D 18m	0.619	0.032	0.622	0.030	–-0.003	0.044	–-0.091; 0.084	.943
QALYs	0.881	0.039	0.929	0.033	–-0.048	0.052	–-0.151; 0.056	.479

^aTotal costs include intervention costs ($2004 for HORYZONS group). P-values for costs are based on GLM (family gamma, link log) adjusted for baseline costs; P-values for QALYs are based on GLM (family Gaussian, link identity) adjusted for baseline utility; P-values for PSP are based on OLS.

Bootstrapped incremental costs, outcomes and ICERs are provided in Table 3, indicating that Horyzons was dominant, which means it was less costly and resulted in better PSP change scores irrespective of the perspective adopted. In total, 58% of the bootstrapped iterations fell in the southeast (ie, dominant) quadrant when adopting a societal perspective (Figure 1). Further, 38% fell in the southwest quadrant, which is associated with lower costs but also fewer health benefits. Supplementary Figure 3 shows the cost-effectiveness plane from a healthcare sector’s perspective with similar findings. In the CUA, the Horyzons intervention resulted in fewer costs but also yielded fewer QALYs, with most of the bootstrapped iterations falling into the southwest quadrant (see Supplementary Figures 4 and 5). Cost and outcomes for the Horyzons intervention group by user profile are presented in Table 4, indicating a trend of decreasing costs and improved outcomes with greater levels of user engagement.

Table 3.

Bootstrapped incremental cost and outcomes, ICER and distribution of bootstrapped iterations on cost-effectiveness plane

	Incremental costs, mean (95% CI)	Incremental outcomes, mean (95% CI)	ICER (95% CI)	NE	NW inferior	SW	SE dominant
CEA ($/PSP) a
Health care sector	–$4857.37 (–10 767.82; 613.93)	0.504 (–3.717; 4.714)	Dominant (SE: dominant; NW: dominated)	1.1%	2.7%	37.9%	58.3%
Societal	–$5144.03 (–10 710.42; 809.19)	0.489 (–3.707; 4.624)	Dominant (SE: –511; NW: –58)	1.4%	2.8%	37.8%	58%
CUA ($/QALYs) b
Health care sector	–$4952.46 (–10 217.13; 202.12)	–0.031 (–0.077; 0.016)	$160 366 (SW) (SE: dominant; NW: dominated)	0.3%	2.7%	87.8%	9.2%
Societal	–$5290.31 (–11 030.00; -23.43)	–0.031 (–0.077; 0.016)	$170 306 (SW) (SE: dominant; SW: 1853)	2%	2.1%	88.4%	9.3%

^aPSP change scores adjusted for sex and age; Costs adjusted for baseline cost, sex, and age.

^bQALYs adjusted for baseline utilities, sex, and age; costs adjusted for baseline cost, sex, and age.

CEA = cost-effectiveness analysis; PSP = personal and social performance scale; NE = north-east quadrant; NW=north-west quadrant; SW=south-west quadrant; SE=south-east quadrant; CUA=cost-utility analysis; QALYs=quality-adjusted life years.

Fig. 1.

Cost-effectiveness plane for CEA—societal perspective. Note: In the northeast quadrant, the intervention is cost-effective if the ICER falls under the specified value-for-money criterion because the intervention is more effective and more costly than the comparator. In the southeast quadrant, the intervention is less costly and more effective than the comparator (ie, dominant), therefore the intervention is likely to be excellent value-for-money. In the southwest quadrant, the intervention is less costly and less effective, therefore the decision to adopt the intervention may be based on decision-makers willingness to accept some health loss relative to cost-saving. Finally, in the northwest quadrant, the results show the intervention is associated with greater costs but less health gain, therefore, not a good option to adopt (ie. dominated).

Table 4.

Cost and outcomes for the intervention group by user engagement, unadjusted means (SD)

	Horyzons a				Control (n = 84)
	Total (n = 86)	Low usage (n = 49)	Maintained usage of social components (n = 19)	Maintained usage of both therapy and social components (n = 14)
Costs
Health care costs	19 693.3 (3199.95)	22 239.8 (4258.19)	16 524.4 (4316.9)	13 062 (5489.11)	24 482.9 (3711.48)
Societal costs	21 797.7 (3316.7)	24 583.5 (4413.83)	18 372.1 (4660.64)	14 545.1 (5763.08)	26 928.8 (3995.55)
PSP
PSP baseline	66.64 (1.47)	67.14 (2.07)	69.05 (2.42)	62.21 (3.51)	66.27 (1.40)
PSP 18 m	65.59 (2.41)	65.48 (3.14)	66.94 (3.39)	66.75 (4.33)	64.81 (2.14)
PSP change	–1.053 (2.530)	–1.666 (3.354)	–2.113 (3.469)	4.54 (4.580)	–1.467 (2.300)
QALYs	0.881 (0.040)	0.868 (0.055)	0.889 (0.063)	0.944 (0.086)	0.929 (0.033)

^aFour participants from the Horyzons intervention group could not be allocated to a profile group due to drop-out.

PSP = personal and social performance scale; QALYs = quality-adjusted life years.

Findings from the sensitivity analyses, shown in Supplementary Table 6, demonstrate that the results remained robust to the different scenarios tested. In the CEA, the Horyzons intervention consistently resulted in fewer costs and better PSP outcomes; in the CUA costs were lower and the intervention yielded fewer QALYs, although this difference was not statistically significant. Supplementary Table 7 presents the results of the budget impact analysis. For both perspectives, the estimates in the first year were higher, because we assumed all-young people with psychosis would be treated during the year, while in the subsequent years, we only included new cases. Adopting an optimistic scenario, the expected net budget savings were estimated at $6.3 million from the healthcare perspective and $7.5 million from the societal perspective in year one ($788 164 and $939 312 in subsequent years, respectively). Under the more pessimistic scenario, these estimates reduced to $1.3 million in savings from the healthcare perspective and $1.5 million from the societal perspective in year 1 ($143 877 and $171 428 in subsequent years, respectively).

Discussion

This study reports findings from the economic evaluation of an online intervention designed to provide ongoing support for young people with FEP beyond discharge from specialized services (Horyzons). Young people randomized to the Horyzons intervention had significantly lower overall costs when compared to the TAU group. The online intervention also resulted in small improvements in social functioning, however, there was a significant uncertainty observed in the outcome data. The uncertainty analysis showed that most of the bootstrapped iterations fell in the southeast quadrant (dominant), indicating that Horyzons is both cost-effective and cost-saving. When using QALYs as an outcome measure, Horyzons remained cost saving but most of the bootstrapped iterations fell in the southwest quadrant, indicating that the intervention resulted in fewer costs but also yielded fewer QALYs.

Our analysis of the costs and outcomes of the Horyzons group by the level of user engagement sheds light on the outcome differences between groups, revealing that Horyzons users who showed consistent engagement with the social and therapy components experienced greater improvements in both social functioning and QoL compared with young people with lower usage and those allocated to TAU. Thus, the losses in QoL in the Horyzons group (which were numerically small and not statistically different) are likely to represent the heterogeneity of user engagement with the Horyzons intervention and a dose-response effect, where more consistent usage led to greater treatment benefits and cost-effectiveness, although we are unable to establish causality. Three user profiles have been explored in a previous analysis of Horyzons data, showing that the maintained therapy and social group had higher negative symptoms at baseline and showed statistically significant improvements in social functioning, negative symptoms and overall psychiatric symptom severity compared with the other user profiles with lower usage of the online platform and the TAU group.⁵¹ The study concluded that although social network is a key ingredient to increased sustained engagement, it is important to engage young people with therapeutic content to achieve long-term recovery.

A key strength of our study was the use of a comprehensive RUQ. However, this may have resulted in response fatigue as indicated by the proportion of missing values, which might have also been the result of the 6-month recall period adopted in the RUQ. Although a previous study used an even longer recall period (ie, 12 months) to capture service use in people living with psychosis,⁵² the proportion of recall error is increased with longer recall periods, usually resulting in underreporting of service use.⁵³ The use of administrative data in the sensitivity analyses confirmed findings from the base case analyses, indicating the robustness of the findings irrespective of missing values. Additionally, this study adopted two perspectives as recommended,³² capturing not only healthcare costs but also broader societal costs. Yet, not all-societal costs were captured, such as criminal justice costs, informal care costs as well as education costs. While we attempted to measure costs due to presenteeism, the error in the RUQ meant that we had to apply assumptions and our results need to be interpreted carefully. Further, the use of the human capital approach compared with the friction cost approach may have overestimated productivity cost.⁵⁴

QALYs were generated using the AQoL-8D, which performed best in previous studies within the context of mental health.^29,30 However, we would like to acknowledge the availability of mental health-specific preference-based measures, such as the CORE-6D⁵⁵ or the ReQoL-UI.⁵⁶ It is unknown to what extent the use of a mental health-specific preference-based measure would have influenced our findings. We did not present cost-effectiveness acceptability curves given bootstrapped iterations spanned all-four quadrants in the cost-effectiveness plane and the uncertainty associated with the willingness to accept health losses in the south-west quadrant.⁵⁷

Overall, the results indicate that Horyzons offers a cost-effective and likely cost-saving approach for improving social functioning in young people with FEP after they are discharged from specialized FEP services. While large net budget savings in the Australian context were estimated using the trial data, these are likely conservative estimates given that a national roll-out is expected to lead to a significantly more efficient and optimized caseload for clinicians. These findings have important implications for delivering FEP services in Australia and internationally. Sustaining the benefits of specialist FEP services and improving long-term recovery is a critical global research and clinical priority in psychosis treatment and across youth mental health.⁵⁸ Horyzons provides a novel, promising, engaging, and cost-effective intervention to do so.

Funding

The study was funded by the Mental Illness Research Fund (MIRF, Victorian Government, Australia); the Australian National Health and Medical Research Council (NHMRC); and the Telstra Foundation.

Conflict of Interest statement

Nothing to declare.