Protocol for economic evaluation alongside a cluster-randomised controlled trial of a psychoeducational intervention for the primary prevention of postnatal mental health problems in first-time mothers

Abstract

Introduction

Postnatal mental health problems, which are an international public health priority, are a suitable target for preventive approaches. The financial burden of these disorders is borne across sectors in society, including health, early childhood, education, justice and the workforce. This paper describes the planned economic evaluation of What Were We Thinking, a psychoeducational intervention for the prevention of postnatal mental health problems in first-time mothers.

Methods and analysis

The evaluation will be conducted alongside a cluster-randomised controlled trial of its clinical effectiveness. Cost-effectiveness and costs-utility analyses will be conducted, resulting in estimates of cost per percentage point reduction in combined 30-day prevalence of depression, anxiety and adjustment disorders and cost per quality-adjusted life year gained. Uncertainty surrounding these estimates will be addressed using non-parametric bootstrapping and represented using cost-effectiveness acceptability curves. Additional cost analyses relevant for implementation will also be conducted. Modelling will be employed to estimate longer term cost-effectiveness if the intervention is found to be clinically effective during the period of the trial.

Ethics and dissemination

Approval to conduct the study was granted by the Southern Health (now Monash Health) Human Research Ethics Committee (24 April 2013; 11388B). The study was registered with the Monash University Human Research Ethics Committee (30 April 2013; CF12/1022-2012000474). The Education and Policy Research Committee, Victorian Government Department of Education and Early Childhood Development approved the study (22 March 2012; 2012_001472). Use of the EuroQol was registered with the EuroQol Group; 16 August 2012.

Trial registration number

The trial was registered with the Australian New Zealand Clinical Trials Registry on 7 May 2012 (registration number ACTRN12613000506796).

Strengths and limitations of this study.

• Prospectively planned data collection for the purposes of economic evaluation alongside the clinical trial of effectiveness.

• Will provide decision-makers with valuable evidence when considering any potential implementation of What Were We Thinking, a novel psychoeducational intervention for the prevention of postnatal mental health problems.

• Limited duration of data collection in the trial.

• Lack of data on willingness-to-pay for prevention of postnatal mental health problems.

Background

Postnatal depression (PND) is an international public health priority, being the most common cause of postnatal morbidity with a prevalence in high-income countries of approximately 13%, and presenting a challenging target for prevention.^1–4 In the short term, PND is associated with the woman's own health, quality of life and interactions with her baby, plus practical caregiving factors such as breastfeeding and sleep management.^5–9 In the longer term, women who experience PND are more likely to experience recurrent or chronic mental health problems (including but not limited to postnatal mental health problems with subsequent pregnancies) and difficulties in the maternal–infant and intimate partner relationships, including intimate partner violence.^{10 11} Their children are more likely to have psychological, behavioural, cognitive and health problems.¹²

As a result, the burden of postnatal mental health problems and their consequences are borne not only by families and the healthcare system, but also by other sectors in society including early childhood, education, justice and the workforce. As with depression at other life phases, women with PND use more health services than non-depressed women, not limited to mental health services.^13–16 In general, depression is associated with reduced work time and productivity.¹⁷ Women's absence from the workforce may be prolonged by postnatal mental health problems, which may also affect the work productivity of her partner and extended family as they care for her and/or the baby.

A major focus in this field is non-psychotic depression; however, a range of mental health problems manifest in the postnatal period, including adjustment disorders, anxiety, bipolar affective disorder and disorders of maternal–infant bonding.^{7 18} Postnatal anxiety has similar prevalence to and is often comorbid with depression.¹⁹ When considering postnatal mental health problems, it is important not to overlook these other disorders and their associated burden.

Postnatal mental health problems are recognised as a suitable target for preventive approaches, with the potential to avert the burden to women, their children and families, as well as the social and economic costs.²⁰ Despite these incentives, the search for successful preventive interventions has met with limited success. A recent systematic review found promise in certain programmes, such as professional home visits, telephone-based peer support and individual psychotherapy.²¹ However, these interventions may be better suited as targeted approaches for women at increased risk, rather than for primary prevention.

What Were We Thinking (WWWT) is a psychoeducational intervention that may be suitable for primary prevention of postnatal mental health problems. In a before and after controlled study (n=364), WWWT was found to reduce the prevalence of postpartum mental health problems in women without a history of psychiatric disorder.²² It involves group-based delivery of the programme in one 6 h session, where both parents along with the infant are encouraged to attend.

The Sleep, Parenting and Relationships in a Community Setting (SPARCS) trial is a cluster-randomised controlled trial examining the effectiveness of WWWT, as delivered by Maternal and Child Health (MCH) nurses to groups of first-time parents (FTP) in preventing non-psychotic postnatal mental health problems.²³ MCH centres in Victoria, Australia provide a free, universal service including 10 visits between birth and school age, with a focus on parenting and the health and development of the child. They are jointly funded by the Victorian Department of Education and Early Childhood Development and local governments. MCH services see more than 90% of women who give birth in Victoria each year.²⁴ Many MCH centres facilitate FTP's groups, which provide an opportunity for education of new parents, as well as for social connections to form between parents in a local community.²⁵ WWWT may be suitable for integration into FTP groups.

This paper describes the protocol for the economic evaluation of WWWT, to be conducted alongside the SPARCS trial of its clinical effectiveness. The evaluation will address the question of whether WWWT provided by MCH nurses is a cost-effective intervention for the prevention of PND, postnatal anxiety and adjustment disorders in first-time mothers, compared with usual MCH care alone.

Methods

Design

Full details of the trial protocol are provided in a separate paper. The SPARCS trial is cluster-randomised, with the MCH centre as the unit of randomisation (owing to the nature of the intervention, it is not feasible to randomise by individuals). Six Local Government Areas (LGAs; from a total of 31) in the greater metropolitan area of Melbourne, Australia will be selected to participate. The LGAs will be ranked by the Socio-Economic Indexes for Areas (SEIFA) Index of Relative Advantage and Disadvantage, and two will be included from each of the low, middle and high tertiles of socioeconomic advantage.²⁶ Within these six LGAs, 48 MCH centres will be randomly allocated to intervention or control arms, with stratification by LGA. Although it is not possible to blind MCH staff to allocation, measures will be taken to minimise contamination across sites, and participants will be blinded to the intervention.

The economic evaluation will be conducted alongside the trial to examine the difference in costs and outcomes between the intervention and control arms. If the intervention is found to be both cost-saving and associated with equivalent or improved outcomes, then it is said to dominate the comparator. If (as is more likely) the intervention incurs additional costs, but provides additional health and/or utility gains, it is not immediately apparent whether the intervention would be preferred to the comparator. In these situations, an economic evaluation comparing costs and outcomes can be informative for decision-makers.

We will conduct cost-effectiveness and cost-utility analyses (CEA and CUA) to explore and quantify the costs per health (or utility) gain. The difference between these two is the measure of effectiveness employed: CEA uses outcomes in natural units (such as cases prevented) while CUA uses Quality Adjusted Life Years (QALYs). QALYs weight periods of time spent in a particular state by the quality of life for that state.

Whether or not any gain achieved is worth the additional costs is in the domain of decision-makers, but the results of CEA and CUA provide a basis for understanding the opportunity costs of investing scarce health resources in one area relative to another.²⁷ The economic evaluation will take a public sector perspective, considering costs and outcomes relevant to government departments of health, early childhood and human services (given the funding arrangements for MCH in the Victorian setting), as well as out-of-pocket (OOP) costs incurred by the participants.

Study population

Trial participants will be first-time mothers who: reside in the same LGA and receive care from one of the 48 selected MCH centres, have given birth within 2 weeks prior to enrolment, and have sufficient English language proficiency to complete structured telephone interviews. Women who agree to participate, after being provided with details of what participation will entail, will provide their written consent.

Intervention and comparator

The intervention consists of a single 6 h group session along with WWWT-informed postnatal care provided by trained MCH nurses throughout the study period. The session will provide information and training plus the opportunity for discussion and hands-on practice regarding managing infant behaviour, the intimate partner relationship and fatigue. MCH nurses at centres in the intervention arm of the trial will receive training on the principles of WWWT and its delivery. These trained nurses will deliver the session to participating women and their partners or other support people in group settings, within 10 weeks of the baby's birth, in addition to any usual FTP group sessions. Unlike most FTP group sessions, WWWT will be delivered on a Saturday rather than a weekday to facilitate the attendance of the women's partners. The number of WWWT sessions run per centre will be tailored to the number of first-time mothers in the area.

Women in the comparator arm of the trial will receive usual care, including standard FTP groups, from MCH nurses who have not been trained in WWWT. Centres which share staff with other participating centres will be excluded to avoid cross-contamination with the intervention.

Measures of outcome

The measures of outcome employed in the economic evaluation and the timing of their collection are presented in table 1. Data for outcome assessment will be collected by means of computer-assisted telephone interviews (CATIs). The baseline CATI will be conducted before delivery of the WWWT session, and the follow-up CATI when the infant is 6 months of age.

Table 1.

Overview of outcome measures

Measure	Means of collection	Timing of collection
Prevalence of depression, anxiety and adjustment disorder	CIDI conducted via telephone interview	Baseline: prior to WWWT session delivery Follow-up: 6-month post partum
Quality of life	EQ-5D-3L conducted via telephone interview	Baseline: prior to WWWT session delivery Follow-up: 6-month post partum

CIDI, Composite International Diagnostic Interview V.3.0; WWWT; What Were We Thinking.

The primary measure of effectiveness for CEA will be combined prevalence of depression, anxiety and adjustment disorders in the previous 30 days. Diagnostic and Statistical Manual for Mental Disorders, Fourth Edition (DSM-IV) diagnoses of depression and anxiety will be measured by the Composite International Diagnostic Interview V.3.0 (CIDI), while measurement of adjustment disorders will employ the Patient Health Questionnaire (see ref. 25 for details). The CIDI is a standardised structured interview which can be administered, as in this trial, by telephone interview, and which yields diagnosis of psychiatric disorders according to the DSM-IV criteria.²⁸

The EQ-5D-3L measure of health-related quality of life will be used to calculate QALYs for the CUA.²⁹ The EQ-5D-3L is one of the most widely used multiattribute utility measures and is regularly employed in health economic evaluations. Completion of the EQ-5D-3L involves responding to a series of questions across five dimensions of health-related quality of life: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Each dimension is scored over three levels: no problems, some problems or extreme problems. The EQ-5D-3L has been shown to identify differences in quality of life between people with differing severity of depression and anxiety, and to detect changes over time in those conditions.^30–33 Responses to the EQ-5D-3L will be scored using preference weights developed for the Australian population, which convert the five responses into a single summary index, where a score of 1 reflects perfect health and 0 is equivalent to dead.³⁴ QALYs will be estimated for each individual in the trial by estimating the area under the quality of life curve.²⁷

Measures of resource use and cost

An overview of resource use and cost measures to be employed in the economic evaluation is presented in table 2. The included costs are those that are likely to differ across the intervention and control groups, specifically the costs of: developing the intervention, training of MCH staff, WWWT session delivery on a Saturday within the trial, materials used during the sessions, and use of health and other services by participants during the follow-up period. The costs of developing WWWT will be obtained from the developers.

Table 2.

Overview of cost measures

Cost component	Means of collection	Timing of collection	Source of data
Cost of WWWT development	Interviews with programme developers	After completion of development	Developers of WWWT
Cost of training MCH staff	Interviews with project team, administrative records	After completion of training	Trial team
Cost of delivering WWWT	Trial records	After completion of all WWWT sessions	Trial records
Health service use	Telephone interview	6-month post partum for the period since baseline interview	Quantity—participant interview (including record card) Unit cost—see table 3
Other service use	Telephone interview	6-month post partum for the period since baseline	Quantity—participant interview (including record card) Unit cost—see table 3

MCH, Maternal and Child Health; WWWT, What Were We Thinking.

The costs of delivering a WWWT session will be affected by whether it would be additional to existing FTP group sessions or if it would replace some elements of the other services they offer. This will be established by asking MCH coordinators about the feasibility of integrating WWWT into existing FTP groups and how the inclusion of WWWT would affect the programme of sessions. The additional cost of making staff available to deliver WWWT on Saturdays will be calculated based on the higher rate of pay applicable to MCH nurses on Saturdays compared with their usual working hours.

Data on service use by mother and infant will be collected from participants in both intervention and control arms during the follow-up CATIs (see table 3). After the baseline interviews, participants will be provided with a record card, on which they will be asked to record all instances of health or other service use, including OOP costs. Parents will be advised to keep the record card with the baby's usual health records book, which is generally carried to all health-related appointments. During the follow-up CATI, they will be asked to refer to this card, and to report any other service use not listed on the card.

Table 3.

Information to be gathered on self-reported service use and sources of cost data

Service type	Source of unit costs
MCH	MCH data
MCH home visit	MCH data
Attendance at MCH-run parents’ group	MCH data
GP (other than immunisation)	MBS/OOP
Clinic nurse (other than immunisation)	MBS/OOP
Obstetrician/gynaecologist	MBS/OOP
Psychologist	MBS/OOP
Counsellor	MBS/OOP
Psychiatrist	MBS/OOP
Lactation consultant	OOP
Home visit from a Mothercraft nurse or another person specialising in sleep and settling	OOP
Dietician/nutritionist	OOP
Physiotherapist	MBS/OOP
Complementary healthcare practitioner (eg, chiropractor, naturopath, traditional Chinese medicine practitioner)	OOP
Paediatrician	MBS/ OOP
Residential Early Parenting Service	DHS/ OOP
Day-stay early parenting program	DHS/ OOP
Admission to a psychiatric mother–baby unit	WIES
Hospital emergency department attendance	WIES
Hospital outpatients clinic	MBS
Admission to hospital overnight	WIES
Social worker/family support services	DHS
Support groups including the Post and Antenatal Depression Association (PANDA), Australian Breastfeeding Association	Relevant organisation
Telephone helplines including MCH Line, nurse on call, Australian Breastfeeding Association	Relevant organisation
Online therapy resources	OOP
Other healthcare practitioners or services	OOP
Prescribed medicines.	PBS
Over-the-counter medicines	PBS
Unmet need for any of the listed services and why not able to access the service.	NA

DHS, Department of Human Services; GP, general practitioner; MBS, Medicare Benefits Scheme; MCH, Maternal and Child Health; NA, not available; OOP, self-reported out of pocket costs; PBS, Pharmaceutical Benefits Scheme; WIES, Weighted Inlier Equivalent Separation (weights for casemix funding calculation).

Unit costs for health service use will be estimated using data from the Medicare Benefits Schedule (MBS) and Pharmaceutical Benefits Scheme (PBS), plus any additional OOP costs reported by participants.^{35 36} Services not covered in MBS or PBS data will be valued at market prices and, where necessary, will reflect any OOP costs incurred by participants. Each participant's aggregate service use cost will be calculated from the total of instances of service use multiplied by the unit cost for that service.

All resources will be valued in 2013/2014 Australian dollars, and the 6-month trial follow-up means there is no requirement to apply discounting. The expense of developing WWWT was incurred earlier over a period of some years prior to this trial, and will be inflated based on the Consumer Price Index to reflect 2013/2014 prices.

Sample size

For sample size calculations, we assumed that the prevalence of the primary outcome would be 25% in the control group, based on results from the earlier before and after study of WWWT, which used the same measure (incorporating adjustment disorders along with PND and postnatal anxiety). We calculate that the minimum sample size required to detect as statistically significant a difference of 12.5 percentage points in the primary outcome measure (25% prevalence in the control group and 12.5% in the intervention group) is 184 participants per group, with a type 1 error of 5%, 80% power and allowing for 10% attrition between baseline and follow-up.

Notably, the sample size is that prescribed by the trial effect size, rather than the economic outcomes. Sample size estimation using economic end points is possible, but it is both time consuming, thus generally infeasible prior to obtaining funding, and will more likely produce a sample size that is larger than that required for the clinical effect; this increases the cost of a trial, and thus deceases the likelihood of successfully receiving funding.³⁷ It was therefore necessary to trade off these issues, and as with many economic evaluations, the sample was powered by the clinical outcome of interest.³⁸

Cost-effectiveness and cost-utility analyses

We will calculate the total costs for the intervention and control groups, as well as the average cost per participant, incorporating the cost of development, training and delivery of WWWT for the intervention group. Using regression analysis, we will control for differences in characteristics of participants (such as age, socioeconomic status, history of mental health problems, marital status), characteristics of MCH centres (such as SEIFA, rural/urban, size of service) and baseline EQ-5D-3L scores. This will also allow us to better manage skewed data, which is likely to be the case; we expect high proportions of participants to have zero service use costs and QALYs of 1 (full health). From these regression analyses, we will estimate the average cost per participant, the prevalence of postnatal mental health problems, and the average QALY achieved per participant for intervention and control groups.

Costs and outcomes will be combined into a single measure, the incremental cost-effectiveness ratio (ICER), which is the difference between intervention and control groups in costs divided by difference in outcomes. Results of the CEA will be expressed as cost per percentage point reduction in the combined 30-day prevalence of depression, anxiety and adjustment disorders. Results of the CUA will be expressed as cost per QALY gained. The ICER from the CEA will be comparable with other interventions using the same outcome measure (prevention of PND, postnatal anxiety and adjustment disorders), while that from the CUA will be comparable more broadly, as QALYs are not specific to the clinical condition. Information on cost per QALY allows decision-makers to consider efficient allocation of funding across divergent clinical disciplines, in this case allowing comparison between the cost-effectiveness of WWWT and that of prevention and treatment interventions for mental health, other perinatal interventions as well as unrelated health interventions.

Addressing uncertainty

As we will have individual-level data on costs and outcomes for the period of trial follow-up, we will evaluate uncertainty of the cost-effectiveness and cost-utility estimates using non-parametric bootstrapping.³⁹ Although standard statistical methods can be used to estimate SEs for the costs and outcomes, it is the combined uncertainty surrounding the ICER that is of most interest; in particular, we cannot assume independence of costs and effects. Bootstrapping produces an estimate of the joint distribution of costs and effectiveness that does not rely on assumptions about the nature of this distribution.

To represent decision uncertainty surrounding cost-effectiveness, we will employ cost-effectiveness acceptability curves.⁴⁰ These display the proportion of the estimates produced by bootstrapping that would be ‘acceptable’ (below a given willingness-to-pay (WTP) threshold), over a range of these WTP thresholds.

We will also perform scenario analysis, incorporating different extremes of uncertain values in order to estimate a base case, ‘best case’ and ‘worst case’ scenarios, and other policy-relevant scenarios, including different service provision arrangements in other Australian jurisdictions.²⁷

Modelling

Our base case will have a time horizon of up to 6 months, the period of the trial follow-up. If the intervention demonstrates clinical effectiveness in that period, we will employ decision analytic modelling to estimate cost-effectiveness beyond the trial period, incorporating the effect of the intervention on long-term correlates of postnatal mental health problems. It is acknowledged that the uncertainty in these estimates is likely to be significant due to the limitations of data available on connections between maternal mental health and later outcomes, but nonetheless it is likely to be of interest to policymakers to extrapolate the results beyond the trial.

Additional cost analysis to inform implementation

We will report on total costs of the intervention, as well as its components: development, training and delivery. The costs of developing the programme will be accounted over the projected lifetime of the programme, 5 years. We will consider how costs of the intervention might differ if WWWT were to be delivered in MCH services more broadly. This will include such factors as the need for refresher training, training of new staff members and the ongoing availability of staff on Saturdays.

To assist with the understanding of the policy implications of this intervention, costs (and cost savings) will be identified as accruing to the public sector (State and Commonwealth government departments of health, early childhood and human services), to private health insurers or to participants and their families.

Approvals and registration

The study was registered with the Monash University Human Research Ethics Committee (30 April 2013; CF12/1022-2012000474). Use of the EuroQol was registered with the EuroQol Group; 16 August 2012.

The trial was registered with the Australian New Zealand Clinical Trials Registry on 7 May 2012 (registration number ACTRN12613000506796).²³

Discussion

The value society places on prevention of postnatal mental health problems is unknown, nor is there an explicit WTP threshold for cost per QALY in the Australian health system. Some guidance can be derived given the past behaviours of decision-makers (ie, approvals and rejections for funding of interventions by government agencies). A review of decisions on reimbursement of drugs in Australia found that in the 1990s the decision-making agency generally did not recommend approval of drugs with a cost per life-year saved of higher than $A76 000 (1998/99 figures).⁴¹ The stated WTP from the general population may also be indicative; a more recent population-based study reported that an Australian stated WTP for an additional QALY was $A64 000.⁴² The figure of US$50 000 per QALY gained is often considered an approximation for a threshold, but this figure has been used for many years and any threshold that exists may vary with other factors (such as the value society places on the availability of treatments for particular conditions).^{42 43}

Adding to the uncertainty surrounding decision-makers’ WTP is that the duration of follow-up in the trial, as with all in this field, precludes collection of data on the long-term correlates of the mental health outcomes. The perceived value of preventing a case of PND, postnatal anxiety or adjustment disorder may depend on the decision-makers’ acceptance of the posited causal association between postnatal mental health problems and long-term problems for mothers, their children and partners.

Despite these uncertainties, this economic evaluation will provide decision-makers with valuable data to inform any future implementation of this innovative intervention for primary prevention of postnatal mental health problems.