Abstract
Introduction
Prevention of overweight and obesity in childhood is a priority because of associated acute and chronic conditions in childhood and later in life, which place significant burden on health systems. Evidence suggests prevention should engage a range of actions and actors and target multiple levels. The Whole of Systems Trial Of Prevention Strategies for childhood obesity (WHO STOPS) will evaluate the outcomes of a novel systems-based intervention that aims to engage whole communities in a locally led multifaceted response. This paper describes the planned economic evaluation of WHO STOPS and examines the methodological challenges for economic evaluation of a complex systems-based intervention.
Methods and analysis
Economic evaluation alongside a stepped-wedge cluster randomised controlled trial in regional and rural communities in Victoria, Australia. Cost-effectiveness and cost–utility analyses will provide estimates of the incremental cost (in $A) per body mass index unit saved and quality adjusted life year gained. A Markov cohort model will be employed to estimate healthcare cost savings and benefits over the life course of children. The dollar value of community resources harnessed for the community-led response will be estimated. Probabilistic uncertainty analyses will be undertaken to test sensitivity of results to plausible variations in all trial-based and modelled variables. WHO STOPS will also be assessed against other implementation considerations (such as sustainability and acceptability to communities and other stakeholders).
Ethics and dissemination
The trial is registered by the Australian New Zealand Clinical Trials Registry (ACTRN12616000980437). Full ethics clearances have been received for all methods described below: Deakin University’s Human Research Ethics Committee 2014-279, Deakin University’s Human Ethics Advisory Group-Health (HEAG-H) HEAG-H 194_2014, HEAG-H 17 2015, HEAG-H 155_2014, HEAG-H 197_2016, HEAG-H 118_2017, the Victorian Department of Education and Training 2015_002622 and the Catholic Archdiocese of Ballarat. Trial findings (including economic evaluation) will be published in peer-reviewed journals and presented at international conferences. Collected data and analyses will be made available in accordance with journal policies and study ethics approvals. Results will be presented to relevant government authorities with an interest in cost-effectiveness of these types of interventions.
Trial registration number
ACTRN12616000980437; Pre-results.
Keywords: childhood obesity, community child health, systems-based intervention, economic evaluation, cost effectiveness
Strengths and limitations of this study.
The protocol for this novel and complex intervention is guided by the Consolidated Health Economic Evaluation Reporting Standards guidelines for economic evaluation and draws lessons from the literature on the economic evaluation of complex public health interventions.
Pragmatic solutions are discussed for the core challenges this complex intervention poses for economic evaluation (eg, defining intervention boundaries; measurement and attribution of costs to Whole of Systems Trial Of Prevention Strategies (WHO STOPS)).
A range of data collection approaches will be employed to identify and measure the resources contributed across multiple sectors in participating communities.
WHO STOPS will be assessed against other implementation considerations (strength of evidence, equity, acceptability to stakeholders, sustainability, feasibility of implementation and potential side effects).
This protocol provides the core elements for the economic evaluation of this adaptive multifaceted whole of systems approach. Further decisions may be required that have not yet been considered.
Introduction
Childhood obesity causes a range of acute and chronic conditions reducing mental and physical health and well-being.1–3 Obese children are at higher risk of becoming obese adults, a major risk factor for diabetes, cardiovascular disease and some cancers.1 4 5 WHO estimates that 23% of children living in developed countries are overweight or obese.6 In Australia, the most recent National Health Survey (2014–2015) indicates that 27% of children aged 7–12 years are overweight or obese.7 It is estimated the direct costs of obesity to the health system were $8.6 billion in the 2011/2012 financial year.8 Preventing the onset of overweight and obesity during childhood could improve physical and psychological well-being, and education outcomes for children, and reduce the health burden and healthcare costs during childhood and later life.9–12
While many different types of interventions to prevent or treat childhood obesity have been trialled,13 14 these have generally shown limited feasibility and/or success. In the absence of successful interventions, invasive procedures such as bariatric surgery have increasingly been used to reduce body mass index (BMI) in teens with obesity as well as children as young as 9 years.15 However, these procedures are costly to the health system and are a high risk rather than population-level solution.9 Downstream interventions targeting at-risk children and their parents through screening and provision of primary-care-based interventions have generally been ineffective in reducing BMI.16 Of the more upstream interventions, few achieved reductions in the BMI of children,13 and those that did, have not demonstrated long-term benefits.13 17 The most promising strategies for preventing childhood obesity appear to be multifaceted—engaging a range of actions and actors to target multiple risk factors across multiple levels (eg, individuals, schools and whole communities).13 18
Building on the existing evidence, the Whole of Systems Trial Of Prevention Strategies for childhood obesity (WHO STOPS) will evaluate the impact of a novel intervention that aims to engage whole communities in a locally led multifaceted response.19 This systems-based intervention is underpinned by systems thinking20 21 and collective impact.19 22 WHO STOPS aims to reduce childhood BMI-z and obesity prevalence by supporting community leaders to change food and physical activity (PA) environments and related behaviours.19
Assessing cost-effectiveness of new interventions is critical to priority setting and funding decisions.23 24 There are few trials employing systems-based thinking for childhood obesity prevention and consequently little rigorous evaluation of cost-effectiveness of these approaches.25 This is due to the relatively recent emergence of this approach to childhood obesity prevention and also the methodological challenges for economic evaluation, where well-established economic evaluation frameworks provide only limited guidance on assessing complex and adaptive interventions.26 27 That said, modelling and trial-based evidence suggests multifaceted preventive strategies—particularly targeting schools—have real potential to be cost-effective,28–30 making the case for a rigorous economic evaluation of WHO STOPS. This paper presents a protocol (incorporating a discussion of key challenges) for the economic evaluation of the WHO STOPS childhood obesity intervention to address the research question:
From the funder/organiser and societal perspectives, what is the cost-effectiveness of the WHO STOPS childhood obesity intervention compared with current practice in regional and rural communities in Victoria, Australia?
About the trial
Study design
WHO STOPS is a stepped-wedge, cluster randomised controlled trial (C-RCT). The trial design is described in detail in Allender et al.19 In brief, 10 dispersed clusters or ‘natural communities’ (based on existing local government, health service and education boundaries) in the South Western Region of Victoria (Australia) were randomly assigned to receive the intervention at step 1 (2017)—referred to as intervention communities, or step 2 (2019) referred to as control communities (see table 1). Note that the timing of implementation has been updated from Allender et al.19 The region has 360 000 inhabitants and population clusters range in size from around 3200 to 20 800 people.
Table 1.
Community | Preintervention baseline (2015–2016) |
Step 1 (2017–2018) |
Step 2 (2019–2020) |
Group 1 (five communities) | Control | Control | Intervention |
Group 2 (five communities) | Control | Intervention | Intervention |
The intervention
The intervention has three main components.
Community engagement and facilitation (component 1)
First, community leaders will engage in at least two group model building (GMB) sessions. The research team will facilitate the construction of a causal loop diagram (CLD) that visually describes from the community leaders’ points of view, the shared understanding of the drivers of childhood obesity in their community and the interactions between such drivers.20 21 31 Figure 1 provides an example of a community leaders’ CLD.32 All community members are then invited to participate in a whole of community session (or sessions), where participants will (1) review the CLD, (2) identify points across the CLD where community-led actions to reduce obesity-related risk factors can be designed and implemented and (3) form community action groups to take ownership of these proposed actions.
Built on a behavioural and anthropometry school-based monitoring programme, childhood obesity and associated risk factor data collected in primary schools of the same community (see Crooks et al 33) will be presented during the GMB and whole of community sessions.
Backbone organisation (component 2)
The WHO STOPS process uses a collective impact approach,19 22 which seeks to identify and support a local ‘backbone’ organisation that will take significant responsibility for fostering, supporting and monitoring the community-led actions (described in component 3). It is anticipated that this role will include facilitating WHO STOPS action planning and review meetings, tracking and providing feedback of community-led actions to the community. During a pilot testing phase in a proximal, comparable town, the backbone organisation was a locally based public health organisation that allocated personnel time to these tasks.
Components 1 and 2 capture the reproducible element of the intervention and will be directly facilitated by the implementation specialists from the WHO STOPS research team. The timing of the establishment of the backbone organisation, however, may occur prior to, during or after component 1. The end of the whole of community session(s), where community-led actions are agreed on, will mark the full implementation of the intervention dose.
Community-led actions (component 3)
Component 3 consists of the planning and implementation by community members, of the suite of actions identified in their whole of community sessions, as well as any actions subsequently taken in the community that were motivated by component 1. The aim will be to undertake numerous actions across multiple points in the community, thereby addressing a range of obesity-related risk factors. Actions might be led by community organisations (including local government, health services, schools and sporting clubs), businesses, community groups or individuals. Any resources required to implement the proposed actions will be sourced by communities, primarily from within existing community resources. This strategy will result in component 3 being unique in each of the five intervention communities.
Control communities
Current practice will be observed in control (step 2) communities. This will include any local strategies targeting obesity-related risk factors implemented at the community level. Current practice itself is dynamic as communities introduce new and phase-out old local strategies. Implementation of the intervention in control communities will be completed 2 years after implementation in intervention (step 1) communities (2019) (see table 1).
Challenges for conducting economic evaluation of a whole of system intervention
The characteristics of this novel systems-based childhood obesity prevention strategy introduce a number of practical challenges for the application of standard economic evaluation methods.
Defining the intervention
From the funder/organiser perspective, the intervention could be viewed as limited to component 1, with the establishment of a backbone organisation (component 2) and the range of community actions generated (component 3)—important intermediate outcomes. However, from a broader societal perspective, those (component 3) community actions are a prerequisite for obesity-related behaviour change and as such can be considered an active component of the intervention. From this perspective, understanding the resources harnessed from within the community to affect any potential change in childhood overweight and obesity is important. Adding further complexity, each community will develop its own non-prescribed set of actions—as a result of differing priorities (determined via component 1), and at least partly as a result of the capacity provided by their backbone organisation and the approach it takes. As a result, each community’s component 3 will be unique, non-standardised and tailored to its own specific needs and resource capacity.
Assessing the costs
While costing components 1 and 2 will be relatively straightforward, the task of costing component 3 will be large and complex. Implemented actions (component 3) will occur across numerous sectors/settings (eg, health, education, local government, transport, local commerce, sport and recreation), making identification of costs difficult. Implementing parties may be well defined (eg, local department of health, local municipal government, school, sports club) or smaller and informal (eg, households or a group of parents). Beyond identification, ‘best practice’ microcosting of the potentially hundreds of discrete actions that may result across multiple communities is not feasible given this and most research projects’ resource constraints.
Attributing costs of a community action to the intervention in itself provides challenges particularly when:
An action has a set of aims and/or target group broader than children.
An action was motivated by observing other activities in theirs or other communities, rather than resulting from participation at the original community GMB sessions (ie, how many degrees of separation should be allowed for spin-off actions to be included?).
The intervention was only a tipping point, where years of community attention afforded a given action brought the community to a point of readiness to implement.
An action is somewhat distal to the primary aim of the intervention but was identified as an obesity-related risk factor by that community (eg, targeting parental drug and alcohol use to improve parenting skills).
While not unique to this intervention, this research is part of a much larger research effort and there is a genuine risk of overburdening community members with data collection at the risk of undermining community support.34 The costing method applied requires achievement of a balance between data comprehensiveness, feasibility and community sensitivity.
Assessing the benefits
The benefits of the whole of systems intervention may extend beyond the primary outcome and target population. For example, any resulting improvement in a child’s eating and PA behaviours may extend to other household members,35 or the strengthening of a community’s networks and leadership may improve its capacity to address other health and non-health related issues.36 Further, the intervention may result in multiple community actions, which are iterative in their development and interact in a non-linear fashion creating intended and unintended consequences, either of a positive or negative nature.37 38 Such a systems-based intervention seeks to create ‘system shifts’ and establish new societal norms around obesity-related behaviours; predicting the nature and extent of change beyond the trial period is difficult. These innovative intervention strategies require novel assessment techniques or adaptation of existing methods. Scenario analyses using system dynamics models37 may provide a framework to predict the likelihood of such system shifts (the potential use of which will be explored in this trial).
Lessons from the literature
The Consolidated Health Economic Evaluation Reporting Standards guidelines for reporting economic evaluations26 remain generally appropriate for this systems-based intervention, and some relevant lessons can be drawn from economic evaluations of complex public health interventions that share some of these challenges.35 38–42 However, the level of detail embedded in these resources is insufficient to provide practical guidance on all of the methodological decisions required.43 Frew35 argues (in the context of childhood obesity interventions) that such challenges require creativity, with decision rules made and justified on a case-by-case basis, while keeping the needs of decision-makers foremost in mind.
Methods and analysis
Economic evaluation overview
A cost-effectiveness evaluation will be conducted with incremental cost-effectiveness ratios calculated for the cost ($A) per BMI unit saved and quality adjusted life year (QALY) gained. Results will be analysed at the commencement of step 2 implementation (2019), when a comparison of the intervention versus current practice can be made. Results will also be analysed after 4 years (2 years post step 2 implementation (2021)) to identify the evolution and sustainability of community responses (including resource use) and any treatment effect. Analyses of intention-to-treat and as-per-protocol (ie, reaching component 3) treatment effects will be undertaken. Costs and benefits will be modelled over the rest of life, until the study cohort of children has either died or reached 100 years of age. The dollar value of community resources harnessed for $1 investment into component 1 will also be estimated. All costs will be inflated to current Australian dollars for the year of study completion using the all-items Consumer Price Index from the Australian Bureau of Statistics. All costs and benefits will be converted to present values using an annual discount rate of 5% in the base case, and annual rates of 3.5% and 0% in sensitivity analysis.44
Two perspectives will be taken. First, a funder/organiser perspective will be adopted, where the relevant intervention costs pertain to components 1 and 2. This perspective is broadly equivalent to what Frew35 describes as a ‘local authorities’ perspective, where a community’s leaders in local government, health services and primary schools (each having remit over health and well-being of children in their communities), will (collectively) be most likely responsible for sourcing funds to facilitate the first component of a WHO STOPS style strategy beyond the trial setting, as well as providing resources for the backbone role (component 2). As such, this perspective will have most utility for local decision-makers. Depending on the funding source of the identified backbone organisations, it is possible that this funder/organiser perspective will overlap significantly with a state government perspective. Second, a societal perspective will be taken, which will include the costs of resources contributed by the broader community through component 3, as well as future health system cost offsets. While it is expected that community actions will largely be resourced by reallocations of existing resources and funds, the associated opportunity costs require identification. This perspective will be of value for funding decision-making at higher levels (eg, state, national) where broader comparisons of relative cost-effectiveness within and across health silos are made, as well as for communities considering such a strategy, so they are fully aware that any treatment effect observed in this trial may have been mediated by the scale of community resources (eg, volunteer hours) contributed in those communities.
Given the burden associated with costing, the costing of components 2 and 3 will be restricted to two intervention communities and two control communities. These will be selected by the research team in consultation with external partners using the following considerations: (1) there is comparability in the population size of intervention and control communities, (2) the selected communities have some generalisability from the perspective of decision-makers, and (3) the backbone organisation agrees to participate in data provision and collection for the economic evaluation. To capture broader, less quantifiable issues that are of concern to policy-makers, WHO STOPS will also be assessed against other implementation considerations (strength of evidence, equity, acceptability to stakeholders, sustainability, feasibility of implementation and potential side effects) as per the approach developed and employed by Carter et al.45 These will be assessed by the research team in consultation with backbone organisations and other community partners.
Identification, measurement and valuation of outcomes
Health and health-related behavioural outcomes
Primary (BMI change) and secondary (PA and dietary behaviours) outcome data will be collected from children aged between about 8 and 12 years at participating primary schools using opt-out (passive) consent as described in Crooks et al 33 and the intervention effect assessed as described in Allender et al.19 PA and dietary behaviour questions will be self-completed, with students given structured prompts on how to answer the questions or to clarify terminology (as is the case for Health-Related Quality of Life (HRQoL) Questionnaire—discussed below).
Quality of life
HRQoL data will be collected (see Crooks et al 33) using the PedsQLTM 4.0 Child Report (8–12 years). The PedsQL is a non-preference-based 23 item instrument that assesses functioning across physical, emotional, social and school domains, where responses are transformed to a score on a 0–100 scale; higher scores reflect better HRQoL.46 Given PedsQL is a non-preference-based HRQoL instrument, an algorithm will be developed to enable conversion of PedsQL overall scores of study participants to the preference-based Children’s Health Utility 9 Dimension Index.47 48 Specifically, a dataset of around 1800 Australian children aged between 10 and 12 years will be employed. Following best practice methods, the optimal mapping algorithm will be chosen based on a series of econometric techniques using a number of goodness-of-fit measures.47 This will enable estimation of any resulting QALY gains.47
System changes
Proxy indicators of system change will be measured at baseline and followed up annually. The number of community actions will be tracked (and dollar value of resources utilised estimated) as proxies of community-level engagement. Social network analysis methods will be used to measure structural changes in community leadership networks.49 Assessments of changes in the obesity policy, infrastructure and leadership environment will be measured through a readiness to change analysis.50 These indicators will inform analysis of the other implementation considerations.
Future health and HRQoL benefits
An existing multistate life table Markov model will be used. Described in detail in Brown et al,51 the model estimates (for the 2010 Australian population) the extent to which changes in BMI and PA (independent of BMI) impact on the incidence and associated healthcare costs of osteoarthritis of the knee and hip, breast cancer, colon cancer, endometrial cancer, kidney cancer, ischaemic heart disease, hypertensive heart disease, stroke and type 2 diabetes; all causally related to obesity. The model was built in Excel (Microsoft Office 2003) and uses the add-in tool Ersatz (EpiGear, V.1.0) for uncertainty analysis.
Identification and measurement of costs
Table 2 summarises the cost inclusions and data collection strategy for components 1, 2 and 3. The planned approach for identification and measurement of costs will use a community’s backbone organisation to track community actions and collect data on related resource use for planning and implementation of given actions. A community action register, which is designed for tracking community actions and associated resource use, will be provided to each community’s backbone organisation. This will facilitate data collection for economic evaluation purposes as well as aid each community’s own monitoring and evaluation. This approach has been developed in collaboration with team members of such a backbone organisation and takes into consideration their capacity to collect data for research purposes, which may be beyond their own data needs for evaluation and community feedback.
Table 2.
Included costs | Data sources | Timing of cost data collection |
Component 1 (community engagement and facilitation) | ||
BMI data collection. Costs included: personnel time, travel costs, equipment (scales, tablets, etc) | Project administrative records. | Every 2 years. |
GMBs and whole of community sessions. Costs included: personnel time (facilitators and participants), travel, venue/catering, printing/stationery, STICK-E software licence* |
Project administrative records, meeting attendance sheets. | Ongoing over 3–9 month period of implementation. |
Personnel time and related administrative costs in organising, preparing and reporting results of GMB sessions, supporting backbone organisations and maintenance of community support website. | Project administrative records. | Annually. |
Component 2 (the role of the backbone organisation) | ||
Personnel time and related administrative costs in organising meetings of local WHO STOPS working groups and committees, advising and supporting other organisations and community members to plan and implement actions, track and feedback progress and ‘stories’ to community. | Project administrative records, key informant interviews. | Annually. |
Meeting facilitation and travel costs. | Project administrative records, key informant interviews. | Annually. |
Communication costs including printing and dissemination of newsletters and advertising of meetings and activities. | Project administrative records, key informant interviews. | Annually. |
Component 3 (community-led actions) |
Included costs | Relevant stakeholder | Data sources | Timing of cost data collection |
Community participation in backbone facilitated meetings. | Local authorities (eg, local government, health services, primary schools) and broader community. | Backbone organisation administrative records. | Annually |
Community actions. Costs included:
|
Local authorities (eg, local government, health services, primary schools) | Backbone’s action register, community case studies, document review and key informant interviews. |
Baseline (IC) then at years 2 and 4 (IC and CC). |
Primary schools (additional) | Survey of school actions. | At years 2 and 4 (IC and CC). | |
Broader community | Review of current community actions during whole of community sessions (component 1) Action register, community case studies. Key informant interviews. |
Baseline (ICs) and year 2 (CC). At years 2 (IC) and 4 (IC and CC). |
*Systems Thinking in Community Knowledge Exchange (STICK-E) is a web-based software developed to aid group model building (GMB) sessions.
CC, control communities; IC, intervention communities; WHO STOPS, Whole of Systems Trial Of Prevention Strategies.
Should an action result in an organisation moving funds between obesity-related programmes (ie, no net change in obesity-related programme funding), the costs associated with the newly funded activity will not be included in the cost-effectiveness analysis, though the new activity will be identified and documented. Given the intractability of such data — any changes in household expenditure on food and PA, and changes in revenue flowing to local retailers as a result of component 3 actions will be excluded. The costs of designing and developing the process for engaging and facilitating community actions (component 1), and Systems Thinking in Community Knowledge Exchange software will be excluded as they largely pre-exist this trial and will have wider use beyond this childhood obesity prevention intervention.
To assess the extent to which an identified action is attributable to WHO STOPS, at least two backbone team members (or relevant key informants) will respond to the following questions for each identified action: (1) Was the action commenced after Component 3 was implemented? (2) Is there a known link between WHO STOPS and the action? (3) Were any participants in planning or implementing the given action also involved in any WHO STOPs GMB sessions? (4) Was the implemented action intended to directly or indirectly address childhood obesity? (5) What proportion of the target population were children? (6) Were new resources allocated to obesity-related actions? (7) To what extent do you think the WHO STOPS intervention motivated implementation of the given action—(select one response: not at all, a little, somewhat, a lot, completely).
Given the intrinsic variability of component 3 of the intervention, it is possible that comprehensive costing of only two of the five communities might not provide an accurate representation of the potentially large between-community heterogeneity of actions and costs involved in this component. As a way to investigate this variability, all communities will be encouraged to monitor community actions and resource contributions using the community action register. Prior to finishing the economic evaluation, results from the costing of the two selected intervention communities will be discussed with backbone organisations from all intervention communities. Those backbone organisations from communities not comprehensively costed will be asked to consider those results alongside the actions registered in their own registers, and reflect on the extent to which findings appear consistent with their own communities. If perceived heterogeneity is raised, this will be acknowledged as a limitation in the published economic evaluation results.
The collection of resource use data relating to current practice in the control communities will differ in some respects. Backbone organisations will not be actively established by the research team until close to step 2 implementation. This will minimise researcher-led contamination of the control communities, but will result in a reliance on retrospective identification of community actions in those communities (as set out in table 1) and raises the risk of failing to identify ‘current practice’ actions that occurred. This potential for recall bias reinforces the importance of taking steps to assess attribution of actions to WHO STOPS in the intervention communities.
Valuation of resource use
The time contributions of individuals (professional and volunteer time) will be costed using opportunity cost principles. Resource use of non-health sector goods and services will be valued at market prices and be informed by best available evidence from Australian-based studies. Where relevant, health resources will be costed as per the Manual of Resource Items for use in submissions to the Commonwealth of Australia’s Pharmaceutical Benefits Advisory Committee.44
Where the data collection strategy results in insufficient detail for an identified community action, evidence of costs may be drawn from comparable community-based obesity prevention activities. The anticipated large number of community-led actions likely to be identified and logged in a backbone’s action register will deem it infeasible to collect detailed data on resource use for each registered action. As such, each action will be classified into small, medium and large (in terms of resource intensity) by backbone organisation personnel. A sample from each classification will then be costed in detail, with results extrapolated.
Uncertainty and scenario analyses
It is important to note some of the challenges being faced that may have implications for the estimated treatment effect. The intervention aims to make system-level changes, some of which will take longer than the trial period to occur. Further, due to the variability we have observed in the time it takes communities to reach the component 3 stage (community actions), it is likely there will be variable effects measured across communities after 2 years. The 4-year analyses to be undertaken will be comparing a maximum of 4 years of exposure (step 1 communities) versus a maximum 2 years of exposure (step 2 communities) rather than comparing against a current practice control. However, BMI change data are being collected from a small number of schools in 13 communities external to the WHO STOPS study.52 The methods of data collection and general characteristics of the surveyed schools differ from the current study. So, while these external communities do not constitute the ideal control group, several of the external communities are comparable (in terms of population and geographic proximity to major urban centres). These external communities will provide an indication of BMI trends in non-WHO STOPS settings. This will facilitate the estimation of plausible variations in treatment effects at 2 and 4 years for use in sensitivity analyses.
It is possible that other family members of targeted children and the broader community may also benefit from the WHO STOPS intervention in terms of BMI change.35 Furthermore, the intervention may result in (1) productivity gains (given potentially reduced child absenteeism from school for obesity-related reasons leading to lower parent absenteeism from work), and (2) improved future income levels arising from improved schooling outcomes.53 54 Best available evidence on such broader potential costs and benefits will be sought and included in scenario analyses.
Given uncertainty around the maintenance of community responses and treatment effects beyond the trial period, modelling of future benefits and health cost savings will test a range of plausible assumptions of decayed and maintained treatment effect. These may range from full decay over 5 or 10 years through to 100% maintenance of effect. In the event that within trial analyses of the proxy indicators of system change suggest an accentuation of treatment effect is plausible, modest accentuation of treatment effect will also be modelled.
Extensive analyses will be undertaken to test the sensitivity of results to plausible variations in all trial-based and modelled variables, including assumptions around the maintenance of any observed changes in BMI (as discussed above), PA and fruit and vegetable consumption as well the costs of alternative approaches to GMB facilitation (where local community members are trained to facilitate). Further, in the event that BMI changes are observed in children, the potential impact on the intervention’s cost-effectiveness of broader ‘family effects’ will be investigated in scenario analyses. The potential for system dynamics models to estimate the impact of ‘system-wide’ changes on future obesity-related behaviours and prevalence will be explored and considered for use in scenario analyses.
Conclusion
Obesity is associated with poorer health and quality of life—its prevalence is high and rising in many countries.55 Childhood obesity can have detrimental health and well-being implications during childhood and is a major predictor of obesity in adulthood and its serious and expensive associated conditions.5 Community-based strategies have been effective in achieving some reductions in population BMI, but these have not been sustained. The WHO STOPS intervention builds on this evidence base. It seeks to harness existing community resources and expand the extent of local engagement in obesity-risk reduction, across whole communities.
Evaluating the cost-effectiveness of this novel systems-based intervention will help policy-makers by assessing the resource use implications of achieving any observed intervention effect. This protocol considers the main challenges posed by the economic evaluation of such a complex intervention designed to produce systems change. This protocol registers our intent to conduct this evaluation alongside the WHO STOPS Childhood Obesity Trial, and describes for transparency, the predetermined approaches for addressing the methodological challenges described and the analyses planned a priori. This does not preclude additional hurdles arising during the course of the project forcing additions to or deviations from this plan, but these will be openly documented during the reporting of results.
Supplementary Material
Footnotes
Contributors: RS led the conceptual design and writing of this work. MM made substantial contributions to the conceptual design of the work as well as substantial contributions to the writing of this work. Coauthors PN, PF, KB, AB, JM, NC, CS, CB, LM, LO and SA all made substantial contributions to the conceptual design of the methods described in this Economic Evaluation Protocol, and all made important contributions in revising the manuscript critically for important intellectual content. All authors have approved of the final version of the submitted manuscript.
Funding: This study is supported by an NHMRC Partnership Project titled “Whole of Systems Trial of Prevention Strategies for childhood obesity: WHO STOPS childhood obesity” (APP1114118). The work described herein has also received funding support from the Western Alliance. Study researchers (listed in title page) are researchers within the NHMRC Centre for Research Excellence in Obesity Policy and Food Systems (APP1041020). The authors would also like to acknowledge the support from the Victorian Department of Health and Human Services, Victorian Department of Education and Training, South West Primary Care Partnership (PCP), Southern Grampians Glenelg PCP, Portland District Health, Western District Health Service, Colac Area Health, Portland Hamilton Principal Network of Schools (NoS), Colac Corangamite NoS, Warnambool and District NoS, Glenelg Shire Council, Southern Grampians Shire Council.
Patient consent: Not required.
Ethics approval: Full ethics clearances have been received for all methods described in the manuscript: Deakin University’s Human Research Ethics Committee (DU-HREC) 2014-279, Deakin University’s Human Ethics Advisory Group-Health (HEAG-H) HEAG-H 194_2014, HEAG-H 17 2015, HEAG-H 155_2014, HEAG-H 197_2016, HEAG-H 118_2017), the Victorian Department of Education and Training 2015_002622 and the Catholic Archdiocese of Ballarat.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1. Williams EP, Mesidor M, Winters K, et al. . Overweight and Obesity: prevalence, consequences, and causes of a growing public health problem. Curr Obes Rep 2015;4:363–70. 10.1007/s13679-015-0169-4 [DOI] [PubMed] [Google Scholar]
- 2. Han JC, Lawlor DA, Kimm SY. Childhood obesity. Lancet 2010;375:1737–48. 10.1016/S0140-6736(10)60171-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Pulgarón ER. Childhood obesity: a review of increased risk for physical and psychological comorbidities. Clin Ther 2013;35:A18–32. 10.1016/j.clinthera.2012.12.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Singh AS, Mulder C, Twisk JW, et al. . Tracking of childhood overweight into adulthood: a systematic review of the literature. Obes Rev 2008;9:474–88. 10.1111/j.1467-789X.2008.00475.x [DOI] [PubMed] [Google Scholar]
- 5. Llewellyn A, Simmonds M, Owen CG, et al. . Childhood obesity as a predictor of morbidity in adulthood: a systematic review and meta-analysis. Obes Rev 2016;17:56–67. 10.1111/obr.12316 [DOI] [PubMed] [Google Scholar]
- 6. Ng M, Fleming T, Robinson M, et al. . Global, regional and national prevalence of overweight and obesity in children and adults 1980-2013: A systematic analysis. Lancet 2014;384:766–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Australian Bureau of Statistics. National Health Survey: first results, 2014-15. Canberra, Australia: Australian Bureau of Statistics, Commonwealth of Australia, 2016. [Google Scholar]
- 8. Duckett S HS, S H, W T. A sugary drinks tax: recovering the community costs of obesity. Melbourne, Australia: Grattan Institute, 2016. [Google Scholar]
- 9. Wake M, Baur LA, Gerner B, et al. . Outcomes and costs of primary care surveillance and intervention for overweight or obese children: the LEAP 2 randomised controlled trial. BMJ 2009;339:b3308 10.1136/bmj.b3308 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pan L, Sherry B, Park S, et al. . The association of obesity and school absenteeism attributed to illness or injury among adolescents in the United States, 2009. J Adolesc Health 2013;52:64–9. 10.1016/j.jadohealth.2012.04.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Hayes A, Chevalier A, D’Souza M, et al. . Early childhood obesity: association with healthcare expenditure in Australia. Obesity 2016;24:1752–8. 10.1002/oby.21544 [DOI] [PubMed] [Google Scholar]
- 12. Colagiuri S, Lee CM, Colagiuri R, et al. . The cost of overweight and obesity in Australia. Med J Aust 2010;192:260–4. [DOI] [PubMed] [Google Scholar]
- 13. Waters E, de Silva-Sanigorski A, Hall BJ, et al. . Interventions for preventing obesity in children. Cochrane Database Syst Rev 2011:CD001871 10.1002/14651858.CD001871.pub3 [DOI] [PubMed] [Google Scholar]
- 14. Luckner H, Moss JR, Gericke CA. Effectiveness of interventions to promote healthy weight in general populations of children and adults: a meta-analysis. Eur J Public Health 2012;22:491–7. 10.1093/eurpub/ckr141 [DOI] [PubMed] [Google Scholar]
- 15. Treadwell JR, Sun F, Schoelles K, Review S. Systematic review and meta-analysis of bariatric surgery for pediatric obesity. Ann Surg 2008;248:763–76. 10.1097/SLA.0b013e31818702f4 [DOI] [PubMed] [Google Scholar]
- 16. Swinburn BA, Sacks G, Hall KD, et al. . The global obesity pandemic: shaped by global drivers and local environments. Lancet 2011;378:804–14. 10.1016/S0140-6736(11)60813-1 [DOI] [PubMed] [Google Scholar]
- 17. Shaya FT, Flores D, Gbarayor CM, et al. . School-based obesity interventions: a literature review. J Sch Health 2008;78:189–96. 10.1111/j.1746-1561.2008.00285.x [DOI] [PubMed] [Google Scholar]
- 18. Wilkie HJ, Standage M, Gillison FB, et al. . Multiple lifestyle behaviours and overweight and obesity among children aged 9–11 years: results from the UK site of the International Study of Childhood Obesity, Lifestyle and the Environment. BMJ Open 2016;6(2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Allender S, Millar L, Hovmand P, et al. . Whole of systems trial of prevention strategies for childhood obesity: WHO STOPS Childhood Obesity. Int J Environ Res Public Health 2016. 13:1143 10.3390/ijerph13111143 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston: Irwin McGraw-Hill, 2000. [Google Scholar]
- 21. Hovmand PS. Group model building and community-based system dynamics process. Community based system dynamics. New York, NY: Springer, 2014:17–30. [Google Scholar]
- 22. Kania J, Kramer M. Collective impact. Stanford Social Innovation Review 2011;9:36–41. [Google Scholar]
- 23. Oortwijn W, Mathijssen J, Banta D. The role of health technology assessment on pharmaceutical reimbursement in selected middle-income countries. Health Policy 2010;95:174–84. 10.1016/j.healthpol.2009.12.008 [DOI] [PubMed] [Google Scholar]
- 24. Stafinski T, Menon D, Philippon DJ, et al. . Health technology funding decision-making processes around the world. Pharmacoeconomics 2011;29:475–95. 10.2165/11586420-000000000-00000 [DOI] [PubMed] [Google Scholar]
- 25. Malakellis M, Hoare E, Sanigorski A, et al. . School-based systems change for obesity prevention in adolescents: outcomes of the Australian Capital Territory ’It’s Your Move!'. Aust N Z J Public Health 2017;41:490–6. 10.1111/1753-6405.12696 [DOI] [PubMed] [Google Scholar]
- 26. Husereau D, Drummond M, Petrou S, et al. . Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ 2013;346:f1049 10.1136/bmj.f1049 [DOI] [PubMed] [Google Scholar]
- 27. Drummond MF, Stoddart GL, Torrance GW. Methods for the Economic Evaluation of Health Care Programmes. Oxford: Oxford University Press, 2005. [Google Scholar]
- 28. McAuley KA, Taylor RW, Farmer VL, et al. . Economic evaluation of a community-based obesity prevention program in children: the APPLE project. Obesity 2010;18:131–6. 10.1038/oby.2009.148 [DOI] [PubMed] [Google Scholar]
- 29. Haby MM, Vos T, Carter R, et al. . A new approach to assessing the health benefit from obesity interventions in children and adolescents: the assessing cost-effectiveness in obesity project. Int J Obes 2006;30:1463–75. 10.1038/sj.ijo.0803469 [DOI] [PubMed] [Google Scholar]
- 30. Moodie ML, Herbert JK, de Silva-Sanigorski AM, et al. . The cost-effectiveness of a successful community-based obesity prevention program: the be active eat well program. Obesity 2013;21:2072–80. 10.1002/oby.20472 [DOI] [PubMed] [Google Scholar]
- 31. Vennix JA. Group model building: facilitating team learning using system dynamics. Chichester: Wiley, 1996. [Google Scholar]
- 32. Allender S, Owen B, Kuhlberg J, et al. . A community based systems diagram of obesity causes. PLoS One 2015;10:e0129683 10.1371/journal.pone.0129683 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Crooks N, Strugnell C, Bell C, et al. . Establishing a sustainable childhood obesity monitoring system in regional Victoria. Health Promot J Austr 2017;28:96-102 10.1071/HE16020 [DOI] [PubMed] [Google Scholar]
- 34. Gubbels JS, Mathisen FK, Samdal O, et al. . The assessment of ongoing community-based interventions to prevent obesity: lessons learned. BMC Public Health 2015;15:216 10.1186/s12889-015-1563-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Frew E. Economic evaluation of childhood obesity interventions: reflections and suggestions. Pharmacoeconomics 2016;34:733–40. 10.1007/s40273-016-0398-8 [DOI] [PubMed] [Google Scholar]
- 36. Johnson K, Collins D, Shamblen S, et al. . Long-Term sustainability of evidence-based prevention interventions and community coalitions survival: a five and one-half year follow-up study. Prev Sci 2017;18:610–21. 10.1007/s11121-017-0784-2 [DOI] [PubMed] [Google Scholar]
- 37. Sterman JD. Learning from evidence in a complex world. Am J Public Health 2006;96:505–14. 10.2105/AJPH.2005.066043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. BMJ 2008;336:1281–3. 10.1136/bmj.39569.510521.AD [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Hawe P, Shiell A, Riley T, et al. . Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. J Epidemiol Community Health 2004;58:788–93. 10.1136/jech.2003.014415 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Weatherly H, Drummond M, Claxton K, et al. . Methods for assessing the cost-effectiveness of public health interventions: key challenges and recommendations. Health Policy 2009;93:85–92. 10.1016/j.healthpol.2009.07.012 [DOI] [PubMed] [Google Scholar]
- 41. Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health 2013;13:1001 10.1186/1471-2458-13-1001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Husereau D, Jacobs P, Manns B, et al. ; Economic evaluation of complex health system interventions: a discussion paper. Edmonton Alberta, Canada: Institute of Health Economics, 2014. [Google Scholar]
- 43. Lung TW, Muhunthan J, Laba TL, et al. . Making guidelines for economic evaluations relevant to public health in Australia. Aust N Z J Public Health 2016;41:115–7. 10.1111/1753-6405.12601 [DOI] [PubMed] [Google Scholar]
- 44. Australia C. Guidelines for preparing submissions to the Pharmaceutical Benefits Advisory Committee (Version 5.0). Canberra, 2016. [Google Scholar]
- 45. Carter R, Moodie M, Markwick A, et al. . Assessing cost-effectiveness in obesity (ACE-obesity): an overview of the ACE approach, economic methods and cost results. BMC Public Health 2009;9:419 10.1186/1471-2458-9-419 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Varni JW, Seid M, Kurtin PS. PedsQL 4.0: reliability and validity of the pediatric quality of life inventory version 4.0 generic core scales in healthy and patient populations. Med Care 2001;39:800–12. 10.1097/00005650-200108000-00006 [DOI] [PubMed] [Google Scholar]
- 47. Wailoo AJ, Hernandez-Alava M, Manca A, et al. . Mapping to estimate health-state utility from non-preference-based outcome measures: an ISPOR Good Practices for outcomes research task force report. Value Health 2017;20:18–27. 10.1016/j.jval.2016.11.006 [DOI] [PubMed] [Google Scholar]
- 48. Stevens K. Valuation of the Child Health Utility 9D Index. Pharmacoeconomics 2012;30:729–47. 10.2165/11599120-000000000-00000 [DOI] [PubMed] [Google Scholar]
- 49. Valente TW, Palinkas LA, Czaja S, et al. . Social network analysis for program implementation. PLoS One 2015;10:e0131712 10.1371/journal.pone.0131712 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Plested B, Edwards R, Jumper-Thurman P. Community readiness: a handbook for successful change. Colorado: CO: Tri-Ethnic Center for Prevention Research, 2006. [Google Scholar]
- 51. Brown V, Moodie M, Cobiac L, et al. . Obesity-related health impacts of fuel excise taxation- an evidence review and cost-effectiveness study. BMC Public Health 2017;17:359 10.1186/s12889-017-4271-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Strugnell C, Millar L, Churchill A, et al. . Healthy together Victoria and childhood obesity-a methodology for measuring changes in childhood obesity in response to a community-based, whole of system cluster randomized control trial. Arch Public Health 2016;74:16 10.1186/s13690-016-0127-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Black N, Johnston DW, Peeters A. Childhood obesity and cognitive achievement. Health Econ 2015;24:1082–100. 10.1002/hec.3211 [DOI] [PubMed] [Google Scholar]
- 54. Cawley J. The economics of childhood obesity. Health Aff 2010;29:364–71. 10.1377/hlthaff.2009.0721 [DOI] [PubMed] [Google Scholar]
- 55. Hruby A, Hu FB, Fb H. The Epidemiology of obesity: a big picture. Pharmacoeconomics 2015;33:673–89. 10.1007/s40273-014-0243-x [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.