Long-term economic outcomes for interventions in early childhood: protocol for a systematic review

Abstract

Introduction

Investment in early childhood produces positive returns: for the child, the family and the community. Benefits have been shown to be significant within certain parameters, but a systematic review of the economic evidence across multiple sectors including health, education and social welfare will have the capacity to inform policy relative to the full range of social determinants. This review will take a broad approach, encompassing a range of costs and benefits to enable the identification of the most beneficial investments in early childhood and to highlight gaps in current research.

Methods and analysis

Economic evaluations incorporating both costs and long-term outcomes of early childhood interventions and programmes will be included. Outcomes may be valued in monetary units or quantified non-monetary units (eg, quality-adjusted life years (QALY), disability-adjusted life years (DALY)). Results will be expressed as a ratio according to the outcome; with monetary outcomes expressed as cost-benefit ratios or return on investment, and non-monetary outcomes expressed as cost per QALY or DALY. The target population is children aged 0–5 years.

Extensive database searches across sectors will be undertaken. The review will involve five phases: defining the research question, identifying relevant studies, selecting studies, extracting and collating data, and summarising and reporting results. The search commenced in 2019 and the expected end date is December 2020.

Ethics and dissemination

The findings of this review will inform policymakers and practitioners in public health, education, social welfare and primary care settings. The publication plan includes a series of academic publications, and policy papers prepared and disseminated through Telethon Kids Institute networks. Exemption from ethics approval was granted by the University of Western Australia Human Ethics Office (RA/4/20/5677).

PROSPERO registration number

CRD42020145901.

Strengths and limitations of this study.

• Holistic approach to review evidence of early childhood investment.

• Identification of gaps in the literature.

• Internationally based.

• Different methodological approaches according to discipline and/or sector.

• Contrasting priorities for investment according to income level of country.

Introduction

Evidence indicates that investment in early childhood generates positive returns, for the child, the family and the wider community.^1–5 Benefits to children in the short term include the development of resilience, improved cognitive skills, reduced school absenteeism and reduced risk of disease. Longer term outcomes include better employment pathways,^{1 2} improved health,^{3 6–8} reduced dependency on welfare (including social services, incarceration and juvenile justice)^9–11 and reduction in inequality.^{3 5} This is particularly so for children who struggle to pursue their full potential, due to poor health,^6–8 lack of opportunities to learn^1–5 and/or deprivation of care.^{12 13} Ultimately these benefits have implications for improving national productivity and gross domestic product.^{1 2 5} Conversely the cost of failing to adequately support children has implications for the child and the community, and, through the social and economic implications of that failure, to the national economy.¹⁴ While the benefits of investment in early childhood have been shown to be significant within certain parameters, to date there is limited evidence addressing investment across multiple sectors (including health, education and social welfare).¹⁵

Much of the research on successful early childhood interventions and programmes supports the need to intervene prior to formal school entry. For instance, developmental neuroscience emphasises ages 0–5 years as the optimal time to intervene³ based on the rate of change that occurs physically and mentally during this period. Interventions initiated between the ages of 0 and 3 years have been shown to yield the highest economic returns, particularly for children experiencing adversity.⁶

While the benefit of early childhood investment is well established per se, priorities for the best allocation of resources are less clear. Understanding the comparative ‘value’ of a programme⁵ is central to enabling policy choices. This ‘value’ can be expressed in terms of cost-effectiveness (the cost required to achieve a given outcome), as cost-benefit (the ratio of cost inputs to dollar benefits) or as return on investment (ROI), which compares net profit with cost inputs. Typically, health programmes are valued according to cost per quality-adjusted life year (QALY) or cost per disability-adjusted life year (DALY) whereas in education, value is more typically presented in terms of cost-benefit or ROI.

Consistent with the WHO definition of health as ‘a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity’,¹⁶ individual outcomes are specified broadly, encompassing a child’s whole life potential. For example, optimal health for the young child includes the development of resilience and the opportunities to achieve throughout their life.

A systematic review of the peer-reviewed literature across all sectors will synthesise the economic evidence on interventions directed towards infants and children (those aged 0–5 years), for legislators, practitioners and policymakers. Given the complementarity of interventions a systematic review of evidence that is not siloed by sector has the capacity to broadly address the range of social determinants.

The aim of this study is to systematically review the economic evidence on population interventions targeted at children aged 0–5 years across multiple sectors including health, education and social welfare.

Methods and analysis

Eligibility criteria

The review will include economic evaluation studies in peer-reviewed journals containing both costs and non-monetary outcomes (including cost-minimisation, cost-utility, cost-effectiveness) or costs and outcomes valued in monetary terms (cost-benefit and other ROI). The sectoral differences in approach to economic evaluations, for example, the use of cost-benefit analysis in the education sector and cost-effectiveness in health, will be transparently addressed. Given the potential variation in methodological approach, we will present economic ratios specific to each sector or within subgroups such as vulnerable and low socioeconomic groups.

Participants will primarily be infants and children aged 0–5 years, but may also include alternative groups such as parents, where outcomes relate to children. Studies including broader population age groups will be reviewed where data relating to 0–5 year-olds can be separately extracted.

Interventions may have broad scope including vaccination, nutrition and health management; early childhood learning, access to early childhood programmes and the organisation of early learning; and the impact of poverty, maternal employment and parental incarceration. Acute care or diagnostic choices in tertiary care will not be included.

All interventions will be assessed against a control, although the comparator may be ‘no intervention’. Additional components of quality will be assessed within a set template and studies may be excluded if they do not meet the agreed criteria. Only studies that involve both an investment in early childhood and a measured benefit will be reviewed. Studies will be excluded if they do not report both costs (inputs) and benefits (outputs). Full inclusion and exclusion criteria are listed in table 1.

Table 1.

Inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria
Economic evaluations with investment in early childhood and measured long-term benefit	No outcomes quantified
Peer-reviewed journal articles	Supplements, letters, conference abstracts and proceedings
Participants: children aged 0–5 years and others (eg, parents where outcomes relate to children aged 0–5 years)	No separate extraction for 0–5 years possible
	Published prior to 2000
	Interventions of acute care, treatment, diagnostic choices

Information sources

Following advice from generalist and specialist librarians (health and medical, business and economics), the following electronic databases have been identified as information sources: MEDLINE (Ovid), EMBASE (Ovid), PsycInfo (Ovid), National Health Service Economic Evaluation Database (NHS EED), Econ Lit, Paediatric Economic Database Evaluation (PEDE) and Education Resources Information Center (ERIC).

Reference lists of included studies will be hand searched to identify further papers. Authors of particularly germane studies may be contacted for additional pertinent material where considered necessary. References cited in relevant systematic reviews will be searched for additional eligible studies.

Search strategy

To account for indexing variation between electronic databases, tailored search specifications have been developed for each database. For example, databases dedicated to predefined criteria, such as ‘paediatric’, do not require inclusion of that criterion in the search. The search strategy to be applied to the Ovid platform (MEDLINE, EMBASE, PsycInfo) will be used in a modified form for all other databases. For example, in MEDLINE the search will combine exp Child, Preschool/, INFANT/, early childhood.mp and (preschool child* or infant* or prekindergarten or kinder* or child*).mp. This will be specified as: all infant (birth to 23 months) or newborn infant (birth to 1 month) or infant (1–23 months) or preschool child (2–5 years). Cost-Benefit Analysis/, exp Quality-Adjusted Life Years/, (return on invest* or ‘cost and cost analysis’ or economic evaluation or QALY or DALY or ‘quality adjusted life year*’ or ROI or cost benefit*) will be combined. The complete search strategy is published as online supplementary material.

The search in all databases will be limited to peer-reviewed journals published since 2000; classified as journal article (ie, no conference proceedings, supplements, or letters); and relating to human research. There will be no limitation on language of publication.

Study records

Data management

Search results will be imported into Covidence¹⁷ and automatically deduplicated. Additional duplicates will be manually identified and removed once full text comparison is undertaken.

Selection process

The CoLab Economics Systematic Review Consultancy Group comprises internationally recognised professional staff, consultants and senior economists. This group provided advice on the design of the study and will be involved in other facets of the data collection and analysis.

Two reviewers will independently examine the titles and abstracts retrieved through the search process to identify studies that do not meet the stated eligibility criteria. At this level any economic evaluation measuring cost-effectiveness, cost-utility or ROI from interventions affecting the immediate and/or lifetime health, education, labour or social welfare outcomes of infants and children, other than clinical treatment outcomes, will be included. Lack of consensus between reviewers on study inclusion will be resolved through discussion. Where necessary a third reviewer will be consulted. Full texts of the remaining potentially relevant records will be retrieved in order to proceed to independent evaluation by two reviewers.

Data collection

An extraction template, using an adapted checklist, will be piloted and refined by staff, and circulated to the Consultancy group for further piloting, discussion and approval. Standard checklists will be considered and compared throughout the process. Data items included in the extraction may be modified following piloting. At least two independent reviewers will extract data from the full text of selected studies. Extracted data will be compared and conflicts between the reviewers discussed and resolved.

Data items

The characteristics of included studies will be described based on the following:

1. General categorising information (sector information, category, condition, intervention, study question).

2. Country and income level (using World Bank classifications).¹⁸

3. Population characteristics and reason for selection (age group, stratification, target criteria, number of participants).

4. Perspective of the economic evaluation (societal, government, and so on).

5. Time horizon.

6. Economic study type and outcome ratio.

7. Comparator(s) or base case and justification for comparator.

8. Costing approach (currency, index year for costs, standardisation methods, cost components, cost sources, discount rate).

9. Outcomes (measure of benefit in evaluation, discount rate applied in outcomes).

10. Sensitivity analyses (parameters, methods and results).

11. Credibility and conflicts of interest (eg, funder).

12. Modelling (type, structure, assumptions, sources, cycle length and validation).

13. Feasibility/necessity to contact study authors to obtain missing information.

14. Quality (subjective).

    1. Research methods (Population, Intervention, Comparator, Outcomes).

    2. Risk of bias—selection, performance, detection, attrition, reporting.

    3. Imprecision—CIs, sample size.

    4. Inconsistency.

    5. Publication bias.

At the full text retrieval stage reviewers will examine the economic methods used, the usability of studies containing partially age-relevant data and the significance of studies that require modelling inputs. The breadth of the interventions suggests that an extensive discussion will be required to consider the value of literature where methods have not strictly complied with standard methodologies.

Risk of bias in individual studies

The quality of the selected studies will be transparently reported based on the economic evaluation methods used, validity of assumptions and possible inherent biases. Two independent reviewers will assess the certainty of evidence considering risk of bias, imprecision, inconsistency, indirectness and publication bias. Five sources of bias in reported outcomes will be assessed: selection, performance, detection, attrition and reporting. Study authors who are contactable may be requested to provide pertinent and available missing data.

For each included study, both reviewers will assess the risk of bias and assign ratings of low, high or unclear risk of bias. Disagreement will be resolved via discussion between the two reviewers until they reach consensus or through referral to an independent third reviewer. Final decisions will be recorded in a ‘Risk of bias’ table with a rationale for each decision.

Data

Synthesis

Results will be reported consistent with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 checklist; a PRISMA diagram and checklist will be included at the completion of the review.

In consideration of differing economic evaluation methodologies across sectors, a narrative synthesis of data will be undertaken. Narrative format ‘Summary of Findings’ tables will present the key results of the review, including ranking of the certainty of the evidence on which these results are based. Tables will be developed according to the outcome variable, or the means of reporting ROIs. Information on population, intervention, comparison, setting and certainty of evidence will also be reported.

Primary comparisons will be made within sector (eg, health sector using QALYs). Secondarily, cross-sectoral comparisons will be explored using willingness-to-pay thresholds to convert health outcomes to monetary units.

Patient and public involvement

This research will be done without patient involvement. Patients were not invited to comment on the study design and were not consulted to develop patient-relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Discussion

This systematic review will be the first to consider cross-sector economic evaluation of investment in early childhood. While this presents some methodological challenges, the results will provide invaluable information to assist in ranking whole-of-government priorities and forming policy recommendations. The project reflects a global perspective, with national policy implications.

The lack of consistent outcome measures between sectors represents a potential limitation of the review in comparing cost-effectiveness. For example, the conversion of outcomes to monetary units is contentious because of the assumptions required. Furthermore, benefits of early childhood intervention may be difficult to quantify because they are intangible (eg, resilience or self-belief) or because of extrapolation methods of predicting far into the future (eg, adult employment). Moreover, while the cost-effectiveness and ROI of outcomes in the health, social welfare and education sectors can be measured or estimated, impacts on inequality, intergenerational outcomes and benefits at a societal level may be less easily quantified. Such limitations will be addressed transparently and within subgroups where necessary.

Summary evidence will identify gaps in the literature and provide policymakers with cross-sectoral information relevant to strategic decision-making for early childhood investment.

Ethics and dissemination plan

The findings of this review will inform policymakers and practitioners in public health, education, social welfare and primary care settings. A series of academic publications is planned, focusing on overall findings and subsets of data. Policy papers addressing key areas will be disseminated through Telethon Kids Institute networks to policymakers and practitioners. Exemption from ethics approval was granted by the University of Western Australia Human Ethics Office (RA/4/20/5677).