Economic Evaluation of Interventions for Children with Neurodevelopmental Disorders: Opportunities and Challenges

Abstract

Economic evaluation is a tool used to inform decision makers on the efficiency of comparative healthcare interventions and inform resource allocation decisions. There is a growing need for the use of economic evaluations to assess the value of interventions for children with neurodevelopmental disorders (NDDs), a population that has increasing demands for healthcare services. Unfortunately, few evaluations have been conducted to date, perhaps stemming from challenges in applying existing economic evaluation methodologies in this heterogeneous population. Opportunities exist to innovate methods to address key challenges in conducting economic evaluations of interventions for children with NDDs. In this paper, we discuss important considerations and highlight areas for future work. This includes the paucity of appropriate instruments for measuring outcomes meaningful to children with NDDs and their families, difficulties in the measurement of costs due to service utilization in a wide variety of sectors, complexities in the measurement of caregiver and family effects and considerations in estimating long-term productivity costs. Innovation and application of evaluation approaches in these areas will help inform decisions around whether the resources currently spent on interventions for children with NDDs represent good value for money, or whether greater benefits for children could be generated by spending money in other ways.

Key Points for Decision Makers

Productivity costs or the loss of value of gross earnings resulting from NDDs for either a child with NDD or their caregivers/family should be incorporated.

Outcome measures need to be assessed for their ability to capture meaningful clinical change for the NDD population.

The evolving dependency relationships between the health of a child with NDDs and the quality of life of family members needs to be incorporated into the evaluations.

Background

Economic evaluation provides information on costs and consequences of healthcare interventions to prioritize interventions that could improve population health [1]. The decision problem addressed by economic evaluation considers the choice between different types of healthcare interventions by assessing the value for money, given a budget constraint. Neurodevelopmental disorders (NDDs) are a heterogeneous group of conditions with onset in the developmental period, characterized by impairments in personal, social, educational or occupational functioning. Manifesting in early development and ranging from specific limitations of learning or control of executive functions to global impairments of social skills and intelligence, these functional limitations have implications for the well-being of the child, the family, and society [2]. When resources are limited, decision makers are increasingly turning to evidence generated by economic evaluation to make decisions about resource allocation, but little evidence exists regarding economic evaluations of NDD interventions [3, 4].

While the number of economic evaluations in child health grows rapidly [5], few economic evaluations are specific to NDDs. Beyond the core challenges all evaluations working with a pediatric population face [6–10], NDDs warrant specific consideration. In this paper, we highlight some key challenges and opportunities for this population related to developing outcome measures meaningful to children with NDD [8, 11], costing service utilization in different sectors [1], incorporation of caregiver and family effects [12] and estimating long-term productivity costs [7].

Economic Evaluation Methods

Perspective and Time Horizon

The perspective in an economic evaluation provides a framework for analysis and determines what costs and effects to include and how to value them. The healthcare sector perspective includes formal healthcare costs borne by third-party payers or paid for out-of-pocket by families. The Guidelines for the Economic Evaluation of Health Technologies in Canada (CADTH) recommends reporting a reference case from a publicly funded healthcare payer perspective [13]. It should reflect the costs incurred by Canadian public payer and health effects for patients and their informal caregivers.

NDD interventions are complex and often include non-health outcomes such as education performance [14], employment [15], (social) participation [16], criminal activities [17], better captured by adopting a societal perspective. The societal perspective is a broader perspective, capturing health and non-health costs or effects such as time costs of children and families in seeking and receiving care, time costs of informal (unpaid) caregivers, transportation costs, effects on future productivity and consumption. The National Institute for Health and Care Excellence (NICE) recommends a broader perspective regarding cost, indicating that the cost in a reference case should use National Health Services and personal social services perspectives. NICE advises that the outcomes in a reference case should measure all the direct health effects relevant to patients and carers [18]. The second panel on cost-effectiveness in health and medicine in the USA (USA panel on CEA) recommends reporting the reference case analysis from a healthcare sector or a societal perspective [19].

For children with NDD, a lifespan time horizon can be important because the impact of interventions is expected to manifest in adulthood and throughout their lifespan. The time horizon should reflect resources consumed and outcomes experienced over the lifetime of participants as part of, or as a result of, an intervention. The USA panel on CEA, CADTH, and NICE recommend adopting a time horizon long enough to capture all relevant differences in the future costs and outcomes associated with the alternatives compared. However, longer time horizons introduce higher uncertainty because of variation in discount rates [20]. When creating long-term and lifetime models to capture the effects of an intervention, different discount rates should be considered [21].

Measurement of Costs

Costing involves the systematic identification, measurement, and valuation of resources used in the intervention and the comparators, and should reflect the opportunity cost for children with a NDD and their families utilizing these resources. Costs can be categorized into costs of care, and productivity costs. Costs of care can include expenditure on healthcare, therapeutic, behavioral or educational services, transportation, caregivers and other special needs services for a child with a NDD. Productivity costs include a reduction in the ability of either the child with a NDD (when they become adults), or parent of a child with a NDD, to sustain paid employment.

Children with NDDs need a range of services during their lifetime; often extending beyond healthcare services, utilization occurs in social services, rehabilitation, the education and criminal justice systems. A systematic review on the economic consequences of child and adolescent mental illness in the UK found that only 6% of the costs fall on the health system, and the majority of costs were from outside (social services, productivity, education and the criminal justice system) [22]. Furthermore, the needs of services for children with NDD diagnosis change as setting and age change [10]. For instance, a child with autism spectrum disorder (ASD) may require a home-based special educator or early intervention therapies in early childhood [23], special education during school years [24] and employment support or vocational training during the transition into adulthood [25]. Consequently, compiling and quantifying service usage in monetary units for an economic evaluation can be challenging [26]. While adopting a narrower public health-care system perspective might be an option, failure to incorporate other relevant sources and settings may bias the results when the majority of service usage is outside the healthcare system.

Productivity costs are often captured as the loss of value of gross earnings resulting from the NDD for the child, parents (or caregivers) and other family members relative to a comparator [27]. A lack of social skills, adequate training, appropriate working environment and education, or appropriate supports [28], means that adolescents with a NDD have enormous difficulties in getting a job, and keeping it, as they transition from school to employment. For example, in Canada, the employment rate of working-age adults with a developmental disability (22.3%) was less than one-third of the rate for people without a disability (73.6%) [29].

The impact on productivity for parents and caregivers can be considered in terms of time spent in formal or informal labor markets. Primary caregivers of children with a NDD often must give up work, reduce working hours, or change jobs to provide support and care. An estimated 85% of individuals with ASD need some measure of care and assistance from their parents and families for the duration of their lives [30]. The employment stability of family members in the household (including parents and siblings) is also affected by the health of a child with a NDD [31]. Mothers of children with disabilities are 3–11% less likely to work, and the effect (13–15%) is larger if the child is severely disabled [32].

When compiling and quantifying service usage in monetary terms, there are two main challenges. First, establishing a causal relationship can be difficult in practice. For example, distinguishing the hours of care provided by caregivers related to NDD compared to usual care is a challenge. Second, quantifying productivity for children in terms of the child’s future earnings is problematic. Persons with a NDD make less money on average than persons without a NDD [29]. While not unique to NDD, incorporating these productivity costs in an economic evaluation warrants careful consideration. For example, if we use a human capital approach to estimate the productivity costs, then costs associated with each day of lost work will be lower for an individual with NDD than for individuals without NDD. Given the high productivity costs for wealthy individuals, the economic evaluation would prioritize interventions for affluent individuals rather than interventions more prevalent among those who earn lower wages. However, failure to incorporate the impacts of early interventions on productivity in adulthood could potentially underestimate the effectiveness of the intervention.

Measurement of Outcomes (Effectiveness)

Measurement of health outcomes in children with NDDs typically involves assessment of physical, social, psychological and cognitive functioning. Outcomes should reflect the experience and well-being of children with NDDs who are utilizing supports and services. For reference case analysis, all three guidelines (USA panel on CEA, CADTH, and NICE) recommend reporting the health consequences of changes in morbidity or mortality in quality-adjusted life years (QALYs) [10–12].

Deriving a QALY in pediatric populations is a challenge. In a recent review of pediatric economic evaluations published between 1980 and 2014, only 24.9% were cost-utility analyses with the majority being cost-effectiveness analyses (63.9%) [5]. To date, very few evaluations of interventions for children with a NDD have reported QALYs, due to challenges with measurement. Instruments used in economic evaluation of interventions for NDD are summarized in Table 1 (general health profile measures) and Table 2 (preference-based measures) [11, 33]. Typically, natural units such as dependency-free life years [34, 35] clinical- or disease-specific outcomes [36–38] or general health profile measures (Table 1) that are familiar to clinicians and patients are used to measure consequences of NDD interventions. A recent cost-effectiveness analysis of a communication-focused therapy for pre-school children with ASD highlights challenges with clinical or disease-specific measures [37]. The primary outcome measure reflected autism symptom severity, and without the societal value relative to a unit improvement, it is difficult to compare interventions even within NDDs.

Table 1.

Examples of general health profile measures applicable to children with neurodevelopmental disorders (NDDs)

Instrumentsa	Domains	Number of items/questionnaires	Age	Raters	Used in children with NDD
Child Health and Illness Profile (CHIP): CHIP-CE and CHIP-AE [65]	CHIP-CE: satisfaction, comfort, resilience, risk avoidance CHIP-AE: satisfaction, discomfort, risk avoidance and resilience	45 and 108	CHIP-CE: 6–11 years CHIP-AE: 11–17 years	Child self-report/proxy report (parent)	ASD [66]
Child Health Questionnaire (CHQ): CHQ-PF28, CHQ-PF50 andCHQ-CF87 [67]	Physical functioning, role/social limitations, general health Perceptions, bodily pain/discomfort, family activities, parent impact, mental health, self-esteem, general behavior, family cohesion and change in health	28, 50 and 87	CHQ-PF28: 4–11 years CHQ-PF 50:5-18 years CHQ-CF87: 10 years or above	Child self-report/proxy report (parent)	Epilepsy [68], ADHD [69]
Functional Status II-R [70]	Communications, mobility, mood, energy, play, sleep, eating and toileting patterns	43 and 14	0–16 years	Child self-report/proxy report (parent)	Not found
Generic Children’s Quality of life Measure (GCQ) [71]	Worry, happiness, relationships with parents, general satisfaction, support, health/appearance, attainments	25	6–14 years	Child self-report	Not found
Health Status Questionnaire [72]	Malformation, neuromotor function, seizure, hearing, communication, vision, cognitive and other physical disability	8 clinical domains	2 years or older	Proxy report (parents, healthcare professionals)	Not found
KIDSCREEN: KIDSCREEN-52, KIDSCREEN-27 and KIDSCREEN-10 [73]	Physical well-being, psychological well- being, moods and emotions, autonomy, parents, relations and home life, peers and social support, school environment, bullying and financial support	52, 27 and 10	8–18 years	Child self-report/proxy report	ASD [74]
KINDL Questionnaire [75]	Psychological well-being, social relationships, physical functioning, everyday life activities	24 and disease specific module	3–17 years	Child self-reported/proxy report (parent)	CP [76]
Pediatric Quality of Life Inventory (PedsQLTM) [77]	Physical, social, emotional and school	23 and 35	2–18 years	Child self-report/proxy report (parent)	ASD [78], Cerebellar malformations [79]
The Inventory of Measuring Quality of Life in Children and Adolescents (ILK questionnaire) [80]	School, family, social contact with peers, interests and recreational activities, physical health, psychological health, overall assessment of the quality of life, exposure to diagnostic and therapeutic	9 thematic areas	6–18 years	Child self-report/proxy report (adults and their medical doctors or therapists)	ASD [81]
The TNO-AZL Questionnaires for Children’s Health-Related Quality-of-Life (TACQOL) [82]	General physical functioning/complaints; functioning: motor, daily, cognitive, social, global emotional (negative and positive)	56 and 43	6–15 years	Child self-report/proxy-report (others administered by parents)	Not found

ASD autism spectrum disorder, ADHD attention-deficit-hyperactivity disorder, CP cerebral palsy

^aThis is not a comprehensive list of the general health profile measures, but gives examples of instruments used most commonly in the field currently

Table 2.

Examples of preference-based health-related quality-of-life (HRQoL) measures applicable to children with neurodevelopmental disorders

Instrumentsa	Domains	Number of items/questionnaires	Age	Raters	Used in children with NDD?
16 Dimensional (16D) [83]	Mobility, vision, hearing, breathing, sleeping, eating, speech, excretion, school and hobbies, mental function, discomfort and symptoms, depression, distress, vitality, appearance and friends	16	12–15 years	Child self-report	Specific language impairment [84], Prader-Willi syndrome [85]
17 Dimensional (17D) [86]	Mobility, vision, hearing, breathing, sleeping, eating, speech, excretion, school and hobbies, discomfort and symptoms, depression, vitality, appearance, friends, concentration, anxiety, learning and memory	17	8–11 years	Child self-report	Specific language impairment [87]
Child Health Utility 9D (CHU9D) [88]	Worried, sad, pain, tired, annoyed, schoolwork/homework, sleep, daily routine and ability to join in activities	9 dimensions with 5 levels of response options per dimension	7–11 years	Child self-report	Not found
Euro QoL five-dimension questionnaire for youth (EQ-5D-Y)/EQ-5D[89]	Mobility, looking after myself, doing usual activities, having pain or discomfort, feeling worried, sad or unhappy	5 dimensions with 3 levels of response options per dimension and Visual Analogue Scale (VAS)-0 (the worst health you can imagine)-100 (the best health you can imagine)	4–11 years (4–7: proxy version, 8–11: self-report)	Child self-report/proxy report	ADHD [90], Spina bifida [91]
Health utilities index (HUI): HUI Mark 1, HUI Mark 2 and HUI Mark 3 [92]	HUI2: sensation, mobility, emotion, cognition, self-care, pain and fertility HUI3: vision, hearing, speech, ambulation, dexterity, cognition and pain	15 and 40	5 years and older (5–8, 8–12 and 13+ years)	Child self-report/proxy report (parent)	ASD [39], FASD [93]
Quality of well-being scale, self-administered (QWB-SA) [94]	Mobility, physical activity and social activity	N/A	N/A	Child self-report	ASD [39]
Standard gamble (SG) [95]	N/A	N/A	N/A	Child self-report/proxy report (parent)	ADHD [96, 97]
Time trade off (TTO) [98]	N/A	N/A	N/A	Child self-report/proxy report (parent)	Not found

ADHD attention-deficit-hyperactivity disorder, ASD autism spectrum disorder, FASD fetal alcohol spectrum disorder, N/A not applicable

^aThis is not a comprehensive list of the preference-based HRQoL, but gives examples of instruments most commonly used in the field currently

Many general health profile measures are adapted from measures developed for adult populations and not designed for NDDs. Each instrument has different domains, and adapting to a NDD intervention can be difficult. For example, the Child Health Questionnaire (CHQ), KIDSCREEN and Child Health and Illness Profile (CHIP) have a significant number of items related to social dimensions such as school, family, and peers, whereas the Pediatric Quality of Life Inventory (PedsQL) has more items that focus on physical and emotional functioning (Table 1). A recent study by Janssens et al looked at these instruments in the context of NDD and found a lack of evidence on responsiveness and measurement error, making it difficult to distinguish meaningful (clinically important) change [11]. Of the instruments assessed in this study, the PedsQL and CHQ were most evaluated in NDD populations, but findings were not satisfactory. Ultimately, the main disadvantage of using either a general health profile or disease-specific measure for economic evaluation is the lack of standardization into a scale that can elicit a QALY.

Few studies relating to children with a NDD have used preference-based measures as an outcome measure (Table 2). There are also differences in the domains captured in preference-based measures. The reliability and validity of preference-based instruments in children with NDD need to be examined. For instance, a study by Tilford et al [39] comparing preference-based and clinical measures in ASD, found that the Health Utilities Index (HUI)-3 to be a good measure of effectiveness and should be included in clinical trials. Other studies are needed to assess these instruments in other NDDs for their ability to capture meaningful clinical changes.

Using direct approaches such as standard gamble or time trade-off for deriving a QALY is another approach, but can be as problematic for children with NDD as in other pediatric populations. Children with a NDD are more likely to experience emotional and behavioral dysregulation, social exclusion, isolation and poorer academic performance than their peers who do not have a disability [40]. This may influence a child’s understanding of health and well-being, meaning they may not have the capacity to understand the concept of time in a way that would allow them to indicate their preference.

If preference-based measures are not feasible, general health profile measures can be converted into QALYs using mapping [41]. This is an emerging opportunity in the field of NDDs as it allows for adaptation of data collected from commonly used general health profile measures such as the KIDSCREEN or PedsQL to estimate a QALY. Algorithms for mapping general health profile measures into health utilities have been reported for some conditions [42]; however, very few algorithms have been created specifically for children with NDD [43].

Ideally, outcome measures should reflect a child’s experience and perception. Due to feasibility, reliability, and validity issues, proxies such as parents, caregivers and healthcare professionals are used. Proxy reporters can be effective for visible signs and symptoms but are less accurate for subjective measures such as quality of life (QoL), emotion, and utility. Reporting is impacted by the proxy’s knowledge, experiences, and expectations [6]. This is particularly important as each NDD has a unique etiology and wide heterogeneity of symptoms and co-morbidities such as mental health considerations [44].

The limitations of QALYs in capturing non-health outcomes has resulted in some researchers calling for more comprehensive outcome measures [45, 46]. Sen’s “Capability approach” has been proposed as one alternative [47–49]. The capability approach is based on the view of living as a combination of various ‘doings and beings,’ with the QoL to be assessed in terms of the “capability to achieve valuable functioning” [50], p31. The essence of the capability approach is that “an individual’s well-being should not be measured according to what they actually do (that is their function) but what they can do (their capabilities) [49], p850. In this context, the capability approach offers the potential for a richer set of dimensions in the evaluation of interventions for children with NDD [47]. A number of capability-based instruments/questionnaires (Investigating Choice Experiments Capability Measure [ICECAP], Oxford Capability Instruments [OxCAP], Adult Social Care Outcome Toolkit [ASCOT], etc.) have been developed for use in healthcare. However, application of this approach in economic evaluations of interventions for children with NDDs requires development and testing of the capability-based questionnaire/instruments for this heterogeneous population.

Caregivers and Family Effects

The notion that children are not ‘isolated individuals,’ but rather have a social circle of parents, siblings, other relatives and friends has emerged in the health economic literature [12]. There are two main effects, described as ‘caregiving effects’ and ‘family effects,’ which are spillover effects. The caregiving effect refers to the impact of a child’s health on the QoL and economic well-being of a caregiver, and family effects refer more broadly to the impact of a child’s health on QoL and economic well-being of family members, applying to parents, siblings and other relatives living in the household. The importance of incorporating spillover effects in economic evaluations of health interventions has been described [51, 52]. Several guidelines (US panel on CEA, CADTH and NICE) recommend including spillover effects for caregivers in the reference case analysis [13, 18, 19]; however, applications to specific conditions and treatments remain limited.

An intervention can have positive and negative effects on the QoL and economic well-being of both the child with NDD and their caregivers and family members. Parents of children with NDD report poorer physical and mental health and experience higher levels of family distress than parents who are not raising children with NDD [53]. A longitudinal population study looked at the impact on the maternal and paternal health of having a child with a disability in the household. After controlling for previous health status and other sociodemographic characteristics, the health of mothers declined compared to fathers [54]. Caregivers of children with NDD experience more health and psychological problems such as stress, problematic family functioning, migraine headaches and asthma compared to caregivers of neurotypical children [53]. Informal caregivers’ health suffers as the severity of disability and needs of patient increases [55, 56]. Siblings of children with a chronic illness may also be impacted, as a meta-analysis showed fewer peer activities and lowered developmental cognitive scores compared to siblings without a chronic illness [57].

Acknowledging these dynamic, complex and changing dependency relationships between a child’s health and the QoL of family members, a number of researchers have proposed different methods of incorporating family and caregiver effects into economic evaluation. Basu and Meltzer developed a theoretical framework based on a utility function for measuring family spillover effects and have shown how medical treatment could provide direct and indirect effects on the welfare of all family members [58]. Al-Janabi et al advanced research on health spillovers by developing a framework, which involves the inclusion of two multiplier effects: multiplier effects for health benefits generated and displaced by a new intervention [59].

A few methodological approaches for isolating and measuring spillover health effects of caregivers and family members (separately) have been introduced in the literature. Brouwer et al and Poley et al have used the EQ-5D to measures the health-related QoL of rheumatoid arthritis caregivers and parents providing informal care to young patients with congenital anomalies [56, 60]. They found that effects of illness were extended to other members of the family. Similarly, using a modified time trade-off method, Basu et al evaluated the impact on the QoL of partners due to a patient experiencing prostate cancer [61]. The spouse of the patient was asked to trade off his or her life based on their expected burden if the patient developed one of the prostate cancer-related health states. Using standard gamble (SG) methods, Prosser et al asked family members of Alzheimer/dementia, arthritis, cancer and depression patients to value the spillover effects [62] and found that effects of illness extend beyond the individual patient to caregivers, children, a spouse, and family members. Spillover effects on caregivers can also be measured using the CarerQol, a carer-related QoL instrument [63, 64]. While numerous methods to incorporate caregiver and family effects have been suggested in the literature, there is a lack of standardized methodology, and more empirical and theoretical work related to NDD is warranted.

Conclusion

There is an urgent need for identifying or developing the most appropriate instruments or methods that take into consideration the measurement challenges present for economic evaluations of interventions for children with a NDD. This paper is the first step to highlighting these challenges and outlines considerations when conducting an economic evaluation in this field. Children with NDD present unique challenges for health-specific outcome measures, as they often lack cognitive, communication and social abilities to respond to questionnaires on health and well-being. Furthermore, there is uncertainty about the measurement of costs, for instance when and how to measure costs for caregivers, and how to estimate and incorporate productivity costs in economic evaluation. Further research on methods to measure spillover effects on family members and caregivers and productivity costs of NDD is needed. Understanding the heterogeneity of children with NDDs in terms of their patterns of healthcare utilization, resources used, and dependency relationships with caregivers is important to ensure the measurement of costs and consequences are more representative in the economic evaluation.

Author Contribution

Both authors contributed to the manuscript. RL led the manuscript drafting and JZ leading conception of the manuscript, drafting and revisions.

Compliance with Ethical Standards

Ethical approval

No specific ethical approval was sought for this leading article.

Conflict of interest

Ramesh Lamsal and Jennifer Zwicker work on economic evaluations of interventions for children with NDD.

Funding

RL and JZ are supported by grants from the Kids Brain Health Network, a Network of Center of Excellence and CHILD-BRIGHT, funded by CIHR. The Social Policy and Health Research Unit is supported by the Sinneave Family Foundation.