Abstract
Background
Psychosocial (e.g., anxiety or behavior) problems lead to costs not only in the healthcare sector but also in education and other sectors. As psychosocial problems develop during the critical period of establishing educational trajectories, education costs are particularly relevant in the context of psychosocial problems among children and adolescents.
Objectives
This study aimed to gain insights into the methods used for the inclusion of education costs in health economics studies and into the proportion of the education costs in relation to the total costs associated with a condition or an intervention.
Methods
We systematically searched the PubMed, Embase, SSCI, CINAHL, PsycINFO, ERIC, and Econlit databases in August 2019 for economic evaluations of mental health, psychosocial and educational interventions, and cost-of-illness studies of mental, behavioral, and neurodevelopmental disorders conducted from a societal perspective in populations of children and adolescents. An additional search was conducted in February 2021 to update the review.
Results
In total, 49 articles were included in the analysis. The most common cost items were special education, school absenteeism, and various educational professionals (educational psychologist). A variety of methods were employed for the identification, measurement, and/or valuation of education costs. The proportion of education costs to the total costs of condition/intervention ranged from 0 to 67%, with the mean being 18.5%.
Discussion
Since education costs can constitute a significant proportion of the total costs of an intervention or condition, including them in health economics studies might be important in informing optimal resource allocation decisions. Although various methods are available for including education costs in health economics studies, further research is needed to develop evidence-based methods for producing comparable estimates.
Supplementary Information
The online version contains supplementary material available at 10.1007/s40273-021-01049-y.
Key Points for Decision Makers
Psychosocial problems among children and adolescents often lead to costs in the education sector. However, these costs are not often included in health economics studies. |
While a variety of methods for identification, measurement, and valuation of education costs in health economics are available, systematic guidance on which methods are the most optimal is lacking. This hampers the comparability and transferability of study results. |
The results of this study suggest that the proportion of the education costs in relation to the total costs associated with psychosocial problems can be substantial. Therefore, considering these costs for inclusion in health economics studies can be important for informing optimal resource allocation decisions. |
Introduction
Individuals affected by psychosocial problems not only have an increased need for healthcare but also their ability to participate in education and work or enjoy leisure time may be diminished. Hence, the economic impact of psychosocial problems is cross-sectoral and is not limited to the healthcare sector [1]. In fact, a large proportion of the costs associated with psychosocial problems occurs in sectors other than healthcare, including education, employment and productivity, informal care, and criminal justice [1, 2]. In the absence of a commonly accepted definition, for this study we defined psychosocial problems as a range of psychological problems (e.g., learning difficulties, anxiety, behavioral issues) and social problems (e.g., educational or family difficulties) that affect one’s “functioning in daily life, his or her environment and/or life events” [3]. As psychosocial problems often begin to develop in childhood or adolescence, their onset might occur during the period when individuals establish their educational trajectories [4]. Therefore, the economic impact of psychosocial problems on the education sector is likely to be particularly evident in the context of common conditions among children and adolescents (e.g., autism and attention-deficit/hyperactivity disorder). Research shows that supporting individuals with psychosocial problems can incur far more costs than attributable to their peers without these problems [5, 6]. Psychosocial problems among children and adolescents can lead to diminished school productivity and school performance, increased need for additional support at school, early dropout, and potentially reduced educational attainment, all of which are associated with substantial overall societal losses and excessive costs for the education sector in particular [2, 5, 7, 8]. Several health economic studies in the psychosocial domain have also demonstrated that education costs can constitute a large proportion of the total costs and even be the most expensive cost category [9, 10]. Therefore, including education costs in health economics research studies could be crucial to ensuring optimal resource allocation decision making [11].
Despite recent methodological developments in identifying, measuring, and valuing education costs [12–16], their inclusion in costing research remains limited [17]. This could be attributed to several reasons. First, a narrow perspective is still predominant in the majority of current pharmacoeconomic guidelines, many of which neglect costs other than healthcare and productivity costs [18]. Second, explicit methodological guidance on how to collect, measure, and value education costs in health economics studies is limited. The current methods for the identification, measurement, and valuation of the education costs in health economics focus only on a limited number of costs and do not allow for producing valid and internationally comparable estimates. Third, the evidence regarding the impact of education costs on the results of health economics studies is lacking. Although several studies have demonstrated that the proportion of education costs to total costs can be significant [9, 10], a systematic synthesis of the existing evidence could provide a sound rationale for including education costs in health economics research studies in the psychosocial domain.
Given these research gaps, a systematic review of health economics studies, i.e., economic evaluations and cost-of-illness studies, in the psychosocial domain was conducted to fulfill the aims of this research. As discussed in more detail in the methods section, potentially relevant studies included economic evaluations of psychosocial interventions and cost-of-illness studies of mental health, behavioral, and neurodevelopmental disorders. First, this study aimed to gain insight into whether studies in the psychosocial domain included education costs, and, if so, what methods they used to identify, measure, and value education costs. This would provide an overview of the available methods for including education costs in health economics studies and highlight current knowledge gaps, information that can direct further research in this field. Second, this study aimed to estimate the proportion of the total costs that education costs account for, to provide justification for including education costs in health economics studies.
Methods
This study adopted a systematic review methodology standardized in health economics, based on the methods described by van Mastrigt et al. [19]. Reporting of the results was based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [20].
Data Collection
To identify health economics research studies that included costs in the education sector, seven electronic databases were searched in August 2019: PubMed, Embase (Ovid), SSCI (Web of Science), CINAHL (EBSCO), PsycINFO (EBSCO), ERIC (EBSCO), and Econlit (EBSCO). The selection of databases and the search string were developed and consolidated by the research team in collaboration with an information specialist. An example of the search string in PubMed is available in the electronic supplementary material (ESM)-1. An additional search using the same approach was carried out in February 2021 and covered the period between August 2019 and February 2021.
Inclusion and exclusion criteria for the identified studies were based on the PICOS framework [21]. They included studies with a population of children and adolescents aged 0–25 years (P), investigating psychosocial and educational interventions (I), regardless of the comparator (C) or the outcomes (O), relating to societal costs, and designed either as a full economic evaluation or cost-of-illness study (S). Psychosocial interventions were defined as “interpersonal or informational activities, techniques, or strategies that target biological, behavioral, cognitive, emotional, interpersonal, social, or environmental factors with the aim of improving health functioning and well-being” [22]. Both economic evaluations and cost-of-illness studies were considered for inclusion only if costs in the education sector were included as intersectoral costs associated with a condition or an intervention. Studies that included costs in the education sector as part of the intervention costs (e.g., training of professionals, materials) were considered beyond the scope of the current review and excluded, as there is research that provides detailed methodological guidance on how to include intervention costs in economic evaluations [23]. The relevance of cost-of-illness studies was determined based on the International Classification of Diseases, Eleventh Revision; the relevant conditions were mental health and behavioral and neurodevelopmental disorders [24]. Studies were excluded when the abstract and/or the full text was not available in the English language or could not be accessed.
Identification and Screening of Relevant Studies
All hits were imported into Endnote X8.2 and de-duplicated using the method determined by Bramer et al. [25]. One reviewer (IP) performed title and abstract screening. The second reviewer (LJ) screened 10% of randomly assigned titles and abstracts. A similar procedure was employed for the full-text screening. Differences in inclusion were resolved by discussion between the two reviewers. The ultimate inclusion of studies based on full-text screening was determined in the discussion between the two reviewers.
Data Extraction and Analysis
The number of studies that were excluded during full-text screening because they did not include education costs and the number of studies that complied with the inclusion criteria were used to calculate the proportion of the studies that included education costs. Data extraction for the studies that complied with the inclusion criteria was conducted by multiple reviewers independently. One reviewer extracted data from all studies (IP); two reviewers (RD, LS) extracted data from 50% of the included studies each. The data on the general characteristics of the selected articles and the methods used to identify, measure, and value education costs were extracted using a predefined template adapted from Wijnen et al. [26], which was piloted on 10% of the included studies by two reviewers (IP and LJ) (ESM 2).
The reported unit costs were recalculated to €, year 2021 values, using an online inflation tool [27] and a currency converter [28] and presented in ranges per reported measurement unit (e.g., per day, per year). When the original year for reported unit costs was not provided, the year of the article’s publication was used for the recalculation. For studies that included costs in the education sector and reported the breakdown of total costs and/or incremental cost-effectiveness ratio calculation, the percentage of total costs that was accounted for by costs in the education sector was determined. When multiple groups (i.e., treatment arms) were compared in an article, the percentage of education costs was calculated separately for each group. The overall proportion of the education costs was calculated as the average of every estimate. In addition, this proportion was also calculated by averaging the mean estimates per article to avoid bias. The percentages of education costs for each condition were summarized and presented separately. When articles reported the inclusion of costs in the education sector but did not provide the breakdown of total costs, the authors of these articles were contacted for further information.
Results
The database search conducted in August 2019 resulted in 10,236 hits. After de-duplication, the titles and abstracts of 6248 articles were screened, and 5937 of them were excluded. The full texts of 311 records were assessed for eligibility, resulting in 270 records being excluded. The additional search in February 2021 resulted in 1705 hits. After de-duplication, the titles and abstracts of 1048 articles were screened and 995 of them were excluded. The full texts of 53 records were assessed for eligibility, resulting in 45 records being excluded. In total, 41 records identified in the search in August 2019 and eight records identified in the search in February 2021 fulfilled the inclusion criteria and were included in the analysis. The process of the literature search and selection is shown in the PRISMA flowchart (Fig. 1).
General Study Characteristics
In total, 49 studies fulfilled the inclusion criteria and were included for data extraction and data analysis [9, 10, 29–75]. The majority of the studies (n = 16 [33%]) were conducted in the Netherlands, the USA (n = 8 [16%]), and Canada (n = 7 [14%]), followed by the UK (n = 5 [10%]), Sweden (n = 5 [10%]), Germany (n = 2 [4%]), Australia (n = 1 [2%]), Spain (n = 1 [2%]), and Ireland (n = 1 [2%]). Furthermore, two studies focused on multiple countries [36, 45], and one study did not focus on any particular country [55]. Concerning the study design, over one-half of the studies were cost-of-illness studies (n = 26); the remaining studies were either trial-based (n = 12) or model-based (n = 8) economic evaluations or a combination of both (n = 1) [51]. One study described an economic evaluation study using a quasi-experimental design [74]. One study combined a cost-of-illness study with a model-based economic evaluation [32]. The majority of the studies focused on mental and neurodevelopmental disorders among children and adolescents, including autism spectrum disorder (n = 10), attention-deficit/hyperactivity disorder (n = 8), and anxiety disorders (n = 6).
Methods for Including Education Costs
Identification
In total, 92 education cost items were extracted from the 49 studies and clustered based on similar descriptions (ESM 3). The most commonly identified education cost items were special education (n = 22), school absenteeism (n = 17), different types of professionals in the education sector (n = 17), and additional educational support services such as tutoring and homework help (n = 8). Two studies reported using the classification scheme by Drost et al. [14] to identify relevant education costs [41, 73]. In five studies, relevant cost categories were identified based on the literature search [9, 45, 52, 55, 64]. In one study, cost selection was based on data included in administrative records [51]. In addition, the authors of five studies justified their selection of cost categories as including all potentially relevant costs based on the definition of a societal perspective [30, 33, 38, 47, 71] and education costs in particular as “typical components of direct nonmedical costs” [43], “important cost for many children with autism” [47], and “important resource units” [72]. The majority of the studies (70%) did not report any specific methods used for identifying the education cost items.
Measurement
For measuring education costs, 27 studies used self-reported data, 13 studies derived data from the literature (i.e., other studies), six studies used other secondary sources (e.g., national-level statistics data, government reports, administrative records), two studies used a combination of primary and secondary data [45, 74], and one study used literature combined with expert opinion [64]. Figure 2 illustrates the frequency of the use of the measurement methods per cost item. For example, special education was measured most often using literature (n = 8) or self-reported data (n = 8), followed by a combination of methods (n = 4) and other secondary sources (n = 2). The figure shows that most education cost items (n = 65) were measured using self-reported data. It is important to note that, in Figs. 2 and 3, the extracted cost items are presented based on the clustering as described above.
The majority of studies collected self-reported data from parents or other primary caregivers (n = 21); fewer studies collected self-reported data from children or adolescents alone or in addition to caregivers (n = 5), study investigators (n = 1), or government officials (n = 1). The most common methods for collecting self-reported data were questionnaires (n = 21), followed by cost diaries (n = 5), and case report forms (n = 1). The resource-use measurement questionnaires that were used included the (modified) Treatment Inventory of Costs in Patients with psychiatric disorders (TiC-P) (n = 6), the modified Client Service Receipt Inventory (n = 1), the Child and Adolescent Service Use Schedule (n = 2), the Carer Service Use Schedule (n = 1), the Expenditures for Health and Social Service Utilization Questionnaire (n = 1), the Child and Adolescent Services Assessment (n = 1), the Studying the Costs of Parental Expenditures instrument (n = 1), or other study-specific resource-use measurement instruments (n = 5). Nine articles did not report what specific resource-use measurement instruments were used to collect self-reported data.
Valuation
The unit costs for the extracted education cost items (n = 92) were derived from various secondary data sources (n = 65 [70%]), calculated by the authors using secondary data (n = 12 [13%]), taken at face value (e.g., out-of-pocket expenses) (n = 5 [5%]), or calculated based on the market price of a given service (n = 1 [1%]). Nine items (10%) were not valued or were included as either intangible costs (i.e., they were measured but not valued) or as the impact on a caregivers’ costs (e.g., lost productivity, time, or money). The secondary sources of unit costs included official public sources such as national-level public data (e.g., government reports) (n = 34), costing guidelines (n = 12), literature (n = 11), or a combination of literature and public sources (n = 1). For seven items, the sources of unit costs were not reported. For the 30 extracted items, the unit costs were not reported.
Figure 3 illustrates the frequency of use of different methods for the valuation of education cost items. For example, the unit cost of school absenteeism was most frequently developed by the authors (n = 8) or extracted from secondary national sources (n = 4). In three studies, school absenteeism was included as an intangible cost, and two studies used costing guidelines. ESM 3 provides a detailed overview of the measurement and valuation methods applied to measure and value the extracted items. The reported unit costs of the education cost items are presented in Table 1 in €, year 2021 values, per measurement unit. The items for which the unit of measurement was not reported are presented in separate rows.
Table 1.
Item | Measurement unit | Range (€, year 2021 values) | Sources |
---|---|---|---|
Special education | Per day | 15.4–15.6 | [64, 70] |
Per month | 449.8 | [69] | |
Per year | 588.1–56,534.5 | [10, 42, 45, 47, 56, 58, 59, 71] | |
Average lifetime cost | 7949.1–13,852.8 | [32, 57] | |
Not reported | 677–1771.32 | [62] | |
School absenteeism | Per hour | 5.3–11.4 | [33, 34, 65, 66, 71] |
Per day | 28.6–68 | [38, 48, 53, 54, 63] | |
School presenteeism | Per day | 61.6–68 | [48, 53, 63] |
Remedial teaching | Per visit | 63.6 | [64] |
Per hour | 63.1 | [69] | |
Aggregate societal expenditures for education | Per year | 5599.3–6981.7 | [9, 55] |
Study/homework help | Per hour | 44.1 | [48] |
Tutoring help | Per hour | 48.7 | [53] |
Educational psychologist | Per hour | 48.5 | [62] |
Not reported | 51 | [38] | |
Attendance officer | Per hour | 47 | [41, 73] |
Teacher training | Per visit | 83.4 | [64] |
General/specialist services at school | Not reported | 25.3 | [40] |
Education welfare officer | Not reported | 27.6 | [38] |
Family liaison officer (school) | Per hour of client work | 67.5 | [38] |
School counselor | Per hour | 14.9 | [30] |
Special needs assistant | Per hour | 18.6 | [62] |
Teacher | Not reported | 48.2 | [38] |
Parental out-of-pocket expenses related to child's education | Annual per patient and total costs | 64.3 | [9] |
Financing of education | Per year | 13,846.7 or 27,693.4a | [44] |
Not reported | 193–1094 | [62] | |
Aggregate savings for the education sector | Per year per child | 8185.40 | [61] |
Productivity gains from school attainment | Per high school diploma | 171,133–23,7784.9 | [74] |
aDepending on disability level
Contribution of the Education Costs to the Total Costs
Of the 49 included studies, 31 contained sufficient detailed information to calculate the proportion of the education costs. Contact with the authors did not allow us to collect any additional information either because of nonresponse or lack of time to provide further information. Because some studies included multiple groups or treatment arms, the number of estimates (n = 72) was higher than the number of studies (n = 31). The average proportion of the education costs based on the average of every estimate was 18.48% (n = 72), ranging from 0 to 78.5%. A slightly higher proportion of 20.37% was estimated based on the average of the means per study (n = 31). Figure 4 illustrates the percentage of the education costs to total costs. The majority of the estimates (64% [n = 46]) lay below the mean value of 18.48%. Approximately one-quarter of the estimates (22% [n = 16]) were within the range between the mean and 40%. Only 13% (n = 10) of the estimates were above 40%, in the range of 44.3–78.5%. The highest estimates (above 50% of the total costs) were reported in six studies; one was an economic evaluation of an exposure-based cognitive behavioral therapy for adolescents diagnosed with irritable bowel syndrome [63]; five were cost-of-illness studies of autism spectrum disorder, intellectual disability, attention-deficit/hyperactivity disorder, and mental problems in general in a population of children and adolescents [9, 10, 36, 44, 52]. A summary of the studies is provided in ESM 4.
The largest number of estimates was provided for the studies that investigated autism spectrum disorder (n = 18), anxiety disorders (n = 9), and attention-deficit/hyperactivity disorder/attention-deficit disorder (n = 8). The highest proportion of education costs was observed for intellectual disability disorder (67%), but this was based on a single study [44]. Studies that focused on autism spectrum disorder and attention-deficit/hyperactivity disorder/attention-deficit disorder accounted for the second and third highest proportion of education costs: 30 and 24%, respectively.
Discussion
Psychosocial problems have a far-reaching impact on all societal sectors, including the education sector [2]. The economic impact of psychosocial problems on the education sector is especially relevant for children and adolescents [5]. Nevertheless, little is known about including costs from the education sector in health economics studies in the psychosocial domain. The aim of this study was to investigate whether health economics studies in the psychosocial domain that focused on child and adolescent populations included costs from the education sector, and, for those studies that did, what methods the authors used to identify, measure, and value these costs. In addition, for the studies that included education costs, this study aimed to estimate the proportion of education costs to total costs.
The findings of this study revealed that only approximately 20% of the health economics studies relating to the psychosocial domain included education costs. Regarding the country of origin of the included studies, the consideration of education costs is explicitly recommended only in the Dutch, Canadian, and Irish pharmacoeconomic guidelines [76–78]. However, in the absence of evidence-based guidelines in other countries regarding including education costs in health economics studies, it is difficult to assess whether the education costs should have been included in each particular study. For example, the guidelines of the National Institute of Health and Care Excellence in the UK suggest considering broader costs if these are substantially influenced by an intervention but do not provide any further guidance in terms of the types of costs to be included and how to determine their relevance [79]. This means that the decision whether to include education costs oftentimes depends on the researchers, who might not be aware of the relevance of the education costs and might be constrained by a lack of resources, the requirements of the national pharmacoeconomic guidelines, and other factors. In addition, the decision to include education costs might also lead to the need for additional resources (e.g., extended questionnaires, more time for data analysis), which are often scarce.
The education cost items identified in this review comprise services, professionals, lost productivity (i.e., absenteeism and presenteeism), and the financial expenses on education. The list also includes disciplinary measures (i.e., detention, expulsion, etc.), which were relevant in the context of attention-deficit/hyperactivity disorder, although these were not valued. In comparison with the existing classifications [14, 16], this study identified more services and educational professionals. However, intangible consequences, such as reduced educational attainment or problems with school entry, were scarcely captured in the reviewed studies. It is important to note that the costs associated with reduced school attainment become more apparent in the longer term, which makes it more challenging to capture these costs in studies with shorter time horizons of 1 or 2 years. In addition, the most frequently included education cost items, “special education,” “absenteeism from school,” and “additional educational support (e.g., tutoring, homework help),” correspond with the findings of the recent review of health-related resource-use measurement instruments by Mayer et al. [13]. However, less tangible resource-use items such as “school functioning” and “social functioning” were also not included in the reviewed studies. This might be attributed to the limited time horizon of the studies as well as the difficulty of valuing these costs. Furthermore, it can be argued that these costs can also be captured by quality-of-life measurement instruments. This highlights the relevance of the issue of double counting, which researchers need to be aware of.
A variety of methods were employed for measuring and valuating the education costs. Self-reported data were used more frequently than secondary data sources. Furthermore, the studies that were based on self-reported data included more cost items than the studies that relied on secondary sources, which often provide pooled estimates. Resource-use measurement questionnaires were preferred over other self-reported data collection methods, with TiC-P being the most commonly used. There was considerable variation between the estimates for comparable items, which could be attributed to the selection of the measurement and valuation methods, among other factors. In addition, the content of the items was not always clearly based on the name and/or the semantic description of the item, which further limited comparability. For example, special education was the most commonly included education cost in the reviewed studies. However, it was not always clear how the authors defined special education and, based on the definition, what type of service was measured and valued. Above all, the findings suggest that the variability in the methodologies used to include education costs and the general lack of transparent reporting limits meaningful comparison between the studies. This also indicates the need for the development of evidence-based harmonized methods and instruments.
The proportion of education costs in the reviewed studies varied from 0 to 78.5%. The highest proportion (above 50% of the total costs) was observed in studies of autism spectrum disorder, intellectual disability, attention-deficit/hyperactivity disorder, and mental problems in general in populations of children and adolescents. It is important to note that the proportion of the education costs in the reviewed studies might have been influenced by the choice of study design as well as by the number of cost categories and the number of the education cost items included in each particular study. Nevertheless, this finding suggests that education costs can constitute a significant proportion of the total costs and thus are likely to influence the results of the study. This also provides a rationale for considering including education costs in health economics studies in the psychosocial domain to ensure optimal resource allocation in society.
Methodological Reflections
To the best of our knowledge, this is the first study to systematically review health economics literature in relation to the inclusion of education costs. Broad inclusion criteria were employed to incorporate studies with various designs. Furthermore, a detailed overview of possibly relevant cost items and currently available methods for including education costs in health economics studies was presented. Nevertheless, the study is also subject to several limitations. First, the context of this study was limited to the psychosocial domain, whereas education costs can also be relevant for other disease areas (e.g., infectious diseases) [17]. Using the methods of the current study to investigate economic evaluations and cost-of-illness studies in other contexts could provide a more complete overview of the available methods for including education costs. Second, the inclusion criteria for this study were fairly broad, which allowed for collecting more data but did not allow for making meaningful comparisons between studies that included education costs and studies that did not, because of heterogeneity. Third, the involvement of a second reviewer in the screening process was limited to a random sample of 10% of the studies at each stage. Fourth, the findings of the current study depended on the quality of reporting in the included articles, particularly regarding transparency and the level of detail when reporting the methods that were used for the identification, measurement (e.g., measurement units, resource-use measurement instrument used), and valuation (e.g., unit costs and their sources) of the education costs. Furthermore, the lack of detail in several studies did not allow for calculation of the proportion of the education costs. Although several attempts were made to contact the authors of the studies, we were unable to gather additional information, either because of lack of time or nonresponse.
It is also important to note that the definition of the education sector and what falls under it might not always be clear. The reviewed studies included several cost items that might not necessarily fall within the education sector. For example, government subsidy on education might fall within the government sector, as it does in the Canadian costing guidelines [77]. Another example is out-of-pocket expenses for education, which might in fact fall within the category of patient and family costs, as these costs are incurred by the household. In addition, training of professionals (e.g., nurses) who will be delivering an intervention might also be an example of an education cost. In this study, we included training of professionals as part of intervention costs, although it was considered an education cost if the intervention was set in the education sector but not if the intervention was set in another sector (e.g., healthcare sector). While this could be considered a limitation, the literature provides detailed methodological guidance on how to include intervention costs in economic evaluations [23]. Therefore, further investigation of these studies was beyond the scope of the current article.
Implications for Further Research
The findings of this study demonstrate that the incorporation of education costs in health economic studies can be hindered because of the lack of methodological guidance and difficulties collecting reliable comparable data to estimate these costs. These challenges dictate the need for further research in this direction. First, the relevance of including education costs depends on various factors such as target population, condition and/or intervention of interest, and national setting, among others. Further research could focus on investigating the relevance of specific education cost items in relation to these factors, for example, which costs are most relevant for the child population with attention-deficit/hyperactivity disorder. Second, the use of standardized terminology might help facilitate the comprehensibility and comparability of the cost items. One example of a classification system that can facilitate the comparability of the services is the standard taxonomy for description, mapping, and comparison of services for long-term care (Description and Evaluation of Services and Directories in Europe for Long-Term Care [DESDE-LTC]) [80]. It has been widely used in health services research to enable comparison of services across settings [81]. Application of this system to other sectors, including the education sector, might help facilitate the comparability of the services included in future health economics studies. Third, reduced educational attainment because of psychosocial problems in childhood and adolescence can influence one’s labor market outcomes [5], which might be especially notable for individuals whose educational needs have not been met. Accounting for this relationship would be particularly relevant for studies that adopt a long-term timeframe and would require further research to develop appropriate methods for capturing this effect in health economic evaluations and cost-of-illness studies. It is also important to consider the potential issue of double counting that might arise because costs associated with educational attainment and consequences on the labor market would be interlinked. Fourth, future research is needed to investigate the appropriate methods for the valuation of school productivity, since school productivity is conceptually different from work-related productivity.
Conclusion
Education costs can constitute a large proportion of the total costs in health economics studies set in the psychosocial domain, especially in populations of children and adolescents. Therefore, inclusion of education costs, if those are substantially affected given the study context, could be crucial for informing optimal resource-allocation decisions. The findings of this study demonstrate that, although methods for the identification, measurement, and valuation of the education costs are available, the lack of methodological guidance regarding the proper use of the methods leads to considerable variation and limits the comparability of the studies. This indicates the need for further research in this direction, especially with respect to the development of evidence-based methodological recommendations for the inclusion of the education costs in health economics studies. By demonstrating the broader impact of psychosocial conditions and interventions to treat them on the education sector, health economics research can play a crucial role in optimizing the use of limited societal resources. Further research in this direction also has the potential to alleviate the urgent problem of underfunding of the mental health domain by attracting additional (cross-sectoral) resources.
Supplementary Information
Below is the link to the electronic supplementary material.
Acknowledgments
This manuscript was discussed at the Lowlands Health Economic Study Group (lolaHESG) conference in 2020. The authors thank Prof. Judith Bosmans and all others who were present and contributed to the discussion.
Declarations
Funding
I. Pokhilenko and LMM Janssen are partially funded by the PECUNIA project, which has received funding from the European Union’s Horizon 2020 research and innovation program under Grant agreement no. 779292.
Conflicts of interest
Irina Pokhilenko, Luca M. M. Janssen, Silvia M. A. A. Evers, Ruben M. W. A. Drost, Lena Schnitzler, and Aggie T. G. Paulus have no conflicts of interest that are directly relevant to the content of this article.
Availability of data and material
All data collected and analyzed during this study are included in this published article and its supplementary information files.
Ethics approval
Not applicable.
Consent
Not applicable.
Author contributions
All authors contributed to the study conception and design. Data collection and analysis were performed by IP, LMMJ, RMWAD, and LS. The first draft of the manuscript was written by IP, and all authors commented on all versions of the manuscript. All authors read and approved the final manuscript.
References
- 1.OECD, EU. Health at a glance: Europe 2018. 2018.
- 2.Doran CM, Kinchin I. A review of the economic impact of mental illness. Aust Health Rev. 2019;43(1):43–48. doi: 10.1071/AH16115. [DOI] [PubMed] [Google Scholar]
- 3.Vannieuwenborg L, Buntinx F, De Lepeleire J. Presenting prevalence and management of psychosocial problems in primary care in Flanders. Arch Public Health. 2015;73(1):1–6. doi: 10.1186/s13690-015-0061-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kessler RC, Amminger GP, Aguilar-Gaxiola S, Alonso J, Lee S, Ustun TB. Age of onset of mental disorders: a review of recent literature. Curr Opin Psychiatry. 2007;20(4):359. doi: 10.1097/YCO.0b013e32816ebc8c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Beecham J. Annual research review: child and adolescent mental health interventions: a review of progress in economic studies across different disorders. J Child Psychol Psychiatry. 2014;55(6):714–732. doi: 10.1111/jcpp.12216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Scott S, Knapp M, Henderson J, Maughan B. Financial cost of social exclusion: follow up study of antisocial children into adulthood. BMJ. 2001;323(7306):191. doi: 10.1136/bmj.323.7306.191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Currie J, Stabile M. Child mental health and human capital accumulation: the case of ADHD. J Health Econ. 2006;25(6):1094–1118. doi: 10.1016/j.jhealeco.2006.03.001. [DOI] [PubMed] [Google Scholar]
- 8.Hultkrantz L, Karpaty P, Vimefall E. Education-earnings linkage for assessing societal benefits of interventions for children and youth in Sweden. Psychosoc Interv. 2017;26(3):171–179. doi: 10.1016/j.psi.2017.06.001. [DOI] [Google Scholar]
- 9.Le HH, Hodgkins P, Postma MJ, Kahle J, Sikirica V, Setyawan J, et al. Economic impact of childhood/adolescent ADHD in a European setting: the Netherlands as a reference case. Eur Child Adolesc Psychiatry. 2014;23(7):587–598. doi: 10.1007/s00787-013-0477-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Järbrink K. The economic consequences of autistic spectrum disorder among children in a Swedish municipality. Autism. 2007;11(5):453–463. doi: 10.1177/1362361307079602. [DOI] [PubMed] [Google Scholar]
- 11.Ten JB. arguments for a societal perspective in the economic evaluation of medical innovations. Eur J Health Econ. 2009;10:357–359. doi: 10.1007/s10198-009-0173-2. [DOI] [PubMed] [Google Scholar]
- 12.Shearer J, McCrone P, Romeo R. Economic evaluation of mental health interventions: a guide to costing approaches. Pharmacoeconomics. 2016;34(7):651–664. doi: 10.1007/s40273-016-0390-3. [DOI] [PubMed] [Google Scholar]
- 13.Mayer S, Paulus AT, Łaszewska A, Simon J, Drost RM, Ruwaard D, et al. Health-related resource-use measurement instruments for intersectoral costs and benefits in the education and criminal justice sectors. Pharmacoeconomics. 2017;35(9):895–908. doi: 10.1007/s40273-017-0522-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Drost R, Paulus A, Ruwaard D, Evers S. Inter-sectoral costs and benefits of mental health prevention: towards a new classification scheme. J Ment Health Policy Econ. 2013;16(4):179–186. [PubMed] [Google Scholar]
- 15.Drost RM, Paulus AT, Ruwaard D, Evers SM. Valuing inter-sectoral costs and benefits of interventions in the healthcare sector: methods for obtaining unit prices. Expert Rev Pharmacoecon Outcomes Res. 2017;17(1):77–84. doi: 10.1586/14737167.2016.1141679. [DOI] [PubMed] [Google Scholar]
- 16.Pokhilenko I, Janssen LM, Evers SM, Drost RM, Simon J, König H-H, et al. Exploring the identification, validation, and categorization of costs and benefits of education in mental health: the PECUNIA project. Int J Technol Assess Health Care. 2020;36(4):325–331. doi: 10.1017/S0266462320000203. [DOI] [PubMed] [Google Scholar]
- 17.Drost RM, van der Putten IM, Ruwaard D, Evers SM, Paulus AT. Conceptualizations of the societal perspective within economic evaluations: a systematic review. Int J Technol Assess Health Care. 2017;33(2):251–260. doi: 10.1017/S0266462317000526. [DOI] [PubMed] [Google Scholar]
- 18.ISPOR. Pharmacoeconomic guidelines around the world. n.d. https://tools.ispor.org/peguidelines/. Accessed 12 Aug 2019.
- 19.van Mastrigt GA, Hiligsmann M, Arts JJ, Broos PH, Kleijnen J, Evers SM, et al. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: a five-step approach (part 1/3) Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):689–704. doi: 10.1080/14737167.2016.1246960. [DOI] [PubMed] [Google Scholar]
- 20.Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264–269. doi: 10.7326/0003-4819-151-4-200908180-00135. [DOI] [PubMed] [Google Scholar]
- 21.Tacconelli E. Systematic reviews: CRD's guidance for undertaking reviews in health care. Lancet Infect Dis. 2010;10(4):226. doi: 10.1016/S1473-3099(10)70065-7. [DOI] [Google Scholar]
- 22.England MJ, Butler AS, Gonzalez ML. Psychosocial interventions for mental and substance use disorders: a framework for establishing evidence-based standards. Washington, DC: National Academy Press; 2015. [PubMed] [Google Scholar]
- 23.Johns B, Baltussen R, Hutubessy R. Programme costs in the economic evaluation of health interventions. Cost Eff Resour Alloc. 2003;1(1):1. doi: 10.1186/1478-7547-1-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.WHO. International classification of diseases for mortality and morbidity statistics (11th Revision). 2018. https://icd.who.int/en. Accessed 7 Aug 2019.
- 25.Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc JMLA. 2016;104(3):240. doi: 10.3163/1536-5050.104.3.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Wijnen B, Van Mastrigt G, Redekop W, Majoie H, De Kinderen R, Evers S. How to prepare a systematic review of economic evaluations for informing evidence-based healthcare decisions: data extraction, risk of bias, and transferability (part 3/3) Expert Rev Pharmacoecon Outcomes Res. 2016;16(6):723–732. doi: 10.1080/14737167.2016.1246961. [DOI] [PubMed] [Google Scholar]
- 27.Inflation tool. 2020. https://www.inflationtool.com/. Accessed 16 Apr 2021.
- 28.Converter XC. https://www.xe.com/currencyconverter/. Accessed 16 Apr 2021.
- 29.Economic costs associated with mental retardation, cerebral palsy, hearing loss, and vision impairment—United States, 2003. MMWR Morb Mortal Wkly Rep. 2004;53(3):57–9. [PubMed]
- 30.Arora S, Goodall S, Viney R, Einfeld S. Societal cost of childhood intellectual disability in Australia. J Intellect Disabil Res. 2020;64(7):524–537. doi: 10.1111/jir.12732. [DOI] [PubMed] [Google Scholar]
- 31.Barrett B, Byford S, Sharac J, Hudry K, Leadbitter K, Temple K, et al. Service and wider societal costs of very young children with autism in the UK. J Autism Dev Disord. 2011 doi: 10.1007/s10803-011-1393-8. [DOI] [PubMed] [Google Scholar]
- 32.Beaulieu E, Rajabali F, Zheng A, Pike I. The lifetime costs of pediatric abusive head trauma and a cost-effectiveness analysis of the Period of Purple crying program in British Columbia, Canada. Child Abuse Negl. 2019;97:104133. doi: 10.1016/j.chiabu.2019.104133. [DOI] [PubMed] [Google Scholar]
- 33.Bodden DH, Dirksen CD, Bögels SM. Societal burden of clinically anxious youth referred for treatment: a cost-of-illness study. J Abnorm Child Psychol. 2008;36(4):487–497. doi: 10.1007/s10802-007-9194-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Bodden DH, Dirksen CD, Bogels SM, Nauta MH, De Haan E, Ringrose J, et al. Costs and cost-effectiveness of family CBT versus individual CBT in clinically anxious children. Clin Child Psychol Psychiatry. 2008;13(4):543–564. doi: 10.1177/1359104508090602. [DOI] [PubMed] [Google Scholar]
- 35.Bodden DHM, Stikkelbroek Y, Dirksen CD. Societal burden of adolescent depression, an overview and cost-of-illness study. J Affect Disord. 2018;241:256–262. doi: 10.1016/j.jad.2018.06.015. [DOI] [PubMed] [Google Scholar]
- 36.Buescher AV, Cidav Z, Knapp M, Mandell DS. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr. 2014;168(8):721–728. doi: 10.1001/jamapediatrics.2014.210. [DOI] [PubMed] [Google Scholar]
- 37.Byford S, Cary M, Barrett B, Aldred CR, Charman T, Howlin P, et al. Cost-effectiveness analysis of a communication-focused therapy for pre-school children with autism: results from a randomised controlled trial. BMC Psychiatry. 2015;15:316. doi: 10.1186/s12888-015-0700-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Creswell C, Violato M, Cruddace S, Gerry S, Murray L, Shafran R, et al. A randomised controlled trial of treatments of childhood anxiety disorder in the context of maternal anxiety disorder: clinical and cost-effectiveness outcomes. J Child Psychol Psychiatry. 2019 doi: 10.1111/jcpp.13089. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Day C, Briskman J, Crawford MJ, Foote L, Harris L, Boadu J, et al. An intervention for parents with severe personality difficulties whose children have mental health problems: a feasibility RCT. Health Technol Assess. 2020;24(14):1. doi: 10.3310/hta24140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Domino ME, Burns BJ, Silva SG, Kratochvil CJ, Vitiello B, Reinecke MA, et al. Cost-effectiveness of treatments for adolescent depression: results from TADS. Am J Psychiatry. 2008;165(5):588–596. doi: 10.1176/appi.ajp.2008.07101610. [DOI] [PubMed] [Google Scholar]
- 41.Drost RM, Paulus AT, Jander AF, Mercken L, de Vries H, Ruwaard D, et al. A web-based computer-tailored alcohol prevention program for adolescents: cost-effectiveness and intersectoral costs and benefits. J Med Int Res. 2016;18(4):e93. doi: 10.2196/jmir.5223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Ericson L, Magnusson L, Hovstadius B. Societal costs of fetal alcohol syndrome in Sweden. Eur J Health Econ. 2017;18(5):575–585. doi: 10.1007/s10198-016-0811-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Ganz ML. The lifetime distribution of the incremental societal costs of autism. Arch Pediatr Adolesc Med. 2007;161(4):343–349. doi: 10.1001/archpedi.161.4.343. [DOI] [PubMed] [Google Scholar]
- 44.Genereaux D, van Karnebeek CD, Birch PH. Costs of caring for children with an intellectual developmental disorder. Disabil Health J. 2015;8(4):646–651. doi: 10.1016/j.dhjo.2015.03.011. [DOI] [PubMed] [Google Scholar]
- 45.Greenmyer JR, Klug MG, Kambeitz C, Popova S, Burd L. A multicountry updated assessment of the economic impact of fetal alcohol spectrum disorder: costs for children and adults. J Addict Med. 2018;12(6):466–473. doi: 10.1097/ADM.0000000000000438. [DOI] [PubMed] [Google Scholar]
- 46.Hakkaart-van Roijen L, Zwirs BW, Bouwmans C, Tan SS, Schulpen TW, Vlasveld L, et al. Societal costs and quality of life of children suffering from attention deficient hyperactivity disorder (ADHD) Eur Child Adolesc Psychiatry. 2007;16(5):316–326. doi: 10.1007/s00787-007-0603-6. [DOI] [PubMed] [Google Scholar]
- 47.Järbrink K, Knapp M. The economic impact of autism in Britain. Autism. 2001;5(1):7–22. doi: 10.1177/1362361301005001002. [DOI] [PubMed] [Google Scholar]
- 48.Jolstedt M, Wahlund T, Lenhard F, Ljotsson B, Mataix-Cols D, Nord M, et al. Efficacy and cost-effectiveness of therapist-guided internet cognitive behavioural therapy for paediatric anxiety disorders: a single-centre, single-blind, randomised controlled trial. Lancet Child Adolesc Health. 2018;2(11):792–801. doi: 10.1016/s2352-4642(18)30275-x. [DOI] [PubMed] [Google Scholar]
- 49.Konnopka A, Konig HH. The health and economic consequences of moderate alcohol consumption in Germany 2002. Value Health. 2009;12(2):253–261. doi: 10.1111/j.1524-4733.2008.00444.x. [DOI] [PubMed] [Google Scholar]
- 50.Konnopka A, König HH. Direct and indirect costs attributable to alcohol consumption in Germany. Pharmacoeconomics. 2007;25(7):605–618. doi: 10.2165/00019053-200725070-00006. [DOI] [PubMed] [Google Scholar]
- 51.Kuklinski MR, Oxford ML, Spieker SJ, Lohr MJ, Fleming CB. Benefit-cost analysis of Promoting First Relationships®: implications of victim benefits assumptions for return on investment. Child Abuse Negl. 2020;106:104515. doi: 10.1016/j.chiabu.2020.104515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Leijdesdorff SMJ, Huijs CEM, Klaassen RMC, Popma A, van Amelsvoort T, Evers S. Burden of mental health problems: quality of life and cost-of-illness in youth consulting Dutch walk-in youth health centres. J Ment Health. 2020 doi: 10.1080/09638237.2020.1836555. [DOI] [PubMed] [Google Scholar]
- 53.Lenhard F, Ssegonja R, Andersson E, Feldman I, Ruck C, Mataix-Cols D, et al. Cost-effectiveness of therapist-guided internet-delivered cognitive behaviour therapy for paediatric obsessive-compulsive disorder: results from a randomised controlled trial. BMJ Open. 2017;7(5):e015246. doi: 10.1136/bmjopen-2016-015246. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Oosterhoff M, Over EAB, van Giessen A, Hoogenveen RT, Bosma H, van Schayck OCP, et al. Lifetime cost-effectiveness and equity impacts of the Healthy Primary School of the future initiative. BMC Public Health. 2020;20(1):1887. doi: 10.1186/s12889-020-09744-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Pelham WE, Foster EM, Robb JA. The economic impact of attention-deficit/hyperactivity disorder in children and adolescents. J Pediatr Psychol. 2007;32(6):711–727. doi: 10.1093/jpepsy/jsm022. [DOI] [PubMed] [Google Scholar]
- 56.Penner M, Rayar M, Bashir N, Roberts SW, Hancock-Howard RL, Coyte PC. Cost-effectiveness analysis comparing pre-diagnosis autism spectrum disorder (ASD)-targeted intervention with Ontario's Autism Intervention Program. J Autism Dev Disord. 2015;45(9):2833–2847. doi: 10.1007/s10803-015-2447-0. [DOI] [PubMed] [Google Scholar]
- 57.Peterson C, Florence C, Thomas R, Klevens J. Cost-benefit analysis of two child abuse and neglect primary prevention programs for US states. Prev Sci. 2018;19(6):705–715. doi: 10.1007/s11121-017-0819-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Piccininni C, Bisnaire L, Penner M. Cost-effectiveness of wait time reduction for intensive behavioral intervention services in Ontario, Canada. JAMA Pediatr. 2017;171(1):23–30. doi: 10.1001/jamapediatrics.2016.2695. [DOI] [PubMed] [Google Scholar]
- 59.Popova S, Lange S, Burd L, Rehm J. The economic burden of fetal alcohol spectrum disorder in Canada in 2013. Alcohol Alcohol. 2016;51(3):367–375. doi: 10.1093/alcalc/agv117. [DOI] [PubMed] [Google Scholar]
- 60.Quintero J, Ramos-Quiroga JA, Sebastian JS, Montanes F, Fernandez-Jaen A, Martinez-Raga J, et al. Health care and societal costs of the management of children and adolescents with attention-deficit/hyperactivity disorder in Spain: a descriptive analysis. BMC Psychiatry. 2018;18(1):40. doi: 10.1186/s12888-017-1581-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Reynolds AJ, Temple JA, White BA, Ou SR, Robertson DL. Age 26 cost-benefit analysis of the child-parent center early education program. Child Dev. 2011;82(1):379–404. doi: 10.1111/j.1467-8624.2010.01563.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Roddy A, O'Neill C. The economic costs and its predictors for childhood autism spectrum disorders in Ireland: how is the burden distributed? Autism. 2019;23(5):1106–1118. doi: 10.1177/1362361318801586. [DOI] [PubMed] [Google Scholar]
- 63.Sampaio F, Bonnert M, Olen O, Hedman E, Lalouni M, Lenhard F, et al. Cost-effectiveness of internet-delivered cognitive-behavioural therapy for adolescents with irritable bowel syndrome. BMJ Open. 2019;9(1):e023881. doi: 10.1136/bmjopen-2018-023881. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Schawo S, van der Kolk A, Bouwmans C, Annemans L, Postma M, Buitelaar J, et al. Probabilistic Markov model estimating cost effectiveness of methylphenidate osmotic-release oral system versus immediate-release methylphenidate in children and adolescents: which information is needed? Pharmacoeconomics. 2015;33(5):489–509. doi: 10.1007/s40273-015-0259-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Simon E, Dirksen C, Bögels S. An explorative cost-effectiveness analysis of school-based screening for child anxiety using a decision analytic model. Eur Child Adolesc Psychiatry. 2013;22(10):619–630. doi: 10.1007/s00787-013-0404-z. [DOI] [PubMed] [Google Scholar]
- 66.Simon E, Dirksen C, Bogels S, Bodden D. Cost-effectiveness of child-focused and parent-focused interventions in a child anxiety prevention program. J Anxiety Disord. 2012;26(2):287–296. doi: 10.1016/j.janxdis.2011.12.008. [DOI] [PubMed] [Google Scholar]
- 67.Thanh NX, Jonsson E. Costs of fetal alcohol spectrum disorder in Alberta, Canada. Can J Clin Pharmacol. 2009;16(1):e80–e90. [PubMed] [Google Scholar]
- 68.Thurston S, Paul L, Loney P, Ye C, Wong M, Browne G. Associations and costs of parental symptoms of psychiatric distress in a multi-diagnosis group of children with special needs. J Intellect Disabil Res. 2011;55(3):263–280. doi: 10.1111/j.1365-2788.2010.01356.x. [DOI] [PubMed] [Google Scholar]
- 69.van der Kolk A, Bouwmans CA, Schawo SJ, Buitelaar JK, van Agthoven M, Hakkaart-van Roijen L. Association between societal costs and treatment response in children and adolescents with ADHD and their parents. A cross-sectional study in the Netherlands. Springerplus. 2015;4:224. doi: 10.1186/s40064-015-0978-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.van der Schans J, Kotsopoulos N, Hoekstra PJ, Hak E, Postma MJ. Cost-effectiveness of extended-release methylphenidate in children and adolescents with attention-deficit/hyperactivity disorder sub-optimally treated with immediate release methylphenidate. PLoS ONE. 2015;10(5):e0127237. doi: 10.1371/journal.pone.0127237. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.van Steensel FJ, Dirksen CD, Bögels SM. A cost of illness study of children with high-functioning autism spectrum disorders and comorbid anxiety disorders as compared to clinically anxious and typically developing children. J Autism Dev Disord. 2013;43(12):2878–2890. doi: 10.1007/s10803-013-1835-6. [DOI] [PubMed] [Google Scholar]
- 72.Vermeulen KM, Jansen DE, Knorth EJ, Buskens E, Reijneveld SA. Cost-effectiveness of multisystemic therapy versus usual treatment for young people with antisocial problems. Crim Behav Ment Health. 2017;27(1):89–102. doi: 10.1002/cbm.1988. [DOI] [PubMed] [Google Scholar]
- 73.Wansink HJ, Drost RM, Paulus AT, Ruwaard D, Hosman CM, Janssens JM, et al. Cost-effectiveness of preventive case management for parents with a mental illness: a randomized controlled trial from three economic perspectives. BMC Health Serv Res. 2016;16(1):228. doi: 10.1186/s12913-016-1498-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Weimer DL, Moberg P, French F, Tanner-Smith EE, Finch AJ. Net benefits of recovery high schools: higher cost but increased sobriety and educational attainment. J Ment Health Policy Econ. 2019;22(3):109–120. [PMC free article] [PubMed] [Google Scholar]
- 75.Zhao X, Page TF, Altszuler AR, Pelham WE, 3rd, Kipp H, Gnagy EM, et al. Family burden of raising a child with ADHD. J Abnorm Child Psychol. 2019;47(8):1327–1338. doi: 10.1007/s10802-019-00518-5. [DOI] [PubMed] [Google Scholar]
- 76.Hakkaart-van Roijen L, Van der Linden N, Bouwmans C, Kanters T, Tan SS. Kostenhandleiding. Methodologie van kostenonderzoek en referentieprijzen voor economische evaluaties in de gezondheidszorg In opdracht van Zorginstituut Nederland Geactualiseerde versie; 2015.
- 77.Lee KM, McCarron CE, Bryan S, Coyle D, Krahn M, McCabe C. Guidelines for the economic evaluation of health technologies: Canada. 4th ed. Ottawa: CADTH; 2017.
- 78.HIQA . Guidelines for the Economic Evaluation of Health Technologies in Ireland 2010. Health Information and Quality Authority Publisher; 2010. [Google Scholar]
- 79.NICE. Guide to the methods of technology appraisal. http://www.nice.org.uk/about.nice/howwework/devnicetech/technologyappraisalprocessguides/guidetothemethodsoftechnologyappraisaljsp. 2008.
- 80.Salvador-Carulla L, Alvarez-Galvez J, Romero C, Gutiérrez-Colosía MR, Weber G, McDaid D, et al. Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: the eDESDE-LTC study. BMC Health Serv Res. 2013;13(1):218. doi: 10.1186/1472-6963-13-218. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Romero-López-Alberca C, Gutiérrez-Colosía MR, Salinas-Pérez JA, Almeda N, Furst M, Johnson S, et al. Standardised description of health and social care: a systematic review of use of the ESMS/DESDE (European Service Mapping Schedule/Description and Evaluation of Services and DirectoriEs) Eur Psychiatry. 2019;61:97–110. doi: 10.1016/j.eurpsy.2019.07.003. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.