National Healthcare Economic Evaluation Guidelines: A Cross-Country Comparison

Abstract

Background and Objectives

Globally, a number of countries have developed guidelines that describe the design and conduct of economic evaluations as part of health technology assessment (HTA) or pharmacoeconomic analysis for decision making. The current scoping review was undertaken with an objective to summarize the recommendations made on methods of economic evaluation by the national healthcare economic evaluation (HEE) guidelines.

Methodology

A comprehensive search was undertaken in the website repositories of the International Society for Pharmacoeconomic and Outcomes Research (ISPOR) and Guide to Economic Analysis and Research (GEAR), and websites of national HTA agencies and ministries of health of individual countries. All guidelines in the English language were included in this review. Data were extracted with respect to general and methodological characteristics, and a descriptive analysis of recommendations made across the countries was undertaken.

Results

Overall, our review included 31 national HEE guidelines, published between 1997 and August 2020. Nearly half (45%) of the guidelines targeted the evaluation of pharmaceuticals. The nature of the guidelines was either mandatory (31%), recommendatory (42%), or voluntary (16%). There was a substantial consensus among the guidelines on several key principles, including type of economic evaluation (cost-utility analysis), time horizon of the analysis (long enough), health outcome measure (quality-adjusted life-years) and use of sensitivity analyses. The recommendations on study perspective, comparator, discount rate and type of costs to be included (particularly the inclusion of indirect costs) varied widely.

Conclusion

Despite similarity in the overall processes, variation in several recommendations given by various national HEE guidelines was observed. This is perhaps unsurprising given the differences in the health systems and financing mechanisms, capacity of local researchers, and data availability. This review offers important lessons and a starting point for countries that are planning to develop their own HEE guidelines.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41669-020-00250-7.

Key Points for Decision Makers

This review provides important implications for low- and middle-income countries and can act as a starting point for these countries to develop their own national guidelines.

The recommendations on the study perspective and the costs to be included varied widely.

There is a lack of consensus on whether and how to account for indirect costs.

There is no consensus on how to calculate the rate for discounting.

Introduction

Increased demand for healthcare services as a result of demographic and epidemiological transition, and an accelerated introduction of new drugs and technologies has increased the pressure on healthcare budgets worldwide [1]. Given the finite nature of healthcare resources, it is essential to inform resource allocation decisions through evidence-based mechanisms such as health technology assessment (HTA) [2].

The World Health Organization passed a resolution for HTA in 2014, recognizing the importance of HTA in supporting countries to make cost-effective resource allocation decisions [3]. Many high-income countries, as well as an increasing number of middle-income countries, worldwide have established HTA systems to support priority-setting decisions [4–8]. In order for policy decisions based on HTA evidence to be applied fairly, it is important that a uniform and transparent process is in place to ensure that all HTA evaluations are carried out according to rigorous standards of best practices [9]. Since economic evaluation is one of the mainstays of HTA, several countries have developed guidelines that address the design and conduct of economic evaluations. These guidelines assist researchers on what methods to use while conducting economic evaluations as part of HTA [10].

In general, the national healthcare economic evaluation (HEE) guidelines have been developed as methodological guides for analysts for conducting and reporting economic evaluations [11, 12]. In addition, these HEE guidelines become important resources for users to evaluate the quality of any HTA evaluation. In some cases, these guidelines are mandatory and are issued by the government authorities to define formats for listing new drugs for reimbursement purposes [10–13]. In other cases, these guidelines may not be mandatory, however their use is highly encouraged to improve standardization and transparency in the economic evaluations undertaken [14]. Such guidelines may be either recommendatory or voluntary in nature [10, 13, 14].

In low- and middle-income countries (LMICs), the use of HTA in resource allocation planning and decision making is even more important considering the limited availability of healthcare resources. While an increase in the number of economic evaluations being undertaken in LMICs is witnessed, their quality remains questionable [15, 16]. Systematic reviews of economic evaluations highlight the lack of consistency in methodological and reporting standards in the absence of common guidelines [17, 18]. Researchers in LMICs often resort to international guidance documents such as the WHO Guide to Cost-Effectiveness Analysis and the International Decision Support Initiative’s (iDSI) reference case [19, 20]. However, indigenously developed HEE guidelines that account for local contextual factors and priorities are the gold standard that each country must work towards.

The objective of this paper was to review and compare the recommendations made in the national HEE guidelines for HTA or pharmacoeconomic analysis. To date, some attempts to describe and compare national HEE guidelines have been made, however the majority of these reviews focused on guidelines developed in European countries [13, 21, 22]. Moreover, in light of methodological advancements occurring in the field of health economics in recent years, several of these guidelines have also been updated. With growing interest internationally in the use of HTA, new guidelines have been developed in Asian and African nations that have not been captured in earlier reviews [23, 24]. We undertook this review to provide a comparative review of the recommendations made by the national HEE guidelines. This would be a useful starting point for countries that are beginning to form their own national HEE for HTA.

Methodology

Literature Search

A comprehensive search was performed to identify national HEE guidelines published until 31 August 2020. All forms of guidelines or recommendations for undertaking HEE (for reimbursement and methodological purposes) were considered, e.g. HTA guidelines, pharmacoeconomic guidelines or economic evaluation guides. We first searched two website repositories that included international economic evaluation information: International Society for Pharmacoeconomic and Outcomes Research (ISPOR) and Guide to Economic Analysis and Research (GEAR) [10, 25]. These repositories were considered as they are the most comprehensive databases on HEE guidelines, which identify guidelines by undertaking online searches of databases such as Medline and Google Scholar [26]. Furthermore, these repositories are updated at regular intervals (approximately 6 months) by experts in the field of health economics [10, 26]. In addition, the websites of national HTA agencies and ministries of health of individual countries were also searched. A list of countries to be searched was compiled, referring to the member lists of international HTA organizations, including the International Network of Agencies for Health Technology Assessment (INAHTA; 33 countries), European Network for Health Technology Assessment (EUnetHTA; 29 countries) and Health Technology Assessment Agencies in Asia (HTAsialink; 15 countries) [27–29]. All guidelines in the English language were included in the review. For country guidelines that had multiple versions, the most recent version was retrieved and reviewed.

We excluded quality assurance and reporting guidelines such as the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist, Quality of Health Economic Studies (QHES) and Drummond’s checklist as these primarily provide recommendations on the reporting of HEEs rather than technical recommendations on the conduct of economic evaluations [30–32]. Guidelines that solely provided recommendations related to a subsection of economic evaluations, such as costing, modelling or budget impact analysis (BIA), were not considered in this review.

Data Extraction and Synthesis

Data were extracted using a template (electronic supplementary sheet 1) that had two components; first, general guideline characteristics (year of publication, type of guideline, purpose of the guideline, nature of the guideline, agency responsible for its creation, etc.); and second, methodological characteristics pertaining to the principles of economic evaluation (comparator, study perspective, target population, analytic technique, preferred costs to be included, measure for health outcome, study time horizon, discount rate, modelling, uncertainty analysis, equity analysis and BIA).

A descriptive analysis of recommendations across the national HEE guidelines was undertaken. Based on the nature of the national HEE guidelines, i.e. mandatory, recommendatory (use highly recommended) or voluntary (no obligation), the countries were divided into three groups and the recommendations were compared.

Results

In total, 47 national HEE guidelines were identified [33–79]. While Iran and Switzerland reportedly have specific guidelines, complete versions of these are not available online, hence they could not be included [70, 79]. Twelve guidelines were excluded as these were available in languages other than English [65–69, 71–77]. This resulted in a total of 33 guidelines for inclusion in the review. Another guideline (regions of Spain—Catalonia) was excluded because it was regional in nature and did not apply to the entire country [78]. Furthermore, guidelines for Bhutan did not provide any methodological details. These were general guidelines for the organizational set-up of the country’s HTA wing, thus Bhutan was also excluded [64]. Finally, a set of 31 national HEE guidelines, published between 1997 and August 2020 were included in the review (Fig. 1) [33–63].

Fig. 1.

PRISMA flowchart of included national healthcare economic evaluation guidelines

General Characteristics

As per the World Bank classification of countries, 26 of 31 national HEE guidelines belonged to high-income countries [33–39, 41–45, 47–50, 52–57, 59, 60, 62, 63], while five were from middle-income countries [40, 46, 51, 58, 61]. Until August 2020, none of the low-income countries had developed an HEE guideline. Furthermore, more than half (n = 18) of the national HEE guidelines were from European countries [34–36, 38, 39, 41–45, 47, 49, 53–56, 58, 62].

Year of Publication

The first ever national HEE guideline identified in this review was produced in Australia in 1992, followed by Canada in 1994 [80, 81]. These two countries have since produced the fifth and fourth version of their guidelines, respectively [33, 37]. The majority of the national HEE guidelines included in this review (n = 23, 74%) were published after 2010 (Fig. 2) [33, 35, 37, 38, 40–47, 49–54, 56–58, 61–63]. Of these, five were new guidelines introduced for the first time, whereas the remainder were updated versions of previously published guidelines [38, 40, 46, 57, 58]. In 2019 alone, five countries had released revised versions of their national HEE guidelines [42, 47, 50, 51, 56].

Fig. 2.

Timeline of the publication of national healthcare economic evaluation guidelines

Nature of the Guidelines

While 14 (45%) of the national HEE guidelines targeted the evaluation of pharmaceuticals, the remaining (n = 17, 55%) catered to a broader mix of healthcare technologies [33, 35, 39, 40, 48, 49, 51–53, 55, 56, 58, 60]. Only 10 national HEE guidelines (32%) were mandatory in nature [33, 36, 42–44, 50, 53, 55, 61, 62], while others were either recommendatory (n = 13, 42%) [34, 37, 38, 40, 41, 45, 47, 49, 51, 52, 54, 56, 60] or voluntary (n = 5, 16%) [39, 57–59, 63]. Three national HEE guidelines (Baltic States, Indonesia and Israel) did not explicitly specify serving as either purpose [35, 46, 48].

Summary of Recommendations

A summary of findings from the review, according to the principles of economic evaluation, is presented in Table 1. All the national HEE guidelines recognized and provided recommendations on the basic principles of economic evaluation but differed in the extent to which details were provided. There was also considerable variation in the methods to be followed.

Table 1.

Recommendations made by national healthcare economic evaluation guidelines on principles of economic evaluation

Characteristic	Recommendation	N (%)	Mandatory guidelines	Recommendatory guidelines	Voluntary guidelines
Analytic technique	Primarily/preferably CUA	15 (49)	Australia*, Finland*, Japan, Norway, Portugal*, Thailand, The Netherlands	Canada, Croatia, England and Wales, Ireland, New Zealand, Scotland	Indonesia, Israel
	CUA or CEA	10 (32)	Belgium, France, Germany	Hungary, Italy, Malaysia, Poland, Taiwan	Denmark, US
	Any technique (justify)	6 (19)	–	Austria, Egypt	South Africa, Spain, Singapore, Baltic
Perspective	Payer	15 (48)	Australia, Finland, Francea, Germany, Japan	Canada, Hungary, Malaysia, New Zealand, Poland	Baltic, Israel, Singapore, South Africa, US
	Payer (costs), Patients/Societal (outcomes)	5 (16)	Belgium	Croatia, England and Wales, Ireland, Scotland
	Societal	8 (26)	Norway, Portugal, Thailand, The Netherlands	Taiwan	Denmark, Indonesia, Spain
	Any (justified)	3 (10)	–	Austria, Egypt, Italy
Comparator	Most Commonly used/Standard therapy	17 (55)	Portugal, Thailand, The Netherlands	Canada, Croatia, Egypt, Ireland, Italy, Austria, Hungary, England and Wales	Baltic, Indonesia Israel, South Africa, Spain, US
	Most likely to be replaced	9 (29)	Australia, Japan, Norway	Malaysia, New Zealand, Poland, Scotland, Taiwan	Singapore
	All relevant alternatives	3 (10)	Belgium, Germany, France, Norway, Thailand	Canada, England and Wales	–
	Not specified	2 (6)	Finland (most appropriate)	–	Denmark
Costs	Direct medical costs	14 (45)	Australia, Belgium	Canada, Croatia, Egypt, England and Wales, Ireland, Malaysia, New Zealand, Scotland	Baltic, Israel, South Africa, Singapore
	Direct costs (medical and nonmedical)	6 (20)	France, Finland, Germany, Japan, Norway	Hungary	-
	Direct and Indirect costs	9 (29)	Portugal (also intangible costs), Thailand, The Netherlands	Poland, Italy, Taiwan	Indonesia, Spain, Denmark (also intangible costs)
	All relevant costs	2 (6)	–	Austria	US
Outcomes	Primarily QALY	12 (39)	Australia, Finland, japan, Norway, Thailand, The Netherlands	Canada, England and Wales, Ireland, New Zealand, Scotland	Israel
	QALY and natural units	13 (42)	Germany	Croatia, Egypt, Austria, Hungary, Italy, Malaysia	Baltic, Denmark, Singapore, Spain, US
	QALY, Life years	6 (19)	Belgium, France, Portugal	Taiwan, Poland	Indonesia (also DALY), South Africa
Uncertainty analysis	Both PSA and DSA should be done	10 (33)	Belgium, Germany, Finland, Japan, The Netherlands	Hungary, Poland, New Zealand	Indonesia, South Africa, US
	PSA preferred	8 (26)	France, Norway Thailand	England and Wales, Canada, Croatia, Ireland, Malaysia
	DSA mandatory, PSA optional	7 (23)	Australia	Egypt, Italy, Scotland	Singapore, Spain
	DSA only	2 (6)	–	Austria, Taiwan
	Not specified	4 (12)	Portugal	–	Baltic, Denmark, Israel

CEA cost-effectiveness analysis, CUA cost-utility analysis, Baltic states Latvia, Lithuania, Estonia, DALY disability-adjusted life-years, DSA deterministic sensitivity analysis, PSA probabilistic sensitivity analysis, QALY quality-adjusted life-year, US United States

^aFrance: collective perspective *CUA is preferred over other types of economic analysis

(1) Type of Economic Analysis

While all mandatory and most of the recommendatory (85%) guidelines clearly prescribed the analytical technique as either a cost-utility analysis (CUA) or cost-effectiveness analysis (CEA), nearly half of the voluntary guidelines (50%) were open-ended and stated that any method of economic evaluation may be chosen (Table 1). Cost-benefit analysis (CBA) was the least recommended method and was usually recommended only as an additional analysis in cases where monetary effects of the interventions being compared were considered to be important [33, 34, 37, 43]. The majority (68%) of the national HEE guidelines also acknowledged the fact that in special scenarios where the effectiveness of the competing technologies did not differ, the use of cost-minimization analysis (CMA) will be considered acceptable [33–36, 40, 42, 43, 45–47, 50–60]; however, a few guidelines (32%) did not explicitly mention CMA as an acceptable technique [37–39, 41, 44, 48, 49, 61–63]. Furthermore, guidelines for Thailand and Canada identified CMA as a costing exercise and not a full economic evaluation, with the latter specifically recommending against its use [37].

(2) Perspective

Fifteen (48%) national HEE guidelines recommended using the payers’ perspective, with varying terms to describe this [33, 35, 37, 42–45, 48, 50–52, 54, 57, 58, 63], while four national HEE guidelines specified it as healthcare system perspective [33, 35, 48, 57]. Other terms used to describe perspective included publicly funded healthcare payer [35, 48], statutory health insurance (SHI) [44], third-party payer [58], funder [52] and healthcare decision maker [63]. Eight national HEE guidelines (26%) recommended a societal perspective for primary analysis [39, 46, 53, 55, 59–62], while 10 countries (26%) recommended a societal perspective for an additional analysis if required [33, 35, 37, 38, 44, 45, 47, 51, 54, 63]. Three countries (10%) stated that any perspective relevant to the research question may be considered [34, 40, 49]. Twenty-six national HEE guidelines recommended using a common perspective for both costs and outcomes, however the remaining five (16%) guidelines stated that a different perspective should be used for costs (payer perspective) and outcomes (either societal or patients and individuals whose health is affected) (Table 1) [36, 38, 41, 47, 56].

Nevertheless, even within the societal perspective, variation in the recommendation on type of costs to be included was observed. Several guidelines recommend including all costs and outcomes within and outside the health system, including other sectors such as education, housing, judicial system, etc. [33, 35, 37–39, 43–45, 47, 62]. Other guidelines recommended including all healthcare-related costs irrespective of the payer, including direct medical, direct non-medical and indirect costs [46, 51, 54, 59–61]. The guidelines for Portugal recommended that in addition to the direct medical, non-medical and indirect costs, intangible costs should also be included under societal perspective [55]. Furthermore, the Norwegian guidelines recommended using a societal perspective but with limitations, where the inclusion of indirect costs due to productivity losses was optional [53].

(3) Target Population

A large number of national HEE guidelines recommend that the target population is the one that is most likely to receive the proposed intervention in clinical practice (approved or potential users) and should be clearly described [33, 36–38, 41, 42, 44–57, 62]. The guidelines for France state that the target population consists of all individuals whose health is directly or indirectly affected by the intervention [43], while the Baltic guidelines recommend that the analysis should be performed on the entire population included in the clinical trial [35]. Other remaining guidelines do not identify specific features of the target population but state that the target population should be clearly defined, with details on demographic and disease-specific characteristics [34, 46, 47, 58–61, 63].

(4) Choice of Comparator

The majority of national HEE guidelines (17, 55%) across the three groups recommended that the comparator chosen in the economic evaluation should be the one that is most commonly used, either as routine practice/current care or in accordance with the standard treatment guidelines [34, 35, 37, 38, 40, 41, 45–49, 55, 58, 59, 61–63]. The second most frequently recommended comparator (9, 29%) was the alternative that was most likely to be replaced with the introduction of the new alternative (Table 1) [33, 50–54, 56, 57, 60]. However, some countries such as France and Germany do not identify any specific alternative as the comparator but recommend that all alternatives that compete with the intervention should be used as comparators [43, 44]. Similarly, Canada, England, Norway and Thailand also recommend that all appropriate comparators should be identified [37, 41, 53, 61], while Belgium recommends identifying the relevant comparator from all available alternatives using an efficiency frontier [36]. In addition to the primary comparator, certain guidelines also recommend using additional comparators that include the cheapest, most efficacious, or current best practices [38, 54, 55, 61].

(5) Time Horizon

In general, most of the national HEE guidelines recommended that the time horizon should be conceptually based on the natural course of the disease and the anticipated effects of the intervention. Several national HEE guidelines advocated for the use of lifetime horizon, especially when there is a long-term impact on a patient’s health, such as in the case of chronic diseases [33, 35, 37, 41, 43, 47, 52, 53, 57, 61, 62]. The guidelines for Germany stated that the time horizon should represent at least the duration of the randomized controlled trial evidence that informs the analysis, and should preferably be extended to a lifetime for chronic diseases [44].

(6) Discount Rate

National HEE guidelines of all countries except Belgium, Poland and The Netherlands recommended that a common rate should be used for discounting both costs and outcomes [33–35, 37–53, 55–61, 63]. Figure 3 shows that the discount rate recommended varied across countries, with a discount rate of 3% (10 guidelines) being the most commonly suggested rate, followed by 5% (7 guidelines). In addition, most countries (28, 90%) also recommended sensitivity analyses around the discount rate, including no discounting [31, 32, 34–36, 38–45, 47–50, 52–61, 63]. Guidelines for Belgium, Poland and The Netherlands recommended using a lower discount rate for outcomes (1.5%, 1.5% and 3.5%, respectively) compared with costs (3%, 4% and 5%, respectively) [36, 54, 62]. Thailand initially recommended using a common discount rate (3%), however if the time horizon was more than 30 years, discounting at a rate of 4% for costs and 2% for consequences was recommended [61]. Spain and Taiwan also recommended differential discounting for costs and outcomes, however this was recommended as part of sensitivity analyses and not for base-case analyses [59, 60]. Guidelines for France recommended using a lower discount rate for both costs and outcomes if the time horizon was longer than 30 years (2%) [43].

Fig. 3.

Discount rates recommended by national healthcare economic evaluation guidelines. SA South Africa

The basis for the selected discount rate was usually not justified; where reported, it was either cited as being based on long-term bond rates or merely for the purpose of consistency with the existing recommendation (Table 2).

Table 2.

Justifications of the rate chosen for discounting costs and outcomes

Country	Discount rate	Justifications
Australia	5% for costs and outcomes	–
Austria	5% for costs and outcomes	–
Baltic	5% for costs and outcomes	–
Belgium	3% for costs and 1.5% for outcomes	The choice of discount rate for costs is based on the return on risk-free government bonds, currently around 3%, in Belgium. The choice of discount rate for outcomes is based on the expected change in the value of health over time and the expected relative changes in budgets and productivity over time
Canada	1.5% for costs and outcomes	Recent empirical evidence on the long-term cost of borrowing for Canadian provinces. The discount rate is expressed in real (i.e. constant, inflation-adjusted) terms, which is consistent with valuing resources in real dollars. Nominal provincial bond rates were adjusted for inflation using the Bank of Canada’s target inflation rate (currently 2% per year), and a weighted average of the real provincial bond rates was calculated based on the relative proportion of the population represented by each province
Croatia	5% for costs and outcomes	Calculated mean of base rate for four quarters within the respective year, over the last 3 years
Denmark	Not mentioned	–
Egypt	3.5% for costs and outcomes	–
England and Wales	3.5% for costs and outcomes	Recommendations of the UK Treasury for the discounting of costs. Based on the social rate of time preference and catastrophic risk rate of 1%, expected income growth rate of 2%, pure time preference rate of 0.5%
Finland	3% for costs and outcomes	No justification
France	4% for costs and outcomes up to 30 years and reduction of 2% thereafter	This social discount rate reflects the ‘value of time’ to society and has been set at 4% since 2005. The discount rate depends on a pure preference rate for the present, an elasticity of marginal utility of consumption, and a growth rate of per capita consumption
Germany	3% for costs and outcomes	Consistency
Hungary	3.7% for costs and outcomes	Calculations based on domestic empirical data using the Ramsey equation
Indonesia	3% for costs and outcomes	–
Ireland	4% for costs and outcomes	Rate set by the Department of Finance
Israel	3% for costs and outcomes	–
Italy	3% for costs and outcomes	3%, is more appropriate for comparing the different existing studies
Japan	2% for costs and outcomes	–
Malaysia	3% for costs and outcomes	–
New Zealand	3.5% for costs and outcomes	Social rate of time preference calculated based on the risk-free, long-term government bond rate, adjusted for inflation
Norway	4% for costs and outcomes	Ministry of Finance for public projects with moderate systematic risk. The discount rate is a real interest rate so that prices in the pharmacoeconomic analysis must not be adjusted for inflation
Poland	5% for costs and 3.5% for outcomes	–
Portugal	5% for costs and outcomes	Based on the real long-term market interest rate (4–5% in recent years). The choice of 5% was influenced by the fact that it is the one used in most of the countries
Scotland	3.5% for costs and outcomes	–
Singapore	3% for costs and outcomes	Based on the return on risk-free government bonds, which are currently about 3% in Singapore
South Africa	5% for costs and outcomes	–
Spain	3% for costs and outcomes	–
Taiwan	5% for costs and outcomes	Long-term market interest rate, which is approximately 4–5% in Taiwan
Thailand	3% for costs and outcomes; time horizon >30 years 4% for costs and 2% for outcomes	–
The Netherlands	4% for costs and 1.5% for outcomes	–
United States	3% for costs and outcomes	US Panel recommendations

(7) Costs

The majority (6, 60%) of the mandatory national HEE guidelines recommended the inclusion of both direct medical and non-medical costs, while the majority of the recommendatory (9, 69%) and voluntary (4, 50%) national HEE guidelines suggested the inclusion of direct medical costs only (Table 1). The recommendations on the type of costs to be included were in line with the recommendations on the study perspective. Guidelines recommending a healthcare system/healthcare payer/funder perspective recommended including only direct medical costs [33, 35–38, 40, 41, 45, 47, 48, 51, 52, 57, 58], while guidelines recommending a ‘payer’ perspective recommended the inclusion of both direct medical and non-medical costs [42–45, 50]. The inclusion of indirect costs was suggested by those HEE guidelines that recommended using a societal perspective for the primary analysis [34, 39, 46, 49, 55, 59–62], or as a secondary analysis with the results presented separately [33, 36–38, 40, 42–44, 47, 50, 51, 54, 57]. Other guidelines that strictly forbade the inclusion of indirect costs were in accordance with the recommendations on study perspective [41, 45, 52, 58]. Moreover, recommendations on the preferred methodology to estimate indirect costs also varied. While some national HEE guidelines recommended using a human capital approach (HCA) [34, 49, 50, 60, 61], others recommended the use of a friction cost approach (FCA) [37, 39, 44, 54, 62]. Additionally, Portugal and Denmark recommended the inclusion of intangible costs under the societal perspective [39, 55].

Furthermore, in addition to the type of costs, recommendations also varied depending on who bore the cost. While Australia and Singapore recommended the inclusion of all direct medical costs borne by the patient as well as the payer (public or private), Germany on the other hand recommended that only those direct costs that will be reimbursed by SHI should be included [33, 44, 57].

Regarding the cost and price source, the national HEE guidelines usually provided a general recommendation stating that national cost data and price lists should be used. The guidelines acknowledged that it is difficult to transfer cost data from other settings, hence emphasis should be given to the use of local cost data.

(8) Measuring Health Outcomes

All mandatory national HEE guidelines recommended that quality-adjusted life-years (QALYs) should be used as health outcome measures. Recommendatory national HEE guidelines did not explicitly advocate for the use of any particular outcome measure and argued that the evaluator may choose either natural units (clinical endpoints) or QALYs, depending on the decision problem and the analytic technique being followed. This was also the case for the majority (5, 63%) of the voluntary national HEE guidelines (Table 1). Guidance was also provided on the methods to be used for the valuation of utilities. Quality-of-life valuation using generic health instruments was most widely recommended (24, 77%) [35–43, 45–47, 50–57, 59–62]. Of these, nearly three-quarters recommended using the EQ-5D tool [33, 35–38, 40, 41, 43, 45–47, 49–54, 56, 57, 60–63], while other suggested tools included the Short Form–6 dimensions (SF-6D), Health Utilities Index (HUI) and Quality of Well Being (QWB) questionnaires (Table 3). Furthermore, there was consensus among the national HEE guidelines that a nationally representative tariff value-set should be used as much as possible. Disability-adjusted life-years (DALYs) were recommended by the Indonesian guideline only as a health outcome measure [46].

Table 3.

Recommended health-related quality-of-life instruments by national guidelines

Country	Recommended health-related quality-of-life instruments	Whether EQ-5D is preferred
Australia	HUI2, HUI3, EQ-5D-3L, EQ-5D-5L, SF-6D, AQoL, and CHU-9D	–
Austria	–	–
Baltic	EQ-5D and HUI
Belgium	EQ-5D (for adults), EQ-5D-Y (for youngsters), or SF-6D	Yes
Canada	EQ-5D, HUI, SF-6D	–
Croatia	EQ-5D	Yes
Denmark	–	–
Egypt	EQ-5D, SF-6D or similar generic measures	–
England	EQ-5D	Yes
Finland	–	–
France	EQ-5D, HUI3	–
Germany	–	–
Hungary	General (EQ-5D, SF-36) or disease-specific instruments	–
Indonesia	EQ-5D, HUI, QWB, or WHO-QoL	Yes
Ireland	EQ-5D or SF-6D	–
Israel	–	–
Italy	If possible, the use of both a specific and generic instrument (SF-36, HUI, EQ-5D) is suggested	–
Japan	EQ-5D, SF-6D, HUI	–
Malaysia	EQ-5D	Yes
New Zealand	EQ-5D, other instruments may be used subject to justifications	Yes
Norway	EQ-5D, SF-6D, 15D	–
Poland	Preference based instruments with validated questionnaires in Polish (EQ-5D)	Yes
Portugal	Generic (SF-36, Sickness Impact Profile or Nottingham Health Profile) and specific instruments at the same time	–
Scotland	EQ-5D	Yes
Singapore	EQ-5D, SF-36, HUI3, AQoL	Yes
South Africa	–	–
Spain	–	–
Taiwan	SF-36, WHO-QoL, QWB, EQ-5D, HUI	–
Thailand	EQ-5D, HUI, SF-6D, QWB	Yes
Netherlands	Generic instruments, preferably EQ-5D	Yes
United States	EQ-5D, HUI, SF-6D, or QWB	–

AQoL Assessment of Quality of Life, CHU-9D Child Health Utility-9 dimensions, EQ-5D EuroQol 5-Dimensions, EQ-5D-3L EQ-5D 3-level, EQ-5D-5L EQ-5D 5-level, HUI Health Utilities Index, SF-6D Short Form-6 dimensions, SF-36 36-item Short Form, QWB Quality of Well Being, WHO-QoL World Health Organization Quality of Life, 15D 15-dimensional

(9) Modelling

All national HEE guidelines acknowledged that modelling techniques may be applied, especially in scenarios where data from clinical trials were required to be extrapolated to longer time frames [33–63]. The extent to which guidance was provided for undertaking model-based economic evaluations varied, however there was consistency between the broad recommendations. Many national HEE guidelines suggested following guidelines for good modelling practices, such as those developed by the ISPOR taskforce [33, 36–38, 40, 41, 44, 47, 51, 53, 57, 61–63]. The majority of the guidelines recommend that the model structure should be consistent with the nature of the disease and the clinical pathway of the health condition. The guidelines suggest that any model type may be chosen, however the structure should be as simple as possible; a few guidelines preferred decision tree and Markov models [43, 46, 51–53, 55, 60]. Furthermore, the guidelines suggest that model assumptions should be duly justified and the data inputs should be clearly reported with sources. Additionally, many guidelines recommend that model validity (external, internal and cross-validity) should also be documented [33, 36, 37, 40, 41, 43, 44, 47, 51, 53, 54, 56–58, 60–63].

(10) Uncertainty Analysis

All national HEE guidelines included in this review recommended undertaking some form of sensitivity analysis to explore the impact of uncertainty (Table 1). While a few guidelines (Baltic states, Denmark, Israel and Portugal) merely stated that uncertainties should be addressed appropriately, the majority of the remaining guidelines recommended identifying the type of uncertainty (parameter, structural or methodological) and to specify the methods to address these. Deterministic sensitivity analyses (DSA), both univariate and multivariate, are recommended for addressing uncertainties (Table 1). In addition to DSA, the majority (n = 25, 81%) of the national HEE guidelines also recommended using probabilistic sensitivity analysis (PSA) [33, 36–38, 40–47, 49–54, 56–59, 61–63]. While some of these guidelines specifically preferred PSA [37, 38, 41, 43, 47, 51, 53, 61], others suggested that PSA should be undertaken, although they did not make it mandatory [33, 40, 49, 56, 57, 59].

In addition to PSA and DSA, scenario-based analyses, including best-case and worst-case scenarios, were also recommended by many guidelines [37, 40, 41, 43, 44, 47, 50, 51, 53, 56, 58, 62, 63].

(11) Equity Issues

While some (10, 32%) of the national HEE guidelines mentioned that equity implications of the technology are important and should be considered [33, 37, 38, 40, 41, 47, 56, 60, 61], the majority (21, 68%) of the national HEE guidelines did not explicitly discuss how equity should be addressed [34–36, 39, 42–46, 48–55, 57–59, 62, 63]. The national HEE guidelines that discussed equity recommended that equal weighting of QALYs, irrespective of age, sex and socioeconomic status, will be preferred in the base-case analysis [33, 37, 38, 40, 41, 47, 56, 60, 61].

(12) Budget Impact Analysis

Several national HEE guidelines propose BIA be undertaken alongside CEA [33, 36, 44–46, 48, 49, 51–54, 56, 57, 60–63]. A subsection specifying technical requirements on how to undertake BIA is included as part of the main guideline of these countries. In general, the national HEE guidelines recommended using the perspective of the budget holder and the current mixed treatment scenario as the comparator. The time horizon recommended was generally short term, ranging between 1 and 6 years, with 3 years being the most common time horizon. Most of the national HEE guidelines opined that discounting should not be done while undertaking BIA, however New Zealand recommended using a higher rate of 8% against the 3.5% recommended for CEA [52].

(13) Cost-Effectiveness Thresholds

The majority (77%) of the national guidelines do not recommend any specific threshold value for deciding whether or not an intervention is cost effective [33–40, 42–44, 48–50, 52–60, 62], and state that multiple factors beyond cost effectiveness are important to guide reimbursement and investment decisions [36–38, 43, 44, 52]. A few guidelines recommend that the incremental cost-effectiveness ratios may be compared with gross domestic product (GDP)-based thresholds, i.e. Hungary (three times the GDP per capita) [45], Indonesia (GDP per capita adjusted for purchasing power parity, Rp43,000,000) [46], Malaysia (GDP per capita, RM42,000) [51] and Ireland (£20,000–£45,000) [47]; however, these may only be used to present the results of economic evaluations, not for decision making. Furthermore, countries such as England and Thailand suggest empirically calculated thresholds of £20,000–£30,000/QALY gained and ฿160,000/QALY gained, respectively [41, 61].

Discussion

This review was undertaken with the objective of reviewing national HEE guidelines developed by different countries in order to compare and contrast the recommendations regarding methodological principles of practice. This review provides the first comprehensive account of national HEE guidelines globally, as well as the similarities and differences in practice among these worldwide.

In the past two decades, we have witnessed a steady increase in the conduct of economic evaluations to inform health policy decisions [15]; however, fair policy decisions based on economic evidence require analysts to adhere to a uniform and transparent standard of practice [20]. National HEE guidelines provide clear set rules that govern practice in this respect and are useful, both for the analysts to generate economic evidence and for decision makers to aid in rational decision making.

The development of national HEE guidelines originated in the early 1990s, when Australia developed guidelines for pharmacoeconomic submissions, with Canada and other European countries following shortly after [80, 81]. The development of guidelines is dependent on multiple contextual factors, supplemented by the economic and political environment of a country. Furthermore, each country varies in its health system design and performance. With the mechanism of financing being largely insurance-based in the European region, the government provides for most drugs and technologies [82]. As a result, the guidelines in these countries were developed with the objective of assisting in the reimbursement decision-making process [21].

The use of economic evidence for decision making is slowly gaining impetus in the Asian and African regions [83]; however, these countries are at an uneven pace of developing guidelines. For instance, Thailand, which has been a forerunner in the use of economic evidence, has integrated HTA into policy making, including its use for the development of health benefit packages and determining pharmaceutical reimbursement lists [84]. It formulated its first version of the national guideline in 2008, which was updated in 2014 [85]. More recently, four Asian and two African nations have formulated national HEE guidelines, with the most recent developed by Singapore in 2018 [57]. A common phenomenon documented in the development of these guidelines in Asian and African countries was that they were formulated after a detailed review of existing guidelines worldwide [51, 85, 86]. In this context, our paper presents a useful resource for countries that are aiming to develop national guidelines in the future, and the methodological principles for health economic evaluations.

We observed that the national HEE guidelines from Europe and other high-income countries were by far more comparable with each other because of the considerably similar healthcare systems. The reliance on out-of-pocket expenditure is low and the insurance cover is almost universal [87]. Moreover, these countries have developed standard treatment guidelines and drug reimbursement systems. The national HEE guidelines of these countries provided in-depth guidance on most of the principles. With methods for economic evaluation being widely developed, and with the availability of reliable data sources, these guidelines give clear recommendations. On the other hand, national HEE guidelines from middle-income countries were more flexible and allowed use of alternate approaches depending on the data availability and researcher capacity.

Nevertheless, it was observed that there is substantial consensus among the national HEE guidelines on several key areas, including type of analytic technique, study time horizon and sensitivity analysis. Although the national HEE guidelines provided straightforward recommendations on the aforementioned principles, there were certain issues where there was a deficit of clear-cut guidance. The recommendations on study perspective and the type of costs to be included (particularly the inclusion of indirect costs) varied widely. Several countries did not specify which perspective should be followed and left it to the wisdom of the analyst to choose the perspective (subject to justifications) [34, 40, 41, 49]. Furthermore, even when recommending the same perspective, for instance the payers’ perspective, a variation was observed because different guidelines defined ‘payer’ differently. While some countries considered the healthcare system or the government as the payer [37, 38, 41, 44, 47, 48, 51, 52, 54, 56], others included both the government as well as the individual whose health is affected as the payer [33, 36, 42, 43, 50, 57]. This variance further reflected upon the cost categories to be included in a given perspective. The latter group of countries recommended the inclusion of out-of-pocket expenses, in addition to healthcare costs, from the payers’ perspective, while the former limit costs to direct medical costs borne by the healthcare system/insurer, to be included in the payers’ perspective.

Similarly, in relation to the recommendation to take the societal perspective, some countries recommended including only direct medical and non-medical costs, while others suggested indirect costs should also be included. Barring Denmark and Portugal, there was agreement on the exclusion of intangible costs from a societal perspective, as it is considered these costs have already been valued in outcomes such as quality of life [39, 55]; however, the inclusion of indirect costs from a societal perspective remain controversial.

Furthermore, the methodology to determine indirect costs was ambiguous and lacked consensus. While countries such as Austria, Japan, Taiwan and Thailand recommended the use of HCA [34, 50, 60, 61], other countries favoured the use of FCA [37, 39, 44, 54, 62]. The former argued that HCA was usually better in scenarios where there was short-term absenteeism, while the latter stated that HCA overestimates the cost of lost production, and thus FCA, which factors in the concept of unemployment and labour reserve, results in more realistic representation; However, the absence of reliable estimates for country-specific friction periods limited the use of FCA. Nevertheless, some countries left it to the analyst to choose which method they wanted to use [43, 51, 59]. Overall, the inclusion of productivity loss and its methodology was contentious, with the majority of countries advocating to include it in additional analyses, and to present results with and without the inclusion of productivity losses.

Another area with significant variation in the national HEE guidelines was the concept of discounting. The majority of the national HEE guidelines did not justify the basis for opting a specific discount rate [33–35, 39, 40, 42, 44, 46, 48, 50, 51, 54, 56, 58, 62]. Often the discount rate chosen was more of an imitation effect chosen to maintain consistency across studies [36, 56, 59]. Furthermore, the plausible assumptions on which the discount rate was calculated was usually not stated clearly. This is particularly important to guide countries in deriving specific discount rates.

This review provides useful insights for countries that have recently institutionalized HTA and are anticipating the development of national HEE guidelines in the near future. The lack of common methodological standards often results in poor quality of economic evaluations, thus limiting their use in decision making [17, 18, 20]. Developing a common set of standards is urgently needed in these countries to increase the quality and comparability of evidence, and thereby increasing usability for policy formation. These countries have several options. First, they can replicate the methods and recommendations of the existing guidelines, which may promote standardization and transferability. However, the complex nature of the health systems and financing mechanisms limits this option of complete replication. The second option available to these countries is to start de novo, which may not be an efficient methodology. We recommend that these countries could instead choose a middle path, which involves adapting the existing national HEE guidelines to cater to their local health system needs, current data availability, local researcher capacity, health financing system, and the nature of predominant use for HTA and how it is communicated.

The results of this review should be interpreted in light of following limitations. First, we had to exclude 12 national HEE guidelines that were not available in the English language. Furthermore, we acknowledge that several countries may have HTA systems in place, with the existence of internal guidelines and processes; however, guidelines were not available in the public domain and hence could not be included. Moreover, while systematic reviews are considered as the gold standard, we did not undertake a systematic search in scientific databases. However, we feel this would not bias our results since the national HEE guidelines are usually published as official documents by the national organizations rather than as research papers. Second, the ISPOR web repository is considered to be the best source of national guidelines, which we further supplemented by undertaking a search in the GEAR database and country-specific HTA agency websites. Additionally, searching HEE-specific databases such as the Tufts Cost-Effectiveness Analysis Registry would not affect the yield of the present review since this registry archives only original cost per QALY studies (CUA) and excludes all methodological articles, including guidelines [88, 89]. Thus, we believe the chances of missing any major national HEE guidelines are low. Lastly, a quality appraisal could not be undertaken because the national HEE guidelines have varied recommendations and no generalized tool is available to assess the quality of such guidelines.

Conclusion

There is substantial variation in several key recommendations given by the national HEE guidelines that have been developed globally. This is perhaps unsurprising given the differences in the health systems and financing frameworks, capacity of local researchers, data availability, and the purpose of HTA among different countries. This review offers some lessons for countries that are planning to develop their own guidelines; however, in addition to the technical guidelines, the countries need a system in place to review the evidence generated through HEE and to use this for decision making.

Supplementary Information

Below is the link to the electronic supplementary material.

Declarations

Funding

No separate funding was received to conduct this review. The review was undertaken by Deepshikha Sharma as part of her doctoral research, which was supported by the Indian Council of Medical Research, New Delhi, India.

Conflict of interest

Deepshikha Sharma, Arun Kumar Aggarwal, Laura E. Downey and Shankar Prinja declare they have no conflicts of interest.

Data availability

All data generated and analysed in this review are available within the manuscript and its supplementary information file.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Code availability

Not applicable.

Author contributions

DS: Literature search and data analysis, writing of the original draft and revising the final version of the manuscript. AKA: Study conception and design, interpretation of results, and original draft review and editing. LED: Interpretation of results, and original draft review and editing. SP: Study conception and design, interpretation of results, and original draft review and editing. All authors read and approved the final version of the manuscript.