Abstract
Background:
During times of fiscal austerity, means of reimbursement decision-making are of particular interest for public health theory and practice. Introduction of advanced health technologies, growing health expenditures and increased public scrutiny over drug reimbursement decisions have pushed governments to consider mechanisms that promote the use of effective health technologies, while constraining costs.
Aims:
The study’s aim was to explore the current rationale of the drug reimbursement decision-making framework in Bulgaria. Our pilot research focused on one particular component of this process – the criteria used – because of the critical role that criteria are known to have in setting budgets and priorities in the field of public health. The analysis pursued two objectives: to identify important criteria relevant to drug reimbursement decision-making and to unveil relationships between theory and practice.
Study Design:
Cross-sectional study.
Methods:
The study was realized through a closed-ended survey on reimbursement criteria among four major public health stakeholders – medical professionals, patients, health authorities, and industry. Empirical outcomes were then cross-compared with the theoretical framework, as defined by current Bulgarian public health legislation. Analysis outlined what is done and what needs to be done in the field of public health reimbursement decision-making.
Results:
Bulgarian public health stakeholders agreed on 15 criteria to form a tentative optimal framework for drug reimbursement decision-making. The most apparent gap between the empirically found preferences and the official legislation is the lack of consideration for the strength of evidence in reimbursement decisions.
Conclusion:
Bulgarian policy makers need to address specific gaps, such as formal consideration for strength of evidence, explicit role of efficiency criteria, and means to effectively empower patient and citizen involvement in public health decision-making. Drug reimbursement criteria have to be integrated into legitimate public health decision support tools that ensure the achievement of national public health objectives. These recommendations could be expanded to all Eastern European countries who share common public health problems.
Keywords: Bulgaria, decision-making, decision support models, health technology assessment, reimbursement, reimbursement criteria
Drug reimbursement decision-making – an emerging challenge for today’s public health
During times of fiscal austerity, means of reimbursement decision-making are of particular interest and importance for the public health theory and practice (1,2). Introduction of advanced health technologies, growing health expenditures and increased public scrutiny over drug reimbursement decisions have pushed governments to consider mechanisms that promote the use of effective health technologies, while constraining costs (3,4). Health technology assessment (HTA) has been largely promoted during the last decade for helping health authorities to innovate and reform public health, including reimbursement decision-making (5). HTA weights clinical and economic evidence, combining these considerations into an incremental cost-effectiveness ratio (ICER) (6). This final indicator has been used to foster informed reimbursement decision-making. However, despite its tremendous advantages, HTA remains a technical tool only. At the end of the day, drug reimbursement decisions are made by public health authorities, who must take into account various other factors as well.
Drug reimbursement decision-making – an even bigger problem in Eastern Europe
Over the years, many jurisdictions have adopted HTA to guide public health reimbursement decision-making. So have the Eastern European countries (7-11). Nevertheless, challenges remain when it comes to the role of HTA in public health decision-making as well as to human resource capacities of these countries (11). Lack of technical expertise and poor governance can limit the use of HTA (10). Implementation of objective and verifiable criteria for decisions, and the availability of remedies for negative decisions are often stated among the most difficult barriers on the way to successfully address HTA issues in Eastern Europe (7). Moreover, there is a globally growing interest in the systematic setting of priorities in public health reimbursement decision-making (1).
In Bulgaria, there is neither a specialized legislation for HTA, nor an explicit public health entity to perform such activities. The country has only recently started officially implementing HTA in drug reimbursement decision-making (9). The Ordinance on the terms, rules and procedure for regulation and registration of prices for medicinal products was adopted at the end of 2011 to provide a more sophisticated base for reimbursement decisions (12). Under this legal act, the National Council on prices and reimbursement of medicinal products was established, which makes reimbursement decisions based on a defined set of criteria. The Council, however, only appraises industry-submitted HTA reports. It does not perform assessment tasks itself (9,12).
Aim of the study
The study’s aim was to explore the current rationale of the drug reimbursement decision-making framework in Bulgaria. Our pilot research focused on one particular component of this process – the criteria used – because of the critical role criteria are found to have in setting budgets and priorities in the field of public health (1). The analysis pursued two objectives: to identify important criteria relevant to drug reimbursement decision-making and to unveil relationships between theory and practice. These pilot results would lay down a base for the subsequent elaboration of a balanced decision support tool that could serve public health policy in Bulgaria.
MATERIALS AND METHODS
Empirical assessment of drug reimbursement decision-making criteria
Overall, 58 distinct reimbursement criteria were identified and integrated into a closed-ended survey. This tentative list of reimbursement variables to be explored was taken from a recent literature review by Guindo et al. (13). This particular study was selected because of its rigorous classification of decision criteria for resource allocation. Respondents were asked to assess the empirical relevance of each criterion by answering the polar (yes or no) question: should this criterion be considered in drug reimbursement decision-making? A short explanation of the meaning of all criteria (as defined by Guindo et al. (13)) was provided too. The survey instrument was pretested with 7 respondents for accessibility and conceptual distinctiveness.
Selection of respondents
Target respondents included 4 groups of public health stake-holders from Bulgaria: medical professionals, heading university hospital clinics; chairs of patient organizations; health authorities (reimbursement decision-makers, working at the macro level); market access and governmental affairs executives of pharmaceutical companies.
Individual respondents were deliberatively selected. This was due to the fact that reimbursement decision-making is a very specific problem with few people having experience and expertise. Respondents’ eligibility was determined by past or present participation (prior works, publications, positions held, etc.) in decision-making on drug reimbursement with public funds in Bulgaria. Participants were additionally chosen to assure variation with regard to age, sex, geography, pathology and governance type. The total number of respondents was 40, with 10 coming from each target group. It is also acknowledged that in reality, different stakeholders may have different roles and impacts on drug reimbursement decision-making.
In order to confirm participation, each respondent was required to provide informed consent before the study. Four respondents declined to take part and were replaced. Ethics committee approval was not necessary. Questionnaires were e-mailed to respondents, who had 2 weeks to complete and return them. All 40 confirmed participants provided a completed survey. Descriptive statistical analysis was performed using Microsoft Excel 2002, v10.0 (Microsoft Corporation; Redmond, Washington, USA).
Cross-comparison with the current reimbursement decision-making framework in Bulgaria
After identifying drug reimbursement criteria, considered relevant and consensually agreed, our study compared this set to the list of reimbursement decision-making variables, currently defined and used in Bulgaria (9,12).
Official reimbursement decision-making criteria include 5 indicators, each having a weighting of a different amount of points. Clinical effectiveness scores for up to 45 points, assessing the therapeutic benefit of health technology, its impact on quality of life, life expectancy, whether the technology reduces underlying complications and whether it offers additional clinical benefits. Safety considerations are responsible for up to 30 points. Pharmacoeconomics gives up to 40 points, assessing cost-effectiveness and budget impact. Two more criteria score 20 points each – if the health technology has therapeutic alternative and if the health technology is indicated for conditions of high public health interest. A medicinal product should score a total of 60 points at least in order to be recommended for reimbursement with public funds in Bulgaria. Important gaps between this reimbursement framework and the empirically stated preferences of the public health stakeholders in the survey were examined.
RESULTS
Empirical assessment through a closed-ended survey among four public health stakeholder groups
In total, 10 reimbursement criteria (out of 58 surveyed) received a median agreement percentage of 80% and more in all 4 stakeholder groups (Table 1, Appendix 1). Health benefits were the only unanimously agreed individual criterion for explicit consideration in drug reimbursement decision-making. Five of these 10 criteria extensively characterized the health outcomes and benefits of the health technology. Another 3 adjusted for the clinical context of the condition targeted by the health technology. Two more completed the detailed assessment by adding economic and scientific evidence perspectives. Apart from this top-tier group, 5 additional criteria received an agreement percentage of 90% and more in at least 1 of the 4 groups, giving further insight into the specific decision-making concerns of each public health stakeholder group. Combined, these 15 criteria gave empirically a tentative optimal framework for the drug reimbursement decision-making process in Bulgaria.
TABLE 1.
Cross comparison of the empirical and theoretical reimbursement decision-making criteria
| Empirical assessment through a closed-ended survey among four stakeholder groups | Theoretical assessment as defined in the current official reimbursement legislation | ||||||
|---|---|---|---|---|---|---|---|
|
|
|
||||||
| Reimbursement decision-making criterion | Medical professionals (n=10) | Patients (n=10) | Health authorities (n=10) | Industry (n=10) | Overall agreement percentage (median) | Reimbursement decision-making criterion | Weight (actual points given) |
| Reimbursement criteria of 80% and more overall agreement | |||||||
|
|
|||||||
| Health benefits | 100% | 90% | 100% | 100% | 100% | Clinical effectiveness | 45 |
| Efficacy/effectiveness | 100% | 70% | 100% | 90% | 95% | Clinical effectiveness | 45 |
| Strength of evidence | 100% | 70% | 80% | 100% | 90% | Currently not considered | n/a |
| Population effect | 90% | 70% | 90% | 80% | 85% | Public health effect | 20 |
| Safety | 80% | 70% | 80% | 90% | 80% | Safety | 30 |
| Individual effect | 80% | 90% | 80% | 70% | 80% | Clinical effectiveness | 45 |
| Disease burden | 90% | 60% | 70% | 100% | 80% | Currently not considered | n/a |
| Treatment alternatives | 80% | 80% | 80% | 100% | 80% | Therapeutic alternative | 20 |
| Need | 70% | 80% | 80% | 100% | 80% | Currently not considered | n/a |
| Budget impact | 90% | 40% | 70% | 100% | 80% | Pharmacoeconomics | 40 |
| Reimbursement criteria of 90% and more agreement in at least one of the stakeholder groups | |||||||
|
|
|||||||
| Life-saving | 80% | 90% | 70% | 70% | 75% | Currently not considered | n/a |
| Cost-effectiveness | 100% | 40% | 80% | 70% | 75% | Pharmacoeconomics | 40 |
| Clinical guidelines and practices | 90% | 70% | 70% | 60% | 70% | Currently not considered | n/a |
| Access | 70% | 70% | 90% | 60% | 70% | Currently not considered | n/a |
| Disease severity | 60% | 30% | 90% | 30% | 45% | Currently not considered | n/a |
n/a: not applicable
Analyzing the differences within the 4 public health stake-holder groups revealed several important variations. Economic criteria received a relatively lower agreement percentage by the patient representatives. Budget impact and cost-effectiveness were both only 40% agreed by patients, while their overall median agreement percentage was 80% and 75%, respectively. On the other hand, patient stakeholders gave the highest agreement percentages to purely medical factors such as individual effect and life-saving nature. Medical professionals generally demonstrated higher agreement percentages compared to the overall median scores, as this was the case for all 15 criteria but one. The best consensus within a single stakeholder group was, however, reached among the industry representatives. Six of the 10 top criteria identified showed 100% agreement percentage from the pharmaceutical industry representatives. Regarding the 5 additional criteria, health authorities expressed interest in clinical (disease severity) and equity (access) factors. Medical professionals unanimously agreed on the cost-effectiveness and availability of clinical guidelines and best practices.
Gaps identified in the current public health reimbursement decision-making framework
The most apparent gap between the empirical preferences, found in the survey, and the theoretical scores, as defined by the present Bulgarian coverage decision-making framework, was the lack of consideration for the strength of the evidence which is used to support reimbursement decisions. By overall agreement percentage, this criterion was ranked third out of 58 criteria, gathering almost unanimous support from all four public health groups. At the same time, the current legal framework does not take this factor into account. Furthermore, respondents wanted reimbursement decisions to be more accountable for the medical context of the health technology in question, integrating factors like disease severity and life-saving. Health needs and access issues were also agreed as relevant for consideration. The current reimbursement decision-making framework only accounts for special considerations like therapeutic alternative and public health interest, but misses other factors, which are perceived as equally important by public health stakeholders (Table 1).
DISCUSSION
Identifying and implementing drug reimbursement decision-making criteria
The idea that public health authorities utilize explicit criteria for reimbursement decision-making is neither controversial nor novel (1). Nevertheless, this process remains complicated. Difficulty arises because drug reimbursement decision-making includes competing equity and efficiency obligations. It results in different levels of funding and opposing interests of the stakeholders involved (14). The most apparent problem here before public health authorities is what reimbursement criteria to use and in which manner to implement them in real-world settings. Concentrating on a limited number of indicators seems more functional (15). However, it usually leads to the marginalization of other relevant considerations and affected groups (16). A broader number of criteria ensure greater public involvement and transparency, but it could also initiate endless debates among competing stakeholders.
Eastern European countries could not directly transpose a set of drug reimbursement decision-making criteria from other countries and regions. Local public health resources, needs and expectations strongly differ even from one Eastern European country to another. These differences impact the relative importance of the individual criteria, making any analytical decision-making framework unique to its own public health settings. Reimbursement criteria may be the same, but local public health considerations are different. Political interests and societal preferences vary. Most importantly, national public health systems operate within different scopes and resources.
The outcomes and benefits of the health technologies in question, therapeutic context and impact of the conditions targeted constitute the core of any drug reimbursement decision-making framework. Indeed, if a health technology demonstrates a significant clinical added value and is supported by sound scientific evidence, it would be highly illogical to leave this technology out of public health. Nowadays, reimbursement decisions are, however, not so much between effective and ineffective health technologies, or necessary and unnecessary ones. Rather, choices are often between technologies that are somewhat effective and/or needed (17).
Our cross-comparison analysis confirmed that drug reimbursement legislation in Bulgaria should establish a means to address the role of other factors, such as economic impact, fairness, ethics and overall context, which are more susceptible to deviant interpretations. One crucial gap to be filled is the formal consideration for the strength of evidence used in drug reimbursement. This criterion is not only used to support public health decision-making, but also to lend legitimacy to decisions and actions pursued (18). The interpretation of evidence in decision-making is, however, influenced by several factors, such as organizational support, credibility, relevance and applicability in practice, political support and legislative constraints (19). These and other criteria, which stand for equity, efficiency and political context, actually make public health reimbursement decision-making frameworks unique and very specific to their own local settings. Equity criteria relate to tackling inequalities and distributional impact. Efficiency criteria refer to achieving the largest impact at the lowest cost and the total number of beneficiaries (20). Determining the drivers behind these categories of criteria is essential to improve the overall applicability of the drug reimbursement decision-making (21). Further research of these criteria beyond their direct meaning and elaboration of rational ways in which they can be integrated into public health policy decisions are needed.
Integration of efficiency, equity and political considerations into public health reimbursement decision-making framework
Use of the cost-effectiveness criterion allows decision-makers to assess the effectiveness of health technologies, while the budget impact criterion provides them with information on the impact of the adoption and use of a technology in a particular jurisdiction. This combined background is crucial, because it answers fundamental questions of whether a health technology presents a value for the money and what resources will eventually be necessary to implement this decision (22,23). The actual implementation of these and other efficiency criteria is, however, very often unclear and non-transparent. This problem is highlighted even in countries where credibility and accountability have long been established as a mandatory requirement in public health decision-making (24). Niezen et al. (23) concluded that despite the fact that regulators almost always demand a budget impact estimate, they seem reluctant to formally include it as a rationing criterion. Budget impact is believed to be lacking scientific rigor, thus not representing a rational use of evidence-based and explicit knowledge. The experience of Rocchi et al. (24) confirmed similar problems for the use of cost-effectiveness. Despite the formal requirement for economic evidence, there is virtually no information available on how cost-effectiveness is being used by public health authorities in decision-making process.
The importance of efficiency criteria is undeniable. This is why skepticism and concerns expressed by public health stakeholders need to be properly addressed in order not to undermine the overall legitimacy of public health decisions. Consistency in drug reimbursement decision-making does not necessarily mean uniformity of decisions. However, it implies a capacity to explain how seemingly different decisions are reached by different groups or at different times (24).
Efficiency criteria have become an important and frequently used tool in reimbursement decision-making, especially in the case of innovative health technologies. Nevertheless, there are concerns expressed about the impact of economic evaluations in terms of fairness (25,26). This is why equity criteria have been introduced to balance public health decisions, namely to ensure fair distribution of health benefits in society (27). There is a particular need for public health decisions in terms of achievement of overall health policy goals, i.e. ensuring the availability, accessibility and affordability of relevant health technologies to populations in need in timely and adequate fashion (28). Otherwise, access delays and increased financial burden mean generating significant health inequalities within the society.
The effective mechanisms for public involvement are very likely to determine the practical implementation of equity criteria like access, vulnerable populations and solidarity. The active role of citizen in public health decision-making is seen as a way to ensure a better, patient-focused health system (29). Despite this strong emphasis on stakeholder involvement in drug reimbursement decisions, there is no consensus regarding its definition and there is still limited evidence on the actual intake of these procedures (30). This is a serious constraint to the full implementation of the equity criteria. The appropriate recognition, reflection and inclusion of these values will increase the likelihood of meaningful public health policies and enhance the efficiency and reputation of national public health systems.
Efficiency and equity are not the only criteria that shape drug reimbursement decision-making. Political factors provide an important context for all public health decisions. Decision-makers may not always tend to be benevolent maximizers of social welfare. Different advocacy groups try to exercise influence on authorities to prioritize health technologies according to their objectives. As a result, public health decision-makers are more likely to use intuitive or heuristic approaches to simplify the complexity of reimbursement decision-making (16). This is actually the reason to see many well-designed approaches to drug reimbursement decision-making failing. For example, it is not because setting a threshold for ICER is good or bad; it is because of the political factors that the subsequent application of this mechanism becomes loose and selective (22). For this reason, public health practitioners admit that political interests are among the most important decision-making criteria, which are to be weighed against other considerations. Some stakeholders even regard political pressure as an essential constraint within which public health systems have to operate (1). Researchers explained this phenomenon with the need for an effective public health leader engagement in the political arena to ensure sufficient funding (1). It is apparent that public health reimbursement decisions have real-life political consequences, thus making it difficult to avoid such considerations.
Implications on public health policy
Public health authorities are facing a series of challenges in today’s drug reimbursement decision-making. They have to balance limited budgets and increased expectations, formal requirements and informal constraints. Our pilot results would lay down a base for subsequent elaboration of an advanced decision support tool that could serve public health policy in Bulgaria. This study identified and explored a short list of criteria, rated relevant and appropriate in Bulgarian settings and consensually agreed by all groups. Bulgarian health policy makers need to address specific gaps, such as formal consideration for the strength of evidence, explicit role of efficiency criteria, and means to effectively empower patient and citizen involvement in public health decision-making.
Determining a set of criteria for drug reimbursement decision-making is an emerging health policy challenge, because of the critical role these variables play in setting public health budgets and priorities. While addressing this issue, public health authorities should keep in mind that a broader participation, both in terms of multidisciplinary expertise and stakeholder involvement, is an optimal way to lend public health decision-making transparency and legitimacy. Reimbursement criteria and decisions should be in line with health policy’s overall aim of ensuring the availability, accessibility and affordability of relevant health technologies to populations in need in a timely and adequate fashion. The balanced selection of reimbursement criteria and their subsequent consistent and coherent application in decision-making enhance the efficiency and reputation of national public health systems. These policy recommendations could be greatly expanded to all Eastern European countries, who share common public health problems.
APPENDIX 1. Agreement percentage per all 58 individual criteria surveyed* (13)
| Criterion | Medical professionals | Patients | Health authorities | Industry | Overall agreement percentage (median) | |||
|---|---|---|---|---|---|---|---|---|
| Number of responses | 10 | 10 | 10 | 10 | 40 | |||
| Category 1 – Health outcomes and benefits of intervention | ||||||||
| Health benefits | 100% | 90% | 100% | 100% | 100% | |||
| Efficacy/effectiveness | 100% | 70% | 100% | 90% | 95% | |||
| Life-saving | 80% | 90% | 70% | 70% | 75% | |||
| Safety | 80% | 70% | 80% | 90% | 80% | |||
| Patient-reported outcomes | 70% | 40% | 40% | 50% | 45% | |||
| Quality of care | 70% | 50% | 70% | 50% | 60% | |||
| Category 2 – Type of health benefit | ||||||||
| Population effect (prevention) | 90% | 70% | 90% | 80% | 85% | |||
| Individual effect (medical service) | 80% | 90% | 80% | 70% | 80% | |||
| Category 3 – Impact of the disease targeted by intervention | ||||||||
| Disease severity | 60% | 30% | 90% | 30% | 45% | |||
| Disease determinants | 30% | 10% | 20% | 30% | 25% | |||
| Disease burden | 90% | 60% | 70% | 100% | 80% | |||
| Epidemiology | 70% | 30% | 60% | 70% | 65% | |||
| Category 4 – Therapeutic context of intervention | ||||||||
| Treatment alternatives | 80% | 80% | 80% | 100% | 80% | |||
| Need | 70% | 80% | 80% | 100% | 80% | |||
| Clinical guidelines and practices | 90% | 70% | 70% | 60% | 70% | |||
| Pre-existing use | 60% | 20% | 20% | 70% | 40% | |||
| Category 5 – Economic impact of intervention | ||||||||
| Cost | 80% | 40% | 60% | 70% | 65% | |||
| Budget impact | 90% | 40% | 70% | 100% | 80% | |||
| Broad financial impact | 60% | 40% | 50% | 70% | 55% | |||
| Poverty reduction | 60% | 60% | 30% | 10% | 45% | |||
| Cost-effectiveness | 100% | 40% | 80% | 70% | 75% | |||
| Value | 50% | 20% | 20% | 20% | 20% | |||
| Efficiency and opportunity costs | 10% | 10% | 20% | 10% | 10% | |||
| Resources | 50% | 40% | 60% | 10% | 45% | |||
| Insurance premiums | 60% | 10% | 30% | 20% | 25% | |||
| Category 6 – Quality and uncertainty of evidence | ||||||||
| Evidence available | 60% | 50% | 40% | 10% | 45% | |||
| Strength of evidence | 100% | 70% | 80% | 100% | 90% | |||
| Relevance of evidence | 80% | 50% | 70% | 40% | 60% | |||
| Evidence characteristics | 50% | 40% | 60% | 30% | 45% | |||
| Research ethics | 80% | 50% | 30% | 50% | 50% | |||
| Evidence requirements | 70% | 60% | 40% | 40% | 50% | |||
| Category 7 – Implementation complexity of intervention | ||||||||
| Legislation | 60% | 60% | 60% | 30% | 60% | |||
| Organizational requirements and capacity to implement | 50% | 50% | 30% | 30% | 40% | |||
| Skills | 80% | 60% | 50% | 40% | 55% | |||
| Flexibility of intervention | 70% | 50% | 30% | 20% | 40% | |||
| Characteristics of implementation | 60% | 50% | 50% | 50% | 50% | |||
| Appropriate use | 60% | 60% | 30% | 20% | 45% | |||
| Barriers and acceptability | 70% | 20% | 20% | 20% | 20% | |||
| Integration and system efficiencies | 50% | 60% | 70% | 40% | 55% | |||
| Sustainability | 80% | 60% | 70% | 40% | 65% | |||
| Category 8 – Priorities, fairness and ethics | ||||||||
| Population priorities | 60% | 20% | 40% | 50% | 45% | |||
| Access | 70% | 70% | 90% | 60% | 70% | |||
| Vulnerable and needy population | 80% | 50% | 80% | 70% | 75% | |||
| Equity, fairness and justice | 30% | 50% | 50% | 40% | 45% | |||
| Utility | 80% | 20% | 30% | 20% | 25% | |||
| Solidarity | 50% | 60% | 20% | 40% | 45% | |||
| Ethics and moral aspects | 60% | 40% | 50% | 30% | 45% | |||
| Category 9 – Overall context | ||||||||
| Mission and mandate of health system | 30% | 20% | 10% | 30% | 25% | |||
| Overall priorities | 80% | 60% | 40% | 50% | 55% | |||
| Financial constraints | 40% | 20% | 60% | 20% | 30% | |||
| Incentives | 20% | 10% | 20% | 0% | 15% | |||
| Political aspects | 20% | 20% | 20% | 0% | 20% | |||
| Historical aspects | 30% | 10% | 10% | 0% | 10% | |||
| Cultural aspects | 40% | 40% | 0% | 10% | 25% | |||
| Innovation | 80% | 50% | 40% | 60% | 55% | |||
| Partnership and leadership | 40% | 60% | 20% | 20% | 30% | |||
| Citizen involvement | 50% | 40% | 60% | 20% | 45% | |||
| Stakeholder interests and pressures | 50% | 10% | 30% | 0% | 20% | |||
Footnotes
Ethics Committee Approval: N/A.
Informed Consent: Written informed consent was obtained from the patients who participated in this study.
Peer-review: Externally peer-reviewed.
Author contributions: Concept - G.I., R.S.; Design - G.I., R.S.; Supervision - R.S.; Data Collection &/or Processing - G.I.; Analysis &/or Interpretation - G.I., R.S.; Literature Search - G.I., R.S.; Writing - G.I., R.S.; Critical Reviews - G.I., R.S.
Conflict of Interest: No conflict of interest was declared by the authors.
Financial Disclosure: The authors declared that this study has received no financial support.
REFERENCES
- 1.Leider JP, Resnick B, Kass N, Sellers K, Young J, Bernet P, et al. Budget- and priority-setting criteria at state health agencies in times of austerity: a mixed-methods study. Am J Public Health. 2014;104:1092–9. doi: 10.2105/AJPH.2013.301732. http://dx.doi.org/10.2105/AJPH.2013.301732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Sibbald SL, Gibson JL, Singer PA, Upshur R, Martin DK. Evaluating priority setting success in healthcare: a pilot study. BMC Health Serv Res. 2010;10:131. doi: 10.1186/1472-6963-10-131. http://dx.doi.org/10.1186/1472-6963-10-131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Simoens S. Health economic assessment: a methodological primer. Int J Environ Res Public Health. 2009;6:2950–66. doi: 10.3390/ijerph6122950. http://dx.doi.org/10.3390/ijerph6122950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Stoykova M, Mussurlieva N. Some problems of prevention and promotion of dental health in Bulgaria. Dentalna Medicina. 2011;93:178–82. [Google Scholar]
- 5.Chabot I, Rocchi A. Oncology drug health technology assessment recommendations: Canadian versus UK experiences. Clinicoecon Outcomes Res. 2014;6:357–67. doi: 10.2147/CEOR.S66309. http://dx.doi.org/10.2147/CEOR.S66309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Eichler HG, Kong SX, Gerth WC, Mavros P, Jönsson B. Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge? Value Health. 2004;7:518–28. doi: 10.1111/j.1524-4733.2004.75003.x. http://dx.doi.org/10.1111/j.1524-4733.2004.75003.x. [DOI] [PubMed] [Google Scholar]
- 7.Kolasa K, Kalo Z, Zah V, Dolezal T. Role of health technology assessment in the process of implementation of the EU Transparency Directive: relevant experience from Central Eastern European countries. Expert Rev Pharmacoecon Outcomes Res. 2012;12:283–7. doi: 10.1586/erp.12.12. http://dx.doi.org/10.1586/erp.12.12. [DOI] [PubMed] [Google Scholar]
- 8.Kolasa K, Schubert S, Manca A, Hermanowski T. A review of Health Technology Assessment (HTA) recommendations for drug therapies issued between 2007 and 2009 and their impact on policymaking processes in Poland. Health Policy. 2011;102:145–51. doi: 10.1016/j.healthpol.2011.05.001. http://dx.doi.org/10.1016/j.healthpol.2011.05.001. [DOI] [PubMed] [Google Scholar]
- 9.Iskrov GG, Raycheva RD, Stefanov RS. Insight into reimbursement decision-making criteria in Bulgaria: implications for orphan drugs. Folia Med (Plovdiv) 2013;55:80–6. doi: 10.2478/folmed-2013-0032. http://dx.doi.org/10.2478/folmed-2013-0032. [DOI] [PubMed] [Google Scholar]
- 10.Lopert R, Ruiz F, Chalkidou K. Applying rapid ‘de-facto’ HTA in resource-limited settings: experience from Romania. Health Policy. 2013;112:202–8. doi: 10.1016/j.healthpol.2013.07.019. http://dx.doi.org/10.1016/j.healthpol.2013.07.019. [DOI] [PubMed] [Google Scholar]
- 11.Gulácsi L, Rotar AM, Niewada M, Löblová O, Rencz F, Petrova G, et al. Health technology assessment in Poland, the Czech Republic, Hungary, Romania and Bulgaria. Eur J Health Econ. 2014;15(Suppl 1):S13–25. doi: 10.1007/s10198-014-0590-8. http://dx.doi.org/10.1007/s10198-014-0590-8. [DOI] [PubMed] [Google Scholar]
- 12. Ordinance on the terms, rules and procedure for regulation and registration of prices for medicinal products. Adopted by Council of Ministers Decree No. 97 of 19 April 2013. Promulgated in the State Gazette No. 40 of 30 April 2013.
- 13.Guindo LA, Wagner M, Baltussen R, Rindress D, van Til J, Kind P, et al. From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking. Cost Eff Resour Alloc. 2012;10:9. doi: 10.1186/1478-7547-10-9. http://dx.doi.org/10.1186/1478-7547-10-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rosenberg-Yunger ZR, Daar AS, Thorsteinsdóttir H, Martin DK. Priority setting for orphan drugs: an international comparison. Health Policy. 2011;100:25–34. doi: 10.1016/j.healthpol.2010.09.008. http://dx.doi.org/10.1016/j.healthpol.2010.09.008. [DOI] [PubMed] [Google Scholar]
- 15.Sussex J, Rollet P, Garau M, Schmitt C, Kent A, Hutchings A. A pilot study of multicriteria decision analysis for valuing orphan medicines. Value Health. 2013;16:1163–9. doi: 10.1016/j.jval.2013.10.002. http://dx.doi.org/10.1016/j.jval.2013.10.002. [DOI] [PubMed] [Google Scholar]
- 16.Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14. doi: 10.1186/1478-7547-4-14. http://dx.doi.org/10.1186/1478-7547-4-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Stolk EA, Poley MJ. Criteria for determining a basic health services package. Recent developments in The Netherlands. Eur J Health Econ. 2005;6:2–7. doi: 10.1007/s10198-004-0271-0. http://dx.doi.org/10.1007/s10198-004-0271-0. [DOI] [PubMed] [Google Scholar]
- 18.Dupont AG, Van Wilder PB. Access to orphan drugs despite poor quality of clinical evidence. Br J Clin Pharmacol. 2011;71:488–96. doi: 10.1111/j.1365-2125.2010.03877.x. http://dx.doi.org/10.1111/j.1365-2125.2010.03877.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lorenc T, Tyner EF, Petticrew M, Duffy S, Martineau FP, Phillips G, et al. Cultures of evidence across policy sectors: systematic review of qualitative evidence. Eur J Public Health. 2014;24:1041–7. doi: 10.1093/eurpub/cku038. http://dx.doi.org/10.1093/eurpub/cku038. [DOI] [PubMed] [Google Scholar]
- 20.Defechereux T, Paolucci F, Mirelman A, Youngkong S, Botten G, Hagen TP, et al. Health care priority setting in Norway a multicriteria decision analysis. BMC Health Serv Res. 2012;12:39. doi: 10.1186/1472-6963-12-39. http://dx.doi.org/10.1186/1472-6963-12-39. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Tony M, Wagner M, Khoury H, Rindress D, Papastavros T, Oh P, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv Res. 2011;11:329. doi: 10.1186/1472-6963-11-329. http://dx.doi.org/10.1186/1472-6963-11-329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Iskrov G, Stefanov R. Post-marketing access to orphan drugs: a critical analysis of health technology assessment and reimbursement decision-making considerations. Orphan Drugs: Research and Reviews. 2014;4:1–9. http://dx.doi.org/10.2147/ODRR.S43409. [Google Scholar]
- 23.Niezen MG, de Bont A, Busschbach JJ, Cohen JP, Stolk EA. Finding legitimacy for the role of budget impact in drug reimbursement decisions. Int J Technol Assess Health Care. 2009;25:49–55. doi: 10.1017/S0266462309090072. [DOI] [PubMed] [Google Scholar]
- 24.Rocchi A, Menon D, Verma S, Miller E. The role of economic evidence in Canadian oncology reimbursement decision-making: to lambda and beyond. Value Health. 2008;11:771–83. doi: 10.1111/j.1524-4733.2007.00298.x. http://dx.doi.org/10.1111/j.1524-4733.2007.00298.x. [DOI] [PubMed] [Google Scholar]
- 25.Cho E, Park EC, Kang MS. Pitfalls in reimbursement decisions for oncology drugs in South Korea: need for addressing the ethical dimensions in technology assessment. Asian Pac J Cancer Prev. 2013;14:3785–92. doi: 10.7314/apjcp.2013.14.6.3785. http://dx.doi.org/10.7314/APJCP.2013.14.6.3785. [DOI] [PubMed] [Google Scholar]
- 26.Stolk EA, van Donselaar G, Brouwer WB, Busschbach JJ. Reconciliation of economic concerns and health policy: illustration of an equity adjustment procedure using proportional shortfall. Pharmacoeconomics. 2004;22:1097–107. doi: 10.2165/00019053-200422170-00001. http://dx.doi.org/10.2165/00019053-200422170-00001. [DOI] [PubMed] [Google Scholar]
- 27.Green CJ, Maclure M, Fortin PM, Ramsay CR, Aaserud M, Bardal S. Pharmaceutical policies: effects of restrictions on reimbursement. Cochrane Database Syst Rev. 2010:CD008654. doi: 10.1002/14651858.CD008654. http://dx.doi.org/10.1002/14651858.cd008654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Sax P, Shmueli A. Impact of pharmaceutical regulation and policies on health system performance goals in Israel. Adv Health Econ Health Serv Res. 2010;22:77–101. doi: 10.1108/s0731-2199(2010)0000022007. http://dx.doi.org/10.1108/S0731-2199(2010)0000022007. [DOI] [PubMed] [Google Scholar]
- 29.Pizzo E, Doyle C, Matthews R, Barlow J. Patient and public involvement: how much do we spend and what are the benefits? Health Expect. 2014 doi: 10.1111/hex.12204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Rosenberg-Yunger ZR, Thorsteinsdóttir H, Daar AS, Martin DK. Stakeholder involvement in expensive drug recommendation decisions: an international perspective. Health Policy. 2012;105:226–35. doi: 10.1016/j.healthpol.2011.12.002. http://dx.doi.org/10.1016/j.healthpol.2011.12.002. [DOI] [PubMed] [Google Scholar]