Abstract
Background: Various studies have used multiple attribute decision making (MADM) techniques to assess and rank health technologies. The goal of the present study was to prioritize health technologies using various techniques of MADMs in combination with decision rules.
Methods: The study is an applied research using multi-attribute decision making (MADM) methods. This study extracted the attributes related to health technology assessment from global literature and experts’ opinions. In this study, two different types of experts were consulted: the first type, including three experts in the field of the decision-making techniques, on the subject of setting priority on health focusing on MADM; and the second one consists of seven experts in the field of HTA, asked about the selection of attributes and determination their importance. Candidate health technologies were individually weighted and ranked using TOPSIS, SAW and VIKOR by the weight and decision matrix. The results obtained from various techniques were combined and ranked using Copeland’s technique to obtain the final ranking of health technologies. To determine HTA type reports, decision rules were defined. All models were designed via MS Excel.
Results: This study chose eight technologies according to six tradeoff attributes. These attributes included health benefits at the population level, vulnerable population size, availability of alternative technologies, budget impact, financial protection, and quality of evidence. Their exact weights were 0.25, 0.121, 0.146, 0.132, 0.167 and 0.181, respectively. Also, safety and uncertainty about the cost-effectiveness were considered as the veto and decision rules respectively. Copeland’s method was therefore used to combine the methods: Whereas HT2 (The technology for treating patients suffering from varicose) was ranked the highest priority and HT3 (The palliative method for patients who suffer from various cancers) was ranked the lowest (for preventing from any ethical issue, the exact name of each technology wasn’t mentioned).
Conclusion: Finally, in accordance with decision rules which are based on various conditions of "uncertainty about the cost-effectiveness", it is recommended that full health technology assessment report be performed on three technologies, rapid health technology assessment report be performed on four others, and, finally no prioritizing for health technology assessment be made on one of them.
Keywords: Topic Selection, Health Technology Assessment, Multiple Attribute Decision Making
↑ What is “already known” in this topic:
Health technologies provide an opportunity to experience better healthcare services for governments, providers, and patients, although these advancements can increase healthcare expenditures. Given that the number of health technologies requiring assessment is more than the resources available, all health technology assessment organizations should prioritize their research projects.
→ What this article adds:
To the best of our knowledge, no one has been yet utilized the Copeland’s mixed model in combination with decision rules to aggregate the different ranks of health technologies obtained from different decision-making techniques. Hence, the current study uses three well-known multiple attribute models available in the literature of operations research to expand the previous model and perform operations with a number of requested technologies and then integrates their different results.
Introduction
In recent years, many policymakers have advocated the use of new technologies in the health sector. These technologies provide an opportunity to experience better healthcare services for governments, providers, and patients (1), although these advancements can increase healthcare expenditures (2). Due to developments in medical technologies and the gap between the demand for health services and the available resources, health care priority setting is important for the policy agenda in all developed countries (3). Today, providing health services increasingly depends on complex technologies that have been used exclusively for public health services and medical objectives. Developing and utilizing such technologies has been very beneficial to social health, but it has not been free of expenditures (4).
However, resource limitation has provided a choice, which means we can use the available resources in the best way possible. Using service prioritization methods as well as learning how to use them is very important in reaching this goal (5).
Using medical technologies has grown significantly in recent years, which can be of help in diagnosing and treatment. On the other hand, providing unlimited and uncontrolled access to technology can cause induced demand and overuse of health services; this has been a problem in many developed and developing countries, which has extremely increased expenditures. Therefore, technologies are evaluated using a systematic method before they are licensed. In this process, necessary steps are taken to show how available resources should be used as efficiently as possible (6). Given that the number of health technologies requiring assessment is more than the resources available, all health technology assessment organizations should prioritize their research projects (7).
Health Technology Assessment (HTA) process began in Iran in 2007 in the capacity of a secretary at the Department of Health in the Ministry of Health and Medical Education. Then, health technology assessment began in 2010 as an independent office under the supervision of the Office of Technology Assessment, Standardization, and Tariffs at the Department of Curative Affairs (8). For topic selection, this office has so far mainly focused on attributes like the burden of disease, expenditures, and organizational, moral and public acceptability indexes (8).
This study complements the previous two studies (9, 10), regarding choosing subjects to assess health technologies, which were carried out using a combined multiple attribute decision-making model. To the best of our knowledge, no one has been yet utilized the Copeland’s mixed model in combination with decision rules to aggregate the different ranks of health technologies obtained from different decision-making techniques. Hence, the current study aims to use three well-known multiple attribute models available in the literature of operations research to expand the previous model and perform operations with a number of requested technologies and then integrates their different results.
Methods
The study is an applied research that uses some multi-attribute decision making (MADM) techniques in order to assess health technologies. This research is based on performing operations research modeling of the multiple attribute decision making. As was mentioned in the Introduction section, this model is a continuation and expansion of the two previous studies (9, 10). The main question of this study was: which technology and topic are on the top of the priority list for doing HTA; therefore, the first step of our study was obtaining a request list for HTA. Therefore, in this retrospective study, eight technologies that were sent to Iran’s National Institute of Health Research for performing HTA were selected. In fact, they were used to explain and test our model; therefore, seven stages were designed as follows.
Stage 1. Specifying the Experts
Experts were consulted in this study for various reasons, including selecting multiple attribute decision-making models for the study, the selection of relevant attributes related to health technology assessment, weighting the stage of attributes, and determining the values of some attributes qualitatively. For the first reason, three Iranian academic members of health economics and health policy who had expertise in setting priority on health focusing on MADM and for the other reasons seven top experts who worked in the field of HTA in Iran were questioned.
Stage 2. Determining the Attributes Affecting the Ranking of Health Technologies
At this stage, the attributes prioritizing health technologies in Iran were identified with a systematic and comparative review in the previous model. The identified attributes were categorized into five categories of attributes related to a) efficacy, b) disease and the target population, c) alternative technologies, d) economic factors, and e) evidence (9, 10).
Stage 3. Extracting Final Attributes Through Comparative Studies and Experts’ Opinion
In the previous model, nine attributes were finalized from both the systematic review and the expert opinions. These attributes were as follows: safety, efficacy/effectiveness, population size, vulnerable population size, availability of alternative technologies, cost-effectiveness in other countries, budget impact, financial protection, and quality of evidence (9, 10).
As mentioned before, some changes have been made in this new model regarding the above-mentioned attributes. These changes were mainly related to the role of each attribute. All attributes in the previous model were considered as trade-off ones (10). This approach was wrong; for example if a technology had a high mortality incidence, it could not trade off for its cost-effectiveness, or duplicated value of efficacy. Hence, for solving these kinds of issues, first, safety was considered as a veto attribute. Second, efficacy/effectiveness was merged with population size and a new attribute was defined as “health benefits at the population level”. Third, the cost-effectiveness in other countries was changed into a criterion of "uncertainty about the cost-effectiveness," which was for the first time considered in the Dutch model, was defined as decision rule (11, 12). At this step, this criterion will be a positive one. That is, the greater the uncertainty is, it will result in a greater need for performing health technology assessment. The average scores of experts’ opinions were considered as the final score of uncertainty on cost-effectiveness. Hence, in this model, one veto attribute, six trade-off attributes, and one decision rule attribute were considered.
Stage 4. Quantifying Attributes
For most attributes, the scales which were designed in previous models were used (10), but for some attribute (safety, effectiveness, population size, and cost-effectiveness in other countries), changes were designed as follows:
First of all, five conditions were considered for the safety attribute. The definition of each degree regarding this attribute was derived from a definition of safety based on the Cancer Therapy Evaluation Program (13) (Table 1).
Table 1. Various States of the Safety Attribute and their Corresponding Degree .
| Safety Degree | Condition |
| Degree 1 | No Adverse Events |
| Degree 2 | Minor and Slight Adverse Event |
| Degree 3 | Severe Adverse Event |
| Degree 4 | Life-threatening Adverse Event |
| Degree 5 | Fatal Adverse Event |
It is worth mentioning that the safety attribute in this new model has a veto state, which was disregarded at the stage of trade-off with other attributes. In other words, each health technology that is in the 3rd, 4th, or 5th stage of the safety attribute was eliminated from the assessment stage and not investigated.
Second, for measuring “Health Benefits at the Population Level” (combining effectiveness and population size), a new code was developed (Table 2). This is an innovative approach because for measuring efficacy with the aim of HTA research priority setting, efficacy should be considered at the level of the population (in the previous model, these two attributes were considered as trade-off form and this was a wrong approach). Hence some new code were defined for measuring this merged attribute.
Table 2. Coding for "Health Benefits at the Population Level” .
| Effectiveness Code | Population Size Code | Level | Health Benefits at the Population Level Code |
| 5 | 5 | Very high | 5 |
| 5 | 4 | High | 4 |
| 5 | 3 | Medium | 3 |
| 5 | 2 | low | 2 |
| 5 | 1 | Very low | 1 |
| 4 | 5 | Very high | 5 |
| 4 | 4 | High | 4 |
| 4 | 3 | Medium | 3 |
| 4 | 2 | Low | 2 |
| 4 | 1 | Very low | 1 |
| 3 | 5 | High | 4 |
| 3 | 4 | Medium | 3 |
| 3 | 3 | Low | 2 |
| 3 | 2 | Low | 2 |
| 3 | 1 | Very low | 1 |
| 2 | 5 | Medium | 3 |
| 2 | 4 | Medium | 3 |
| 2 | 3 | Low | 2 |
| 2 | 2 | Low | 2 |
| 2 | 1 | Very low | 1 |
| 1 | 5 | Low | 2 |
| 1 | 4 | Low | 2 |
| 1 | 3 | Very low | 1 |
| 1 | 2 | Very low | 1 |
| 1 | 1 | Very low | 1 |
Moreover, due to inadequate evidence for the two attributes of budget impact and financial protection, for estimation of them, seven top HTA experts were questioned to determine the condition of this technology in these attributes given the available evidence for each technology (a five-point scale designed in the previous model was used ) (10).
Stage 5. Specifying the Technologies Selected for Ranking and Data Extraction
At this stage, eight technologies sent to Iran’s National Institute of Health Research were retrospectively selected. It is noteworthy that, in this research, the exact name of the technology was not used. Instead, the function of the technology was defined (Table 3). After that, with regards to data extraction, a systematic review (using the PRISMA approach) was done to extract the relevant data for each technology. This systematic review was conducted based on the eight specified attributes in the period of 2005 to 2018 from the most relevant and up-to-date databases (like PubMed/Medline, Social Sciences Database, and Google Scholar in English), (all the investigated technologies had the standard Safety degree).
Table 3. Health Technologies Selected for Assessment and Ranking .
| No. | The Technology Function | Abbreviation |
| 1 | The technology for treating refractive disorders of the eye | HT1 |
| 2 | The technology for treating patients suffering from varicose | HT2 |
| 3 | The palliative method for patients who suffer from various cancers | HT3 |
| 4 | The procedure for hemorrhoid surgery | HT4 |
| 5 | The technology for treating diabetic foot wounds | HT5 |
| 6 | The diagnostic technology for patients who suffer from metastatic cancers | HT6 |
| 7 | The technology for treating burn patients | HT7 |
| 8 | The blood coagulation technology for hemorrhage indications | HT8 |
Stage 6. The Attributes’ Weighting andExecuting Decision Making Models and Combining the Results
At this stage, various multiple attribute of decision-making techniques were introduced to 3 experts. According to the nature of the study, they were then asked to select their desired techniques. The result was to use a pairwise comparison table to extract expert opinion and weight the attributes (7 experts). After determining the final attributes affecting health technology selection, their weight was calculated. For this purpose, seven top HTA experts were questioned. First, each expert was given a paired 6x6 comparison table and was asked to state his or her preferences regarding the relative importance of the attributes, such that for the a-i-j element, the i line indicates the preference of the attribute to the column j attribute. The inconsistency rate was then calculated for the tables, and the tables with an inconsistency rate lower than 0.1 were combined. For this purpose, the geometric mean of equal elements was calculated for consistent pairwise comparison matrices and, finally, the weights of the attribute were calculated according to the data from the combined tables using the eigenvector method. The weights of attributes were calculated from the final combined pairwise comparison table using the eigenvector method.
In order to rank health technologies, the consensus was to use three methods, which were TOPSIS (the technique for order preferences by similarity to an ideal solution), SAW (The simple additive weighting method), and VIKOR (The VlseKriterijumska Optimizacija I Kompromisno Resenje). These techniques categorized in the MADM class are used to rank the alternatives with respect to different criteria. The interested readers can refer to (14) to study these techniques in detail. However, these techniques may lead to different ranks of alternatives. Therefore, the Copeland’s method was used to summarize and combine the results and determine the final ranking of health technologies (considering that techniques lead to a different ranking of health technologies). After this ranking, a proposed cut-off point (which was determined by authors as innovation of this research) was proposed for determining the proper HTA type on the basis of Copeland’s final ranking. In this decision-making method, a pairwise comparison matrix is created for different alternatives. The logic of this method is on the basis of the number of “win or lose” (preferences) of each alternative compared to another one (“Win” is shown with M and “lose” or “equal preference” is shown with X). Then the final score of each alternative is calculated from “Wins – Loses”. Finally, the options are ranked in ascending order based on the difference between “win and lose”.
These weighting methods and the three MADM techniques were selected because of the following logic: first, a pairwise comparison matrix was used to extract experts’ opinions and the weight of the attributes. This technique was employed because it could better reflect the importance of attributes regarding the research goal, which was HTA topic selection. Second, the compliance of characteristics of different categories of MADM methods presented by Hwang and Yoon (1981) was investigated with the nature of our problem (15). We figured out that the methods classified in the “cardinal preference of attribute” category were the most appropriate methods for ranking HTA. There are several methods in this category exposed in detail to experts to select the most desirable. The consensus was to use three methods, which were SAW, TOPSIS, and VIKOR.
Stage 7. Defining Decision Rules for "Uncertainty about the Cost-Effectiveness"
At this stage, this criterion will not be a trade-off one and acts as a decision rule. In terms of the resulted ranking from the first step, the highest-ranking (50% -upper limit) should have the "uncertainty about the cost-effectiveness" of very high or high. If they do not have this condition, they will be transferred to the second category namely rapid health technology assessment. Medium limit technologies (50 to 90% - medium limit) should have “uncertainty about the cost-effectiveness” of the medium. If this criterion is low or very low for these technologies, they should be transferred to the third category namely no performing of health technology assessment. If they have the condition of high or very high, they should be transferred to the first category. For the lowest limit technologies (10% - lower limit), they should have “uncertainty about the cost-effectiveness” of low or very low. In case this criterion is medium for these technologies, then they should be transferred to the second category. For eight technologies that were tested in this research, there was little information about cost-effectiveness. Therefore, a five-point scale was defined to estimate the cost-effectiveness and seven top HTA experts were requested to determine the condition technology in this attribute, given the available evidence (Table 4).
Table 4. Coding for “Uncertainty about the cost-effectiveness.” .
| The code | “Uncertainty about the cost-effectiveness”: How much of the unpredictability on the cost-effectiveness does the technology have before the assessment? |
| Level | |
| 5 | Very High |
| 4 | High |
| 3 | Some extent |
| 2 | Low |
| 1 | Very Low |
Results
For the purpose of this research, the selected health technologies were ranked in several steps using different approaches explained above. The findings of each step were as follows:
Determining the Weight of Attributes
The Health Benefits at the Population Level and Quality of Evidence, with the weights of 0.25 and 0.181 respectively, were the most important in selecting health technologies according to experts’ opinions (Table 5).
Table 5. Determining the Attributes Affecting Health Technologies’ Topic Selection (Attributes’ Weighting) .
| Attributes |
Health Benefits at the Population Level |
Vulnerable population |
Availability rate of alternative technologies |
Budget impact |
Financial protection |
Quality of evidence |
| Weight | 0.25 | 0.121 | 0.146 | 0.132 | 0.167 | 0.181 |
Forming the Decision Matrix
In order to form this matrix, the codes obtained for each technology were extracted from evidence (some attributes were explicitly or implicitly inferred from the related articles while others were inferred from Grey literature) (Tables 6 and 7).
Table 6. Codes Extracted for Each Technology .
| No. | Title | Codes |
| 1 | HT1 |
This technology is in safety degree one according to the safety standard and enters the prioritization phase. Effectiveness: Code 4 - This technology arguably increases patients’ life years and quality. Population size: Code 5 - Refractive disorders are very common in human societies. Vulnerable population size: Code 5 - Vulnerable populations, like children under 5 and the elderly, are more likely to suffer this disorder. Availability of alternative technologies: Code 2 - High availability of alternative technologies with the same safety and effectiveness for the target population. The quality of evidence: Code 2 - Randomized controlled trial with medium quality. Budget impact: Code 2 - No significant changes in the costs or additional Charges. Financial protection: Code 3 - 50 to 75 % of probability for covering by the basic health insurance package. |
| 2 | HT2 |
This technology is in safety degree two according to the safety standard and enters the prioritization phase. Effectiveness: Code 3 - This technology can be as effective as other alternative treatments. Population size: Code 5 - About 10 to 20 percent of the population suffer from varicose veins (8 million people) Vulnerable population size: Code 5 - Is more common among elderly women over 65. Availability of alternative technologies: Code 3 - Appropriate availability of alternative technologies with the same safety and effectiveness for the target population (laser and surgery). The quality of evidence: Code 5 - high quality systematic review. Budget impact: Code 2 - No significant changes in the costs or additional Charges. Financial protection: Code 2 -Less than 50 % of probability for covering by the basic health insurance package |
| 3 | HT3 |
This technology is in safety degree two according to the safety standard and enters the prioritization phase (minor adverse event). Effectiveness: Code 1 - This technology arguably slightly increases patients’ quality of life. Population size: Code 5 - Population sizes larger than 500 thousand people (suffering from various types of cancer). Vulnerable population size: Code 5 - Large share of the vulnerable population with various types of cancer. Availability of alternative technologies: Code 3 - Appropriate availability of alternative technologies with the same safety and effectiveness for the target population. The quality of evidence: Code 1 - Other studies (cohort, case series and quasi-random studies etc.) with any quality. Budget impact: Code 2 -No significant changes in the costs or additional Charges. Financial protection: Code 2 -Less than 50 % of probability for covering by the basic health insurance package |
| 4 | HT4 |
This technology is in safety degree two according to the safety standard and enters the prioritization phase (minor adverse event). Effectiveness: Code 4 - This technology arguably increases patients’ life years and quality. Population Size: Code 5 - About half the country’s population suffers from hemorrhoids. Vulnerable population size: Code 4 - Most affected people are 45 to 65 years old. Availability of alternative technologies: Code 3 - Appropriate availability of alternative technologies with the same safety and effectiveness for the target population. The quality of evidence: Code 2 - - Randomized controlled trial with medium quality. Budget impact: Code 3 - Very slight change in the amount of savings. Financial protection: Code 3 - 50 to 75 % of probability for covering by the basic health insurance package. |
| 5 | HT5 |
This technology is in safety degree two according to the safety standard and enters the prioritization phase. Effectiveness: Code 1 – This technology can increase patients ‘quality of life to some extent. Population size: Code 5 - Population sizes larger than 500 thousand people (suffering from diabetes). Vulnerable population size: Code 5 - Large share of the vulnerable population with diabetes. Availability of alternative technologies: Code 2 - High availability of alternative technologies with the same level of safety and effectiveness. The quality of evidence: : Code 5 - High quality systematic review Budget impact: Code 3 - Very slight change in the amount of savings. Financial protection: Code 3 - 50 to 75 % of probability for covering by the basic health insurance package. |
| 6 | HT6 |
This technology is in safety degree one according to the safety standard, and enters the prioritization phase. Effectiveness: Code 4 - Considering the false negatives, it is arguably not 100% accurate, and the disease being screened and diagnosed can be treated. Population size: Code 5 - Population size larger than 500 thousand people (suffering from various types of cancer). Vulnerable population size: Code 5 - Large share of the vulnerable population with various types of cancer. Availability of alternative technologies: Code 5 - Partial availability of alternative technologies with the same safety and effectiveness for the target population. The quality of evidence: Code 1 - Other studies (cohort, case series and quasi-random studies etc.) with any quality. Budget impact: Code 2 -No significant changes in the costs or additional Charges. Financial protection: Code 2 - Less than 50 % of probability for covering by the basic health insurance package |
| 7 | HT7 |
This technology is in safety degree two according to the safety standard and enters the prioritization phase. Effectiveness: Code 4 - This technology arguably increases patients’ life years and quality. Population size: Code 5 - Considering this technology’s application in various wound infection indications, the size of the target population will be over 500 thousand. Vulnerable population size: Code 5 - Vulnerability groups suffer more wound infections. Availability of alternative technologies: Code 2 - High availability of alternative technologies with the same level of safety and effectiveness. The quality of evidence: Code 2 - Systematic review with medium or low quality of other studies. Budget impact: Code 3 -Very slight change in the amount of savings. Financial protection: Code 3 - 50 to 75 % of probability for covering by the basic health insurance package. |
| 8 | HT8 |
This technology is in safety degree one according to the safety standard and enters the prioritization phase. Effectiveness: Code 4 - This technology arguably increases patients’ life years and quality. Population Size: Code 5 - Considering this technology’s application in various hemorrhage indications, the size of the target population will be over 500 thousand. Vulnerable population size: Code 5 - These groups will suffer more hemorrhages due to their vulnerability. Availability of alternative technologies: Code 2 - High availability of alternative technologies with the same safety and effectiveness for the target population The quality of evidence: Code 4 - High quality randomized controlled trial. Budget impact: Code 2 -No significant changes in the costs or additional Charges. Financial protection: Code 2 - Less than 50 % of probability for covering by the basic health insurance package |
Table 7. Final Decision Matrix .
|
Alternative |
Health Benefits at the Population Level |
Vulnerable population |
Availability rate of alternative technologies |
Budget impact |
Financial protection | Quality of evidence |
| HT1 | 5 | 5 | 2 | 2 | 3 | 2 |
| HT2 | 4 | 5 | 3 | 2 | 2 | 5 |
| HT3 | 2 | 5 | 3 | 2 | 2 | 1 |
| HT4 | 5 | 4 | 3 | 3 | 3 | 2 |
| HT5 | 2 | 5 | 2 | 3 | 3 | 5 |
| HT6 | 5 | 5 | 5 | 2 | 2 | 1 |
| HT7 | 5 | 5 | 2 | 3 | 3 | 2 |
| HT8 | 5 | 5 | 2 | 2 | 2 | 4 |
Ranking Health Technologies Using Multiple Attribute Decision Making Techniques
In this step, health technologies were individually investigated using three techniques chosen by experts, SAW, TOPSIS, and VIKOR. These rankings are shown in Table 8.
Table 8. Final Ranking via SAW, TOPSIS, and VIKOR .
| MADM Models Alternatives | SAW | TOPSIS | VIKOR |
| HT1 | 7 | 7 | 4 |
| HT2 | 1 | 1 | 2 |
| HT3 | 8 | 8 | 8 |
| HT4 | 3 | 4 | 3 |
| HT5 | 2 | 3 | 7 |
| HT6 | 6 | 5 | 6 |
| HT7 | 5 | 6 | 1 |
| HT8 | 4 | 2 | 5 |
Extracting the Final Ranking of Health Technologies Using Copeland’s Method
Various techniques often lead to different rankings of options. For example, HT8 is ranked 4, 2, 5 in SAW, TOPSIS and VIKOR, respectively. Therefore, other methods should be used to combine the rankings obtained through various techniques, one of which is Copeland’s method. According to experts’ opinions, and combining the results obtained through various methods, three health technologies, namely HT2, HT5 and HT8, were more important than others (Table 9).
Table 9. Final Ranking Matrix using Copeland’s Method .
|
alternatives alternatives |
HT1 | HT2 | HT3 | HT4 | HT5 | HT6 | HT7 | HT8 | No. of Wins |
| HT1 | - | X | M | X | X | X | X | X | 1 |
| HT2 | M | - | M | M | M | M | M | M | 8 |
| HT3 | X | X | - | X | X | X | X | X | 0 |
| HT4 | M | X | M | - | X | M | M | X | 4 |
| HT5 | M | X | M | M | - | M | M | X | 5 |
| HT6 | M | X | M | X | X | - | X | X | 2 |
| HT7 | M | X | M | X | X | M | - | X | 3 |
| HT8 | M | X | M | X | M | M | M | - | 5 |
| No. of Loses | 6 | 0 | 7 | 3 | 2 | 5 | 4 | 2 | - |
| (Wins – Loses) | -5 | 8 | -7 | 1 | 3 | -3 | -1 | 3 | - |
| Final Rank | 7 | 1 | 8 | 4 | 2 | 6 | 5 | 2 | - |
Specifying the Report Types (Classification) by Defining Decision Rules
It is recommended to assess health technology according to the following range (from high priorities to low ones):
1) Technologies that rank higher than 50% (upper limit): Full health technology assessment
2) Technologies that rank between 50% and 90% (medium limit): Rapid health technology assessment
3) Technologies that rank lower than 10% (lower limit): No health technology assessment
According to this categorization of technologies and decision rules, which were explained in the Methods section (stage 7), the recommended report types were as follows: (Table 10).
Table 10. Determining of HTA Reports .
| Alternatives | Final Rank (Copeland) | The Code of "uncertainty about the cost-effectiveness" | HTA Type |
| HT1 | 7 | 4 | Full HTA |
| HT2 | 1 | 4 | Full HTA |
| HT3 | 8 | 2 | No HTA |
| HT4 | 4 | 4 | Full HTA |
| HT5 | 2 | 3 | Rapid HTA |
| HT6 | 6 | 3 | Rapid HTA |
| HT7 | 5 | 3 | Rapid HTA |
| HT8 | 2 | 3 | Rapid HTA |
Discussion
Based on the evidence gathered, and in accordance with decision rules which are on the basis of various conditions of “uncertainty about the cost-effectiveness”, it is recommended that a full health technology assessment report be performed on three technologies (HT1, HT2, HT4), rapid health technology assessment report be performed on four other techniques (HT5, HT6, HT7, HT8) and, finally, no prioritizing for health technology assessment be made on one of them (HT3). The model designed in this study can also be compared with the health technology assessment prioritization model in the Netherlands. Various methods for categorizing, scoring and weighting policymaking attributes for this domain have been used in this model. These attributes included disease burden, potential health benefits for the patient, number of patients, direct intervention costs for each patient, financial consequences of intervention over time, and its effect on health system policies (11). The current model can also be compared with the EVIDEM health technology assessment prioritization model. While the model designed for the present study used a one to five scales, this model uses a zero to three scale to value each attribute. Moreover, the attributes shared by these two models included improving effectiveness, safety, alternative interventions, the impact of budget on health programs, completeness and consistency of existing evidence, and the relevance and reliability of present evidence (16). The present model can also be compared with the health technology assessment prioritization model in Lithuania. This model searched the research literature and used a qualitative study to design and test the health technology assessment prioritization model. The attributes relevant to this model were the health benefit, evidence, assessment timing, expected benefits to policymakers, and social, legal and moral concepts (17). It appears that health benefits and evidence were two common attributes with the model designed in this study. This model can be compared with the PriTec prioritization tool developed by the Galician Health Technology Assessment Agency, which was used for the prioritization of obsolete technologies. The attributes which were used in this tool were population, frequency of the disease, burden of the disease, frequency of use, patients’ preferences, efficacy, adverse risks, costs, organization and other implications. This inferred technology was used for the appraisal phase and not for the topic selection. Another difference between our model and the PriTec prioritization tool was the merging of the evidence and experts’ opinions at the same time. However, the value of each attribute was measured on the basis of experts’ opinions in the PriTec prioritization tool (18, 19).
The difference between this model and the topic selection model from the Canadian Agency for Drugs and Technologies in Health regarded the type of multi-attribute decision-making model. In the Canadian model, the overall relative weight extraction and the final priority setting were performed based on the analytic hierarchy process (AHP) (20), whilst in the model used in this study, only the relative weight was extracted through paired comparisons (based on experts’ opinions), and the related evidence was applied to the final model (via designing the related coding and just for “the budget impact, the financial protection and the uncertainty about the cost-effectiveness”. Due to the lack of evidence, experts’ opinions were used). This subject was designed in order to prevent the full and unlimited interventions of medical beneficiary groups for the selection of health technologies that are assumed as the stewardship scope.
The results of this study prove the applicability of the designed model as a retrospective operation executed with mixed prioritization of eight technologies requested for health technology assessment performed by Iranian researchers and ordered by Iran’s National Institute of Health Research. This kind of approach can improve the HTA system in Iran which is now mentioned as one of the Strengths of the Iranian system in the current decade (21).
Conclusion
It is recommended that a full health technology assessment report be performed on three technologies, a rapid health technology assessment report be performed on four others, and, finally no prioritizing for health technology assessment be made on one of them. It is recommended that HTA agencies use this model to prioritize the requests for health technology assessment by providing a summarized list of requested health technologies each month to a team from internal members involved in health technologies assessment, who determine the type of reports required for health technology assessment by extracting the recommended codes from the latest related papers to these technologies and placing them in the model. For localizing of this model in other countries, attributes which played as a decision rule in this research may be different in other societies depend on their main values. Considering the advancement of health technologies, it appears that HTA agencies can use this model to allocate their research budget for health technology assessment, which can by itself improve the impact of health technology assessment reports in making health macro-policies and optimizing financial resource allocations in the health domain.
Study Limitations
On the three economic attributes, namely “the budget impact, the financial protection, and the uncertainty about the cost-effectiveness”, information at the international and domestic levels was scarce. Therefore, the researcher provided evidence on the effectiveness, target population and availability of alternative technologies as well as the cost of technology to a group of top HTA experts, who were involved in the project, and asked them to submit their own estimates on the mentioned attributes. However, it seems that for making accurate and robust estimates, designing a framework will be essential in this regard.
Acknowledgments
The authors would like to express very great appreciation to Professor Rob Baltussen for his valuable suggestions and comments, especially in performing “MADM with decision rules” during the development of this research. The authors acknowledge the Islamic Republic of Iran’s National Institute of Health Research for providing relevant data and financial support.
Ethical issues
This study was approved by the ethical committee of the Islamic Republic of Iran’s National Institute of Health Research, under the license no: IR.TUMS.NIHR.REC.1395.13
Conflict of Interests
The authors declare that they have no competing interests.
Cite this article as: Mobinizadeh M, Mohamadi E, Arman H, Nasiripour AA, Olyaeemanesh A, Mohamadi S. Topic selection for health technology assessment: An approach combining multiple attribute decision making and decision rules. Med J Islam Repub Iran. 2021 (27 Mar);35:40. https://doi.org/10.47176/mjiri.35.40
Footnotes
Conflicts of Interest: None declared
Funding:This study was supported and funded by the Islamic Republic of Iran’s National Institute of Health Research.
References
- 1.Farnoodi S. Presenting a Framework for Evaluation of Health Technologies in Health and Medical System of Iran; Case Study: Robolens Robot. J Sci Technl Policy. 2010;2(3):75–86. [Google Scholar]
- 2.Abbasi M, Arabloo J. Ethics in health technology assessment. Bioethics J. 2012;2(3):101–16. [Google Scholar]
- 3.Carlsson P. Health technology assessment and priority setting for health policy in Sweden. Int J Technol Assess Health Care. 2004;20(01):44–54. doi: 10.1017/s0266462304000777. [DOI] [PubMed] [Google Scholar]
- 4.Teerawattananon Y, Tantivess S, Yothasamut J, Kingkaew P, Chaisiri K. Historical development of health technology assessment in Thailand. Int J Technol Assess Health Care. 2009;25(S1):241–252. doi: 10.1017/S0266462309090709. [DOI] [PubMed] [Google Scholar]
- 5.Dehnavieh R, Rashidian A, Maleki M, Tabibi SA, Ibrahimi H, Pour SNH. Criteria for Priority-setting in Iran Basic Health Insurance Package: Exploring the Perceptions of Health Insurance Experts. Health MED. 2010;7(2):1542. [Google Scholar]
- 6.Ravaghi H, Sari AA, Sarvari S, Mobinizadeh MR. the Effectiveness of PET-Scan in Diagnosis and Treatment of Non-Small Cell Lung Carcinoma (NSCLC) and Lymphoma: A Comprehensive Review of Literature. J Isfahan Med School. 2012 Feb 20;29(167) [Google Scholar]
- 7.Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care. 2007;23(03):310–315. doi: 10.1017/s026646230707050x. [DOI] [PubMed] [Google Scholar]
- 8.Doaee SH, Olyaeemanesh A, Emami SH, Mobinizadeh M, Abooee P, Nejati M. et al. Development and Implementation of Health Technology Assessment: A Policy Study. Iran J Public Health. 2013;4(2(Supple1)):50. [PMC free article] [PubMed] [Google Scholar]
- 9.Mobinizadeh M, Raeissi P, Nasiripour AA, Olyaeemanesh A, Tabibi SJ. The health systems' priority setting criteria for selecting health technologies: A systematic review of the current evidence. Med J Islam Repub Iran. 2016;30:329. [PMC free article] [PubMed] [Google Scholar]
- 10.Mobinizadeh M, Raeissi P, Nasiripour AA, Olyaeemanesh A, Tabibi SJ. A model for priority setting of health technology assessment: the experience of AHP-TOPSIS combination approach. Daru. 2016 Dec;24(1):10. doi: 10.1186/s40199-016-0148-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Oortwijn W, Banta D, Vondeling H, Bouter L. Identification and priority setting for health technology assessment in The Netherlands: actors and activities. Health Policy. 1999;47(3):241–253. doi: 10.1016/s0168-8510(99)00020-2. [DOI] [PubMed] [Google Scholar]
- 12.Baltussen R, Marsh K, Thokala P, Diaby V, Castro H. et al. Multicriteria decision analysis to support health technology assessment agencies: benefits, limitations, and the way forward. Value Health. 2019 Nov 30;22(11):1283–1288. doi: 10.1016/j.jval.2019.06.014. [DOI] [PubMed] [Google Scholar]
- 13. Cancer Therapy Evaluation Program, Common Terminology Criteria for Adverse Events, Version 3.0, DCTD, NCI, NIH, DHHS March 31, 2003 (http://ctep.cancer.gov), Publish Date: August 9, 2006.
- 14. Tzeng GH, Huang JJ. Multiple Attribute Decision Making: Methods and applications. Taylor & Francis Group, LLC. 2011.
- 15. Hwang CL, Yoon K. Multiple Attribute Decision Making: Methods and Applications. Lecture Notes in Economics and Mathematical Systems, 186, Now York: Springer-Verlag. 1981. 10.1007/978-3-642-48318-9. [DOI]
- 16. Anonymous. [Cited Sep 2014]. Available from: https://www.evidem.org/docs/2014/EVIDEM-v2-3-Decision-criteria-2013-12-24.pdf.
- 17.Jankauskiene D, Petronyte G. A model for hta priority setting: experience in Lithuania. Int J Technol Assess Health Care. 2013;29(4):450. doi: 10.1017/S0266462313000470. [DOI] [PubMed] [Google Scholar]
- 18. Anonymous. [Cited Feb 2020]. Available from: http://vortal.htai.org/index.php?q=node/181.
- 19. Anonymous. [Cited Feb 2020]. Available from: https://htai.org/wpcontent/uploads/2018/02/Prioritisation20of20potential20obsolete20technologies20-20Alberto20Ruano-Ravina.pdf.
- 20.Husereau D, Boucher M, Noorani H. Priority setting for health technology assessment at CADTH. Int J Technol Assess Health Care. 2010;26(3):341–7. doi: 10.1017/S0266462310000383. [DOI] [PubMed] [Google Scholar]
- 21.Olyaeemanesh A, Behzadifar M, Mousavinejhad N, Behzadifar M, Heydarvand S, Azari S, Martini M, Bakhtiari A, Luigi Bragazzi N. Iran’s Health System Transformation Plan: A SWOT analysis . Med J Islam Repub Iran . 2018(12 May);32:39. doi: 10.14196/mjiri.32.39. [DOI] [PMC free article] [PubMed] [Google Scholar]