Skip to main content
NCBI home page
Wiley - PMC COVID-19 Collection logoLink to Wiley - PMC COVID-19 Collection
. 2021 Nov 15;29(1-2):80–91. doi: 10.1002/mcda.1772

A hybrid approach based on ELECTRE III‐genetic algorithm and TOPSIS method for selection of optimal COVID‐19 vaccines

Roberto Louis Forestal 1,✉, Shih‐Ming Pi 1
PMCID: PMC8646624  PMID: 40477443

Abstract

COVID‐19 pandemic poses unprecedented challenges to the world health system, prompting academics and health professionals to develop appropriate solutions. Researchers reported different COVID‐19 vaccines introduced by institutions and companies around the globe, which are at different stages of development. However, research developing an integrated framework for selecting and ranking the optimal potential vaccine against COVID‐19 is minimal. This paper aimed to fill this gap by using a hybrid methodology based on ELimination Et Choice Translating REality III (ELECTRE III)–Genetic Algorithm (GA) and Technique of Order Preference Similarity to the Ideal Solution (TOPSIS) approach to select the optimal SARS‐CoV‐2 vaccine. ELECTRE III method yields a fathomable analysis of the concordance index, while GA is known for its ability to disaggregate decision‐making preferences from holistic decisions. TOPSIS is preferred for picking an ideal and an anti‐ideal solution. Thus, combining ELECTRE III‐GA and TOPSIS is considered the best model to assess vaccines against the pandemic. The results confirm that the best vaccines rely on a high level of safety, efficacy, and availability. Our developed evaluation framework can help healthcare professionals and researchers gain research information and make critical decisions regarding potential vaccines against the disease.

Keywords: COVID‐19 vaccine, ELECTRE III, genetic algorithm, MCDM, TOPSIS

1. INTRODUCTION

At the writing of this paper, more than 12 months have passed since World Health Organization (WHO) declared Covid‐19 a pandemic. This pandemic has already caused the dramatic loss of millions of lives around the world. It poses unprecedented challenges to the global health system, urging the academic world and health professionals to work quickly on a solution. Studies on SARS‐CoV‐2 vaccine intentions showed that the majority of interviewees want to receive a COVID‐19 vaccine. Given the need for a vaccine increasing demand, research institutions and pharmaceutical corporations used different methods to design potential COVID‐19 vaccines platforms such as non‐replicating viral vector, RNA‐based, inactivated virus, protein subunit, and DNA‐based vaccines. For example, Kumar et al. (2021) designed a promiscuous subunit vaccine from a pathogenic sequence, while Enayatkhani et al. (2021) developed a multi‐epitope through a filtering pipeline. The reported vaccine of both studies was antigenic and capable of generating some robust immune responses.

Previous authors also used computational models, and multicriteria decision‐making (MCDM) approaches to widely inspect the novel coronavirus vaccine. For instance, Mohammed et al. (2020) used a decision matrix (DM) that embedded a mix of 10 evaluation criteria and 12 diagnostic models for COVID‐19. An integrated MCDM method is proposed. The Technique of Order Preference Similarity to the Ideal Solution (TOPSIS) is applied for benchmarking and ranking purposes. At the same time, Entropy, used to calculate the weights of criteria. The results revealed that the benchmarking and selection problems associated with COVID‐19 diagnosis models could be effectively solved using the integration of Entropy and TOPSIS. The linear support vector machine (SVM) classifier is selected as the best diagnosis model for COVID‐19 with the closeness coefficient value of 0.9899 for their case study data. Furthermore, the proposed methodology has solved the significant variance for each criterion in terms of ideal best and worst best value and the issue when specific diagnosis models have the same ideal best value.

However, our research acknowledges a substantial gap in the use of MCDM to select COVID‐19 vaccines, as noted by Abdelwahab et al. (2021), despite the fact that Hezam et al. (2021) highlighting the importance of vaccine classification uses a neutrosophic analytic hierarchy process (AHP) and TOPSIS method to rank COVID‐19 vaccine alternatives based on priority groups. Consequently, much of the existing literature focuses on the vaccine distribution process (Jakhar et al., 2021) instead of considering optimal vaccination approaches (Moore et al., 2021). ELimination Et Choice Translating Reality III (ELECTRE III) and genetic algorithm (GA) have reportedly outperformed common approaches in selecting optimal solutions while TOPSIS confirms its effectiveness in measuring the relative performance for each alternative (Aiello et al., 2013; Chen, 2021; Leyva‐López & Fernández‐González, 2003). Therefore, using a hybrid methodology based on ELECTRE–GA and TOPSIS approach can provide valuable insights into the optimised scenarios building (Marchetti & Wanke, 2020; Rohaninejad et al., 2015; Vafaeinejad, 2016). Concerning the motivation, our analysis also contributes to emerging literature studying COVID‐19 vaccines, their safety, and effectiveness.

The remainder of the paper is structured as follows. Section 2 details the literature review. Section 3 presents the methodology used, including data selection and decision‐making assessment. Section 4 and Section 5 present the empirical case study as well as results and discussion. Section 6 discusses managerial implications, while the final Section 7 concludes the analysis and elaborates on the limitations of the paper.

2. STATE OF THE ART AND PROBLEM STATEMENT

2.1. COVID‐19 vaccine development and effectiveness

The novel COVID‐19 pandemic caused by SARS‐CoV‐2 is a public‐ health emergency of international concern, thus calling for the development of a safe and effective vaccine to protect against the infection. It is essential to assess people's intention to be vaccinated. Studies such as Faasse and Newby (2020) and Head et al. (2020) conducted an online survey to examine the role of perceived risk, media coverage, and vaccination intentions among thousands of respondents. Findings revealed that the majority of interviewees were at least moderately concerned about a widespread COVID‐19 outbreak. Worrying about the epidemic and closely monitoring media coverage were consistent forecasting parameters of health protection behaviours and vaccination intentions. With growing demand, the availability of a COVID‐19 vaccine is touted as the solution to controlling the current COVID‐19 pandemic, reducing the number of infections and deaths, and facilitating the resumption of our previous lifestyle (Abdullahi et al., 2021; Begum et al., 2021).

Scholars reported that many research institutions and pharmaceutical companies have already embarked on the race to develop COVID‐19 vaccines, which involved the adaptive immune system. Their analysis recoursed to immunoinformatics and advocated a sequence analysis of the pathogenic strains of SARS‐CoV‐2 to design vaccine candidates. These vaccine candidates may provide meaningful, timely directives for an effective vaccine against SARS‐CoV‐2 (Chen et al., 2021; Kumar et al., 2021; Oyarzun et al., 2021). Moreover, SARS‐CoV‐2 spike glycoprotein followed by epitopes' use in constructing a multi‐epitope peptide vaccine construct (MEPVC) is also an attractive candidate for a vaccine, antibodies, and inhibitor development because of the many roles it plays in attachment, fusion, and entry into the host cell. The MEPVC revealed robust host immune system simulation with high immunoglobulins, cytokines, and interleukins (Enayatkhani et al., 2021; Saha et al., 2021).

Other researchers designed and presented different kinds of vaccines against the novel coronavirus. For instance, Khurana et al. (2021) and Malabadi et al. (2021) proposed a development framework of a potential COVID‐19 vaccine based on nanotechnology. They highlight the utility of nanomedicine in alleviating the COVID‐19 health crisis. In addition, it is also crucial to mention that previous researchers argued the need to optimise SARS‐CoV‐2 vaccine confidence, availability, and efficacy. For example, Mehrotra et al. (2021) provide a set of clinical endpoints based on clinical and statistical reasoning to facilitate a harmonised assessment and comparison of the efficacy of novel coronavirus vaccines. Ensuring public confidence in vaccine safety and effectiveness is also crucial to facilitate uptake. Bartsch et al. (2020) apply a computational model of the U.S. simulating the spread of COVID‐19 coronavirus and vaccination and reveal that the vaccine efficacy has to be at least 60% when vaccination coverage is 100%. Moreover, to extinguish an ongoing epidemic, the vaccine efficacy has to be at least 60% when coverage is 100% and at least 80% when coverage drops to 75% to reduce the peak by 85–86%, 61–62%, and 32% when vaccination occurs after 5, 15, and 30% of the population, respectively, have already been exposed to COVID‐19 coronavirus. A vaccine with an efficacy between 60 and 80% could still obviate the need for other measures under certain circumstances such as much higher, and in some cases, potentially unachievable, vaccination coverages.

2.2. Multiple criteria decision making

So far, we have displayed studies—primarily qualitative research—examining the demand, availability, and efficacy of COVID‐19 vaccines. Previous authors also used the MCDM approach, which involves multiple attributes when dealing with COVID‐19 pandemic and vaccination decisions (Batur Sir, 2021; Clemente‐Suárez et al., 2021; Ni et al., 2021). Mohammed et al. (2020) used a decision matrix (DM) that embedded a mix of 10 evaluation criteria and 12 diagnostic models for COVID‐19. An integrated MCDM method is proposed where TOPSIS is applied for the benchmarking and ranking purpose while Entropy is used to calculate the weights of criteria. The results revealed that the benchmarking and selection problems associated with COVID‐19 diagnosis models could be effectively solved using the integration of Entropy and TOPSIS. The linear SVM classifier is selected as the best diagnosis model for COVID‐19 with a closeness coefficient value of 0.9899 for their case study data. Furthermore, the proposed methodology has solved the significant variance for each criterion in terms of ideal best and worst best value, besides issues when specific diagnosis models have the same ideal best value.

Based on available published evidence and clinical practice, Sayan et al. (2020) evaluated diagnostic tests of coronavirus disease (COVID‐19) by MCDM methods, namely, fuzzy preference ranking organisation method for enrichment evaluation (fuzzy PROMETHEE) and fuzzy TOPSIS. Several parameters such as computerised tomography of the chest, CoV‐19 antigen detection, and chest X‐ray were evaluated by fuzzy linguistic scale to compare the diagnostic tests. This scale consists of selected parameters that possessed different weights, determined by the experts' opinions of the field. The results indicated that the most effective diagnosis method of COVID‐19 was chest CT. It is interesting to note that the methods consistently used in diagnosing viral diseases were ranked in second place for the diagnosis of COVID‐19. Angelis et al. (2021) advocate the need for an MCDM approach to assessing the value of COVID‐19 vaccines, calling for clinical, manufacturing, and cost aspects to be complemented with societal value considerations to inform decisions on development, reimbursement, and pricing of vaccines. Two important implications can be drawn from the results of this study.

First, the value of COVID‐19 vaccines should be estimated against other health interventions, which can be informed by large‐scale surveys of public preferences with established methodologies such as discrete choice experiments. Second, vaccine procurement costs could be used to derive cost‐to‐value ratios to inform resource allocation decisions within a fixed budget, similar to a portfolio optimisation approach. In addition, Hezam et al. (2021) highlight the need to identify priority groups when allocating COVID‐19 vaccine doses since they assumed it is almost impossible to vaccinate everyone. Their methods include the neutrosophic AHP and TOPSIS. Neutrosophic AHP has been used to evaluate criteria related to age, health status, a woman's status, and the kind of job. In contrast, TOPSIS has been employed to rank the COVID‐19 vaccine alternatives. Their results indicate that the most suitable vaccine for patients and health workers prioritises other alternative vaccines.

Although much work has been done on the novel coronavirus, obtaining an integrated solution to limit the current COVID‐19 disease remains challenging. Many studies conducted so far resulted in the urgent need to develop vaccines. Still, less has proposed a comprehensive framework to select and rank those potential vaccines candidates. Our research aims to apply a hybrid methodology based on ELECTRE III‐Genetic Algorithm and the TOPSIS approach to select and outclass the best SARS‐COV‐2 vaccines.

3. METHODOLOGY

This section discusses one possible methodology for assessing COVID‐19 candidate vaccines using indicators. Following Aiello et al. (2013) and Majumder, Biswas, and Majumder et al. (2020), our research applied ELECTRE‐GA and TOPIS approaches to selected and outranked candidate vaccines in clinical evaluation aiming at providing active acquired immunity to the novel coronavirus.

3.1. Selection of criteria

Identifying appropriate criteria for COVID‐19 candidate vaccines is essential for reliable outcomes. We examined several criteria sets proposed by World Health Organization (WHO), Centers for Disease Control and Prevention (CDC), U.S. Food and Drug Administration (FDA), Coalition for Epidemic Preparedness Innovations (CEPI), and Gavi, the Vaccine Alliance. Those indicators are as follows:

3.1.1. Safety (SAF)

The FDA has issued guidance for the industry on the steps required for developing and ultimately licensing vaccines to prevent COVID‐19—these are the same rigorous safety standards required for all vaccines. We imply that data from animal and human studies support no apparent risk of enhanced disease in vaccines by safety. Moreover, we also imply a lack of significant disease enhancement risk supported by clinical and/or pre‐clinical data from relevant/suitable animal model(s) and unexpected severe findings that could require more investigations. To calculate vaccine safety, we use a crude ratio from Whelan (2009). This percentage is defined as the number of subjects exposed to a vaccine and experiencing a particular adverse event (such as fever, headaches, muscle pain), n, divided by the total number of subjects exposed to the vaccine, N, regardless of the duration of follow‐up:

Crude rateCR=nN×100
Safety rateSR=100−CR

3.1.2. Efficacy (VEF)

An efficacious COVID‐19 vaccine could reduce the likelihood of infection of an individual, severity of disease in an individual, or degree of transmission within a population. In other words, it refers to the percentage reduction in the incidence of the disease in a vaccinated group compared to an unvaccinated group under optimal conditions (Nichols et al., 2018). We use the following equation from Orenstein et al. (1985) to estimate vaccine efficacy:

Vaccine EfficacyVEF=ARU−ARVARU×100

where ARU is the attack rate of COVID‐19 disease in the unvaccinated population, and ARV relates to the attack rate of the disease in the vaccinated population.

3.1.3. Effectiveness (EFF)

Once the efficacy of a vaccine has been asserted, determining its effectiveness is essential to establish the vaccine's uptake. Efficacy determines if the vaccine works, whereas effectiveness monitors the benefits of vaccination for the community. Vaccine effectiveness refers to preventing COVID‐19 disease depending on its potency and proper administration to individuals capable of responding (Andrew & McNeil, 2021). It can be measured by considering the percentage of COVID‐19 cases vaccinated (PCV), and the proportion of the population vaccinated (PPV) following Cohen et al. (2007).

Vaccine effectivenessEFF=1−PCV1−PPVPPV1−PCV

3.1.4. Stability (STA)

Stability data are sufficient to assure delivery of dose to be tested. The stability of vaccines has a significant impact on the success of immunisation programmes worldwide. Optimising the use of vaccines depends heavily on maintaining optimal storage conditions. A vaccine is now expected to remain stable at standard refrigerator temperatures of 2–8°C (36°–46 °F) for an average period of 1 week while shipping and long‐term storage conditions are expected to remain at standard freezer temperatures of −20°C (−4 °F; Dadari & Zgibor, 2021; Dumpa et al., 2019; World Health Organization, 2018). However, some vaccines are sensitive to freezing, some to heat and others to light. Therefore, replacing a traditional models of vaccine stability with a version that yielded cost‐saving appears to be more appropriate in appraising vaccine stability as Lee et al. (2017) proposed. This refers to the relative stability (r) of the vaccine i, which is the ratio between the minimum costs associated with other vaccines and vaccine i. PC, TC, and SC refer to procurement, transport, and storage costs, respectively.

Relative stabilityr=mCosti
Costi=PCi+TCi+SCi
m=MinCosti

The proposed set of indicators is organised into measurement units and information sources, as shown in Table 1.

TABLE 1.

Review of the selected criteria

Criteria and abbreviation Brief definition Measurement units Information source
Safety (SAF) We imply that data from animal and human studies support no apparent risk of enhanced disease in vaccines by safety. Adverse event profile supports advancement to the subsequent phase 0–100 rating score CDC, FDA
Efficacy (VEF) Evidence that the selected dose induces adequate immune responses in humans that might confer protection. 1–100 rating score CDC, WHO
Effectiveness (EFF) The ability in preventing COVID‐19 disease depends on its potency and proper administration to individuals capable of responding 1–100 rating score WHO
Stability (STA) Stability data are sufficient to assure delivery of dose to be tested 0–100 rating score WHO
Open in a new tab

An evaluation of coronavirus vaccine candidates has been performed using two multi‐criteria approaches. The first one is based on ELECTRE III, whose first version was proposed by Bernard Roy and his colleagues at the SEMA consultancy company. The ELECTRE III method was chosen in our work because it provides a simple and understandable analysis of the Concordance index. Indeed, the fundamental basis of the ELECTRE methodology is a pairwise comparison between alternatives using two types of indices: Concordance and Discordance. For each ordered pair of options (A, B), Concordance and Discordance indices are assigned. A Concordance index referred to the measurement of the arguments in favour of A outranks B, and a Discordance index may shed some doubt upon the latter statement (Figueira et al., 2016). In other words, this method aims to obtain a subset or kernel N of project options such that any action which is not in N is outranked by at least one action in N (Roy, 2013).

3.2. ELECTRE III‐Genetic algorithm model

3.2.1. Genetic algorithm

GA are the heuristic search and optimisation techniques that mimic natural evolution. This algorithm is developed with the ability to disaggregate preferences of DM from holistic decisions. The GA generates an initial population conforming to decision variables which are then converted into inter‐criteria parameters (see Figure 1). Each element in the population is evaluated by its fitness, which concerns the set of inter‐criteria parameters that better represent the decision‐making's preference when the set is used to construct the fuzzy outranking relation, then used in a multi‐criteria classification. Once the GA has passed through all the generations, the final result presents a set of inter‐criteria parameters. When used by ELECTRE III generates rankings similar or identical to the entry ranking defined by the decision‐makers (Carrillo et al., 2017).

FIGURE 1.

FIGURE 1

Open in a new tab

Simple genetic algorithm adapted from Sheppard (2017)

3.2.2. Procedure of the ELECTRE III‐GA

We consider the following steps to derive GA intercriteria parameters for our proposed ELECTRE III‐GA model.

Phase 1: Prepare the input data. Input data refers to the problem, which is the performance matrix and DM preference. The DM generates holistic judgments from the paired alternatives (a_b) or as an order of alternatives. Also, the DM establishes criteria importance, ordering the best criterion on the first position and the worst criterion on the last position.

Phase 2: Run the GA. The GA includes the ELECTRE III method to aggregate DM preference, constructing a valued outranking relation named preferential model. The GA applies the genetic operators and evolves the population in each iteration. The fitness function of the GA is Kendall's rank correlation index.

Phase 3: Consider the inter‐criteria parameters as output data. The outcomes of GA show the best population. This results in different sets of inter‐criteria parameters, which generates a ranking corresponding to the preferences of the DM.

3.3. TOPSIS approach

The second multi‐criteria approach we use to evaluate coronavirus vaccine candidates is TOPSIS. Introduced in 1981, TOPSIS is based on picking an ideal and an anti‐ideal solution and comparing the distance of each one of the alternatives to those (Chen & Hwang, 1992).

3.4. Proposed model

As described previously, the search to find a vaccine against COVID‐19 consists of two subprocesses: selecting and ranking problems. In this paper, vaccines candidates against the novel coronavirus have been studied. The proposed approach to solve this problem is based on three major steps: (1) Description of the multi‐criteria ranking problem, (2) Derivation of intercriteria parameters, and (3) Ranking of the preference order. To implement the first and second steps, the ELECTRE III‐GA model is used, while the last step involves TOPSIS methodology, as shown in Figure 2.

FIGURE 2.

FIGURE 2

Open in a new tab

Simplified representation of our proposed model

4. EMPIRICAL CASE STUDY

The next instance of the ranking problem discusses an empirical study of the following actual selection and ranking problem.

4.1. Sample data and decision making

Sample data involves 108 COVID‐19 vaccine candidates in clinical trials extracted from six major databases, including WHO, U.S. National Library of Medicine, Chinese Clinical Trial Registry, Pan African Clinical Trials Registry, EU Clinical Trials Register, and Australian New Zealand Clinical Trials Registry (ANZCTR) websites as 30 July 2021. Those 108 applicants are labelled as A1, A2…, A108. In addition, we have conducted personal online interviews with six professionals, including a medical doctor, a public health nurse, and four University students. The medical doctor is a graduate of the University of Notre Dame. He trained in internal medicine at the University of Southern California. He is a clinical associate professor at Florida State University College of Medicine. The second participant worked as a frontline health employee at Little Haiti Health Center for more than 10 years. The university students are all from Taiwan.

4.2. Threshold values and criteria weights

Using the original ELECTRE method, performance matrix, thresholds, weights (relative importance of the criteria), and final ranking were calculated and presented in this subsection. The DM was supported in defining its preferences and uncertainties through the indifference threshold (q), the preference threshold (p), and the veto threshold (v) for all five criteria by using a linear format. We define two columns of numeric values for each threshold, one for the slope (label beginning by alpha) and another for the interception (label beginning by beta), as shown in Table 2.

TABLE 2.

Threshold values

Criterion αq βq αp βp αv βv
C1. Safety (SAF) 0.08 −2000 0.13 −3000 0.9 50,000
C2. Efficacy (VEF) 0.02 0 0.05 0 0 0
C3. Effectiveness (EFF) 0.1 −0.5 0.2 −1 0.5 3
C4. Stability (STA) 0 3 0 5 0 15
Open in a new tab

Evaluation and prioritisation of COVID‐19 candidate vaccines developed by WHO are considered here. We complete the data preparation by imposing a direction of evaluation for each criterion and a direction of definition for each threshold, as shown in Table 3. All applicants were also evaluated by the experts using the criteria and scale. All criteria were treated as quantitative ones.

TABLE 3.

Evaluation/threshold direction and criteria weights

Criterion
SAFF VEF EFF STA
Weight 2.5 2.5 1.0 1.5
Criteria evaluation direction 1 1 1 1
Threshold direction −1 1 1 1
Open in a new tab

5. RESULTS AND DISCUSSIONS

5.1. Derivation of intercriteria parameters and outranking using ELECTRE III‐GA process

Based on the additional information pointed out, we applied ELECTRE III to construct a fuzzy outranking relation. The credibility matrix indicates the reliability of the outranking hypothesis. Suppose the concordance index is greater than or equal to the discordance index for all criteria. In that case, the degree of credibility is equal to the concordance index. If the concordance index is strictly lower than the discordance index, then the degree of credibility is equal to the concordance index lowered to the importance of these discordances. Then, each alternative is linked with the other alternative with two arrows related to the credibility index. The distillation procedure is then used to outrank the alternatives. The name distillation has been chosen for the analogy with alchemists, who distill liquid mixtures to extract a magic ingredient. Moreover, we used Python to run the GA to exploit the outranking relation and derive a final ranking of the alternatives in decreasing the order of preferences. The computation in the GA was realised with the following parameters: chromosome_length = 10; population_size = 47; maximum_generation = 200. Final results can be presented as follows: A10 (P = 46; NP = 0), A16 (P = 44; NP = 1) and A41 (P = 44; NP = 1) having the least non‐preference score are at the most preferable alternatives while A42 (P = 1; NP = 43), A44 (P = 1; NP = 44) and A26 (P = 0; NP = 46) are considered as the least preferable alternatives. Alternatives like A7, A9, A33, and A37 share the same preference score of 40 (P = 40). Likewise, labels A1, A5, A8, A36, and A38 are preferred 35 times compared to other alternatives. A preference score of 21 is shared between labels A15, A17, and A25, while alternatives A27, A40, and A46 are preferred 15 times compared to others (Table 4).

TABLE 4.

Summary of outranking matrix

Label Vaccine or trial name P NP R
A10 BNT162 46 0 0
A16 INO‐4800 44 1 0
A41 UB‐612 44 1 0
A7 JNJ‐78436735 40 3 0
A9 mRNA‐1273 40 3 0
A33 COVAX‐19 40 3 0
A37 FINLAY‐ FR‐1 40 3 0
A1 CoronaVac 35 7 1
A5 Ad5‐nCoV 35 7 1
A8 NVX‐CoV2373 35 7 1
A36 FINLAY‐FR‐2 35 7 1
A38 EpiVacCorona 35 7 4
A34 ACTRN12620000674932 34 12 0
A18 ZyCoV‐D 33 13 0
A20 Covaxin 32 14 0
A45 LNP‐nCoVsaRNA 31 15 0
A47 NCT04450004 25 16 5
A2 NCT04053010 25 16 1
A3 BBIBP‐CorV 25 16 1
A21 NCT04473690 25 16 1
A28 GRAd‐COV2 25 16 1
A35 MVC‐COV1901 25 16 1
A29 NCT04552366 24 22 0
A15 ChiCTR2000039462 21 23 0
A17 Covidvax 21 23 0
A25 ACTRN12620000817943 21 23 0
A23 NCT04608305 19 26 1
A12 CVnCoV 18 26 2
A13 NCT04470609 18 27 1
A27 NCT04591717 15 29 0
A40 NCT04546841 15 29 0
A46 ChiCTR2000034112 15 29 0
A30 VXA‐CoV2‐1 14 32 0
A11 NCT04466085 11 33 2
A14 QazCovid‐in 11 33 1
A22 NCT04537208 11 33 1
A31 NCT04569383 10 36 0
A32 NCT04405908 9 37 0
A24 LUNAR‐COV19 (ARCT‐021) 7 38 0
A39 ChiCTR2000037518 7 38 0
A43 NCT04497298 6 40 0
A6 Sputnik V 4 41 1
A4 AZD1222 3 41 2
A19 GX‐19 2 42 2
A42 V590 1 43 2
A44 ChiCTR2000037782 1 44 1
A26 bacTRL‐Spike 0 46 0
Open in a new tab

Note: R means that an alternative a is not comparable to other alternatives; P refers to the preference of an alternative over the other alternatives; NP means how an alternative a is not preferred to the others.

5.2. Ranking results by TOPSIS method

From the ELECTRE III‐GA model, we extracted 15 actions based on Preference (P > 30) and Non‐preference (NP < 15), as presented in Table 5. Then, we used Excel Spreadsheet to determine the weighted normalised decision matrix and Python compiler to plot the Ideal and anti‐ideal solutions (Figure 3).

TABLE 5.

Input data extracted from ELECTRE III‐GA computation

Criteria Safety Efficacy Effectiveness Stability
Weight 0.25 0.25 0.25 0.25
Input alternatives A10 BNT162 99 98 95 90
A16 INO‐4800 99 96 90 85
A41 UB‐612 95 96 90 95
A7 JNJ‐78436735 99 95 95 90
A9 mRNA‐1273 95 95 90 90
A33 COVAX‐19 95 95 90 80
A37 FINLAY‐ FR‐1 97 95 90 90
A1 CoronaVac 96 94 95 80
A5 Ad5‐nCoV 99 94 90 90
A8 NVX‐CoV2373 98 94 90 95
A36 FINLAY‐ FR‐2 95 94 90 90
A38 EpiVacCorona 94 96 80 85
A34 ACTRN12620000674932 98 95 80 75
A18 ZyCoV‐D 94 94 80 75
A20 Covaxin 90 93 80 85
Open in a new tab

FIGURE 3.

FIGURE 3

Open in a new tab

Ideal solution, anti‐ideal solution, and closeness coefficient

Table 6 ranks the novel coronavirus vaccines. According to our model, the best COVID‐19 vaccine is BNT162b2, with a relative closeness of 0.96. It is designed by Pfizer, one of the world's largest pharmaceutical companies, based in New York, collaborating with German biotech company BioNTech. This vaccine candidate is based on the injection of extracts of the genetic material of a virus, in this case, messenger RNA (mRNA), into human cells. It stimulates the production of viral proteins that mimic the coronavirus, causing the immune system to recognise its presence. Any successful vaccine based on this technology would be the first mRNA vaccine approved for human use. This vaccine requires two doses 21 days apart.

TABLE 6.

Final ranking for the Covid‐19 vaccines selection problem

Vaccine name Origin
Di*
Di−
Ci*
 Result‐rank
BNT162 Germany/China/USA 0.00 0.07 0.96 1
INO‐4800 USA/International 0.01 0.05 0.79 6
UB‐612 Taiwan/USA 0.01 0.05 0.80 4
JNJ‐78436735 Belgium 0.01 0.05 0.80 3
mRNA‐1273 USA 0.01 0.07 0.92 2
COVAX‐19 Australia/South Korea 0.02 0.05 0.76 7
FINLAY‐FR‐1 Cuba 0.03 0.04 0.61 8
CoronaVac China 0.03 0.04 0.59 11
Ad5‐nCoV China 0.04 0.03 0.42 12
NVX‐CoV2373 USA 0.01 0.05 0.79 5
FINLAY‐FR‐2 Cuba 0.03 0.04 0.60 9
EpiVacCorona Russia 0.03 0.04 0.59 10
ACTRN12620000674932 Australia/USA 0.07 0.03 0.30 14
Open in a new tab

The second‐best vaccine showed a relative closeness of 0.92. It was introduced by the Massachusetts‐based biotech company, Moderna Therapeutics in collaboration with the National Institutes of Health under the name mRNA‐1273. This vaccine candidate also relies on injecting snippets of mRNA into human cells to trigger an immune response. This vaccine requires two doses, 4 weeks apart.

JNJ‐78436735 (relative closeness of 0.80) is the third‐best vaccine against COVID‐19. It is implemented by Johnson & Johnson, one of the world's largest multinational corporations, based in New Jersey, specialising in healthcare and pharmaceutical products. Johnson & Johnson is developing an adenovector vaccine, which introduces a piece of DNA from SARS‐CoV‐2 into the common cold‐causing adenovirus that has been genetically changed so that it cannot replicate in the body. This vaccine builds on the technology Johnson & Johnson used to develop an Ebola vaccine and vaccine candidates for Zika and HIV.

With a relative closeness of 0.80, UB‐612, which is the fourth‐best vaccine candidate, consists of the Spike protein S1 subunit Receptor Binding Domain (RBD) genetically fused to a single chain Fc domain of human IgG1 (S1‐RBD‐sFc), combined with proprietary peptides representing T helper (Th) and cytotoxic T‐cell (CTL) epitopes on S2 subunit, Membrane and Nucleocapsid structural protein components of SARS‐CoV‐2. It was built by COVAXX, a subsidiary of United Biomedical Inc (UBI) headquartered in the United States, in collaboration with German biotech company BioNTech Asia. COVAXX said the trial is partly supported by a grant from the Ministry of Health and Welfare in Taiwan of up to NTD 430 million (approximately $15 million). It is important to notice how well Taiwan Government's response to COVID‐19 spread. Taiwan has until now contained the spread of Covid‐19.

A biotechnology company named Novavax and based in Gaithersburg, Maryland, introduced the NVX‐CoV2373. Novavax (relative closeness of 0.79) has bioengineered the coronavirus spike proteins, the parts that help the virus invade cells but cannot replicate or cause COVID‐19. Its vaccine candidate combines those proteins into a knucklebone‐shaped nanoparticle. This can be injected with its proprietary Matrix‐M adjuvant—a compound that stimulates immune cells—to elicit an immune response. The vaccine is administered in two doses, 21 days apart.

INO‐4800 (relative closeness: 0.79), introduced by the American biotechnology company, Inovio Pharmaceuticals is the sixth‐best vaccine candidate against COVID‐19 according to our model. It is a DNA vaccine candidate matched to the novel coronavirus SARS‐CoV‐2. This vaccine contains the plasmid pGX9501, which encodes for the entire length of the Spike glycoprotein of SARS‐CoV‐2.

6. MANAGERIAL IMPLICATIONS

The proposed integrated evaluation framework can be used for a more effective assessment of coronavirus vaccines alternatives. Considering assessment is a multidimensional problem by its nature, a single model cannot explain the whole evaluation process for deciding on the most appropriate COVID‐19 vaccine candidate. Our study combined safety, efficacy, stability, implementation, and availability attributes in a developed vaccine evaluation model to address this gap.

The reason for integrating ELECTRE and TOPSIS in our model is their strength in evaluating complex interrelationships among various aspects and handling internal dependencies (Sangaiah et al., 2017). To check its validity, the model is assessed by experts to understand its advantages and drawbacks better. The results align with (Bartsch et al., 2020), stating that the appropriate vaccine against the disease should be at least 80% effective.

This developed evaluation framework can help healthcare professionals and researchers, especially in developing countries, make proper judgments and gain research information by examining five criteria. Thus, the model does not simply follow a traditional approach and can provide decision‐makers with a valuable tool to collect information in complex relationships that can be useful for the undecided.

This selection model provided in this study can be used worldwide. There are some differences that users should keep in mind when applying this model: the level of importance of the criteria may vary in different situations. This study can be helpful to researchers to understand this selection problem better theoretically and health officials to better design good evaluation systems.

7. CONCLUSIONS AND FUTURE RESEARCH

7.1. Conclusions

As the COVID‐19 pandemic continues to challenge the global healthcare system, calling for the development of a safe and effective vaccine to protect against infection is urgent when we write this paper. Research institutions and pharmaceutical companies around the globe have already embarked on the race to develop COVID‐19 vaccines, which are at different stages of development. World Health Organization and other databases identified more than 180 vaccines in pre‐clinical development. This paper aims to evaluate those available vaccine alternatives and select the most suitable option. In decision and evaluation processes, there are many factors involving subjective and qualitative judgments and require different complex factors. In such processes, MCDM methods can be effectively employed to choose the most suitable COVID‐19 vaccine correctly.

After performing a detailed state of the art and considering expert opinions, an evaluation model is developed. A hybrid MCDM approach based on ELECTRE III‐Genetic Algorithm integrated with TOPSIS methodology is considered for selecting the most appropriate vaccine against the pandemic. The obtained results showed that BNT162b2 (Pfizer), mRNA‐1273 (Moderna Therapeutics), JNJ‐78436735 (Johnson & Johnson), UB‐612 (COVAXX), and INO‐4800 (Inovio) are ranking among the top and best vaccines against COVID‐19 according to our model.

This study contributes to the literature by extending practical applications of ELECTRE, Genetic Algorithm, and TOPSIS. ELECTRE III method provides a simple and understandable analysis of the Concordance index; however, it is time‐consuming. A GA is developed because of its ability to disaggregate preferences of DM from holistic decisions. For picking an ideal and an anti‐ideal solution and comparing the distance of each of the alternatives, TOPSIS is the best model. Thus, for these reasons, combining ELECTRE III‐GA and TOPSIS provide successful results in reaching strategic decisions.

7.2. Limitations and future research

Although we have shown an essential contribution to the literature, it is important to notice some limitations that future research can consider. First, the compilation of 108 vaccine alternatives following the WHO framework was systematic. Still, our assessment of the inclusion criteria is based on our subjective judgement. Therefore, some critical criteria such as supply chain, environment, and race are missing in this review. Future work can extend this framework by considering other criteria and other databases to perform assessment and evaluation on COVID‐19 vaccines.

A second limitation is sensitivity analysis. A sensitivity analysis is essential to test the robustness of the results obtained from the decision model by varying the values of the weights and thresholds and observing the effect on the outcome. Subsequent work may consider adding a robustness test to check the overall validity of this model. These limitations can provide future research avenues and can step on the contributions established by this paper.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

ACKNOWLEDGEMENTS

The authors are grateful to the anonymous referees for their most helpful suggestions that have improved the quality and legibility of this article.

Forestal, R. L. , & Pi, S.‐M. (2022). A hybrid approach based on ELECTRE III‐genetic algorithm and TOPSIS method for selection of optimal COVID‐19 vaccines. Journal of Multi‐Criteria Decision Analysis, 29(1‐2), 80–91. 10.1002/mcda.1772

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.

REFERENCES

  1. Abdelwahab, S. F. , Issa, U. H. , & Ashour, H. M. (2021). A novel vaccine selection decision‐making model (VSDMM) for COVID‐19. Vaccine, 9(7), 718 Retrieved from https://www.mdpi.com/2076-393X/9/7/718 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Abdullahi, I. N. , Emeribe, A. U. , Adekola, H. A. , Abubakar, S. D. , Dangana, A. , Shuwa, H. A. , Nwoba, S. T. , Mustapha, J. O. , Haruna, M. T. , Olowookere, K. A. , Animasaun, O. S. , Ugwu, C. E. , Onoja, S. O. , Gadama, A. S. , Mohammed, M. , Daneji, I. M. , Amadu, D. O. , Ghamba, P. E. , Onukegbe, N. B. , … Ahmad, A. E. (2021). Leveraging on the genomics and immunopathology of SARS‐CoV‐2 for vaccines development: Prospects and challenges. Human Vaccines & Immunotherapeutics, 17(3), 620–637. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aiello, G. , La Scalia, G. , & Enea, M. (2013). A non dominated ranking multi objective genetic algorithm and electre method for unequal area facility layout problems. Expert Systems with Applications, 40(12), 4812–4819. 10.1016/j.eswa.2013.02.026 [DOI] [Google Scholar]
  4. Andrew, M. K. , & McNeil, S. A. (2021). Influenza vaccination and the evolution of evidence‐based recommendations for older adults: A Canadian perspective. Vaccine, 39, A36–A41. 10.1016/j.vaccine.2020.09.011 [DOI] [PubMed] [Google Scholar]
  5. Angelis, A. , Baltussen, R. , & Tervonen, T. (2021). The need for novel approaches in assessing the value of COVID‐19 vaccines. American Journal of Public Health, 111(2), 205–208. 10.2105/ajph.2020.306066 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bartsch, S. M. , O'Shea, K. J. , Ferguson, M. C. , Bottazzi, M. E. , Wedlock, P. T. , Strych, U. , McKinnell, J. , Siegmund, S. S. , Cox, S. N. , Hotez, P. J. , & Lee, B. Y. (2020). Vaccine efficacy needed for a COVID‐19 coronavirus vaccine to prevent or stop an epidemic as the sole intervention. American Journal of Preventive Medicine, 59(4), 493–503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Batur Sir, G. D. (2021). Pain treatment evaluation in COVID‐19 patients with hesitant fuzzy linguistic multicriteria decision‐making. Journal of Healthcare Engineering, 2021, 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Begum, J. , Mir, N. A. , Dev, K. , Buyamayum, B. , Wani, M. Y. , & Raza, M. (2021). Challenges and prospects of COVID‐19 vaccine development based on the progress made in SARS and MERS vaccine development. Transboundary and Emerging Diseases, 68(3), 1111–1124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Carrillo, P. Á. , López, J. L. , & Chavira, D. G. (2017). Deriving parameters and preferential model for a total order in ELECTRE III. Paper presented at the 2017 4th International Conference on Systems and Informatics (ICSAI).
  10. Chen, P. (2021). Effects of the entropy weight on TOPSIS. Expert Systems with Applications, 168, 114186. [Google Scholar]
  11. Chen, S.‐J. , & Hwang, C.‐L. (1992). Fuzzy multiple attribute decision making methods Lecture notes in economics and mathematical systems, (pp. 289–486). Berlin, Heidelberg: Springer. [Google Scholar]
  12. Chen, Y. , Cheng, L. , Lian, R. , Song, Z. , & Tian, J. (2021). COVID‐19 vaccine research focusses on safety, efficacy, immunoinformatics, and vaccine production and delivery: A bibliometric analysis based on VOSviewer. BioScience Trends., 15, 64–73. [DOI] [PubMed] [Google Scholar]
  13. Clemente‐Suárez, V. J. , Navarro‐Jiménez, E. , Ruisoto, P. , Dalamitros, A. A. , Beltran‐Velasco, A. I. , Hormeño‐Holgado, A. , Laborde‐Cárdenas, C. C. , & Tornero‐Aguilera, J. F. (2021). Performance of fuzzy multi‐criteria decision analysis of emergency system in covid‐19 pandemic. an extensive narrative review. International Journal of Environmental Research and Public Health, 18(10), 5208. 10.3390/ijerph18105208 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Cohen, C. , White, J. M. , Savage, E. J. , Glynn, J. R. , Choi, Y. , Andrews, N. , Brown, D. , & Ramsay, M. E. (2007). Vaccine effectiveness estimates, 2004–2005 mumps outbreak, England. Emerging infectious diseases, 13(1), 12–17. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dadari, I. K. , & Zgibor, J. C. J. J. O. G. H. (2021). How the use of vaccines outside the cold chain or in controlled temperature chain contributes to improving immunization coverage in low‐and middle‐income countries (LMICs): A scoping review of the literature. [DOI] [PMC free article] [PubMed]
  16. Dumpa, N. , Goel, K. , Guo, Y. , McFall, H. , Pillai, A. R. , Shukla, A. , Repka, M. A. & Murthy, S. N. (2019). Stability of vaccines. AAPS PharmSciTech, 20(2), 1–11. 10.1208/s12249-018-1254-2 [DOI] [PubMed] [Google Scholar]
  17. Enayatkhani, M. , Hasaniazad, M. , Faezi, S. , Gouklani, H. , Davoodian, P. , Ahmadi, N. , Einakian, M. A. , Karmostaji, A. , & Ahmadi, K. (2021). Reverse vaccinology approach to design a novel multi‐epitope vaccine candidate against COVID‐19: An in silico study. Journal of Biomolecular Structure and Dynamics, 39(8), 2857–2872. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Faasse, K. , Newby, J. (2020). Public perceptions of covid‐19 in australia: perceived risk, knowledge, health‐protective behaviors, and vaccine intentions. Frontiers in Psychology, 11, 1–11. 10.3389/fpsyg.2020.551004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Figueira, J. R. , Mousseau, V. , & Roy, B. (2016). ELECTRE methods. In Greco S., Ehrgott M. & Figueira J. R. (Eds.), Multiple criteria decision analysis (2nd ed., pp. 155–185). Springer. [Google Scholar]
  20. Head, K. J. , Kasting, M. L. , Sturm, L. A. , Hartsock, J. A. , & Zimet, G. D. (2020). A national survey assessing SARS‐CoV‐2 vaccination intentions: Implications for future public health communication efforts. Science Communication, 42(5), 698–723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hezam, I. M. , Nayeem, M. K. , Foul, A. , & Alrasheedi, A. F. (2021). COVID‐19 vaccine: A neutrosophic MCDM approach for determining the priority groups. Results in Physics, 20, 103654. 10.1016/j.rinp.2020.103654 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Jakhar, J. , Naik, S. S. , & Prasad, S. (2021). COVID‐19 vaccination prioritization for persons with severe mental illness. Asian Journal of Psychiatry, 64, 102792–102792. 10.1016/j.ajp.2021.102792 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Khurana, A. , Allawadhi, P. , Khurana, I. , Allwadhi, S. , Weiskirchen, R. , Banothu, A. K. , Chhabra, D. , Joshi, K. , & Bharani, K. K. (2021). Role of nanotechnology behind the success of mRNA vaccines for COVID‐19. Nano Today, 38, 101142. 10.1016/j.nantod.2021.101142 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Kumar, N. , Admane, N. , Kumari, A. , Sood, D. , Grover, S. , Prajapati, V. K. , Chandra, R. , & Grover, A. (2021). Cytotoxic T‐lymphocyte elicited vaccine against SARS‐CoV‐2 employing immunoinformatics framework. Scientific Reports, 11(1), 7653. 10.1038/s41598-021-86986-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lee, B. Y. , Wedlock, P. T. , Haidari, L. A. , Elder, K. , Potet, J. , Manring, R. , Connor, D. L. , Spiker, M. L. , Bonner, K. , Rangarajan, A. , Hunyh, D. & Brown, S. T. (2017). Economic impact of thermostable vaccines. Vaccine, 35(23), 3135–3142. 10.1016/j.vaccine.2017.03.081 [DOI] [PMC free article] [PubMed]
  26. Leyva‐López, J. C. , & Fernández‐González, E. (2003). A new method for group decision support based on ELECTRE III methodology. European Journal of Operational Research, 148(1), 14–27. 10.1016/S0377-2217(02)00273-4 [DOI] [Google Scholar]
  27. Majumder, P. , Biswas, P. , & Majumder, S. (2020). Application of new topsis approach to identify the most significant risk factor and continuous monitoring of death of covid‐19. Electronic Journal of General Medicine, 17(6), 1–12. 10.29333/ejgm/7904 [DOI] [Google Scholar]
  28. Malabadi, R. B. , Meti, N. T. , & Chalannavar, R. K. (2021). Applications of nanotechnology in vaccine development for coronavirus (SARS‐CoV‐2) disease (Covid‐19). International Journal of Research and Scientific Innovations, 8(2), 191–198. [Google Scholar]
  29. Marchetti, D. , & Wanke, P. (2020). Efficiency of the rail sections in Brazilian railway system, using TOPSIS and a genetic algorithm to analyse optimized scenarios. Transportation Research Part E: Logistics and Transportation Review, 135, 101858. [Google Scholar]
  30. Mehrotra, D. V. , Janes, H. E. , Fleming, T. R. , Annunziato, P. W. , Neuzil, K. M. , Carpp, L. N. , Benkeser, D. , Brown, E. R. , Carone, M. , Cho, I. , Donnell, D. , Fay, M. P. , Fong, Y. , Han, S. , Hirsch, I. , Huang, Y. , Huang, Y. , Hyrien, O. , Juraska, M. , & Gilbert, P. B. (2021). Clinical endpoints for evaluating efficacy in covid‐19 vaccine trials. Annals of Internal Medicine, 174(2), 221–228. 10.7326/m20-6169 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mohammed, M. A. , Abdulkareem, K. H. , Al‐Waisy, A. S. , Mostafa, S. A. , Al‐Fahdawi, S. , Dinar, A. M. , Alhakami, W. , Baz, A. , Al‐Mhiqani, M. N. , Alhakami, H. , Arbaiy, N. , Maashi, M. S. , Mutlag, A. A. , Garcia‐Zapirain, B. , & De La Torre Diez, I. (2020). Benchmarking methodology for selection of optimal covid‐19 diagnostic model based on entropy and topsis methods. IEEE Access, 8, 99115–99131. 10.1109/access.2020.2995597 [DOI] [Google Scholar]
  32. Moore, S. , Hill, E. M. , Dyson, L. , Tildesley, M. J. , & Keeling, M. J. (2021). Modelling optimal vaccination strategy for SARS‐CoV‐2 in the UK. PLoS Computational Biology, 17(5), e1008849. 10.1371/journal.pcbi.1008849 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Ni, L. , Chen, Y.‐W. , & de Brujin, O. (2021). Towards understanding socially influenced vaccination decision making: An integrated model of multiple criteria belief modelling and social network analysis. European Journal of Operational Research, 293(1), 276–289. [Google Scholar]
  34. Nichols, M. K. , Andrew, M. K. , Hatchette, T. F. , Ambrose, A. , Boivin, G. , Bowie, W. , Chit, A. , Dos Santos, G. , ElSherif, M. , Green, K. , Haguinet, F. , Halperin, S. A. , Ibarguchi, B. , Johnstone, J. , Katz, K. , Lagacé‐Wiens, P. , Langley, J. M. , LeBlanc, J. , Loeb, M. , … McNeil, S. A. (2018). Influenza vaccine effectiveness to prevent influenza‐related hospitalizations and serious outcomes in Canadian adults over the 2011/12 through 2013/14 influenza seasons: A pooled analysis from the Canadian Immunization Research Network (CIRN) Serious Outcomes Surveillance (SOS Network). Vaccine, 36(16), 2166–2175. 10.1016/j.vaccine.2018.02.093 [DOI] [PubMed] [Google Scholar]
  35. Orenstein, W. A. , Bernier, R. H. , Dondero, T. J. , Hinman, A. R. , Marks, J. S. , Bart, K. J. , & Sirotkin, B. (1985). Field evaluation of vaccine efficacy. Bulletin of the World Health Organization, 63(6), 1055–1068. [PMC free article] [PubMed] [Google Scholar]
  36. Oyarzun, P. , Kashyap, M. , Fica, V. , Salas‐Burgos, A. , Gonzalez‐Galarza, F. F. , McCabe, A. , Jones, A. R. , Middleton, D. , & Kobe, B. (2021). A proteome‐wide immunoinformatics tool to accelerate t‐cell epitope discovery and vaccine design in the context of emerging infectious diseases: an ethnicity‐oriented approach. Frontiers in Immunology, 12, 1–25. 10.3389/fimmu.2021.598778 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rohaninejad, M. , Kheirkhah, A. , Fattahi, P. , & Vahedi‐Nouri, B. (2015). A hybrid multi‐objective genetic algorithm based on the ELECTRE method for a capacitated flexible job shop scheduling problem. The International Journal of Advanced Manufacturing Technology, 77(1–4), 51–66. [Google Scholar]
  38. Roy, B. (2013). Multicriteria methodology for decision aiding (Vol. 12). Springer Science & Business Media. [Google Scholar]
  39. Saha, R. , Ghosh, P. , & Burra, V. P. (2021). Designing a next generation multi‐epitope based peptide vaccine candidate against SARS‐CoV‐2 using computational approaches. Biotech, 11(2), 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Sangaiah, A. K. , Gopal, J. , Basu, A. , Subramaniam, P. R. , Sangaiah, A. K. , Gopal, J. , Basu, A. & Subramaniam, P. R. (2017). An integrated fuzzy DEMATEL, TOPSIS, and ELECTRE approach for evaluating knowledge transfer effectiveness with reference to GSD project outcome. Neural Computing and Applications, 28(1), 111–123. 10.1007/s00521-015-2040-7 [DOI] [Google Scholar]
  41. Sayan, M. , Sarigul Yildirim, F. , Sanlidag, T. , Uzun, B. , Uzun Ozsahin, D. , & Ozsahin, I. (2020). Capacity evaluation of diagnostic tests for COVID‐19 using multicriteria decision‐making techniques. Computational and Mathematical Methods in Medicine, 8, 1560250. 10.1155/2020/1560250 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Sheppard, C. (2017). Genetic algorithms with python. Smashwords Edition. [Google Scholar]
  43. Vafaeinejad, A. (2016). Cropping pattern optimization by using of TOPSIS and genetic algorithm based on the capabilities of GIS. Iranian journal of Ecohydrology, 3(1), 69–82. [Google Scholar]
  44. Whelan, S. F. (2009). Mortality in Ireland at advanced ages, 1950‐2006: Part 1: Crude rates. Annals of Actuarial Science, 4(1), 33–66. [Google Scholar]
  45. World Health Organization . (2018). Controlled temperature chain (CTC)‐beyond the traditional cold chain. Author. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Articles from Journal of Multi-Criteria Decision Analysis are provided here courtesy of Wiley

RESOURCES