Rise of multiattribute decision‐making in combating COVID‐19: A systematic review of the state‐of‐the‐art literature

Abstract

Considering the coronavirus disease 2019 (COVID‐19) pandemic, the government and health sectors are incapable of making fast and reliable decisions, particularly given the various effects of decisions on different contexts or countries across multiple sectors. Therefore, leaders often seek decision support approaches to assist them in such scenarios. The most common decision support approach used in this regard is multiattribute decision‐making (MADM). MADM can assist in enforcing the most ideal decision in the best way possible when fed with the appropriate evaluation criteria and aspects. MADM also has been of great aid to practitioners during the COVID‐19 pandemic. Moreover, MADM shows resilience in mitigating consequences in health sectors and other fields. Therefore, this study aims to analyse the rise of MADM techniques in combating COVID‐19 by presenting a systematic literature review of the state‐of‐the‐art COVID‐19 applications. Articles on related topics were searched in four major databases, namely, Web of Science, IEEE Xplore, ScienceDirect, and Scopus, from the beginning of the pandemic in 2019 to April 2021. Articles were selected on the basis of the inclusion and exclusion criteria for the identified systematic review protocol, and a total of 51 articles were obtained after screening and filtering. All these articles were formed into a coherent taxonomy to describe the corresponding current standpoints in the literature. This taxonomy was drawn on the basis of four major categories, namely, medical (n = 30), social (n = 4), economic (n = 13) and technological (n = 4). Deep analysis for each category was performed in terms of several aspects, including issues and challenges encountered, contributions, data set, evaluation criteria, MADM techniques, evaluation and validation and bibliography analysis. This study emphasised the current standpoint and opportunities for MADM in the midst of the COVID‐19 pandemic and promoted additional efforts towards understanding and providing new potential future directions to fulfil the needs of this study field.

1. INTRODUCTION

The coronavirus disease 2019 (COVID‐19) caused by severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) is currently transmitting dramatically worldwide, thereby resulting in millions of infections and deaths among the human population. ¹ Similar to all global crises in human history, the pandemic is causing unprecedented health and economic disruptions in numerous countries. ² , ³ As a result, centres for disease control and prevention have recommended avoiding any gatherings with more than a specific number of people, including all forms of social activities, schools, and events globally. ⁴ The majority of countries worldwide have imposed movement restriction orders. ⁵ COVID‐19 is not the first global pandemic to hit the world ⁶ but is the first one to cause such damage with respect to the economy and loss of human lives. ⁷ Several different viruses and pandemics, including Ebola, ⁸ Middle East respiratory syndrome coronavirus and SARS, have emerged in the past. However, all of these so‐called pandemics did not have an impact as serious as that of COVID‐19. ⁹ Considering the harm that this virus has caused, the whole scientific community is working continuously towards assisting and helping one another in such times. ¹⁰ Medical doctors and frontliners across the globe are dealing with the pandemic and cases at the expense of their health and even lives. ¹¹ Scientists from all domains are proposing solutions to mitigate the effects of this global issue. ¹² Accordingly, we were motivated to ask an important question, which is presented as follows.

1.1. Can digital technology help in mitigating the effects of the COVID‐19 pandemic?

The new situation is favouring the transition to digital solutions in numerous industries and society as a whole from computer science to artificial intelligence (AI), ¹³ , ¹⁴ machine learning (ML), ¹⁵ and deep learning ¹⁶ to assist with medical decisions and from social science comes solutions to people with behavioural issues from being locked up because of the pandemic. ¹⁷ From industrial science, strategies and maintain the economy have also been proposed. ¹⁸ Thousands of published studies across all scientific domains are dedicated to COVID‐19 and are proposing ideas, solutions, and means to mitigate the effect of the virus. ¹⁹ Such widespread participation shows people's solicitude in harnessing their ultimate capabilities to assist with their respected domains. ²⁰ Although those digital solutions have shown considerable significance in mitigating the pandemic, governments and decision makers are sometimes hesitant to make decisions. ²¹ In this context, another important question is raised as follows.

1.2. Why is decision‐making a critical challenge during the COVID‐19 pandemic?

Closing a country's economy, shutting down operations or even formulating decisions in such times will not only affect the country's economy but might also leave the population vulnerable and endangered. ²² At the same time, governments are not only affected by the difficulty in making such big decisions. At times, even researchers are also incapable of making sound decisions, particularly when several concerns are at stake. ²³ Considering this circumstance, certain decisions will have varying effects if they are applied in different settings or even different countries, particularly given the variations in the priorities of people and governments across the globe. Decision‐making in itself is no easy task, and every aspect contributing to the action must be carefully measured and considered before the action is applied. Although such consideration can be impossible for each individual to do, this consideration is not impossible for decision support systems once they are fed with the appropriate parameters and aspects to enforce any resolution. ²⁴ However, towards that end, we have to answer another question, which is formulated as follows.

1.3. How to make a sound decision and what is the suitable technique to achieve that?

The field of decision science is uniquely concerned with making optimal choices based on available information, and its most appropriate approach is known as multicriteria decision‐making (MCDM). ²⁵ This technique assists in making decisions, particularly when different parameters and settings might shift between the significance of a decision. The most acceptable classification for the MCDM approach has been made on the basis of two main categories, namely, multiple objective decision‐making (MODM) and multiple‐attribute decision‐making (MADM). ²⁶ Such classification considered different purposes and data types. From a practical viewpoint, the MODM is not associated with problems in which the alternatives have been preidentified. ²⁷ , ²⁸ In this case, the main concern is to plan/design the most suitable alternative with respect to limited resources. ²⁹ , ³⁰ By contrast, MADM is associated with problems in which the alternatives must be identified. In this category, the main concern is to provide a selection/prioritisation/ranking for a finite number of alternative options. ³¹ , ³² Undoubtedly, the profound capabilities of both groups shine most when needed. Those groups stand on the shoulders of giants during difficult times, and no other time is greater than what we are experiencing right now in the COVID‐19 pandemic.

When reviewing the academic literature of MODM, different interesting studies have been published to combat the COVID‐19 pandemic. Considering the medical waste management during the pandemic, reference ³³ developed a novel model called mixed‐integer linear programming for formulating the sustainable multitrip location‐routing problem with time windows. Two sequential objectives were achieved by reference. ³⁴ Such objectives are to examine the related literature while systematically focusing on a process approach used in pandemic planning and analyse the pandemic research and prepare a case study for the Czech Republic's pandemic plan using a process approach. Reference ³⁵ presented a deep model approach based on a multilayer long short‐term memory network for forecasting medical equipment demand and spread the virus during the COVID‐19 outbreak. The authors of [36] explained, formulated and solved the problem of sustainable medical waste management for the COVID‐19 pandemic.

From another perspective, several studies declared that the impact of government strategies on fighting the current pandemic has been majorly assisted by MADM methods. Such methods have been proven effective in evaluating and/or selecting various aspects of healthcare data‐focused applications. ¹⁰ According to the aforementioned significance of MODM and MADM methods in making as much as possible optimum decisions, another question has been raised.

1.4. What is the research scope of this systematic review?

To comprehensively analyse the academic literature studies based on an interconnected and sequential story, the scope of this review exclusively focuses on utilising the MADM methods in different aspects of the COVID‐19 pandemic. The reason behind such focus is the significant contributions of research efforts that were performed by scientists by evaluating/benchmarking/selecting different perspectives during the COVID‐19 pandemic. In this regard, MADM can be formulated by constructing a decision matrix on the basis of an intersection between the aspects or decisions (i.e., alternatives). That is, a person would like to select their evaluation criteria based on importance, defined as their weights. ³⁷ , ³⁸ Their selection can also be represented as the outcome of mental and reasoning processes to determine the most suitable option from several alternatives concerning predefined attributes or criteria. ³⁹ MADM techniques are used in situations that require selecting the most appropriate alternative among several candidates or even ranking or prioritising them on the basis of different evaluation criteria. In its essence, MADM is used in several cases to assist decision makers having difficulty expressing a specific preference for relevant alternatives under several criteria, mainly when relying on unreliable, ambiguous or incomplete information. ⁴⁰ , ⁴¹ , ⁴² , ⁴³ MADM is unique in finding suitable decision issues with appropriate solutions. This aspect enables MADM to rise across multiple current means for addressing real‐life issues, specifically when several factors are influencing the decision‐making aspect.

In sum, the role of MADM techniques in solving complex real‐world problems controlled by multiattribute to come out with appropriate decisions among available alternatives encouraged researchers to utilise MADM in enormous applications and study it more precisely. The COVID‐19 outbreak that affected all life facilities, mainly the health system, led to an unprecedented situation, where decision makers need to make a fast and correct decision. Furthermore, MADM has enormous benefits as support for decision science and has been used in the context of COVID‐19 with different applications, such as logistics ⁴⁴ and health. ¹⁰ Identifying all these areas of COVID‐19 where MADM has been used will enable researchers to assess its usage and determine its integration to solve real‐world problems in the presence of the pandemic. However, another technical question related to MADM has been raised as follows:

1.5. How to perform MADM? In other words, what are the methodological steps of MADM?

In terms of how MADM is done, scientists have both agreed and disagreed on the steps needed in this process. Nevertheless, the main flow (Figure 1) of using MADM can be illustrated as follows:

• Decision goals: In this step, a decision maker or a person needs decision assistance by setting goals that suit the decision‐making. The goals can be related to selecting and prioritising among alternatives or even weighting a set of criteria.

• Identification of alternatives: In this step, a decision maker will make up his/her mind on the items he/she needs to favour or choose from.

• Identification of evaluation criteria: In this step, we will determine how to measure the criteria that are to be used in the MADM process. Some criteria are calculated on the basis of a fixed measured value, whereas others are subjectively known. Hence, such a process has to be identified.

• Construction of the decision matrix: In this step, a decision matrix will be constructed, which includes all the alternative options to which the authors want to apply MADM and their corresponding influencing criteria.

• Selection of MADM methods: In this step, the most suited MADM technique will be selected. Some MADM methods assign weight (importance) levels to criteria, whereas other techniques deal with the matrix straight away considering that weight is already defined. Some MADM approaches can do both. Thus, selecting the most suitable approach for a specific case is a worthy aspect of consideration.

• Normalisation and/or weighting matrices: In this step, the entire matrix is filled with data for both alternatives and their criteria. The data come in various scales and types. Thus, this step aims to normalise the matrix to unify it for the MADM selection process mentioned previously. Moreover, in the criteria weighting performed by MADM weighting methods, the obtained weights for the evaluation criteria can be used and multiplied with the normalised matrix to produce a weighted matrix.

• Goal: In this step, the goal of the decision maker is finally resolved by the MADM technique, and the decision maker's desirable outcome is finally apparent across different alternatives/criteria from which he/she was sceptical to select.

Figure 1.

MADM procedure. MADM, multiattribute decision‐making

After answering the aforementioned questions, let us close up this section by asking a final important question, which is presented as follows:

1.6. What are the research novelty and contributions?

To the best of the authors' knowledge, no research attempted to comprehensively review the use of MADM in COVID‐19 applications. Thus, the motive behind the work presented in this study is to introduce an intensive analysis of the MADM techniques and demonstrate all the current work directions in the context of COVID‐19 to provide an exhaustive analysis for researchers. The contributions of this study are as follows:

• 1.A comprehensive systematic review and analysis based on the systematic literature review (SLR) protocol are presented to categorise and taxonomise the related literature review into four main categories (i.e., medical, social, economic and technological).
• 2.The issues and challenges related to other COVID‐19 case studies or the theoretical level of MADM methods are determined.
• 3.Numerous statistical analysis results for the data sets used are presented to cover the indispensable role of MADM techniques in handling the unprecedented effect of the COVID‐19 pandemic.
• 4.The types of evaluation criteria in the context of MADM based on COVID‐19 with their application in each category and subcategory of the proposed taxonomy are mapped.
• 5.The originality of MADM methods and types with the trend of extending or developing new techniques aligned with the issues and challenges is discussed. The objective is to understand how such methods assist decision makers and researchers across all scientific domains in combating the global issue.
• 6.Various validation and evaluation approaches are investigated and discussed to prove the reliability of MADM.
• 7.Bibliography analysis in terms of nationality, journals and MADM environments (i.e., crisp and fuzzy) is presented.
• 8.Potential future directions and key solutions are presented to solve the uncovered points, particularly in the medical sector.

2. SLR PROTOCOL

In this study, the SLR protocol steps were followed. This protocol helps in achieving an exhaustive understanding of the research interest and enriches future studies with further information. Moreover, the well‐structured process of SLR compared with traditional review approaches improves the research synthesis in identifying relevant studies based on the identified metrics. The SRL approach is considered an advanced method because of its wide impact on various research fields and scientific disciplines. This approach is mainly composed of a set of processes, including research scope identification, searching mechanism, study selection and extraction and synthesis of information.

2.1. Information source

The collected information for this study was based on the strategical search method followed by the SLR and meta‐analysis (PRISMA) phases, as shown in Figure 2. Four reliable database search engines were used for searching, filtering, extracting and drafting this survey: (1) 'Scopus', which contains numerous research publications on different scientific domains; (2) 'IEEE Xplore', which includes publication work on multidisciplinary technologies from different domains; (3) 'ScienceDirect', which provides wide access to research works from various academic fields; (4) 'Web of Science' which contains a wide spectrum of works in many fields, such as social sciences, arts, and humanities. The selected databases have widely appeared in numerous published SLRs in high‐impact scientific journals, which have academic resilience and scientific soundness. Thus, these databases were considered adequate and most suitable for this review.

Figure 2.

Systematic literature review protocol

2.2. Search strategy

The search was carried out on 30 November 2020, followed by two other iterative search rounds on 27 December 2020 and 16 April 2021 to ensure that more updated and recent literature was included. The search was implemented using the advanced search boxes of the search engine databases mentioned above. Boolean operators were used for the search (i.e., AND and OR), and two groups of keywords (i.e., queries) were used in the process, as shown in Figure 2. The previous process was performed to retrieve the most relevant articles. In searching and filtration, the content based on various types of publications, such as journal articles, conference papers, reviews and research articles, was selected. This option was considered efficient for covering the most recent and related publications in the designated topic of this review.

2.3. Study selection

The research procedure was composed of three succeeding steps, namely, collection of articles, scanning the title and abstract and full‐text reading. First, the initial number of collected articles from the nominated databases was 879, and duplicate articles (n = 26) were scanned across all the databases. Second, the abstracts and titles of all extracted articles were scanned on the basis of the inclusion criteria discussed in Section ‎2.4 to identify the relevant articles and verify their relevance. All articles that matched the inclusion criteria would be included in the final round. Lastly, full‐text reading was performed. Any study that did not meet the identified criteria was excluded from this review. Among the collected articles, 720 were irrelevant papers and thus were excluded. The final set of articles that complied with the identified criteria of this review was 51. Full‐text reading was performed to extract useful and valuable information (i.e., data extracted) to be analysed for this review.

2.4. Inclusion and exclusion criteria

The defined inclusion and exclusion criteria were imposed while attempting to identify the most related articles during the study selection process. The date of publication was set from the time that COVID‐19 made its presence in 2019 until April 2021. According to the additional criteria, all papers, including reviews and research papers, were limited to those in the English language across all the selected databases. The remaining criteria were concerned with the inclusion of all papers on COVID‐19 based on MADM techniques, which focused on discussing how MADM was integrated into different applications during the COVID‐19 pandemic.

3. LITERATURE REVIEW ANALYSIS

In this stage, 51 articles were classified into four major categories: (1) medical (n = 30), (2) social (n = 4), (3) economic (n = 13), and (4) technological (n = 4). These major categories were linked to their corresponding subcategories while considering the nature of their contribution, where MADM was applied in the presence of COVID‐19, as shown in Figure 3. Furthermore, several sequential analyses were performed to the MADM‐based COVID‐19 taxonomy. These analyses included issues and challenges (Section 3.1), contribution and description for each included study in such taxonomy (Section 3.2), the data set used in the literature (Section 3.3), evaluation criteria used (Section 3.4), MADM technique used (Section 3.5), validation and evaluation approaches used (Section 3.6) and bibliography analysis for such literature review (Section 3.7).

Figure 3.

Taxonomy of multiattribute decision‐making‐based COVID‐19 studies [Color figure can be viewed at wileyonlinelibrary.com]

3.1. Issues and challenges encountered

In this section, the limitations and issues of decision science in the literature review should be identified to understand the role of MADM with respect to the presence of COVID‐19 application. By knowing its issues, researchers can either take one of the two different approaches while using MADM. (1) They can begin by working on MADM at the theoretical level and attempt to mitigate the issues of the techniques. Alternatively, they can develop new ones that aim to solve the existing issues in previous approaches. ⁴⁵ , ⁴⁶ (2) The second group of scientists can opt to work with some of the issues arising while attempting to apply MADM in different applications. ⁴⁷ , ⁴⁸ In general, most of the general issues and problems of MADM can be identified from the following:

• Multiattribute: a complex decision issue, where a decision is influenced by several factors that cannot be placed on a single scale (e.g., all good or all bad). A conflict among these criteria causes difficulty for humans when making decisions. ⁴⁹

• Inconsistency: an MADM challenge that mostly occurs in pairwise comparison approaches in practice, wherein a decision maker gives inconsistent comparisons when determining the weight. ⁴⁹

• Time consumption in a pairwise comparison: an MADM issue, which shows that conducting a comparison is lengthy and at times requires proper explanation to the person making the comparison. ⁵⁰

• Unnatural comparison: comparing two different quantities is not a natural process. In such a case, a decision‐making expert has difficulty providing his/her judgment. Hence, the comparison must be a natural process based on similar quantities to allow experts to have a more instinctive and easier decision‐making process. ¹⁰

• Vagueness: an MADM issue also known as fuzziness, which is an inherent property of the decision‐making process. 'It arises as a result of the psychological perspective of humans or due to the impreciseness in the physical nature of a problem. Along with this property, another psychological aspect, namely, the nondeterministic nature of the decision maker measured by priority index is also closely associated with the process of decision‐making'. ⁵¹

• Normalisation: using different techniques for the normalisation process might provide different ranking/weighting results. Thus, different evaluation scales must be unified, and their values must be converted into unitless numbers. ¹⁰

• Distance measurement: this issue is related to the distance measurement between the ideal solution and the alternatives established. Some MADM methods identify the costs and benefits of evaluation attributes using the Euclidean distance between separate attributes and the ideal solution. Conducting two different references for comparing alternatives cause this issue. ¹⁰

• Outranking: the issue of rank preservation and reversal is the basis to ensure the effectiveness of MADM methods, which is often a critical issue to make life more organised and orderly. In this issue, prioritising the alternatives is inverted when an alternative is added or eliminated from the list of the established alternatives. ⁵¹

• Trade‐off: a situational decision that causes reducing or eliminating one quality, quantity or trait of a set or design in consideration of improvements in other aspects. In other words, a trade‐off is when one evaluation criterion increases, the other must decrease without any interconnected relation between them. ⁵²

• Conflict criteria: this issue is most similar to the trade‐off issue. However, the difference is that the relationship among the evaluation criteria exists, and one criterion might be affected by others. ⁵²

• Importance of criteria: as mentioned before, the decision in MADM shall be made on the basis of different evaluation criteria. However, the importance of such criteria varies from one to another according to the expert preferences as a subjective concept or even on the basis of the nature of the data of each criterion as an objective principle. ⁵²

• Data variation: The data variation issue in MADM occurs when some alternatives have defeated others and could be selected as the best ones according to some criteria, whereas other alternatives can be prioritised over the previous alternatives according to different criteria. ⁴⁹

Across all the screened and analysed studies, most of the abovementioned issues were apparent in the literature and identified, as presented in Table 1.

Table 1.

Issues and MADM challenges

Ref	Category	Subcategory	MADM issues											Challenge
			Multiattribute	Importance criteria	Data variation	Trade‐off	Conflict	Vagueness	Time consumed in comparison	Distance measurement	Unnatural comparison	Normalisation	Reverse rank
[49]	Medical	Strategies	×	×	×			×						Selecting a suitable strategic guideline for the reorganisation of healthcare systems on the hazard of pandemic spread and infection capability.
[51]			×	×			×	×					×	Evaluating and ranking of intervention strategy options applied by various countries during the COVID‐19 pandemic.
[53]			×	×			×	×						Scrutinising on designing lockdown relaxation efforts that consider public health and economic revival without documented experience to depend on.
[54]			×	×		×								Analysing and prioritising the activities for COVID‐19 infection control and prevention.
[55]		Policies	×	×			×							Developing public policies to eliminate the pressure of COVID‐19 on healthcare systems, which is commonly known as 'flattening the curve'.
[56]			×	×			×							Assessing the severity of the urban COVID‐19 epidemic situation involved in emergency medical management during the time of humanitarian catastrophe.
[57]			×	×			×	×						Determining the best alternatives that will help in choosing the safest path to overcome this deadly infection.
[58]		Tools	×	×	×			×						Selecting antivirus veils during the COVID‐19 pandemic is used as an example to show the feasibility and advantages of the proposed method.
[59]			×	×		×		×					×	Selecting an antivirus face mask as a protection measure for the COVID‐19 pandemic; the obtained results are aligned with existing ones.
[60]			×	×			×	×		×				Selecting the available COVID‐19 diagnostic tests.
[61]			×	×		×	×							Selecting the best COVID‐19 diagnosis model.
[62]			×	×			×	×						Diagnosing COVID‐19 by assessing the patient's symptoms and CT images.
[52]			×	×		×	×							Evaluating and benchmarking AI classification methods used for detecting COVID‐19 infection in the medical images.
[63]			×	×	×			×						Selecting authentic labs from available alternative labs for the COVID‐19 test.
[64]			×	×				×						Assessing the estimation for each feature in complex decision‐making problems to reduce the impact of the preposterous estimation and its application in antivirus mask selection.
[65]			×	×			×	×						Identifying the patient with COVID‐19 earlier is important to contain the fast spread of this virus.
[50]		Services	×	×			×	×	×				×	Prioritising the factors affecting the quality of hospital services from the viewpoint of patients and their companions in hospitals.
[37]			×	×		×								Prioritising patients with COVID‐19 based on multibiological laboratory test biomarkers.
[66]			×	×		×								Prioritising asymptomatic COVID‐19 patients.
[10]			×	×		×				×	×	×		Selecting the most suitable CP and prioritising patients with COVID‐19.
[67]			×					×						Exploring the most efficient method for surviving from COVID‐19 besides medical treatments.
[68]			×	×		×								Determining the appropriate CP for the most severe COVID‐19 cases in distributed hospital management aspects and ranking COVID‐19 patients and donors instantaneously.
[69]			×	×			×							Ranking the most urgent needed groups in society to prioritise the vaccine recipient order.
[70]			×	×		×								Prioritising COVID‐19 mild/not critical cases for hospital admission in healthcare settings with limited services and resources.
[71]			×	×			×	×						Identifying the hospital preparation level during disaster time is an important and necessary matter.
[72]			×	×				×				×		Determining the importance of decision makers in the hierarchical structure during the decision‐making process and its application in selecting the makeshift hospital during the COVID‐19 pandemic.
[73]			×	×			×	×						Evaluating and selecting the disposal healthcare waste techniques during and after the COVID‐19 pandemic.
[74]			×	×				×						Evaluating and selecting COVID‐19 medicine for mild patient cases by handling the unbalance and impact of experts' evaluation in relation to the individual risk tolerance.
[75]			×	×			×	×	×					Identifying and prioritising healthcare systems to deal with different types of wastes and their dimensions.
[76]			×	×	×			×						Selecting an appropriate antiviral therapy in treating the mild cases of COVID‐19.
[77]	Social		×					×						Analysing the reaction of various social factors meant to minimise the spread of the coronavirus.
[78]			×	×		×		×						Selecting an appropriate emergency means avoiding further escalation of the COVID‐19 crisis.
[79]			×	×										Identifying musculoskeletal disorders' risk level among handheld device users during the COVID‐19 pandemic as a result of work from home with an inappropriate workstation environment.
[80]			×	×				×						Evaluating and ranking the risk factors of COVID‐19 transmission and spread.
[44]	Economic	Supply chain	×	×			×	×						Evaluating the impact of policies on aspects of the supply chain.
[81]			×	×			×							Ranking and determining the importance of factors influencing the agility of pharmaceutical companies in the COVID‐19 pandemic.
[82]			×	×		×		×						Assessing the risk exposure and determining short‐ and long‐term strategic plans for agricultural supply chains.
[83]			×	×			×							Identifying the essential factors that can help the industrial sector (i.e., companies) to enhance their resilience in dealing with the crisis of the COVID‐19 pandemic and overwhelm it.
[84]			×	×		×								Identifying the supply chain risks during sudden disruption, such as the COVID‐19 pandemic, and exploring strategies to reduce the impact of these risks in the electronics sector.
[85]		Green economy	×	×			×	×						Exploring and prioritising green fuel technology alternatives for environmental sustainability.
[86]			×	×			×	×						Analysing and identifying strategies for achieving green economic efficiency.
[87]		Transportation	×	×	×									Identifying transportation alternatives to provide effective solutions for selecting a sustainable situation and improving the urban context.
[88]		Environment	×	×			×							Assessing and analysing the impacts of COVID‐19‐induced lockdowns on the quality of the environment in general.
[89]		Business and corporations	×	×	×		×	×						Evaluating and analysing the impact of COVID‐19 on the performance of construction companies financially in different related sectors.
[90]			×	×			×							Evaluating and ranking the risk grade in the investment of electric power grid to reduce risk loss and enhance risk management in the context of socioeconomic development.
[91]			×											Determining quarantined areas inside the country to control local tourism and reduce the negative impact of COVID‐19 on the hospitality sector industry.
[92]			×	×			×	×						Assessing and ranking private health insurance companies to improve health services during the COVID‐19 pandemic.
[93]	Technological		×	×			×							Defining and prioritising the importance and the net cause and effect relationship of the smart system for medical waste disposal.
[94]			×	×	×									Developing an auction mechanism in the way that the agents of network service can automatically select edge computing offers to support their customers.
[95]			×	×		×		×						Evaluating and ranking the critical challenges of DT intervention to control the COVID‐19 outbreak.
[96]			×	×			×	×						Selecting and evaluating e‐learning websites to improve users' performance and learning and teaching quality and reduce subjectivity.

For the issues presented in all previous sections, multiattribute and importance criteria appeared in 100% of all articles within the medical category. As for data variation, four studies had the issue with a 13.33% occurrence rate, followed by a trade‐off with nine studies with a 30% occurrence rate. Meanwhile, conflicting criteria and vagueness were present in 15 and 19 studies with a 50% and 63.33% occurrence rate, respectively. For unnatural comparison, only one study reported the issue with a 3.33% occurrence rate. However, time consumption was present in two studies with a 6.67% occurrence rate, and the same outcome occurred for distance measurement and normalisation. The last group was reverse ranking, with three studies and a 10% occurrence rate. For the social science with respect to MADM, the multiattribute issue was apparent in all four studies with a 100% occurrence rate. However, three studies accounted for criteria importance with a 75% occurrence rate, and three studies accounted for vagueness issues. Only one study tackled trade‐offs with a 25% occurrence rate. For the economic part of the literature of MADM, multiattribute was again apparent in all studies with a 100% occurrence rate, whereas the importance of criteria dominated in all studies except one (n = 12) with a 92.30% occurrence rate, followed by conflicting criteria with nine studies and a 69.23% occurrence rate. Vagueness occurred in six studies with a 46.15% occurrence rate, followed by data variation and trade‐off with two studies with a 15.38% occurrence rate. The last category was technological with a 100% occurrence rate for the multiattribute issue and importance of criteria, followed by conflict and vagueness, which occurred in two studies with a 50% occurrence rate. Furthermore, data variation and trade‐off were observed in each study.

All of these MADM issues emerged in numerous studies, and each study in this review dealt with these topics from various perspectives. Some works opted to develop MADM and made it more advanced mathematically, whereas some dealt with these issues on the basis of the nature of the criteria and attributes taken into the MADM decision. Others were keen on extending MADM approaches or even combining them to address their respective issues and problems regardless of their occurrence. However, how the studies dealt with the issues represented the contributions that they carried in their respected research. As much as this section addressed these main issues, the following portion highlights how research contributions were made in various fields after addressing the respective MADM issues.

3.2. Contributions

As shown in Figure 3, the final set of articles was categorised into four major categories, namely, medical (Section 3.2.1), social (Section 3.2.2), economic (Section 3.2.3), and technological (Section 3.2.4). These categories were classified on the basis of a common theme inspired by the reference and agreed upon during the authors' discussions.

3.2.1. Medical

This category discussed studies that in one way or another touched on a medical aspect in applying MADM to COVID‐19. The section contains a total of 30 studies. The subcategory contains major topics, where MADM was applied with regard to (1) strategies, (2) policies, (3) tools, and (4) services.

The first set of studies discussing MADM with strategies included four studies. In the first study, ⁴⁹ an MADM approach was used for the selection of sustainable strategic guidelines for the reorganisation of a healthcare system under the conditions of the COVID‐19 pandemic. The authors claimed that such an MADM approach would assist in crises, such as the current pandemic, which made the integration of the method necessary for establishing adequate medical care to patients and reducing dangers for the population caused by the COVID‐19 pandemic to a minimum. The researchers used four strategic guidelines, which were defined along with five criteria for the evaluation in their study. In the following study, ⁵¹ the authors discussed the hospitals' confrontation with a wide range of alternatives represented in preparing preventive measures to confront COVID‐19. However, the study did not determine which alternatives were better. The authors used MADM to identify these alternatives, arrange them according to their importance and make a comparison of various preventive strategies between countries globally. In another study, ⁵⁴ the authors discussed that many health organisations, such as the World Health Organization (WHO), and many governmental agencies were struggling with proper guidelines and remedial activities to reduce the spread of COVID‐19. In their pursuit, they indicated that applying all recommendations at the same level by people is an impractical expectation. The reason is the existence of numerous social and physical measures, such as social and physical distancing, wearing antiviral masks, avoiding any unnecessary travel, maintaining hygiene, eating healthy food, and monitoring health conditions. Thus, measures that are more effective in curtailing the spread of COVID‐19 should be prioritised compared with those that have less effect. Consequently, an MADM analysis is used to analyse and prioritise the precautionary measures of COVID‐19. The fourth study ⁵³ discussed an important issue associated with the presence of COVID‐19. The authors claimed that the mental and physical well‐being of people under lockdowns has become an emerging concern. Moreover, governments have been continuously pursuing lockdown relaxation efforts for public health and economic restart. Moreover, they have attempted to maintain existing strategies while avoiding succeeding waves of cases. Such serious issue would be considered a conflicting criterion, particularly when the government relies on a trial‐and‐error approach. Therefore, MADM emerges in scenarios with intertwined causal relationships among guideline protocols for the relaxation strategy.

The second set of studies discussing MADM with policies included three studies. In the first work, ⁵⁵ the authors claimed that governments are challenged to come up with policies to help reduce the COVID‐19 burden on healthcare systems, which have exceeded their capacity. The authors called it the ‘flattening the curve' concept. Moreover, the authors looked at the issue and recognised that traditional means are impossible to implement, particularly given the fast spread of COVID‐19 and community attributes that influence the spread at the municipal level. Nevertheless, such a position always remains a tough one to make given all these issues, and finding a solution in a short amount of time without causalities is nearly impossible. Once again, MADM is pioneered in scenarios regarding infected patients and bed capacity to provide support for effective public health policies for national, regional and local intervention. In the following study, ⁵⁶ the authors asserted that their ultimate and harmonic integration is needed considering the importance of emergency medical services during the COVID‐19 pandemic. They highlighted that the response to the pandemic affects the rate of mitigating its problems. They also discussed methods to respond effectively to the urban epidemic situation during a complex environment, which has become a global challenge. Moreover, emergency decision‐making can be considered an MADM problem, which involves multiple criteria or attributes with qualitative and quantitative aspects. Therefore, by using MADM, we can evaluate the severity of the urban COVID‐19 epidemic situation within a complex humanitarian crisis environment. The last study in the policies subsection by Ashraf et al. ⁵⁷ discussed the importance of controlling the spread of COVID‐19 in emergencies worldwide. The authors claimed that emergency events are known to have aspects of short duration and data, harmfulness and ambiguity. Policy makers are often rationally bounded under the uncertainty, complexity and vagueness of the information and issues. Decision issues become complex, making them priorities that need to be addressed. Therefore, these authors posited that MADM for controlling the transmission and spread of COVID‐19 is a very effective approach that can also be used for other complicated problems, such as risk evaluation, emerging technology, uncertain decision‐making, project installation and site selection.

The third set of studies discussing MADM with tools included nine studies. The first two studies discussed masks. In the study of Yang et al., ⁵⁸ the issue of mask selection during the COVID‐19 pandemic was discussed. The authors argued that antivirus mask selection given the shortage of these tools during the pandemic is a serious concern. People often believe that the best masks are either expensive or only those used by frontliners. This misconception leads to the inappropriate and excessive purchase and the use of personal protective equipment, which has exacerbated the mask shortage. In reality, masks should be considered with factors, such as reusability, quality of raw materials and the situations of people wearing them to optimise their use and allocate medical resources in the best way possible. When such an issue arises, MADM assists in selecting a reasonable antivirus mask on the basis of the needs and situations of different people, thereby ensuring practical significance. Another work ⁵⁹ expressed that considering the COVID‐19 pandemic, the task of choosing the right face mask has become a problem that numerous people face because of the lack of specific standards. This problem has been addressed using MADM. Another mask‐related work was presented by Yang et al., ⁶⁴ wherein the authors discussed the importance of mask selection during the COVID‐19 pandemic. They utilised MADM techniques to select among the six most common and available masks. They also analysed the advantages and disadvantages and also made a comparative analysis with graphical interpretation. Aside from masks, other authors discussed the diagnosis tests for COVID‐19. The authors in Sayan et al. ⁶⁰ stated that diagnosing patients quickly and accurately is important in their treatment. Moreover, the existence of several diagnosis tests creates confusion in selecting the most appropriate method. Prioritising one over the other is a difficult decision, particularly given various parameters, such as high sensitivity, high specificity, low false positivity, low false negativity, high usability and low cost. Therefore, the authors considered the MADM solution in the evaluation of the effectiveness of seven different diagnostic tests for COVID‐19. They concluded that the effectiveness of diagnostic tests varies according to the patient's specific condition and each country's own resources. In Mohammed et al., ⁶¹ the authors shed light on the fact that COVID‐19 diagnosis has been widely applied with the use of AI models. These intelligent methodologies are highly desirable in helping health organisations in the selection of a COVID‐19 diagnosis system. However, similar to any other technology, they are ambiguous and have drawbacks, hindering their ultimate usage. The authors claimed that owing to the availability of several ML models, selecting one over the other method is no easy thing to do, and evaluating and benchmarking COVID‐19 ML models are considered their main challenges. They argued that multiple conflicting criteria exist in the evaluation and benchmarking process, and in this regard, MADM comes as an intermediary in addressing such decision issues. Another study in this subcategory ⁶² argued that accurately diagnosing COVID‐19 using CT is sometimes difficult and uncertain as the symptoms for the disease have similarities with other viral lung diseases, namely, H1N1, H5N1, SARS, and hantavirus. Therefore, the authors recognised the challenge of such a decision problem and agreed that CT imaging results would be identified as a decision issue with multiple conflicting criteria. Hence, they pioneered the use of MADM in the context of environmental uncertainty, wherein the symptoms and CT imaging results are the criteria and the alternatives are the viral lung diseases, including COVID‐19. The authors believed that such work can assist medical staff in monitoring the spread of COVID‐19 with accurate results of the effectiveness of the evaluation. Another interesting work ⁵² discussed the potential of using MADM in the detection of COVID‐19. The authors proposed a methodology with three phases for evaluating and benchmarking the COVID‐19 AI classification techniques, which were applied for diagnosing the disease. 'The researchers discussed the phase of identification, which illustrates the data sets and requires preprocessing, and identified the evaluation criteria used in the evaluation and benchmarking of COVID‐19 AI classification techniques and the number and type of techniques.' The output of the phase had four decision matrices, which were followed by the integration of MCDA methods to deal with the issue. The final phase in their work included the objective and subjective validations for ranking COVID‐19 AI classification techniques. Another diagnosis study ⁶³ presented a strategy based on the operators to tackle MADM issues. The authors used COVID‐19 as a practical example for the selection of a suitable laboratory for testing to demonstrate the application of their proposed strategy. The last study in this category ⁶⁵ presented that the emergence of COVID‐19 led to many casualties globally. The authors discussed the importance of detecting COVID‐19 in patients and proper treatment along with awareness because it can assist to control COVID‐19. Therefore, the authors proposed the fuzzy cloud‐based COVID‐19 diagnosis assistant, which aims to distinguish patients as confirmed, suspected or probable COVID‐19 cases. The diagnosis tool categorised the patients into four categories as mild, moderate, severe, or critical. The tool was proven in successfully monitoring the COVID‐19 pandemic and reducing its transmission rate among society.

The fourth set of studies discussing MADM with services included 14 studies. The first three studies discussed patients. The authors in Shirazi et al. ⁵⁰ discussed patients' dissatisfaction with hospital services in the presence of COVID‐19. They indicated that despite having resources and existing medical capacities, dissatisfaction with improper allocation hinders patients from having the ultimate health experience with respect to their needs, desires and expectations. They argued that service factors from the patients' viewpoints varied and that prioritising them would maintain the quality of the service. Therefore, MADM was introduced in such a scenario to mitigate the decision issue. The approach helped to identify which satisfaction factors were to be prioritised under normal conditions and during the COVID‐19 pandemic to succeed and survive in today's competitive environment. Another study ³⁷ argued that COVID‐19 causes variations in the laboratory examinations of patients. The authors shed light on the speed of geographical expansion and the sudden increase in the numbers of infected patients. This rise should quickly emphasise the growing number of patients, thereby leaving individuals at medical institutions with tough decisions. Therefore, medical professionals have opted to use MADM to prioritise patients while considering the urgency for solutions to minimise risks for deteriorating conditions. The latter was proposed to overcome the challenges and complications of patients with severe COVID‐19 on the basis of laboratory examination criteria, which assist in differentiating among mild, serious, and critical COVID‐19 conditions. Such an approach can be used later to assign appropriate care with prompt and effective treatment. The last patient study ⁶⁶ revealed that given the rapid spread of the global pandemic, numerous scientists are doing their best to tackle this crisis. The authors claimed that new challenges are outlined from various medical perspectives, thereby causing several decision issues and challenges that require a novel design solution, particularly for asymptomatic patients because of multilaboratory criteria, criterion importance and trade‐off among these criteria, which are challenging to differentiate. The researchers used integrated MADM to handle the prioritisation of patients with COVID‐19 and detect the health conditions of asymptomatic carriers. Other studies discussed treatment, particularly the study of Albahri et al. ¹⁰ who presented interesting ideas. The authors claimed that people who have recently recovered from COVID‐19 have antibodies against the virus circulating in their blood. Transfusing these antibodies to deteriorating patients could theoretically help boost their immune systems. At the same time, decision issues arise as to whom the antibodies should be administered first given all the biologically different and conflicted criteria and the compliance with national health requirements and known standard routine procedures. The authors proposed a rescue framework for the transfusion of the best convalescent plasma (CP) to the most critical patients while considering ML and MADM, which can greatly assist the health sector when such decisions are to be made. Another study ⁶⁷ discussed exploring the most efficient method for surviving COVID‐19 in addition to medical treatments. For this purpose, the authors introduced a novel concept of the MADM technique and explored the most effective alternative cure for COVID‐19, which is undoubtedly based on the theory and work they applied. Another treatment study ⁶⁸ stated that ‘as COVID‐19 spreads across the world, the transfusion of efficient CP to the most critical patients can be the primary approach to preventing the spread of the virus and treating the disease. This strategy is considered an intelligent computing concern where several criteria affect the decision. Therefore, the authors proposed a novel CP‐transfusion intelligent framework for rescuing COVID‐19 patients across centralised/decentralised telemedicine hospitals based on the matching component process to provide CP efficiently from eligible donors to the most critical patients using MADM methods.' In this category, another study ⁶⁹ discussed that given the presence of COVID‐19 vaccines and with the need to provide all the people of a country with vaccines, governments must identify priority groups for allocating COVID‐19 vaccine doses. Therefore, the authors formulated strategies with the assistance of MADM methods to classify and rank the most deserving groups in society so that they receive the vaccine first. Similar to Mishra et al., ⁷⁶ more treatment work discussed that choosing the best antiviral therapy to treat the mild symptoms of COVID‐19 was quite a complex, challenging and uncertain decision. The reason is the inexistence of ultimate medication for COVID‐19 at that time and the emergence of other forms of treatment including several antiviral therapies, which were used to treat the mild symptoms of COVID‐19 disease. In that respect, the authors used hesitant fuzzy sets (HFSs) as part of hesitant fuzzy decision‐making, which have been proven effective and valuable to express uncertain information in real‐world issues, to choose five methods or medicines to treat the mild symptoms of COVID‐19 and demonstrate the practicability and efficacy of the developed idea in real‐life applications. Another treatment study ⁷⁴ discussed that owing to the impact of COVID‐19, treating patients with possible medicines is warranted, but at the same time, people who have COVID‐19 infection will suffer from fever, cough, shortness of breath, dyspnoea, and other symptoms. Unfortunately, no specific treatment is available for diseases caused by COVID‐19. In that regard, MADM was utilised for the medicine selection of COVID‐19. In the following topic for resource utilisation, the authors in De Nardo et al. ⁷⁰ raised the issue of hospital admission during the COVID‐19 pandemic with a hospital‐bed shortage. The researchers claimed that for critically ill patients, this shortage has been a major challenge because even countries with robust healthcare systems and strong economies can be rapidly overwhelmed by such emergency, let alone those in low‐ and middle‐income settings. Prioritising access to care in these situations is considered a serious issue with various criteria influencing the decision. To address the latter, MADM has been proven useful in such context and used in identifying noncritical COVID‐19 patients who should be admitted to hospitals because of the risk of rapid clinical deterioration and in improving the response of health systems. The next study regarding hospitals and their resources ⁷¹ suggested that the unexpected emergence of natural and man‐made disasters globally is worth discussing, particularly the importance of hospital preparedness, which is the first reference point for people to obtain healthcare services. At the same time, numerous uncertainties and decision issues are emerging, thereby hindering the ability to grasp what these hospitals can truly do. Therefore, identifying hospitals with low levels of preparedness will be crucial for disaster preparedness planning. The authors argued that the healthcare sector is currently facing a great struggle because of the COVID‐19 pandemic. Evidently, preparing hospitals for such disasters should be made quickly and reliably. Therefore, MADM has been used to evaluate hospital disaster preparedness and has been proven to be a suitable approach considering that methods measuring hospital disaster preparedness levels are lacking. Another study ⁷² discussed the importance of hospital selection in times of COVID‐19. The authors highlighted the hospital selection problem for COVID‐19 when the pandemic had broken out in Wuhan, China. At that time, the government stressed to ‘leave no one unattended'. Therefore, the Chinese government ordered the design and conversion of makeshift hospitals in Wuhan. Undoubtedly, this effort raised serious decision problems on hospital selection to efficiently provide treatment for COVID‐19 patients with mild symptoms. In that regard, MADM was used to select the best among many available options. More resource works were concerned with different topics, including the disposal of medical waste. The authors ⁷³ discussed that 'many authorities especially in the developing nations are battling to select the best health care waste (HCW) disposal technique for the effective treatment of the medical wastes during and post COVID‐19 era'. In that regard, the selection of the best technique requires considering various tangible and intangible criteria, which can be framed as an MADM problem. Therefore, the authors proposed a framework for the selection of the best HCW disposal technique on the basis of socio‐technical and triple bottom line perspectives. Another work ⁷⁵ discussed the importance of recognising various dimensions of seven basic types of waste in healthcare processes. In that regard, MADM was used to prioritise the identified wastes and their dimensions. The authors also developed a framework of leanness assessment and validation for the healthcare system.

3.2.2. Social

This category discussed studies that one way or another touched on a social aspect related to applying MADM with COVID‐19. The category contains four studies. In the first work, ⁷⁷ the impact of COVID‐19 on numerous vital centres worldwide must be examined by determining the social factors that countries are facing. The most critical issue among these social aspects is still unclear. Therefore, MADM has been applied to reach a logical arrangement of these factors and address the corresponding problem. As a result, several preventive measures have been classified to confront COVID‐19. The cases have several differences, and the best procedure has not been identified. Hence, MADM has been used to explore the best preventive method. The next study ⁷⁸ indicated that given the emergency factors that increased during the COVID‐19 pandemic, several organisations have suffered from decision‐making problems for their emergency operations associated with these times. Therefore, to address these issues, the authors used the MADM technique to discuss the decision‐making process under uncertainty, which can assist them in decision‐making problems in emergencies. Another work ⁸⁰ discussed COVID‐19 risk factors. The authors indicated that the COVID‐19 outbreak in December 2019 severely affected the world. The epidemic is spreading fast through various manners as the virus is very contagious. The authors discussed that one method of containing the virus is to sort out the risk factors and rank them in terms of contagion. Therefore, they evaluated risk factors involved in the spread of COVID‐19 and ranked them using MADM methods. The next work ⁷⁹ discussed the risk level for musculoskeletal disorders among handheld device users. The authors indicated that owing to COVID‐19, 'The work‐from‐home (WFH) situation increased the handheld device (HHD) users to work in uncomfortable postures for longer times because of the absence of proper designed workstations. This issue leads to different type of musculoskeletal disorders (MSDs) among the HHD users. For that, the authors proposed multicriteria decision‐making approach for identifying the risk level of MSDs among HHD users'.

3.2.3. Economic

This category discussed studies that in one way or another touched on an economic aspect related to the application of MADM on COVID‐19. A total of 13 studies were obtained. The subcategory contains major topics, where MADM was applied with regard to (1) supply chain, (2) green economy, (3) transportation, (4) environment, and (5) business and corporations.

The first set of studies discussing MADM with the supply chain included five studies. Grida et al. ⁴⁴ shed light on the disruption caused by COVID‐19 and its economic impact. They argued that the COVID‐19 crisis in supply chains is derived from three main aspects: (1) supply, (2) demand, and (3) logistics. Several works in the literature have investigated the effect of preventive strategies on supply chain policies in light of COVID‐19. In this regard, MADM was used to generate sufficient information for decision makers in industrial firms to address uncertainty issues, which are considered high‐decision issues associated with the three supply chain policies mentioned previously. The next study ⁸¹ explored the issues that disrupted supply chain agility for pharmaceutical companies during the COVID‐19 crisis and the vital problems related to the priority of factors that affect supply chain agility for pharmaceutical companies, specifically because of the slow financial turnover in pharmaceutical companies and their need for urgent financial resources. Thus, MADM was used to increase forecast accuracy, lower production costs, optimise the use of resources and set the criteria for selecting the perfect suppliers to increase speed and flexibility in production. Next, the authors in Sharma et al. ⁸² explained that the COVID‐19 pandemic has created problems for supply companies regarding flexibility and response during and after COVID‐19 in terms of building systems that are more responsive and resistant to future crises of industrial companies. Therefore, MADM was used to identify the most important factors that help rebuild industries and societies to resist crises and be more flexible. In the last two supply chain studies, the authors in Khurana et al. ⁸³ discussed numerous unprecedented risks during the COVID‐19 pandemic resulting from disruptions in agricultural supply chains. In addition, the critical risks during this global challenge remain unclear. Using MADM methods, the risks of agricultural supply chains were determined by the order of importance, namely, demand risks, financial risks, logistics and infrastructure risks, management and operational risks, policy and regulation risks and biological and environmental risks. Another supply chain disruption work ⁸⁴ discussed the importance of electronic products to society, but the COVID‐19 situation pushed almost all human direct communication to become virtual (i.e., online). This event indicated that the supply of critical raw materials for electronics manufacturing is becoming increasingly precarious to social, geopolitical, and technical disruptions, which are more likely to be increased in the future because of the global health crises and their impact on climate change. In that regard, the authors 'investigated supply chain disruption risks in the electronics sector by applying metrics that capture supply, demand, socio‐political, and environmental risks in a multicriteria framework using TOPSIS to almost 40 metals and minerals that provide critical functionality to electronic products'.

Aside from the supply chain, another topic for the green economy was also discussed in Shah et al. ⁸⁵ The authors claimed that the COVID‐19 situation constitutes an opportunity to shift towards a green economy because of the curfew issues that have affected various companies, constituting a favourable opportunity to reduce carbon production. However, the problem that companies face lies in setting priorities for waste‐to‐energy conversion. Thus, MADM with fuzzy priorities for converting waste to energy was formed to reduce uncertainties regarding the most appropriate technology for the conversion. Another green energy work was discussed by Yao. ⁸⁶ The authors argued that COVID‐19 causes serious threats not only to global health but also to the worldwide development regime. 'The experts, economists, policymakers, and the governments expressed their pledges and determinations to adapt and mitigate climate change, making them start adopting green growth and development strategies, and one of the major challenges has been promulgating and strictly implementing environmental regulations and policies for green growth and development'. In that regard, they utilised MADM to analyse environmental regulation. They identified alternative strategies to achieve while considering different main and subcriteria in the context of environmental regulations.

Another economy‐related topic associated with transportation was discussed in Moslem et al. ⁸⁷ The authors believed that countries suffered because of the transport sector's sustainability and faced numerous environmental challenges, which were caused by the challenge of identifying transportation alternatives. This problem was observed after imposing social distancing measures owing to COVID‐19. To determine transportation alternatives, MADM was applied to identify options to provide effective solutions for selecting a sustainable situation and improving the urban context.

The next work ⁸⁸ comprehensively studied the changes in environmental conditions caused by the imposition of lockdowns because of COVID‐19. They had different alternatives in their study. By relying on an MADM method, the authors were able to derive the weight of indicators to study them properly. They also gained an understanding of the environmental response to the COVID‐19‐pandemic‐imposed lockdown situation and assessment after the lockdown situation.

Furthermore, four studies discussed business and corporations. In Altuntas and Gok, ⁹¹ the authors discussed another important use of MADM given the gap in the scientific literature with respect to making the right quarantine decisions to decrease the negative effect of a pandemic on the hospitality industry. The authors claimed that quarantine decisions during a pandemic should be taken by using the systematic method, and no universal guideline regarding such thing exists, specifically during the COVID‐19 pandemic. Towards this end, the authors used the decision‐making trial and evaluation laboratory (DEMATEL) method to help countries reduce the negative impact of quarantine decisions on the hospitality industry because of the COVID‐19 pandemic. Another work ⁸⁹ discussed that owing to the emergence and impact of COVID‐19, each country's economic development was affected, and all governmental agencies are encouraging the construction industry to develop an advanced infrastructure related to health, transport, education, and housing, particularly with regard to the operations and financial performance of construction sector companies. Therefore, the MADM process was utilised because no comprehensive study was conducted on the evaluation of the financial performance of construction companies. The authors determined the priority of financial ratios and the ranking of construction companies with the proposed techniques. The next work ⁹² discussed that the medical sector has been affected by COVID‐19‐related decisions and issues that were also reflected in business and the economy. They discussed the insurance business and elaborated that assessing and ranking private health insurance companies provide insurance agencies, insurance customers, and authorities with a reliable instrument for the insurance decision‐making process. In that respect, insurance companies were ranked on the basis of their healthcare services provided in Turkey during the COVID‐19 outbreak through an MADM. The authors concluded that the introduced approach met the insurance assessment problem during the COVID‐19 pandemic. In Duan et al., ⁹⁰ the authors discussed the impact of COVID‐19 on electric power grid investment. 'They discussed that owing to COVID‐19 socio‐economic development is undergoing changes in China, and it is in that regard urgent to evaluate the risk of electric power grid investment in China under new socio‐economic development situation, which can help the investors manage risk and reduce risk loss. In their work, the MADM method was proposed for risk evaluation of electric power grid investment in China under new socio‐economic development situation'.

3.2.4. Technological

This category discussed studies that in one way or another touched on a technological aspect related to the application of MADM. A total of four studies were obtained. In the first work, ⁹³ the authors discussed that wastes pose a great risk not only to the environment but also to public health. Particularly with the COVID‐19 outbreak, the proper disposal of waste has become a vital risk to people's lives. Hence, one problem is related to defining the criteria for a smart system for the disposal of medical wastes of industrial companies. By using MADM to determine the relationship between healthcare centres and waste disposal companies, the researchers could mitigate the issues and resolve them in the best way possible. The next study ⁹⁴ indicated that telecom service providers have been facing an unprecedented challenge to meet customer demands during the COVID‐19 period because of the increase in employees working from home. Users face challenges related to the price and quality of these services. Therefore, the use of MADM was discussed to develop methods to support end‐users. The third work ⁹⁵ explained that digital technologies are considered one of the essential aspects of daily life at present. However, the emergence of COVID‐19 has created numerous challenges in terms of strengths, weaknesses, opportunities, and threats. Thus, the application of MADM was discussed to determine the importance of these factors in the health sector in relation to digital technologies. The researchers concluded that the first factor is health information systems, followed by the lack of digital knowledge, digital stratification, and economic interventions. Another technological aspect was more related to using online sources and technologies for educational purposes. The authors in Gong et al. ⁹⁶ discussed that 'network teaching has been widely developed under the influence of COVID‐19 pandemic to guarantee the implementation of teaching plans and protect the learning rights of students'. They also shed light on the existence of various online sources in that regard. Moreover, selecting a particular website for online teaching can directly affect the performance of educators and students and promote the quality of online teaching. Consequently, e‐learning website selection can be considered an intricate MADM problem, and experts' evaluations of the performance of e‐learning websites are often imprecise and vague because of the subjectivity in human thinking.

3.3. Data set

In this section, studies on various COVID‐19 applications based on MADM approaches are discussed and analysed in terms of whether the data set used or suggested to use multiple factors represented by the primary data set, secondary data set, sample size and number of experts. The primary data represent the data set collected during the research and approved by the ethics committee. Meanwhile, the secondary data are those adopted and obtained by other researchers to help authors address multiple problems and test their approach. Table 2 presents a summary of the data sets used in the literature. ⁵²

Table 2.

Summary of the data set used

Ref.	Primary	Secondary	Description	Data availability			Category	Subcategory
				Completely presented	Partially presented	Not presented
[70]		×	It adopted patient's data to identify the factors that prioritise patients.		×		Medical	Services
[37]		×	A data set from six patients was used.	×
[66]	×		Real and simulated data sets of 56 patients with different health conditions were used. Seven experts from healthcare management participated.	×
[50]	×	×	Data set service of eight private hospitals in the city of Sari was used. Two decision makers.		×
[71]	×		Four selected Turkish hospitals were used.		×
[69]	×		Three experts from related sectors participated.	×
[68]	×		The data set included simulation data of 80 patients and 80 donors. Four centralised/decentralised telemedicine hospitals. Three experts from related sectors participated.	×
[67]	×		A group of medical experts was included to provide data for assessment.	×
[76]	×	×	Five medicines were used as an alternative to control COVID‐19 patients. Three doctors agreed to participate. Self‐report questionnaire was developed for related doctors and patients.	×
[74]	×	×	Four experts were invited. Four medicines for mild COVID‐19 were included.	×
[72]	×		One senior and five junior decision makers participated. Five makeshift hospitals in Wuhan were used for proving the concept.	×
[73]	×		Six decision makers participated and shared their judgment through the questionnaire. Tamil Nadu City was selected as one of the fastest developing states in India.		×
[75]	×		Selected hospital from Pune City in India. The data set used a questionnaire to collect expert feedback.	×
[60]	×	×	The data set used seven SARS‐CoV‐2 diagnostic tests in healthcare.	×				Tools
[59]	×	×	Data of approximately six mask types were used. Four patients were considered for testing.	×
[61]	×	×	The data set was the chest X‐ray for selecting an ideal model for diagnosing COVID‐19.	×
[62]	×		Primary symptoms of five viral chest diseases and CT scans. Three doctors participated.	×
[58]	×		Six types of medical and nonmedical masks were evaluated.	×
[63]	×		The data set included five authentic labs for the COVID‐19 test.	×
[64]	×		Six different antivirus veils available in the market were assessed. The individual shared their assessment based on four metrics.	×
[65]	×		The questionnaire was used for patients and doctors to prioritise the administration of medication to COVID‐19 patients. Five specialists (DMs) participated.	×
[55]	×		Geodatabase for estimating the risks of the Brazilian health system was used.	×				Policies
[57]	×		A panel of three domain experts participated.	×
[56]	×		Five cities from around the world were selected to participate in this study. A data management platform of an official WeChat account was used to generate the data of crisp numbers.	×
[53]	×		A focus group discussion of three experts that participated to complete the direct relation matrix.	×				Strategies
[49]	×		The data set was collected from 30 domain experts using a questionnaire to identify the criteria.	×
[54]	×		This study used several experts to evaluate the prevention and control actions.	×
[51]	×		The survey was used by seven experts for evaluating 15 intervention strategies.	×
[81]	×		The data set was collected by distributing a questionnaire to the pharmaceutical company.	×			Economic	Supply Chain
[82]	×		Information on risk factors was collected from 20 experts (organisations) using online discussion sessions and from the literature.	×
[83]	×		The survey was distributed to 10 healthcare experts.	×
[44]	×		A questionnaire was used to collect evaluation information on the nine policies from experts.	×
[84]		×	Data were obtained from the US Geological Survey, 2020.		×
[87]	×		Data collected from 400 participants in two iterative searches using a questionnaire followed BWM logic steps.		×			Transportation
[85]	×		Sixteen domain experts participated in the evaluation process.	×				Green Economy
[86]	×		The data set of this study was determined according to the set of experts to identify alternative strategies to achieve green economic efficiency.	×
[88]	×	×	The data set was derived from Landsat 8 OLI and TIRS data.	×				Environment
[89]	×		Twenty construction companies with respect to the financial ratio were used as a data set.	×				Business and Corporations
[90]	×		The data set used was a survey of five experts in grid investment.	×
[91]	×	×	The travel information among regions in Turkey from Turkey Statistical Institute (TurkStat) was used as a data set.	×
[92]	×		Five experts shared in this assessment. Ten private health insurance companies.	×
[78]	×		Data were collected from health experts about the emergency factors and alternatives.	×			Social
[77]		×	Data from the WHO were used.		×
[80]	×	×	Data set was combined from literature, media survey and physicians' feedback.	×
[79]	×		Four decision makers with research specialisation of related domains helped to provide the requested data.	×
[95]	×		The data set used was a survey of three experts about digital technology in the health sector.	×			Technological
[94]	×		A simulated network service provider data set was generated to be used as a data set.		×
[93]	×		Three Indian smart cities participated in this study. Five to seven experts from each city were included in evaluating the smart healthcare waste disposal plans.	×
[96]	×		The data set was used on the basis of four experts' feedback for e‐learning websites.	×

Table 2 presents various employed data sets based on each category and subcategory of the proposed taxonomy. In the literature, the experimental test was implemented either using a primary or secondary data set or both. ⁹⁷ Out of 51 studies, two studies ¹⁰ , ⁵² did not include the primary and secondary data sets in their study design. The authors in Albahri et al. ⁵² proposed a theoretical framework based on MADM methods as a future direction to evaluate and benchmark AI diagnosis techniques used in classifying COVID‐19 medical images. Meanwhile, the authors in Albahri et al. ¹⁰ stated that the developed MADM framework has not been tested using a data set of infected cases because of the lockdown and global pandemic outbreak. The majority of the 49 remaining studies (36; 73.5%) used only the primary data set, nine studies (18%) combined primary and secondary data sets and four studies (8%) used only the secondary data set in the experimental test. Furthermore, most researchers were keen to publish the data set and present it within the study work. However, only eight studies (16%) presented the data sets and the results partially. In general, the availability of the data set is either fully presented, partially presented or not shared, which mainly followed the journal policy of the availability statement. However, the researcher can access the data set that was not directly published either through contacting the authors, referring to the supplementary file or contacting the journal itself. Most of the time, the data set would be shared if requested, unless if it includes sensitive data. ⁹⁸

3.4. Evaluation criteria

This section analyses previous studies in the literature that focused on using decision‐making with COVID‐19 in terms of the evaluation criteria used in each case study. The extracted criteria were classified within the two main types of evaluation criteria: subjective and objective criteria. The researchers used either the first or the second type or the combination of the two types depending on the research in question. Table 3 illustrates the evaluation criteria used in each of the taxonomic categories shown in Figure 3. Furthermore, subjective criteria are defined on the basis of human judgment or perception, which rely on individuals' tastes and preferences. These factors reflect some desirable properties of the object that need to be evaluated. By contrast, objective criteria are defined as factual, measurable information. Their characteristics are evident and do not rely on human judgment or perception.

Table 3.

Evaluation criteria

Ref	Category	Subcategory		Evaluation criteria
				Criteria	Subcriteria	Criteria type
						Objective	Subjective	Hybrid
[49]	Medical	Strategies		C1, C2, C3, C4, C5, C6	NA		×
[53]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15	NA		×
[51]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15	NA		×
[54]				C1	S1, S2, S3		×
				C2	S1, S2, S3
				C3	S1, S2, S3, S4, S5
				C4	S1, S2, S3
				C5	S1, S2, S3
				C6	S1, S2, S3, S4
				C7	S1, S2, S3, S4
[55]		Policies		C1	S1, S2, S3, S4, S5	×
				C2	S1, S2
				C3	S1
				C4	S1
				C5	S1
[56]				C1, C2, C3, C4, C5, C6, C7	NA			×
[57]				C1, C2, C3, C4	NA			×
[45, 46, 51]		Tools		C1, C2, C3, C4	NA		×
[60]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15	NA		×
[61]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10	NA	×
[62]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14	NA			×
[52]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24	NA	×
[63]				C1, C2, C3	NA		×
[65]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10	NA			×
[37]		Services		C1, C2, C3, C4, C5, C6, C7, C8	NA	×
[66]				C1, C2, C3, C4, C5, C6, C7, C8	NA	×
[70]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12	NA	×
[50]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10	NA		×
[10]				C1, C2, C3, C4, C5, C6	NA	×
[67]				C1, C2, C3, C4	NA			×
[71]				C1	S1, S2, S3, S4, S5, S6, S7			×
				C2	S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12
				C3	S1, S2, S3
				C4	S1, S2, S3, S4
				C5	S1, S2, S3, S4, S5, S6
				C6	S1, S2, S3, S4
[69]				C1	S1, S2, S3, S4, S5, S6		×
				C2	S1, S2, S3
				C3	S1, S2, S3
				C4	S1, S2, S3
[68]				C1, C2, C3, C4, C5	NA	×
[72]				C1, C2, C3, C4, C5, C6, C7, C8	NA			×
[73]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10	NA			×
[74]				C1, C2, C3, C4	NA			×
[75]				C1	S1, S2, S3			×
				C2	S1, S2, S3
				C3	S1, S2, S3
				C4	S1, S2, S3
				C5	S1, S2, S3
				C6	S1, S2, S3
				C7	S1, S2, S3
[76]				C1, C2, C3, C4, C5, C6, C7	NA			×
[77]	Social			C1, C2, C3	NA	×
[78]				C1, C2, C3, C4, C5, C6, C7, C8	NA		×
[79]				C1	S1, S2, S3, S4, S5			×
				C2	S1, S2, S3, S4
				C3	S1, S2, S3
[80]				C1, C2, C3	NA		×
[44]	Economic		Supply chain	C1, C2, C3, C4, C5, C6, C7, C8, C9	NA		×
[81]				C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23	NA		×
[83]				C1	S1, S2, S3, S4		×
				C2	S1, S2, S3, S4
				C3	S1, S2, S3, S4, S5
				C4	S1, S2, S3, S4
				C5	S1, S2, S3, S4
				C6	S1, S2, S3, S4
				C7	S1, S2, S3, S4
				C8	S1, S2, S3, S4
				C9	S1, S2, S3, S4
[82]				C1	S1, S2, S3, S4		×
				C2	S1, S2, S3, S4
				C3	S1, S2, S3, S4, S5, S6, S7
				C4	S1, S2, S3, S4
				C5	S1, S2, S3, S4, S5, S6, S7
				C6	S1, S2, S3, S4
				C7	S1, S2
				C8	S1, S2, S3, S4, S5
				C9	S1, S2, S3, S4, S5
[84]				C1	S1, S2, S3, S4	×
				C2	S1, S2, S3, S4
				C3	S1, S2, S3, S4
				C4	S1, S2, S3, S4
[85]			Green economy	C1	S1, S2, S3		×
				C2	S1, S2, S3
				C3	S1, S2, S3
[86]				C1	S1, S2, S3, S4, S5, S6			×
				C2	S1, S2, S3, S4, S5
				C3	S1, S2, S3, S4, S5, S6
				C4	S1, S2, S3, S4, S5
				C5	S1, S2, S3, S4, S5
				C6	S1, S2, S3, S4, S5
				C7	S1, S2, S3
				C8	S1, S2, S3, S4, S5
				C9	S1, S2, S3, S4
				C10	S1, S2, S3, S4
[87]			Transportation	C1, C2, C3, C4, C5, C6, C7, C8	NA		×
[88]			Environmental	C1, C2, C3, C4, C5	NA		×
[89]			Business and corporations	C1, C2, C3, C4, C5, C6	NA	×
[90]				C1	S1, S2, S3		×
				C2	S1, S2, S3
				C3	S1, S2, S3
				C4	S1, S2, S3
				C5	S1, S2, S3
[91]				C1, C2	NA		×
[92]				C1, C2, C3, C4, C5, C6, C7	NA			×
[93]	Technological			C1, C2, C3, C4, C5, C6, C7	NA	×
[94]				C1, C2, C3, C4	NA	×
[95]				C1	S1, S2, S3, S4, S5, S6		×
				C2	S1, S2, S3, S4, S5
				C3	S1, S2, S3, S4, S5, S6
				C4	S1, S2, S3, S4, S5, S6
[96]				C1, C2, C3, C4, C5, C6, C7	NA			×

Table 3 shows the distribution of objective, subjective and hybrid criteria (a combination of subjective and objective criteria) that were used by the studies included in this review (Table A1 in the appendix A illustrates the description of the main and subcriteria of each reviewed study). Of the 51 studies, 14 studies representing 27.45% of the exhaustive studies used objective criteria. Then, 22 studies comprising 43.13% used subjective criteria. Meanwhile, 15 studies constituting 29.41% used hybrid criteria. Figure 4 depicts the visualisation of the overall distribution of objective, subjective and hybrid criteria used in the literature. In the medical category, which represented 58.82% of the total studies, 8 studies representing 26.67% used objective criteria, 11 studies representing 36.67% used subjective criteria and 11 studies representing 36.67% used hybrid criteria. The distribution of objective criteria in terms of numbers and percentages among the medical subcategory revealed that the objective criteria were most frequently used in the services subcategory with 17% of the total studies in the medical category. This group was followed by the tool subcategory with 7%, whereas the policies subcategory comprised 3% of the total studies in the medical category. By contrast, subjective criteria were most frequently used in the tools subcategory with 17%, followed by strategies with 13% and services with 7%. Most hybrid criteria were used in services with 23%, followed by policies and tools with 7% each in the total studies of the medical category.

Figure 4.

Distribution of objective, subjective, and hybrid criteria [Color figure can be viewed at wileyonlinelibrary.com]

Furthermore, Figure 5 shows the distribution of the mentioned criteria for each category and its subcategory individually. In the economic category, which represented 25.49% of the total studies, nine studies representing 69.23% used subjective criteria, two studies representing 15.38% used objective criteria and two studies representing 15.38% used hybrid criteria. The distribution of subjective criteria in terms of numbers and percentages in the economic subcategory demonstrated that subjective criteria were most frequently used in the supply chain subcategory with 31% of the total studies in the economic category. This group was followed by the green economy, environmental, transportation and business and corporations subcategories with 8% each, as shown in Figure 5. In the social category, which represented 7.84% of the total studies, two studies representing 3.92% used subjective criteria, one study representing 1.96% used objective criteria and one study representing 1.96% used hybrid criteria. In the technological category, which represented 7.84% of the total studies, two studies representing 3.92% used objective criteria, one study representing 1.96% used subjective criteria and one study representing 1.96% used hybrid criteria.

Figure 5.

Distribution of objective, subjective, and hybrid criteria [Color figure can be viewed at wileyonlinelibrary.com]

3.5. MADM techniques

This section discusses the application of different MADM techniques in solving issues related to COVID‐19 in the medical, social, economic, or technological sectors, which were considered in the proposed taxonomy of this study. In MADM, all possible options are evaluated against each criterion. These assessments help to make the right decision and select the most suitable option(s). Theoretically, MADM techniques have three different directions: ranking, weighting, and hybrid methods.

Ranking methods work on prioritising the alternatives or distinguish the most suitable alternative among others, whereas weighting methods compute the weight of each criterion on the basis of the related importance against other criteria. Hybrid methods take advantage of integrating ranking and weighting methods together. ⁷⁸ Notably, some MADM methods, such as the analytic hierarchy process (AHP), ⁶⁹ subjective and objective decision by opinion score method (SODOSM), ¹⁰ and weighted aggregated sum product assessment (WASPAS) ⁹⁵ were developed to accomplish both tasks, compute criteria weights and rank alternative priorities. Moreover, weighting methods can be categorised into three different types: subjective, objective and hybrid weighting methods (both subjective and objective weighting). Objective weighting methods use the data values for each criterion to measure the weights of the criteria. ⁶¹ For subjective weighting methods, the accumulated expert consensus is used to assign and calculate the subjective agreements for each criterion. The hybrid weighting methods use the characteristics of both previous methods. That is, subjective and objective processes are combined in a single method as in SODOSM, ¹⁰ Alternately, subjective and objective methods are integrated as in the level‐based weight assessment (LBWA) model and an objective method called measuring attractiveness by a categorical‐based evaluation technique (MACBETH) method ⁴⁹ and AHP with spherical fuzzy entropy. ⁷⁸

As mentioned in Table 1, assessing the importance of related criteria is one of the main issues in solving COVID‐19‐related problems in the medical, social, economic or technological sectors. Thus, the distribution analysis of MADM techniques under each category of our taxonomy is unnecessary because all 51 studies computed the weight of related criteria and ranked its studied alternatives in one way or another. Moreover, the applied weighting methods were a combination of subjective, objective and hybrid weighting methods (Table 4).

Table 4.

Weighting methods

Ref	Category	Subcategory	Subjective methods	Objective methods	Hybrid methods
[49]	Medical	Strategies	–	–	LBWA + MACBETH
[51]			Direct weight	–	–
[53]			Direct weight	–	–
[54]			AHP	–	–
[55]		Policies	AHP	–	–
[56]			–	Entropy	–
[57]			Direct weight	–	–
[58]		Tools	Direct weight	–	–
[59]			Direct weight	–	–
[60]			Direct weight	–	–
[61]			–	Entropy	–
[62]			BWM	–	–
[52]			AHP	–	–
[63]			Direct weight	–	–
[64]			Direct weight	–	–
[65]			Fuzzy assessment	–	–
[50]		Services	FAHP	–	–
[37]			AHP	–	–
[66]			–	Entropy	–
[10]			–	–	SODOSM
[67]			Direct weight	–	–
[70]			–	PAPRIKA	–
[71]			FAHP	–	–
[69]			Neutrosophic AHP	–	–
[68]			AHP	–	–
[72]			TrIT2F‐BWM	–	–
[73]			Direct weight	–	–
[74]				–	Cross‐entropy of UPLZs
[75]			FAHP	–	–
[76]			Hesitant Fuzzy divergence measure	–	–
[77]	Social		FAHP	–
[78]			–	–	AHP + Spherical Fuzzy Entropy
[79]			BWM	–	–
[80]			FAHP	–	–
[44]	Economic	Supply chain	BWM		–
[82]			–	FLQOWA	–
[83]			AHP	–	–
[81]			Direct weight	–	–
[84]			Direct weight	–	–
[85]		Green economy	FANP	–	–
[86]			FAHP	–	–
[87]		Transportation	BWM	–	–
[88]		Environmental	FAHP	–	–
[89]		Business and corporations	–	Entropy	–
[90]			Bayesian BWM	–	–
[91]			Direct weight	–	–
[92]			IF‐MARCOS	–	–
[93]	Technological		DEMATEL	–	–
[94]			Direct weight	–	–
[95]			SWARA	–	–
[96]			LHF‐BWM	–	–

Of the 51 studies, 41 representing 80.39% of the exhaustive studies used subjective weighting methods. Six studies representing 11.76% used objective weighting methods, and four studies representing 7.84% used hybrid weighting. The distribution of subjective methods in terms of numbers and percentages showed that the direct weighting method (also called the direct rating method) ⁹⁹ was the most frequently used among the studies with 34.15%. In this method, the rate/importance of each criterion was directly assigned by the decision maker(s). This criterion was followed by the fuzzy‐AHP (F‐AHP) with 17.07% of the total studies; AHP with 14.63%; the best–worst method (BWM) with 9.76%; and fuzzy analytic network process, DEMATEL, stepwise weight assessment ratio analysis, neutrosophic AHP, linguistic HFSs BWM (LHF‐BWM), Bayesian BWM (BBWM), trapezoidal interval type‐2 fuzzy BWM (TrIT2F‐BWM), fuzzy assessment (FA), HF‐divergence measure and IF‐MARCOS with 2.44% for each method. The distribution of objective methods demonstrated that the entropy method was the most frequently used among the studies at 66.67%. Potentially, all pairwise rankings of all possible alternatives (PAPRIKA) and fuzzy linguistic quantifier order weighted aggregation (FLQOWA) followed the entropy method with 16.67% each. In the same line, the distribution of hybrid methods including SODOSM, LBWA with MACBETH, AHP with spherical fuzzy entropy and cross‐entropy of probabilistic linguistic z‐number with unbalanced semantics (UPLZs) was equal to 33.33% frequency each. In light of the hybridisation definition mentioned before, SODOSM is a hybrid method that combines two assessment processes (subjective and objective) in a single method. In contrast to LBWA with MACBETH and AHP with spherical fuzzy entropy, the subjective and objective methods are integrated to perform the required assessments.

For the distribution of MADM ranking methods used by studies included in this review, 10 out of 51 studies representing 19.61% from the exhaustive studies used the TOPSIS ranking method two times separately and were integrated eight times with the weighting method. Four studies representing 7.84% used the F‐AHP method for ranking, four studies representing 7.84% used VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) and three studies representing 5.88% used AHP. The rest of the ranking methods include the following: TODIM, compressed proportional assessment (COPRAS), spherical normal fuzzy, intuitionistic fuzzy set (IFS), fuzzy preference ranking organisation method for enrichment of evaluations, fuzzy Ranking of Alternatives through Functional mapping of criterion sub‐intervals into a Single Interval (fuzzy RAFSI), IF‐DEMATEL, Fermatean fuzzy sets (FFNs), preference ranking organisation method for enrichment of evaluations, SODOSM, PmFT‐TOPSIS, PAPRIKA, F‐TOPSIS, FLQOWA, BWM, F‐VIKOR, DEMATEL, WASPAS, LHFSs‐TODIM, IMEEM, HFS‐TOPSIS, complex q‐rung orthopair uncertain linguistic weighted partitioned Bonferroni mean (CQROULWPBM), multiobjective optimisation on the basis of a ratio analysis plus the full MULTIplicative form (MULTIMOORA), FA, HF‐ARAS, and IF‐MARCOS. Each of these methods was used by one study, representing 1.96% each.

Moreover, for the distribution of integrated MADM methods used by studies included in this review, 26 of the 51 studies representing 50.98% of the exhaustive studies intended to integrate the MADM methods (ranking and weighting) to solve the related issues of COVID‐19. Noticeably, integrating TOPSIS as a ranking method with weighting methods was the most common practice in this review compared with other MADM methods with 42.30%.

On the other hand, the distribution of originality of MADM methods showed that 63% of the overall studies used an existing method, 25% of the studies extended an existing method and 12% proposed new MADM methods. The devastating effect of the COVID‐19 pandemic has created an unprecedented situation that needs a professional decision support system that can effectively handle the related issues mentioned in Table 1. The result of this review revealed 13 extensions of existing methods, namely, LBWA–MACBETH–RAFSI, ⁴⁹ PmF‐TOPSIS, ⁶⁷ WASPAS, ⁹⁵ AHP‐Entropy‐TOPSIS‐GRA, ⁷⁸ distance measure for IFSs, ⁵⁹ LHF‐BWM‐TODIM, ⁹⁶ BBWM with improved matter‐element extension model (BBWM‐IMEEM), ⁹⁰ cross‐entropy of UPLZs‐MULTIMOORA, ⁷⁴ Entropy‐F‐VIKOR, ⁸⁹ FAHP‐HFS‐TOPSIS, ⁸⁰ HF‐divergence measure‐HF‐ARAS, ⁷⁶ TrIT2F‐BWM‐VIKOR, ⁷² and IF‐MARCOS. ⁹² These methods mainly focus on handling the inaccuracies and uncertainties (vagueness issue) mentioned in Table 1. Among the six new proposed MADM methods that were developed to overcome the limitation of the existing methods, five were under the medical category, namely, SODOSM, ¹⁰ FFNs, ⁶³ spherical normal fuzzy weighted Bonferroni mean (SpNoFWBM), ⁵⁸ FA, ⁶⁵ and CQROULWPBM, ⁶⁴ whereas FLQOWA ⁸³ was under the economic category. SODOSM ¹⁰ was designed to provide better services in the medical sector, which can compute the importance of various criteria with their values within CP and patient decision matrices. In addition, SODOSM can calculate the weight of these criteria subjectively and objectively without increasing the inconsistency in the reliability of the output of decision‐making in ranking the priority of alternative CPs/patients. FFNs ⁶³ use the powerful FFSs in handling uncertain information to provide a better decision in selecting an authentic laboratory for the COVID‐19 test. The SpNoFWBM ⁵⁸ uses the Bonferroni mean operator to improve the process of selecting better antivirus masks similarly to CQROULWPBM. ⁶⁴ FA ⁶⁵ is used to improve the identification process and provide fast remote diagnosis assistance on the cloud. Lastly, FLQOWA ⁸³ is used to solve economic problems where the information is imprecise and under uncertain conditions. FLQOWA combines two operators (fuzzy ontologies and aggregation) to provide a very powerful solution to decision problems and improve the reliability of the results.

3.6. Validation and evaluation

In general, the validity of any proposed work (e.g., method, technique, model and framework) is a crucial step; this vital process becomes indispensable with regard to human life. MADM methods tackle numerous COVID‐19 cases in different categories related to multiattribute complex problems where decision‐making is a challenge. However, the development of a new decision support approach cannot be applicable unless the evaluation and/or validation is achieved. In the literature included in this study, several approaches were used to evaluate and/or validate the proposed decision support approaches. Notably, the evaluation and/or validation in MADM coincides with the principle of testing the reliability and stability of the decision results. ⁵⁶ Six common approaches are presented to evaluate and/or validate the MADM results in the related literature as follows:

• Comparative analysis: One of the most common approaches to verify the strength, steadiness and validity of a proposed model, method or technique in MADM is comparative analysis. In this approach, the superiority of the proposed decision support approach is tested compared with the existing one. The proposed and existing competitors are exposed to the same test situation and circumstances and applied to the same data set to guarantee the efficiency and reliability of the suggested one. ⁵⁷

• Objective validation (mean ± SD): This statistical analysis is applied to validate the ranking results of the proposed MADM framework. In this approach, the rank results of the alternatives are divided into (n) groups. Each group consists of a set of the studied alternatives. The mean ± SD is used to compute each group accordingly using Equations (1) and (2).

  $$\text{Mean}=\frac{1}{n}\sum _{i=1}^n{x}_i,$$ (1)

  $$\mathit{\text{SD}}=\sqrt{\frac{1}{N-1}\sum _{i=1}^N{(x_i-\overline{x})}^2}.$$ (2)

  For example, suppose we divided the rank results of the alternatives into three groups, according to this division, the best value should be for the first group, which must be supported by the mean ± SD result. The statistical measurement of all following groups should be poorer than the first and preceding groups, whereas such measurement should be better or equal to the posterior groups to ensure that the alternative prioritisation undergoes systematic ranking. ⁵²

• Subjective validation (feedback and evaluation from specialists and experts of the study field): This approach depends on the subjective judgment of the specialised experts who are responsible for evaluating and/or validating the outcome of the proposed method. The central and crucial part of this approach is the characteristic and standardisation method applied to determine the experts of the studied field. On this basis, the selected experts will be involved in the evaluation process to provide their feedback and judgment subjectively. ⁵²

• Sensitivity analysis (tweaking the parameters): This analysis tests the influence of changing parameters on the decision‐making results. The values of the operator's parameters ($p,q)$ within a specific interval are tweaked accordingly to observe their impact. This sensitivity can indicate and evaluate the validity and stability of the ranking results in the proposed methodology compared with other MADM methodologies by competitors. ⁵⁶

• Experimental evaluation (case study): In some studies, the experimental assessment by deploying a case study of a real or laboratory data set is used to evaluate the effectiveness of the proposed decision support systems, particularly if the study tests community behaviours or examines the accuracy of diagnosing diseases, as in Pamučar et al. ⁴⁹

• Spearman's rank correlation test: This correlation test is used to examine the difference in the ranking results of the proposed MADM method in competing with existing methods. Spearman's rank correlation test is formulated in Equations (3) and (4).

$$r_s=1-6\sum _{i=1}^n{\left(d_i\right)}^2/n(n^2-1),$$ (3)

$$Z=r_s\sqrt{n-1},$$ (4)

where $r_s \text{and }Z$ are two indicators to assess the relationships between two ranking results. $n$ is the number of alternatives, $d_i$ is the degree of the difference between two comparative ranking results for each alternative and $r_s\in [-\mathrm{1,1}]$. The strength of this relationship increases whenever the value of $|r_s|$ is closer to 1. The value of $Z \text{if equal or greater than }1.645$ indicates a positive relationship; otherwise, the rankings fluctuate. ⁵⁸

As shown in Table 5, the studies included in this review applied different approaches to evaluate and/or validate the decision results. Of the 51 studies, 28 included the evaluation and validation procedures in their process for conformity. In the taxonomy categories, 16 out of the 28 studies with a 57.14% occurrence rate that applied the verification process to their results were under the medical category; six with 21.42% occurrence rate were under the economic category, whereas three studies in each social and technological category with 10.71% occurrence rate used the evaluation and validation process. Unsurprisingly, the comparative study was the most common and dominant approach for testing the reliability and efficiency of the MADM results. This approach was used in 19 works (67.86%) to examine the validity of the decision outcomes. Some studies used only comparative analysis as in [57, 67, 72, 89], whereas others combined this analysis with other approaches to increase the accuracy of the evaluation results as in [90, 94, 96]. This process was followed by sensitivity analysis, which was employed in 10 studies with a 37.71% occurrence rate. Meanwhile, experimental evaluation, objective validation (mean ± SD), subjective validation and Spearman's rank correlation test were the least applied methods with occurrence rates of 14.29%, 10.71%, 10.71%, and 7.14%, respectively. Thus, regardless of the applied approach, evaluation and/or validation is essential for checking the reliability of the proposed MADM method's results.

Table 5.

Summary of the validation and evaluation approaches

Ref	Category	Subcategory	Comparative analysis	Objective validation	Subjective validation	Sensitivity analysis	Experimental evaluation	Spearman's rank correlation test	Results of validation and evaluation
[57]	Medical	Policies	×						The proposed model was compared with most decision‐making methods. Results proved that the flexibility that the spherical fuzzy set provided is more efficient.
[56]						×	×		Sensitivity analysis was used to test the validity of hybrid TODIM, in addition to the experimental evaluation of the urban epidemic situation.
[37]		Services			×				The validity of the proposed framework was proved using medical expertise.
[66]				×					The systematic decision results were proved objectively using a statistical test.
[67]			×						The results of the proposed model were congruous with those of existing MADM methods.
[68]				×					The prioritisation of matching between patients and donors across the centralised or decentralised hospitals was validated statistically.
[72]			×			×			Comparison with existing MADM methods and sensitivity tests proved the superiority of the proposed framework.
[73]			×			×			Fuzzy TOPSIS and fuzzy VIKOR were compared with their hybridisation, and the decision sensitivity was tested.
[74]			×			×			The sensitivity and efficiency of the MULTIMOORA method were tested and proved by comparing with TOPSIS and VIKOR.
[53]		Strategies					×		An experimental evaluation was conducted on the Philippine government's response to the lockdown as proof of concept.
[58]		Tools	×					×	The proposed method was analysed using Spearman's rank correlation test and comparative analysis to verify its efficacy.
[62]							×		The efficiency and accuracy of the proposed model were evaluated using five different lung diseases.
[63]			×						The comparative analysis study proved that the proposed operators were superior compared with the other existing operators and the FF TOPSIS method.
[52]				×	×				Objective and subjective tests used to prove the ranking results were conducted systematically.
[59]			×						The comparison analysis results showed consistent outcomes between the proposed distance function and the existing ones.
[64]			×						The explore operator provides accurate decisions, whereas the ones being compared failed.
[78]	Social	Emergency factors	×						The comparative studies demonstrated the superiority of the proposed decision support system over the existing techniques.
[77]		Social distancing factors	×						The proposed model can play an efficient role in reducing the spread of COVID‐19 compared with other existing models.
[80]		Risk factors	×			×			Different fuzzy forms were compared to test the sensitivity of the obtained rank results in relation to the most significant risk factors.
[83]	Economic	Supply chain				×			The 'Role of Governance (RG)' as the highest priority among other factors was used to analyse the sensitivity and stability of ranking results.
[92]			×			×		×	The proposed method compared with other IF‐MCDM methods support the outcome, and its stability and results correlation is proved.
[85]		Green economy			×				The expert feedback supported the ranking results of the proposed model, where the gasification alternative was kept as the first rank constantly.
[89]		Business and corporations	×						The entropy‐fuzzy VIKOR model showed better performance and accuracy compared with the VIKOR model.
[90]			×			×			The comparison results were consistent, and the sensitivity of risk factors to weight changes is related to the performance value tested.
[76]			×			×			The results of the proposed model were consistent with those of existing methods. Sensitivity analysis proved the reliability of the results.
[95]	Technological		×						The performance of the proposed method was validated by applying comparative analysis with the HF‐COPRAS method.
[94]			×				×		The experimental evaluation and comparison analysis proved the efficiency of the proposed model in helping customers to make the right choice.
[96]			×			×			The proposed method was compared with existing methods to determine its reliability, and the changing weight of experts shows the impact on the results.

3.7. Bibliography analysis

In this section, the country in which the study was performed was statistically analysed, as shown in Figure 6. The journals that were targeted to publish this type of research and studies were presented. Moreover, the statistical information about the amount of resources used in previous studies in terms of their nature (i.e., whether the values were fuzzy numbers or crisp numbers) is presented in this section.

Figure 6.

Distribution by country of study [Color figure can be viewed at wileyonlinelibrary.com]

Figure 6 shows the 16 countries in which the studies included in this review were conducted. The selected articles' geographical distribution in terms of numbers and percentages shows that the most prolific contributors were from India ⁵⁹ , ⁶³ , ⁶⁵ , ⁷³ , ⁷⁵ , ⁷⁶ , ⁷⁹ , ⁸⁰ , ⁸² , ⁸³ , ⁸⁸ , ⁹³ with 12 studies; China ⁵⁶ , ⁵⁸ , ⁷² , ⁷⁴ , ⁸⁶ , ⁹⁰ , ⁹⁶ with seven studies; Turkey, ⁵¹ , ⁶⁰ , ⁷¹ , ⁹¹ , ⁹² Pakistan, ⁵⁴ , ⁵⁷ , ⁶⁷ , ⁷⁸ , ⁸⁵ and Malaysia ¹⁰ , ³⁷ , ⁵² , ⁶⁸ , ⁸⁹ with five studies each; Saudi Arabia ⁶⁹ , ⁷⁷ , ⁹⁴ with three studies; Iraq, ⁶¹ , ⁶⁶ Egypt, ⁴⁴ , ⁶² Iran, ⁵⁰ , ⁸¹ and Italy ⁷⁰ , ⁸⁷ with two studies each; and Serbia, ⁴⁹ Philippines, ⁵³ Brazil, ⁵⁵ Vietnam, ⁹⁵ Taiwan, ⁶⁴ and the USA ⁸⁴ with one study each.

Additionally, the total number of journals that included articles in different categories was 39. The distribution of journals in terms of the numbers and percentages of published articles in this review showed that the most prolific journals were Applied Soft Computing Journal ⁷⁶ , ⁹² , ⁹⁵ , ⁹⁶ and Journal of Cleaner Production, ⁷³ , ⁸³ , ⁸⁵ , ⁹³ with four papers each. These journals were followed by the Sustainability (MDPI) ⁴⁹ , ⁸⁷ , ⁸⁸ with three published articles and IEEE Access, ⁶¹ , ⁷⁴ International Journal of Intelligent Systems, ⁵⁹ , ⁷⁸ Results in Physics, ⁶⁹ , ⁸⁰ and Soft Computing ⁵⁷ , ⁷⁹ with two articles each. Meanwhile, one article was published in each of the following journals: Journal of Healthcare Engineering‐Hindawi, ⁵¹ Socioeconomic Planning Sciences, ⁵³ Science of the Total Environment, ⁵⁵ Complexity, ⁵⁶ International Journal of Environmental Research and Public Health, ⁵⁸ Computational and Mathematical Methods in Medicine, ⁶⁰ Health Informatics Journal, ⁶² Journal of Infection and Public Health, ⁵² Mathematical Problems in Engineering, ⁶³ International Journal of Healthcare Management, ⁵⁰ International Journal of Information Technology and Decision Making, ³⁷ Artificial Intelligence in Medicine, ⁶⁶ Computer Methods and Programs in Biomedicine, ¹⁰ International Journal of Biomathematics, ⁶⁷ International Journal of Infectious Diseases, ⁷⁰ International Journal of Disaster Risk Reduction, ⁷¹ International Journal of Intelligent Engineering and Systems, ⁷⁷ Transportation Research Interdisciplinary Perspectives, ⁴⁴ Systematic Reviews in Pharmacy, ⁸¹ International Journal of Logistics Research and Applications, ⁸² Journal of Latex Class Files, ⁹⁴ Applied Intelligence, ⁶⁸ Entropy, ⁸⁹ Environmental Science and Pollution Research, ⁸⁶ International Journal of Hospitality Management, ⁹¹ Journal of Healthcare Leadership, ⁵⁴ Knowledge‐Based Systems, ⁷² Materials Today: Proceedings, ⁶⁵ Mathematics, ⁹⁰ Operational Research Society of India (OPSEARCH), ⁷⁵ Resources, Conservation & Recycling, ⁸⁴ and Symmetry. ⁶⁴

In the application of MADM methods, the criteria can be crisply or fuzzily defined. In this review, 19 studies used a crisp environment, representing 37.25% of the total studies. Meanwhile, 32 works used a fuzzy environment, representing 62.75% of the total studies (Figure 7). The distribution in terms of numbers and percentages showed that the fuzziest method was triangular fuzzy numbers (TFN), ⁴⁹ , ⁵⁰ , ⁵¹ , ⁶⁰ , ⁶⁵ , ⁷¹ , ⁷³ , ⁷⁵ , ⁷⁷ , ⁸⁰ , ⁸⁵ , ⁸⁶ , ⁸⁹ which was used by 13 studies. Then, the triangular neutrosophic set (TNS), ⁴⁴ , ⁶² , ⁶⁹ spherical fuzzy number set (SFN) ⁵⁷ , ⁵⁸ , ⁷⁸ and IFS ⁵³ , ⁵⁹ , ⁹² were all used by three studies. Hesitant fuzzy numbers (HFN) ⁷⁶ , ⁹⁵ was used by two studies. Meanwhile, Fermatean fuzzy set (FFSs), ⁶³ intuitionistic m‐polar fuzzy topology (ImFT), ⁶⁷ order weighted aggregation (OWA) operator, ⁸² fuzzy membership function‐fuzzy linear (FMF‐FL), ⁸⁸ linguistic HFSs (LHFSs), ⁹⁶ trapezoidal interval type‐2 fuzzy (TrIT2F), ⁷² complex q‐rung orthopair uncertain linguistic sets (CQROULSs), ⁶⁴ and Probabilistic Linguistic Z‐number with Unbalanced semantics (UPLZ) ⁷⁴ were used by each study.

Figure 7.

Distribution by fuzzy sets used [Color figure can be viewed at wileyonlinelibrary.com]

4. POTENTIAL FUTURE DIRECTIONS

For future research, a further examination should focus on the role of MADM in different unused contexts to mitigate any issues that might arise when a similar incident recurs. As discussed in Section 3.2, the related studies were analysed and taxonomised in four main categories: medical, social, economic, and technological. Furthermore, several studies determined customers' intentions and trust in social commerce during COVID‐19. ¹⁰⁰ , ¹⁰¹ , ¹⁰² However, the previous studies did not consider and assess the multigroup modelling for customers and firms during COVID‐19, which is considered a research gap. Such a research gap is related to economic or/and social categories of the proposed taxonomy and might be addressed in future research.

In this context, the evaluation and ranking of the customers of each firm would determine the most intention of customers to use social commerce. The social commerce firms would give high support, such as discounts, coupons, points of purchase and gifts, to such customers to reduce the costs of advertising and increase marketing promotion. ¹⁰⁰ , ¹⁰³ From another perspective, the evaluation and ranking of the firms can provide valuable insights into the best firm on the basis of using social commerce. ¹⁰¹ Furthermore, the social commerce firms would improve social commerce construct. The firms listed at the last of the ranking can adhere to firms with the highest‐ranking models. Firms with the highest ranking may develop their social commerce construct to respond to commerce challenges for such a pandemic. ¹⁰⁴ However, the evaluation and ranking of firms and customers of each firm are considered a challenge because of the following four main issues: (1) the supporting criteria of trust in social commerce during COVID‐19 for firms and customers within each firm should be identified and determined, (2) weight is assigned for each supported criterion of firms and customers, (3) the supported multievaluation criteria must be considered in the evaluation of firms and customers, ¹⁰⁴ and (4) the maximisation and minimisation goals for social commerce criteria of firms and customers increase the variation complexity, resulting in the fourth issue of data variation. ¹⁰⁵

Accordingly, four sequential methodology stages will be proposed as follows (Figure 8). First, based on literature review analysis, identifying the related criteria of firms and customers for trust in social commerce during COVID‐19 must be achieved. Second, based on the identified criteria, a multigroup structural equation modelling approach must be adopted to assess and determine the supporting criteria of firms and customers for trust in social commerce during COVID‐19. Third, an artificial neural network technique must be developed to assign weight in supported criteria of firms and customers. Fourth, two decision matrices should be formulated. (I) firms' decision matrix will be presented on the basis of the intersection of the 'supported and weighted firms' criteria' and 'firms list'. (ii) Customers' decision matrices within each firm will be presented based on the intersection of the 'supported and weighted customers criteria' and ‘customers list'. Then, based on the formulated decision matrices, MADM methods must be used for ranking the firms and customers thoroughly. ¹⁰⁶ , ¹⁰⁷ Moreover, the multiexpert decision‐making might be addressed to improve the consistency of the individual opinion of experts involved in the evaluation and ranking of firms and customers. ¹⁰⁸

Figure 8.

Potential future directions of social commerce based on MADM methods [Color figure can be viewed at wileyonlinelibrary.com]

Such future work is vital and has two main significances. First, the social commerce firms will improve when promoting the social commerce of people during COVID‐19 by recognising the most vital customers to provide them with effective services. Second, the social commerce firms will improve the social commerce construct emerging from the COVID‐19 crisis worldwide.

5. CONCLUSION

Since the outbreak of the COVID‐19 pandemic, the number of studies that reflect researchers' efforts is continually increasing. Studying the influencing criteria and finding the best substitute in the various sectors and aspects that are crucially affected by COVID‐19 to make a proper decision are necessary. In this study, a comprehensive review of how MADM approaches assisted with different applications in the midst of COVID‐19 were systematically conducted. Following the SLR protocol presented in Figure 2, four reliable scientific database engines have been searched three different times to extract the relevant publications of COVID‐19 based on MADM techniques. In total, out of the initial publication numbers (i.e., 879 studies), 51 studies were classified into four main categories including medical (n = 30), social (n = 4), economic (n = 13), and technological (n = 4). Those categories were associated with their specific subcategories, as shown in Figure 3. Moreover, the medical studies were distributed into four subcategories, namely, strategies, policies, tools, and services. Regarding the social category, the studies focused on different social aspects including social distancing factors, emergency factors, risk factors, and risk level. The economic studies discussed interesting topics, such as supply chain, green economy, transportation, environment and business and corporations. Moreover, the technological studies discussed the topics of smart systems, telecom service, digital technology, and E‐Learning.

Furthermore, a comprehensive analysis was conducted for each category to address the issues and challenges, the literature contributions, the employed data set, evaluation criteria types, MADM techniques types, validation and evaluation approach and different bibliography analysis. Several issues and challenges were analysed and discussed including multiattribute, inconsistency, time consumption, unnatural comparison, vagueness, normalisation, distance measurement, outranking, trade‐off, conflict criteria, the importance of criteria, and data variation. In this regard, multiattribute and importance criteria issues were clearly discussed in all articles within all categories except for the economic category, which showed that the issue of the importance of criteria dominated in all studies except for one study only. The main analysis of the data set was discussed and presented in terms of the primary and secondary data sets, sample size and number of experts. This analysis showed that the majority of studies used the primary data set, combined primary and secondary data sets and secondary data sets. Going through the conducted analysis of the evaluation criteria types, two types of evaluation criteria (i.e., subjective and objective criteria) were discussed. Previous studies used either only one type or combined the two types depending on the identified criteria in their case. The analysis showed that most studies used the objective criteria type, subjective criteria type and both. Furthermore, the academic literature showed that three different directions in using MADM techniques exist, namely, ranking, weighting and hybrid methods. The main highlight of such analysis showed that the existing MADM methods were most frequently used followed by the extended existing MADM method and proposed new MADM methods. In the analysis of validation and evaluation approaches, six common approaches were found and discussed, namely, comparative analysis, objective validation, subjective validation, sensitivity analysis, experimental evaluation and Spearman's rank correlation test. In this regard, comparative study was most frequently used for testing the reliability and efficiency of the results followed by sensitivity analysis, experimental evaluation, objective validation, subjective validation and Spearman's rank correlation test. In addition, the final set of articles were distributed based on different bibliography analyses including the country of study, journals and fuzzy set used. This analysis has shown that the most prolific contributors were from India, the most prolific journal was Applied Soft Computing Journal and the fuzziest method was TFN. As a future recommendation, the authors of this study intend to give insight into the second scope of MCDM methods (i.e., MODM), which was also used to combat the COVID‐19 pandemic.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

Mohammed Assim Alsalem, Rawia Mohammed, Osamah Shihab Albahri, and Abdullah Hussein Alamoodi: Data curation, Writing‐Original draft preparation and Visualization. Aws Alaa Zaidan, Kareem Dawood, Alhamzah Alnoor, Ahmed Shihab Albahri, Bilal Bahaa Zaidan, and Hassan Alsattar: Investigation, Software, and Writing‐Reviewing. Osamah Shihab Albahri, Mamoun Alazab, and Fawaz Jumaah: Conceptualization, Methodology, Supervision and Editing.

APPENDIX A.

Table A1.

Evaluation criteria description

Ref	Category	Subcategory	Evaluation criteria
			Criteria	Subcriteria
[49]	Medical	Strategies	C1: Assessment of which strategic guideline provides the best results in the circumstances considered	NA
			C2: Assessment of the quality of services provided for patients treated in an infectious disease ward of a clinical centre
			C4: Assessment of the quality of services provided in noninfectious disease units of a clinical centre
			C5: Assessment of the electiveness of preventing the spread of infection in the considered region/country
			C6: Assessment of the ratio of the cost for implementing the strategic guideline to the quality of performance of the healthcare center within the strategic guideline
[53]			C1: Compliance of minimum public health standards	NA
			C2: Limited movement of persons
			C3: 24‐hr curfew of minors and senior citizens
			C4: Work in government at full operational capacity
			C5: Limited operational capacity of diplomatic missions and international organisations
			C6: Full operational capacity of category I industries
			C7: Minimum of 50% operational capacity of category II industries.
			C8: Maximum of 50% operational capacity of category III industries
			C9: Limited operations of malls and shopping centers
			C10: Allowed operation of essential public and private construction projects
			C11: Nonoperation of category IV industries
			C12: Nonoperation of hotels or similar establishments
			C13: Suspension of physical classes
			C14: Prohibition of mass gatherings
			C15: Reduced capacity of public transportation
[51]			C1: Quarantine/lockdown of patients and those suspected of infection	NA
			C2: Internal border restrictions are reducing the ability to move freely (transportation) within a country
			C3: Social distancing
			C4: Health monitoring
			C5: Public awareness campaigns
			C6: Restriction of nonessential businesses
			C7: Restrictions of mass gatherings
			C8: External border restrictions are reducing the ability to exit or enter a country
			C9: Closure of schools
			C10: Government policies that affect the country's health resources (materials and health worker
			C11: Formation of new task units/bureaus and government policies changing administrative capacity to respond to the crisis
			C12: Common health testing (independent of suspected infection)
			C13: Curfew
			C14: Restriction of nonessential government services
			C15: Declaration of emergency
[54]			C1: Social distance (SD)	S1: Travel
				S2: 3Cs
				S3: Shaking hands and hugging people
			C2: Hygiene (H)	S1: Wearing a mask
				S2: Hand wash
				S3: Workplace sanitization
			C3: Sharing personal items (SPI)	S1: Phone
				S2: Combs
				S3: Crockery
				S4: Sanitary item
				S5: Stationary
			C4: Unnecessary touching things (UTT)	S1: Public objects
				S2: Own body part
				S3: Doorknob/stair bannisters
			C5: Healthy Diet (HD)	S1: Mix whole grain foods
				S2: Avoid processed, baked and fried foods
				S3: Drink enough water
			C6: Immunity/Fitness (I/F)	S1: Exercise regularly
				S2: Get adequate sleep
				S3: Minimize stress
				S4: Maintain a healthy weight
			C7: Monitor your health (MH)	S1: Shortness of breath
				S2: Fever
				S3: Sore throat
				S4: Congestion/runny nose
[55]		Policies	C1: Land use	S1: Airport: Count of national and international flights
				S2: Port: Quantity of goods movement by port
				S3: Interstate bus terminal: The number of passengers that travelled to each municipality by interstate bus
				S4: Educational Institutions: accounted for elementary schools, middle schools, high schools, colleges, and universities
	S5: Urbanisation: The number of people living in urban areas and in rural areas
	C2: Socioeconomic		S1: Income
			S2: Business activities: Information on the proportion of people above 18 years old works in different economic sectors
	C3: Population		S1: Age Group: 0–9 years old, 10–19 years old, 20–64 years old, and above 64 years old.
	C4: Health conditions		S1: Age Group: The number of hospital admissions for heart diseases, lung diseases, and diabetes for each age group in 0–9 years old, 10–19 years old, 20–64 years old, and above 64 years old.
	C5: Healthcare system		S1: Hospital beds and number of staff: The number of hospital beds and staffs
[56]	C1: Confirmed diagnosis rate per one million population		NA
	C2: The cure rate of a confirmed case
	C3: The mortality rate of confirmed cases
	C4: The growth rate of confirmed cases in three days
	C5: Number of beds per capita of confirmed cases
	C6: People's awareness of the epidemic
	C7: Government's attitude toward the epidemic. Obviously, for evaluating the severity of the urban epidemic situation
[57]	C1: High temperature and high humidity: High temperature and humidity positively correlate with the number of COVID‐19 cases		NA
	C2: Resources and facilities: Infrastructure: Hospitals, Lab, and in‐patient beds Transport: Ambulance and other vehicles Skilled personnel: Doctors and nurses Working cost: Cost for the building, medical equipment, drugs, fees for staff commitment and other various purposes
	C3: Socio‐demographic variable: People who work in urban areas, where the hygienic levels are lower more likely to get COVID‐19 infection
	C4: Quarantine: People may spend most of their time at home, due to this period no more spreading of COVID‐19 deduction
[45, 46, 51]	Tools	C1: Leakage rate: The adhesiveness of the mask structure design to cover the human face	NA
		C2: Reutilizability/Reusability
		C3: Quality of raw materials
		C4: Filtration and efficiency: Which means the filtration efficiency of non‐oily 0.3 µm particles is greater than 95%, and it must also have medical protection requirements such as surface moisture resistance and blood barrier
[60]		C1: No symptom‐window period	NA
		C2: Early stage of infection
		C3: The active phase of infection
		C4: Late of the recurrent stage of infection
		C5: Past infection
		C6: Recovery stage of infection
		C7: Cost
		C8: Accessibility
		C9: Usability
		C10: Equipment
		C11: Education
		C12: Sensitivity
		C13: Specificity
		C14: False positivity
		C15: False negativity
[61]		C1: Recall: It measures the classifier's ability to produce all applicable subject	NA
		C2: Precision tests the classifier's ability to reject irrelevant subjects
		C3: Accuracy refers to a measurement's closeness parameter when reading the data value against the actual data values
		C4: F‐Score: a recall and precision weighted average, where a score of F1 achieves its worst at 0 and best value at 1. The recall and precision relative contribution to the score of F1 are similar
		C5: Area Under the Curve (AUC): is a metric of degree or separability
		C6: FPR: output predicted as false positive (FP)
		C7: FNR: output predicted as false negative (FN)
		C8: TNR: output predicted as true negative (TN)
		C9: TPR: output predicted as true positive (TP).
		C10: Time Complexity: the execution time to provide a diagnosis result
[62]		Symptoms Criteria are:	NA
		C1: Chills
		C2: Nasal congestion
		C3: Headache
		C4: Cough
		C5: Sore throat
		C6: Sputum production
		C7: Fatigue
		C8: Shortness of breath
		C9: Fever
		CT imaging result criteria are:
		C10: Nonappearance of Both Ground‐Glass Opacities and Consolidation (Normal CT)
		C11: The occurrence of Ground‐Glass Opacities
		C12: The occurrence of Ground‐Glass Opacities with or without Consolidation
		C13: The occurrence of Ground‐Glass Opacities with Consolidation without effusion
		C14: The occurrence of Ground‐Glass Opacities with Consolidation effusion
[52]		Binary classification criteria are:	NA
		C1: Accuracy: Overall effectiveness of a classifier
		C2: Precision: Class agreement of the data labels with the positive labels given by the classifier
		C3: Recall: Effectiveness of a classifier to identify positive labels
		C4: F score: Relations between positive data labels and those given by a classifier
		C5: Specificity: How effectively a classifier identifies negative labels
		C6: Area Under the Curve (AUC): Classifier's ability to avoid false classification
		Multiclass classification criteria are:
		C7: Average accuracy: Average per‐class effectiveness of a classifier
		C8: Error rate: Average per‐class classification error
		C9: Precisionµ: Agreement of the data class labels with those of classifiers if calculated from the sums of per‐sample decisions
		C10: Recallµ: Effectiveness of a classifier to identify class labels if calculated from the sums of per‐sample decisions
		C11: F scoreµ: Relations between positive data labels and those given by a classifier based on the sums of per‐sample decisions
		C12: PrecisionM: Average per‐class agreement of the data class labels with those of a classifier
		C13: RecallM: Average per‐class effectiveness of a classifier to identify class labels
		C14: FscoreM: Relations between positive data labels and those given by a classifier based on a per‐class average
		Multilabelled classification criteria are:
		C15: Exact match ratio: Average per‐sample exact classification
		C16: Labelling F score: Average per‐sample classification with partial matches
	C17:Retrieval F score: Average per‐class classification with partial matches
	C18:Hamming loss: Average per‐example per‐class total error
	Hierarchical classification criteria are:
	C19: Precision↓: Positive agreement on subclass labels concerning the subclass labels given by a classifier
	C20: Recall↓Positive agreement on subclass labels concerning the subclass labels given by data
	C21: F‐score↓: Relations between data positive subclass labels and those given by a classifier
	C22: Precision↑: Positive agreement on superclass labels concerning the superclass labels given by a classifier
	C23: Recall↑: Positive agreement on superclass labels concerning the superclass labels given by data
	C24: F‐score↑: Relations between data positive superclass labels and those given by a classifier
[63]	C1: Time limits	NA
	C2: Accurate results
	C3: Location flexibility for the client
[65]	C1: Chronic obstructive pulmonary disease (COPD)	NA
	C2: Dry cough
	C3: fever
	C4: presence of weakness
	C5: difficulty in breathing
	C6: respiratory rate
	C7: oxygen saturation
	C8: WBC availability
	C9: CRP elevation
	C10: lung texture abnormality
[37]	Services	C1: C‐reactive protein (CRP), mg/L: CRP is \an annular (ring‐shaped), pentameric protein can be found in blood plasma, whose circulating concentrations rise in response to inflammation	NA
		C2: Eosinophils, x109/L: Eosinophils or acidophils are \a variety of white blood cells and one of the immune system components accountable for combating multicellular parasites and certain infections in vertebrates
		C3: Eosinophil ratios: The reference range is between 0.4% and 8%
		C4: Lymphocytes, x109/L: They include WBCs and are one of the body's main types of immune cells; they are made in the bone marrow and can be found in blood and lymph tissues
		C5: Lymphocyte ratios: The reference range is between 20% and 50%.
		C6: Neutrophils, x109/L: A type of WBC (or granulocyte) protects humans from infection among other functions
		C7: Neutrophil ratios: The reference range is between 40% and 75%.
		C8: White blood cell count (WBC), x109/L: Comprises leukocytes or leucocytes, the cells of the immune system involved in protecting the body against infectious diseases and foreign invaders
[66]		C1: White blood cell count	NA
		C2: Neutrophil count
		C3: Lymphocyte count
		C4: Haemoglobin
		C5: Blood platelet count
		C6: Albumin
		C7: C‐reactive protein
		C8: Interleukin
[70]		C1: Age (18–50, 50–70, and >70)	NA
		C2: Body mass index (BMI: <30, 30–40, and >40
		C3: Comorbidities (diabetes, preexisting respiratory/cardiovascular diseases, and onco‐haematological diseases)
		C4: Respiratory rate (<20 breaths/min and >20 breaths/min)
		C5: PaO2 (>80 mmHg, 70–80 mmHg, and <70 mmHg)
		C7: Peripheral oxygen (O2) saturation (>96%, 92–96%, and <92%)
		C8: Findings at chest X‐ray (normal, consolidation, and bilateral interstitial lung abnormalities)
		C9: Modified Early Warning Score‐MEWS (Subbe et al., 2001), a clinical scoring system including pulse rate, respiratory rate, systolic blood pressure, body temperature, and neurological symptoms (score: 0–2 and 3–4)
		C10: Duration of symptoms before hospital evaluation (<3 days, 4–7 days, and >7 days)
		C11: C‐reactive protein (CRP: normal/high by local cut off)
		C12: Living with vulnerable people (i.e., people with comorbidities, pregnant women, or immunosuppressed patients).
[50]		The Criteria considered under normal conditions are:	NA
		C1: Tangibility: Appearance of physical facilities, equipment, personnel and communication equipment
		C2: Reliability: The service provider organisation's ability to provide service according to its promises accurately and consistently
		C3: Responsiveness: The tendency and desire of the organisation to help patients and provide timely and prompt service
		C4: Assurances: Knowledge, skills and competencies of staff and the organisation's ability to induce trust and confidence in patients
		C5: Empathy: Taking care of the patient and personal attention to each patient
		C6: Admission: Accelerating the patient admission process
		Criteria considered during the outbreak of COVID 19 are:
		C7: Agility: The agility of a hospital reflects the hospital's affective response to internal and external changes and threats and turns these changes into hospital growth opportunities
		C8: Resilience: Resilience represents the hospital's capacity to absorb, adapt, and change when faced with a shock so that it can maintain its structure and perform its current activities. The hospital is resilient when it can provide health services during and after disasters and save lives
		C9: Updating: Speed of access to new information and studies on recent advances in the treatment of COVID 19
		C10: Updating: Speed of access to new information and studies on recent advances in the treatment of COVID 19
[10]		C1: PAO2/FIO2	NA
		C2: C‐Reactive protein
		C3: Cytokine/chemokines
		C4: Albumin
		C5: IgM/IgG
		C6: Peroxiredoxin II
[67]		C1: Respiratory issues	NA
		C2: High‐grade fever
		C3: Cough
		C4: Shortness of breath
[71]		C1: Hospital buildings: It is defined as the structural preparedness of hospitals when facing disaster events. It covers installed capacity, location, and infrastructure quality criteria	S1: Physical infrastructure
			S2: Location
			S3: Number of floors
			S4: Capacity
			S5: Disaster gathering area
			S6: Insulation
			S7: Ventilation
		C2: Equipment: It is about nonstructural preparedness. This criterion focuses on the availability of quality and quantity of the equipment	S1: Medicine
			S2: Potential hazardous substance
			S3: Material safety management
			S4: Medical equipment for ES
			S5: Power generator
			S6: Drinking water
			S7: Tent
			S8: Food
	S9: Bed
	S10: Triage tag
	S11: Finance
	S12: Supply source
	C3: Communication: It concerns the information technology ability of the hospital under disaster	S1: Emergency network
		S2: Communication tools/device
		S3: Information quality
	C4: Transportation: This criterion specifies the hospital's transportation resources and the roads' status to the hospital in case of disaster	S1: Number of vehicles
		S2: Helipad space
		S3: Safety
		S4: Accessibility
	C5: Personnel: This criterion shows the capability of the hospital regarding the qualification and quantity of personnel	S1: Education
		S2: Disaster drill
		S3: Emergency response team
		S4: Coordination
		S5: Number of personnel
		S6: Working hours
	C6: Flexibility: This criterion considers hospitals' ability to expand their capacity to admit patients' most significant possible number.	S1: Flexibility in the use of facilities
		S2: Contingency staff
		S3: Blood bank
		S4: The supply chain of medicines and supplies
[69]	C1: Age index AC	S1: Older adults with health problem OWH: This group refers to older adults whose age is more than 60 years old and have some health problems
		S2: Older adults without health problem OUH: It indicates to the elderly but in good health
		S3: Adult people with health problem AWH: This category refers to the age group between 18 to less than 60. Besides, it is assumed that they suffer from health problems such as lack of immunity, diabetes, pressure, and other diseases that may cause death if they are infected with COVID‐19
		S4: Adults people without health problem AUH: It refers to young people in good health condition
		S5: Kids with health problem KWH: This group's age is less than 18 years; moreover, the kids suffer from health problems
		S6: Kids without health problem KUH: It indicates to the kids in good health
	C2: Health state index HS	S1: For people with high‐risk health problems PHR
		S2: People with health problems PWH: It refers to the individuals having health problems but not severe diseases
		S3: Healthy people PUH: It indicates the individuals in good health condition
	C3: Women state index WC	S1: Pregnant WP: During the pregnancy, the women usually have a weak immunity system and are susceptible to disease, so pregnant women have been classified as a community group that has priority to take the COVID‐19 vaccine over others
		S2: Lactating women WL: It points to breastfeeding women and has priority because being infected with the COVID‐19 virus will have severe complications for her and her infant kid
		S3: Others women WO: It refers to other women who are not pregnant or breastfeeding
	C4: Job kind index JK	S1: Health workers HP: All individuals are working in health care places who have direct or indirect exposure to patients or infectious materials as well as people who are not directly involved in patient care but who may be exposed to infectious agents while working in a health care environment, Such as doctors, nurses, lab technicians, and administrative staff. Health workers are the first line of defense to fight the COVID‐19 virus
		S2: Essential workers EW: This class is essential for the life continuity and maintenance of basic services, such as logistics, supply, agriculture, transport, education, hygiene, energy, security, armed forces, and the judiciary. But priority should be given to those who cannot work remotely more than others
		S3: Other workers OS: It indicates to the workers in the other sectors
[68]	C1: PAO2/FIO2: is defined as the ratio of the partial pressure of arterial oxygen to the percentage of inspired oxygen, and its reference range must be between 100 and 300	NA
	C2: C‐reactive protein: CRP is a serum amyloid P component belonging to the pentraxin family of calcium‐dependent ligand‐binding proteins. It serves as a marker of inflammation and ranges between 8 and 250
	C3: IL‐6 pg/ml (Cytokines): is released by T cells and activated macrophages during the acute‐phase response following injury or trauma and may lead to inflammation or infection; it should be between 6 and 300. IL‐6 has pro‐ and anti‐inflammatory properties
	C4: Albumin: is an essential binding and transport protein for various substances in plasma and maintains the osmotic pressure of blood. The reference range is between 5 and 55
	C5: IgM ELISA titer: is used to detect immunoglobulin M (IgM) and IgG antibodies against capsular and O antigens of Haemophilus influenza. It ranges between 100 and 800
[72]			C1: Traffic convenience	NA
			C2: Environmental protection
			C3: Geographical position
			C4: Infrastructure
			C5: Regional communication convenience
			C6: Capacity
			C7: Reconstruction difficulty
			C8: Reconstruction cost
[73]			C1: Annual operating cost	NA
			C2: Public acceptability
			C3: Reliability
			C4: Treatment efficiency
			C5: Human resource requirement
			C6: Treatment system capacity
			C7: Waste residuals
			C8: Toxic emissions and health effects
			C9: Operational safety
			C10: Infrastructure rRequirement
[74]			C1: Antiviral activity	NA
			C2: Coolify
			C3: Ease breathing
			C4: Side effect
[75]			C1: Transportation	S1: Patient movement
				S2: Movement for equipment
				S3: Improper layout
			C2: Inventory	S1: Excessive material stock
				S2: Excessive data storage
				S3: Emergency medicine unavailability
			C3: Motion	S1: Staff movement
				S2: Equipment unavailability
				S3: Staff location
			C4: Waiting	S1: Waiting of Diagnosis
				S2: Waiting for Admission
				S3: Waiting for discharge
			C5: Overproduction	S1: Lab tests
				S2: Scheduled follow‐up
				S3: Unwanted treatment
			C6: over processing	S1: Patients information
				S2: Test duplication
				S3: Process duplication
			C7: Defects	S1: Readmission
				S2: Equipment errors
				S3: Procedural errors
[76]			C1: Anorexia	NA
			C2: Cough
			C3: Fatigue
			C4: Fever
			C5: Myalgia
			C6: Shortness of breath
			C7: Sputum production
[77]	Social		C1: Number of susceptible	NA
			C2: Number of infected
			C3: Number of removed (as death or recovered)
[78]			C1: Clinical management: Vaccination after the virus's dispersal is a massively effective means of reducing these deadly diseases	NA
			C2: First‐aid training: This disease spread very quickly, so to control this virus first, trained, or avoid people of these disease symptoms
			C3: Increased personal protective equipment: The lack of testing kits is another factor, the situation will be improved with increased production of testing kits, the loosing of confirmation criteria and the local governments' decision to threat and finally quarantine all suspected cases
			C4: Trained technician: It is swift to share the virus's genetic makeup to enable the rest of the world to begin developing specific screening and start working on potential vaccines
			C5: Banned intra‐city transportation: The disease caused by the virus is severe. For local people's safety, it is necessary for local government that takes a step or announced to banned intra‐city transportation to force patients to local community clinics
			C6: Global uncertainty: The economic fallout from Coronavirus: Rapid reduction in the transport and hospitality sectors will damage the economy in the short term and harm consumption and trade in the first quarter
			C7: Country‐level coordination and planning: Each government needs the best level of cooperation with its province/states to prepare to overcome the novel uncertainty in the form of COVID‐19
			C8: Monitoring: Every government should appoint health and emergency decision‐making experts to assess and track every country's current situation and advise how to improve the situation
[79]			C1: Individual Factors	S1: Age
				S2: Gender
				S3: Obesity
				S4: Smoking
				S5: Physical activity
			C2: Psychosocial Factors	S1: Job Strain
				S2: Job Satisfaction
				S3: Repetitive Work
				S4: Task/activity type
			C3: Physical Factors	S1: Duration work
				S2: Posture
				S3: Force exertion
				S4: Poor Design
[80]			C1: Doctor's opinion	NA
			C2: Literature review
			C3: Media
[44]	Economic	Supply chain	C1: Policy 1: International and domestic air suspension.	NA
			C2: Policy 2: Close the commercial centres, shops, restaurants, cafes and nightclubs.
			C3: Policy 3: Suspending tourist activities.
			C4: Policy 4: Suspension of religious rites.
			C5: Policy 5: Internal transportation and High‐Speed Rail train services have been suspended.
			C6: Policy 6: Postponing studies in schools and universities.
			C7: Policy 7: Reducing the employment rate in some jobs determined by the state.
			C8: Policy 8: Converting the work system to remote work from home.
			C9: Policy 9: Suspension of maritime traffic.
[81]			C1: Market change	NA
			C2: Change in competitiveness
			C3: Change in customer needs
			C4: Change in social needs
			C5: Technology change
			C6: Improve product quality
			C7: Reduce product life
			C8: Reduce production costs
			C9: R&D innovation and development
			C10: Increasing the number of suppliers
			C11: Continuous assessments and point improvement points
			C12: Responsiveness
			C13: Persons
			C14: Flexibility
			C15: Speed
			C16: Solve liquidity problems including receipt of claims
			C17: Participation
			C18: Timely delivery
			C19: Reduce my Lead
			C20: Solve liquidity problems including receipt of claims
			C21: Customer relationship;
			C22: Information systems
			C23: Product basket management
[83]			C1: Supply‐side	S1: Supplier quality problems
				S2: Payment default by the supplier for the availed services (transportation, wages etc.).
				S3: The underperformance of logistics providers
				S4: Supply shortages
			C2: Demand‐side	S1: Uncertain & unanticipated demand
				S2: Inadequate Information on demand
				S3: Changes in food safety requirements
				S4: Transportation issues
			C3: Logistics and infrastructural	S1: Inadequate road infrastructure
				S2: Increase in fuel costs
				S3: Lack of transportation infrastructure
				S4: Conflicts, labour disputes, labour shortages
				S5: Lack of infrastructure and service units
			C4: Policy and regulatory	S1: Uncertain monetary, fiscal, tax policies
				S2: Uncertain legal policies and enforcements
				S3: Uncertain trade and market policies
				S4: Uncertain land policies and tenurial systems
			C5: Financial	S1: Lack of financial support
				S2: Delays in accessing financial support
				S3: Uncertain credit support
				S4: Uncertain interest and exchange rate policies
			C6: Biological and environmental	S1: Pests, diseases, yield losses
				S2: Contamination related to inadequate sanitisation and illnesses
				S3: Contamination affecting food safety
				S4: Degradation of processed food products
			C7: Management and operational	S1: Poor management decisions
				S2: Poor quality control
				S3: Planning and forecast errors
				S4: Use of outdated inputs
			C8: Weather‐related	S1: Periodic deficit/excess rainfall
				S2: Extreme drought
				S3: Flooding
	S4: Extreme winds/cyclone
	C9: Political	S1: Political instability/crisis
		S2: Trade interruptions/restrictions
		S3: Changes in the political environment
		S4: Legislation risks
[82]	C1: Economics, trade and commerce: Efforts to revive the industries can be made through banks and special financial aid packages provided by the government	S1: Support to industries & customers extended through banks
		S2: Additional loans are required for enabling MSMEs and stressed sectors for overcoming the loss of businesses
		S3: National economic and investment policies are to be favoured
		S4: Emphasis should be on the creation of supply‐chains integrated within geographical boundaries
	C2: IT preparedness: Advanced IT infrastructure commensurate to new needs	S1: IT infrastructure and its reach.
		S2: Capacity‐building for updating industrial technologies.
		S3: Improved training of the workforce and ensuring its availability.
		S4: The customised facility, latest technologies & cybersecurity tools for MSMEs.
	C3: Roles of governance: Roles of governance is paramount for the revival of industries in post‐ COVID‐19	S1: Provide Economic stimulus packages to provide low‐cost money to industries to help them to restart
		S2: They targeted social security programmes for those below the poverty line
		S3: Close cooperation of Government and industry to improve efficiency and resilience of production and supply chain management issues
		S4: Measures are to be adopted to ensure demand for locally produced products
		S5: Healthy and quick decision‐making and effective implementation of the selected approaches
		S6: New norms for personal Hygiene and sanitation
		S7: New healthcare norms
	C4: Supply chain and logistics: Facilitate the barrier‐free movements of goods and services so that timely delivery can be accomplished within the country. The focus should be to overcome intra‐country barriers	S1: Smooth movement of goods and services across all supply chains
		S2: Smooth movement of workers should be a priority
		S3: Smooth movement of essential goods
		S4: Comprehensive and excellent quality healthcare facility to be provided
	C5: Industry protocol: New norms which have to be followed by industries to facilitate the restart and resilience in the likelihood of similar or far worse crises	S1: The safe return of workers to the workplace
		S2: Organise standard operating procedures on modes to commute that will allow social distancing
		S3: Shift management.
		S4: Flexible production size.
		S5: Workers safety and health to be the paramount agenda
		S6: Transparency
		S7:Welfare scheme and its effective implementation
	C6: Environmental issues: Drastic reduction in pollution level is observed, i.e., the planet revives with no humans into play	S1: New environmental assessment models and norms.
		S2: Pollution is made from industrial production, and humanity consumption level has been reduced.
		S3: Newer energy sources, renewable sources of energy.
		S4: Energy‐efficient devices to be used
	C7: Capacity building: Capacity building helps in easy incorporation of the technology	S1: Industries must focus on realignment, retraining and re‐skilling of their employees
		S2: Digitisation, automation and artificial intelligence will have to be accelerated in every stream. Hands‐on training on automation and artificial intelligence is required so that the future of societies is secured
	C8: Change management: Stakeholders to be prepared mentally and trained to take the new normal	S1: The workers' morale should be boosted by assuring them that their health is the company's priority; by removing the fear of the disease and providing them with a liveable wage, even when the company is temporarily 'closed.'
		S2:Training for the 'new normal'.
		S3: Psychological issues and management
		S4: Awareness to be created among individuals on the importance of maintaining hygiene
		S5: Safety of the consumers is to be ensured
	C9: Organisational culture: Change management facilitates effective implementation of the efforts taken by the government	S1: Organisational should coalesce as a team for sailing through this complex environment.
		S2: Wearing the masks has to be made compulsory inside the premises.
		S3: Proper hygiene in the toilet area and near the work area is to be ensured.
		S4: The regular medical check‐up should be conducted inside the premises of the industries.
		S5: The top management's commitment to ensure the safety & well‐being of the workforce includes all of the workers in reverse and forward supply chains.
[84]		C1: Supply risk	S1: Global reserve
			S2: Ore concentration
			S3: Static index of depletion
			S4: Byproduct production percentage
		C2: Demand risk	S1: Annual mine production
			S2: Consumption by the electronics sector
			S3: Price
			S4: Price volatility
		C3: Sociopolitical risk	S1: Geographical concentration of production
			S2: Net import reliance in the U.S.
			S3: Socio‐political weighted geographic concentration
			S4: Social hotspots
		C4: Environmental risk	S1: Global warming potential
			S2: Cumulative energy demand
			S3: Mineral resource demand
			S4: Supply chain ecotoxicity
[85]	Green economy	C1: Energy security: Energy security is the effective management of energy supply from domestic as well as external sources	S1: Resilience: It displays an energy system's resilience to survive any shock and bounce back from it swiftly with minimum effect on supplies
			S2: Dependency: Energy dependency shows how much an energy system relies on imported energy. Energy systems that mostly depend on imported energy are more vulnerable in securing energy supply amidst energy shocks or global disasters
			S3: Capability: It reflects the energy system's capacity to meet the current and future energy demands in a reliable manner
		C2: Energy equity: Energy equity dimension emphasises equitable access to adequate and quality energy on an affordable price for domestic and commercial use	S1: Quality: Quality of energy encompasses access to modern and improved forms of energy that is resilient to extreme events and grid outages. It also includes modern fuels for cooking and heating that do not have negative impacts on human health
			S2: Affordability: Provide energy access at a price that is affordable to disadvantaged and low‐income communities
			S3: Accessibility: Given the COVID‐19 emergency response, access to energy has become the core need. Energy services are needed to power healthcare facilities, supply clean water for hygiene, and enable communication services that keep people connected while continuing social distancing.
		C3: Environmental sustainability: Environmental sustainability represents a transition to low carbon and sustainable energy sources e mainly RE sources e to reduce energy led to environmental harm and mitigate potential climate change impacts	S1: Decarbonisation: It refers to the minimisation of the carbon intensity of energy generation. Decarbonizing is a vital part of complying with the emissions reduction target submitted by the countries during the Paris agreement
			S2: Emission and Pollution Control: It includes emissions reduction, which is achieved through a corresponding decline in fossil fuels usage and an increase in RE sources' share
			S3: Resource Productivity: Improving resource productivity is essential to enable the economy to grow in a lasting way out of the financial crisis caused by the pandemic
[86]		C1: Socioeconomic Development Policies	S1: Sustainable DevelopmentInitiative (SDI)
			S2: Green Civil Society (GCS)Initiative(s)
			S3: Ensuring StakeholderParticipation
			S4: Gender Mainstreaming
			S5: Sectoral and RegionalDevelopment Initiatives
			S6: Social Inclusion in GreenEconomy
		C2: Green Growth Agenda promulgated by the government	S1: Inclusive and CollaborativePlanning
			S2: Promote Green Growth Patterns(GGPs)
			S3: Simulate Green Investment
			S4: Government InvestmentIncentives (GIIs)
			S5: Sustainable Special EconomicZone (SSEZs) Development
		C3: Green Industrial Development (GID)	S1: Green Product Innovation (GPI)
			S2: Green Craft Innovation (GCI)
			S3: Green Innovation Initiative (GII)for Green Industrial Growth(GIG)
			S4: Industrial Specialization
			S5: Industrial Diversity
			S6: Industrial Competition
		C4: Environmental Regulations	S1: Administrative EnvironmentalRegulations (AERs)
			S2: Market‐based EnvironmentalRegulations (MERs)
			S3: Monitoring and EvaluationSystem Development
			S4: Public Participation inEnvironmental Regulation andCompliance
			S5: Land/Planning laws
		C5: Resource Efficiency	S1: Minimization of EnvironmentalRisk
			S2: Sustainable Public Procurement(SPP)
			S3: Reducing Waste throughIndustrial Symbiosis
			S4: Reduce Resources and EnergyConsumption
			S5: Efficient Land Use
		C6: Technological Initiatives and Innovation (TII)	S1: Direct Government Funding andTax Incentives
			S2: Intellectual Property Laws
			S3: Research and Development(R&D)
			S4: Green Technology Innovation
			S5: Marketization Innovation
		C7: Green Energy Production and Consumption Practices	S1: Green Energy Initiative
			S2: Energy‐Saving TechnologyAdoption
			S3: Green Energy Transmission andDistribution System
		C8: Blue‐Green Infrastructure Development	S1: Blue Infrastructure Development
			S2: Green Stormwater ManagementSystem Development
			S3: Recycling Infrastructure
			S4: Transport Infrastructure
			S5: Green Buildings
		C9: Pollution Control, and Waste Management	S1: Air Pollution Control
			S2: Wastewater Management
			S3: Solid Waste Management(SWM)
			S4: Shared and Circular Economy Promotion
		C10: Labor Policies	S1: Skill Development
			S2: Occupational Safety and Health (OSH)
			S3: SME Development
			S4: Green Jobs
[87]	Transportation	C1: Bus	NA
		C2: Tram
		C3: Taxi
		C4: Train
		C5: Car Sharing
		C6: Multiple Modes
		C7: Walk
		C8: Car
[88]	Environment	C1: Particulate Matters PM10 concentration	NA
		C2: Land Surface Temperature (LST)
		C3: Normalized Deferent Moisture Index (NDMI)
		C4: Normalised Difference Vegetation Index (NDVI
		C5: Normalised Difference Water Index (NDWI
[89]		Business and Corporations	C1: Return on equity (ROE)	NA
			C2: Return on asset (ROA)
			C3: Earnings per share (EPS)
			C4: Debt to equity ratio (DER)
			C5: Debt to assets ratio (DAR)
			C6: Current ratio (CR)
[90]			C1: Economic risk	S1: Financing risk
				S2: Foreign exchange fluctuation risk
				S3: Economic downturn risk
			C2: Social risk	S1: Inconformity between electric power plan and local development plan
				S2: Land acquisition risk
				S3: Electricity policy fluctuation risk
			C3: Environmental risk	S1: Climatic condition risk
				S2: Geographical condition risk
				S3: Irresistible risk (such as nature disaster and COVID‐19)
			C4: Technical risk	S1: Power grid engineering quality risk
				S2: Security incidents risk
				S3: Poor power grid engineeringmanagement risk
			C5: Marketable risk	S1: Electricity price fall risk
				S2: Delayed payback period risk
				S3: Raw materials price increase risk
[91]			C1: Cause group	NA
			C2: Net receivers' group
[92]			C1: Effectiveness: means the effectiveness of the medical centers and doctors	NA
			C2: Responsibility: means the caring responsibility for patients and their family who infected with COVID‐19
			C3: Network: number of healthcare provider (i.e., clinic or hospital) for the COVID‐19 patients
			C4: Support: depicts the 24/7 provided service for issues related to COVID‐19
			C5: Age: range of covered age
			C6: Payback period: the speed of reimbursement procedure
			C7: The premium price: the private health insurance cost fee
[93]	Technological		C1: RFID labelling of waste: The RFID labelling of waste has been considered as an essential method to track waste	NA
			C2: GPS and GIS tracking of waste collection vehicles: The Global Positioning System (GPS) and Geographic Information System (GIS) help track vehicles of the waste disposal firms
			C3: The common user interface for uploading waste generation data: The collection and feeding of the data related to waste generation are another critical information point useful for planning a smart city's waste disposal
			C4: Digitisation of chimneys at waste disposal sites: The chimneys of the healthcare waste disposal plants need attention as disposing of waste leads to the generation of toxic and harmful gas emissions
			C5: Direct monitoring of the hospital's temporary storage sites.
			C6: Digitally connected healthcare centres, waste disposal firms, and pollution control board: The healthcare centres include the primary, secondary, and tertiary types of hospitals and generate healthcare waste from patient care for disposal
			C7: Pollution control board's feedback app to the public and other stakeholders: The data help in improving the lacunas of a system by analysing critical Information. This data is captured in various forms with the use of RFID, sensors, GPS and GIS.
[94]			C1: Bandwidth: Is the rate of data transfer for a fixed period	NA
			C2: Latency: Is the amount of time it takes for data to travel from one point to another
			C3: Storage: Many users can access a centralised repository for digital data, and they use high‐speed connections to provide fast performance
			C4: Price/cost
[95]			C1: Strengths	S1: Digital treatment (DTs): DTs, especially robotics, mobile application, and thermal imaging via AI, can be utilised for different purposes, including self‐assessment for COVID‐19, information circulation through social media, and identifying infected people
				S2: Awareness & training: As mentioned, DTs in different platforms such as online learning portals, social media, and websites can boost awareness of people around COVID‐19 news
				S3: Health hazard prediction: ICT interventions can benefit people in hazard mapping and epidemic and pandemic prediction through data analytic and AI
				S4: Nonvirtual support: DTs can help the healthcare system physically apart from their virtual support. DTs in drones, modern robots, and 3D Printers can improve the humanitarian supply management and logistics and medical transportation during COVID‐19 outbreak
				S5: Medicine development: DTs can be used to discover new drugs and vaccines in the form of biotechnical platforms
				S6: Economic interventions: Many economists have mentioned that economic status has directly or indirectly affected by DTs, especially ICT
			C2: Weakness	S1: Digital divide: Many areas in the world are deprived of the internet and limited resources so that the digital divide is considered another weakness of digital health intervention.
				S2: Lack of Digital knowledge: Digital literacy aids people in getting knowledge easy and fast through advanced technology. At the same time, they need to boost their technical skills to take full advantage of DTs in their daily works
				S3: Cost inefficiency: A massive amount of time, energy, and money are required to employ DTs in different fields, including the healthcare system, education system, and supply chain management.
				S3: Mock Information: One of the biggest crisis management problems is controlling unauthentic Information, misleading both individuals and medical personnel
				S4: Lack of reliable data: A large amount of information is accessible in ICT domains such as social media and website
				S5: Insecure applications: The governments have managed to use big data and tracing applications to trace movements of people during the COVID‐19 pandemic so that people may be concerned about a lack of privacy
			C3: Opportunities	S1: Accurate prediction system: It is believed that countries can use data analytic and artificial intelligence to predict the imminent epidemic precisely
				S2: Automatisation of Healthcare: There is a growing interest in the application of automation and robots in the healthcare system
				S3: Research framework in Epidemiology: The most important lesson out of the COVID‐19 pandemic is that the world does not have enough preparedness to cope with Coronavirus
				S4: Digital health education: Digital health applies ICT in health services through related internet‐based technologies
				S5: Digital diagnosis: Obtaining an accurate, swift various diagnosis of the source of human disease is a vital step of diagnostic medicine, specifically in viral infections, to boost healthcare services delivery
				S6: Health information systems (HIS): Health Information Systems (HIS) is considered the interplay among the technologies, individuals, and processes to carry out practices and management in delivering vital Information to boost healthcare performance quality
			C4: Threats	S1: Nondigital interferences: Religions and political organisations may interfere with the rapid and free growth of ICT since there is a dichotomy between their beliefs and political value and DT
				S2: Digital stratification: Considering the wealth of nations, digital health services' diffusion rate is wary in a different country
				S3: Privacy worries: Many tasks have moved online in light of the COVID‐19 pandemic affecting the privacy of people
				S4: Unaffordability: Although the operational cost is improved through DTs' application, especially the combination of AI and IoT, some scholars believe that it may not be affordable in some cases
				S5: Exacerbation of paranoia: A new type of phobia, namely 'Corona phobia', has recently been generated due to the COVID‐19 pandemic
				S6: Infodemic risk: Infodemic is as dangerous as pandemic since fake news and false rumours have significant adverse effects on society
[96]			C1: User interface	NA
			C2: Personalization
			C3: Interactivity
			C4: Security
			C5: Complete content
			C6: Navigation
			C7: Right and understandable content

Alsalem MA, Mohammed R, Albahri OS, et al. Rise of multiattribute decision‐making in combating COVID‐19: A systematic review of the state‐of‐the‐art literature. Int J Intell Syst. 2022;37:3514‐3624. 10.1002/int.22699