A novel project portfolio selection framework towards organizational resilience: Robust Ordinal Priority Approach

Abstract

The COVID-19 pandemic has affected the world's economic condition significantly, and construction projects have faced many challenges and disruptions as well. This should be an alarm bell for project-oriented organizations to be prepared for such events and take necessary actions at the earliest time. In this regard, project-oriented organizations should establish their business based on the resilience concept, making them flexible in dealing with risks and decreasing the recovery time after disruptions. The current study proposes a practical conceptual framework for project-oriented organizations to select the most appropriate portfolio based on organizational resilience strategy. First, portfolios are identified, and the projects are clustered based on organizational resilience strategy using the Elbow and Fuzzy C-Means methods. The projects' scores are then determined employing the stakeholders' opinions and Robust Ordinal Priority Approach (OPA-R), which can handle the uncertainty of the input data. After that, each portfolio's score is determined using the obtained scores of the projects, and the best portfolio linked to the organizational resilience strategy is selected. The application of the proposed method to a project-oriented organization is examined, and its usage for the managers of project-oriented organizations is discussed in detail.

1. Introduction

Due to the COVID-19 pandemic, a considerable number of businesses were damaged or even shut down as they had failed to prepare for such an event. Project-oriented organizations are no exception, they also experienced business losses as a consequence of this pandemic. The main challenges are labor shortages, financial pressure on the organizations, and disruption in project supply chain management. It may affect the contractual agreements in the construction projects as well, which are critical for the organizations (Ritter & Peckett, 2020). Since the world is facing a number of uncertainties, organization should be prepared for the occurrence of unexpected events. In this regard, organizations should have some characteristics such as flexibility, proactivity, persistence, and coping ability. Hence, project-oriented organizations need to consider the resilience concept in their strategies. Organizational resilience can be defined as an organization's ability to predict and respond to sudden disruptions caused by risks and unpredicted events to keep the business successful (Duchek, 2020). Indeed, the resilience concept can contribute to the organizations to be robust against environmental factors and uncertain circumstances. Organizations have to update their strategies and tactics based on environmental and marketplace conditions; otherwise, they may be eliminated from the survival cycle. They should select those projects which are currently aligned with their strategies. The question of how the project-oriented organizations can choose the portfolio based on the resilience concept may arise. How can they make such an important decision while numerous uncertainties surround the world, and there are many scenarios for selection? Which scenario can be suitable for them in the short and long term? These are the main challenges of the current study that need to be discussed in detail. However, to the best of our knowledge, no study has considered the resilient strategy in the project portfolio selection problem (PPSP).

Portfolio management can be defined as managing some portfolios and sub-portfolios to achieve the strategic objectives at the organization level (Project Management Institute, 2017b). The scholars proposed different portfolio selection frameworks based on various environmental, social, and financial goals. Moreover, they used mathematical modeling, multiple criteria decision-making (MCDM), simulation and other methods to solve portfolio selection problems. The Ordinal Priority Approach (OPA) is a novel development in the MCDM context (Ataei, Mahmoudi, Feylizadeh, & Li, 2020). It was established based on the linear programming approach for decision-making with the ordinal data. This method can handle group decision-making problems with incomplete data as well. Moreover, it can determine the weight of experts, criteria, and alternatives simultaneously. However, it cannot consider the uncertainties of the input data, which is a major concern for the decision-makers. To extend the OPA under uncertain situations, the current study proposed the Robust Ordinal Priority Approach (OPA-R), which can consider various scenarios during the decision-making process. In the present study, robustness is employed as a tool to cover the uncertainties which arise from the lack of reliability of experts' opinions regarding the resilient criteria. Based on the information mentioned above, the contributions of the current study can be summarized as follows:

• -The proposed framework considers the resilient strategy to make project-oriented organizations resilient. This approach can help the policymakers to make the organizations flexible.
• -The current study proposes the Robust Ordinal Priority Approach (OPA-R), a novel approach for determining the weight of the projects under uncertainty based on the organization's requirement.
• -The current study employs powerful approaches for aggregating the stakeholders' opinions, calculating the optimal number of portfolios, and clustering the projects. In this regard, the Ordered Weighted Aggregation (OWA) operator, Fuzzy C-Means, and Elbow methods are utilized.

The current study is organized as follows: First, a review is provided on the PPSP from various aspects. After that, the proposed framework and its steps are explained in detail and several large-scale examples are addressed. Next, a case study is executed in a project-oriented organization. Finally, the discussion and conclusion are provided with some suggestions and future direction for the portfolio selection problem.

2. Literature review

Since selecting projects to align with the organization's strategy plays a vital role in the success of project-oriented organizations, many scholars have paid attention to this subject in recent years. The PPSP can be addressed using various methods, including quantitative and qualitative approaches. It should be noted that quantitative approaches are more popular among scholars, while many positive results are acknowledged by implementing them in real case studies. The quantitative methods can be divided into three major groups. The first group defines the PPSP as an MCDM problem. The second group stated that the PPSP is a mathematical model and tries to solve it using linear programming, nonlinear programming, data envelopment analysis (DEA), etc. The third group uses a combination of MCDM and a mathematical model to solve PPSP. In the current study, we selected some recent research items from each group and explained their methodology.

MCDM methods contribute to the decision-makers for finding the optimal alternatives based on some defined criteria. It has a wide application in various fields of study such as project management, supply chain management, infrastructure management, etc. Smith-Perera et al. (2010) utilized Analytic Network Process (ANP) method for PPSP. They employed a strategic project index to analyze the strategies, and then they prioritized the project portfolios using the ANP method. They believe that ANP can perfectly aggregate the experts' opinions, and their approach has enough accuracy to be utilized in real-world problems. Strang (2011) utilized the analytical hierarchy process (AHP) method to propose a selection portfolio methodology in the nuclear industry. The main reason for using the AHP method was determining the weight of criteria based on the experts' opinions. In this research, the inconsistency ratio for the pairwise comparison matrix was determined to check the reliability of the experts' opinions. Vetschera and De Almeida (2012) employed the PROMETHEE method, an outranking MCDM method, to select the optimal portfolio. They stated that their approach requires low computational cost and obtains a reasonable approximation based on the c-optimal concept. Bhattacharyya (2015) proposed a grey-based MCDM method to select the portfolio in R&D projects under uncertainty. In this research, the various interdependencies attributes were incorporated to achieve more accurate results in real-world practice. Song et al. (2019) stated that the conventional frameworks could not handle uncertain criteria; hence, they proposed stochastic multi-criteria acceptability analysis to cope with PPSP under uncertainty. Ma et al. (2020) presented a framework for PPSP with a sustainable perspective. They used Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) in the decision-making process and fuzzy set theory to incorporate the uncertainty. It should be noted that the scholars employed various MCDM methods in the PPSP context. However, the OPA is a recent development in the MCDM field, which works with ordinal data and requires simple input data. Moreover, it requires low computational cost and time, while the weights of experts, criteria, and alternatives can be obtained after a few steps.

Since real-world problems can be formulated and simulated as mathematical models, it widely used by engineers, managers, and decision-makers. Carazo et al. (2010) presented a multi-objective binary programming model with taking the organization objectives into consideration for PPSP. The model was solved utilizing the metaheuristic approach, which might not find the global optimum solution. Interestingly, this model could find the optimal time for starting the selected portfolio projects with an objective viewpoint. Cho and Shaw (2013) proposed a mathematical model for the IT project portfolio selection problem. Their model could consider the impact of synergy enhancement on portfolio risk and return. Hassanzadeh et al. (2014) presented a multi-objective binary integer programming model for PSPP in R&D projects. The objective functions and constraints were uncertain, and a robust approach was employed to handle the uncertainty. Sharifi and Safari (2016) presented a chance-constrained programming model by bringing up the financial aspects into the PPSP at risk. Their efforts were concentrated on maximizing the return value and minimizing the risk impacts during portfolio selection. Pérez et al. (2018) stated that the PPSP is a complicated problem with the inherent uncertainty in the input data. They have proposed a mathematical model with fuzzy parameters for both selection and planning project portfolios. Albano et al. (2019) proposed a mixed-integer nonlinear optimization model to incorporate four important factors including, alignment of the strategy, balancing the portfolios, maximization of the value, and future preparedness. They employed two real-world cases from Brazil and Canada to check the reliability of the model. Li et al. (2020) suggested a mixed-integer polynomial model for PPSP and used a branch-and-bound algorithm for computations. They utilized a novel linearization method to solve the problem with fewer continuous variables than linear reformulations. RezaHoseini et al. (2020) presented a linear multi-objective by incorporating sustainable dimensions as objectives. The main advantage of the proposed approach was its ability to split the projects at any time and continue them at another time point. The mathematical modeling was broadly employed by the researchers for PPSP. It is worth mentioning the OPA is an MCDM method built based on a linear programming approach. Therefore, the OPA can be categorized into the mathematical modeling group as well.

Some scholars employed the combination of MCDM and mathematical modeling approaches. Gupta et al. (2013) engaged the AHP and Fuzzy MCDM to determine the ethical and financial performances. After that, they presented three hybrid mathematical models for portfolio selection which were based on ethical and financial performances. Solimanpur et al. (2015) pointed out portfolio selection problem is a multiple-objective decision. Therefore, they proposed a zero-one mathematical programming model with two objective functions, solved by a genetic algorithm. Since various solutions were obtained using the genetic algorithm, they used the AHP method to choose the most desired solution based on the decision maker's view. Haghighi-Rad and Rowzan (2018) utilized the TOPSIS method and a multi-objective zero-one integer programming model for solving the PPSP. Their model contained three objectives: maximizing the profit, minimizing the total risk, and maximizing the strategies' values. Mavrotas and Makryvelios (2020) presented a framework entitled Iterative Trichotomic Approach. They pointed out this approach can handle the subjectivity and uncertainty in the PPSP. This approach provided an integrated framework with the aid of mathematical programming, multiple criteria analysis, and Monte Carlo simulation for portfolio selection in R&D projects.

Based on the review mentioned above, various approaches were proposed by scholars to solve PPSP. The current study aims to provide an integrated framework for the portfolio selection problem by proposing the OPA-R. This approach can suggest the optimal numbers of the portfolios and assign the projects to the appropriate portfolios based on the resilient strategy in the organization. The aggregation operator can detect and decrease the subjectivity in experts' opinions. In the end, the OPA-R can calculate the weights of the experts, criteria, and projects associated with the most robust scenario. In the next section, the proposed approach is explained in detail.

3. Project portfolio selection framework

This section aims to propose an intelligent approach for PPSP, which can contribute to the organizations to implement resilience strategy in an uncertain environment. The steps of the proposed framework are presented in Fig. 1 , briefly.

Fig. 1.

The steps of the proposed framework.

As shown in Fig. 1, the proposed approach has three main stages: identifying portfolios, scoring projects, and portfolio selection. In the first stage, the portfolios are identified based on the resilient strategy. After that, the projects' scores are determined using the OPA-R, which can consider various scenarios during the project scoring process. Finally, each portfolio's score is calculated, and the policymakers analyze the results to make the best possible decision.

In the proposed framework, the evaluation and optimization methods are employed at the same time. The logic of the framework is illustrated in Fig. 2 for more clarification. As shown in Fig. 2, the evaluation methods are employed to identify and provide the portfolios. On the other hand, optimization methods are utilized to calculate the weight of the projects. Moreover, the evaluation methods such as the correlation formula are employed to check the similarity of the solutions of the scenarios which are obtained through the optimization methods.

Fig. 2.

The logic of the proposed framework.

The rest of the current section aims to explain the mentioned stages in detail and provide a comprehensive guideline for implementing them in project-oriented organizations.

3.1. Identifying portfolios

Before implementing an effective portfolio, it is essential to identify the portfolios in the organization. Portfolios should be defined based on the long-term organizational strategies and goals for growth in the competitive market. These strategies should be specified by the organizations' key stakeholders, who may be the organization's owner, project managers, functional managers, etc. Hence, identifying the organizations' portfolios can be achieved by passing the steps, including identifying key stakeholders, identifying organizational strategies, and clustering the projects based on these strategies.

3.1.1. Identifying key stakeholders

The stakeholders in an organization are a group of people who can affect or be affected by the activities/decisions of the organization in the short and long-term (Project Management Institute, 2017a). In most cases, the list of stakeholders in a project is mentioned in the project charter, which can help extracting the organizations' stakeholders. The stakeholders can be classified into various groups such as external/internal, resistor/supporter/neutral/, etc. (Project Management Institute, 2017a). The critical point is specifying the key stakeholders who play a vital role in the future of the organization. It should be noted that the most common method for identifying the key stakeholders is the Power-Interest matrix which is shown in Fig. 3 . In the current study, the same approach is employed, and it is suggested to the policymakers as well.

Fig. 3.

Power-interest matrix (Olander & Landin, 2005).

As can be seen from Fig. 3, the key stakeholders enjoy a high level of power and interest in the organization. These stakeholders have a high impact on the organizational strategies, which is a critical factor in project portfolio management.

3.1.2. Identifying resilient strategies and data collection

Various perspectives have been presented about organizational resilience in recent years (Duchek, 2020). For example, Vogus and Sutcliffe (2007) mentioned that organizational resilience keeps the balance when unexpected events occur and an organization rises out of those conditions fortified (adaptation attribute). Somers (2009) believed that organizational resilience was not just about the survival of the organization but also about identifying potential risks and taking preventive measures to ensure that the organization could function properly if faced with threats (anticipation attribute). Linnenluecke et al. (2012) stated that organizational resilience is the ability of the organization to recover from an extreme weather occurrence and absorb its impact (recovery attribute). Based on the aforementioned definitions regarding organizational resilience, a resilient organization should have the ability to anticipate the potential risks and cope with and adapt to unexpected events. These are the main strategies in a resilient organization that can contribute to the organization being resistant in the current uncertain world. The concept of organizational resilience is illustrated in Fig. 4 , proposed by Duchek (2020) for the first time and extended in the current study for project-oriented organizations. As can be seen from Fig. 4, the first phase of resilience is anticipation, which needs the availability of resources. The resources may include the human resource, financial resources, etc. The second phase is coping, which refers to the time of an unexpected event. In this phase, the organization requires social resources such as sharing knowledge, learning, and various services to deal with an unexpected event. It should be noted that the organization still needs financial and human resources during the second phase. The last phase is the adaption of the organization with an unexpected event. In this phase, the organization requires some changes to decrease the impact of the unexpected event. Any changes in the organization requires power and responsibility and other resources such as financial and human resources.

Fig. 4.

Organizational resilience concept (it is extended from Duchek (2020)).

From another aspect, Fig. 4 illustrates the relationship between the portfolio and organizational resilience. It reminds us that the resilience aspects should be considered as a strategy in the portfolio selection process in the project-oriented organizations. Since each portfolio can consist of some programs and projects that reflect the value of the related portfolio in the organization, the key stakeholders should provide the score for the projects in each resilient strategy, and then the score of the portfolios can be measured. In this regard, the stakeholders should answer three main questions: (1) To what extend does the project align with the anticipation capability in the organization, and can the risks of the project be predicted? (2) To what extent does the project align with the coping ability of the organization during unexpected events? (3) To what extent does the project align with making the organization flexible and improving the power of change after unforeseen circumstances? These responses ought to be used for extracting the portfolios based on the resilient strategies. It should be noted that the key stakeholders have already been identified in Section 3.1 (a).

3.1.3. Data aggregation and clustering

Data aggregation is a sensitive process that needs a high level of accuracy to achieve good results. When we face a group of experts, it is challenging to decrease the effect of subjective viewpoints on the final results. Therefore, there is a need to employ a method to detect the subjective views and assign a lower weight to them to decrease their impact on the final results. In this regard, Xu (2005) proposed a method to detect the experts' false opinions and reduce their impact on the final results. The Ordered Weighted Aggregation (OWA) operator is presented as Eq. (1):

$${\mathit{OWA}}_w\left(a_1,a_2,\cdots ,a_n\right)=\sum _{j=1}^n{w_{j}b}_j{b}_j\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{h}\mathrm{e}j\mathrm{t}\mathrm{h}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{o}\mathrm{f}{a}_j(j=1,2,\cdots ,n)$$ (1)

The critical point is determining the values of $W_j$ because the false or biased information should have less weight compared with other opinions. Hence, the $W_j$ is calculated using Eq. (2), which is established based on the normal distribution.

$$W_j=\frac{e^{-\frac{{(j-{\mu }_n)}^2}{2{\sigma }_n^2}}}{{\sum }_{i=1}^n{e}^{-\frac{{(i-{\mu }_n)}^2}{2{\sigma }_n^2}}}j=1,2,\cdots ,n$$ (2)

${\mu }_n$ and ${\sigma }_n$ are the mean and standard deviation of the collection of $(1,2,\cdots ,n)$ respectively and could be calculated as follows:

$${\mu }_n=\frac{1}{n}\frac{n(1+n)}{2}=\frac{1+n}{2}$$

$${\sigma }_n=\sqrt{\frac{1}{n}\sum _{i=1}^n{(i-{\mu }_n)}^2}$$

The concept of calculating the weight of the OWA based on the normal distribution is illustrated in Fig. 5 .

Fig. 5.

The concept of calculating.$W_j$

As shown in Fig. 5, this approach assigned a low weight to the false or biased information.

After aggregating the experts' opinions, the projects should be clustered based on the resilient strategy. Clustering techniques are useful for classifying projects into similar groups. Since the clustering factors are the organizational strategies, the output of the clustering would be the portfolios of the organization. One of the promising methods in the clustering context is the Fuzzy C-Means method. The Fuzzy C-Means was proposed by Dunn (1973) for the first time and later extended by Bezdek (1981). Let us assume that $t$ represents the number of strategies and $d$ implies the number of projects in the organization. Hence, the project set is $X=\left\{X_1,X_2,\cdots ,X_d\right\}$, and the $j^{\mathit{th}}$ project can be defined as $X_j=\left\{x_{1j},x_{2j},\cdots ,x_{\mathit{tj}}\right\}$ where $(j=1,2,\cdots d)$. In other words, each project has a score in each strategy. On the other hand, we aim to categorize these projects into $c$ clusters (portfolios). The center of the cluster can be defined as $v_i=\left\{v_{1i},v_{2i},\cdots ,v_{\mathit{ti}}\right\}$ for $(i=1,2,\cdots c)$. Considering these definitions, the steps of the Fuzzy C-means can be presented as follows (Aswani Kumar & Srinivas, 2010):

• Step 1: First, the optimal number of clusters (portfolios) should be determined using the Elbow method. Also, threshold value $\epsilon$ and degree of fuzziness $m$ should be chosen. The $U={[u_{\mathit{ji}}]}_{d\times c}$ matrix should be initialized where $u_{\mathit{ji}}$ is the membership degree of the $j$ ^th project in the $i$ ^th cluster (portfolio) and $U$ is the fuzzy partition matrix.

• Step 2: The value of the $v_i$ should be calculated using Eq. (3).

$$v_i=\frac{{\sum }_{j=1}^d{u}_{\mathit{ji}}^m.X_j}{{\sum }_{j=1}^d{u}_{\mathit{ji}}^m}i=1,2,\cdots c$$ (3)

• Step 3: The value of Euclidean distance ($d_{\mathit{ji}}$) should be calculated using Eq. (4). This value represents the distance from $X_j$ to $v_i$. After that, $u_{\mathit{ij}}$ should be calculated using Eq. (5) and $U={[u_{\mathit{ji}}]}_{d\times c}$ should be updated.

$$d_{\mathit{ji}}=\sqrt{\sum _{p=1}^d{(v_{\mathit{pi}}-x_{\mathit{pj}})}^2}$$ (4)

$$u_{\mathit{ij}}=\frac{1}{{\sum }_{p=1}^c{\left(\frac{d_{\mathit{ji}}}{d_{\mathit{jp}}}\right)}^{\frac{2}{m-1}}}m\ne 1$$ (5)

• Step 4: The value of the objective function should be computed using Eq. (6). When the difference between two consecutive objective functions is less than $\epsilon$, the algorithm should be stopped; else, we should go to Step 2 and repeat the algorithm.

$$J_{\mathit{Fuzzy}}=\sum _{j=1}^d\sum _{i=1}^c{u}_{\mathit{ji}}^m{d}_{\mathit{ji}}^2$$ (6)

The Fuzzy C-Means results include the fuzzy partition matrix that reflects each project's membership degree in each cluster and cluster center. To sum up, the experts' opinions regarding the projects in each strategy can be aggregated, and then, the projects can be arranged into portfolios based on their similarity in terms of the resilient strategies using the Fuzzy C-Means algorithm.

3.2. Scoring projects

Each project has a specific value for each organization to be selected. Organizations always like to consider different criteria during the project selection process and also to calculate the score of the project based on those criteria. In this regard, the current section is divided into two sub-sections; the first section is related to identifying the standard criteria. The second section proposes the OPA-R for calculating the projects' scores based on the organizational requirements under uncertainty.

3.2.1. Identifying criteria and data collection

There are various definitions for project resilience which is proposed by scholars in recent years. Meanwhile, Kutsch et al. (2016) stated that the project's success depends on standing resilience. They mentioned that resilience means awareness, understanding, preparing for the crisis, staying stable, and continuously recovering from them. Blay (2017) conducted very comprehensive research on the project resilience and its definition and dimensions. Blay (2017) defined project resilience as ‘the capability of a project to respond to, prepare for and reduce the impact of disruption caused by the drifting environment and project complexity’. The dimensions of the project resilience are defined as Proactivity, Coping ability, Flexibility, and Persistence. Naderpajouh et al. (2020) described project resilience as the capacity to manage the project under various scenarios such as disruptions due to stressors. Based on these studies and the definitions of project resilience, the criteria for scoring the projects in terms of degree of resilience are summarized in Table 1 .

Table 1.

The criteria for project scoring.

Measurement	Criteria	Description
Project Resilience	Proactivity (C1)	The capability of a project in terms of identifying potential risks.
	Coping ability (C2)	The capability of a project in terms of managing during an unexpected event.
	Flexibility (C3)	The capability of a project to be changed after an unexpected event
	Persistence (C4)	The capability of a project to be continued despite difficult conditions.
Project Profitability	Net Present Value (NPV) (C5)	The variation of the present value of cash inflows and outflows during a period of time
	Initial Investment (C6)	The amount of money that a project requires to be started

As shown in Table 1, the project profitability criteria, including Net Present Value (NPV) and Initial Investment (C6), are mentioned as well. It should be noted that profitability is an important factor in any organization, and it is a primary reason for establishing a company in most cases. Profitability was considered in the studies conducted by Han et al., 2004, Guo et al., 2016, Huang, 2012, and Silva et al. (2015), yet this is the first time profitability and resilient (Presilient) criteria are considered in PPSP simultaneously. After defining the criteria, the questionnaire should be designed and sent to the key stakeholders to opine the projects based on these criteria.

3.2.2. Solving the problem using Robust Ordinal Priority Approach

Ordinal Priority Approach (OPA) is a method in the MCDM field which is formed based on the ordinal data. The OPA can solve various types of MCDM problems, including group decision-making, individual decision-making and determining the weights of the experts, criteria, and alternatives. It was extended under fuzzy and grey environment by the scholars as well (Mahmoudi et al., 2020, Mahmoudi et al., 2021). The OPA was utilized to calculate the weights of sustainability attributes and ranks of the barriers to blockchain adoption (Sadeghi, Mahmoudi, & Deng, 2021). The benefits of employing the OPA are summarized as follows:

• •The weights of the experts, criteria, and alternatives in the group decision-making problems can be calculated without using other MCDM methods.
• •Requires simple comparison among experts, criteria, and alternatives as input data, making the decision-making process practical, simple, and agile.
• •Supports incomplete input data when the experts do not have enough knowledge to opine a criterion/alternative.

The OPA model is established based on the linear programming approach. In the core model of the OPA, the following sets, indexes, variables, and parameters are used (Mahmoudi, Deng, Javed, & Yuan, 2021):

Sets
I	Set of experts$\forall i\in I$
J	Set of criteria$\forall j\in J$
K	Set of alternatives$\forall k\in K$
Indexes
$i$	Index of the experts$\left(1,\dots ,p\right)$
$j$	Index of preference of the criteria$\left(1,\dots ,n\right)$
$k$	Index of the alternatives$\left(1,\dots ,m\right)$
Variables
Z	Objective function
$W_{\mathit{ijk}}^r$	Weight (importance) of $k$th alternative based on $j$th criterion by $i$th expert at $r$th rank
Parameters
$i$	The rank of expert$i$
$j$	The rank of criterion$j$
$k$	The rank of alternative$k$

With considering the above definitions, the steps of the OPA can be explained as follows:

• Step 1: The experts should be identified, and their order should be prioritized using ordinal numbers. In the current study, the experts are prioritized based on their power and interest in the organization.

• Step 2: The criteria should be specified and prioritized by each expert.

• Step 3: The alternatives (projects) should be prioritized in each criterion by each expert.

• Step 4: Using the information from Steps 1 to 3, Model (7) should be constructed and solved using an optimization software like Excel, MATLAB, LINGO, etc.

Max Z (7)

S.t:

$$Z\le i\left(j\left(k\left({W_{\mathit{ijk}}}^r-{W_{\mathit{ijk}}}^{r+1}\right)\right)\right)\forall i,j,kandr$$

$$Z\le ijm{W_{\mathit{ijk}}}^m\forall i,jandk$$

$$\sum _{i=1}^p\sum _{j=1}^n\sum _{k=1}^m{W}_{\mathit{ijk}}=1$$

$$W_{\mathit{ijk}}\ge 0\forall i,jandk$$

where Z: Unrestricted in sign

After solving Model (7), the weights of alternatives are obtained using Eq. (8):

$$W_k=\sum _{i=1}^p\sum _{j=1}^n{W}_{\mathit{ijk}}\forall k$$ (8)

The weights of criteria are obtained using Eq. (9):

$$W_j=\sum _{i=1}^p\sum _{k=1}^m{W}_{\mathit{ijk}}\forall j$$ (9)

The weights of experts are obtained using Eq. (10):

$$W_i=\sum _{j=1}^n\sum _{k=1}^m{W}_{\mathit{ijk}}\forall i$$ (10)

However, in most cases, the experts are not entirely sure about the data in Steps 2 and/or 3, and the input data are uncertain. Therefore, there is a need to employ an uncertainty approach to handle the experts' doubts while calculating the weights. In this regard, the current study proposes the Robust OPA (OPA-R), which can consider various scenarios as input data and present the most robust scenario as a solution to the decision-maker. Various classifications can be defined in robust optimization based on the type of model. However, one of the most common classifications is probabilistic and non-probabilistic robustness models. From this aspect, the OPA-R model is considered as a non-probabilistic model because the primary sources of the uncertainty in the input data are experts' doubts regarding the criteria and alternatives and/or decision-maker doubts about the order of the experts. In the proposed approach, the interaction among the projects which are located in the same portfolio can have a positive or negative impact on the weight of the portfolios. This issue can be incorporated by considering the changes in the input data. For example, when two projects have a positive interaction with each other, the input rank should be decreased in the projects. When the interaction between two projects is negative, the input rank should be increased. This issue can be considered in the scenarios.

Theorem 1:

If the input ranks of the MCDM problem of the OPA consist of uncertainty (face several scenarios), and the scenarios' solutions are correlated significantly, the scenario with the highest value in the objective function of the OPA is recognized as robust scenario.

Proof ofTheorem 1: Wald's maximin model is a well-known and influential approach to deal with the non-probabilistic models under severe uncertainty (Sniedovich, 2008). Wald's maximin model can be presented as follows (Wald, 1945):

$$v^{\ast }:=\underset{d\in D}{\max }\underset{\mathit{s}∊\mathit{S}(d)}{\min }f(d,s)$$ (11)

$D$ represents the decision space, $S(d)$ implies a set of feasible states related to decision $d$, and $f(d,s)$ represents the outcome of decision $d$ and state $s$. If the constraints are incorporated into Model (11), the following model can be resulted where the form of constraints is appropriate for maximizing the objective function.

$$v^{\ast }:=\underset{d\in D}{\max }\underset{\mathit{s}∊\mathit{S}(d)}{\min }\left\{f\left(d,s\right):g\left(d,s\right)\le 0,\forall s∊S(d)\right\}$$ (12)

$$\mathit{Min}-Z$$ (13)

S.t:

$$Z-i\left(j\left(k\left({W_{\mathit{ijk}}}^r-{W_{\mathit{ijk}}}^{r+1}\right)\right)\right)\le 0\forall i=1...p,j=1...nandk=1\cdots (m-1)$$

$$Z-p\ast n\ast r\left(-\sum _{i=1}^p\sum _{j=1}^n\sum _{k=1}^{m-1}{W_{\mathit{ijk}}}^r+{W_{\mathit{ijk}}}^{r+1}\right)\le 0$$

$$Z-ijm{W_{\mathit{ijk}}}^m\le 0\forall i=1...p,j=1...nandk=1\cdots (m-1)$$

$$Z-p\ast n\ast m\left(1-\sum _{i=1}^p\sum _{j=1}^n\sum _{k=1}^{m-1}{W}_{\mathit{ijk}}\right)\le 0$$

$$W_{\mathit{ijk}}\ge 0\forall i=1...p,j=1...nandk=1\cdots (m-1)$$

where Z: Unrestricted in sign

Model (12) implies that the problem should be solved for each scenario separately, then, the scenario with the highest value in the objective function is the robust solution. So, to propose the OPA-R model, Model (7) should be constructed based on the form of Model (12). In this regard, the objective function in Model (7) should be multiplied by “-1″ to become a minimization type like Model (12). Moreover, constraints should be constructed in the form of $g\left(d,s\right)\le 0$. Finally, Model (13) is obtained, which should be constructed and solved for each scenario.

It should be noted, constraint ${\sum }_{i=1}^p{\sum }_{j=1}^n{\sum }_{k=1}^m{W}_{\mathit{ijk}}=1$ should be changed in Model (7) to adjust all constraints to Wald's maximin model. To overcome this barrier, we used the variable change of$W_{\mathit{pnm}}=$ $1-{\sum }_{i=1}^p{\sum }_{j=1}^n{\sum }_{k=1}^{m-1}{W}_{\mathit{ijk}}$. After solving Model (13) for each scenario, based on the Wald's maximin approach, the robust scenario could be distinguished using the following equation:

$$s_{\mathit{robust}}=\underset{d\in D}{\mathrm{m}\mathrm{a}\mathrm{x}}\left\{Z_1,Z_2,\cdots ,Z_s\right\}$$ (14)

Therefore, the scenario with the highest value in the objective function in the OPA model is recognized as robust scenario and Theorem 1 is proved.

We could find the robust solution, yet the scenarios should not significantly differ in the scenario set. Indeed, if there is a considerable uncertainty (inconsistency among the scenarios), making an appropriate decision is difficult. In order to check the correlation among the scenarios, the ranks of the scenarios could be utilized. In this regard, assume that $m$ represents the number of alternatives and $d$ donates the difference between the two rank series. Therefore, $r_{\mathit{spearman}}$ implies the Spearman’s Rank Correlation Coefficient between two rank series, which is shown in Eq. (15):

$$r_{\mathit{spearman}}=1-\left[\frac{6.{\sum }_{k=1}^m{(d_k)}^2}{m(m^2-1)}\right]$$ (15)

Since Spearman’s Rank Correlation Coefficient is a non-parametric statistic test, there is no need to check the primary assumptions such as the type of distribution, and it can be employed for any ordinal data series. The value of Spearman’s Rank Correlation Coefficient is always between +1 and −1. The value close to −1 and +1 shows strong correlation and the value close to 0 represents a weak correlation between the rank series. The critical values for Spearman’s Rank Correlation Coefficient at different levels of $\alpha$ and various sample sizes were mentioned in Kokoska and Nevison (1989). In the current study, the value of $\alpha$ is 0.05 and Spearman’s Rank Correlation Coefficient should be equal or higher than the critical values at the $\alpha$ level. It is worth mentioning that if the scenarios have a significant difference (weak correlation), the decision-maker should re-evaluate the input data which were provided by the experts. In this case, various options could be selected, including adding new expert(s) or discussing how to identify the reason of the significant difference in the scenarios with the experts (feedback mechanism can be used as well). If a slight difference in the scenarios is observed, the robust scenario, which can be identified using Eq. (14), should be selected by the decision-maker.

Calculation of the ordinal robustness plays an essential role in the problem as well. In order to calculate the ordinal robustness index (ORI), there is a need to calculate the agreement (consistency) among the ranks of scenarios. Since there are more than two scenarios in most cases, the methods such as Spearman or Kendall's Tau cannot be employed. To overcome this barrier, Fleiss' kappa can be utilized (Fleiss, 1971), which is suitable for calculating the reliability of agreement (consistency) among a group of scenarios as well as for ordinal and nominal data. Therefore, the $\mathit{ORI}$ can be calculated using Eq. (16).

$$\mathit{ORI}=\frac{\stackrel{-}{P}-{\stackrel{-}{P}}_e}{1-{\stackrel{-}{P}}_e}$$ (16)

where $1-{\stackrel{-}{P}}_e$ implies a degree of robustness that is attainable above chance and $\stackrel{-}{P}-{\stackrel{-}{P}}_e$ implies the degree of robustness which is achieved actually above the chance. Where all scenarios have the same results, the value of $\mathit{ORI}$ is 1, and where there is not any agreement (consistency) among the scenarios, the $\mathit{ORI}$ is 0, and in rare situations, it may be negative. The value of $\stackrel{-}{P}$ in Eq. (16) can be calculated through Eq. (17) where $m$ $\left(k=1\cdots m\right)$ represents the number of alternatives (projects) and $s\left(1\cdots S\right)$ represents the number of scenarios. Since the current study uses ranked data, the number of categories is equal to the number of alternatives, and the index of categories is shown by $q$. Moreover, $s_{\mathit{kq}}$ implies the number of scenarios which assigned the $k$ ^th alternative to $q$ ^th category.

$$\stackrel{-}{P}=\frac{1}{mS\left(S-1\right)}\left(\sum _{k=1}^m\sum _{q=1}^m{s_{\mathit{kq}}}^2-mS\right)$$ (17)

The value of ${\stackrel{-}{P}}_e$ in Eq. (16) can be determined using Eq. (18).

$${\stackrel{-}{P}}_e=\sum _{q=1}^m{\vartheta }_q^2$$ (18)

where

$${\vartheta }_q=\frac{1}{mS}\sum _{k=1}^m{s}_{\mathit{kq}}$$ (19)

The $\mathit{ORI}$ is a useful index to demonstrate the reliability of the problem and the calculation can be performed using SPSS software. Table 2 can be used for interpreting the value of robustness which is adopted from Landis & Koch (1977).

Table 2.

Interpreting robustness value.

$\mathit{ORI}$	Interpretation
<0	Poor robustness
0.01–0.20	Slight robustness
0.21–0.40	Fair robustness
0.41–0.60	Moderate robustness
0.61–0.80	Substantial robustness
0.81–1.00	Almost perfect robustness

3.3. Portfolio selection

In previous sections, the portfolios are identified, and the score of the projects is calculated. However, the most important step is selecting the optimal portfolio for the organization, which is the main purpose of the current study.

3.3.1. Scoring the portfolios

Each portfolio includes some projects which reflect the value of that portfolio. Since the projects' scores based on the organization strategies were determined already, a suitable approach such as bottom-up could be used to calculate the score of each portfolio. Fig. 6 illustrates the relationship among the portfolios, programs, and projects in the organization. As can be seen from Fig. 6, the portfolios can be included the projects and programs simultaneously. To calculate each portfolio's score, the score of the programs and projects in the subset of the portfolio should be totalized based on the bottom-up approach. The bottom-up approach has a wide range of applications in project management, such as estimating the cost, time, etc.

Fig. 6.

Bottom-up approach for calculating portfolio score.

After calculating the score of each portfolio, the most suitable portfolio for the organization could be selected. However, the organization may need to select more than one portfolio based on the available resources.

3.3.2. Analyzing the results

Calculating the score of the portfolios is not enough for making the final decision. Especially when the organization needs to select more than one portfolio. The decision-maker should check and compare the score of the portfolios with each other. When the difference of the scores between the portfolios with the top portfolio is significant, it may be useful to do more analyses before selecting other portfolios. It implies that the rank of the portfolio is not sufficient; the most crucial factor is the score of the portfolio and its variation with the other portfolios. In some cases, making sub-optimal decisions can impose a high amount of financial losses on the organization. Hence, it may be better to avoid selection in these cases.

4. Numerical examples and experimental results

In this section, the proposed framework is employed to solve various sizes of problems. Moreover, the robustness, runtime, and other characteristics of the problems are discussed and reported in detail.

4.1. Identifying portfolios

First of all, the input data should be collected from the experts regarding the alignment of the projects with the organizational resilience and the related strategies. Here, there are three various examples with different sizes. Since these problems are large-scale, their input data is provided as supplementary materials to save space. It should be noted the input data are generated hypothetically for the problems. Problem 1 includes 10 experts, 6 criteria, and 80 projects. Problem 2 consists of 12 experts, 6 criteria, and 100 projects. Problem 3 consists of 14 experts, 6 criteria, and 120 projects. For aggregation the experts' opinions, the OWA operator is utilized, which can decrease the impact of bias information on the results. After the data aggregation process, the Elbow method is used to extract the number of portfolios for each problem, the results are presented in Fig. 7 . Based on Fig. 7, the number of portfolios for Problem 1 is 21, where the percentage variance explained is around 90%. Likewise, the number of portfolios for Problem 2 and Problem 3 are 23 and 25, respectively. After that, the clustering process is done to categorize similar projects into portfolios based on resilient organizational strategies. The results are illustrated in Fig. 7 for each problem separately. This step can help the organization when there are a lot of projects in the organization. If there are a few number of projects, this step may be handled manually without employing the clustering process. The clustering process is done using the MATLAB 2016b version on a computer with a Core i5 processor, 64 bits, 2.4 GHz, a Windows 10 operating system, and 4 GB of RAM. It should be noted that the runtime of problems 1 to 3 is reported as 58, 75, and 117 millisecond (ms), respectively.

Fig. 7.

The results of Elbow and fuzzy C-Means for problems 1 to 3.

4.2. Scoring projects

In this step, the data regarding the projects in each criterion should be collected. After that, Model (7) should be constructed and solved for each problem separately (Appendix A). After solving the models for all scenarios, the value of $r_{\mathit{spearman}}$ should be checked. More details regarding the problems are illustrated in Table 3 . As can be seen from Table 3, the minimum value of $r_{\mathit{spearman}}$ is higher than the critical value. Therefore, the scenarios do not have a significant difference. Next, the robust scenario should be selected. Based on Theorem 1, the scenario with the maximum value in the objective function is the robust scenario. In this regard, the values of the objective functions of all scenarios in each problem are calculated and illustrated in Fig. 8 . Based on Fig. 8, Scenario 32 is the robust scenario for problem 1, Scenario 64 is the robust scenario for problem 2, and Scenario 92 is the robust scenario for problem 3. The information regarding the robust scenario is presented in Table 3. As shown in Table 3, the runtime is increased with increasing the dimension of the problem. Since the models are linear programming, little time is needed to solve them. The $\mathit{ORI}$ of the problems is determined using Eqs. (14), (15), (16), (17). The value of $\mathit{ORI}$ depends on the uncertainty of the input data. It seems that the uncertainty in Problem 2 is less than other problems.

Table 3.

The information of the model of problems 1 to 3.

	Problem 1	Problem 2	Problem 3
Total variables	4801	7201	10,081
Total constraints	9599	14,400	20,157
Model type	LP	LP	LP
Runtime (MS)	570	940	1550
NS	32	64	128
NPR	80	100	120
NC	6	6	6
NE	10	12	14
NPO	21	23	25
Minimum$r_{\mathit{spearman}}$	0.9966	0.99753	0.99746
Ordinal Robustness Index	0.523 (Moderate robustness)	0.566 (Moderate robustness)	0.534 (Moderate robustness)

Note: LP: Linear Programming; NS: Number of scenarios; NPR: Number of projects; NPO: Number of portfolios; NC: Number of criteria; NE: Number of experts

Fig. 8.

The objective function value for scenarios in problems 1 to 3.

4.3. Portfolio selection

Considering the score of the projects in the robust scenario, the weight of the portfolio in each problem is calculated. Fig. 9 illustrates the importance of the portfolios for problem 1. As can be seen from Fig. 9, portfolio 21 achieved the highest weight and is suitable for selection.

Fig. 9.

The weight of the portfolios for Problem 1

The weights of the portfolios for problem 2 are illustrated in Fig. 10 . As can be seen from Fig. 10, portfolio 17 achieved the highest importance among all portfolios. Moreover, its difference is significant from other portfolios, which makes it suitable for selection compared to other portfolios. The weights of the portfolios for problem 3 are illustrated in Fig. 11 . As can be seen from Fig. 11, portfolio 7 achieved the highest weight among all portfolios. Moreover, its difference is not significant, and the decision-maker can select other portfolios in the second and third positions as well if required.

Fig. 10.

The weight of the portfolios for Problem 2

Fig. 11.

The weight of the portfolios for Problem 3

In this section, three different sizes of problems were solved using the proposed framework. However, these examples may not be enough to implement the proposed approach for real-world problems. Therefore, in the next section, a case study is addressed carefully to show the application of the proposed approach for real-world problems step by step.

5. Case study

In order to demonstrate the application of the proposed framework, a project-oriented organization is selected. The mission of this organization is to manufacture various types of refinery equipment that are unique from different aspects. This organization was established in 1889 as a small manufacturing company, and then it was developed after a few years and currently has almost 2000 staff. Here, we aim to implement the proposed approach in this organization, and the steps are provided in the rest of the current section.

5.1. Identifying portfolios

First, the key stakeholders were identified in the organization using the Power-Interest matrix, which was explained in Section 3.1 (a). Here, there were 5 key stakeholders who had high power and interest in the organization. The primary strategy of the organization was considered to be resilient, which can make it sustainable against unexpected events. The resilience strategies were explained in Section 3.1 (b). Here, strategy 1 is Anticipation capacity, strategy 2 is Coping ability, and strategy 3 is Adaptation capacity. There are 23 projects which were opined by the key stakeholders using the Likert Scale. These values are shown in Table B.1 (Appendix B). For aggregating the key stakeholders' opinions, the OWA operator was employed with objective weights through the normal distribution. OWA can decrease the weights of false and unfair views automatically. The steps of the OWA were explained in Section 3.1 (c). After that, the projects should be categorized into portfolios based on the strategies. Using the aggregated data and the Elbow method, the projects can be categorized into 7 portfolios with 90% percentage variance explained. Fig. 12 illustrates the optimal number of portfolios based on the similarity degree, which was provided using the MATLAB software.

Fig. 12.

The results of the Elbow method to find the optimal number of portfolio

In this step, the projects should be clustered using the Fuzzy C-Means method explained in Section 3.1 (c). After applying Fuzzy C-Means, each project has a specific membership degree in each portfolio. Projects were assigned to the portfolio with the maximum value of membership degree. Fig. 13 illustrates the Fuzzy C-Means method results, and each portfolio is depicted with a specific color.

Fig. 13.

The results of the Fuzzy C-Means

As a result, the portfolios and their projects are summarized in Table 4 . Totally, there are 7 portfolios and 23 projects while Portfolio 1 contains the highest number of projects and Portfolio 6 contains the lowest number of projects.

Table 4.

The portfolios and their projects.

Portfolios	Projects
Portfolio 1	P1, P4, P8, P9, P18, P20
Portfolio 2	P2, P6, P17, P19, P21
Portfolio 3	P3, P11, P12, P22
Portfolio 4	P16, P23
Portfolio 5	P5, P13, P15
Portfolio 6	P10
Portfolio 7	P7, P14

5.2. Scoring projects

In order to calculate the score of each portfolio, the score of each project should be determined first. The essential criteria were identified and explained in Section 3.2 (a). The key stakeholders' opinions regarding the projects in each criterion are presented in Table C.1 (Appendix C). The important point is the uncertainties in the input data in Table C.1, which should be incorporated during project scoring. For example, the key stakeholder 2 was not entirely sure about his opinion associated with Project 14 in Criterion 1. Hence, two ranks were provided, which could be 1st or 2nd. As a second example, the key stakeholder 3 was not sure about his opinion regarding Project 8 in Criterion 1. It is worth mentioning that the OPA is a flexible model which can handle incomplete data as well. Therefore, where the expert is not entirely sure about his opinion, it can be considered as missed data. Considering the data in Table C1, the scenarios in Table 5 are possible.

Table 5.

The possible scenarios for the case study.

Scenario Number	The rank of P14 in C1 by Stakeholder 2	The rank of P08 in C1 by Stakeholder 3	The rank of P14 in C4 by Stakeholder 3	The rank of P17 in C5 by Stakeholder 4
1	2	3	7	4
2	2	3	7	Missed
3	2	3	Missed	4
4	2	3	Missed	Missed
5	2	Missed	7	4
6	2	Missed	7	Missed
7	2	Missed	Missed	4
8	2	Missed	Missed	Missed
9	1	3	7	4
10	1	3	7	Missed
11	1	3	Missed	4
12	1	3	Missed	Missed
13	1	Missed	7	4
14	1	Missed	7	Missed
15	1	Missed	Missed	4
16	1	Missed	Missed	Missed

Based on the explained procedure in Section 3.2 (b), each scenario should be solved separately using LINGO software. The stakeholders' priority is considered as Stakeholder 2 > Stakeholder 1 > Stakeholder 4 > Stakeholder 3 > Stakeholder 5 based on their power and interest. Later, the objective function values of the Model (7) for all scenarios were obtained, which are shown in Table 6 . As shown in Table 6, Scenario 8 reached the highest value that can be considered a robust solution based on Theorem 1.

Table 6.

The object value for all scenarios.

Scenario Number	Objective Function	Rank
Scenario 1	0.007772	8
Scenario 2	0.007779	6
Scenario 3	0.007775	7
Scenario 4	0.007782	4
Scenario 5	0.007780	5
Scenario 6	0.007786	2
Scenario 7	0.007783	3
Scenario 8	0.007789	1
Scenario 9	0.007613	16
Scenario 10	0.007619	14
Scenario 11	0.007616	15
Scenario 12	0.007622	12
Scenario 13	0.007620	13
Scenario 14	0.007627	10
Scenario 15	0.007623	11
Scenario 16	0.007629	9

It should be noted that the scenarios should be correlated significantly. If there is a weak correlation among the scenarios and the scenarios have a huge gap, the decision-maker should re-evaluate the input data, take corrective action, add a new stakeholder, etc. The results of the obtained scenarios in Table 6 are compared using Eq. (15) in Fig. 14 .

Fig. 14.

The scenarios comparison

As can be seen from Fig. 14, the lowest value of Spearman’s Rank Correlation Coefficient is 0.979, which is significantly more than the critical value at level 0.05. Therefore, the scenarios in Table 6 are strongly correlated, and Scenario 8 can be selected as the robust solution for the problem (See Theorem 1). The stakeholders’ weights, associated with Scenario 8, are presented in Fig. 15 .

Fig. 15.

The stakeholders' weights are obtained through the robust scenario

As shown in Fig. 15, Stakeholder 2 and Stakeholder 1 reached the first and second positions, respectively. On the other hand, Stakeholder 5 achieved the last position. It should be noted that these weights were provided based on the ordinal priority of the stakeholders, which had already been provided as input data. However, the stakeholders' opinions related to the criteria and projects are effective in determining the weights as well. These weights imply the role of each stakeholder in the final decision, which can be useful for the decision-maker to manage the power of the stakeholders during the decision-making process.

Moreover, the weights of the criteria are illustrated in Fig. 16 . Based on the stakeholders' opinions, C1 achieved the first position with the highest weight, which implies the proactivity capability of the projects is considerably important in the organization. Later, C3 reached the second position, which is related to the flexibility of the projects. It is interesting to mention that these factors play a vital role in organizational resilience.

Fig. 16.

The weights of criteria are obtained through the robust scenario

The weights of the projects are presented in Fig. 17 , which illustrates the importance of each project based on the stakeholders' opinions in all criteria.

Fig. 17.

The weights of projects are obtained through the robust scenario

As can be seen from Fig. 17, Projects 9 and 8 achieved the highest weights, respectively. It implies that these projects are exceedingly valuable for the organization to be selected. On the other hand, Projects 17 and 22 have the lowest value for the organization.

5.3. Portfolio selection

In the previous sections, we identified the portfolios in the organization, and also, the weight of each project was calculated based on the key stakeholders' viewpoints. Using Table 4, Fig. 17, and the bottom-up approach in Fig. 6, the weights of the portfolios are resulted and presented in Fig. 18 .

Fig. 18.

The weight of the portfolios

As shown in Fig. 18, Portfolio 1 is the best portfolio for the organization, which strongly aligns with the organizational strategies and requirements. Portfolios 2 and 5 reached the second and third positions with only a slight difference with each other. However, Portfolios 2 and 5 have a significant difference in terms of weight with Portfolio 1. Therefore, if the organization requires more than one portfolio, the managers should be careful in choosing other portfolios. Based on this information, the rank of the portfolio is not enough for making the right decision, and the weight of the portfolios can be helpful as well. Because the difference among the portfolio can be extracted through their weights and making sub-optimal decisions can increase the risk of failure.

6. Discussion and conclusion

Businesses and organizations were recently affected by the unexpected occurrence of COVID-19. Some organizations even shut down and were unable to survive in the current situation. Organizational resilience is a useful approach that can help the organization remain sustainable during an unexpected event. Indeed, organizational resilience can be defined as an organization's ability to predict and respond to the sudden disruptions caused by risks and unpredicted events and to keep the business successful. It should be considered as a strategy in the organization, and the activities should be aligned with that. On the other hand, portfolio management refers to managing some portfolios and sub-portfolios to achieve the strategic objectives in project-oriented organizations. In this regard, the current study proposed a comprehensive framework for portfolio selection with the view of organizational resilience and related strategies.

In the proposed approach, the stakeholders' opinions were aggregated using the OWA operator, which could detect unfair information with the aid of normal distribution and decrease their impact on the final results. Later, the Fuzzy C-Means and Elbow method were used to cluster the projects into the portfolios. Then, the OPA-R was proposed based on Wald's maximin model, which could handle various scenarios as input data and was sustainable against the uncertainties and changes. The results imply that the objective function in the OPA represents the robustness of the problem, which is a new achievement. In the end, the OPA-R provided the weights of the stakeholders, criteria, and projects. Using the bottom-up approach, the portfolios scores could be determined and, finally, the most appropriate portfolio for the organization was selected.

The proposed framework provides several benefits for the organization, which can be short and long term. Since the proposed approach considers profitability and resilience simultaneously, the profitability viewpoint can bring short-term benefits, and the resilience perspective can bring long-term benefits. Selecting resilient projects can increase the capability of the organization to identify the potential risk, manage unexpected events, continue despite difficult conditions such as COVID-19. Indeed, selecting resilient projects can increase the flexibility of the organization and increase the probability of survival in the future.

The main limitation of the proposed approach is dealing with an enormous scenario in the real-world situation, which can increase the computational time. Therefore, the scholars can propose a new approach that can handle many scenarios. The proposed approach can be improved to consider a high level of interaction among projects as well. Moreover, the proposed framework can be extended under the fuzzy environment to enable the robust fuzzy OPA to handle the different types of uncertainty.

CRediT authorship contribution statement

Amin Mahmoudi: Writing – original draft, Conceptualization, Methodology, Software, Visualization. Mehdi Abbasi: Methodology, Software, Conceptualization. Xiaopeng Deng: Supervision, Validation, Investigation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A

Table A1.

The score of the projects for problems 1 to 3.

Project number	Problem 1		Problem 2		Problem 3
	Weight	Rank	Weight	Rank	Weight	Rank
001	0.025334	15	0.022031	15	0.019083	16
002	0.027375	9	0.023729	9	0.020425	9
003	0.024828	18	0.021732	17	0.018840	17
004	0.023894	20	0.020996	20	0.018253	20
005	0.027838	8	0.024027	8	0.020659	8
006	0.024628	19	0.021540	18	0.018675	18
007	0.028739	5	0.024831	5	0.021304	5
008	0.025235	16	0.022075	14	0.019110	15
009	0.026532	11	0.023283	10	0.020071	10
010	0.027875	7	0.024331	6	0.020892	6
011	0.025702	14	0.022571	13	0.019495	13
012	0.029794	4	0.025643	3	0.021952	3
013	0.025975	12	0.022724	12	0.019628	12
014	0.033706	1	0.028803	1	0.024457	1
015	0.031179	2	0.026683	2	0.022784	2
016	0.026569	10	0.022786	11	0.019677	11
017	0.028392	6	0.024158	7	0.020780	7
018	0.024858	17	0.021487	19	0.018650	19
019	0.030019	3	0.025434	4	0.021780	4
020	0.025718	13	0.022004	16	0.019118	14
021	0.011389	36	0.011203	36	0.011467	37
022	0.011179	39	0.011022	40	0.011249	40
023	0.012165	31	0.011768	31	0.011683	33
024	0.012084	32	0.011745	32	0.011686	32
025	0.011505	35	0.011257	35	0.011634	34
026	0.011664	34	0.011443	34	0.011629	35
027	0.011092	40	0.011025	39	0.011299	38
028	0.011278	38	0.011129	38	0.011275	39
029	0.012404	29	0.011974	30	0.011913	30
030	0.011344	37	0.011188	37	0.011546	36
031	0.012654	28	0.012258	27	0.012161	28
032	0.012283	30	0.011976	29	0.011933	29
033	0.011965	33	0.011681	33	0.011739	31
034	0.013022	26	0.012490	26	0.012514	26
035	0.012700	27	0.012234	28	0.012354	27
036	0.013941	24	0.013041	24	0.012740	24
037	0.014360	22	0.013350	22	0.012942	22
038	0.014127	23	0.013153	23	0.012893	23
039	0.013762	25	0.012905	25	0.012521	25
040	0.014715	21	0.013546	21	0.013118	21
041	0.008094	43	0.008524	45	0.008773	43
042	0.009003	41	0.009220	41	0.009321	41
043	0.008009	46	0.008531	43	0.008732	46
044	0.007933	48	0.008462	47	0.008714	47
045	0.008052	44	0.008528	44	0.008755	45
046	0.007946	47	0.008435	48	0.008664	48
047	0.007868	49	0.008401	49	0.008660	49
048	0.007683	52	0.008332	51	0.008576	52
049	0.007771	50	0.008353	50	0.008629	50
050	0.007542	54	0.008128	53	0.008447	54
051	0.007319	57	0.007963	56	0.008311	56
052	0.007351	56	0.007983	55	0.008304	57
053	0.007202	58	0.007852	58	0.008212	59
054	0.008033	45	0.008508	46	0.008761	44
055	0.007746	51	0.008270	52	0.008568	53
056	0.007650	53	0.008077	54	0.008606	51
057	0.007363	55	0.007872	57	0.008391	55
058	0.007142	59	0.007712	59	0.008237	58
059	0.008430	42	0.008675	42	0.009207	42
060	0.006522	60	0.007176	60	0.007790	60
061	0.002600	70	0.004238	70	0.005153	70
062	0.003026	61	0.004561	61	0.005411	61
063	0.002719	69	0.004313	69	0.005201	69
064	0.002878	65	0.004432	64	0.005314	64
065	0.002931	62	0.004473	62	0.005370	62
066	0.002743	67	0.004334	67	0.005231	66
067	0.002735	68	0.004333	68	0.005210	68
068	0.002755	66	0.004343	66	0.005226	67
069	0.002889	63	0.004449	63	0.005331	63
070	0.002882	64	0.004409	65	0.005292	65
071	0.002324	77	0.003991	77	0.004946	77
072	0.002470	74	0.004117	73	0.005064	73
073	0.002485	73	0.004105	74	0.005056	74
074	0.002203	79	0.003920	79	0.004869	80
075	0.002495	72	0.004127	72	0.005067	72
076	0.002270	78	0.003875	80	0.004872	79
077	0.002399	76	0.003976	78	0.004910	78
078	0.002468	75	0.004038	76	0.005000	76
079	0.002513	71	0.004065	75	0.005030	75
080	0.001730	80	0.004157	71	0.005078	71
081			0.001926	91	0.003204	90
082			0.002029	87	0.003268	87
083			0.001858	93	0.003129	93
084			0.002238	84	0.003457	83
085			0.001920	92	0.003195	92
086			0.002038	86	0.003279	86
087			0.001979	89	0.003219	89
088			0.001789	96	0.003061	95
089			0.001981	88	0.003241	88
090			0.001959	90	0.003197	91
091			0.001732	99	0.003013	99
092			0.001745	97	0.003043	97
093			0.001833	94	0.003105	94
094			0.001742	98	0.003013	98
095			0.001669	100	0.002980	100
096			0.002180	85	0.003394	85
097			0.002269	83	0.003451	84
098			0.002272	82	0.003474	82
099			0.002531	81	0.003668	81
100			0.001797	95	0.003047	96
101					0.000761	112
102					0.000794	108
103					0.000744	115
104					0.000818	105
105					0.000746	114
106					0.000833	102
107					0.000810	107
108					0.000832	103
109					0.000849	101
110					0.000747	113
111					0.000712	116
112					0.000704	118
113					0.000784	110
114					0.000830	104
115					0.000816	106
116					0.000647	120
117					0.000706	117
118					0.000651	119
119					0.000788	109
120					0.000767	111

Table A2.

The weight of the criteria for problems 1 to 3.

Criteria	Problem 1		Problem 2		Problem 3
	Weight	Rank	Weight	Rank	Weight	Rank
C1	0.217682	1	0.209809	1	0.212911	1
C2	0.207113	2	0.201665	2	0.19933	2
C3	0.133317	5	0.140529	5	0.138287	5
C4	0.193923	3	0.189658	3	0.187583	3
C5	0.166679	4	0.168772	4	0.173225	4
C6	0.081285	6	0.089567	6	0.088663	6

Table A3.

The weight of the experts for problems 1 to 3.

Expert ID	Problem 1		Problem 2		Problem 3
	Weight	Rank	Weight	Rank	Weight	Rank
Expert 1	0.340953	1	0.3219136	1	0.307241	1
Expert 2	0.1707381	2	0.1611543	2	0.153777	2
Expert 3	0.1139417	3	0.107524	3	0.102588	3
Expert 4	0.0854563	4	0.080643	4	0.076941	4
Expert 5	0.0683185	5	0.0644793	5	0.061553	5
Expert 6	0.0569708	6	0.053762	6	0.051294	6
Expert 7	0.0488321	7	0.0460817	7	0.043966	7
Expert 8	0.0426845	8	0.0402886	8	0.038444	8
Expert 9	0.0379806	9	0.0358413	9	0.034196	9
Expert 10	0.0341244	10	0.0322133	10	0.030742	10
Expert 11			0.0293247	11	0.027979	11
Expert 12			0.0267742	12	0.025647	12
Expert 13					0.023674	13
Expert 14					0.021958	14

Appendix B

Table B1.

The key stakeholders' opinions associated with the projects alignment with the strategies.

Projects	Key Stakeholder 1			Key Stakeholder 2			Key Stakeholder 3			Key Stakeholder 4			Key Stakeholder 5
	Strategy 1	Strategy 2	Strategy 3	Strategy 1	Strategy 2	Strategy 3	Strategy 1	Strategy 2	Strategy 3	Strategy 1	Strategy 2	Strategy 3	Strategy 1	Strategy 2	Strategy 3
P1	A	SD	SA	NN	SD	A	A	D	NN	D	A	SD	D	SD	SA
P2	SA	D	SD	A	D	SD	A	SD	D	NN	D	D	SD	D	SA
P3	SD	SA	D	SD	SA	D	D	A	D	SD	SD	SA	D	A	SD
P4	SA	D	SA	A	D	SA	SA	SD	A	NN	D	A	A	D	A
P5	D	SA	SA	D	A	SA	SD	A	A	SD	D	SA	D	NN	A
P6	NN	SD	NN	NN	SD	NN	A	SD	NN	NN	SD	SD	A	NN	D
P7	SA	A	SD	SA	NN	SD	SA	A	SD	A	A	SD	D	SA	SD
P8	SA	SD	SA	A	SD	SA	SA	SD	SA	D	SD	D	A	D	A
P9	NN	D	A	NN	D	SA	D	D	A	D	D	A	SA	D	NN
P10	SD	SA	SD	SD	A	SD	SD	A	SD	SD	A	SD	NN	D	SD
P11	NN	A	NN	A	A	NN	NN	NN	NN	D	SA	NN	A	A	D
P12	D	A	NN	D	A	NN	D	A	NN	D	SA	NN	D	NN	NN
P13	SD	SA	A	D	SA	SA	SD	A	NN	SD	NN	A	SD	A	A
P14	SA	A	SD	SA	A	SD	A	A	SD	SA	D	SD	D	NN	SD
P15	D	A	SA	D	NN	SA	D	A	D	D	D	A	D	A	SA
P16	SA	D	SD	SA	SD	SD	NN	SD	A	SA	D	SD	SA	D	SD
P17	D	SD	SD	D	SD	SD	SD	SD	SD	D	SD	SD	D	SD	SD
P18	SA	SD	SA	A	SD	NN	NN	SD	A	A	D	SA	A	SD	D
P19	NN	SD	NN	A	SD	NN	NN	D	NN	NN	D	NN	NN	SD	NN
P20	SA	SD	A	SA	SD	A	D	SD	SD	NN	SD	SA	SA	SD	NN
P21	A	D	D	A	D	D	NN	SD	D	NN	SD	D	A	D	D
P22	SD	NN	NN	SD	A	NN	SD	D	NN	SD	NN	NN	SD	NN	NN
P23	SA	SD	SA	A	SD	A	A	SD	SA	A	SD	A	A	SD	A

Note: Strongly Disagree (SD) = 1, Disagree (D) = 2, Neither Disagree Nor Agree (NN) = 3, Agree (A) = 4, and Strongly Agree (SA) = 5

Appendix C

Table C1.

The key stakeholders' opinions associated with the projects in the criteria.

Experts	Rank of Criteria	Criteria	The rank of each project in each criterion
			P1	P2	P3	P4	P5	P6	P7	P8	P9	P10	P11	P12	P13	P14	P15	P16	P17	P18	P19	P20	P21	P22	P23
Stakeholder 1	1	C1	10	1	23	9	16	12	2	3	14	21	13	18	22	4	17	5	19	8	15	7	11	20	6
	4	C2	23	11	2	12	1	22	8	18	15	3	6	7	4	5	9	13	21	19	20	16	14	10	17
	3	C3	7	23	16	3	1	14	19	4	9	20	11	12	8	21	2	22	18	5	15	10	17	13	6
	5	C4	1	9	23	10	16	12	2	3	21	14	13	18	20	6	17	5	19	8	15	7	11	22	4
	2	C5	18	9	7	17	13	4	19	8	1	10	16	2	20	11	3	21	5	14	22	6	12	15	23
	6	C6	17	9	8	18	14	3	19	7	4	10	16	1	20	11	2	21	6	12	22	5	13	15	23
Stakeholder 2	1	C1	13	5	21	6	16	14	1	7	15	22	8	17	18	1–2*	19	3	20	9	10	4	11	23	12
	4	C2	21	12	2	11	5	22	9	17	14	7	4	6	1	3	10	15	20	18	19	16	13	8	23
	2	C3	9	18	16	4	2	15	19	6	5	20	10	11	1	22	3	23	21	13	14	7	17	12	8
	5	C4	4	2	19	5	6	15	18	7	11	22	16	9	3	20	1	21	23	17	13	12	14	10	8
	3	C5	18	9	7	17	13	5	19	8	1	10	16	3	20	11	2	21	4	14	22	6	12	15	23
	6	C6	17	8	9	19	13	4	18	7	2	11	15	1	22	10	3	21	6	12	20	5	14	16	23
Stakeholder 3	2	C1	4	5	14	1	19	6	2	3*	15	20	9	16	21	7	17	10	22	11	12	18	13	23	8
	3	C2	10	16	3	14	6	17	1	15	12	7	9	4	8	2	5	18	20	19	11	23	21	13	22
	1	C3	11	17	15	5	3	14	19	1	4	22	10	8	9	20	16	6	23	7	12	21	18	13	2
	5	C4	1	4	14	5	19	6	2	3	15	16	9	21	20	7*	17	10	22	11	12	18	13	23	8
	4	C5	19	7	9	17	13	5	18	8	1	10	16	3	20	11	2	22	4	14	21	6	12	15	23
	6	C6	18	9	7	17	13	5	19	8	2	10	16	3	20	11	1	21	4	15	22	6	12	14	23
Stakeholder 4	1	C1	10	16	14	5	3	13	18	1	4	9	19	7	8	20	15	6	21	17	11	22	23	12	2
	4	C2	3	8	17	9	10	18	4	19	11	5	1	2	6	12	13	14	20	15	16	21	23	7	22
	2	C3	17	14	3	6	1	23	18	15	7	19	10	11	5	20	8	21	22	2	13	4	16	12	9
	5	C4	7	3	20	16	4	14	19	2	9	23	11	12	8	21	1	22	18	13	15	10	17	5	6
	3	C5	17	7	9	19	13	5	18	8	1	10	15	3	20	11	2	22	4*	16	21	6	12	14	23
	6	C6	17	8	9	19	13	4	18	7	2	11	15	1	22	10	3	21	6	12	20	5	14	16	23
Stakeholder 5	1	C1	13	20	14	4	21	5	15	6	1	11	7	16	22	17	18	2	19	8	12	3	9	23	10
	2	C2	20	17	3	13	7	6	1	14	11	16	2	8	5	9	4	12	21	23	22	18	15	10	19
	4	C3	3	2	19	6	5	15	18	7	11	22	14	9	4	20	1	21	23	17	13	12	16	10	8
	5	C4	3	2	19	5	6	15	20	7	11	22	16	9	4	18	1	21	23	17	13	12	14	10	8
	3	C5	19	9	7	17	13	5	18	8	3	10	16	1	20	11	2	21	4	14	22	6	12	15	23
	6	C6	18	9	8	19	14	4	17	7	2	11	16	1	20	10	3	22	5	12	21	6	13	15	23

Note: Project (P), Criteria (C), * Stakeholders are not sure about these data.

Appendix D. Supplementary data

The following are the Supplementary data to this article: