Dataset on a Benchmark for Equality Constrained Multi-objective Optimization

Abstract

In this Data in Brief, we provide the source code for the equality constrained multi-objective optimization benchmark problems EqDTLZ 1–4 and EqIDTLZ 1–2 proposed in the research article “A Benchmark for Equality Constrained Multi-objective Optimization” [1]. Further, we provide the codes for the multi-objective evolutionary algorithms NSGA-II, NSGA-III, aNSGA-III, GDE3, MOEA/D/D and PPS and their numerical approximations on the above mentioned test functions. All codes are provided in Matlab using the PlatEMO classes version 2.0 in order to test different algorithms.

Specifications Table

Subject	Control and Optimization
Specific subject area	Constrained Multi-objective Optimization
Type of data	Table Figure Text file Matlab file
How data were acquired	Matlab program, PlatEMO platform
Data format	Raw Analysed
Parameters for data collection	The matlab files provide the proposed test functions, for this case D is the number of decision variables, M is the number of objectives, N is the population size, c and r are the centre and radius of each constraint. Figures show the approximation of the Pareto set/front computed by the selected MOEAs for different number of: objectives (M = 3 or M = 4), constraints (p = 1 and p = 2) and function evaluations (150,000 and 500,000) Table present the performance indicator value and the feasibility radio for a specific test function (Eq-DTLZ4) for
Description of data collection	For the figures present the approximation of the solution sets over 30 independent runs for each MOEA. The matlab files include the PlatEMO 2.0 code of each test function of the proposed benchmark.
Data accessibility	The data are in this article
Related research article	Oliver Cuate, Lourdes Uribe, Adriana Lara, and Oliver Schütze. A Benchmark for Equality Constrained Multi-objective Optimization, Swarm and Evolutionary Computation. DOI: https://doi.org/10.1016/j.swevo.2019.100619 in press [1].

Value of the Data •This dataset shows approximations of the Pareto sets and ${\text{Δ}}_2$ fronts computed by different state-of-the-art MOEAs for the proposed benchmark for different budgets of function calls. •With these data we provide a starting point for future research in the reliable treatment of equality constrained MOPs. •This dataset includes the coded test functions in order to run over PlatEMO platform. In this way, readers can reproduce the experiments described in Ref. [1] or propose new MOEAs that handle this type of MOPs. •With the test functions we provide, new experiments can be designed by varying the number of objectives, variables or constraints. Thus, harder test problems can be used with the benefit that the analytical Pareto set is given.

1. Data

The dataset contains the obtained solution sets of the proposed equality constrained MOPs for the following state-of-the-art MOEAs: ANSGA-III, GDE3, MOEAD/D/D, NSGA-II, NSGA-III and PPS (see Images folder). Also the coded test functions can be found in EqDTLZ folder.

Fig. 1 presents the obtained approximation of ANSGA-III on Eq-DTLZ1 and a budget of 150,000 function evaluations. (a) and (b) shows the one with best value. (c) and (d) shows the one with best HV value. Although many solutions are feasible, the distributions are not satisfactory.

Fig. 1.

ANSGA-III approximations on Eq-DTLZ1 and a budget of 150,000 functions evaluations.

Fig. 2 presents the obtained approximation of GDE3 on Eq-DTLZ1 and a budget of 150,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value. Although many solutions are feasible, most of them are far from the Pareto set/front.

Fig. 2.

GDE3 approximations on Eq-DTLZ1 and a budget of 150,000 functions evaluations.

Fig. 3 presents the obtained approximation of MOEA/D/D on Eq-DTLZ1 and a budget of 150,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value. Though we observe a better variation of the solution, the overall distribution is still poor.

Fig. 3.

MOEA/D/D approximations on Eq-DTLZ1 and a budget of 150,000 functions evaluations.

Fig. 4 presents the obtained approximation of NSGA-II on Eq-DTLZ1 and a budget of 150,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value. This MOEA obtained the best approximation, however, still not satisfying.

Fig. 4.

NSGA-II approximations on Eq-DTLZ1 and a budget of 150,000 functions evaluations.

Fig. 5 presents the obtained approximation of NSGA-III on Eq-DTLZ1 and a budget of 150,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value.

Fig. 5.

NSGA-III approximations on Eq-DTLZ1 and a budget of 150,000 functions evaluations.

Fig. 6 presents the obtained approximation of PPS on Eq-DTLZ1 and a budget of 150,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value.

Fig. 6.

PPS approximations on Eq-DTLZ1 and a budget of 150,000 functions evaluations.

Fig. 7 presents the obtained approximation of ANSGA-III on Eq-DTLZ2 and a budget of 500,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value.

Fig. 7.

ANSGA-III approximations on Eq-DTLZ2 and a budget of 500,000 functions evaluations.

Fig. 8 presents the obtained approximation of GDE3 on Eq-DTLZ2 and a budget of 500,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value. The plots in decision space are three dimensional projections. Though most solutions appear to be on the Pareto set, they are in fact far away as can be seen in the respective plots in the image space.

Fig. 8.

GDE3 approximations on Eq-DTLZ2 and a budget of 500,000 functions evaluations.

Fig. 9 presents the obtained approximation of MOEA/D/D on Eq-DTLZ2 and a budget of 500,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value. This method is capable of finding some Pareto optimal solutions, but the distribution is still poor.

Fig. 9.

MOEA/D/D approximations on Eq-DTLZ2 and a budget of 500,000 functions evaluations.

Fig. 10 presents the obtained approximation of NSGA-II on Eq-DTLZ2 and a budget of 500,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value. This MOEA obtained the best approximation, however, still not satisfying.

Fig. 10.

NSGA-II approximations on Eq-DTLZ2 and a budget of 500,000 functions evaluations.

Fig. 11 presents the obtained approximation of NSGA-III on Eq-DTLZ2 and a budget of 500,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value.

Fig. 11.

NSGA-III approximations on Eq-DTLZ2 and a budget of 500,000 functions evaluations.

Fig. 12 presents the obtained approximation of PPS on Eq-DTLZ2 and a budget of 500,000 function evaluations. (a) and (b) shows the one with best ${\text{Δ}}_2$ value. (c) and (d) shows the one with best HV value.

Fig. 12.

PPS approximations on Eq-DTLZ2 and a budget of 500,000 functions evaluations.

Table 1 presents the performance indicator values (${\text{Δ}}_2$ and HV) and the ratio of feasible solutions for Eq-DTLZ4 with different number of objectives and equality constraints for a long run (1,000,000 functions evaluations).

Table 1.

Performance for the long run of Eq-DTLZ4 of the different MOEAs.

METHOD	${\text{Δ}}_2$	HV	RATIO OF FEASIBILITY
EQ-DTLZ4 (M = 3 and p = 1)
ANSGA-III	2.8724e-04	2.1059e-10	1.00
GDE3	4.3166e-01	2.1056e-10	1.00
MOEADD	1.2032e-05	2.0252e-10	1.00
NSGA-II	4.5451e-04	2.1059e-10	0.99
NSGA-III	2.8724e-04	2.1059e-10	1.00
PPS	5.4675e-03	2.1058e-10	0.99
EQ-DTLZ4 (M = 4 and p = 1)
ANSGA-III	4.9884e-03	8.9390e-15	1.00
GDE3	7.8509e-01	8.8768e-15	1.00
MOEADD	5.7475e-06	8.6261e-15	1.00
NSGA-II	4.0930e-03	8.9390e-15	1.00
NSGA-III	4.9884e-03	8.9390e-15	1.00
PPS	1.2327e-02	7.4647e-15	1.00
EQ-DTLZ4 (M = 4 and p = 2)
ANSGA-III	4.0995e-04	1.7954e-20	0.98
GDE3	1.0044e+00	0.0000e+00	0.06
MOEADD	4.3665e-04	1.3263e-20	1.00
NSGA-II	1.8104e-03	1.8035e-20	1.00
NSGA-III	4.0995e-04	1.7954e-20	0.98
PPS	9.5984e-03	1.7933e-20	1.00

Fig. 13 shows the Pareto set/fronts approximations of Eq-DTLZ4 with M = 3 and p = 1 for: (a) ANSGA-III, (b) GDE3, (c) MOEA/D/D, (d) NSGA-II, (e) NSGA-III and (f) PPS for the long run (1,000,000 function calls).

Fig. 13.

Approximations on Eq-DTLZ4 with M = 3 and p = 1 of the selected MOEAs for a budget of 1,000,000 functions evaluations.

Fig. 14 shows the Pareto set/fronts approximations of Eq-DTLZ4 with M = 4 and p = 1 for: (a) ANSGA-III, (b) GDE3, (c) MOEA/D/D, (d) NSGA-II, (e) NSGA-III and (f) PPS for the long run (1,000,000 function calls).

Fig. 14.

Approximations on Eq-DTLZ4 with M = 4 and p = 1 of the selected MOEAs for a budget of 1,000,000 functions evaluations.

2. Experimental design, materials, and methods

We have selected six state-of-the-art MOEAs in order to test the proposed benchmark. For all experiments we use the PlatEMO framework [2], where we have executed 30 independent runs of the following MOEAs: NSGA-II, NSGA-III, Adaptive NSGA-III, MOEA/D/D, GDE3 and PPS; using 50, 000, 100, 000 and 150, 000 function calls for MOPs with M = 3; and using 200, 000, 300, 000 and 500, 000 function calls for MOPs with M = 4. For the long run, we execute all the selected MOEAs for 1,000,000 function evaluations for Eq-DTLZ4 with different number of objectives and constraints We have selected six state-of-the-art MOEAS in order to test the proposed benchmark. For all experiments we use the PlatEMO framework [2], where we have executed 30 independent runs of the following MOEAs: NSGA-II, NSGA-III, Adaptive NSGA-III, MOEA/D/D, GDE3 and PPS; using 50, 000, 100, 000 and 150, 000 function calls for MOPs with M = 3; and using 200, 000, 300, 000 and 500, 000 function calls for MOPs with M = 4. For the long run, we execute all the selected MOEAs for 1,000,000 function evaluations for Eq-DTLZ4 with different number of objectives and constraints. We also use the Kruskal-Wallis test as statistical significance proof to validate the numerical approximations. For this, we apply the test to each function considering the different algorithms as the groups and each run as the sample. We set $\propto =0.05$; if the $p$ value of the test is less than $\propto$ then we reject the null hypothesis, which is that the sample data from each group comes from the same distribution, for more details see Ref. [1].

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article: