A hybrid conjugate gradient algorithm for constrained monotone equations with application in compressive sensing

Abstract

Combining the projection method of Solodov and Svaiter with the Liu-Storey and Fletcher Reeves conjugate gradient algorithm of Djordjević for unconstrained minimization problems, a hybrid conjugate gradient algorithm is proposed and extended to solve convex constrained nonlinear monotone equations. Under some suitable conditions, the global convergence result of the proposed method is established. Furthermore, the proposed method is applied to solve the ${\ell }_1$-norm regularized problems to restore sparse signal and image in compressive sensing. Numerical comparisons of the proposed algorithm versus some other conjugate gradient algorithms on a set of benchmark test problems, sparse signal reconstruction and image restoration in compressive sensing show that the proposed scheme is computationally more efficient and robust than the compared schemes.

Applied mathematics; Computer science; Conjugate gradient method; Projection method; Convex constraints; Compressive sensing

1. Introduction

Let C be a non-empty, closed and convex subset of $R^n$ and $G:R^n\to R^n$ be a continuous and monotone mapping. By monotonicity, it means for all $x,y\in R^n$, the function satisfies

$${(G(x)-G(y))}^T(x-y)\ge 0.$$ (1)

In this paper, we are interested in finding solution of the nonlinear monotone equations with convex constraints of the form

$$G(x)=0,x\in C.$$ (2)

It has been found that various problems with vast applications in interdisciplinary areas can be elegantly modeled using (2). For instance, the power flow equations [1], compressive sensing [2], the economic equilibrium problem [3]. To this effect, researchers have focused on numerical methods for solving (2). Several algorithms have been proposed for solving (2). For example, Newton method, Quasi-Newton method, Levenberge Marquardt method and a series of their variants, see [4], [5], [6], [7], [8] for an overview of these results. It is not surprising that the mentioned methods are attractive, this is due to their rapid convergence from a sufficiently good initial guess. However, they are not suitable for handling large scale nonlinear systems of equations because at each iteration, the Jacobian matrix or its approximation is needed.

In recent times, influenced by the projection method developed by Solodov and Svaiter [9], some of the first-order optimization methods such as the conjugate gradient (CG) method which are well known for solving large-scale unconstrained optimization problems and characterized by their simplicity and low storage have been extended by several researchers to solve (2) (see [2], [10], [11], [12], [13], [14]). Most of these extensions and newly developed methods are variants of the well known CG-method which is one of the foremost vital numerical methods for unconstrained optimization. Some of the earliest conjugate gradient algorithms includes the Fletcher-Reeves (FR) method [15], the Polak Ribiére-Polyak (PRP) method [16], [17], the Hestenes-Steifel (HS) method [18], the Liu-Storey (LS) [19] method, Dai-Yuan (DY) method [20].

Motivated by the good practical behavior of the LS method and strong convergence of the FR method, Djordjević [21] proposed a hybrid conjugate gradient algorithm for solving unconstrained minimization problem. Numerical experiment indicates that the proposed algorithm is efficient and superior to other conjugate gradient algorithms such as the conjugate gradient descent algorithm which is often referred to as CG_DESCENT [2]. Recently, the CG_DESCENT was extended to solve large-scale nonlinear convex constraint monotone equations by Xiao and Zhu [2]. The method was shown to be efficient in solving monotone equations arising from compressive sensing. Can the method of Djordjević be extended to solve constrained monotone equations inheriting the good practical behavior of the LS method and strong convergence of the FR method? Also, how about the computational performance of the method? The focus of this article is to give a positive answer to these questions.

The main contribution of this paper is to propose, analyse and test a hybrid conjugate gradient algorithm combined with the projection technique of Solodov and Svaiter to solve problem (2). Furthermore, with the reformulation of the ${\ell }_1$-norm problem as a non-smooth monotone equation [22], the proposed algorithm is used to solve sparse signal and image restoration problem. In addition, we show that the proposed algorithm exhibit some appealing properties. For instance, the algorithm exhibit less number of iterations and function evaluations. Under some mild assumptions, the global convergence of the algorithm is established. Numerical experiment indicates that the proposed algorithm is efficient, robust and competitive.

This paper is structured as follows: In section 2, we recall some preliminaries. Next, we give the description of our proposed algorithm. Analysis of its global convergence is given in section 3. Numerical results obtained from testing the new method to solve some benchmark test problems are reported in section 4. Finally, we end this paper with section 6 where we demonstrate application of the proposed method in recovery sparse signal and image restoration.

2. Preliminaries and algorithm

Given an initial point $x_0$, an iterative scheme for (2) generally generate a sequence of iterates $\{x_k\}$ by

$$x_{k+1}:=x_k+{\alpha }_k{d}_k,k=0,1,\cdots ,$$ (3)

where ${\alpha }_k>0$ is the step length which is computed by a certain line search and $d_k$ is the search direction usually satisfying

$$d_k^{T}G(x_k)\le -c{\Vert G(x_k)\Vert }^2,$$ (4)

with positive constant c. If G is the gradient of a real-valued function $g:R^n\to R$, the descent condition means that $d_k$ is a direction of sufficient descent g at $x_k$. For convenience, we abbreviate $G(x_k)$ as $G_k$.

To describe our algorithm, we recall the projection map denoted as $P_C$, which is a mapping from $R^n$ onto the nonempty convex set C, that is

$$P_C(x):=\arg \min \{\Vert x-y\Vert :y\in C\},$$

which has the well known nonexpansive property

$$\Vert P_C(x)-P_C(y)\Vert \le \Vert x-y\Vert ,\forall x,y\in R^n.$$ (5)

In what follows, we assume that G satisfies the following assumptions.

Assumption 1

The mapping G is Lipschitz continuous on $R^n$. That is,

$$\Vert G(x)-G(y)\Vert \le L\Vert x-y\Vert \forall x,y\in R^n.$$

Assumption 2

The solution set $C^{⁎}$ is nonempty.

In this paper, we propose the search direction based on the method proposed in [21]. Specifically, $d_k$ is determined by

$$d_k:=\{\begin{array}{cc}-G_k+{\beta }_k(I-\frac{G_k{G}_k^T}{{\Vert G_k\Vert }^2})w_{k-1}\hfill & \text{if }k>0\text{,}\hfill \\ -G_k\hfill & \text{if }k=0\text{,}\hfill \\ \hfill \end{array}$$ (6)

where $w_k:=z_k-x_k={\alpha }_k{d}_k$ and parameter ${\beta }_k$ computed as a convex combination of LS and FR methods. That is,

$${\beta }_k:=(1-{\vartheta }_k){\beta }_k^{LS}+{\vartheta }_k{\beta }_k^{FR}$$ (7)

$$:=(1-{\vartheta }_k)\frac{G_k^T{y}_{k-1}}{-G_{k-1}^T{d}_{k-1}}+{\vartheta }_k\frac{{\Vert G_k\Vert }^2}{{\Vert G_{k-1}\Vert }^2}$$ (8)

Substituting (8) into (6), we have

$$d_k=-G_k+((1-{\vartheta }_k)\frac{G_k^T{y}_{k-1}}{-G_{k-1}^T{d}_{k-1}}+{\vartheta }_k\frac{{\Vert G_k\Vert }^2}{{\Vert G_{k-1}\Vert }^2})(w_{k-1}-\frac{w_{k-1}^T{G}_k}{{\Vert G_k\Vert }^2}{G}_k)$$ (9)

We select ${\vartheta }_k$ such that the search direction $d_k$ satisfies the famous conjugacy condition. That is,

$$d_k^T{y}_{k-1}=0.$$ (10)

We have

$$0=-G_k^T{y}_{k-1}+{\beta }_k(w_{k-1}^T{y}_{k-1}-\frac{w_{k-1}^T{G}_k}{{\Vert G_k\Vert }^2}{G}_k^T{y}_{k-1}),$$ (11)

rearranging gives, ${\vartheta }_k=\frac{1}{({\beta }_k^{FR}-{\beta }_k^{LS})}(\frac{G_k^T{y}_{k-1}}{\mathrm{\Lambda }}-{\beta }_k^{LS})$ where $\mathrm{\Lambda }=(w_{k-1}^T{y}_{k-1}-\frac{w_{k-1}^T{G}_k}{{\Vert G_k\Vert }^2}{G}_k^T{y}_{k-1})$.

Next, we formally present a hybrid conjugate gradient algorithm as a convex combination of LS and FR method for solving (2). For simplicity, we refer to this algorithm as HLSFR algorithm.

Algorithm 2.1

(HLSFR)

Input. Choose any random point $x_0\in C$, the positive constants: $\xi \in (0,1)$, $\varrho \in (0,2)$ $Tol>0$, $\gamma >0$. Set $k=0$.

Step 0. Compute $G_k$. If $\Vert G_k\Vert \le Tol$, stop. Otherwise, compute the search direction $d_k$ by

$$d_k:=-G_k+{\beta }_k(I-\frac{G_k{G}_k^T}{{\Vert G_k\Vert }^2})w_{k-1}$$

If ${\vartheta }_k\in (0,1)$, then compute ${\beta }_k$ using (8)

If ${\vartheta }_k\ge 1$, then compute ${\beta }_k={\beta }_k^{FR}$

If ${\vartheta }_k\le 1$, then compute ${\beta }_k={\beta }_k^{LS}$

If $\mathrm{\Lambda }=0$, we set ${\vartheta }_k=0$.

Step 1. Let ${\alpha }_k=\max \{{\xi }^m|m=0,1,2,\cdots \}$ be determined by the following line-search

$$-G{(x_k+{\xi }^m{d}_k)}^T{d}_k\ge \gamma {\xi }^m{\Vert d_k\Vert }^2.$$ (12)

Step 2. Compute the trial point

$$z_k:=x_k+{\alpha }_k{d}_k.$$ (13)

Step 3. If $z_k\in C$ and $\Vert G(z_k)\Vert \le Tol$, stop. Otherwise, compute

$$x_{k+1}:=P_C[x_k-\varrho {\phi }_{k}G(z_k)]$$ (14)

where

$${\phi }_k:=\frac{G{(z_k)}^T(x_k-z_k)}{{\Vert G(z_k)\Vert }^2}$$

Step 4. Set $k:=k+1$ and go to step 1.

Remark 2.2

It is clear to see that the ${\beta }_k$ we propose is similar to that proposed in [21] but with different definition on ${\vartheta }_k$ and $d_k$. The search direction $d_k$ defined by (6) originated in [23] which was originally used in solving unconstrained optimization problem. Here, the method is extended to solve nonlinear monotone equations with convex constraints.

Lemma 2.3

Let $d_k$ be the search direction generated by (6). Then, $d_k$ always satisfies the sufficient decent condition (4). That is,

$$G_k^T{d}_k=-{\Vert G_k\Vert }^2$$ (15)

for all $k\ge 0$.

Proof

From the definition of $d_k$ (6), it is easy to see that (15) holds. □

3. Convergence analysis

Lemma 3.1

The line search is well defined. That is, for all $k\ge 0$, there exists a non negative integer m satisfying (12).

Proof

We begin by contradiction. Suppose there exist $k_0\ge 0$ such that (12) is not satisfied for any nonnegative integer m, that is

$$-G{(x_{k_0}+{\xi }^m{d}_{k_0})}^T{d}_{k_0}<\gamma {\xi }^m{\Vert d_{k_0}\Vert }^2,\forall m\ge 1.$$

Using the continuity of G and letting $m\to \infty$ yields

$$-G_{k_0}^T{d}_{k_0}\le 0$$

which contradicts (15). This completes the proof. □

Lemma 3.2

The HLSFR algorithm is well defined.

Proof

The first step is to notice that, from the line search (12), if ${\alpha }_k\ne {\xi }^m$, then $\overline{{\alpha }_k}={\xi }^{-1}{\alpha }_k$ does not satisfy (12), that is,

$$-G{(x_k+{\xi }^{-1}{\alpha }_k{d}_k)}^T{d}_k<\gamma {\xi }^{-1}{\alpha }_k{\Vert d_k\Vert }^2.$$ (16)

Equation (16) combined with (15), we have

$${\Vert G_k\Vert }^2=-G_k^T{d}_k={(G(x_k+{\xi }^{-1}{\alpha }_k)-G(x_k))}^T{d}_k-G{(x_k+{\xi }^{-1}{\alpha }_k)}^T{d}_k\le {\xi }^{-1}{\alpha }_{k}L{\Vert d_k\Vert }^2+{\xi }^{-1}{\alpha }_k\gamma {\Vert d_k\Vert }^2={\xi }^{-1}{\alpha }_k(L+\gamma ){\Vert d_k\Vert }^2.$$ (17)

Since G is a Lipschitz continuous function, then the above inequality is valid. Thus, from (17),

$${\alpha }_k\ge \min \{1,\frac{\xi }{(L+\gamma )}\frac{{\Vert G_k\Vert }^2}{{\Vert d_k\Vert }^2}\}$$ (18)

This proves Lemma 3.2. □

Lemma 3.3

Suppose G is monotone and Lipchitz continuous on $R^n$ and the sequence $\{x_k\}$ is generated by (14) in HLSFR algorithm, then there exists $\varpi >0$ such that

$$\Vert G_k\Vert \le \varpi .$$ (19)

Proof

Recall that, from the nonexpansiveness of the projection operator, it holds that for any $x_{⁎}\in C^{⁎}$,

$${\Vert x_{k+1}-x_{⁎}\Vert }^2={\Vert P_C[x_k-\varrho {\phi }_{k}G(z_k)]-x_{⁎}\Vert }^2\le {\Vert x_k-\varrho {\phi }_{k}G(z_k)-x_{⁎}\Vert }^2=\Vert x_k-x_{⁎}\Vert -2\varrho {\phi }_{k}G{(z_k)}^T(x_k-x_{⁎})+{\varrho }^2{\phi }_k^2{\Vert G(z_k)\Vert }^2=\Vert x_k-x_{⁎}\Vert -2\varrho \frac{G{(z_k)}^T(x_k-z_k)}{{\Vert G(z_k)\Vert }^2}G{(z_k)}^T(x_k-x_{⁎})+{\varrho }^2{\left(\frac{G{(z_k)}^T(x_k-z_k)}{\Vert G(z_k)\Vert }\right)}^2\le \Vert x_k-x_{⁎}\Vert -2\varrho \frac{G{(z_k)}^T(x_k-z_k)}{{\Vert G(z_k)\Vert }^2}G{(z_k)}^T(x_k-z_k)+{\varrho }^2{\left(\frac{G{(z_k)}^T(x_k-z_k)}{\Vert G(z_k)\Vert }\right)}^2={\Vert x_k-x_{⁎}\Vert }^2-\varrho (2-\varrho ){\left(\frac{G{(z_k)}^T(x_k-z_k)}{\Vert G(z_k)\Vert }\right)}^2$$ (20)

$$\le {\Vert x_k-x_{⁎}\Vert }^2.$$ (21)

The above inequality (21) implies that the sequence $\{\Vert x_k-x_{⁎}\Vert \}$ is a decreasing sequence. Therefore, the sequence $\{x_k\}$ is bounded, that is

$$\Vert x_k\Vert \le \varsigma ,\varsigma >0.$$ (22)

In addition, we obtain

$$\Vert x_{k+1}-x_{⁎}\Vert \le \Vert x_k-x_{⁎}\Vert \le \Vert x_{k-1}-x_{⁎}\Vert \le \cdots \Vert x_0-x_{⁎}\Vert .$$ (23)

Using the Lipchitz continuity of G, we have

$$\Vert G_k\Vert =\Vert G_k-G(x_{⁎})\Vert \le L\Vert x_k-x_{⁎}\Vert \le L\Vert x_0-x_{⁎}\Vert .$$ (24)

Setting $\varpi =L\Vert x_0-x_{⁎}\Vert$ proves Lemma 3.3. □

Lemma 3.4

Let $\{z_k\}$ and $\{x_k\}$ be sequences generated by (13) and (14) respectively under Assumption 1, Assumption 2 using HLSFR Algorithm, then $-G(z_k)$ is a descent direction of the function $\frac{1}{2}{\Vert x-x_{⁎}\Vert }^2$ at the point $x_k$ where $x_{⁎}\in C^{⁎}$.

Proof

At $x_k$, the function $\frac{1}{2}{\Vert x-x_{⁎}\Vert }^2$ has a gradient of $x_k-x_{⁎}$. By monotonicity property (1), it can be seen that

$$G{(z_k)}^T(x_k-x_{⁎})=G{(z_k)}^T(x_k+z_k-z_k-x_{⁎})=G{(z_k)}^T(z_k-x_{⁎})+G{(z_k)}^T(x_k-z_k)\ge G{(x_{⁎})}^T(z_k-x_{⁎})+G{(z_k)}^T(x_k-z_k)=G{(z_k)}^T(x_k-z_k)\ge \gamma {\alpha }_k^2{\Vert d_k\Vert }^2=\gamma {\Vert x_k-z_k\Vert }^2>0,$$ (25)

which indicates that the function $\frac{1}{2}\Vert x-x_{⁎}\Vert$ has a descent direction $-G(z_k)$ at the iteration point $x_k$. □

Lemma 3.5

Suppose Assumption 1, Assumption 2 hold and the sequence $z_k$ and $x_k$ are generated by (13) and (14) respectively in HLSFR algorithm. Then,

• (i)$\{z_k\}$ is bounded
• (ii)${\lim }_{k\to \infty }\Vert x_k-z_k\Vert =0$
• (iii)${\lim }_{k\to \infty }\Vert x_k-x_{k+1}\Vert =0$.

Proof

• (i)
  From (23), we know that the sequence $\{x_k\}$ is bounded. From (25), we have

  $$G{(z_k)}^T{x}_k-z_k\ge \gamma {\Vert x_k-z_k\Vert }^2.$$ (26)

  Utilizing (19) and (1), we have

  $$G{(z_k)}^T(x_k-z_k)={(G(z_k)-G_k)}^T(x_k-z_k)+G_k^T(x_k-z_k)\le \Vert G_k\Vert \Vert x_k-z_k\Vert \le \varpi \Vert x_k-z_k\Vert .$$

  Combined with (26), it is easy to deduce that

  $$\Vert x_k-z_k\Vert \le \frac{\varpi }{\gamma }.$$

  Then, we obtain,

  $$\Vert z_k\Vert \le \frac{\varpi }{\gamma }+\Vert x_k\Vert$$

  Hence the sequence $\{z_k\}$ is bounded owing to the boundedness of $\{x_k\}$.
• (ii)
  From inequality (20), we get

  $$\Vert x_{k+1}-x_{⁎}\Vert \le {\Vert x_k-x_{⁎}\Vert }^2-\varrho (2-\varrho )\frac{{[G{(z_k)}^T(x_k-z_k)]}^2}{{\Vert G(z_k)\Vert }^2}\le {\Vert x_k-x_{⁎}\Vert }^2-\varrho (2-\varrho )\frac{{\gamma }^2{\Vert x_k-z_k\Vert }^4}{{\Vert G(z_k)\Vert }^2},$$

  which means

  $$\varrho (2-\varrho ){\Vert x_k-z_k\Vert }^4\le \frac{{\Vert G(z_k)\Vert }^2}{{\gamma }^2}({\Vert x_k-x_{⁎}\Vert }^2-{\Vert x_{k+1}-x_{⁎}\Vert }^2).$$

  From the fact that the function G is continuous and the sequence $\{z_k\}$ is bounded, we know that the sequence $\{\Vert G(z_k)\Vert \}$ is bounded. Hence, there exist a positive ${\varpi }_1>0$ such that $\Vert G(z_k)\Vert \le {\varpi }_1$ and furthermore

  $$\varrho (2-\varrho )\sum _{k=0}^{\infty }{\Vert x_k-z_k\Vert }^4\le \frac{{\varpi }_1^2}{{\gamma }^2}\sum _{k=0}^{\infty }({\Vert x_k-x_{⁎}\Vert }^2-{\Vert x_k-x⁎\Vert }^2)=\frac{{\varpi }_1^2}{{\gamma }^2}{\Vert x_0-x_{⁎}\Vert }^2<+\infty .$$

  Hence,

  $$\underset{k\to \infty }{\lim }{\alpha }_k\Vert d_k\Vert =\underset{k\to \infty }{\lim }\Vert x_k-z_k\Vert =0.$$ (27)
• (iii)
  From the nonexpansiveness of the projection operator, we have

  $$\Vert x_k-x_{k+1}\Vert =\Vert x_k-P_C[x_k-\varrho {\phi }_{k}G(z_k)]\Vert \le \Vert x_k-(x_k-\varrho {\phi }_{k}G(z_k))\Vert =\Vert \varrho {\phi }_{k}G(z_k)\Vert \le \varrho \Vert x_k-z_k\Vert .$$

   □

The following theorem establishes the global convergence of HLSFR algorithm.

Theorem 3.6

Suppose conditions of Assumption 1, Assumption 2 hold. Then, the sequence $\{x_k\}$ generated by (14) in HLSFR method converges globally to a solution of (2).

$$\underset{k\to \infty }{\lim \inf }\Vert G_k\Vert =0.$$ (28)

Proof

Suppose (28) does not hold, meaning there exists a constant ${\epsilon }_0>0$ such that

$$\Vert G_k\Vert \ge {\epsilon }_{0}k\ge 0.$$ (29)

By (15), we know

$$\Vert G_k\Vert \Vert d_k\Vert \ge -G_k^T{d}_k\ge {\Vert G_k\Vert }^2,$$

which implies

$$\Vert d_k\Vert \ge \Vert G_k\Vert \ge {\epsilon }_0,\forall k\ge 0.$$ (30)

By (6), we have

$$\begin{gathered} \Vert d_k\Vert =\Vert -G_k+((1-{\vartheta }_k)\frac{G_k^T{y}_{k-1}}{-G_{k-1}^T{d}_{k-1}}+{\vartheta }_k\frac{{\Vert G_k\Vert }^2}{{\Vert G_{k-1}\Vert }^2})w_{k-1} \\ -((1-{\vartheta }_k)\frac{G_k^T{y}_{k-1}}{-G_{k-1}^T{d}_{k-1}}+{\vartheta }_k\frac{{\Vert G_k\Vert }^2}{{\Vert G_{k-1}\Vert }^2})\frac{G_k{G}_k^T{w}_{k-1}}{{\Vert G_k\Vert }^2}\Vert \le \Vert G_k\Vert +2(\frac{\Vert G_k\Vert \Vert y_{k-1}\Vert }{|G_{k-1}^T{d}_{k-1}|}+\frac{{\Vert G_k\Vert }^2}{{\Vert G_{k-1}\Vert }^2}){\alpha }_{k-1}\Vert d_{k-1}\Vert \le \varpi +\frac{2\varpi (2\varsigma L+\varpi )}{{\epsilon }_0^2}{\alpha }_{k-1}\Vert d_{k-1}\Vert \end{gathered}$$

for all $k\in \mathbb{N}$. Since (27) hold, it follows that for every ${\epsilon }_1>0$ there exist $k_0$ such that ${\alpha }_{k-1}\Vert d_{k-1}\Vert <{\epsilon }_1$ for every $k>k_0$. Choosing ${\epsilon }_1={\epsilon }_0^2$ and $\mathbb{H}=\max \{\Vert d_0\Vert ,\Vert d_1\Vert ,\cdots ,\Vert d_{k_0}\Vert ,{\mathbb{H}}_1\}$ where ${\mathbb{H}}_1=\varpi (1+4\varsigma L+2\varpi )$, it holds that

$$\Vert d_k\Vert \le \mathbb{H}$$ (31)

for every $k\in \mathbb{N}$. Integrating with (18), (29), (30) and (31), we know that for any k sufficiently large

$${\alpha }_k\Vert d_k\Vert \ge \min \{1,\frac{\xi }{(L+\gamma )}\frac{{\Vert G_k\Vert }^2}{{\Vert d_k\Vert }^2}\}\Vert d_k\Vert =\min \{\Vert d_k\Vert ,\frac{\xi }{(L+\gamma )}\frac{{\Vert G_k\Vert }^2}{\Vert d_k\Vert }\}\ge \min \{{\epsilon }_0,\frac{\xi {\epsilon }_0^2}{(L+\gamma )\mathbb{H}}\}$$

The last inequality yields a contradiction with $(ii)$ in Lemma 3.5. Consequently, (28) holds. The proof is completed. □

4. Numerical results

We present a detail report of the numerical experiment in testing the performance of HLSFR. We compared HLSFR with the CGD, PCG, PDY and ACGD methods in [2], [13], [24], [25] respectively. The mapping G is taken as

$$G(x)={(g_1(x),g_2(x),\cdots ,g_n(x))}^T,$$

where the associated initial points for these problems are

$$x_1={(1,\cdots ,1)}^T,x_2={(0.1,\cdots ,0.1)}^T,x_3={(\frac{1}{2},\cdots ,\frac{1}{2_n})}^T,x_4={(1-\frac{1}{n},\cdots ,n-1)}^T,x_5={(0,\frac{1}{n},\cdots ,\frac{n-1}{n})}^T,x_6={(1,\frac{1}{2},\cdots ,\frac{1}{n})}^T,x_7={(\frac{n-1}{n},\frac{n-2}{n},\cdots ,0)}^T,x_8={(\frac{1}{n},\frac{2}{n},\cdots ,1)}^T,x_9=rand(n,1).$$

We made use of the following benchmark test problems in testing the effectiveness and robustness of the methods.

Problem 1

This problem is the Exponential function [26] with constraint set $C=R_{+}^n$, that is,

$$g_1(x)=e^{x_1}-1,g_i(x)=e^{x_i}+x_i-1,\text{for }i=2,3,...,n.$$

Problem 2

Modified Logarithmic function [26] with constraint set $C=\{x\in R^n:{\sum }_{i=1}^n{x}_i\le n,x_i>-1,i=1,2,\dots ,n\}$, that is,

$$g_i(x)=\ln (x_i+1)-\frac{x_i}{n},i=2,3,...,n.$$

Problem 3

The Nonsmooth Function [27] with constraint set $C=R_{+}^n$.

$$g_i(x)=2x_i-\sin |x_i|,i=1,2,3,...,n.$$

Problem 4

The Strictly convex function [28], with constraint set $C=R_{+}^n$, that is,

$$g_i(x)=e^{x_i}-1,i=2,3,\cdots ,n$$

Problem 5

Tridiagonal Exponential function [29] with constraint set $C=R_{+}^n$, that is,

$$g_1(x)=x_1-e^{\cos (h(x_1+x_2))},g_i(x)=x_i-e^{\cos (h(x_{i-1}+x_i+x_{i+1}))},\text{for}2\le i\le n-1,g_n(x)=x_n-e^{\cos (h(x_{n-1}+x_n))},\text{where}h=\frac{1}{n+1}$$

Problem 6

Nonsmooth function [30] with constraint set $C=\{x\in R^n:{\sum }_{i=1}^n{x}_i\le n,x_i\ge -1,1\le i\le n\}$.

$$g_i(x)=x_i-\sin |x_i-1|,i=2,3,\cdots ,n$$

Problem 7

The Trig exp function [26] with constraint set $C=R_{+}^n$, that is,

$$\begin{gathered} g_1(x)=3x_1^3+2x_2-5+\sin (x_1-x_2)\sin (x_1+x_2)g_i(x)=3x_i^3+2x_{i+1}-5+\sin (x_i-x_{i+1})\sin (x_i+x_{i+1})\phantom{g_i(x)=}+4x_i-x_{i-1}{e}^{x_{i-1}-x_i}-3\text{for}i=2,3,...,n-1 \\ {g}_n(x)=x_{n-1}{e}^{x_{n-1}-x_n}-4x_n-3,\text{where h}=\frac{1}{n+1}. \end{gathered}$$

Problem 8

The Penalty 1 function [25] with $C=R_{+}^n$, that is,

$$\begin{gathered} t_i=\sum _{i=1}^n{x}_i^2,c={10}^{-5} \\ {g}_i(x)=2c(x_i-1)+4(t_i-0.25)x_i,i=1,2,3,...,n. \end{gathered}$$

Problem 9

The function $g_i(x)$ with $C=R_{+}^n$ defined by,

$$g_i(x)=8^{\frac{1}{2}}{x}_i-1i=1,2,3,...,n.$$

Problem 10

The function $g_i(x)$ with $C=R_{+}^n$ defined,

$$g_i(x)=e^{x_i^2}+3\sin x_i\cos x_i-1fori=1,2,...,n,$$

The codes for all methods were written on a windows 10 HP personal computer with 2.40 GHz processor, 8 GB RAM of Intel(R) Core (TM) i3-7100U using Matlab R2019b software. We choose the following parameters: $\gamma ={10}^{-4}$, $\xi =0.6$, $\varrho =1.8$ in implementing HLSFR. For the other methods, their parameters were set as in their respective papers. Furthermore, for each test problem, the iterations are terminated when the inequality $\Vert G_k\Vert \le {10}^{-6}$ is satisfied. Failure is declared if the inequality is not satisfied after 1000 iterations. A comprehensive results of our numerical experiment are presented in the appendix section. The columns of the presented tables have the following definitions:

• IP: denotes the initial points

• DIM: denotes the dimension of the problem

• NI: represents iterative number

• NF: denotes iterative number of function evaluation.

• CPU: denotes the CPU time in seconds when the algorithms terminate

• NORM: denotes the final norm equation

From Tables 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12, it is not difficult to see that all methods solved all the test problems successfully. However, the HLSFR method highly performs better compared with CGD, PCG, PDY and ACGD in terms of the iteration number and the number of function evaluations.

To visualize the efficiency of HLSFR algorithms, we adopt the Dolan and More [31] performance profile. Figs. 1, 2 and 3 illustrates the performance profile of the five algorithms, where the performance indices are the total number of iterations, iterative number of function evaluations and CPU time of Tables 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12 in the appendix section. It can be seen that the HLSFR algorithm is the best solver with probability of solving 60% and 68% of the test problems with the least number of iterations and function evaluation respectively.

Figure 1.

Performance profiles for the number of iterations.

Figure 2.

Performance profiles for number of function evaluations.

Figure 3.

Performance profiles for CPU running time.

5. Application in compressive sensing

5.1. General description

Digital image processing plays an important role in medical sciences, biological engineering and other areas of science and engineering [32], [33], [34]. Let $A\in R^{k\times n}(k<n)$ be a linear operator and $\overline{x}$ be a sparse original signal. For a given observation $b\in R^k$ that satisfies

$$b=A\overline{x}.$$

It is necessary to reconstruct the original signal $\overline{x}$ from the linear system $A\overline{x}=b$. However, the framework is typically ill-conditioned and grant infinite solutions. In this case, it is typical to look for the sparsest one among all solutions provided that b is gained from a profoundly sparse signal. As a rule, the Basis Pursuit denoising problem is appropriate

$$\underset{x}{\min }\tau {\Vert x\Vert }_1+\frac{1}{2}{\Vert Ax-b\Vert }_2^2,$$ (32)

where τ is a positive parameter.

In what follows, we give a short overview of the reformulation of (32) into a convex quadratic program by Figueiredo in [35]. Consider any vector x such that $x\in R^n$. The vector x can be rewritten as

$$x=u-v,u\ge 0,v\ge 0,$$

where $u\in R^n,v\in R^n$ and $u_i={(x_i)}_{+}$, $v_i={(-x_i)}_{+}$ for all $i\in [1,n]$ with ${(\cdot )}_{+}=\max \{0,\cdot \}$. Subsequently, we represent the ${\ell }_1$-norm of a vector as ${\Vert x\Vert }_1=e_n^{T}u+e_n^{T}v$, where $e_n$ is an n-dimensional vector with all element one. Hence, (32) can be rewritten as

$$\underset{u,v}{\min }\frac{1}{2}{\Vert b-A(u-v)\Vert }^2+\tau e_n^{T}u+\tau e_n^{T}v,u,v\ge 0.$$ (33)

Moreover, from [35], with no difficulty, (33) can be rewritten as

$$\underset{z}{\min }\frac{1}{2}{z}^{T}Hz+c^{T}z,z\ge 0,$$ (34)

where $z=\left[\begin{array}{c}\hfill u\hfill \\ \hfill v\hfill \\ \hfill \hfill \end{array}\right]$, $c=\tau e_{2n}+\left[\begin{array}{c}\hfill -y\hfill \\ \hfill y\hfill \\ \hfill \hfill \end{array}\right]$, $b=A^{T}y$, $H=\left[\begin{array}{cc}\hfill A^{T}A\hfill & \hfill -A^{T}A\hfill \\ \hfill -A^{T}A\hfill & \hfill A^{T}A\hfill \\ \hfill \hfill \end{array}\right]$.

Obviously, H is a positive semi-definite matrix, which implies that equation (34) is a convex quadratic programming problem. Quite recently, equation (34) was translated into a linearly inequality problem by Xiao and Zhu [2] which is equivalent to

$$G(z)=\min \{z,Hz+c\}=0,$$ (35)

where $G(z)$ is said to be continuous and monotone, see [22], [36]. Therefore, (34) can be effectively solved using the HLSFR method.

5.2. Numerical results

In this subsection, our main focus is utilizing HLSFR Algorithm in the restoration of one dimensional sparse signal and image restoration. We begin the experiment with the restoration of a one dimensional sparse signal from its limited measurement with additive noise. Similar to [2], [37], [38], the quality of restoration is measured by using their mean squared error (MSE) defined by

$$MSE=\frac{1}{n}{\Vert \overline{x}-x\Vert }^2,$$ (36)

where $\overline{x}$ is the original signal and x is the restored signal. The parameters for HLSFR were set as follows: $\xi =0.8$, $\beta =0.9$ and $\gamma ={10}^{-4}$.

The goal of our experiment is to recover a sparse signal of length n from k observations with $k<<n$. Due to the capacity restrictions of the PC, we select a small size signal with signal length of 1029 and sampling measurement of 512. The original signal $\overline{x}$ contains 128 randomly non-zero elements. Furthermore, during experiment, a random Gaussian matrix A using the Matlab command $randn(k,n)$ is generated. In the test, the measurement b is computed by

$$b=A\overline{x}+\delta$$

where δ is the Gaussian noise distributed as $N(0,{10}^{-4})$.

To evaluate the performance of HLSFR, we test it against similar algorithms which were specially designed to solve monotone nonlinear equations with convex constraints and reconstructing sparse signal in compressive sensing. These algorithms include: CGD [2], PCG [24] and IPBDF [39]. For fairness in comparing the algorithms, iteration process of all algorithms started at $x_0=A^{T}b$ and terminated when

$$Tol=\frac{\Vert f_k-f_{k-1}\Vert }{\Vert f_{k-1}\Vert }<{10}^{-5},$$

where $f(x)=\frac{1}{2}{\Vert Ax-b\Vert }_2^2+\tau {\Vert x\Vert }_1$ is the objective function and $f_k$ denotes the function value at $x_k$. See Fig. 4 for the numerical results consisting of the original sparse signal, the measurement and the reconstructed signal by each algorithm. Moreover, in Fig. 5, we give a visual illustration of the performance of each method relative to their convergence behavior from the view of merit function values and relative error as the iteration numbers and computing time increases.

Figure 4.

Reconstruction of sparse signal. From the top to the bottom is the original signal (First plot), the measurement (Second plot), and the reconstructed signals by CGD (Third plot), PCG (Fourth plot) and HLSFR (Fifth plot).

Figure 5.

Comparison results of HLSFR, CGD, PCG and IPDBF algorithm. The x-axes represent the number of iterations (top left and bottom left) and the CPU time in seconds (top right and bottom right). The y-axes represent the MSE (top left and top right) and the function values (bottom left and right).

Comparing the four algorithms in Fig. 4, it is not difficult to see that the original signal was recovered by the four algorithms. However, HLSFR won in decoding sparse signal in compressive sensing. This is reflected by its lesser number of iterations, computing time and lesser MSE. To further illustrate the efficiency HLSFR, we repeated the experiment on 10 different noise samples. Each time the experiment is run, HLSFR proves to be more efficient than the CGD, PCG and IPDBF in terms of iteration numbers and CPU time and most importantly, MSE. See summary in Table 1.

Table 1.

The experimental results of compressed sensing problem via CGD, PCG, IPDBF and HLSFR method.

	CGD			PCG			IPBDF			HLSFR
	CPU	ITER	MSE	CPU	ITER	MSE	CPU	ITER	MSE	CPU	ITER	MSE
	4.48	557	1.36E-03	9.30	164	3.45E-03	9.30	1189	2.85E-03	1.27	155	1.18E-03
	2.00	211	2.63E-03	9.27	179	1.44E-03	9.27	1186	1.32E-03	1.34	165	3.71E-04
	4.50	584	1.18E-03	9.91	175	3.33E-03	9.91	1305	2.29E-03	1.53	189	1.02E-03
	3.36	429	9.00E-04	9.28	197	1.31E-03	9.28	1202	1.35E-03	1.34	169	4.85E-04
	3.63	467	1.16E-03	9.81	267	1.51E-03	9.81	1329	1.76E-03	1.39	171	7.29E-04
	4.22	529	9.89E-04	10.36	156	3.01E-03	10.36	1287	1.72E-03	1.39	169	6.07E-04
	3.55	462	1.43E-03	8.11	217	1.94E-03	8.11	1153	2.47E-03	1.03	166	1.27E-03
	4.17	501	1.36E-03	9.44	164	2.69E-03	9.44	1204	2.20E-03	1.17	181	1.04E-03
	1.92	221	4.18E-03	9.03	169	3.31E-03	9.03	1113	3.01E-03	3.31	182	1.39E-03
	1.91	248	2.92E-03	10.72	212	1.86E-03	10.27	1429	1.68E-03	1.61	201	4.13E-04
Avg	3.37	420.9	1.81E-03	9.52	190	2.38E-03	9.48	1240	2.07E-03	1.54	174.8	8.52E-04

Next, we illustrate the performance of HLSFR algorithm in image restoration. In this experiment, a matrix A (partial DWT matrix) whose k rows are randomly selected from the $n\times n$ DWT matrix. This type of matrix A requires no storage and helps in speeding up the matrix-vector multiplications involving A and $A^T$. The test images we considered are personal images with color which were taken with a digital camera. These images include: TP1, TP2, TP3 and TP4. All test images are of the size $800\times 800$ except for TP1 which is $720\times 720$.

The quality of image restoration is determined by signal-to-ratio (SNR) and the peak signal-to-noise ratios (PSNR). For the image restoration experiment, the chosen parameters for HLSFR are $\xi =0.05,\gamma ={10}^{-4}$. See Fig. 6 for the original, blurred, and restored images by each algorithm.

Figure 6.

Restoration of TP1 (Top), TP2 (Top-middle), TP3 (Bottom-middle) and TP4 (Bottom). From the left is the blurred with noise image, followed by the reconstructed images by CGD, IPDBF and HLSFR.

Furthermore, five Gaussian blur kernel were utilized in testing the efficiency of the methods. Their numerical performance is reported in the table that follows where $TPi(\sigma )$ denotes that the test problem i which is solved by a Gaussian blur kernel with standard deviation σ.

From Table 2, it can be observed that, under the five Gaussian blue kernel, the quality of the restored images by HLSFR is much better than that of CGD, IPBDF. This is reflected by smaller value of the ObjFunc and MSE. Similarly, larger SNR, SSIM and PSNR indicate that the restored images from the blurred images by HLSFR are much more closer to the original one than the recovered ones by CGD and IPDBF in most cases. The MATLAB implementation of the SSIM index can be obtained at http://www.cns.nyu.edu/~lcv/ssim/.

Table 2.

Efficiency Comparison for restoration between CGD, IPDBF and HLSFR under different Gaussian blur Kernels.

IMAGE	CGD						IPDBF						HLSFR
	ObjFun	MSE	SNR	SSIM	PSNR	TIME	ObjFun	MSE	SNR	SSIM	PSNR	TIME	ObjFun	MSE	SNR	SSIM	PSNR	TIME
TP1(4)	1.01E+06	1.60E+05	-0.99	0.02	5.14	3.53	1.01E+06	1.15E+05	0.38	0.02	6.51	5.89	1.01E+06	1.11E+05	0.55	0.02	6.68	26.03
TP1(1)	2.66E+05	2.26E+04	7.46	0.12	13.59	2.81	2.65E+05	1.81E+04	8.42	0.14	14.55	4.67	2.65E+05	1.90E+04	8.2	0.13	14.34	25.94
TP1(0.1)	1.18E+04	2.65E+03	16.71	0.77	22.84	4.72	1.18E+04	2.59E+03	16.81	0.78	22.95	16.14	1.18E+04	2.58E+03	16.83	0.78	22.97	68.36
TP1(0.25)	7.33E+04	6.53E+03	12.87	0.35	19.01	3.86	7.33E+04	5.77E+03	13.41	0.39	19.55	9.56	7.33E+04	5.64E+03	13.52	0.39	19.65	42.81
TP1(6.25)	1.57E+06	3.36E+05	-4.15	0.01	1.98	3.58	1.57E+06	2.62E+05	-3.1	0.01	3.03	8.14	1.57E+06	2.56E+05	-2.98	0.01	3.15	39.63
TP2(4)	1.25E+06	2.06E+05	-1.58	0.11	5	3.97	1.25E+06	1.42E+05	-0.09	0.13	6.49	6.41	1.25E+06	1.39E+05	0.03	0.13	6.61	30.05
TP2(1)	3.24E+05	2.87E+04	6.82	0.35	13.4	3.23	3.24E+05	2.35E+04	7.7	0.38	14.28	5.44	3.23E+05	2.46E+04	7.5	0.37	14.08	30.05
TP2(0.1)	1.25E+04	5.05E+03	14.24	0.81	20.82	5.58	1.25E+04	4.93E+03	14.37	0.82	20.95	15.17	1.25E+04	4.91E+03	14.4	0.82	20.98	67.03
TP2(0.25)	8.81E+04	9.57E+03	11.53	0.56	18.11	4.67	8.81E+04	8.52E+03	12.03	0.58	18.61	11.61	8.81E+04	8.44E+03	12.08	0.59	18.66	48.84
TP2(6.25)	1.93E+06	4.47E+05	-4.83	0.06	1.75	4.28	1.93E+06	3.41E+05	-3.69	0.06	2.89	11	1.93E+06	3.31E+05	-3.59	0.06	2.99	54.53
TP3(4)	1.26E+06	1.99E+05	1.08	0.06	5.24	4.05	1.26E+06	1.42E+05	2.42	0.08	6.58	6.34	1.26E+06	1.38E+05	2.5	0.08	6.66	30.55
TP3(1)	3.30E+05	2.80E+04	9.37	0.25	13.52	3.38	3.30E+05	2.17E+04	10.44	0.28	14.59	5.14	3.28E+05	2.36E+04	10.12	0.28	14.28	31.22
TP3(0.1)	1.64E+04	3.46E+03	18.46	0.78	22.62	5.81	1.64E+04	3.36E+03	18.59	0.79	22.74	14.64	1.64E+04	3.34E+03	18.62	0.79	22.77	67.38
TP3(0.25)	9.20E+04	8.10E+03	14.72	0.46	18.87	4.66	9.19E+04	7.21E+03	15.29	0.48	19.44	11.39	9.20E+04	7.03E+03	15.36	0.49	19.52	49.52
TP3(6.25)	1.93E+06	4.72E+05	-2.74	0.03	1.42	4.95	1.93E+06	3.81E+05	-1.86	0.03	2.3	12.11	1.93E+06	3.74E+05	-1.75	0.03	2.4	57.45
TP4(4)	1.26E+06	2.00E+05	-0.19	0.05	5.1	4.31	1.25E+06	1.49E+05	1.14	0.06	6.42	7.42	1.26E+06	1.44E+05	1.3	0.06	6.58	32.02
TP4(1)	3.30E+05	3.25E+04	7.8	0.19	13.08	4.17	3.30E+05	2.65E+04	8.71	0.21	13.99	5.16	3.29E+05	2.84E+04	8.41	0.21	13.69	36.25
TP4(0.1)	1.71E+04	1.02E+04	13.24	0.63	18.52	4.16	1.70E+04	9.99E+03	13.33	0.64	18.61	9.63	1.71E+04	9.88E+03	13.36	0.64	18.65	42.03
TP4(0.25)	9.33E+04	1.41E+04	11.67	0.37	16.96	4.64	9.32E+04	1.31E+04	12.03	0.39	17.31	10.11	9.33E+04	1.29E+04	12.08	0.39	17.36	38.2
TP4(6.25)	1.93E+06	4.72E+05	-3.76	0.02	1.52	4.53	1.93E+06	3.85E+05	-2.83	0.03	2.45	11.52	1.93E+06	3.75E+05	-2.72	0.03	2.56	55.34

6. Conclusions

We have presented a hybrid conjugate gradient projection method for solving convex constrained nonlinear equations. The algorithm is a convex combination of two conjugate gradient algorithm for solving unconstrained optimization problem [15], [19]. Under some appropriate conditions, the global convergence of the method is established. Results from numerical experiment show that our method is practical, effective and out performs the CGD, PCG, PDY and ACGD for some given convex constraint benchmark test problems with dimension ranging from 5000 to 100,000 and different initial points. Furthermore, one major contribution of this article is the utilization of the proposed algorithm in solving the ${\ell }_1$-norm regularized problem in compressive sensing. Computational results from reconstructing sparse signal and blurred images have shown that the proposed method is competitive with the compared ones.

Declarations

Author contribution statement

Abdulkarim Hassan Ibrahim: Conceived and designed the experiments; Wrote the paper.

Poom Kumam: Contributed reagents, materials, analysis tools or data.

Auwal Bala Abubakar: Performed the experiments; Wrote the paper.

Wachirapong Jirakitpuwapat: Performed the experiments.

Jamilu Abubakar: Analyzed and interpreted the data; Wrote the paper.

Funding statement

This work was supported by Theoretical and Computational Science (TaCS) Center under Computational and Applied Science for Smart research Innovation Cluster (CLASSIC), Faculty of Science, KMUTT. Abdulkarim Hassan Ibrahim was supported by the Petchra Pra Jom Klao Doctoral Scholarship, Academic for Ph.D. Program at KMUTT (Grant No. 16/2561).

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Appendix A.

Table 3.

Numerical results for Problem 1.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	1	3	1.0823	0.00E+00	50	149	0.04673	9.11E-07	19	75	0.092184	4.72E-06	18	72	0.58052	4.26E-07	17	67	0.005714	4.53E-06
	x2	1	3	0.012591	0.00E+00	42	125	0.021608	9.97E-07	18	71	0.035649	5.72E-06	16	64	0.027927	3.45E-07	8	31	0.006222	9.26E-06
	x3	2	6	0.00375	6.28E-16	58	173	0.022573	8.22E-07	30	120	0.04483	8.59E-06	17	68	0.033385	4.70E-07	18	72	0.010303	5.80E-06
	x4	7	21	0.022676	4.00E-07	52	155	0.042289	8.48E-07	19	75	0.013987	5.82E-06	17	68	0.022742	7.21E-07	17	67	0.007245	4.37E-06
	x5	7	21	0.047679	5.23E-07	35	104	0.01894	6.10E-07	18	71	0.022429	9.14E-06	17	68	0.079305	7.15E-07	16	63	0.009693	9.82E-06
	x6	2	6	0.002211	0.00E+00	59	176	0.019966	9.55E-07	34	135	0.027208	7.37E-06	17	68	0.018899	6.95E-07	9	36	0.004608	3.90E-06
	x7	7	21	0.010068	4.00E-07	52	155	0.020642	8.48E-07	19	75	0.015801	5.82E-06	17	68	0.017112	7.21E-07	17	67	0.011732	4.37E-06
	x8	7	21	0.009221	6.60E-07	35	104	0.017981	6.11E-07	18	71	0.014044	9.15E-06	17	68	0.01607	7.16E-07	16	63	0.011277	9.83E-06
	x9	7	21	0.007807	2.16E-07	51	152	0.067866	8.00E-07	18	71	0.007959	9.75E-06	17	68	0.016342	7.32E-07	16	63	0.009217	9.79E-06
5000	x1	1	3	0.004128	0.00E+00	48	143	0.35099	9.34E-07	20	79	0.037975	4.70E-06	18	72	0.091098	9.40E-07	18	71	0.034532	4.03E-06
	x2	1	3	0.063436	0.00E+00	41	122	0.50749	8.34E-07	18	71	0.10733	7.42E-06	16	64	0.051997	7.61E-07	9	36	0.021112	3.89E-06
	x3	2	6	0.012812	6.28E-16	58	173	0.061541	8.22E-07	30	120	0.070734	8.59E-06	17	68	0.061095	4.70E-07	18	72	0.044432	5.80E-06
	x4	8	24	0.016178	5.00E-08	50	149	0.060155	8.63E-07	19	75	0.030924	9.92E-06	18	72	0.068399	5.36E-07	17	67	0.029379	8.99E-06
	x5	8	24	0.021606	5.21E-08	36	107	0.042146	8.19E-07	19	75	0.03437	9.15E-06	18	72	0.062534	5.35E-07	17	67	0.031705	8.79E-06
	x6	2	6	0.006381	0.00E+00	59	176	0.073312	9.55E-07	34	135	0.066134	7.37E-06	17	68	0.05826	6.95E-07	9	36	0.18994	3.90E-06
	x7	8	24	0.023883	5.00E-08	50	149	0.072813	8.63E-07	19	75	0.039391	9.92E-06	18	72	0.064131	5.36E-07	17	67	0.035233	8.99E-06
	x8	8	24	0.025457	5.63E-08	36	107	0.075877	8.19E-07	19	75	0.041779	9.15E-06	18	72	0.058588	5.35E-07	17	67	0.035003	8.80E-06
	x9	7	21	0.025834	5.30E-08	49	146	0.072482	9.52E-07	19	75	0.034482	9.66E-06	18	72	0.062149	5.35E-07	17	67	0.030316	8.81E-06
10000	x1	1	3	0.016629	0.00E+00	47	140	0.10544	9.79E-07	20	79	0.055695	6.64E-06	19	76	0.12181	4.44E-07	18	71	0.056367	5.70E-06
	x2	1	3	0.031943	0.00E+00	40	119	0.080776	8.97E-07	18	71	0.04976	9.50E-06	17	68	0.11546	3.55E-07	9	35	0.051461	5.50E-06
	x3	2	6	0.089531	6.28E-16	58	173	0.1461	8.22E-07	30	120	0.11683	8.59E-06	17	68	0.12369	4.70E-07	18	72	0.058851	5.80E-06
	x4	8	24	0.042885	7.27E-08	49	146	0.1106	8.99E-07	20	79	0.09813	6.17E-06	18	72	0.11517	7.57E-07	18	71	0.058665	5.06E-06
	x5	8	24	0.057914	7.41E-08	37	110	0.072591	6.95E-07	20	79	0.06743	5.79E-06	18	72	0.10486	7.56E-07	18	71	0.055802	4.98E-06
	x6	2	6	0.008709	0.00E+00	59	176	0.14788	9.55E-07	34	135	0.10822	7.37E-06	17	68	0.11873	6.95E-07	9	36	0.044591	3.90E-06
	x7	8	24	0.047393	7.27E-08	49	146	0.10977	8.99E-07	20	79	0.057499	6.17E-06	18	72	0.098904	7.57E-07	18	71	0.055068	5.06E-06
	x8	8	24	0.038672	7.72E-08	37	110	0.076211	6.95E-07	20	79	0.063101	5.79E-06	18	72	0.12217	7.57E-07	18	71	0.060447	4.98E-06
	x9	8	24	0.052426	1.55E-07	47	140	0.20135	8.12E-07	20	79	0.072131	6.16E-06	18	72	0.14623	7.59E-07	18	71	0.061031	4.97E-06
50000	x1	1	3	0.027125	0.00E+00	46	137	0.5439	8.30E-07	21	83	0.2535	6.64E-06	20	80	0.43976	8.84E-07	19	75	0.23021	5.10E-06
	x2	1	3	0.023587	0.00E+00	39	116	0.41363	8.43E-07	19	75	0.23779	8.80E-06	17	68	0.33496	7.93E-07	10	40	0.11264	2.33E-06
	x3	2	6	0.044199	6.28E-16	58	173	0.49551	8.22E-07	30	120	0.46619	8.59E-06	17	68	0.40581	4.70E-07	18	72	0.16705	5.80E-06
	x4	8	24	0.11349	1.68E-07	47	140	0.43289	9.33E-07	21	83	0.261	5.91E-06	19	76	0.41494	5.63E-07	19	76	0.21902	4.48E-06
	x5	8	24	0.091825	1.68E-07	38	113	0.31557	9.33E-07	21	83	0.19767	5.79E-06	19	76	0.38138	5.62E-07	19	76	0.19472	4.46E-06
	x6	2	6	0.038903	0.00E+00	59	176	0.46931	9.55E-07	34	135	0.34305	7.37E-06	17	68	0.33145	6.95E-07	9	36	0.088923	3.90E-06
	x7	8	24	0.11501	1.68E-07	47	140	0.69908	9.33E-07	21	83	0.24347	5.91E-06	19	76	0.40886	5.63E-07	19	76	0.18883	4.48E-06
	x8	8	24	0.21179	1.70E-07	38	113	0.59588	9.33E-07	21	83	0.20193	5.79E-06	19	76	0.40335	5.63E-07	19	76	0.18902	4.46E-06
	x9	8	24	0.11512	4.27E-08	39	116	0.3637	6.65E-07	21	83	0.28083	5.79E-06	19	76	0.52614	5.65E-07	19	76	0.19398	4.46E-06
100000	x1	1	3	0.034912	0.00E+00	45	134	0.77402	9.06E-07	21	83	0.42087	9.39E-06	21	84	0.8809	6.05E-07	19	75	0.46145	7.21E-06
	x2	1	3	0.042479	0.00E+00	39	116	0.59479	7.72E-07	20	79	0.36671	5.52E-06	18	72	0.67054	3.76E-07	10	40	0.24162	3.29E-06
	x3	2	6	0.059236	6.28E-16	58	173	0.76354	8.22E-07	30	120	0.65474	8.59E-06	17	68	0.62133	4.70E-07	18	72	0.43411	5.80E-06
	x4	8	24	0.20832	2.38E-07	46	137	0.64004	9.92E-07	21	83	0.37783	8.27E-06	20	80	0.92749	7.77E-07	19	76	0.37867	6.32E-06
	x5	8	24	0.23556	2.38E-07	39	116	0.65969	7.91E-07	21	83	0.41069	8.19E-06	20	80	0.79866	7.77E-07	19	76	0.36352	6.30E-06
	x6	2	6	0.06482	0.00E+00	59	176	0.80171	9.55E-07	34	135	0.61319	7.37E-06	17	68	0.96643	6.95E-07	9	36	0.21668	3.90E-06
	x7	8	24	0.19377	2.38E-07	46	137	0.71526	9.92E-07	21	83	0.39291	8.27E-06	20	80	0.98442	7.77E-07	19	76	0.41014	6.32E-06
	x8	8	24	0.19487	2.39E-07	39	116	0.57312	7.91E-07	21	83	0.39435	8.19E-06	20	80	0.78821	7.77E-07	19	76	0.39413	6.30E-06
	x9	8	24	0.30738	5.93E-08	41	122	0.71712	9.52E-07	21	83	0.43257	8.21E-06	20	80	0.86416	7.79E-07	19	76	0.42681	6.30E-06

Table 4.

Numerical results for Problem 2.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	8	23	0.018212	2.80E-07	75	223	0.80072	8.04E-07	16	60	0.024105	8.69E-06	17	67	0.014428	7.80E-07	5	14	0.003424	3.60E-08
	x2	7	20	0.010173	7.80E-07	55	163	0.054278	8.99E-07	15	58	0.008816	8.59E-06	13	51	0.014789	7.68E-07	3	8	0.004621	5.17E-07
	x3	14	41	0.026389	8.16E-07	56	166	0.049743	8.94E-07	16	62	0.023068	5.23E-06	14	55	0.008207	3.79E-07	427	1706	0.28339	9.99E-06
	x4	17	50	0.020437	6.68E-07	72	214	0.036264	8.60E-07	19	73	0.015537	6.23E-06	17	67	0.015355	4.48E-07	47	184	0.039464	9.78E-06
	x5	17	50	0.050271	6.68E-07	72	214	0.029064	8.60E-07	19	73	0.013726	6.23E-06	17	67	0.022895	4.48E-07	47	184	0.030139	9.78E-06
	x6	15	43	0.023681	9.98E-07	61	181	0.054219	9.60E-07	18	69	0.013178	6.33E-06	15	59	0.013177	4.21E-07	13	49	0.010699	4.00E-06
	x7	17	50	0.025723	6.68E-07	72	214	0.041652	8.60E-07	19	73	0.01093	6.23E-06	17	67	0.013001	4.48E-07	47	184	0.026531	9.78E-06
	x8	17	50	0.025859	6.68E-07	72	214	0.073851	8.61E-07	19	73	0.011677	6.25E-06	17	67	0.014163	4.48E-07	47	184	0.023996	9.33E-06
	x9	17	50	0.039069	6.39E-07	72	214	0.065443	8.49E-07	19	73	0.011878	6.03E-06	17	67	0.022129	4.48E-07	47	184	0.028836	9.51E-06
5000	x1	8	23	0.032659	6.77E-07	78	232	0.24516	9.05E-07	17	64	0.055484	9.24E-06	18	71	0.069709	5.71E-07	5	14	0.012506	6.26E-09
	x2	8	22	0.037024	7.45E-07	59	175	0.084645	8.02E-07	16	62	0.037537	9.35E-06	14	55	0.042903	5.44E-07	3	8	0.014479	1.75E-07
	x3	14	41	0.050599	8.08E-07	56	166	0.13993	8.87E-07	16	62	0.037155	5.17E-06	14	55	0.040223	3.76E-07	27	106	0.054074	9.55E-06
	x4	18	52	0.050527	9.45E-07	75	223	0.26478	9.71E-07	20	77	0.056123	6.86E-06	17	67	0.075953	9.87E-07	12	44	0.032188	2.83E-06
	x5	18	52	0.14547	9.45E-07	75	223	0.1264	9.71E-07	20	77	0.04529	6.86E-06	17	67	0.064816	9.87E-07	12	44	0.037678	2.83E-06
	x6	15	43	0.050711	9.88E-07	61	181	0.090101	9.52E-07	18	69	0.048865	6.41E-06	15	59	0.052666	4.20E-07	17	65	0.039516	7.74E-06
	x7	18	52	0.069951	9.45E-07	75	223	0.12312	9.71E-07	20	77	0.054758	6.86E-06	17	67	0.078804	9.87E-07	12	44	0.035473	2.83E-06
	x8	18	52	0.11842	9.45E-07	75	223	0.22357	9.72E-07	20	77	0.051195	6.87E-06	17	67	0.052195	9.87E-07	12	44	0.034693	2.83E-06
	x9	18	52	0.10997	9.56E-07	75	223	0.34172	9.71E-07	20	77	0.060697	6.92E-06	17	67	0.05724	9.88E-07	12	44	0.037483	2.82E-06
10000	x1	8	23	0.041183	9.66E-07	80	238	0.26206	8.17E-07	18	68	0.092888	6.50E-06	18	71	0.099253	8.06E-07	5	14	0.020898	3.62E-09
	x2	8	23	0.083979	2.12E-07	60	178	0.16174	9.04E-07	17	66	0.070972	6.60E-06	14	55	0.084051	7.66E-07	3	8	0.015687	1.21E-07
	x3	14	41	0.085061	8.08E-07	56	166	0.13119	8.86E-07	16	62	0.05767	5.17E-06	14	55	0.075646	3.76E-07	35	138	0.13043	9.91E-06
	x4	18	53	0.20063	7.34E-07	77	229	0.22668	8.78E-07	20	77	0.098276	9.69E-06	18	71	0.099754	4.64E-07	12	44	0.052358	3.89E-06
	x5	18	53	0.27686	7.34E-07	77	229	0.2324	8.78E-07	20	77	0.093564	9.69E-06	18	71	0.10881	4.64E-07	12	44	0.064254	3.89E-06
	x6	15	43	0.09867	9.86E-07	61	181	0.16297	9.51E-07	18	69	0.067092	6.42E-06	15	59	0.080194	4.20E-07	17	65	0.087748	8.10E-06
	x7	18	53	0.11891	7.34E-07	77	229	0.34381	8.78E-07	20	77	0.078502	9.69E-06	18	71	0.098208	4.64E-07	12	44	0.054981	3.89E-06
	x8	18	53	0.12076	7.34E-07	77	229	0.36919	8.78E-07	20	77	0.081604	9.69E-06	18	71	0.13537	4.64E-07	12	44	0.074371	3.89E-06
	x9	18	53	0.15044	7.39E-07	77	229	0.35996	8.76E-07	20	77	0.13768	9.68E-06	18	71	0.11891	4.62E-07	12	44	0.057607	3.86E-06
50000	x1	9	25	0.15309	8.71E-07	83	247	1.0802	9.34E-07	19	72	0.3017	7.24E-06	20	80	0.42989	7.70E-07	6	19	0.080077	4.49E-06
	x2	8	23	0.16512	4.78E-07	64	190	0.77041	8.26E-07	18	70	0.29234	7.37E-06	15	59	0.32333	5.78E-07	7	25	0.10219	2.94E-06
	x3	14	41	0.31406	8.07E-07	56	166	0.67991	8.85E-07	16	62	0.28346	5.16E-06	14	55	0.23355	3.75E-07	36	142	0.62329	8.88E-06
	x4	19	56	0.43663	5.77E-07	81	241	1.062	8.03E-07	22	85	0.44496	5.43E-06	19	75	0.39463	3.46E-07	13	47	0.19753	7.28E-06
	x5	19	56	0.38161	5.77E-07	81	241	1.1367	8.03E-07	22	85	0.34541	5.43E-06	19	75	0.48171	3.46E-07	13	47	0.20861	7.28E-06
	x6	15	43	0.30917	9.85E-07	61	181	0.64703	9.50E-07	18	69	0.26254	6.44E-06	15	59	0.27749	4.20E-07	17	65	0.21671	8.33E-06
	x7	19	56	0.3637	5.77E-07	81	241	1.0176	8.03E-07	22	85	0.34772	5.43E-06	19	75	0.38515	3.46E-07	13	47	0.22692	7.28E-06
	x8	19	56	0.36649	5.77E-07	81	241	1.0743	8.03E-07	22	85	0.44399	5.43E-06	19	75	0.46512	3.46E-07	13	47	0.19916	7.28E-06
	x9	19	56	0.63323	5.78E-07	81	241	1.4438	8.06E-07	22	85	0.4979	5.40E-06	19	75	0.38699	3.46E-07	13	47	0.25878	7.27E-06
100000	x1	9	26	0.35624	2.47E-07	85	253	1.9715	8.45E-07	20	76	0.57694	5.13E-06	22	88	1.0154	6.15E-07	6	19	0.15741	6.18E-06
	x2	8	23	0.25119	6.76E-07	65	193	1.4595	9.34E-07	19	74	0.62487	5.22E-06	15	59	0.90318	8.17E-07	7	25	0.24042	4.14E-06
	x3	14	41	0.38415	8.07E-07	56	166	1.1403	8.85E-07	16	62	0.36519	5.16E-06	14	55	0.48722	3.75E-07	36	142	1.1879	9.08E-06
	x4	19	56	0.85542	8.16E-07	82	244	2.0197	9.08E-07	22	85	0.69632	7.67E-06	20	80	1.0938	5.47E-07	13	47	0.42213	8.40E-06
	x5	19	56	0.81838	8.16E-07	82	244	2.2071	9.08E-07	22	85	0.79312	7.67E-06	20	80	0.97473	5.47E-07	13	47	0.41177	8.40E-06
	x6	15	43	0.57124	9.85E-07	61	181	1.3264	9.50E-07	18	69	0.55801	6.44E-06	15	59	0.53508	4.20E-07	17	65	0.51383	8.35E-06
	x7	19	56	0.76892	8.16E-07	82	244	2.1547	9.08E-07	22	85	0.65754	7.67E-06	20	80	0.91264	5.47E-07	13	47	0.39895	8.40E-06
	x8	19	56	0.67393	8.16E-07	82	244	2.1183	9.08E-07	22	85	0.64507	7.67E-06	20	80	0.98593	5.47E-07	13	47	0.41949	8.40E-06
	x9	19	56	0.96709	8.17E-07	82	244	2.5937	9.07E-07	22	85	0.81881	7.66E-06	20	80	0.97391	5.48E-07	13	47	0.53617	8.35E-06

Table 5.

Numerical results for Problem 3.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	14	41	0.0183	4.81E-07	78	233	0.82269	8.87E-07	22	87	0.011525	5.34E-06	17	68	0.011873	7.24E-07	11	43	0.008686	6.30E-06
	x2	12	36	0.023188	7.31E-07	68	203	0.05126	9.14E-07	19	75	0.007828	5.62E-06	15	60	0.013488	4.96E-07	10	39	0.004849	4.44E-06
	x3	F	F	F	F	60	179	0.029411	9.74E-07	16	63	0.008361	7.72E-06	13	52	0.013733	8.83E-07	9	35	0.009361	2.85E-06
	x4	18	54	0.030502	8.81E-07	76	227	0.031992	8.32E-07	21	83	0.013737	6.92E-06	17	68	0.00999	3.88E-07	13	51	0.006173	2.84E-06
	x5	18	54	0.015063	8.81E-07	76	227	0.074951	8.32E-07	21	83	0.012266	6.92E-06	17	68	0.024643	3.88E-07	13	51	0.013775	2.84E-06
	x6	F	F	F	F	64	191	0.04589	8.46E-07	17	67	0.00594	7.68E-06	14	56	0.014141	7.41E-07	F	F	F	F
	x7	18	54	0.030096	8.81E-07	76	227	0.031962	8.32E-07	21	83	0.01521	6.92E-06	17	68	0.013337	3.88E-07	13	51	0.006512	2.84E-06
	x8	18	54	0.033431	8.80E-07	76	227	0.093709	8.33E-07	21	83	0.010497	6.93E-06	17	68	0.012659	3.88E-07	13	51	0.008327	2.85E-06
	x9	18	54	0.020059	8.90E-07	76	227	0.064367	8.34E-07	21	83	0.011911	7.04E-06	17	68	0.013935	3.87E-07	13	51	0.011743	2.79E-06
5000	x1	14	42	0.043669	7.53E-07	82	245	0.20528	8.13E-07	23	91	0.060791	5.98E-06	18	72	0.049102	5.41E-07	12	47	0.026256	3.67E-06
	x2	13	38	0.038964	6.54E-07	72	215	0.169	8.37E-07	20	79	0.038254	6.29E-06	16	64	0.051745	3.74E-07	10	39	0.02449	9.93E-06
	x3	F	F	F	F	60	179	0.092302	9.74E-07	16	63	0.033785	7.72E-06	13	52	0.036974	8.83E-07	9	35	0.024014	2.85E-06
	x4	19	57	0.068222	6.93E-07	79	236	0.13927	9.53E-07	22	87	0.059257	7.76E-06	17	68	0.046376	8.68E-07	13	51	0.029617	6.30E-06
	x5	19	57	0.048611	6.93E-07	79	236	0.14781	9.53E-07	22	87	0.052782	7.76E-06	17	68	0.059893	8.68E-07	13	51	0.027401	6.30E-06
	x6	F	F	F	F	64	191	0.10051	8.46E-07	17	67	0.03014	7.68E-06	14	56	0.04967	7.41E-07	F	F	F	F
	x7	19	57	0.070133	6.93E-07	79	236	0.12815	9.53E-07	22	87	0.046812	7.76E-06	17	68	0.047516	8.68E-07	13	51	0.028182	6.30E-06
	x8	19	57	0.14375	6.93E-07	79	236	0.12428	9.53E-07	22	87	0.064462	7.76E-06	17	68	0.10568	8.68E-07	13	51	0.031773	6.30E-06
	x9	19	57	0.084187	7.01E-07	79	236	0.15892	9.59E-07	22	87	0.064349	7.71E-06	17	68	0.067408	8.70E-07	13	51	0.032564	6.26E-06
10000	x1	15	44	0.13781	4.26E-07	83	248	0.22246	9.20E-07	23	91	0.080213	8.46E-06	18	72	0.093398	7.65E-07	12	47	0.045959	5.18E-06
	x2	13	38	0.061573	9.25E-07	73	218	0.26335	9.47E-07	20	79	0.068726	8.90E-06	16	64	0.070683	5.28E-07	11	43	0.035235	3.65E-06
	x3	F	F	F	F	60	179	0.20574	9.74E-07	16	63	0.086356	7.72E-06	13	52	0.059207	8.83E-07	9	35	0.031756	2.85E-06
	x4	19	57	0.12639	9.80E-07	81	242	0.30108	8.62E-07	23	91	0.083338	5.50E-06	18	72	0.091757	4.11E-07	13	51	0.050319	8.89E-06
	x5	19	57	0.10872	9.80E-07	81	242	0.21114	8.62E-07	23	91	0.074474	5.50E-06	18	72	0.082871	4.11E-07	13	51	0.046841	8.89E-06
	x6	F	F	F	F	64	191	0.16324	8.46E-07	17	67	0.14347	7.68E-06	14	56	0.068477	7.41E-07	F	F	F	F
	x7	19	57	0.11703	9.80E-07	81	242	0.26074	8.62E-07	23	91	0.075044	5.50E-06	18	72	0.089915	4.11E-07	13	51	0.053132	8.89E-06
	x8	19	57	0.19919	9.80E-07	81	242	0.19701	8.63E-07	23	91	0.072729	5.50E-06	18	72	0.088977	4.11E-07	13	51	0.051074	8.90E-06
	x9	19	57	0.13663	9.88E-07	81	242	0.26997	8.62E-07	23	91	0.094142	5.48E-06	18	72	0.14686	4.14E-07	13	51	0.067577	8.93E-06
50000	x1	15	44	0.23977	9.52E-07	87	260	1.8282	8.42E-07	24	95	0.34206	9.48E-06	20	80	0.30151	5.51E-07	13	51	0.14639	3.01E-06
	x2	14	41	0.1817	5.79E-07	77	230	1.3441	8.67E-07	21	83	0.23777	9.97E-06	17	68	0.24967	3.91E-07	11	43	0.17692	8.17E-06
	x3	F	F	F	F	60	179	1.2927	9.74E-07	16	63	0.18235	7.72E-06	13	52	0.20767	8.83E-07	9	35	0.10176	2.85E-06
	x4	20	60	0.2897	7.71E-07	84	251	1.231	9.87E-07	24	95	0.33937	6.16E-06	18	72	0.29045	9.19E-07	14	55	0.19307	5.17E-06
	x5	20	60	0.2563	7.71E-07	84	251	0.98366	9.87E-07	24	95	0.32036	6.16E-06	18	72	0.37354	9.19E-07	14	55	0.19138	5.17E-06
	x6	F	F	F	F	64	191	1.4508	8.46E-07	17	67	0.23677	7.68E-06	14	56	0.20764	7.41E-07	F	F	F	F
	x7	20	60	0.24518	7.71E-07	84	251	1.1827	9.87E-07	24	95	0.3051	6.16E-06	18	72	0.34183	9.19E-07	14	55	0.19257	5.17E-06
	x8	20	60	0.28419	7.71E-07	84	251	1.0601	9.87E-07	24	95	0.32944	6.16E-06	18	72	0.26908	9.19E-07	14	55	0.19161	5.17E-06
	x9	20	60	0.32687	7.73E-07	84	251	1.5334	9.87E-07	24	95	0.47879	6.15E-06	18	72	0.34374	9.17E-07	14	55	0.25619	5.16E-06
100000	x1	15	45	0.3195	9.43E-07	88	263	3.1639	9.53E-07	25	99	0.68617	6.72E-06	21	84	0.64426	4.91E-07	13	51	0.29855	4.26E-06
	x2	14	41	0.34837	8.19E-07	78	233	1.96	9.81E-07	22	87	0.5382	7.06E-06	17	68	0.52521	5.53E-07	12	47	0.24981	3.00E-06
	x3	F	F	F	F	60	179	1.5726	9.74E-07	16	63	0.46066	7.72E-06	13	52	0.38743	8.83E-07	9	35	0.25983	2.85E-06
	x4	21	62	0.41639	6.98E-07	86	257	2.9241	8.94E-07	24	95	0.59391	8.71E-06	20	80	0.77236	4.62E-07	14	55	0.35879	7.30E-06
	x5	21	62	0.54923	6.98E-07	86	257	2.1442	8.94E-07	24	95	0.53777	8.71E-06	20	80	0.59394	4.62E-07	14	55	0.29892	7.30E-06
	x6	F	F	F	F	64	191	1.9524	8.46E-07	17	67	0.39127	7.68E-06	14	56	0.39452	7.41E-07	F	F	F	F
	x7	21	62	0.41948	6.98E-07	86	257	2.2592	8.94E-07	24	95	0.591	8.71E-06	20	80	0.62608	4.62E-07	14	55	0.39671	7.30E-06
	x8	21	62	0.50253	6.98E-07	86	257	2.0441	8.94E-07	24	95	0.57094	8.71E-06	20	80	0.63396	4.62E-07	14	55	0.2969	7.30E-06
	x9	21	62	0.74426	6.99E-07	86	257	2.8709	8.94E-07	24	95	0.75883	8.69E-06	20	80	0.6459	4.62E-07	14	55	0.52403	7.30E-06

Table 6.

Numerical results for Problem 4.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	1	3	0.006154	0.00E+00	76	227	0.42078	8.61E-07	19	75	0.01192	5.73E-06	16	64	0.0095	8.79E-07	17	67	0.011751	5.39E-06
	x2	1	3	0.001905	0.00E+00	68	203	0.026772	8.60E-07	18	71	0.010833	9.93E-06	15	60	0.010303	5.13E-07	10	39	0.007579	3.65E-06
	x3	1	3	0.001996	2.22E-16	59	176	0.033022	9.62E-07	15	60	0.013106	9.35E-06	13	52	0.009199	8.83E-07	10	40	0.010692	4.91E-06
	x4	16	48	0.016692	8.42E-07	74	221	0.031073	8.94E-07	20	79	0.00808	6.89E-06	17	68	0.012899	4.91E-07	18	71	0.011959	6.51E-06
	x5	16	48	0.023724	8.42E-07	74	221	0.03155	8.94E-07	20	79	0.014524	6.89E-06	17	68	0.012656	4.91E-07	18	71	0.008487	6.51E-06
	x6	15	44	0.017326	7.29E-07	62	185	0.035103	9.27E-07	17	67	0.005744	6.62E-06	15	60	0.012038	4.55E-07	15	59	0.008223	6.75E-06
	x7	16	48	0.020707	8.42E-07	74	221	0.022633	8.94E-07	20	79	0.015181	6.89E-06	17	68	0.009832	4.91E-07	18	71	0.014186	6.51E-06
	x8	17	50	0.032877	6.73E-07	74	221	0.024689	8.94E-07	20	79	0.01043	6.91E-06	17	68	0.010587	4.96E-07	18	71	0.009381	6.53E-06
	x9	15	45	0.022213	9.31E-07	74	221	0.044716	9.00E-07	20	79	0.00766	6.77E-06	17	68	0.007763	5.01E-07	18	71	0.010474	6.54E-06
5000	x1	1	3	0.00449	0.00E+00	79	236	0.093652	9.85E-07	20	79	0.027713	6.42E-06	17	68	0.042218	6.59E-07	18	71	0.02803	5.45E-06
	x2	1	3	0.00811	0.00E+00	71	212	0.11582	9.85E-07	20	79	0.026781	5.57E-06	16	64	0.034413	3.86E-07	10	39	0.01728	8.15E-06
	x3	1	3	0.005698	2.22E-16	59	176	0.13671	9.62E-07	15	60	0.019444	9.35E-06	13	52	0.025878	8.83E-07	10	40	0.024215	4.91E-06
	x4	17	51	0.13449	7.31E-07	78	233	0.10456	8.19E-07	21	83	0.036654	7.73E-06	18	72	0.041715	3.69E-07	19	75	0.036873	6.61E-06
	x5	17	51	0.032044	7.31E-07	78	233	0.11744	8.19E-07	21	83	0.032061	7.73E-06	18	72	0.046373	3.69E-07	19	75	0.028516	6.61E-06
	x6	15	44	0.078846	7.29E-07	62	185	0.11264	9.27E-07	17	67	0.026217	6.63E-06	15	60	0.033336	4.55E-07	15	59	0.023959	6.76E-06
	x7	17	51	0.036052	7.31E-07	78	233	0.15566	8.19E-07	21	83	0.029738	7.73E-06	18	72	0.041526	3.69E-07	19	75	0.033766	6.61E-06
	x8	17	51	0.092	7.64E-07	78	233	0.10586	8.19E-07	21	83	0.072791	7.73E-06	18	72	0.061274	3.70E-07	19	75	0.0304	6.62E-06
	x9	17	51	0.042002	6.15E-07	78	233	0.368	8.22E-07	21	83	0.033322	7.70E-06	18	72	0.040994	3.80E-07	19	75	0.032687	6.61E-06
10000	x1	1	3	0.007893	0.00E+00	81	242	0.67484	8.92E-07	20	79	0.051839	9.08E-06	17	68	0.066822	9.32E-07	18	71	0.079653	7.70E-06
	x2	1	3	0.007634	0.00E+00	73	218	0.17572	8.91E-07	20	79	0.074758	7.88E-06	16	64	0.067781	5.46E-07	11	43	0.02922	3.00E-06
	x3	1	3	0.015904	2.22E-16	59	176	0.22518	9.62E-07	15	60	0.033559	9.35E-06	13	52	0.051043	8.83E-07	10	40	0.0272	4.91E-06
	x4	18	53	0.10333	6.69E-07	79	236	0.31815	9.26E-07	22	87	0.050369	5.48E-06	18	72	0.076736	5.22E-07	19	75	0.045885	9.36E-06
	x5	18	53	0.060803	6.69E-07	79	236	0.26202	9.26E-07	22	87	0.050461	5.48E-06	18	72	0.071018	5.22E-07	19	75	0.044874	9.36E-06
	x6	15	44	0.065192	7.29E-07	62	185	0.15996	9.27E-07	17	67	0.039317	6.63E-06	15	60	0.05065	4.55E-07	15	59	0.035413	6.76E-06
	x7	18	53	0.094819	6.69E-07	79	236	0.23701	9.26E-07	22	87	0.062936	5.48E-06	18	72	0.076797	5.22E-07	19	75	0.066942	9.36E-06
	x8	18	53	0.091724	6.84E-07	79	236	0.17905	9.27E-07	22	87	0.078648	5.48E-06	18	72	0.085378	5.23E-07	19	75	0.045675	9.36E-06
	x9	18	53	0.082879	7.19E-07	79	236	0.17998	9.24E-07	22	87	0.067936	5.49E-06	18	72	0.081977	5.28E-07	19	75	0.047638	9.33E-06
50000	x1	1	3	0.01661	0.00E+00	85	254	0.73548	8.17E-07	22	87	0.1758	5.10E-06	19	76	0.23641	9.22E-07	19	75	0.1678	7.79E-06
	x2	1	3	0.015122	0.00E+00	77	230	0.67746	8.16E-07	21	83	0.16936	8.83E-06	17	68	0.19794	4.04E-07	11	43	0.10268	6.70E-06
	x3	1	3	0.015695	2.22E-16	59	176	0.42812	9.62E-07	15	60	0.11693	9.35E-06	13	52	0.15029	8.83E-07	10	40	0.091225	4.91E-06
	x4	18	54	0.26938	8.30E-07	83	248	0.70681	8.49E-07	23	91	0.31199	6.14E-06	19	76	0.28722	6.74E-07	20	79	0.15468	9.47E-06
	x5	18	54	0.22256	8.30E-07	83	248	0.67509	8.49E-07	23	91	0.18519	6.14E-06	19	76	0.2694	6.74E-07	20	79	0.15464	9.47E-06
	x6	15	44	0.25757	7.29E-07	62	185	0.46034	9.27E-07	17	67	0.19891	6.63E-06	15	60	0.1812	4.55E-07	15	59	0.14469	6.76E-06
	x7	18	54	0.2146	8.30E-07	83	248	0.65063	8.49E-07	23	91	0.27739	6.14E-06	19	76	0.24048	6.74E-07	20	79	0.16284	9.47E-06
	x8	18	54	0.14835	8.33E-07	83	248	0.70144	8.49E-07	23	91	0.20178	6.14E-06	19	76	0.31257	6.74E-07	20	79	0.16382	9.47E-06
	x9	18	54	0.16348	8.57E-07	83	248	0.7146	8.49E-07	23	91	0.28552	6.12E-06	19	76	0.23715	6.73E-07	20	79	0.1575	9.51E-06
100000	x1	1	3	0.024111	0.00E+00	86	257	1.3852	9.24E-07	22	87	0.47813	7.21E-06	20	80	0.65758	7.47E-07	20	79	0.36895	4.99E-06
	x2	1	3	0.02247	0.00E+00	78	233	1.17	9.24E-07	22	87	0.34454	6.25E-06	17	68	0.36751	5.71E-07	11	43	0.16275	9.48E-06
	x3	1	3	0.03738	2.22E-16	59	176	0.8849	9.62E-07	15	60	0.28862	9.35E-06	13	52	0.36203	8.83E-07	10	40	0.15124	4.91E-06
	x4	19	56	0.31899	7.52E-07	84	251	1.2631	9.60E-07	23	91	0.44782	8.68E-06	19	76	0.48767	9.54E-07	21	84	0.40317	6.06E-06
	x5	19	56	0.33527	7.52E-07	84	251	1.457	9.60E-07	23	91	0.353	8.68E-06	19	76	0.55845	9.54E-07	21	84	0.31449	6.06E-06
	x6	15	44	0.26252	7.29E-07	62	185	0.96211	9.27E-07	17	67	0.26657	6.63E-06	15	60	0.40234	4.55E-07	15	59	0.21568	6.76E-06
	x7	19	56	0.31033	7.52E-07	84	251	1.2752	9.60E-07	23	91	0.3295	8.68E-06	19	76	0.61574	9.54E-07	21	84	0.41612	6.06E-06
	x8	19	56	0.35272	7.54E-07	84	251	1.239	9.60E-07	23	91	0.39743	8.68E-06	19	76	0.47514	9.54E-07	21	84	0.3471	6.06E-06
	x9	19	56	0.33608	7.50E-07	84	251	1.1977	9.59E-07	23	91	0.41664	8.69E-06	19	76	0.47144	9.54E-07	21	84	0.42077	6.06E-06

Table 7.

Numerical results for Problem 5.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	9	26	0.014308	4.55E-07	81	242	0.43008	8.63E-07	23	91	0.013039	6.09E-06	17	68	0.017566	9.43E-07	12	47	0.006837	5.17E-06
	x2	9	26	0.021676	7.37E-07	83	248	0.046613	8.42E-07	23	91	0.017683	9.28E-06	18	72	0.037333	4.82E-07	12	47	0.013967	7.88E-06
	x3	9	26	0.017498	7.74E-07	83	248	0.044732	8.73E-07	23	91	0.01172	9.63E-06	18	72	0.015285	5.00E-07	12	47	0.01114	8.18E-06
	x4	9	27	0.013582	8.60E-07	82	245	0.073931	8.99E-07	23	91	0.012915	7.93E-06	18	72	0.021834	4.12E-07	12	47	0.011322	6.74E-06
	x5	9	27	0.017552	8.60E-07	82	245	0.087185	8.99E-07	23	91	0.020411	7.93E-06	18	72	0.021438	4.12E-07	12	47	0.007829	6.74E-06
	x6	9	27	0.019158	5.89E-07	83	248	0.04814	8.71E-07	23	91	0.019498	9.61E-06	18	72	0.023522	4.99E-07	12	47	0.012242	8.16E-06
	x7	9	27	0.015932	8.60E-07	82	245	0.052921	8.99E-07	23	91	0.013342	7.93E-06	18	72	0.016194	4.12E-07	12	47	0.01134	6.74E-06
	x8	9	27	0.019722	8.60E-07	82	245	0.04338	8.99E-07	23	91	0.013424	7.92E-06	18	72	0.020161	4.12E-07	12	47	0.013278	6.73E-06
	x9	9	27	0.009656	9.32E-07	82	245	0.063622	9.01E-07	23	91	0.013643	7.93E-06	18	72	0.016972	4.13E-07	12	47	0.00937	6.74E-06
5000	x1	8	24	0.032365	5.92E-07	84	251	0.199	9.89E-07	24	95	0.067194	6.83E-06	18	72	0.075774	7.08E-07	13	51	0.037741	3.01E-06
	x2	8	24	0.032245	5.10E-07	86	257	0.26411	9.65E-07	25	99	0.066083	5.22E-06	19	76	0.088501	3.58E-07	13	51	0.041404	4.59E-06
	x3	9	26	0.04046	6.16E-07	87	260	0.19137	8.01E-07	25	99	0.065242	5.41E-06	19	76	0.093793	3.72E-07	13	51	0.044886	4.77E-06
	x4	10	29	0.041491	5.88E-07	86	257	0.26052	8.24E-07	24	95	0.070926	8.89E-06	18	72	0.07529	9.22E-07	13	51	0.041548	3.92E-06
	x5	10	29	0.040664	5.88E-07	86	257	0.32941	8.24E-07	24	95	0.086487	8.89E-06	18	72	0.091088	9.22E-07	13	51	0.05445	3.92E-06
	x6	9	26	0.050973	8.50E-07	87	260	0.21405	8.01E-07	25	99	0.064145	5.41E-06	19	76	0.11547	3.72E-07	13	51	0.036684	4.77E-06
	x7	10	29	0.039105	5.88E-07	86	257	0.20749	8.24E-07	24	95	0.06261	8.89E-06	18	72	0.08249	9.22E-07	13	51	0.038958	3.92E-06
	x8	10	29	0.04557	5.88E-07	86	257	0.19755	8.24E-07	24	95	0.068231	8.89E-06	18	72	0.085026	9.22E-07	13	51	0.035929	3.92E-06
	x9	10	29	0.036645	4.36E-07	86	257	0.19841	8.24E-07	24	95	0.093715	8.90E-06	18	72	0.085331	9.22E-07	13	51	0.042215	3.93E-06
10000	x1	8	24	0.051154	3.22E-07	86	257	0.40323	8.95E-07	24	95	0.10829	9.66E-06	19	76	0.16951	3.32E-07	13	51	0.070885	4.26E-06
	x2	8	24	0.050852	5.07E-07	88	263	0.38203	8.73E-07	25	99	0.13603	7.38E-06	21	84	0.16399	4.00E-07	13	51	0.065384	6.50E-06
	x3	8	24	0.053192	4.68E-07	88	263	0.35275	9.06E-07	25	99	0.115	7.66E-06	21	84	0.19134	4.15E-07	13	51	0.075127	6.74E-06
	x4	9	27	0.053443	6.58E-07	87	260	0.41228	9.33E-07	25	99	0.11631	6.30E-06	20	80	0.16239	5.88E-07	13	51	0.07051	5.55E-06
	x5	9	27	0.052797	6.58E-07	87	260	0.42477	9.33E-07	25	99	0.13438	6.30E-06	20	80	0.18493	5.88E-07	13	51	0.11233	5.55E-06
	x6	8	24	0.052837	7.94E-07	88	263	0.39233	9.06E-07	25	99	0.1112	7.65E-06	21	84	0.18611	4.15E-07	13	51	0.061267	6.74E-06
	x7	9	27	0.057635	6.58E-07	87	260	0.35563	9.33E-07	25	99	0.12898	6.30E-06	20	80	0.16743	5.88E-07	13	51	0.078467	5.55E-06
	x8	9	27	0.056419	6.58E-07	87	260	0.38846	9.32E-07	25	99	0.13804	6.30E-06	20	80	0.15289	5.88E-07	13	51	0.061632	5.55E-06
	x9	9	27	0.059129	2.70E-07	87	260	0.36412	9.34E-07	25	99	0.12657	6.30E-06	20	80	0.16362	5.88E-07	13	51	0.072629	5.55E-06
50000	x1	8	24	0.13295	6.45E-07	90	269	1.4145	8.20E-07	26	103	0.52217	5.42E-06	22	88	0.79337	3.65E-07	13	51	0.22327	9.53E-06
	x2	8	24	0.15183	9.82E-07	91	272	1.3927	1.00E-06	26	103	0.48147	8.26E-06	24	96	0.77851	7.08E-07	14	55	0.27364	3.78E-06
	x3	9	26	0.18609	4.08E-07	92	275	1.4134	8.30E-07	26	103	0.41064	8.58E-06	24	96	0.8668	7.35E-07	14	55	0.23217	3.92E-06
	x4	9	26	0.17376	7.08E-07	91	272	1.3785	8.54E-07	26	103	0.4928	7.06E-06	23	92	0.77416	7.32E-07	14	55	0.33219	3.23E-06
	x5	9	26	0.18155	7.08E-07	91	272	1.3865	8.54E-07	26	103	0.48041	7.06E-06	23	92	0.65644	7.32E-07	14	55	0.24114	3.23E-06
	x6	9	26	0.1553	4.08E-07	92	275	1.4131	8.30E-07	26	103	0.42987	8.58E-06	24	96	0.78074	7.35E-07	14	55	0.33316	3.92E-06
	x7	9	26	0.19292	7.08E-07	91	272	1.3931	8.54E-07	26	103	0.53661	7.06E-06	23	92	0.70399	7.32E-07	14	55	0.24212	3.23E-06
	x8	9	26	0.15393	7.08E-07	91	272	1.4504	8.54E-07	26	103	0.43061	7.06E-06	23	92	0.70673	7.32E-07	14	55	0.28021	3.23E-06
	x9	9	26	0.14097	5.39E-07	91	272	1.4143	8.54E-07	26	103	0.3998	7.06E-06	23	92	0.73136	7.32E-07	14	55	0.22989	3.23E-06
100000	x1	8	24	0.27797	9.12E-07	91	272	2.9557	9.28E-07	26	103	0.89856	7.67E-06	24	96	1.6424	6.57E-07	14	55	0.60769	3.51E-06
	x2	9	26	0.33938	5.56E-07	93	278	2.9333	9.05E-07	27	107	1.1367	5.86E-06	29	116	2.2829	5.93E-07	14	55	0.53702	5.34E-06
	x3	9	26	0.3373	5.77E-07	93	278	2.8816	9.39E-07	27	107	1.0392	6.08E-06	29	116	2.1516	6.15E-07	14	55	0.59951	5.55E-06
	x4	9	26	0.30358	5.60E-07	92	275	2.9059	9.66E-07	26	103	1.0031	9.98E-06	26	104	1.8211	6.44E-07	14	55	0.54641	4.56E-06
	x5	9	26	0.33247	4.75E-07	92	275	2.9764	9.66E-07	26	103	1.0496	9.98E-06	26	104	1.8442	6.44E-07	14	55	0.60126	4.56E-06
	x6	9	26	0.31425	5.77E-07	93	278	2.917	9.39E-07	27	107	1.0072	6.08E-06	29	116	2.1713	6.15E-07	14	55	0.63201	5.55E-06
	x7	9	26	0.3306	4.75E-07	92	275	3.3714	9.66E-07	26	103	1.0065	9.98E-06	26	104	1.8355	6.44E-07	14	55	0.56667	4.56E-06
	x8	9	26	0.31528	4.75E-07	92	275	2.8726	9.66E-07	26	103	1.0548	9.98E-06	26	104	1.8133	6.44E-07	14	55	0.54508	4.56E-06
	x9	9	26	0.32075	4.76E-07	92	275	3.0658	9.66E-07	26	103	0.95345	9.98E-06	26	104	1.8203	6.44E-07	14	55	0.53629	4.56E-06

Table 8.

Numerical results for Problem 6.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	12	35	0.029049	4.64E-07	38	113	1.5939	6.56E-07	21	83	0.009165	6.52E-06	6	24	0.008392	6.51E-07	11	43	0.005861	3.97E-06
	x2	11	32	0.010305	9.75E-07	37	110	0.022469	9.48E-07	17	67	0.008178	6.98E-06	17	68	0.013304	6.92E-07	10	39	0.012039	2.46E-06
	x3	15	44	0.024444	6.21E-07	38	113	0.022915	7.65E-07	18	71	0.008026	7.24E-06	18	72	0.019236	3.67E-07	10	39	0.007047	4.95E-06
	x4	16	47	0.029762	7.29E-07	37	110	0.022978	6.47E-07	20	79	0.016209	8.69E-06	19	76	0.015778	4.09E-07	11	43	0.00795	3.10E-06
	x5	16	47	0.023724	7.29E-07	37	110	0.035643	6.47E-07	20	79	0.013577	8.69E-06	19	76	0.0169	4.09E-07	11	43	0.007036	3.10E-06
	x6	15	45	0.030054	9.83E-07	38	113	0.026261	7.56E-07	21	83	0.011036	4.87E-06	18	72	0.020828	3.70E-07	12	47	0.007196	7.47E-06
	x7	16	47	0.020398	7.29E-07	37	110	0.037504	6.47E-07	20	79	0.017326	8.69E-06	19	76	0.014674	4.09E-07	11	43	0.008818	3.10E-06
	x8	16	47	0.021973	7.30E-07	37	110	0.025528	6.47E-07	20	79	0.009359	8.68E-06	19	76	0.019374	4.10E-07	11	43	0.007293	3.09E-06
	x9	16	47	0.030921	7.46E-07	37	110	0.060642	6.56E-07	20	79	0.013278	8.78E-06	19	76	0.012923	3.94E-07	11	43	0.010979	3.12E-06
5000	x1	12	36	0.050053	6.79E-07	39	116	0.060036	9.18E-07	22	87	0.04615	7.10E-06	7	28	0.020765	6.10E-08	11	43	0.023684	8.88E-06
	x2	12	35	0.039825	4.65E-07	39	116	0.058258	8.30E-07	18	71	0.031432	7.60E-06	18	72	0.057909	5.59E-07	10	39	0.040338	5.49E-06
	x3	15	44	0.057796	7.99E-07	40	119	0.080503	6.70E-07	19	75	0.042786	7.53E-06	18	72	0.078605	8.22E-07	11	43	0.027138	2.57E-06
	x4	17	50	0.078499	6.25E-07	38	113	0.058221	9.05E-07	21	83	0.04181	9.46E-06	19	76	0.080876	9.15E-07	11	43	0.02473	6.94E-06
	x5	17	50	0.065272	6.25E-07	38	113	0.14559	9.05E-07	21	83	0.042197	9.46E-06	19	76	0.06045	9.15E-07	11	43	0.022927	6.94E-06
	x6	16	47	0.036037	6.68E-07	40	119	0.10331	6.68E-07	21	83	0.046372	4.92E-06	18	72	0.096532	8.22E-07	12	47	0.02725	3.99E-06
	x7	17	50	0.066678	6.25E-07	38	113	0.093066	9.05E-07	21	83	0.036007	9.46E-06	19	76	0.062289	9.15E-07	11	43	0.026667	6.94E-06
	x8	17	50	0.060576	6.25E-07	38	113	0.076496	9.05E-07	21	83	0.053183	9.46E-06	19	76	0.0768	9.16E-07	11	43	0.027605	6.93E-06
	x9	17	50	0.065735	6.26E-07	38	113	0.095071	9.10E-07	21	83	0.050867	9.42E-06	19	76	0.080463	9.21E-07	11	43	0.033035	6.90E-06
10000	x1	12	36	0.094012	9.61E-07	40	119	0.11766	8.12E-07	23	91	0.078114	4.89E-06	7	28	0.048952	8.62E-08	12	47	0.048396	3.03E-06
	x2	12	35	0.064997	6.57E-07	40	119	0.14917	7.34E-07	19	75	0.067982	5.23E-06	18	72	0.11357	7.90E-07	10	39	0.035316	7.77E-06
	x3	16	47	0.082227	5.81E-07	40	119	0.11575	9.48E-07	20	79	0.071007	5.16E-06	19	76	0.11519	4.22E-07	11	43	0.043755	3.61E-06
	x4	17	50	0.091098	8.83E-07	39	116	0.11448	8.00E-07	22	87	0.088341	6.52E-06	20	80	0.10801	4.69E-07	11	43	0.050355	9.81E-06
	x5	17	50	0.089616	8.83E-07	39	116	0.1367	8.00E-07	22	87	0.065983	6.52E-06	20	80	0.11862	4.69E-07	11	43	0.046022	9.81E-06
	x6	16	47	0.14203	9.36E-07	40	119	0.12105	9.46E-07	23	91	0.079529	6.19E-06	19	76	0.10274	4.22E-07	13	51	0.046248	6.38E-06
	x7	17	50	0.095615	8.83E-07	39	116	0.16129	8.00E-07	22	87	0.094775	6.52E-06	20	80	0.121	4.69E-07	11	43	0.072467	9.81E-06
	x8	17	50	0.21555	8.83E-07	39	116	0.12837	8.00E-07	22	87	0.074221	6.52E-06	20	80	0.13616	4.69E-07	11	43	0.039266	9.81E-06
	x9	17	50	0.10375	8.75E-07	39	116	0.3069	7.99E-07	22	87	0.099181	6.53E-06	20	80	0.12777	4.68E-07	11	43	0.04783	9.79E-06
50000	x1	13	38	0.22221	7.00E-07	42	125	1.3933	7.10E-07	24	95	0.31765	5.33E-06	20	80	0.39988	6.95E-07	12	47	0.16422	6.77E-06
	x2	12	36	0.17902	9.62E-07	42	125	0.55424	6.42E-07	20	79	0.2098	5.70E-06	19	76	0.44299	6.42E-07	11	43	0.1288	4.19E-06
	x3	16	47	0.25355	8.83E-07	42	125	0.50726	8.30E-07	21	83	0.33955	5.59E-06	19	76	0.44244	9.44E-07	11	43	0.12579	8.05E-06
	x4	18	53	0.26099	6.54E-07	41	122	0.39556	7.00E-07	23	91	0.23472	7.10E-06	21	84	0.45082	3.80E-07	12	47	0.15312	5.30E-06
	x5	18	53	0.21421	6.54E-07	41	122	0.40101	7.00E-07	23	91	0.31743	7.10E-06	21	84	0.3826	3.80E-07	12	47	0.1681	5.30E-06
	x6	17	50	0.29397	5.53E-07	42	125	0.4652	8.29E-07	21	83	0.28602	5.86E-06	19	76	0.3848	9.44E-07	12	47	0.13501	2.54E-06
	x7	18	53	0.25801	6.54E-07	41	122	0.42786	7.00E-07	23	91	0.25659	7.10E-06	21	84	0.43777	3.80E-07	12	47	0.17412	5.30E-06
	x8	18	53	0.3159	6.54E-07	41	122	0.36268	7.00E-07	23	91	0.24778	7.10E-06	21	84	0.43678	3.80E-07	12	47	0.15937	5.30E-06
	x9	18	53	0.31437	6.55E-07	41	122	0.55496	7.02E-07	23	91	0.37644	7.09E-06	21	84	0.68463	3.81E-07	12	47	0.18109	5.30E-06
100000	x1	13	38	0.46788	9.90E-07	43	128	0.76993	6.28E-07	24	95	0.56137	7.54E-06	20	80	0.83997	9.83E-07	12	47	0.26978	9.58E-06
	x2	13	38	0.33088	4.43E-07	42	125	0.77585	9.08E-07	20	79	0.42556	8.06E-06	20	80	0.7169	7.45E-07	11	43	0.29106	5.93E-06
	x3	16	47	0.42191	8.55E-07	43	128	0.8996	7.34E-07	21	83	0.50609	7.90E-06	20	80	0.79422	9.21E-07	12	47	0.30009	2.74E-06
	x4	18	53	0.43122	9.25E-07	41	122	0.68719	9.90E-07	24	95	0.61866	4.89E-06	20	80	0.83134	6.67E-07	12	47	0.29504	7.49E-06
	x5	18	53	0.46021	9.25E-07	41	122	0.73513	9.90E-07	24	95	0.51049	4.89E-06	20	80	0.87882	6.67E-07	12	47	0.2534	7.49E-06
	x6	17	50	0.55415	7.04E-07	43	128	0.81192	7.34E-07	21	83	0.57768	8.09E-06	20	80	0.8959	9.21E-07	12	47	0.26603	3.24E-06
	x7	18	53	0.51063	9.25E-07	41	122	0.91418	9.90E-07	24	95	0.46009	4.89E-06	20	80	0.74139	6.67E-07	12	47	0.38601	7.49E-06
	x8	18	53	0.4617	9.25E-07	41	122	0.66684	9.90E-07	24	95	0.63668	4.89E-06	20	80	0.94291	6.67E-07	12	47	0.35009	7.49E-06
	x9	18	53	0.72029	9.24E-07	41	122	1.1989	9.92E-07	24	95	0.75387	4.90E-06	20	80	0.80012	6.68E-07	12	47	0.35252	7.48E-06

Table 9.

Numerical results for Problem 7.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	0	0	0.038906	0	0	0	0.78166	0	0	0	0.002616	0	0	0	0.003328	0	0	0	0.000844	0
	x2	12	35	0.033115	4.05E-07	122	365	0.67291	9.16E-07	23	91	0.038102	7.76E-06	36	144	0.16109	6.34E-07	61	243	0.078169	9.62E-06
	x3	12	35	0.0381	7.68E-07	83	248	0.17512	9.01E-07	101	403	0.18844	9.86E-06	36	144	0.26356	6.40E-07	132	527	0.27147	9.23E-06
	x4	13	38	0.039655	4.65E-07	110	329	0.11532	9.44E-07	19	75	0.032265	9.48E-06	36	144	0.18872	6.45E-07	37	147	0.050214	8.52E-06
	x5	13	38	0.024441	3.59E-07	62	185	0.07797	7.88E-07	17	67	0.027818	6.63E-06	24	96	0.11755	5.84E-07	29	115	0.054364	6.75E-06
	x6	14	41	0.04153	4.87E-07	69	206	0.093189	9.49E-07	179	715	0.23112	9.93E-06	34	136	0.18349	7.16E-07	119	475	0.14755	9.02E-06
	x7	13	38	0.052096	4.65E-07	110	329	0.11319	9.44E-07	19	75	0.025705	9.48E-06	36	144	0.1563	6.45E-07	37	147	0.069248	8.52E-06
	x8	13	38	0.031708	3.60E-07	61	182	0.075667	8.88E-07	17	67	0.025078	6.62E-06	22	88	0.14498	4.33E-07	29	115	0.04373	6.84E-06
	x9	17	50	0.049173	3.59E-07	132	395	0.14667	9.21E-07	17	67	0.048771	7.80E-06	32	128	0.16597	2.41E-07	220	879	0.37058	9.69E-06
5000	x1	0	0	0.002789	0	0	0	0.003597	0	0	0	0.007603	0	0	0	0.001808	0	0	0	0.003491	0
	x2	12	35	0.10778	7.57E-07	121	362	0.52844	9.10E-07	20	79	0.12976	8.49E-06	34	136	0.80841	8.36E-07	77	307	0.53458	9.40E-06
	x3	12	36	0.13359	7.78E-07	80	239	0.41913	9.09E-07	100	399	0.69403	9.74E-06	34	136	0.83492	8.69E-07	87	347	0.66276	8.66E-06
	x4	13	38	0.1268	9.52E-07	107	320	0.46613	9.77E-07	19	75	0.12409	5.69E-06	34	136	0.81581	8.49E-07	44	175	0.29489	1.00E-05
	x5	13	38	0.12218	5.01E-07	64	191	0.28878	7.27E-07	18	71	0.14112	5.79E-06	22	88	0.50975	9.54E-07	40	159	0.28649	7.70E-06
	x6	14	41	0.12093	4.83E-07	74	221	0.34861	9.38E-07	611	2443	4.0397	9.79E-06	32	128	0.71834	5.84E-07	116	463	0.90756	9.75E-06
	x7	13	38	0.19395	9.52E-07	107	320	0.50089	9.77E-07	19	75	0.12433	5.69E-06	34	136	0.75098	8.49E-07	44	175	0.28655	1.00E-05
	x8	13	38	0.14336	5.01E-07	64	191	0.33556	7.24E-07	18	71	0.13688	5.79E-06	23	92	0.57897	2.97E-07	31	123	0.24484	7.96E-06
	x9	17	50	0.15414	7.01E-07	106	317	0.48913	9.45E-07	18	71	0.15946	6.36E-06	27	108	0.60206	3.49E-07	42	167	0.33114	9.65E-06
10000	x1	0	0	0.00298	0	0	0	0.002765	0	0	0	0.002035	0	0	0	0.006066	0	0	0	0.002716	0
	x2	12	36	0.2002	5.65E-07	120	359	0.94571	9.61E-07	20	79	0.25766	6.25E-06	34	136	1.4786	6.78E-07	61	243	0.89428	8.97E-06
	x3	13	38	0.16935	3.90E-07	79	236	0.67652	9.34E-07	99	395	1.4137	9.23E-06	34	136	1.477	7.07E-07	135	539	1.8146	9.45E-06
	x4	13	39	0.15451	6.73E-07	107	320	0.91163	9.02E-07	19	75	0.21665	5.57E-06	34	136	1.5588	6.89E-07	23	91	0.28068	6.83E-06
	x5	13	38	0.20324	5.44E-07	65	194	0.50449	7.35E-07	18	71	0.23743	8.53E-06	22	88	1.1074	9.79E-07	26	103	0.36902	7.22E-06
	x6	14	41	0.18092	5.24E-07	76	227	0.6752	9.44E-07	189	755	2.7342	9.94E-06	31	124	1.3468	7.33E-07	131	523	1.709	9.68E-06
	x7	13	39	0.19382	6.73E-07	107	320	0.85944	9.02E-07	19	75	0.21799	5.57E-06	34	136	1.5649	6.89E-07	23	91	0.34965	6.83E-06
	x8	13	38	0.35269	5.45E-07	65	194	0.51017	7.34E-07	18	71	0.25233	8.53E-06	21	84	0.987	7.65E-07	26	103	0.32723	7.24E-06
	x9	17	51	0.27824	8.84E-07	129	386	1.0032	9.30E-07	18	71	0.26393	8.76E-06	27	108	1.2328	2.79E-07	75	299	1.1469	9.17E-06
50000	x1	0	0	0.012584	0	0	0	0.009213	0	0	0	0.009373	0	0	0	0.009031	0	0	0	0.010445	0
	x2	13	38	0.84045	7.66E-07	119	356	4.0401	9.39E-07	19	75	1.0123	9.22E-06	34	136	5.8422	6.35E-07	94	375	4.5319	9.54E-06
	x3	13	38	0.62033	8.66E-07	77	230	2.6023	9.99E-07	96	383	4.4204	8.76E-06	34	136	5.5657	6.70E-07	124	495	5.9186	9.61E-06
	x4	14	41	0.69678	5.16E-07	105	314	3.5696	9.65E-07	19	75	1.0568	8.01E-06	33	132	5.574	6.56E-07	15	59	1.0361	7.21E-06
	x5	13	39	0.63099	4.82E-07	67	200	2.4286	8.48E-07	19	75	1.0308	6.76E-06	23	92	3.962	9.16E-07	29	115	1.5984	8.81E-06
	x6	14	41	0.79236	6.78E-07	77	230	2.7058	9.28E-07	107	427	5.0498	9.28E-06	32	128	5.6124	6.70E-07	178	711	8.4281	9.35E-06
	x7	14	41	0.90717	5.16E-07	105	314	3.5951	9.65E-07	19	75	1.0551	8.01E-06	33	132	5.7912	6.56E-07	15	59	0.85465	7.21E-06
	x8	13	39	0.64677	4.83E-07	67	200	2.4259	8.48E-07	19	75	1.0081	6.76E-06	23	92	3.9011	9.07E-07	29	115	1.5584	8.85E-06
	x9	18	53	1.0288	6.35E-07	121	362	4.0524	9.09E-07	19	75	1.0884	6.26E-06	26	104	4.4559	3.85E-07	115	459	5.6331	9.80E-06
100000	x1	0	0	0.020129	0	0	0	0.044349	0	0	0	0.016332	0	0	0	0.016318	0	0	0	0.02095	0
	x2	13	39	1.4101	4.49E-07	118	353	8.7163	9.89E-07	20	79	1.9685	4.35E-06	33	132	11.752	8.00E-07	127	507	12.9108	9.95E-06
	x3	13	39	1.2614	6.14E-07	77	230	6.1223	9.43E-07	96	383	9.5064	9.73E-06	33	132	11.5596	8.45E-07	183	731	18.0213	9.91E-06
	x4	14	41	1.4097	7.29E-07	105	314	8.316	9.08E-07	20	79	2.1194	3.86E-06	35	140	12.4189	6.03E-07	15	59	1.9333	8.01E-06
	x5	13	39	1.3517	8.61E-07	68	203	5.4894	8.63E-07	19	75	2.0334	9.63E-06	23	92	8.0264	9.84E-07	21	83	2.4448	7.92E-06
	x6	14	41	1.3917	7.81E-07	77	230	5.9139	9.06E-07	108	431	11.1115	8.75E-06	32	128	11.542	5.58E-07	182	727	19.5586	9.37E-06
	x7	14	41	1.7178	7.29E-07	105	314	7.6441	9.08E-07	20	79	2.1252	3.86E-06	35	140	12.3952	6.03E-07	15	59	2.1594	8.01E-06
	x8	13	39	1.3664	8.61E-07	68	203	5.0265	8.63E-07	19	75	2.0399	9.63E-06	24	96	8.698	7.30E-07	21	83	3.0546	7.92E-06
	x9	18	53	1.8446	8.85E-07	121	362	8.769	9.42E-07	19	75	2.0222	8.84E-06	26	104	9.1914	6.07E-07	108	431	12.64	9.99E-06

Table 10.

Numerical results for Problem 8.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	8	20	0.045332	9.80E-07	27	76	0.13731	8.06E-07	9	31	0.15603	7.60E-06	11	42	0.010432	2.67E-07	8	25	0.004228	6.09E-06
	x2	8	20	0.015402	9.80E-07	39	113	0.013511	8.17E-07	9	31	0.011112	7.60E-06	11	42	0.01056	2.67E-07	8	25	0.005784	6.09E-06
	x3	39	113	0.050306	9.61E-07	F	F	F	F	26	99	0.009084	9.77E-06	34	107	0.012388	9.76E-07	213	849	0.091231	1.00E-05
	x4	58	166	0.070369	9.97E-07	F	F	F	F	10	35	0.006974	7.26E-06	11	42	0.008517	2.70E-07	21	81	0.011337	9.91E-06
	x5	58	166	0.0931	9.97E-07	F	F	F	F	10	35	0.006215	7.26E-06	11	42	0.031007	2.70E-07	21	81	0.009868	9.91E-06
	x6	46	134	0.057752	9.80E-07	45	132	0.01524	8.49E-07	9	31	0.005135	7.60E-06	11	41	0.007564	2.67E-07	8	25	0.008325	6.09E-06
	x7	58	166	0.028136	9.97E-07	F	F	F	F	10	35	0.005299	7.26E-06	11	42	0.008569	2.70E-07	21	81	0.016458	9.91E-06
	x8	60	172	0.15433	9.77E-07	44	129	0.014334	9.92E-07	9	31	0.00324	7.60E-06	11	42	0.011897	2.67E-07	8	25	0.007805	6.09E-06
	x9	60	172	0.09293	9.90E-07	73	216	0.019643	9.87E-07	9	31	0.004678	7.60E-06	11	42	0.010021	2.67E-07	8	25	0.006595	6.09E-06
5000	x1	6	15	0.023477	5.40E-07	25	72	0.028521	7.58E-07	7	25	0.014113	1.30E-06	8	31	0.031179	1.59E-07	4	12	0.025081	5.76E-06
	x2	6	15	0.019769	5.40E-07	26	75	0.04814	9.83E-07	7	25	0.016343	1.30E-06	8	31	0.026785	1.59E-07	4	12	0.031653	5.76E-06
	x3	8	22	0.025065	2.74E-07	86	255	0.092447	9.78E-07	17	65	0.033401	8.72E-06	11	34	0.029802	8.53E-07	10	37	0.03622	7.40E-06
	x4	18	52	0.11062	9.23E-07	454	1359	0.59789	9.99E-07	7	25	0.025883	1.42E-06	8	31	0.028639	1.59E-07	64	253	0.15295	9.56E-06
	x5	18	52	0.035124	9.23E-07	F	F	F	F	7	25	0.012562	1.42E-06	8	31	0.044412	1.59E-07	64	253	0.21473	9.56E-06
	x6	13	36	0.032072	8.93E-07	F	F	F	F	7	25	0.013074	1.30E-06	8	30	0.02009	1.59E-07	4	12	0.014082	5.76E-06
	x7	18	52	0.052554	9.23E-07	F	F	F	F	7	25	0.021775	1.42E-06	8	31	0.029879	1.59E-07	64	253	0.30531	9.56E-06
	x8	16	45	0.061545	9.95E-07	F	F	F	F	7	25	0.011686	1.30E-06	8	31	0.026899	1.59E-07	4	12	0.017501	5.76E-06
	x9	16	45	0.03167	9.79E-07	F	F	F	F	7	25	0.016309	1.30E-06	8	31	0.030435	1.59E-07	4	12	0.006705	5.76E-06
10000	x1	8	21	0.03621	3.71E-07	18	51	0.043142	7.45E-07	5	17	0.019474	5.06E-06	12	47	0.17017	7.22E-07	5	17	0.022117	2.19E-06
	x2	8	21	0.034409	3.71E-07	15	42	0.037425	7.64E-07	5	17	0.017798	5.06E-06	11	43	0.086668	7.22E-07	5	17	0.020951	2.19E-06
	x3	15	44	0.098121	9.18E-07	36	106	0.14622	9.67E-07	10	38	0.034521	9.06E-06	8	27	0.039777	7.55E-07	5	18	0.031678	3.45E-06
	x4	8	22	0.045275	8.83E-07	196	585	0.52126	9.96E-07	5	17	0.038537	8.38E-06	11	43	0.080798	7.22E-07	85	337	0.29957	9.75E-06
	x5	8	22	0.027345	8.83E-07	50	147	0.15272	9.44E-07	5	17	0.01883	8.38E-06	11	43	0.084839	7.22E-07	85	337	0.32292	9.75E-06
	x6	8	21	0.043909	3.70E-07	210	627	0.52785	9.77E-07	5	17	0.020962	5.06E-06	11	42	0.077928	7.22E-07	5	17	0.024861	2.19E-06
	x7	8	22	0.038238	8.83E-07	20	57	0.051957	8.78E-07	5	17	0.020192	8.38E-06	11	43	0.079948	7.22E-07	85	337	0.50831	9.75E-06
	x8	9	24	0.043123	7.17E-07	208	621	0.40336	9.88E-07	5	17	0.019995	5.06E-06	11	43	0.083423	7.22E-07	5	17	0.021514	2.19E-06
	x9	9	24	0.12251	6.95E-07	208	621	0.39409	9.94E-07	5	17	0.023803	5.06E-06	11	43	0.10931	7.22E-07	5	17	0.027147	2.19E-06
50000	x1	13	37	0.2028	5.34E-07	9	25	0.10676	4.81E-07	8	30	0.11189	5.15E-06	12	48	0.61588	7.59E-07	5	18	0.095206	2.45E-06
	x2	13	37	0.21638	5.34E-07	9	25	0.082291	5.00E-07	8	30	0.11226	5.15E-06	10	40	0.29855	7.59E-07	5	18	0.18925	2.45E-06
	x3	12	33	0.14161	9.21E-07	F	F	F	F	9	33	0.13984	5.57E-06	12	46	0.25481	7.38E-07	7	25	0.13066	6.08E-06
	x4	10	28	0.11798	8.86E-07	F	F	F	F	8	30	0.087649	5.29E-06	11	44	0.43006	7.59E-07	5	18	0.18536	5.53E-06
	x5	10	28	0.17321	8.86E-07	F	F	F	F	8	30	0.13943	5.29E-06	11	44	0.38657	7.59E-07	5	18	0.092437	5.53E-06
	x6	14	40	0.18836	6.47E-07	117	349	0.91182	9.76E-07	8	30	0.088187	5.15E-06	10	39	0.25857	7.59E-07	5	18	0.09337	2.45E-06
	x7	10	28	0.1297	8.86E-07	F	F	F	F	8	30	0.086955	5.29E-06	11	44	0.44874	7.59E-07	5	18	0.15625	5.53E-06
	x8	12	34	0.19733	7.05E-07	F	F	F	F	8	30	0.14503	5.15E-06	11	44	0.37089	7.59E-07	5	18	0.10386	2.45E-06
	x9	11	32	0.18504	7.94E-07	F	F	F	F	8	30	0.093379	5.15E-06	11	44	0.41393	7.59E-07	5	18	0.078907	2.45E-06
100000	x1	9	25	0.21361	3.85E-07	46	136	0.8254	9.31E-07	6	22	0.12967	6.81E-07	14	56	1.6521	2.19E-07	4	14	0.15074	2.67E-06
	x2	9	25	0.18318	3.85E-07	68	202	1.5356	9.35E-07	6	22	0.16861	6.81E-07	9	36	0.75632	2.19E-07	4	14	0.16524	2.70E-06
	x3	12	34	0.25992	7.10E-07	74	219	1.4336	9.65E-07	7	25	0.17283	1.06E-06	11	42	0.55542	9.91E-07	9	33	0.23368	5.47E-06
	x4	10	28	0.22578	4.70E-07	885	2653	20.7608	9.96E-07	6	22	0.13393	9.73E-07	11	44	1.1324	2.19E-07	8	30	0.28851	8.60E-06
	x5	10	28	0.23293	4.70E-07	887	2659	22.9501	9.99E-07	6	22	0.13266	9.73E-07	11	44	1.2224	2.19E-07	8	30	0.28145	8.60E-06
	x6	12	34	0.30496	8.00E-07	92	274	2.0913	9.82E-07	6	22	0.18112	6.81E-07	9	35	0.45198	2.21E-07	4	14	0.16188	2.70E-06
	x7	10	28	0.20114	4.70E-07	113	337	2.3221	9.65E-07	6	22	0.15632	9.73E-07	11	44	1.0635	2.19E-07	8	30	0.28902	8.60E-06
	x8	10	28	0.2406	5.07E-07	83	247	1.6454	9.53E-07	6	22	0.17352	6.81E-07	11	44	1.176	2.19E-07	4	14	0.13366	2.70E-06
	x9	10	28	0.20577	5.22E-07	121	361	2.7946	9.72E-07	6	22	0.16007	6.81E-07	11	44	1.1805	2.19E-07	4	14	0.17212	2.70E-06

Table 11.

Numerical results for Problem 9.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	8	24	0.005028	5.64E-07	23	68	0.13191	4.46E-07	9	35	0.003296	5.48E-06	11	44	0.006284	4.01E-07	9	35	0.003833	2.48E-06
	x2	8	23	0.008476	8.64E-07	21	62	0.006679	9.28E-07	9	35	0.002772	2.15E-06	11	44	0.008552	1.57E-07	8	31	0.00506	6.41E-06
	x3	8	24	0.005569	3.08E-07	22	65	0.006972	5.61E-07	9	35	0.003386	3.00E-06	11	44	0.010073	2.19E-07	8	31	0.005208	8.93E-06
	x4	8	24	0.0066	2.82E-07	22	65	0.011645	5.14E-07	9	35	0.004007	2.74E-06	11	44	0.008651	2.01E-07	8	31	0.008581	8.18E-06
	x5	8	24	0.009222	2.82E-07	22	65	0.020597	5.14E-07	9	35	0.005052	2.74E-06	11	44	0.007554	2.01E-07	8	31	0.009496	8.18E-06
	x6	8	24	0.006655	3.04E-07	22	65	0.00662	5.53E-07	9	35	0.002685	2.96E-06	11	44	0.009785	2.16E-07	8	31	0.003949	8.81E-06
	x7	8	24	0.005654	2.82E-07	22	65	0.007176	5.14E-07	9	35	0.002847	2.74E-06	11	44	0.006838	2.01E-07	8	31	0.004304	8.18E-06
	x8	8	24	0.013108	2.83E-07	22	65	0.007833	5.15E-07	9	35	0.004705	2.75E-06	11	44	0.009929	2.01E-07	8	31	0.00625	8.19E-06
	x9	8	24	0.008784	2.86E-07	22	65	0.010065	5.19E-07	9	35	0.003325	2.79E-06	11	44	0.005355	1.98E-07	8	31	0.004954	8.01E-06
5000	x1	9	26	0.0176	4.91E-07	23	68	0.039317	9.97E-07	10	39	0.01228	2.07E-06	11	44	0.024799	8.97E-07	9	35	0.02265	5.55E-06
	x2	8	24	0.02893	4.95E-07	22	65	0.03288	9.01E-07	9	35	0.012397	4.81E-06	11	44	0.020903	3.52E-07	9	35	0.2018	2.18E-06
	x3	8	24	0.074084	6.90E-07	23	68	0.023742	5.45E-07	9	35	0.012145	6.71E-06	11	44	0.031888	4.90E-07	9	35	0.032921	3.03E-06
	x4	8	24	0.021806	6.32E-07	23	68	0.022984	4.99E-07	9	35	0.01087	6.14E-06	11	44	0.022776	4.49E-07	9	35	0.020182	2.78E-06
	x5	8	24	0.024323	6.32E-07	23	68	0.024503	4.99E-07	9	35	0.011485	6.14E-06	11	44	0.031036	4.49E-07	9	35	0.046506	2.78E-06
	x6	8	24	0.013959	6.87E-07	23	68	0.064772	5.43E-07	9	35	0.013674	6.68E-06	11	44	0.021317	4.89E-07	9	35	0.018303	3.02E-06
	x7	8	24	0.016761	6.32E-07	23	68	0.036919	4.99E-07	9	35	0.017811	6.14E-06	11	44	0.031354	4.49E-07	9	35	0.023681	2.78E-06
	x8	8	24	0.016687	6.32E-07	23	68	0.047002	5.00E-07	9	35	0.014039	6.14E-06	11	44	0.14942	4.49E-07	9	35	0.013313	2.78E-06
	x9	8	24	0.020212	6.27E-07	23	68	0.022977	5.04E-07	9	35	0.017948	6.10E-06	11	44	0.025546	4.48E-07	9	35	0.017248	2.78E-06
10000	x1	9	26	0.078489	6.94E-07	24	71	0.12993	6.13E-07	10	39	0.020771	2.93E-06	12	48	0.05529	2.51E-07	9	35	0.023868	7.85E-06
	x2	8	24	0.020379	7.00E-07	23	68	0.12961	5.53E-07	9	35	0.03616	6.80E-06	11	44	0.045465	4.97E-07	9	35	0.02255	3.08E-06
	x3	8	24	0.03029	9.76E-07	23	68	0.052404	7.71E-07	9	35	0.020414	9.48E-06	11	44	0.042797	6.93E-07	9	35	0.032979	4.29E-06
	x4	8	24	0.027569	8.93E-07	23	68	0.074305	7.06E-07	9	35	0.01837	8.68E-06	11	44	0.044082	6.35E-07	9	35	0.020341	3.93E-06
	x5	8	24	0.025422	8.93E-07	23	68	0.13022	7.06E-07	9	35	0.075188	8.68E-06	11	44	0.04152	6.35E-07	9	35	0.023121	3.93E-06
	x6	8	24	0.02649	9.74E-07	23	68	0.073075	7.70E-07	9	35	0.01947	9.47E-06	11	44	0.045968	6.92E-07	9	35	0.026716	4.28E-06
	x7	8	24	0.024928	8.93E-07	23	68	0.064781	7.06E-07	9	35	0.027659	8.68E-06	11	44	0.04163	6.35E-07	9	35	0.022106	3.93E-06
	x8	8	24	0.030918	8.93E-07	23	68	0.16717	7.06E-07	9	35	0.023928	8.69E-06	11	44	0.052877	6.35E-07	9	35	0.023894	3.93E-06
	x9	8	24	0.029869	8.95E-07	23	68	0.080044	7.05E-07	9	35	0.027406	8.78E-06	11	44	0.053556	6.35E-07	9	35	0.022737	3.96E-06
50000	x1	9	27	0.10307	3.98E-07	25	74	0.1506	5.95E-07	10	39	0.067349	6.54E-06	14	56	0.21988	2.30E-07	10	39	0.11373	2.67E-06
	x2	9	26	0.088932	6.09E-07	24	71	0.17024	5.37E-07	10	39	0.086555	2.57E-06	12	48	0.17181	2.20E-07	9	35	0.065306	6.88E-06
	x3	9	26	0.095518	8.49E-07	24	71	0.15343	7.49E-07	10	39	0.070467	3.58E-06	12	48	0.15512	3.07E-07	9	35	0.10325	9.60E-06
	x4	9	26	0.07146	7.77E-07	24	71	0.15237	6.86E-07	10	39	0.094863	3.28E-06	12	48	0.18387	2.81E-07	9	35	0.071763	8.79E-06
	x5	9	26	0.088237	7.77E-07	24	71	0.15077	6.86E-07	10	39	0.11394	3.28E-06	12	48	0.16862	2.81E-07	9	35	0.068275	8.79E-06
	x6	9	26	0.076803	8.48E-07	24	71	0.29171	7.49E-07	10	39	0.081884	3.58E-06	12	48	0.1473	3.07E-07	9	35	0.11086	9.59E-06
	x7	9	26	0.078565	7.77E-07	24	71	0.28604	6.86E-07	10	39	0.067458	3.28E-06	12	48	0.17418	2.81E-07	9	35	0.076509	8.79E-06
	x8	9	26	0.086736	7.77E-07	24	71	0.15309	6.86E-07	10	39	0.072279	3.28E-06	12	48	0.21846	2.81E-07	9	35	0.081581	8.79E-06
	x9	9	26	0.097146	7.75E-07	24	71	0.32932	6.89E-07	10	39	0.1241	3.29E-06	12	48	0.15526	2.81E-07	9	35	0.06398	8.80E-06
100000	x1	9	27	0.11399	5.62E-07	25	74	0.6106	8.41E-07	10	39	0.17139	9.25E-06	14	56	0.44534	3.25E-07	10	39	0.18755	3.77E-06
	x2	9	26	0.15248	8.61E-07	24	71	0.28022	7.60E-07	10	39	0.18473	3.63E-06	12	48	0.2881	3.12E-07	9	35	0.12208	9.74E-06
	x3	9	27	0.18602	3.08E-07	25	74	0.31581	4.60E-07	10	39	0.16699	5.06E-06	12	48	0.29146	4.35E-07	10	39	0.16077	2.06E-06
	x4	9	27	0.1428	2.82E-07	24	71	0.48131	9.70E-07	10	39	0.12892	4.63E-06	12	48	0.3706	3.98E-07	10	39	0.18411	1.89E-06
	x5	9	27	0.17049	2.82E-07	24	71	0.40206	9.70E-07	10	39	0.15327	4.63E-06	12	48	0.33931	3.98E-07	10	39	0.13712	1.89E-06
	x6	9	27	0.16253	3.08E-07	25	74	0.27723	4.60E-07	10	39	0.20672	5.06E-06	12	48	0.3074	4.34E-07	10	39	0.13456	2.06E-06
	x7	9	27	0.14438	2.82E-07	24	71	0.26333	9.70E-07	10	39	0.15905	4.63E-06	12	48	0.44484	3.98E-07	10	39	0.13312	1.89E-06
	x8	9	27	0.14597	2.82E-07	24	71	0.24372	9.70E-07	10	39	0.15671	4.63E-06	12	48	0.35288	3.98E-07	10	39	0.19196	1.89E-06
	x9	9	27	0.13731	2.81E-07	24	71	0.27596	9.69E-07	10	39	0.14325	4.64E-06	12	48	0.31933	3.98E-07	10	39	0.13545	1.89E-06

Table 12.

Numerical results for Problem 10.

DIM	IP	HLSFR				CGD				PCG				PDY				ACGD
		NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM	NI	NF	CPU	NORM
1000	x1	1	3	0.003912	0	166	497	0.69036	9.13E-07	31	123	0.25366	6.24E-06	13	52	0.51624	3.30E-07	1	4	0.12136	0
	x2	1	3	0.004081	0	152	455	0.067494	9.17E-07	9	35	0.078956	4.99E-06	13	52	0.28982	2.20E-07	9	35	0.13693	2.37E-06
	x3	1	3	0.005348	2.22E-16	134	401	0.076169	9.21E-07	24	96	0.053115	8.77E-06	12	48	0.63759	4.01E-07	14	56	0.059274	6.22E-06
	x4	16	47	0.040778	7.31E-07	165	494	0.066906	9.31E-07	30	119	0.026624	8.24E-06	16	64	0.050373	6.16E-07	17	67	0.011662	7.71E-06
	x5	16	47	0.031295	7.31E-07	165	494	0.064173	9.31E-07	30	119	0.080555	8.24E-06	16	64	0.17673	6.16E-07	17	67	0.35772	7.71E-06
	x6	13	39	0.031615	7.52E-07	141	422	0.060847	9.25E-07	25	99	0.033508	7.79E-06	17	68	0.036626	8.54E-07	14	55	0.011219	9.07E-06
	x7	16	47	0.02627	7.31E-07	165	494	0.15999	9.31E-07	30	119	0.024998	8.24E-06	16	64	0.029012	6.16E-07	17	67	0.016175	7.71E-06
	x8	16	47	0.02149	7.31E-07	165	494	0.12262	9.32E-07	30	119	0.016311	8.24E-06	16	64	0.037983	7.56E-07	17	67	0.060734	7.73E-06
	x9	16	47	0.040906	6.92E-07	165	494	0.068262	9.35E-07	30	119	0.01445	8.26E-06	16	64	0.030959	7.35E-07	17	67	0.038495	7.76E-06
5000	x1	1	3	0.010604	0	173	518	0.27385	9.76E-07	32	127	0.088049	8.31E-06	13	52	0.11491	7.38E-07	1	4	0.008497	0
	x2	1	3	0.006858	0	159	476	0.25698	9.80E-07	10	39	0.029317	2.23E-06	13	52	0.093039	4.93E-07	9	35	0.026642	5.30E-06
	x3	1	3	0.006665	2.22E-16	134	401	0.19158	9.21E-07	24	96	0.082569	8.77E-06	12	48	0.067394	4.01E-07	14	56	0.058619	6.22E-06
	x4	16	48	0.046446	8.02E-07	172	515	0.26576	9.97E-07	32	127	0.084467	6.53E-06	17	68	0.14568	3.10E-07	18	71	0.044645	6.79E-06
	x5	16	48	0.056196	8.02E-07	172	515	0.39559	9.97E-07	32	127	0.14004	6.53E-06	17	68	0.11512	3.10E-07	18	71	0.096528	6.79E-06
	x6	13	39	0.085966	7.52E-07	141	422	0.29676	9.26E-07	25	99	0.056093	7.80E-06	17	68	0.10168	4.89E-07	14	55	0.037525	9.08E-06
	x7	16	48	0.054039	8.02E-07	172	515	0.26396	9.97E-07	32	127	0.074037	6.53E-06	17	68	0.1111	3.10E-07	18	71	0.042014	6.79E-06
	x8	16	48	0.059202	8.02E-07	172	515	0.27002	9.97E-07	32	127	0.091706	6.53E-06	17	68	0.13293	3.22E-07	18	71	0.097957	6.80E-06
	x9	16	48	0.052719	8.25E-07	172	515	0.35188	9.89E-07	32	127	0.096698	6.52E-06	17	68	0.17819	2.95E-07	18	71	0.052604	6.81E-06
10000	x1	1	3	0.019663	0	177	530	0.46763	9.06E-07	33	131	0.12698	7.00E-06	14	56	0.32629	2.26E-07	1	4	0.011638	0
	x2	1	3	0.014307	0	163	488	0.43325	9.10E-07	10	39	0.075608	3.15E-06	13	52	0.12745	6.97E-07	9	35	0.03964	7.49E-06
	x3	1	3	0.011506	2.22E-16	134	401	0.34974	9.21E-07	24	96	0.094225	8.77E-06	12	48	0.13733	4.01E-07	14	56	0.052292	6.22E-06
	x4	17	50	0.067805	6.93E-07	176	527	0.43952	9.25E-07	32	127	0.15461	9.24E-06	17	68	0.17408	4.43E-07	18	71	0.091337	9.61E-06
	x5	17	50	0.094204	6.93E-07	176	527	0.5584	9.25E-07	32	127	0.12309	9.24E-06	17	68	0.2895	4.43E-07	18	71	0.07598	9.61E-06
	x6	13	39	0.067243	7.52E-07	141	422	0.37855	9.26E-07	25	99	0.098141	7.80E-06	17	68	0.21341	4.50E-07	14	55	0.071792	9.08E-06
	x7	17	50	0.086087	6.93E-07	176	527	0.45244	9.25E-07	32	127	0.12887	9.24E-06	17	68	0.22261	4.43E-07	18	71	0.084721	9.61E-06
	x8	17	50	0.10848	6.93E-07	176	527	0.44368	9.25E-07	32	127	0.14797	9.24E-06	17	68	0.18262	4.50E-07	18	71	0.07619	9.61E-06
	x9	17	50	0.084791	7.39E-07	176	527	0.66094	9.24E-07	32	127	0.1238	9.25E-06	17	68	0.19609	4.36E-07	18	71	0.0846	9.64E-06
50000	x1	1	3	0.040787	0	184	551	2.0394	9.69E-07	34	135	0.59763	9.32E-06	16	64	0.87143	3.29E-07	1	4	0.037983	0
	x2	1	3	0.039382	0	170	509	1.9099	9.73E-07	10	39	0.12867	7.04E-06	14	56	0.60424	3.78E-07	10	39	0.21072	3.00E-06
	x3	1	3	0.035945	2.22E-16	134	401	1.2734	9.21E-07	24	96	0.31854	8.77E-06	12	48	0.36476	4.01E-07	14	56	0.24291	6.22E-06
	x4	17	51	0.26368	7.61E-07	183	548	1.8698	9.89E-07	34	135	0.56398	7.32E-06	16	64	0.68032	5.90E-07	19	75	0.25298	8.44E-06
	x5	17	51	0.23584	7.61E-07	183	548	1.8812	9.89E-07	34	135	0.54906	7.32E-06	16	64	0.67644	5.90E-07	19	75	0.28726	8.44E-06
	x6	13	39	0.20824	7.52E-07	141	422	1.6009	9.26E-07	25	99	0.42261	7.80E-06	17	68	0.62999	4.20E-07	14	55	0.18619	9.08E-06
	x7	17	51	0.25547	7.61E-07	183	548	1.8631	9.89E-07	34	135	0.55708	7.32E-06	16	64	0.65903	5.90E-07	19	75	0.31361	8.44E-06
	x8	17	51	0.24578	7.61E-07	183	548	3.6494	9.89E-07	34	135	0.44942	7.32E-06	16	64	0.68492	5.81E-07	19	75	0.31555	8.44E-06
	x9	17	51	0.27934	7.54E-07	183	548	2.0495	9.90E-07	34	135	0.54755	7.34E-06	16	64	0.59958	5.81E-07	19	76	0.3564	8.42E-06
100000	x1	1	3	0.06954	0	187	560	3.8112	9.99E-07	35	139	1.3182	7.85E-06	16	64	1.6176	4.65E-07	1	4	0.096628	0
	x2	1	3	0.060857	0	174	521	3.8221	9.03E-07	10	39	0.30533	9.96E-06	14	56	1.1566	5.34E-07	10	39	0.34704	4.24E-06
	x3	1	3	0.067384	2.22E-16	134	401	2.7412	9.21E-07	24	96	0.6021	8.77E-06	12	48	0.78582	4.01E-07	14	56	0.62844	6.22E-06
	x4	18	53	0.4831	6.58E-07	187	560	3.8213	9.18E-07	35	139	1.0063	6.17E-06	16	64	1.2757	8.35E-07	20	79	0.79438	4.68E-06
	x5	18	53	0.46322	6.58E-07	187	560	3.835	9.18E-07	35	139	1.256	6.17E-06	16	64	1.5281	8.35E-07	20	80	0.73412	4.68E-06
	x6	13	39	0.33844	7.52E-07	141	422	2.912	9.26E-07	25	99	0.71975	7.80E-06	17	68	1.408	4.17E-07	14	55	0.5084	9.08E-06
	x7	18	53	0.47895	6.58E-07	187	560	4.0575	9.18E-07	35	139	1.242	6.17E-06	16	64	1.3442	8.35E-07	20	80	0.7918	4.68E-06
	x8	18	53	0.48391	6.58E-07	187	560	4.1165	9.18E-07	35	139	1.0061	6.17E-06	16	64	1.4101	8.22E-07	20	80	0.72111	4.68E-06
	x9	18	54	0.49485	3.22E-07	187	560	3.9494	9.19E-07	35	139	1.1207	6.16E-06	16	64	1.3269	8.21E-07	20	80	0.81733	4.68E-06