Derivative-free HS-DY-type method for solving nonlinear equations and image restoration

Abstract

A derivative-free conjugate gradient algorithm for solving nonlinear equations and image restoration is proposed. The conjugate gradient (CG) parameter of the proposed algorithm is a convex combination of Hestenes-Stiefel (HS) and Dai-Yuan (DY) type CG parameters. The search direction is descent and bounded. Under suitable assumptions, the convergence of the proposed hybrid algorithm is obtained. Using some benchmark test problems, the proposed algorithm is shown to be efficient compared with existing algorithms. In addition, the proposed algorithm is effectively applied to solve image restoration problems.

Mathematics; Nonlinear equations; Conjugate gradient; Projection method; Image restoration

1. Introduction

Conjugate gradient (CG) method is a well known method that efficiently solves nonlinear equations of the form

$$H(x)=0,x\in A,$$ (1)

where $H:R^n\to R^n$ and A is a closed and convex subset of the Euclidean space $R^n$. The CG algorithm generates an iterative sequence $\{x_k\}$ via the following formula:

$$x_{k+1}=x_k+{\alpha }_k{d}_k,$$

where ${\alpha }_k$ is the step size obtained from a suitable line search process and $d_k$ is the search direction defined as

$$d_k=-H_k,k=0,$$ (2)

$$d_k=-H_k+{\beta }_k{d}_{k-1},k>0.$$ (3)

The parameter ${\beta }_k$ is called the CG parameter. Throughout this paper, $H_k$ denotes the function evaluation of H at $x_k$, and $\langle \cdot ,\cdot \rangle$ denotes the inner product.

Several CG algorithms for solving (1) have been proposed in literature. For example, Feng et al. [16] proposed a CG based algorithm where the search direction is defined as

$$d_k:=\{\begin{array}{cc}-H_k,\hfill & \text{if }k=0,\hfill \\ -(1+{\beta }_k\frac{\langle H_k,d_{k-1}\rangle }{{\Vert H_k\Vert }^2})H_k+{\beta }_k{d}_{k-1},\hfill & \text{if }k\ge 1,\hfill \end{array}$$ (4)

where ${\beta }_k:=\frac{\Vert H_k\Vert }{\Vert d_{k-1}\Vert }$.

In [27], Liu and Feng proposed a search direction defined as

$$d_k:=\{\begin{array}{cc}-H_k,\hfill & \text{if }k=0,\hfill \\ -{\theta }_k{H}_k+{\beta }_k{d}_{k-1},\hfill & \text{if }k\ge 1,\hfill \end{array}$$ (5)

where

$${\beta }_k:=\frac{{\Vert H_k\Vert }^2}{\langle d_{k-1},w_{k-1}\rangle },{\theta }_k:=c-\frac{\langle H_k,d_{k-1}\rangle }{\langle d_{k-1},w_{k-1}\rangle },$$

$$y_{k-1}:=H_k-H_{k-1}+r{s}_{k-1},s_{k-1}:=x_k-x_{k-1},$$

$$w_{k-1}:=y_{k-1}+t_{k-1}{d}_{k-1},t_{k-1}:=1+\max \{0,-\frac{\langle d_{k-1},y_{k-1}\rangle }{{\Vert d_{k-1}\Vert }^2}\}.$$

The algorithm in [27] was shown to be efficient for solving convex constrained monotone equations. Awwal et al. [10] also proposed a modified HS conjugate gradient algorithm for solving problem (1) as well as signal recovery problem. The search direction is defined as

$$d_k:=\{\begin{array}{cc}-H_k,\hfill & \text{if}k=0,\hfill \\ -{\lambda }_k{H}_k+{\overline{\beta }}_k^{MHS+}{d}_{k-1},\hfill & \text{if}k\ge 1,\hfill \end{array}$$ (6)

where

$${\beta }_k^{MHS+}:=\max \{{\overline{\beta }}_k^{MHS},0\},$$

$${\overline{\beta }}_k^{MHS}:=\frac{{\Vert s_{k-1}\Vert }^2\langle H_k,y_{k-1}\rangle }{\langle s_{k-1},y_{k-1}\rangle \langle d_{k-1},y_{k-1}\rangle }{\theta }_k-\gamma \times {\left(\frac{\Vert s_{k-1}\Vert \Vert y_{k-1}\Vert {\theta }_k}{\langle d_{k-1},y_{k-1}\rangle }\right)}^2\frac{\langle H_k,d_{k-1}\rangle }{\langle s_{k-1},d_{k-1}\rangle },\gamma >\frac{1}{4},$$

$${\lambda }_k:=\frac{{\Vert s_{k-1}\Vert }^2}{\langle s_{k-1},y_{k-1}\rangle },{\theta }_k:=1-\frac{{(H_k^T{d}_{k-1})}^2}{{\Vert H_k|\Vert }^2{\Vert d_{k-1}\Vert }^2},y_{k-1}:=H_k-H_{k-1}+\eta s_{k-1},s_{k-1}:=x_k-x_{k-1},\eta >0.$$

In the same line of research, Awwal et al. [11] further proposed a modified Polak–Ribière–Polyak (PRP) conjugate gradient algorithm with search direction given by

$$d_k:=\{\begin{array}{cc}-H_k,\hfill & \text{if}k=0,\hfill \\ -{\theta }_k{H}_k+{\beta }_k^{KPRP}{s}_{k-1},\hfill & \text{if}k\ge 1,\hfill \end{array}$$ (7)

where

$${\theta }_k:={\lambda }_k+\frac{{\beta }_k^{KPRP}\langle H_k,s_{k-1}\rangle }{{\Vert H_k\Vert }^2},{\lambda }_k:=\frac{{\Vert s_{k-1}\Vert }^2}{\langle s_{k-1},{\psi }_{k-1}\rangle },{\psi }_{k-1}:=H_k-H_{k-1}+\eta s_{k-1},s_{k-1}:=x_k-x_{k-1},\eta >0.$$

Recently, Yuan et al. [34] proposed a conjugate gradient algorithm which is a convex combination of the steepest descent algorithm and a modified Liu-Storey (LS) conjugate gradient algorithm. The search direction defined by Yuan et al. is

$$d_k:=\{\begin{array}{cc}-H_k,\hfill & \text{if}k=0,\hfill \\ -N_k{H}_k+(1-N_k)\frac{\langle H_k,v_{k-1}\rangle d_{k-1}-\langle H_k,d_{k-1}\rangle v_{k-1}}{\max \{2\chi \Vert d_{k-1}\Vert \Vert v_{k-1}\Vert ,-\langle H_{k-1},d_{k-1}\rangle \}},\hfill & \text{if}k\ge 1,\hfill \end{array}$$ (8)

where

$$N_k:=\frac{{\Vert v_{k-1}\Vert }^2}{\langle v_{k-1},u_{k-1}\rangle },u_{k-1}:=s_{k-1}+\max \{0,\frac{-\langle s_{k-1},v_{k-1}\rangle }{{\Vert v_{k-1}\Vert }^2}\}{v}_{k-1}+v_{k-1},\chi \in (0,1),$$

$$s_{k-1}:=x_k-x_{k-1},v_{k-1}:=H_k-H_{k-1}.$$

For more on the conjugate gradient algorithms, the interested reader is referred to [1], [2], [3], [4], [5], [6], [7], [8], [9], [18], [19], [20], [21], [22], [26], [29], [35].

To the best of our knowledge, very few hybrid algorithms for solving (1) are available in the literature. To this end, we explore the strong convergence property of the DY algorithm together with the good practical behavior of the HS algorithm by proposing a conjugate gradient method with a conjugate gradient parameter computed as a convex combination of a modified HS and DY parameters. Furthermore, the hybrid method is applied to solve image restoration problem arising in compressive sensing.

Section 2 highlights the reason behind modifying the HS and DY parameters, suggesting the modification and its advantages, and then describing the proposed algorithm. Section 3 gives some nice properties of the search direction and the convergence analysis of the proposed algorithm. Numerical experiments on some benchmark test problems for solving (1) and image restoration problem are given in Section 4. Finally, Section 5 concludes the paper.

2. Algorithm

This section presents a hybrid conjugate gradient algorithm for solving (1). The CG parameter of the algorithm is a convex combination of a modified HS and DY conjugate gradient parameters, respectively.

We begin by recalling the classical HS and DY conjugate gradient parameters defined as

$${\beta }_k^{HS}:=\frac{\langle H_k,y_{k-1}\rangle }{\langle d_{k-1},y_{k-1}\rangle }$$ (9)

and

$${\beta }_k^{DY}:=\frac{{\Vert H_k\Vert }^2}{\langle d_{k-1},y_{k-1}\rangle },$$ (10)

respectively. Note that if $\langle d_{k-1},y_{k-1}\rangle =0$, then both parameters in (9) and (10) will be undefined. In order to avoid such situation, we replace $\langle d_{k-1},y_{k-1}\rangle$ by $\langle d_{k-1},w_{k-1}\rangle$, where

$$w_{k-1}:=y_{k-1}+t_{k-1}{d}_{k-1},y_{k-1}:=H_k-H_{k-1},t_{k-1}:=1+\max \{0,-\frac{\langle d_{k-1},y_{k-1}\rangle }{{\Vert d_{k-1}\Vert }^2}\}.$$

By the above modification, it is not difficult to see that

$$\langle d_{k-1},w_{k-1}\rangle \ge \langle d_{k-1},y_{k-1}\rangle +{\Vert d_{k-1}\Vert }^2-\langle d_{k-1},y_{k-1}\rangle ={\Vert d_{k-1}\Vert }^2>0,$$ (11)

which means that $\langle d_{k-1},w_{k-1}\rangle \ne 0$ unless the solution of (1) is achieved. Hence we define the modified parameters as

$${\beta }_k^{MHS}:=\frac{\langle H_k,y_{k-1}\rangle }{\langle d_{k-1},w_{k-1}\rangle }$$ (12)

and

$${\beta }_k^{MDY}:=\frac{{\Vert H_k\Vert }^2}{\langle d_{k-1},w_{k-1}\rangle }.$$ (13)

Next we will utilize the good practical performance of ${\beta }_k^{MHS}$ and strong convergence behavior of ${\beta }_k^{MDY}$ by defining a new parameter ${\beta }_k^{HSDY}$ as a convex combination of ${\beta }_k^{MHS}$ and ${\beta }_k^{MDY}$. More precisely,

$${\beta }_k^{HSDY}:=(1-{\theta }_k){\beta }_k^{MHS}+{\theta }_k{\beta }_k^{MDY},$$ (14)

where

$${\theta }_k:=\frac{{\Vert y_{k-1}\Vert }^2}{\langle y_{k-1},{\overline{s}}_{k-1}\rangle }\in (0,1],{\overline{s}}_{k-1}:=s_{k-1}+\max \{0,\frac{-\langle s_{k-1},y_{k-1}\rangle }{{\Vert y_{k-1}\Vert }^2}\}{y}_{k-1}+y_{k-1},s_{k-1}:=x_k-x_{k-1},y_{k-1}:=H_k-H_{k-1}.$$

The parameters ${\overline{s}}_{k-1}$ and ${\theta }_k$ are as defined by Li and Fukushima [25] and Birgin and Martínez [13], respectively. We now propose a hybrid conjugate gradient search direction as

$$d_k:=\{\begin{array}{cc}-H_k,\hfill & \text{if}k=0,\hfill \\ -(1+{\beta }_k^{HSDY}\frac{H_k^T{d}_{k-1}}{{\Vert H_k\Vert }^2})H_k+{\beta }_k^{HSDY}{d}_{k-1},\hfill & \text{if}k\ge 1,\hfill \end{array}$$ (15)

where ${\beta }_k^{HSDY}$ is given by (14).

Remark 2.1

By the definitions of $y_{k-1}$ and ${\overline{s}}_{k-1}$, we have that for $y_{k-1}\ne 0$

$$\langle y_{k-1},{\overline{s}}_{k-1}\rangle \ge \langle y_{k-1},(s_{k-1}-\frac{\langle s_{k-1},y_{k-1}\rangle }{{\Vert y_{k-1}\Vert }^2}{y}_{k-1}+y_{k-1})\rangle =\langle y_{k-1},s_{k-1}\rangle -\frac{\langle s_{k-1},y_{k-1}\rangle }{{\Vert y_{k-1}\Vert }^2}{\Vert y_{k-1}\Vert }^2+{\Vert y_{k-1}\Vert }^2={\Vert y_{k-1}\Vert }^2>0.$$

Therefore,

$$0<\frac{{\Vert y_{k-1}\Vert }^2}{\langle y_{k-1},{\overline{s}}_{k-1}\rangle }\le 1.$$

Remark 2.2

From the definition of ${\beta }_k^{HSDY}$ and ${\theta }_k$, we have

$$|{\beta }_k^{HSDY}|\le |{\beta }_k^{MHS}|+|{\beta }_k^{MDY}|.$$ (16)

To describe the hybrid conjugate gradient algorithm, we first recall the projection map.

Definition 2.3

Let $A\subset R^n$ be a nonempty, closed and convex set. Then for any $x\in R^n$, its projection onto A, denoted by $P_A(x)$, is defined by

$$P_A(x):=\arg \min \{\Vert x-y\Vert :y\in A\}.$$

A known property of $P_A$ is that it is non-expansive, that is,

$$\Vert P_A(x)-P_A(y)\Vert \le \Vert x-y\Vert ,\forall x,y\in R^n.$$ (17)

In what follows, we present the steps of the derivative-free algorithm. Throughout, we refer to the proposed algorithm as Algorithm 1.

Algorithm 1.

Remark 2.1 implies that (14) is a convex combination of (12) and (13).

Remark 2.4

The parameter γ in equation (20) is chosen from the interval $(0,2)$ so as to have the sequence $\{\Vert x_k-\overline{x}\Vert \}$ be non increasing (see Lemma 3.5). In addition, the parameter γ has a significant impact on the numerical performance of Algorithm 1.

3. Convergence analysis

To establish the convergence of Algorithm 1, we begin with the following assumptions:

• $A_1$
  The function H is monotone, that is,

  $$\langle H(x)-H(y),(x-y)\rangle \ge 0,\forall x,y\in R^n.$$
• $A_2$
  The function H is Lipschitz continuous, that is there exists a positive constant L such that

  $$\Vert H(x)-H(y)\Vert \le L\Vert x-y\Vert ,\forall x,y\in R^n.$$
• $A_3$The solution set of problem (1) denoted by $A^{\prime }$ is nonempty.
• $A_4$$H_k\ne 0$ unless the solution of (1) is obtained.

Lemma 3.1

Let $d_k$ be defined by (14)-(15), then $d_k$ satisfies the sufficient descent condition. That is

$$\langle H_k,d_k\rangle =-{\Vert H_k\Vert }^2.$$ (21)

Proof

For $k=0$, we have $\langle H_0,d_0\rangle =-{\Vert H_0\Vert }^2$. For $k\ge 1$, by (14)-(15), we get

$$\langle H_k,d_k\rangle =-(1+{\beta }_k^{HSDY}\frac{\langle H_k,d_{k-1}\rangle }{{\Vert H_k\Vert }^2})\langle H_k,H_k\rangle +{\beta }_k^{HSDY}\langle H_k,d_{k-1}\rangle =-{\Vert H_k\Vert }^2-{\beta }_k^{HSDY}\frac{\langle H_k,d_{k-1}\rangle }{{\Vert H_k\Vert }^2}{\Vert H_k\Vert }^2+{\beta }_k^{HSDY}\langle H_k,d_{k-1}\rangle =-{\Vert H_k\Vert }^2-{\beta }_k^{HSDY}\langle H_k,d_{k-1}\rangle +{\beta }_k^{HSDY}\langle H_k,d_{k-1}\rangle =-{\Vert H_k\Vert }^2.$$ (22)

Remark 3.2

From (21), applying Cauchy-Schwartz inequality we have

$$\Vert d_k\Vert \ge \Vert H_k\Vert .$$ (23)

 □

The following Lemma shows that Algorithm 1 is well-defined.

Lemma 3.3

If assumption $A_2$ holds, then there exists a step size ${\alpha }_k={\rho }^i$ satisfying the line search (18) for some $i\in \mathbb{N}\cup \{0\}$ and $\forall k\ge 0$.

Proof

Suppose there exists $k_0\ge 0$ such that (18) does not hold for any non-negative integer i, that is,

$$-\langle H(x_{k_0}+{\rho }^i{d}_{k_0}),d_{k_0}\rangle <\sigma {\rho }^i{\Vert d_{k_0}\Vert }^2.$$

By assumption $A_2$ and allowing $i\to \infty$, we get

$$-\langle H(x_{k_0}),d_{k_0}\rangle \le 0.$$ (24)

Also from (22), we have

$$-\langle H(x_{k_0}),d_{k_0}\rangle \ge {\Vert H(x_{k_0})\Vert }^2>0,$$

which contradicts (24). The proof is complete. □

Lemma 3.4

Suppose assumption $A_2$ holds. If $\{z_k\}$ and $\{x_k\}$ are defined by (19) and (20) in Algorithm 1, then

$${\alpha }_k\ge \max \{1,\frac{\rho {\Vert H_k\Vert }^2}{(L+\sigma ){\Vert d_k\Vert }^2}\}.$$ (25)

Proof

From the line search (18), if ${\alpha }_k\ne 1$, then ${\alpha }_k^{\prime }={\alpha }_k{\rho }^{-1}$ does not satisfy (18), that is,

$$-\langle H(x_k+{\alpha }_k^{\prime }{d}_k),d_k\rangle <\sigma {\alpha }_k^{\prime }{\Vert d_k\Vert }^2.$$

Using (21) and assumption $A_2$, we have

$${\Vert H_k\Vert }^2\le -\langle H_k,d_k\rangle =\langle H(x_k+{\alpha }_k^{\prime }{d}_k)-H_k,d_k\rangle -\langle H(x_k+{\alpha }_k^{\prime }{d}_k),d_k\rangle \le {\alpha }_k^{\prime }(L+\sigma ){\Vert d_k\Vert }^2.$$

Solving the above inequality for ${\alpha }_k^{\prime }$, the desired result is obtained. □

Lemma 3.5

Let assumptions $A_1$-$A_3$ be fulfilled. If $\{z_k\}$ and $\{x_k\}$ are sequences defined by (19) and (20) in Algorithm 1, then $\{z_k\}$ and $\{x_k\}$ are bounded. Furthermore,

$$\underset{k\to \infty }{\lim }\Vert x_k-z_k\Vert =0,$$ (26)

and

$$\underset{k\to \infty }{\lim }\Vert x_{k+1}-x_k\Vert =0.$$ (27)

Proof

We begin by showing that the sequence $\{x_k\}$ and $\{z_k\}$ are bounded. Suppose $\overline{x}\in A^{\prime }$, then by monotonicity of H, we get

$$\langle H(z_k),x_k-\overline{x}\rangle \ge \langle H(z_k),x_k-z_k\rangle .$$ (28)

From the definition of $z_k$ and (18), we have

$$\langle H(z_k),x_k-z_k\rangle \ge \sigma {\alpha }_k^2{\Vert d_k\Vert }^2\ge 0.$$ (29)

Consequently, by (17), (28), (29), the definition of ${\zeta }_k$ and $\gamma \in (0,2)$, we have

$$\begin{gathered} {\Vert x_{k+1}-\overline{x}\Vert }^2 \\ ={\Vert P_A[x_k-\gamma {\zeta }_{k}H(z_k)]-P_A(\overline{x})\Vert }^2 \\ \le {\Vert x_k-\gamma {\zeta }_{k}H(z_k)-\overline{x}\Vert }^2 \\ ={\Vert x_k-\overline{x}\Vert }^2-2\gamma {\zeta }_k\langle H(z_k),x_k-\overline{x}\rangle +{\Vert \gamma {\zeta }_{k}H(z_k)\Vert }^2 \\ ={\Vert x_k-\overline{x}\Vert }^2-2\gamma \frac{\langle H(z_k),x_k-z_k\rangle }{{\Vert H(z_k)\Vert }^2}\langle H(z_k),x_k-\overline{x}\rangle +{\gamma }^2{\left(\frac{\langle H(z_k),x_k-z_k\rangle }{\Vert H(z_k)\Vert }\right)}^2 \\ \le {\Vert x_k-\overline{x}\Vert }^2-2\gamma \frac{\langle H(z_k),x_k-z_k\rangle }{{\Vert H(z_k)\Vert }^2}\langle H(z_k),x_k-z_k\rangle +{\gamma }^2{\left(\frac{\langle H(z_k),x_k-z_k\rangle }{\Vert H(z_k)\Vert }\right)}^2 \\ \le {\Vert x_k-\overline{x}\Vert }^2-\gamma (2-\gamma ){\left(\frac{\langle H(z_k),x_k-z_k\rangle }{\Vert H(z_k)\Vert }\right)}^2 \\ \le {\Vert x_k-\overline{x}\Vert }^2-\gamma (2-\gamma )\frac{{\sigma }^2{\Vert x_k-z_k\Vert }^4}{{\Vert H(z_k)\Vert }^2}. \end{gathered}$$ (30)

Thus, the sequence $\{\Vert x_k-\overline{x}\Vert \}$ is non increasing and convergent, and hence $\{x_k\}$ is bounded. That is,

$$\Vert x_k\Vert \le b,b>0.$$ (31)

Moreover, from relation (30), we have

$${\Vert x_{k+1}-\overline{x}\Vert }^2\le {\Vert x_k-\overline{x}\Vert }^2,$$ (32)

and we can deduce recursively that

$${\Vert x_k-\overline{x}\Vert }^2\le {\Vert x_0-\overline{x}\Vert }^2,\forall k\ge 0.$$

Therefore from assumption $A_2$, we have that

$$\Vert H_k\Vert =\Vert H_k-H(\overline{x})\Vert \le L\Vert x_k-\overline{x}\Vert \le L\Vert x_0-\overline{x}\Vert .$$

Letting $L\Vert x_0-\overline{x}\Vert =B$, then the sequence $\{H_k\}$ is bounded. That is,

$$\Vert H_k\Vert \le B,\forall k\ge 0.$$ (33)

Now by monotonicity of H,

$$\langle H_k-H(z_k),x_k-z_k\rangle \ge 0,$$

which implies that

$$\langle H_k,x_k-z_k\rangle -\langle H(z_k),x_k-z_k\rangle \ge 0.$$

Hence

$$\langle H(z_k),x_k-z_k\rangle \le \langle H_k,x_k-z_k\rangle .$$ (34)

By the definition of $z_k$, (29), (34) and the Cauchy-Schwartz inequality,

$$\sigma \Vert x_k-z_k\Vert =\frac{\sigma {\Vert x_k-z_k\Vert }^2}{\Vert x_k-z_k\Vert }=\frac{\sigma {\Vert {\alpha }_k{d}_k\Vert }^2}{\Vert x_k-z_k\Vert }\le \frac{\langle H(z_k),x_k-z_k\rangle }{\Vert x_k-z_k\Vert }\le \frac{\langle H_k,x_k-z_k\rangle }{\Vert x_k-z_k\Vert }\le \Vert H_k\Vert .$$ (35)

By (35) and the reverse triangle inequality,

$$\sigma (\Vert z_k\Vert -\Vert x_k\Vert )\le \sigma \Vert z_k-x_k\Vert \le \Vert H_k\Vert .$$

The above relation together with (31) and (33) yield

$$\Vert z_k\Vert \le \frac{1}{\sigma }\Vert H_k\Vert +\Vert x_k\Vert \le \frac{1}{\sigma }B+b.$$

Therefore the sequence $\{z_k\}$ is bounded.

Now, for any $\overline{x}\in A^{\prime }$, the sequence $\{z_k-\overline{x}\}$ is also bounded, that is, there exists a positive constant $\nu >0$ such that

$$\Vert z_k-\overline{x}\Vert \le \nu ,\forall k\ge 0.$$

The above inequality together with assumption $A_2$ yield

$$\Vert H(z_k)\Vert =\Vert H(z_k)-H(\overline{x})\Vert \le L\Vert z_k-\overline{x}\Vert \le L\nu .$$

Therefore, using relation (30), we have

$$\gamma (2-\gamma )\frac{{\sigma }^2}{{(L\nu )}^2}{\Vert x_k-z_k\Vert }^4\le {\Vert x_k-\overline{x}\Vert }^2-{\Vert x_{k+1}-\overline{x}\Vert }^2,$$

which implies

$$\begin{gathered} \gamma (2-\gamma )\frac{{\sigma }^2}{{(L\nu )}^2}\sum _{k=0}^{\infty }{\Vert x_k-z_k\Vert }^4 \\ \le \sum _{k=0}^{\infty }({\Vert x_k-\overline{x}\Vert }^2-{\Vert x_{k+1}-\overline{x}\Vert }^2)={\Vert x_0-\overline{x}\Vert }^2<\infty . \end{gathered}$$ (36)

Relation (36) implies that

$$\underset{k\to \infty }{\lim }\Vert x_k-z_k\Vert =0.$$

In addition, using (17), the definition of ${\zeta }_k$ and the Cauchy-Schwartz inequality,

$$\Vert x_{k+1}-x_k\Vert =\Vert P_A[x_k-\gamma {\zeta }_{k}H(z_k)]-P_A(x_k)\Vert \le \Vert x_k-\gamma {\zeta }_{k}H(z_k)-x_k\Vert =\Vert \gamma {\zeta }_{k}H(z_k)\Vert \le \gamma \Vert x_k-z_k\Vert ,\forall k\ge 0.$$ (37)

It follows that

$$\underset{k\to \infty }{\lim }\Vert x_{k+1}-x_k\Vert =0.$$

 □

Remark 3.6

From (26) and definition of $z_k$,

$$\underset{k\to \infty }{\lim }{\alpha }_k\Vert d_k\Vert =0.$$ (38)

Theorem 3.7

Let the sequence $\{x_k\}$ be generated by (20) in Algorithm 1, then

$$\underset{k\to \infty }{\lim \inf }\Vert H_k\Vert =0.$$ (39)

Proof

Suppose by contradiction that (39) is not true, then there exist $r_0>0$ such that $\forall k\ge 0$,

$$\Vert H_k\Vert \ge r_0.$$ (40)

Inequality (23) together with (40) implies that

$$\Vert d_k\Vert \ge r_0\forall k\ge 0.$$ (41)

Now, from (11) (12), (13), (14), (15), (16), (31), (33), (41), assumption $A_2$ and the Cauchy-Schwartz inequality, we get

$$\Vert d_k\Vert =\Vert -(1+{\beta }_k^{HSDY}\frac{\langle H_k,d_{k-1}\rangle }{{\Vert H_k\Vert }^2})H_k+{\beta }_k^{HSDY}{d}_{k-1}\Vert \le \Vert H_k\Vert +|{\beta }_k^{HSDY}|\frac{{\Vert H_k\Vert }^2\Vert d_{k-1}\Vert }{{\Vert H_k\Vert }^2}+|{\beta }_k^{HSDY}|\Vert d_{k-1}\Vert \le \Vert H_k\Vert +2(|{\beta }_k^{MHS}|+|{\beta }_k^{MDY}|)\Vert d_{k-1}\Vert \le \Vert H_k\Vert +2(\frac{\Vert H_k\Vert \Vert y_{k-1}\Vert }{\langle d_{k-1},w_{k-1}\rangle }+\frac{{\Vert H_k\Vert }^2}{\langle d_{k-1},w_{k-1}\rangle })\Vert d_{k-1}\Vert \le \Vert H_k\Vert +2(\frac{\Vert H_k\Vert \Vert y_{k-1}\Vert }{{\Vert d_{k-1}\Vert }^2}+\frac{{\Vert H_k\Vert }^2}{{\Vert d_{k-1}\Vert }^2})\Vert d_{k-1}\Vert =\Vert H_k\Vert +2\Vert H_k\Vert \left(\frac{\Vert y_{k-1}\Vert +\Vert H_k\Vert }{\Vert d_{k-1}\Vert }\right)=(1+\frac{2(\Vert y_{k-1}\Vert +\Vert H_k\Vert )}{\Vert d_{k-1}\Vert })\Vert H_k\Vert \le (1+\frac{2(L\Vert x_k-x_{k-1}\Vert +\Vert H_k\Vert )}{\Vert d_{k-1}\Vert })\Vert H_k\Vert \le (1+\frac{2(L(\Vert x_k\Vert +\Vert x_{k-1}\Vert )+\Vert H_k\Vert )}{\Vert d_{k-1}\Vert })\Vert H_k\Vert \le (1+\frac{4bL+2B}{r_0})B.$$ (42)

Setting $M=(1+\frac{4bL+2B}{r_0})B$, we have

$$\Vert d_k\Vert \le M.$$ (43)

Multiplying both sides of (25) with $\Vert d_k\Vert$ together with (40), (41) and (43), we have

$${\alpha }_k\Vert d_k\Vert \ge \max \{1,\frac{\rho {\Vert H_k\Vert }^2}{(L+\sigma ){\Vert d_k\Vert }^2}\}\Vert d_k\Vert \ge \max \{r_0,\frac{\rho r_0^2}{(L+\sigma )M}\}.$$

The inequality above contradicts (38) and hence (39) holds. □

4. Numerical experiment

In this section, we perform several experiments to investigate the computational efficiency of the hybrid conjugate gradient algorithm. All programs were written in Matlab R2019b and implemented on an Intel(R) Core (TM) i3-7100U CPU @ 2.40GHz, RAM 8.0GB. To measure the algorithm's efficiency, we compare Algorithm 1 called HSDY with CGD [32], PDY [27] and ACGD [14] in terms of number of iterations, number of function evaluations and CPU running time. Our experiment was carried on a set of nine benchmark test problems with the following:

• 1.Initialization: $\rho =0.8$, $\gamma =1.2$, $\sigma ={10}^{-4}$.
• 2.Dimension: $1000,5000,10000,50,000$ and $100,000$.
• 3.Initial points: $x_1={(0.1,0.1,\cdots ,0.1)}^T$, $x_2={(0.2,0.2,\cdots ,0.2)}^T$, $x_3={(0.5,0.5,\cdots ,0.5)}^T$, $x_4={(1.2,1.2,\cdots ,1.2)}^T$, $x_5={(1.5,1.5,\cdots 1.5)}^T$, $x_6={(2,2,\cdots ,2)}^T,x_7=rand(n,1)$.

The algorithms are terminated by reaching a maximum of 1000 iteration or achieving a solution with

$$\Vert H_k\Vert \le {10}^{-6}.$$

Note that the parameters for the algorithms used for comparison are set as reported in the numerical section of their respective papers. We give a list of the benchmark test problems used in our experiment below where the function H is taken as $H(x)={(h_1(x),h_2(x),\cdots ,h_n(x))}^T$ and $x={(x_1,x_2,\cdots ,x_n)}^T$.

Problem 1 [23] Exponential Function.

$$h_1(x)=e^{x_1}-1,h_i(x)=e^{x_i}+x_i-1,\text{for }i=2,3,\dots ,n,\text{and }A=R_{+}^n.$$

Problem 2 [23] Modified Logarithmic Function.

$$h_i(x)=\ln (x_i+1)-\frac{x_i}{n},\text{for }i=1,2,\dots ,n,\text{and }A=\{x\in R^n:\sum _{i=1}^n{x}_i\le n,x_i>-1,i=1,2,\dots ,n\}.$$

Problem 3 [36] Nonsmooth Function.

$$h_i(x)=2x_i-\sin |x_i|,\text{for }i=1,2,\dots ,n,\text{and }A=R_{+}^n.$$

Problem 4 [24]

$$h_i(x)=\min (\min (|x_i|,x_i^2),\max (|x_i|,x_i^3)),\text{for}i=1,2,\dots ,n,\text{and }A=R_{+}^n.$$

Problem 5 [23] Strictly Convex Function I.

$$h_i(x)=e^{x_i}-1,\text{for }i=1,2,\dots ,n,\text{and }A=R_{+}^n.$$

Problem 6 [30] Strictly convex function II.

$$h_i(x)=\frac{i}{n}{e}^{x_i}-1,\text{for }i=1,2,\dots ,n,\text{and }A=R_{+}^n.$$

Problem 7 [12] Tridiagonal Exponential Function.

$$h_1(x)=x_1-e^{\cos (h(x_1+x_2))},h_i(x)=x_i-e^{\cos (h(x_{i-1}+x_i+x_{i+1}))},\text{for }i=2,\dots ,n-1,h_n(x)=x_n-e^{\cos (h(x_{n-1}+x_n))},h=\frac{1}{n+1}\text{ and }A=R_{+}^n.$$

Problem 8 [33] Nonsmooth Function.

$$h_i(x)=x_i-\sin |x_i-1|,\text{for }i=1,2,\dots ,n,\text{and }A=\{x\in R^n:\sum _{i=1}^n{x}_i\le n,x_i\ge -1,i=1,2,\dots ,n\}.$$

Problem 9 [36]

$$h_1(x)=2x_1+\sin x_1-1,h_i(x)=x_{i-1}+2x_i+\sin x_i-1,\text{for }i=2,\dots ,n-1,h_n(x)=2x_n+\sin x_n-1,\text{ and }A=\{x\in R^n:\sum _{i=1}^n{x}_i\le n,x_i\ge 0,i=1,2,\dots ,n\}.$$

Problem 10 Pursuit-Evasion problem.

$$h_i(x)=\sqrt{8}{x}_i-1,\text{for }i=1,2,\dots ,n,\text{and }A=R_{+}^n.$$

The algorithms' numerical results are reported in Table 2, Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11 of the Appendix section, where “ITER” denotes the number of iterations, “FVAL” denotes the number of function evaluations and “TIME” is the CPU running time in seconds. In order to visualize the behavior of HSDY, we employ the Dolan and Morè performance profile tool [15] for efficiency comparison. The performance profile tool seeks to find how well the solvers perform relative to the other solvers on a set of problems based on the total number of iterations, total number of function evaluations, and the CPU running time. We quickly recall this process.

Denote M as the set of the methods, and E as the set of the experiments (the four methods test one problem with the same number of variables and initial point as one experiment). The parameter $t_{m,e}$ means NITER, NF, or TIME of the method $m\in M$ in the e-th experiment. The performance ratio is computed as $r_{m,e}=t_{m,e}/mi{n}_{m\in M}{t}_{m,e}$. Then the performance profile is determined by

$${\chi }_m(\tau ):=\frac{1}{n_e}size\{e\in E|{\log }_2(r_{m,e})\le \tau ,l\le m\le n_m\},\forall \tau \in R^{+},$$

where $n_m$ denotes the number of methods in the set M. Obviously, the function ${\chi }_m:R\to [0,1]$ is a distribution function for the performance ration. And for any $m\in M$, ${\chi }_m$ is a non-decreasing, piecewise constant, continuous function from the right at each breakpoint. Moreover, ${\chi }_m(\tau )$ is the probability for the method $m\in M$ that ${\log }_2(r_{m,e})$ is within a factor $\tau \in R^{+}$ of the best possible ratio. Thus, when τ takes certain value, for any $m\in M$, the method with high value of ${\chi }_m(\tau )$ is preferable or represents the best method. By this technique, we obtain the Figure 1, Figure 2, Figure 3. Based on the performance profile obtained, we can observe that with respect to number of iterations and function evaluations HSDY algorithm solves and win in over 50 percent of the problems as against CGD, PDY and ACGD with 18, 10 and 33 percent success, respectively. However, with respect to CPU time HSDY algorithm solves and win in over 32 percent of the problems as against CGD, PDY and ACGD with 11, 10 and 48 percent success, respectively. Therefore, we conclude that the HSDY method is more efficient than CGD, PDY and ACGD.

Figure 1.

Performance profiles with respect to the number of iterations.

Figure 2.

Performance profiles with respect to the number of function evaluations.

Figure 3.

Performance profiles with respect to CPU time.

4.1. Image restoration problem

Image restoration problem is usually aimed at recovering sparse original image $\overline{x}$ from a degraded observation b using the equation

$$b=A\overline{x},$$ (44)

where $A\in R^{m\times n}(m<n)$ is a linear map. However, since (44) is ill-conditioned, then the basic pursuit denoising framework (${\ell }_1$-norm problem) is appropriate

$$\underset{x}{\min }f(x)\triangleq \frac{1}{2}{\Vert y-Ax\Vert }_2^2+\tau {\Vert x\Vert }_1,\tau >0,$$ (45)

where $x\in R^m$, $y\in R^n$, $A\in R^{m\times n}$. Throughout this section, we use ${\Vert x\Vert }_1={\sum }_{i=1}^n|x_i|$ and ${\Vert x\Vert }_2$ to denote the ${\ell }_1-$ norm of vector $x\in R^m$ and the Euclidean norm, respectively.

In order to solve (45), we quickly give an overview of its reformulation into a convex quadratic problem by Figueiredo [17]. Any vector $x\in R^n$ can be written as

$$x=u-v,u\ge 0,v\ge 0,$$

where $u\in R^n,v\in R^n$ and $u_i={(x_i)}_{+}$, $v_i={(-x_i)}_{+}$ for all $i=1,2,\cdots n$ with ${(\cdot )}_{+}=\max \{0,\cdot \}$. Subsequently, the ${\ell }_1$-norm of a vector can be represented as ${\Vert x\Vert }_1=e_n^{T}u+e_n^{T}v$, where $e_n$ is an n-dimensional vector with all elements one. Hence, the ${\ell }_1$-norm problem (45) was transformed into

$$\underset{u,v}{\min }\frac{1}{2}{\Vert b-A(u-v)\Vert }^2+\tau e_n^{T}u+\tau e_n^{T}v,u\ge 0,v\ge 0.$$ (46)

From [17], the above equation can be easily rewritten as the quadratic program problem with box constraints

$$\underset{z}{\min }\frac{1}{2}{z}^{T}Dz+c^{T}z,\text{s.t.}z\ge 0,$$ (47)

where

$$z=\left[\begin{array}{c}\hfill u\hfill \\ \hfill v\hfill \\ \hfill \hfill \end{array}\right],y=A^{T}b,c=\tau e_{2n}+\left[\begin{array}{c}\hfill -y\hfill \\ \hfill y\hfill \\ \hfill \hfill \end{array}\right],D=\left[\begin{array}{cc}\hfill A^{T}A\hfill & \hfill -A^{T}A\hfill \\ \hfill -A^{T}A\hfill & \hfill A^{T}A\hfill \\ \hfill \hfill \end{array}\right].$$

Simple calculation shows that D is a semi-definite positive matrix. Hence (47) is a convex quadratic program problem, and it is equivalent to

$$H(z)=\min \{z,Dz+c\}=0.$$ (48)

The function D is vector-valued and the min interpreted as componentwise minimum. With the reformulation, from [28, Lemma 3] and [31, Lemma 2.2], since D is Lipschitz continuous and monotone, then the HSDY algorithm can be effectively used to solve (48).

Next, we apply the proposed hybrid conjugate gradient algorithm in image restoration. In order to evaluate the efficiency of the proposed algorithm in image restoration, we compare it's numerical performance with the CGD algorithm [32] designed for solving monotone equations and image restoration. We consider the following classical test images with color to illustrate the efficiency of the proposed algorithm (Figure 4, Figure 5).

Figure 4.

The benchmark test images. From the Left: Tiffany of size 512 × 512, Girl of size 720 × 576, Mars of size 1280 × 1024 and Malamute of size 1616 × 1080 (right).

Figure 5.

From the Left: The blurred image, the restored image by CGD and HSDY restored (right).

The above test images in Fig. 4 are obtained from http://hlevkin.com/06testimages.htm. All simulations are performed in Matlab (R2019b) on a HP with 2.4GHz processor and 8GB RAM. The parameters for the proposed algorithm are set as $\rho =0.4$, $\sigma ={10}^{-4}$. The quality of restoration by the algorithms are determined using Signal-to-ratio (SNR), Peak signal to noise ratio (PSNR) and Structural similarity index (SSIM). For fairness in comparing the algorithms, iteration process of all algorithms begin from $x_0=A^{T}b$ and terminates when

$$\frac{|f_k-f_{k-1}|}{|f_{k-1}|}<{10}^{-5},$$

where $f(x)=\frac{1}{2}{\Vert Ax-b\Vert }_2^2+\tau {\Vert x\Vert }_1$ is the objective function and $f_k$ denotes the function value at $x_k$. The original, blurred and restored images by each of the algorithms are given in Fig. 5.

In the following table, we report the numerical result for the test images used in this experiment.

From the Table 1, it can be observed that both algorithms were able to restore the blurred images. However, HSDY algorithm restored the images with better performance than that of CGD algorithm. This can be seen from the SNR, PSNR and SSIM values. It can be noticed that the SNR, PSNR and SSIM values of the images restored by our algorithm are about 0.01 to 0.05 larger than those restored by CGD. The MATLAB implementation of the SSIM index can be obtained at http://www.cns.nyu.edu/~lcv/ssim/.

Table 1.

Computational results for image restoration via CGD and HSDY.

Image	CGD			HSDY
	SNR	PSNR	SSIM	SNR	PSNR	SSIM
Tiffany	21.18	23.02	0.924	21.23	23.07	0.9253
Girl	17.4	22.46	0.7513	17.5	22.56	0.7549
Mars	14.86	24.75	0.793	14.87	24.77	0.7933
Malamute	15.47	21.89	0.6144	15.53	21.96	0.6172
Average	17.23	23.03	0.77068	17.28	23.09	0.77268

5. Conclusions

In this article, we proposed a conjugate gradient algorithm where the direction is a convex combination of two well known CG parameters, HS and DY. Independent of any line search, the proposed direction is sufficiently descent and bounded. Global convergence of the proposed algorithm was established under appropriate assumptions. Compared with CGD, PDY and ACGD algorithms, the HSDY algorithm performs better in terms of number of iteration and number of function evaluations. However, in terms of CPU time, ACGD algorithm performs better than HSDY, CGD and PDY. This may be as a result of the less computational cost associated with the ACGD algorithm. Finally, after reformulation, the HSDY algorithm was applied to restore blurred image.

Author contribution statement

A. B. Abubakar: Conceived and designed the experiments; Wrote the paper.

P. Kumam: Contributed reagents, materials, analysis tools or data.

A. H. Ibrahim: Performed the experiments; Wrote the paper.

J. Rilwan: Analyzed and interpreted the data; Wrote the paper.

Funding statement

The authors acknowledge the financial support provided by King Mongkut's University of Technology Thonburi through the “KMUTT 55th Anniversary Commemorative Fund”. The first author was supported by the Petchra Pra Jom Klao Doctoral Scholarship Academic for Ph.D. Program at KMUTT. Moreover, this project was partially supported by the Thailand Research Fund (TRF) and the King Mongkut's University of Technology Thonburi (KMUTT) under the TRF Research Scholar Award (Grant No. RSA6080047).

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Appendix.

See Table 2, Table 3, Table 4, Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11

Table 2.

Computational results for Problem 1.

DIM	INP	HSDY				CGD				PDY				ACGD
		ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	2	7	0.005294	0	42	125	0.035717	9.97E-07	16	64	0.033816	3.45E-07	8	31	0.31249	9.26E-06
	x2	2	7	0.004839	0	45	134	0.024415	9.45E-07	16	64	0.021728	7.03E-07	9	35	0.018068	3.01E-06
	x3	2	7	0.008178	0	48	143	0.018984	9.82E-07	17	68	0.015225	6.22E-07	9	36	0.008979	4.02E-06
	x4	2	7	0.004067	0	50	149	0.019721	9.70E-07	18	72	0.017206	4.54E-07	16	63	0.012829	9.22E-06
	x5	2	7	0.005323	0	51	152	0.025101	8.17E-07	18	72	0.043715	3.65E-07	18	71	0.023907	4.46E-06
	x6	2	7	0.003287	0	51	152	0.015118	8.56E-07	18	72	0.017788	3.80E-07	25	99	0.016718	6.74E-06
	x7	10	40	0.013326	1.35E-07	42	125	0.026073	9.99E-07	17	68	0.01432	7.44E-07	16	63	0.006078	9.85E-06
5000	x1	2	7	0.012334	0	41	122	0.17967	8.34E-07	16	64	0.043568	7.61E-07	9	36	0.019466	3.89E-06
	x2	2	7	0.043668	0	43	128	0.06739	9.81E-07	17	68	0.057717	5.15E-07	9	35	0.017451	6.65E-06
	x3	2	7	0.035024	0	47	140	0.059929	8.05E-07	18	72	0.060327	4.63E-07	9	35	0.017659	8.01E-06
	x4	2	7	0.039673	0	48	143	0.071964	9.93E-07	19	76	0.05853	3.38E-07	17	67	0.023527	8.12E-06
	x5	2	7	0.015735	0	49	146	0.29583	8.36E-07	18	72	0.084602	8.12E-07	18	71	0.035603	8.14E-06
	x6	2	7	0.015648	0	49	146	0.0523	8.76E-07	18	72	0.068756	8.10E-07	26	103	0.053833	7.96E-06
	x7	10	40	0.041804	3.49E-07	41	122	0.049472	9.13E-07	18	72	0.050171	5.39E-07	17	67	0.03582	8.74E-06
10000	x1	2	7	0.017441	0	40	119	0.07883	8.97E-07	17	68	0.085867	3.55E-07	9	35	0.036313	5.50E-06
	x2	2	7	0.016414	0	43	128	0.15836	8.26E-07	17	68	0.090358	7.27E-07	9	35	0.021614	9.39E-06
	x3	2	7	0.022671	0	46	137	0.09256	8.46E-07	18	72	0.098463	6.55E-07	10	40	0.030723	2.12E-06
	x4	2	7	0.020743	0	48	143	0.17944	8.30E-07	19	76	0.10115	4.77E-07	18	71	0.05649	4.58E-06
	x5	2	7	0.020405	0	48	143	0.086187	8.75E-07	20	80	0.10581	4.52E-07	18	71	0.097964	7.86E-06
	x6	2	7	0.019788	0	48	143	0.15679	9.17E-07	19	76	0.11446	5.51E-07	27	107	0.071799	6.22E-06
	x7	10	40	0.058995	4.45E-07	47	140	0.11897	8.60E-07	18	72	0.10054	7.54E-07	18	71	0.050097	4.97E-06
50000	x1	2	7	0.08254	0	39	116	0.32234	8.43E-07	17	68	0.42122	7.93E-07	10	40	0.13626	2.33E-06
	x2	2	7	0.085441	0	41	122	0.58374	9.37E-07	18	72	0.36367	5.44E-07	10	40	0.15326	3.97E-06
	x3	2	7	0.11513	0	44	131	0.57192	9.16E-07	19	76	0.58921	4.86E-07	10	40	0.381	4.67E-06
	x4	2	7	0.13723	0	46	137	0.50817	8.84E-07	20	80	0.74501	9.70E-07	19	75	0.21604	4.10E-06
	x5	2	7	0.090597	0	46	137	0.38682	9.34E-07	22	88	0.69514	8.63E-07	18	71	0.28658	5.06E-06
	x6	2	7	0.077672	0	46	137	0.68552	9.78E-07	23	92	0.65392	8.62E-07	28	111	0.31674	7.69E-06
	x7	11	44	0.34462	9.26E-08	45	134	0.42841	8.72E-07	19	76	0.41133	5.63E-07	19	76	0.33614	4.44E-06
100000	x1	2	7	0.13916	0	39	116	0.58628	7.72E-07	18	72	0.70355	3.76E-07	10	40	0.47701	3.29E-06
	x2	2	7	0.13912	0	41	122	1.0842	8.33E-07	18	72	0.74436	7.69E-07	10	40	0.42673	5.62E-06
	x3	2	7	0.2973	0	44	131	1.0265	7.92E-07	19	76	0.79387	6.88E-07	10	39	0.49391	6.59E-06
	x4	2	7	0.15099	0	45	134	0.7083	9.66E-07	23	92	1.2196	3.63E-07	19	75	0.48892	5.79E-06
	x5	2	7	0.17758	0	46	137	0.90572	7.99E-07	23	92	1.6058	9.61E-07	18	71	0.48887	4.05E-06
	x6	2	7	0.17743	0	46	137	0.71118	8.38E-07	26	104	1.586	3.39E-07	29	115	0.57871	6.05E-06
	x7	11	44	0.88693	1.05E-07	42	125	0.75264	8.90E-07	20	80	1.0036	7.80E-07	19	76	0.4356	6.30E-06

Table 3.

Computational results for Problem 2.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER2	FVAL3	TIME4	NORM5	ITER6	FVAL7	TIME8	NORM9	ITER10	FVAL11	TIME12	NORM13
1000	x1	6	19	0.075419	3.20E-09	55	163	0.034922	8.99E-07	13	51	0.009995	7.68E-07	3	8	0.013547	5.17E-07
	x2	6	19	0.008341	3.95E-09	61	181	0.039079	8.88E-07	15	59	0.016373	3.49E-07	3	8	0.002783	6.04E-06
	x3	6	19	0.003112	6.74E-07	69	205	0.023923	8.33E-07	16	63	0.010609	6.98E-07	4	11	0.004207	4.37E-07
	x4	7	22	0.005022	3.73E-09	76	226	0.024285	9.17E-07	18	71	0.014938	3.52E-07	5	14	0.002837	1.52E-07
	x5	6	19	0.009791	7.44E-07	78	232	0.026448	9.18E-07	18	71	0.010637	5.13E-07	5	14	0.013735	1.10E-06
	x6	8	25	0.003764	2.31E-09	81	241	0.061452	8.64E-07	18	71	0.014604	8.59E-07	6	17	0.003368	1.74E-08
	x7	35	119	0.019558	8.32E-07	72	214	0.044508	8.46E-07	17	67	0.015251	4.46E-07	49	192	0.032623	9.47E-06
5000	x1	6	20	0.026009	5.43E-08	59	175	0.12908	8.02E-07	14	55	0.038414	5.44E-07	3	8	0.006547	1.75E-07
	x2	6	20	0.12215	6.64E-08	64	190	0.097268	9.97E-07	15	59	0.038095	7.63E-07	3	8	0.008986	3.13E-06
	x3	6	19	0.018499	3.01E-07	72	214	0.13132	9.37E-07	17	67	0.050666	5.12E-07	4	11	0.013525	1.42E-07
	x4	7	23	0.010205	6.33E-08	80	238	0.15941	8.26E-07	18	71	0.059115	7.73E-07	5	14	0.010757	3.94E-08
	x5	6	19	0.009583	4.13E-07	82	244	0.13489	8.26E-07	19	75	0.059297	3.75E-07	5	14	0.012614	4.05E-07
	x6	8	26	0.018131	3.91E-08	84	250	0.15659	9.71E-07	19	75	0.063129	6.27E-07	6	17	0.016611	2.36E-09
	x7	26	91	0.067965	4.79E-07	75	223	0.19104	9.64E-07	17	67	0.056431	9.81E-07	12	44	0.030909	2.83E-06
10000	x1	7	27	0.033531	1.22E-07	60	178	0.26364	9.04E-07	14	55	0.072236	7.66E-07	3	8	0.012611	1.21E-07
	x2	7	26	0.073193	1.49E-07	66	196	0.41481	9.00E-07	16	63	0.090965	3.55E-07	3	8	0.01112	2.79E-06
	x3	7	26	0.026457	6.48E-07	74	220	0.31332	8.46E-07	17	67	0.079629	7.23E-07	4	11	0.012578	9.73E-08
	x4	8	30	0.037766	1.42E-07	81	241	0.24192	9.32E-07	19	75	0.097434	3.63E-07	5	14	0.02304	2.56E-08
	x5	7	26	0.02843	8.95E-07	83	247	0.31107	9.33E-07	19	75	0.098211	5.29E-07	5	14	0.017935	2.93E-07
	x6	9	33	0.036326	8.81E-08	86	256	0.2881	8.77E-07	19	76	0.10834	9.51E-07	6	17	0.030783	1.24E-09
	x7	31	103	0.25016	7.79E-08	77	229	0.38156	8.74E-07	18	71	0.10615	4.62E-07	12	44	0.052443	3.92E-06
50000	x1	7	27	0.10949	2.71E-07	64	190	1.097	8.26E-07	15	59	0.27736	5.78E-07	7	25	0.098699	2.94E-06
	x2	7	27	0.12313	3.30E-07	70	208	1.3692	8.23E-07	16	63	0.41807	7.92E-07	9	33	0.19125	2.78E-06
	x3	8	30	0.19674	5.74E-08	77	229	1.4151	9.67E-07	18	71	0.46025	5.36E-07	7	24	0.12953	9.11E-06
	x4	8	30	0.15657	3.16E-07	85	253	1.7181	8.52E-07	21	84	0.42626	3.43E-07	7	23	0.092063	9.18E-06
	x5	8	30	0.11835	8.07E-08	87	259	1.0519	8.53E-07	21	84	0.44697	4.72E-07	9	31	0.21747	6.71E-06
	x6	9	33	0.12498	1.95E-07	90	268	1.1335	8.02E-07	21	84	0.44167	4.77E-07	6	18	0.11336	5.20E-06
	x7	24	85	0.39386	2.08E-07	81	241	1.3997	8.04E-07	19	75	0.40352	3.46E-07	13	47	0.31911	7.22E-06
100000	x1	7	27	0.38406	3.83E-07	65	193	1.8345	9.34E-07	15	59	0.60576	8.17E-07	7	25	0.29458	4.14E-06
	x2	7	27	0.20813	4.67E-07	71	211	1.9771	9.30E-07	17	67	0.63713	3.76E-07	9	33	0.22634	3.93E-06
	x3	8	30	0.22192	8.10E-08	79	235	1.8254	8.75E-07	18	72	0.71361	9.65E-07	8	28	0.22477	3.33E-06
	x4	8	30	0.42291	4.46E-07	86	256	2.2303	9.64E-07	22	88	0.97382	8.28E-07	8	27	0.20603	3.34E-06
	x5	8	30	0.29563	1.14E-07	88	262	2.0458	9.64E-07	22	88	1.2244	8.18E-07	9	31	0.23411	9.46E-06
	x6	9	33	0.25169	2.75E-07	91	271	2.1387	9.07E-07	22	88	1.1689	7.87E-07	6	18	0.17317	7.01E-06
	x7	27	93	1.0224	9.11E-07	82	244	2.8011	9.07E-07	20	80	1.1064	5.47E-07	13	47	0.82264	8.37E-06

Table 4.

Computational results for Problem 3.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	7	28	0.076527	3.11E-07	68	203	0.026761	9.14E-07	15	60	0.023736	4.96E-07	10	39	0.012608	4.44E-06
	x2	7	28	0.006568	5.44E-07	71	212	0.020133	9.32E-07	16	64	0.017029	3.39E-07	10	39	0.007383	8.75E-06
	x3	6	24	0.005007	5.03E-07	75	224	0.022058	9.33E-07	16	64	0.018833	9.24E-07	11	43	0.007943	5.09E-06
	x4	7	26	0.009881	0	78	233	0.02225	9.83E-07	17	68	0.012856	8.94E-07	11	43	0.010864	5.04E-06
	x5	4	14	0.005018	0	78	233	0.030721	9.83E-07	18	72	0.023243	3.60E-07	10	39	0.010809	3.12E-06
	x6	5	18	0.00929	0	78	233	0.021676	9.83E-07	18	72	0.021773	3.47E-07	19	75	0.013426	5.98E-06
	x7	F	F	F	F	76	227	0.033447	8.39E-07	17	68	0.013763	3.87E-07	13	51	0.009796	2.86E-06
5000	x1	7	28	0.040308	6.95E-07	72	215	0.094625	8.37E-07	16	64	0.047156	3.74E-07	10	39	0.022852	9.93E-06
	x2	8	32	0.013879	2.43E-07	75	224	0.097486	8.54E-07	16	64	0.059219	7.58E-07	11	43	0.018541	5.09E-06
	x3	7	28	0.056015	2.25E-07	79	236	0.1104	8.55E-07	17	68	0.13165	6.84E-07	12	47	0.029435	2.96E-06
	x4	F	F	F	F	82	245	0.10971	9.00E-07	18	72	0.084674	6.68E-07	12	47	0.026536	2.93E-06
	x5	F	F	F	F	82	245	0.14196	9.00E-07	18	72	0.048557	8.05E-07	10	39	0.026884	6.97E-06
	x6	7	26	0.02394	0	82	245	0.1428	9.00E-07	18	72	0.047887	7.46E-07	20	79	0.057284	6.05E-06
	x7	F	F	F	F	79	236	0.16578	9.52E-07	17	68	0.068372	8.62E-07	13	51	0.027098	6.31E-06
10000	x1	7	28	0.020916	9.83E-07	73	218	0.27048	9.47E-07	16	64	0.07057	5.28E-07	11	43	0.031564	3.65E-06
	x2	8	32	0.0266	3.44E-07	76	227	0.187	9.66E-07	17	68	0.15287	3.55E-07	11	43	0.035459	7.19E-06
	x3	7	28	0.020868	3.18E-07	80	239	0.37162	9.67E-07	17	68	0.075455	9.67E-07	12	47	0.19288	4.18E-06
	x4	F	F	F	F	84	251	0.20245	8.15E-07	18	72	0.12444	9.44E-07	12	47	0.042381	4.15E-06
	x5	18	70	0.066079	0	84	251	0.24582	8.15E-07	20	80	0.10061	3.38E-07	10	39	0.030878	9.85E-06
	x6	16	62	0.062778	0	84	251	0.26355	8.15E-07	19	76	0.10208	3.50E-07	20	79	0.055843	8.56E-06
	x7	F	F	F	F	81	242	0.38288	8.60E-07	18	72	0.071372	4.10E-07	13	51	0.061279	8.90E-06
50000	x1	8	32	0.089482	4.40E-07	77	230	1.0589	8.67E-07	17	68	0.27716	3.91E-07	11	43	0.16071	8.17E-06
	x2	8	32	0.090952	7.70E-07	80	239	1.069	8.85E-07	17	68	0.31051	7.93E-07	12	47	0.14059	4.18E-06
	x3	7	28	0.081149	7.11E-07	84	251	1.0433	8.86E-07	18	72	0.34128	7.25E-07	12	47	0.1721	9.36E-06
	x4	34	134	0.60874	0	87	260	0.86864	9.33E-07	20	80	0.3433	6.42E-07	12	47	0.1296	9.27E-06
	x5	F	F	F	F	87	260	1.0529	9.33E-07	21	84	0.47206	5.20E-07	11	43	0.12183	5.73E-06
	x6	F	F	F	F	87	260	0.85175	9.33E-07	21	84	0.3755	3.51E-07	21	83	0.27422	8.66E-06
	x7	F	F	F	F	84	251	1.1538	9.88E-07	18	72	0.32114	9.18E-07	14	55	0.22277	5.17E-06
100000	x1	8	32	0.17022	6.22E-07	78	233	1.5125	9.81E-07	17	68	0.53357	5.53E-07	12	47	0.23793	3.00E-06
	x2	9	36	0.19443	2.18E-07	82	245	1.5024	8.01E-07	18	72	0.58373	3.76E-07	12	47	0.25733	5.91E-06
	x3	8	32	0.16554	2.01E-07	86	257	2.2472	8.02E-07	19	76	0.70333	3.40E-07	13	51	0.35897	3.44E-06
	x4	F	F	F	F	89	266	1.752	8.44E-07	22	88	0.73515	6.92E-07	13	51	0.2585	3.41E-06
	x5	17	66	0.471	0	89	266	1.8494	8.44E-07	22	88	0.68718	6.17E-07	11	43	0.23454	8.10E-06
	x6	132	526	3.5281	0	89	266	1.5963	8.44E-07	22	88	0.77704	5.81E-07	22	87	0.47895	5.54E-06
	x7	F	F	F	F	86	257	3.0357	8.92E-07	20	80	0.70259	4.62E-07	14	55	0.4586	7.31E-06

Table 5.

Computational results for Problem 4.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	2	6	0.024499	0	1	2	0.006389	0	2	6	0.003962	0	1	3	0.002633	0
	x2	2	6	0.003216	0	1	2	0.00597	0	2	6	0.002428	0	1	3	0.001686	0
	x3	2	6	0.00275	0	1	2	0.003908	0	2	6	0.002823	0	1	3	0.001745	0
	x4	2	7	0.007904	0	1	3	0.003417	0	2	6	0.003702	0	1	4	0.001703	0
	x5	2	7	0.005708	0	1	3	0.009238	0	2	6	0.003499	0	1	4	0.009774	0
	x6	2	7	0.00378	0	1	3	0.006905	0	2	6	0.004086	0	1	4	0.001727	0
	x7	14	43	0.020016	7.20E-07	1	2	0.006253	0	2	6	0.002982	0	1	3	0.001688	0
5000	x1	2	6	0.014845	0	1	2	0.012768	0	2	6	0.009169	0	1	3	0.005327	0
	x2	2	6	0.011499	0	1	2	0.021509	0	2	6	0.011734	0	1	3	0.004357	0
	x3	2	6	0.01021	0	1	2	0.00778	0	2	6	0.009543	0	1	3	0.006509	0
	x4	2	7	0.012676	0	1	3	0.009875	0	2	6	0.009298	0	1	4	0.004635	0
	x5	2	7	0.015074	0	1	3	0.008298	0	2	6	0.009188	0	1	4	0.006047	0
	x6	2	7	0.037449	0	1	3	0.015326	0	2	6	0.009426	0	1	4	0.008151	0
	x7	16	49	0.066203	3.85E-07	1	2	0.011263	0	2	6	0.009807	0	1	3	0.004956	0
10000	x1	2	6	0.017412	0	1	2	0.011494	0	2	6	0.018116	0	1	3	0.008259	0
	x2	2	6	0.016157	0	1	2	0.009187	0	2	6	0.020425	0	1	3	0.007457	0
	x3	2	6	0.01765	0	1	2	0.008987	0	2	6	0.018587	0	1	3	0.007431	0
	x4	2	7	0.024172	0	1	3	0.010462	0	2	6	0.010272	0	1	4	0.011393	0
	x5	2	7	0.029634	0	1	3	0.009659	0	2	6	0.014652	0	1	4	0.013088	0
	x6	2	7	0.022122	0	1	3	0.009686	0	2	6	0.019707	0	1	4	0.012318	0
	x7	17	52	0.13012	9.93E-07	1	2	0.011239	0	2	6	0.016483	0	1	3	0.024397	0
50000	x1	2	6	0.064229	0	1	2	0.050057	0	2	6	0.060039	0	1	3	0.040544	0
	x2	2	6	0.064088	0	1	2	0.046225	0	2	6	0.059133	0	1	3	0.03768	0
	x3	2	6	0.10784	0	1	2	0.099065	0	2	6	0.060013	0	1	3	0.064391	0
	x4	2	7	0.088382	0	1	3	0.045089	0	2	6	0.0784	0	1	4	0.038811	0
	x5	2	7	0.083801	0	1	3	0.038675	0	2	6	0.058015	0	1	4	0.036204	0
	x6	2	7	0.093509	0	1	3	0.040367	0	2	6	0.05856	0	1	4	0.036631	0
	x7	19	59	0.6776	2.96E-07	1	2	0.039825	0	2	7	0.086232	0	1	3	0.035574	0
100000	x1	2	6	0.13224	0	1	2	0.074679	0	2	6	0.12524	0	1	3	0.071684	0
	x2	2	6	0.12929	0	1	2	0.073843	0	2	6	0.14397	0	1	3	0.1608	0
	x3	2	6	0.13495	0	1	2	0.080754	0	2	6	0.12085	0	1	3	0.16962	0
	x4	2	7	0.25188	0	1	3	0.086601	0	2	6	0.096897	0	1	4	0.15325	0
	x5	2	7	0.14212	0	1	3	0.080458	0	2	6	0.086338	0	1	4	0.11427	0
	x6	2	7	0.13933	0	1	3	0.090445	0	2	6	0.097689	0	1	4	0.12526	0
	x7	19	58	1.6287	4.80E-07	1	2	0.076251	0	2	7	0.1664	0	1	3	0.082949	0

Table 6.

Computational results for Problem 5.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	2	7	0.026363	0	68	203	0.037912	8.60E-07	15	60	0.010859	5.13E-07	10	39	0.007286	3.65E-06
	x2	2	7	0.00667	0	71	212	0.05252	8.29E-07	16	64	0.008327	3.59E-07	10	39	0.003748	5.79E-06
	x3	7	28	0.004789	7.05E-07	74	221	0.018701	8.89E-07	16	64	0.011053	9.42E-07	10	39	0.003453	3.29E-06
	x4	2	7	0.003208	0	76	227	0.017737	9.28E-07	15	60	0.008818	6.44E-07	27	107	0.018976	8.97E-06
	x5	2	7	0.009285	0	76	227	0.017438	9.95E-07	17	68	0.008901	3.91E-07	26	103	0.010516	5.97E-06
	x6	2	7	0.003769	0	77	230	0.017944	8.34E-07	17	68	0.00744	7.89E-07	36	143	0.011102	9.56E-06
	x7	25	100	0.013958	1.28E-07	74	221	0.027053	8.88E-07	17	68	0.006517	5.08E-07	18	71	0.008066	6.34E-06
5000	x1	2	7	0.016039	0	71	212	0.066798	9.85E-07	16	64	0.036477	3.86E-07	10	39	0.015085	8.15E-06
	x2	2	7	0.016731	0	74	221	0.067669	9.49E-07	16	64	0.040955	8.02E-07	11	43	0.060773	3.36E-06
	x3	8	32	0.044007	3.15E-07	78	233	0.069571	8.14E-07	17	68	0.036523	7.00E-07	10	39	0.051635	7.37E-06
	x4	2	7	0.01361	0	80	239	0.072497	8.50E-07	16	64	0.029639	4.74E-07	29	115	0.22226	7.09E-06
	x5	2	7	0.014571	0	80	239	0.072811	9.11E-07	17	68	0.035299	8.74E-07	27	107	0.23414	7.95E-06
	x6	2	7	0.014532	0	80	239	0.077153	9.55E-07	19	76	0.037194	5.11E-07	39	155	0.16734	7.33E-06
	x7	22	88	0.054025	1.34E-07	78	233	0.072848	8.18E-07	18	72	0.031339	3.78E-07	19	75	0.048743	6.57E-06
10000	x1	2	7	0.015997	0	73	218	0.11983	8.91E-07	16	64	0.048042	5.46E-07	11	43	0.029547	3.00E-06
	x2	2	7	0.013746	0	76	227	0.12031	8.59E-07	17	68	0.065157	3.76E-07	11	43	0.026683	4.76E-06
	x3	8	32	0.023001	4.46E-07	79	236	0.15686	9.21E-07	17	68	0.051072	9.90E-07	11	43	0.030984	2.71E-06
	x4	2	7	0.088343	0	81	242	0.15296	9.62E-07	19	76	0.058916	3.70E-07	30	119	0.082432	5.97E-06
	x5	2	7	0.023319	0	82	245	0.12973	8.25E-07	18	72	0.056003	4.15E-07	28	111	0.11422	6.68E-06
	x6	2	7	0.022791	0	82	245	0.14462	8.64E-07	19	76	0.076001	7.22E-07	40	159	0.24325	7.26E-06
	x7	19	76	0.058596	4.90E-07	79	236	0.11755	9.29E-07	18	72	0.05497	5.22E-07	19	75	0.039516	9.34E-06
50000	x1	2	7	0.072536	0	77	230	0.50185	8.16E-07	17	68	0.26349	4.04E-07	11	43	0.080456	6.70E-06
	x2	2	7	0.182	0	79	236	0.50349	9.84E-07	17	68	0.24411	8.40E-07	12	47	0.14451	2.77E-06
	x3	8	32	0.10536	9.97E-07	83	248	0.53529	8.44E-07	18	72	0.25877	7.39E-07	11	43	0.082592	6.06E-06
	x4	2	7	0.11606	0	85	254	0.55283	8.81E-07	20	80	0.2675	6.25E-07	31	123	0.2697	7.94E-06
	x5	2	7	0.067573	0	85	254	0.57931	9.44E-07	20	80	0.2866	8.13E-07	29	115	0.24812	8.89E-06
	x6	2	7	0.27146	0	85	254	0.54595	9.89E-07	22	88	0.36281	9.65E-07	42	167	0.36493	7.96E-06
	x7	35	140	0.58073	9.20E-08	83	248	0.56601	8.52E-07	19	76	0.37417	6.75E-07	20	80	0.15103	9.49E-06
100000	x1	2	7	0.17046	0	78	233	0.93275	9.24E-07	17	68	0.45212	5.71E-07	11	43	0.16787	9.48E-06
	x2	2	7	0.1201	0	81	242	0.99893	8.90E-07	18	72	0.44518	3.98E-07	12	47	0.18465	3.91E-06
	x3	9	36	0.19082	2.82E-07	84	251	1.2408	9.55E-07	19	76	0.55976	9.57E-07	11	43	0.17925	8.57E-06
	x4	2	7	0.12481	0	86	257	1.2052	9.97E-07	22	88	0.61461	3.99E-07	32	127	0.44716	6.68E-06
	x5	2	7	0.12795	0	87	260	1.0739	8.55E-07	24	96	0.77374	3.66E-07	30	119	0.46015	7.48E-06
	x6	2	7	0.13466	0	87	260	1.0289	8.95E-07	26	104	1.139	3.55E-07	43	171	0.71872	7.88E-06
	x7	33	132	1.4716	2.96E-07	84	251	1.1087	9.58E-07	19	76	0.74306	9.53E-07	21	84	0.32836	6.07E-06

Table 7.

Computational results for Problem 6.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	17	64	0.037772	2.30E-07	90	268	0.020927	8.03E-07	19	75	0.016348	6.70E-07	39	150	0.017937	9.70E-06
	x2	16	60	0.008853	7.09E-07	89	265	0.024173	9.07E-07	19	75	0.015405	6.02E-07	22	83	0.006316	5.03E-06
	x3	25	98	0.010148	2.84E-07	88	262	0.019909	8.96E-07	20	79	0.011376	8.17E-07	43	169	0.043252	7.96E-06
	x4	24	96	0.02087	4.30E-07	89	266	0.020242	9.36E-07	20	80	0.01259	4.14E-07	30	119	0.009012	6.05E-06
	x5	18	72	0.014772	5.90E-07	87	260	0.026052	9.16E-07	20	80	0.010596	3.51E-07	29	115	0.018135	6.50E-06
	x6	25	100	0.016414	3.92E-07	84	251	0.017746	8.32E-07	21	84	0.012633	3.89E-07	40	159	0.014639	9.83E-06
	x7	25	99	0.025601	6.27E-07	88	262	0.025166	8.30E-07	28	111	0.019649	7.08E-07	274	1094	0.10413	9.52E-06
5000	x1	23	85	0.10341	7.28E-07	97	289	0.10927	8.47E-07	20	79	0.042236	6.26E-07	30	114	0.049102	9.56E-06
	x2	18	68	0.03886	2.24E-07	96	286	0.1044	9.58E-07	20	79	0.040238	5.64E-07	16	59	0.022496	5.91E-06
	x3	21	82	0.051016	5.48E-07	95	283	0.1131	9.47E-07	21	83	0.045612	7.12E-07	78	309	0.11465	9.70E-06
	x4	21	84	0.068828	9.22E-07	97	290	0.14691	8.05E-07	21	84	0.04965	3.38E-07	31	123	0.047898	8.39E-06
	x5	20	80	0.044595	4.34E-07	94	281	0.11473	9.87E-07	21	84	0.044151	4.47E-07	31	123	0.046734	7.81E-06
	x6	23	92	0.069719	2.05E-07	91	272	0.094334	8.88E-07	21	84	0.079496	6.59E-07	44	175	0.059499	7.37E-06
	x7	27	107	0.071376	4.75E-07	97	289	0.11585	9.02E-07	26	103	0.078455	5.32E-07	551	2202	0.69942	9.86E-06
10000	x1	29	104	0.078859	4.20E-07	100	298	0.17381	8.69E-07	20	79	0.087331	9.79E-07	77	302	0.17561	9.85E-06
	x2	16	60	0.058393	6.53E-07	99	295	0.177	9.83E-07	20	79	0.087591	8.67E-07	16	59	0.035911	7.52E-06
	x3	22	86	0.073928	7.91E-07	98	292	0.34269	9.72E-07	22	87	0.083442	4.07E-07	105	417	0.26591	9.08E-06
	x4	21	84	0.064505	5.48E-07	100	299	0.17374	8.29E-07	23	92	0.091583	4.76E-07	32	127	0.091851	7.17E-06
	x5	20	80	0.24827	2.71E-07	98	293	0.18219	8.14E-07	21	84	0.086638	7.05E-07	32	127	0.072841	8.26E-06
	x6	26	103	0.10573	5.21E-07	94	281	0.20502	9.15E-07	21	84	0.078683	5.31E-07	45	179	0.1068	9.01E-06
	x7	33	131	0.11154	2.64E-07	101	301	0.17152	8.76E-07	24	96	0.092335	4.23E-07	F	F	F	F
50000	x1	32	124	0.99143	6.45E-07	107	319	0.79415	9.16E-07	23	92	0.35679	4.69E-07	F	F	F	F
	x2	30	113	0.41584	3.06E-07	107	319	0.81068	8.28E-07	23	92	0.41153	4.37E-07	31	119	0.26438	7.09E-06
	x3	27	106	0.41431	2.09E-07	106	316	0.78562	8.19E-07	22	88	0.55162	8.93E-07	260	1037	2.2784	9.67E-06
	x4	21	84	0.37436	9.26E-07	107	320	0.83653	8.77E-07	24	96	0.4104	5.83E-07	33	131	0.3266	9.98E-06
	x5	20	80	0.25206	8.03E-07	105	314	0.76095	8.61E-07	24	96	0.58649	5.87E-07	35	139	0.33571	7.19E-06
	x6	28	112	0.43632	4.81E-07	101	302	0.75379	9.68E-07	23	92	0.5915	8.28E-07	49	195	0.45448	8.97E-06
	x7	25	99	0.63375	5.27E-07	107	319	0.80871	9.69E-07	27	108	0.48123	5.17E-07	F	F	F	F
100000	x1	25	92	0.54864	2.57E-07	110	328	1.5899	9.39E-07	24	96	0.77659	8.11E-07	F	F	F	F
	x2	20	76	0.612	5.18E-07	110	328	5.7937	8.50E-07	24	96	0.82618	7.59E-07	110	435	2.2584	9.55E-06
	x3	26	101	1.0136	9.00E-07	109	325	2.528	8.40E-07	23	92	0.6544	4.30E-07	345	1377	6.5905	9.76E-06
	x4	22	88	0.67165	2.45E-07	110	329	2.3072	9.00E-07	25	100	0.82136	3.79E-07	34	135	0.83873	8.65E-06
	x5	22	88	0.50431	7.44E-07	108	323	1.8277	8.84E-07	25	100	0.78133	5.83E-07	36	143	1.1988	8.09E-06
	x6	33	132	1.3726	4.14E-07	104	311	1.5009	9.94E-07	26	104	1.0855	3.96E-07	51	203	1.1313	8.42E-06
	x7	38	151	2.1589	3.08E-07	110	328	1.8097	8.07E-07	24	96	0.9451	9.67E-07	F	F	F	F

Table 8.

Computational results for Problem 7.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	13	52	0.049014	3.38E-07	83	248	0.029584	8.42E-07	18	72	0.019692	4.82E-07	12	47	0.011766	7.88E-06
	x2	13	52	0.011157	3.25E-07	83	248	0.044432	8.09E-07	18	72	0.02792	4.64E-07	12	47	0.012859	7.58E-06
	x3	13	52	0.011071	2.86E-07	82	245	0.029735	8.91E-07	18	72	0.040408	4.08E-07	12	47	0.009501	6.68E-06
	x4	12	48	0.009542	9.81E-07	80	239	0.028477	9.53E-07	17	68	0.025853	8.34E-07	12	47	0.010963	4.57E-06
	x5	12	48	0.007211	7.87E-07	79	236	0.02865	9.56E-07	17	68	0.018811	6.69E-07	12	47	0.011064	3.67E-06
	x6	12	48	0.010764	4.64E-07	77	230	0.027975	8.81E-07	17	68	0.027581	3.94E-07	11	43	0.009541	8.32E-06
	x7	13	52	0.007414	2.88E-07	82	245	0.060821	9.04E-07	18	72	0.01596	4.12E-07	12	47	0.010224	6.77E-06
5000	x1	13	52	0.047409	7.58E-07	86	257	0.17597	9.65E-07	19	76	0.077094	3.58E-07	13	51	0.03398	4.59E-06
	x2	13	52	0.13425	7.29E-07	86	257	0.15229	9.28E-07	19	76	0.071716	3.44E-07	13	51	0.032938	4.42E-06
	x3	13	52	0.061136	6.42E-07	86	257	0.14946	8.17E-07	18	72	0.06957	9.14E-07	13	51	0.17432	3.89E-06
	x4	13	52	0.046722	4.40E-07	84	251	0.1561	8.74E-07	18	72	0.068355	6.26E-07	13	51	0.040044	2.66E-06
	x5	13	52	0.06174	3.53E-07	83	248	0.16109	8.77E-07	18	72	0.076824	5.02E-07	12	47	0.1503	8.22E-06
	x6	13	52	0.17129	2.08E-07	81	242	0.13937	8.08E-07	17	68	0.064813	8.83E-07	12	47	0.067968	4.85E-06
	x7	13	52	0.043468	6.49E-07	86	257	0.15943	8.25E-07	18	72	0.084023	9.21E-07	13	51	0.038606	3.93E-06
10000	x1	14	56	0.098172	2.14E-07	88	263	0.30683	8.73E-07	21	84	0.19854	4.00E-07	13	51	0.06992	6.50E-06
	x2	14	56	0.13613	2.06E-07	88	263	0.32227	8.40E-07	21	84	0.16444	3.85E-07	13	51	0.063023	6.25E-06
	x3	13	52	0.14339	9.09E-07	87	260	0.2824	9.25E-07	20	80	0.1716	5.83E-07	13	51	0.1248	5.50E-06
	x4	13	52	0.078805	6.22E-07	85	254	0.28993	9.89E-07	18	72	0.14452	8.85E-07	13	51	0.086457	3.77E-06
	x5	13	52	0.072213	4.99E-07	84	251	0.34473	9.92E-07	18	72	0.12637	7.10E-07	13	51	0.065901	3.02E-06
	x6	13	52	0.11761	2.94E-07	82	245	0.35285	9.14E-07	18	72	0.14149	4.19E-07	12	47	0.065949	6.85E-06
	x7	13	52	0.13066	9.16E-07	87	260	0.31479	9.32E-07	20	80	0.17773	5.88E-07	13	51	0.068571	5.54E-06
50000	x1	14	56	1.3041	4.80E-07	91	272	1.2027	1.00E-06	24	96	0.73487	7.08E-07	14	55	0.36007	3.78E-06
	x2	14	56	0.27289	4.61E-07	91	272	1.1641	9.61E-07	24	96	0.90501	6.81E-07	14	55	0.28734	3.63E-06
	x3	14	56	0.32062	4.06E-07	91	272	1.1438	8.47E-07	23	92	0.73597	7.26E-07	14	55	0.3264	3.20E-06
	x4	14	56	0.28224	2.78E-07	89	266	1.4693	9.06E-07	21	84	0.79227	5.18E-07	13	51	0.26681	8.42E-06
	x5	14	56	0.41125	2.23E-07	88	263	1.0966	9.08E-07	21	84	0.57761	4.16E-07	13	51	0.40799	6.76E-06
	x6	13	52	1.0888	6.58E-07	86	257	1.0703	8.37E-07	18	72	0.51414	9.36E-07	13	51	0.26852	3.99E-06
	x7	14	56	1.0678	4.10E-07	91	272	1.2746	8.54E-07	23	92	0.7069	7.32E-07	14	55	0.34319	3.23E-06
100000	x1	14	56	0.75182	6.78E-07	93	278	2.5581	9.05E-07	29	116	2.6345	5.93E-07	14	55	1.1438	5.34E-06
	x2	14	56	0.60657	6.52E-07	93	278	2.5153	8.70E-07	28	112	2.7308	6.09E-07	14	55	0.88343	5.14E-06
	x3	14	56	0.7018	5.75E-07	92	275	2.4538	9.58E-07	26	104	1.9893	6.39E-07	14	55	0.6342	4.53E-06
	x4	14	56	0.96233	3.93E-07	91	272	2.4524	8.20E-07	23	92	1.7242	7.03E-07	14	55	0.73688	3.10E-06
	x5	14	56	0.84956	3.16E-07	90	269	2.4397	8.22E-07	22	88	1.4125	3.66E-07	13	51	0.70618	9.56E-06
	x6	13	52	0.71105	9.30E-07	87	260	2.6844	9.47E-07	20	80	1.235	5.97E-07	13	51	0.66641	5.64E-06
	x7	14	56	0.97379	5.79E-07	92	275	2.6021	9.66E-07	26	104	1.9078	6.45E-07	14	55	0.82162	4.56E-06

Table 9.

Computational results for Problem 8.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	7	28	0.046001	3.18E-07	37	110	0.020045	9.48E-07	17	68	0.015162	6.92E-07	10	39	0.008984	2.46E-06
	x2	7	28	0.007813	1.87E-07	37	110	0.016632	6.87E-07	17	68	0.011775	4.34E-07	9	35	0.008669	3.91E-06
	x3	6	24	0.081069	2.92E-07	30	89	0.009618	6.51E-07	5	20	0.003465	4.50E-08	8	31	0.008374	7.43E-06
	x4	7	28	0.006878	4.98E-07	38	113	0.015249	8.05E-07	18	72	0.014285	8.82E-07	11	43	0.008951	5.94E-06
	x5	7	28	0.005169	8.54E-07	38	113	0.018328	8.05E-07	19	76	0.012781	8.09E-07	11	43	0.006471	8.97E-06
	x6	8	31	0.016025	1.55E-07	37	109	0.016374	9.07E-07	18	71	0.011751	5.23E-07	12	46	0.007705	2.87E-06
	x7	36	144	0.042225	4.67E-07	37	110	0.015073	6.43E-07	19	76	0.013893	4.20E-07	11	43	0.011131	3.18E-06
5000	x1	7	28	0.023413	7.11E-07	39	116	0.057071	8.30E-07	18	72	0.046513	5.59E-07	10	39	0.038315	5.49E-06
	x2	7	28	0.021433	4.19E-07	38	113	0.049456	9.60E-07	17	68	0.051541	9.70E-07	9	35	0.022551	8.74E-06
	x3	6	24	0.018545	6.52E-07	31	92	0.051271	9.10E-07	5	20	0.020467	1.01E-07	9	35	0.020604	4.01E-06
	x4	8	32	0.022307	4.61E-08	40	119	0.050984	7.05E-07	19	76	0.067288	7.14E-07	12	47	0.031506	3.21E-06
	x5	8	32	0.041555	7.91E-08	40	119	0.061092	7.05E-07	20	80	0.12388	6.56E-07	12	47	0.024647	4.84E-06
	x6	8	31	0.041277	3.46E-07	39	115	0.05036	7.94E-07	19	75	0.054248	4.22E-07	12	46	0.025142	6.43E-06
	x7	53	212	0.14719	1.54E-07	38	113	0.075481	9.07E-07	19	76	0.060376	9.57E-07	11	43	0.026195	6.83E-06
10000	x1	8	32	0.049977	4.16E-08	40	119	0.095685	7.34E-07	18	72	0.091637	7.90E-07	10	39	0.046469	7.77E-06
	x2	7	28	0.035142	5.92E-07	39	116	0.096207	8.50E-07	18	72	0.10528	4.95E-07	10	39	0.035079	2.98E-06
	x3	6	24	0.035021	9.22E-07	32	95	0.094705	8.05E-07	5	20	0.022878	1.42E-07	9	35	0.031606	5.67E-06
	x4	8	32	0.034488	6.52E-08	40	119	0.12099	9.96E-07	20	80	0.099973	3.66E-07	12	47	0.051198	4.53E-06
	x5	8	32	0.037076	1.12E-07	40	119	0.098777	9.96E-07	20	80	0.13997	9.28E-07	12	47	0.10013	6.85E-06
	x6	8	31	0.0323	4.89E-07	40	118	0.094422	7.02E-07	21	84	0.10796	4.36E-07	12	46	0.044167	9.09E-06
	x7	22	88	0.096849	7.40E-07	39	116	0.14938	7.99E-07	20	80	0.10177	4.70E-07	11	43	0.049262	9.87E-06
50000	x1	8	32	0.14095	9.31E-08	42	125	0.43442	6.42E-07	19	76	0.40396	6.42E-07	11	43	0.15715	4.19E-06
	x2	8	32	0.11665	5.48E-08	41	122	0.38745	7.43E-07	19	76	0.42616	4.02E-07	10	39	0.12398	6.67E-06
	x3	7	28	0.19566	8.53E-08	34	101	0.31753	7.04E-07	5	20	0.075073	3.18E-07	10	39	0.12212	3.06E-06
	x4	8	32	0.13993	1.46E-07	42	125	0.37469	8.72E-07	21	84	0.53496	8.23E-07	13	51	0.23281	2.45E-06
	x5	8	32	0.1386	2.50E-07	42	125	0.42858	8.72E-07	21	84	0.77994	7.14E-07	13	51	0.22145	3.69E-06
	x6	9	35	0.24163	4.53E-08	41	121	0.38481	9.82E-07	21	84	0.54289	9.75E-07	13	50	0.15902	4.90E-06
	x7	26	104	0.47401	1.70E-07	41	122	0.57078	7.03E-07	21	84	0.53163	3.78E-07	12	47	0.208	5.32E-06
100000	x1	8	32	0.42726	1.32E-07	42	125	0.75126	9.08E-07	20	80	0.73251	7.45E-07	11	43	0.34593	5.93E-06
	x2	8	32	0.51417	7.75E-08	42	125	0.7611	6.58E-07	19	76	0.82489	5.69E-07	10	39	0.26349	9.43E-06
	x3	7	28	0.21644	1.21E-07	34	101	0.63728	9.96E-07	5	20	0.13893	4.50E-07	10	39	0.55675	4.32E-06
	x4	8	32	0.35243	2.06E-07	43	128	0.88825	7.71E-07	22	88	1.6662	4.22E-07	13	51	0.39548	3.46E-06
	x5	8	32	0.27367	3.54E-07	43	128	0.75182	7.71E-07	22	88	0.97393	7.50E-07	13	51	0.39804	5.22E-06
	x6	9	35	0.33982	6.40E-08	42	124	0.74606	8.69E-07	22	88	1.2738	5.00E-07	13	50	0.46053	6.94E-06
	x7	32	128	1.7627	6.92E-07	41	122	1.3396	9.90E-07	20	80	0.85032	6.65E-07	12	47	0.42613	7.52E-06

Table 10.

Computational results for Problem 9.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	F	F	F	F	37	110	0.023086	8.43E-07	61	243	0.09031	9.02E-07	F	F	F	F
	x2	F	F	F	F	37	110	0.020827	6.49E-07	58	231	0.052697	5.47E-07	F	F	F	F
	x3	38	152	0.031514	2.82E-07	23	68	0.009165	9.41E-07	21	84	0.019308	5.20E-07	37	147	0.03469	8.91E-06
	x4	F	F	F	F	38	113	0.013047	8.94E-07	54	216	0.048717	6.70E-07	F	F	F	F
	x5	F	F	F	F	38	113	0.014218	8.98E-07	54	216	0.057785	6.60E-07	F	F	F	F
	x6	F	F	F	F	38	113	0.013723	9.01E-07	53	212	0.069308	6.89E-07	F	F	F	F
	x7	44	176	0.067123	6.09E-07	182	545	0.083571	9.73E-07	20	80	0.019251	6.26E-07	35	139	0.021446	7.08E-06
5000	x1	F	F	F	F	39	116	0.060891	7.45E-07	123	491	0.56736	6.81E-07	F	F	F	F
	x2	F	F	F	F	38	113	0.062204	9.15E-07	123	491	0.53609	6.83E-07	F	F	F	F
	x3	55	220	0.29615	3.55E-07	29	86	0.053717	7.57E-07	22	88	0.096799	6.36E-07	16	63	0.039947	8.30E-06
	x4	F	F	F	F	40	119	0.069261	8.18E-07	125	500	0.56269	6.73E-07	F	F	F	F
	x5	F	F	F	F	40	119	0.064567	8.19E-07	124	496	0.52537	6.78E-07	F	F	F	F
	x6	F	F	F	F	40	119	0.17664	8.20E-07	124	496	0.52965	6.77E-07	F	F	F	F
	x7	45	180	0.80349	6.40E-07	218	653	0.43288	9.99E-07	21	84	0.10078	6.41E-07	61	243	0.16723	9.51E-06
10000	x1	F	F	F	F	40	119	0.10638	6.60E-07	197	787	1.5874	5.13E-07	F	F	F	F
	x2	F	F	F	F	39	116	0.10312	8.11E-07	199	795	1.8072	5.08E-07	F	F	F	F
	x3	54	216	0.33739	6.16E-07	31	92	0.092141	6.60E-07	20	80	0.19268	8.77E-07	20	79	0.11165	5.55E-06
	x4	F	F	F	F	41	122	0.11915	7.28E-07	198	792	1.8185	5.13E-07	F	F	F	F
	x5	F	F	F	F	41	122	0.12029	7.29E-07	197	788	2.1149	5.14E-07	F	F	F	F
	x6	F	F	F	F	41	122	0.11893	7.29E-07	199	796	3.0559	5.08E-07	F	F	F	F
	x7	51	204	0.35458	6.46E-07	228	683	0.82583	9.96E-07	21	84	0.16758	9.30E-07	584	2335	4.0181	7.89E-06
50000	x1	F	F	F	F	41	122	0.46388	9.25E-07	442	1767	16.7156	5.34E-07	F	F	F	F
	x2	F	F	F	F	41	122	0.42296	7.10E-07	445	1779	17.28	5.36E-07	F	F	F	F
	x3	49	196	1.5589	7.77E-07	33	98	0.34804	9.63E-07	F	F	F	F	19	75	0.54924	5.44E-06
	x4	F	F	F	F	43	128	0.44945	6.41E-07	F	F	F	F	F	F	F	F
	x5	F	F	F	F	43	128	0.45668	6.41E-07	F	F	F	F	F	F	F	F
	x6	F	F	F	F	43	128	0.47516	6.41E-07	F	F	F	F	F	F	F	F
	x7	345	1380	8.604	6.89E-07	241	722	3.5474	9.71E-07	F	F	F	F	603	2411	10.5638	8.27E-06
100000	x1	F	F	F	F	42	125	0.98452	8.18E-07	F	F	F	F	F	F	F	F
	x2	F	F	F	F	42	125	0.99183	6.28E-07	F	F	F	F	F	F	F	F
	x3	F	F	F	F	34	101	0.80447	9.14E-07	F	F	F	F	18	71	0.49349	5.30E-06
	x4	F	F	F	F	43	128	1.4792	9.07E-07	F	F	F	F	F	F	F	F
	x5	F	F	F	F	43	128	1.4904	9.07E-07	F	F	F	F	F	F	F	F
	x6	F	F	F	F	43	128	1.083	9.08E-07	F	F	F	F	F	F	F	F
	x7	347	1388	20.6668	8.34E-07	248	743	7.9032	9.75E-07	F	F	F	F	635	2539	23.5838	9.36E-06

Table 11.

Computational results for Problem 10.

		HSDY				CGD				PDY				ACGD
DIM	INP	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM	ITER	FVAL	TIME	NORM
1000	x1	8	32	0.003296	2.55E-07	21	62	0.005437	9.28E-07	11	44	0.007768	1.57E-07	8	31	0.003855	6.41E-06
	x2	8	32	0.003388	1.54E-07	21	62	0.004295	5.62E-07	10	40	0.008394	6.84E-07	8	31	0.002801	3.88E-06
	x3	8	32	0.004188	2.26E-07	21	62	0.005776	5.36E-07	10	40	0.004236	6.53E-07	8	31	0.003939	3.70E-06
	x4	9	36	0.006267	9.55E-08	23	68	0.004932	5.84E-07	11	44	0.004763	5.25E-07	9	35	0.004641	3.25E-06
	x5	9	36	0.006791	1.29E-07	23	68	0.005668	7.91E-07	11	44	0.005917	7.11E-07	9	35	0.003534	4.40E-06
	x6	9	36	0.003859	1.86E-07	24	71	0.005536	4.93E-07	12	48	0.004462	2.02E-07	9	35	0.003306	6.32E-06
	x7	9	36	0.003971	3.14E-07	22	65	0.007724	5.33E-07	11	44	0.006534	2.01E-07	8	31	0.003656	8.54E-06
5000	x1	8	32	0.019125	5.70E-07	22	65	0.014653	9.01E-07	11	44	0.014293	3.52E-07	9	35	0.013034	2.18E-06
	x2	8	32	0.054113	3.45E-07	22	65	0.015744	5.46E-07	11	44	0.016762	2.13E-07	8	31	0.006781	8.68E-06
	x3	8	32	0.011737	5.05E-07	22	65	0.020062	5.20E-07	11	44	0.017768	2.03E-07	8	31	0.007112	8.28E-06
	x4	9	36	0.017888	2.13E-07	24	71	0.017493	5.67E-07	12	48	0.020918	2.33E-07	9	35	0.01341	7.27E-06
	x5	9	36	0.021415	2.89E-07	24	71	0.02592	7.68E-07	12	48	0.022415	3.15E-07	9	35	0.008237	9.84E-06
	x6	9	36	0.11241	4.15E-07	25	74	0.020946	4.79E-07	12	48	0.022793	4.53E-07	10	39	0.007845	2.15E-06
	x7	9	36	0.018736	6.87E-07	23	68	0.018921	5.01E-07	11	44	0.019088	4.45E-07	9	35	0.006978	2.79E-06
10000	x1	8	32	0.033212	8.06E-07	23	68	0.028233	5.53E-07	11	44	0.035263	4.97E-07	9	35	0.017547	3.08E-06
	x2	8	32	0.023375	4.88E-07	22	65	0.0273	7.71E-07	11	44	0.038258	3.01E-07	9	35	0.017909	1.86E-06
	x3	8	32	0.023671	7.14E-07	22	65	0.031657	7.36E-07	11	44	0.047492	2.87E-07	9	35	0.022159	1.78E-06
	x4	9	36	0.026937	3.02E-07	24	71	0.029701	8.02E-07	12	48	0.037503	3.29E-07	10	39	0.02233	1.56E-06
	x5	9	36	0.03815	4.09E-07	25	74	0.040412	4.72E-07	12	48	0.036216	4.46E-07	10	39	0.015861	2.11E-06
	x6	9	36	0.026772	5.87E-07	25	74	0.031934	6.78E-07	14	56	0.049499	2.62E-07	10	39	0.019646	3.04E-06
	x7	9	36	0.033292	9.84E-07	23	68	0.029817	7.09E-07	11	44	0.034377	6.38E-07	9	35	0.020563	3.97E-06
50000	x1	9	36	0.097268	1.32E-07	24	71	0.13845	5.37E-07	12	48	0.14051	2.20E-07	9	35	0.060196	6.88E-06
	x2	9	36	0.1234	7.99E-08	23	68	0.16203	7.49E-07	11	44	0.17914	6.74E-07	9	35	0.067492	4.17E-06
	x3	9	36	0.14778	1.17E-07	23	68	0.14158	7.15E-07	11	44	0.14897	6.42E-07	9	35	0.058865	3.98E-06
	x4	9	36	0.097828	6.75E-07	25	74	0.12854	7.79E-07	14	56	0.23801	3.01E-07	10	39	0.062775	3.49E-06
	x5	9	36	0.094243	9.14E-07	26	77	0.15039	4.58E-07	30	120	0.58202	6.32E-07	10	39	0.097264	4.73E-06
	x6	10	40	0.31663	9.61E-08	26	77	0.13534	6.58E-07	16	64	0.29789	3.85E-07	10	39	0.074442	6.79E-06
	x7	10	40	0.11338	1.59E-07	24	71	0.1318	6.84E-07	12	48	0.14104	2.81E-07	9	35	0.066392	8.77E-06
100000	x1	9	36	0.19711	1.87E-07	24	71	0.25717	7.60E-07	12	48	0.31272	3.12E-07	9	35	0.13127	9.74E-06
	x2	9	36	0.19498	1.13E-07	24	71	0.24089	4.60E-07	11	44	0.31678	9.53E-07	9	35	0.13732	5.90E-06
	x3	9	36	0.20014	1.65E-07	24	71	0.24867	4.39E-07	11	44	0.34068	9.09E-07	9	35	0.11735	5.62E-06
	x4	9	36	0.32581	9.55E-07	26	77	0.27984	4.78E-07	30	120	0.87109	6.60E-07	10	39	0.13233	4.94E-06
	x5	10	40	0.30747	9.46E-08	26	77	0.31092	6.48E-07	16	64	0.40418	3.80E-07	10	39	0.15794	6.68E-06
	x6	10	40	0.28105	1.36E-07	26	77	0.32261	9.31E-07	16	64	0.44179	5.45E-07	10	39	0.18209	9.60E-06
	x7	10	40	0.28271	2.26E-07	24	71	0.29624	9.71E-07	12	48	0.27776	3.98E-07	10	39	0.1306	1.89E-06