Second-order optimality conditions for nonlinear programs and mathematical programs

Abstract

It is well known that second-order information is a basic tool notably in optimality conditions and numerical algorithms. In this work, we present a generalization of optimality conditions to strongly convex functions of order γ with the help of first- and second-order approximations derived from (Optimization 40(3):229-246, 2011) and we study their characterization. Further, we give an example of such a function that arises quite naturally in nonlinear analysis and optimization. An extension of Newton’s method is also given and proved to solve Euler equation with second-order approximation data.

Introduction

The concept of approximations of mappings was introduced by Thibault [2]. Sweetser [3] considered approximations by subsets of the space of continuous linear maps $L(X,Y)$, where X and Y are Banach spaces, and Ioffe [4] by the so-called fans. This approach was revised by Jourani and Thibault [5]. Another approach belongs to Allali and Amahroq [1]. Following the same ideas, Amahroq and Gadhi [6, 7] have established optimality conditions to some optimization problems under set-valued mapping constraints.

In this work, we explore the notion of strongly convex functions of order γ; see, for instance, [8–15] and references therein. Let f be a mapping from a Banach space X into $\mathbb{R}$, and let $C\subset X$ be a closed convex set. It is well known that the notion of strong convexity plays a central role. On the one hand, it ensures the existence and uniqueness of the optimal solution for the problem

$$(\mathcal{P})\underset{x\in C}{min}f(x).$$

On the other hand, if f is twice differentiable, then the strong convexity of f implies that its Hessian matrix is nonsingular, which is an important tool in numerical algorithms. Here we adopt the definition of a second-order approximation [1] to detect some equivalent properties of strongly convex functions of order γ and to characterize the latter. Furthermore, for a $C^{1,1}$ function f on a finite-dimensional setting, we show some simple facts. We also provide an extension of Newton’s method to solve an Euler equation with second-order approximation data.

The rest of the paper is written as follows. Section 2 contains basic definitions and preliminary results. Section 3 is devoted to mains results. In Section 4, we point out an extension of Newton’s method and prove its local convergence.

Preliminaries

Let X and Y be two Banach spaces. We denote by $\mathcal{L}(X,Y)$ the set of all continuous linear mappings from X into Y, by $\mathcal{B}(X\times X,Y)$ the set of all continuous bilinear mappings from $X\times X$ into Y, and by ${\mathbb{B}}_Y$ the closed unit ball of Y centered at the origin.

Throughout this paper, $X^{\ast }$ and $Y^{\ast }$ denote the continuous duals of X and Y, respectively, and we write $\langle \cdot ,\cdot \rangle$ for the canonical bilinear forms with respect to the dualities $\langle X^{\ast },X\rangle$ and $\langle Y^{\ast },Y\rangle$.

Definition 1

[1]

Let f be a mapping from X into Y, $\overline{x}\in X$. A set of mappings ${\mathcal{A}}_f(\overline{x})\subset \mathcal{L}(X,Y)$ is said to be a first-order approximation of f at x̄ if there exist $\delta >0$ and a function $r:X\to \mathbb{R}$ satisfying ${lim}_{x\to \overline{x}}r(x)=0$ such that

$$f(x)-f(\overline{x})\in {\mathcal{A}}_f(\overline{x})(x-\overline{x})+\parallel x-\overline{x}\parallel r(x){\mathbb{B}}_Y$$ 1

for all $x\in \overline{x}+\delta {\mathbb{B}}_X$.

It is easy to check that Definition 1 is equivalent to the following: for all $\epsilon >0$, there exists $\delta >0$ such that

$$f(x)-f(\overline{x})\in {\mathcal{A}}_f(\overline{x})(x-\overline{x})+\epsilon \parallel x-\overline{x}\parallel {\mathbb{B}}_Y$$ 2

for all $x\in \overline{x}+\delta {\mathbb{B}}_X$.

Remark 1

If ${\mathcal{A}}_f(\overline{x})$ is a first-order approximation of f at x̄, then (2) means that for any $x\in \overline{x}+\delta {\mathbb{B}}_X$, there exist $A(x)\in {\mathcal{A}}_f(\overline{x})$ and $b\in {\mathbb{B}}_Y$ such that

$$f(x)-f(\overline{x})=A(x)(x-\overline{x})+\epsilon \parallel x-\overline{x}\parallel b.$$

Hence, for any $x\in \mathbb{B}(\overline{x},\delta )$ and $A(x)\in {\mathcal{A}}_f(\overline{x})$,

$$\parallel f(x)-f(\overline{x})-A(x)(x-\overline{x})\parallel \le \epsilon \parallel x-\overline{x}\parallel .$$ 3

If ${\mathcal{A}}_f(\overline{x})$ is norm-bounded (resp. compact), then it is called a bounded (resp. compact) first-order approximation. Recall that ${\mathcal{A}}_f(\overline{x})$ is a singleton if and only if f is Fréchet differentiable at x̄.

The following proposition proved by Allali and Amahroq [1] plays an important role in the sequel in a finite-dimensional setting.

Proposition 1

[1]

Let $f:{\mathbb{R}}^p\to \mathbb{R}$ be a locally Lipschitz function at x̄. Then the Clarke subdifferential of f at x̄,

$${\partial }_{c}f(\overline{x}):=co\{lim\mathrm{\nabla }f(x_n):x_n\in dom\mathrm{\nabla }f\mathit{\text{ and }}{x}_n\to \overline{x}\},$$ 4

is a first-order approximation of f at x̄.

In [6], it is also shown that when f is a continuous function, it admits as an approximation the symmetric subdifferential defined and studied in [16].

The next proposition shows that Proposition 1 holds also when f is a vector-valued function. Let us first recall the definition of the generalized Jacobian for a vector-valued function (see [17, 18] for more details) and the definition of upper semicontinuity.

Definition 2

The generalized Jacobian of a function $g:{\mathbb{R}}^p\to {\mathbb{R}}^q$ at x̄, denoted ${\partial }_{c}g(\overline{x})$, is the convex hull of all matrices M of the form

$$M=\underset{n\to +\mathrm{\infty }}{lim}Jg(x_n),$$

where $x_n\to \overline{x}$, g is differentiable at $x_n$ for all n, and Jg denotes the $q\times p$ usual Jacobian matrix of partial derivatives.

Definition 3

A set-valued mapping $F:{\mathbb{R}}^p\rightrightarrows {\mathbb{R}}^q$ is said to be upper semicontinuous at a point $\overline{x}\in {\mathbb{R}}^p$ if, for every $\epsilon >0$, there exists $\delta >0$ such that

$$F(x)\subset F(\overline{x})+\epsilon \mathbb{B}$$

for every $x\in {\mathbb{R}}^p$ such that $\parallel x-\overline{x}\parallel <\delta$.

Proposition 2

Let $g:{\mathbb{R}}^p\to {\mathbb{R}}^q$ be a locally Lipschitz function at x̄. Then the generalized Jacobian ${\partial }_{c}g(\overline{x})$ of g at x̄ is a first-order approximation of g at x̄.

Proof

Since the set-valued mapping ${\partial }_{c}g(\cdot )$ is upper semicontinuous, for all $\epsilon >0$, there exists $r_0>0$ such that

$${\partial }_{c}g(x)\subset {\partial }_{c}g(\overline{x})+\epsilon {\mathbb{B}}_{\mathcal{L}({\mathbb{R}}^p,{\mathbb{R}}^q)}\text{for all }x\in \overline{x}+r_0{\mathbb{B}}_{{\mathbb{R}}^p}.$$

We may assume that g is Lipschitzian in $\overline{x}+r_0{\mathbb{B}}_{{\mathbb{R}}^p}$. Let $x\in \overline{x}+r_0{\mathbb{B}}_{{\mathbb{R}}^p}$. We apply [17], Prop. 2.6.5, to derive that there exits $c\in ]x,\overline{x}[$ such that

$$g(x)-g(\overline{x})\in {\partial }_{c}g(c)(x-\overline{x})\subset {\partial }_{c}g(\overline{x})(x-\overline{x})+\epsilon {\mathbb{B}}_{\mathcal{L}({\mathbb{R}}^p,{\mathbb{R}}^q)}(x-\overline{x}).$$

Since

$${\mathbb{B}}_{\mathcal{L}({\mathbb{R}}^p,{\mathbb{R}}^q)}(x-\overline{x})\subset \parallel x-\overline{x}\parallel {\mathbb{B}}_{{\mathbb{R}}^q},$$

we have

$$g(x)-g(\overline{x})\in {\partial }_{c}g(\overline{x})(x-\overline{x})+\epsilon \parallel x-\overline{x}\parallel {\mathbb{B}}_{{\mathbb{R}}^q},$$

which means that ${\partial }_{c}g(\overline{x})$ is a first-order approximation of g at x̄. □

Recall that a mapping $f:X\to Y$ is said to be $C^{1,1}$ at x̄ if it is Fréchet differentiable in neighborhood of x̄ and if its Fréchet derivative $\mathrm{\nabla }f(\cdot )$ is Lipschitz at x̄.

Let $\overline{x}\in {\mathbb{R}}^p$, and let $f:{\mathbb{R}}^p\to \mathbb{R}$ be a $C^{1,1}$ function at x̄. The generalized Hessian matrix of f at x̄ was introduced and studied by Hiriart-Urruty et al. [19] is the compact nonempty convex set

$${\partial }_H^{2}f(\overline{x}):=co\{lim{\mathrm{\nabla }}^{2}f(x_n):(x_n)\in dom{\mathrm{\nabla }}^{2}f\mathit{\text{ and }}{x}_n\to \overline{x}\},$$ 5

where $dom{\mathrm{\nabla }}^{2}f$ is the effective domain of ${\mathrm{\nabla }}^{2}f(\cdot )$.

Corollary 1

Let $\overline{x}\in {\mathbb{R}}^p$, and $f:{\mathbb{R}}^p\to \mathbb{R}$ be a $C^{1,1}$ function at x̄. Then, ∇f admits ${\partial }_H^{2}f(\overline{x})$ as a first-order approximation at x̄.

Definition 4

[1]

We say that $f:X\to Y$ admits a second-order approximation at x̄ if there exit two sets ${\mathcal{A}}_f(\overline{x})\subset \mathcal{L}(X,Y)$ and ${\mathcal{B}}_f(\overline{x})\subset \mathcal{B}(X\times X,Y)$ such that

• (i) ${\mathcal{A}}_f(\overline{x})$ is a first-order approximation of f at x̄;
• (ii)
  For all $\epsilon >0$, there exists $\delta >0$ such that

  $$f(x)-f(\overline{x})\in {\mathcal{A}}_f(\overline{x})(x-\overline{x})+{\mathcal{B}}_f(\overline{x})(x-\overline{x})(x-\overline{x})+\epsilon {\parallel x-\overline{x}\parallel }^2{\mathbb{B}}_Y$$

  for all $x\in \overline{x}+\delta {\mathbb{B}}_X$.

In this case the pair $({\mathcal{A}}_f(\overline{x}),{\mathcal{B}}_f(\overline{x}))$ is called a second-order approximation of f at x̄. It is called a compact second-order approximation if ${\mathcal{A}}_f(\overline{x})$ and ${\mathcal{B}}_f(\overline{x})$ are compacts.

Every $C^2$ mapping $f:X\to Y$ at x̄ admits $(\mathrm{\nabla }f(\overline{x}),{\mathrm{\nabla }}^{2}f(\overline{x}))$ as a second-order approximation, where $\mathrm{\nabla }f(\overline{x})$ and ${\mathrm{\nabla }}^{2}f(\overline{x})$ are, respectively, the first- and second-order Fréchet derivatives of f at x̄.

Proposition 3

[1]

Let $f:{\mathbb{R}}^p\to \mathbb{R}$ be a $C^{1,1}$ function at x̄. Then f admits $(\mathrm{\nabla }f(\overline{x}),\frac{1}{2}{\partial }_H^{2}f(\overline{x}))$ as a second-order approximation at x̄.

Proposition 4

Let $f:X\to Y$ be a Fréchet-differentiable mapping. If $(\mathrm{\nabla }f(\overline{x}),{\mathcal{B}}_f(\overline{x}))$ is a bounded second-order approximation of f at x̄. Then $\mathrm{\nabla }f(\cdot )$ is stable at x̄, that is, there exist $c,r>0$ such that

$$\parallel \mathrm{\nabla }f(x)-\mathrm{\nabla }f(\overline{x})\parallel \le c\parallel x-\overline{x}\parallel$$ 6

for all $x\in \overline{x}+r{\mathbb{B}}_X$.

To derive some results for γ-strong convex functions, the following notions are needed.

Definition 5

[8]

Let $\gamma >0$. We say that a map $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ is γ-strongly convex if there exist $c\ge 0$ and $g:[0,1]\to {\mathbb{R}}^{+}$ satisfying

$$g(0)=g(1)=0\text{and}\underset{\theta \to 0}{lim}\frac{g(\theta )}{\theta }=1$$ 7

and such that

$$f(\theta x+(1-\theta )y)\le \theta f(x)+(1-\theta )f(y)-cg(\theta ){\parallel x-y\parallel }^{\gamma }$$ 8

for all $\theta \in [0,1]$ and $x,y\in X$.

Of course, when $c=0$, f is called a convex function. Otherwise, f is said γ-strongly convex. This class has been introduced by Polyak [11] when $\gamma =2$ and $g(\theta )=\theta (1-\theta )$ and studied by many authors. Recently, a characterization of γ-strongly convex functions has been shown in [8]. For example, if f is $C^1$ and $\gamma \ge 1$, then (8) is equivalent to

$$\langle \mathrm{\nabla }f(x),y-x\rangle \le f(y)-f(x)-\frac{c}{\gamma }{\parallel y-x\parallel }^{\gamma },\mathrm{\forall }x,y\in X.$$ 9

Let $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ and $\overline{x}\in domf:=\{x\in X,f(x)<+\mathrm{\infty }\}$ (the effective domain of f). The Fenchel-subdifferential of f at x̄ is the set

$${\partial }_{\mathrm{Fen}}f(\overline{x})=\{x^{\ast }\in X^{\ast }:\langle x^{\ast },y-\overline{x}\rangle \le f(y)-f(\overline{x}),\mathrm{\forall }y\in X\}.$$ 10

Let $\gamma >0$ and $c>0$. The $(\gamma ,c)$-subdifferential of f at x̄ is the set

$${\partial }_{(\gamma ,c)}f(\overline{x})=\{x^{\ast }\in X^{\ast }:\langle x^{\ast },y-\overline{x}\rangle \le f(y)-f(\overline{x})-c{\parallel \overline{x}-y\parallel }^{\gamma },\mathrm{\forall }y\in X\}.$$ 11

For more details on $(\gamma ,c)$-subdifferential, see [8]. Note that if $x\notin domf$, then ${\partial }_{(\gamma ,c)}f(\overline{x})={\partial }_{\mathrm{Fen}}f(\overline{x})=\mathrm{\varnothing }$. Clearly, we have ${\partial }_{(\gamma ,c)}f(\overline{x})\subset {\partial }_{\mathrm{Fen}}f(\overline{x})$. Note that the Fenchel-subdifferential defined by (10) coincides with the Clarke subdifferential of f at x̄ if the function f is convex. We also need to recall the following definitions.

Definition 6

[20]

We say that a map $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ is 2-paraconvex if there exists $c>0$ such that

$$f(\theta x+(1-\theta )y)\le \theta f(x)+(1-\theta )f(y)+cmin(\theta ,1-\theta ){\parallel x-y\parallel }^2$$ 12

for all $\theta \in [0,1]$ and $x,y\in X$.

It has been proved in [20] that if f is a $C^1$ mapping, then (12) is equivalent to

$$\langle \mathrm{\nabla }f(x),y-x\rangle \le f(y)-f(x)+c{\parallel y-x\parallel }^2,\mathrm{\forall }x,y\in X.$$ 13

Main results

In this section, we obtain the main results of the paper related to strongly convex functions of order γ defined by (7)-(8). We begin by showing some interesting facts of functions that admit a first-order approximation.

For any subset A of $X^{\ast }$, we define the support function of A as

$$s(A,x)=sup\{\langle x^{\ast },x\rangle ,x^{\ast }\in A\}.$$ 14

It is well known that, for any convex function f: $X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$, the ‘right-hand’ directional derivative at x in domf (the domain of f ) exists and, for each $h\in X$, is

$$d^{+}f(x)(h)=\underset{t\to 0^{+}}{lim}\frac{f(x+th)-f(x)}{t}.$$

Theorem 1

Let $\overline{x}\in X$. If $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ is convex and continuous at x̄ and if ${\mathcal{A}}_f(\overline{x})\subset X^{\ast }$ is a convex $w(X^{\ast },X)$-closed approximation of f at x̄, then

$${\partial }_{(\gamma ,c)}f(\overline{x})\subset {\mathcal{A}}_f(\overline{x}).$$

Proof

By the definition of ${\mathcal{A}}_f(\overline{x})$, there exist $\delta >0$ and $r:X\to \mathbb{R}$ with ${lim}_{x\to \overline{x}}r(x)=0$ such that, for all $x\in \overline{x}+\delta {\mathbb{B}}_X$, $t\in ]0,\delta [$, and $h\in X$, there exist $A\in {\mathcal{A}}_f(\overline{x})$ and $b\in [-1,1]$ satisfying

$$\frac{f(\overline{x}+th)-f(\overline{x})}{t}-\parallel h\parallel r(\overline{x}+th)b=\langle A,h\rangle \le s({\mathcal{A}}_f(\overline{x});h).$$

By letting $t\to 0^{+}$ the directional derivative of f at x̄ satisfies

$$d^{+}f(\overline{x})(h)\le s({\mathcal{A}}_f(\overline{x});h),\mathrm{\forall }h\in X.$$ 15

Using [21], Prop. 2.24, we get

$$s({\partial }_{\mathrm{Fen}}f(\overline{x});h)\le s({\mathcal{A}}_f(\overline{x});h).$$

Since ${\partial }_{(\gamma ,c)}f(\overline{x})\subset {\partial }_{\mathrm{Fen}}f(\overline{x})$, we deduce that

$$s({\partial }_{(\gamma ,c)}f(\overline{x});h)\le s({\mathcal{A}}_f(\overline{x});h).$$

Hence we conclude that ${\partial }_{(\gamma ,c)}f(\overline{x})\subset {\mathcal{A}}_f(\overline{x})$. □

Proposition 5

Let $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ be a γ-strongly convex function. Assume that ${\mathcal{A}}_f(\overline{x})$ is a compact approximation at x̄. Then ${\mathcal{A}}_f(\overline{x})\cap {\partial }_{(\gamma ,c)}f(\overline{x})\ne \mathrm{\varnothing }$.

Proof

Let $d\in X$ be fixed and define $x_n:=\overline{x}+\frac{1}{n}d$. Using Definition 1, we get, for n large enough, $A_n\in {\mathcal{A}}_f(\overline{x})$ and $b_n\in [-1,1]$ such that

$$\frac{1}{n}\langle A_n,d\rangle =f(\overline{x}+\frac{1}{n}d)-f(\overline{x})-\frac{1}{n}\parallel d\parallel r(x_n)b_n.$$

By γ-strong convexity we obtain

$$\frac{1}{n}\langle A_n,d\rangle \le \frac{1}{n}(f(\overline{x}+d)-f(\overline{x}))-cg\left(\frac{1}{n}\right){\parallel d\parallel }^{\gamma }-\frac{1}{n}\parallel d\parallel r(x_n)b_n.$$

By the compactness of ${\mathcal{A}}_f(\overline{x})$, extracting a subsequence if necessary, we may assume that there exists $A\in {\mathcal{A}}_f(\overline{x})$ such that $\langle A_n,d\rangle \to \langle A,d\rangle$; and hence we obtain

$$\langle A,d\rangle \le f(\overline{x}+d)-f(\overline{x})-c{\parallel d\parallel }^{\gamma }.$$ 16

Assume that $A\in {\mathcal{A}}_f(\overline{x})\cap {\partial }_{(\gamma ,c)}f(\overline{x})$. By the separation theorem there exists $h\in X$ with $\parallel h\parallel =1$ such that

$$\underset{A\in {\mathcal{A}}_f(\overline{x})}{min}\langle A,h\rangle >\underset{x^{\ast }\in {\partial }_{(\gamma ,c)}f(\overline{x})}{sup}\langle x^{\ast },h\rangle .$$

Let $t>0$ sufficiently small, so that

$$\underset{A\in {\mathcal{A}}_f(\overline{x})}{min}\langle A,h\rangle >\frac{f(\overline{x}+th)-f(\overline{x})}{t},$$

in contradiction with relation (16) by taking $d=th$. □

Following a result by Rademacher, which states that a locally Lipschitzian function between finite-dimensional spaces is differentiable (Lebesgue) almost everywhere, we can prove the following result.

Proposition 6

Let $\gamma \ge 1$, $\overline{x}\in {\mathbb{R}}^p$, and let $f:{\mathbb{R}}^p\to \mathbb{R}$ be continuous at x̄. Assume that f is a γ-strongly convex function. Then ${\partial }_{c}f(\overline{x})={\partial }_{(\gamma ,c)}f(\overline{x})$.

Proof

Obviously, we have ${\partial }_{(\gamma ,c)}f(\overline{x})\subset {\partial }_{c}f(\overline{x})$. Now let $A\in {\partial }_{c}f(\overline{x})$. For all n, there exists $x_n\in dom\mathrm{\nabla }f$ such that $x_n\to \overline{x}$ and $\mathrm{\nabla }f(x_n)\to A$. Since f is γ-strongly convex and Fréchet differentiable at $x_n$ for all $n\in \mathbb{N}$, it follows by (9) that

$$\langle \mathrm{\nabla }f(x_n),y-x_n\rangle \le f(y)-f(x_n)-c{\parallel y-x_n\parallel }^{\gamma },\mathrm{\forall }y\in {\mathbb{R}}^p,\mathrm{\forall }n\in \mathbb{N}.$$

Letting $n\to +\mathrm{\infty }$, we get

$$\langle A,y-\overline{x}\rangle \le f(y)-f(\overline{x})-c{\parallel y-\overline{x}\parallel }^{\gamma },\mathrm{\forall }y\in {\mathbb{R}}^p,$$

which means that ${\partial }_{c}f(\overline{x})\subset {\partial }_{(\gamma ,c)}f(\overline{x})$. □

Corollary 2

Let $\gamma \ge 1$, $\overline{x}\in {\mathbb{R}}^p$, and let $f:{\mathbb{R}}^p\to \mathbb{R}$ be continuous at x̄. Assume that f is a γ-strongly convex function. Then, for all $\epsilon >0$, there exists $r>0$ such that

$$f(x)-f(\overline{x})\in {\partial }_{(\gamma ,c)}f(\overline{x})(x-\overline{x})+\epsilon \parallel x-\overline{x}\parallel {\mathbb{B}}_{\mathbb{R}}$$ 17

for all $x\in \overline{x}+r{\mathbb{B}}_{{\mathbb{R}}^p}$, which means that ${\partial }_{(\gamma ,c)}f(\overline{x})$ is a first-order approximation of f at x̄.

Proof

It is clear that ${\partial }_{c}f(\overline{x})$ is a first-order approximation of at x̄. We end the proof by Propositions 1 and 6. □

The converse of Proposition 5 holds if (16) is valid for any $A\in {\mathcal{A}}_f(x)$ and $x\in X$.

Proposition 7

Let $\gamma \ge 1$ and $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$. Assume that, for each $x\in X$, f admits a first-order approximation ${\mathcal{A}}_f(x)$ such that ${\mathcal{A}}_f(x)\subset {\partial }_{(\gamma ,c)}f(x)$. Then f is γ-strongly convex.

Proof

Define $x_{\theta }:=\theta u+(1-\theta )v$ for $\theta \in [0,1]$ and $u,v\in X$. Let us take $A\in {\mathcal{A}}_f(x_{\theta })$. Then

$$\langle A,u-x_{\theta }\rangle \le f(u)-f(x_{\theta })-c{\parallel u-x_{\theta }\parallel }^{\gamma }.$$

Multiplying this inequality by θ, we obtain

$$\left({\mathrm{a}}^{\prime }\right)\theta (1-\theta )\langle A,u-v\rangle \le \theta f(u)-\theta f(x_{\theta })-c{(1-\theta )}^{\gamma }\theta {\parallel u-v\parallel }^{\gamma }.$$

In a similar way, since

$$\langle A,v-x_{\theta }\rangle \le f(v)-f(x_{\theta })-c{\parallel v-x_{\theta }\parallel }^{\gamma },$$

we get

$$\left({\mathrm{a}}^{″}\right)-\theta (1-\theta )\langle A,u-v\rangle \le (1-\theta )f(v)-(1-\theta )f(x_{\theta })-c(1-\theta ){\theta }^{\gamma }{\parallel u-v\parallel }^{\gamma }.$$

We deduce by addition of $({\mathrm{a}}^{\prime })$ and $({\mathrm{a}}^{″})$ that

$$f(x_{\theta })\le \theta f(u)+(1-\theta )f(v)-cg(\theta ){\parallel u-v\parallel }^{\gamma }\text{for all }u,v\in X,$$

where $g(\theta )=(1-\theta ){\theta }^{\gamma }+{(1-\theta )}^{\gamma }\theta$, so that f is γ-strongly convex. □

The next results are devoted to presenting some useful properties of the generalized Hessian matrix for a $C^{1,1}$ function in the finite-dimensional setting and a characterization of γ-strongly convex functions with the help of a second-order approximation.

Proposition 8

Let $\overline{x}\in X$, and let $f:X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ be convex and Fréchet differentiable at x̄. Suppose that f admits $(\mathrm{\nabla }f(\overline{x}),{\mathcal{B}}_f(\overline{x}))$ as a second-order approximation at x̄ and that ${\mathcal{B}}_f(\overline{x})$ is compact. Then there exists $B\in {\mathcal{B}}_f(\overline{x})$ such that

$$\underset{B\in {\mathcal{B}}_f(\overline{x})}{sup}\langle Bd,d\rangle \ge 0,\mathrm{\forall }d\in X.$$ 18

If f is 2-strongly convex, then we obtain

$$\underset{B\in {\mathcal{B}}_f(\overline{x})}{sup}\langle Bd,d\rangle \ge c{\parallel d\parallel }^2,\mathrm{\forall }d\in X,$$ 19

for some $c>0$.

Proof

We prove only the case where f is convex. In a similar way, we can prove the other case. Let $d\in X$ and $\epsilon >0$ be fixed. We get for n large enough $B_n\in {\mathcal{B}}_f(\overline{x})$ and $b_n\in [-1,1]$ such that

$$f(\overline{x}+\frac{1}{n}d)-f(\overline{x})=\frac{1}{n}\langle \mathrm{\nabla }f(\overline{x}),d\rangle +\frac{1}{n^2}\langle B_{n}d,d\rangle +\epsilon \frac{1}{n^2}{\parallel d\parallel }^2{b}_n.$$

Since f is convex, we obtain

$$\langle B_{n}d,d\rangle +\epsilon {\parallel d\parallel }^2{b}_n\ge 0.$$

By the compactness of ${\mathcal{B}}_f(\overline{x})$, extracting a subsequence if necessary, we may assume that there exits $B\in {\mathcal{B}}_f(\overline{x})$ such that $B_n$ converges to B; therefore

$$\langle Bd,d\rangle \ge 0,$$

and hence

$$\underset{B\in {\mathcal{B}}_f(\overline{x})}{sup}\langle Bd,d\rangle \ge 0,\mathrm{\forall }d\in X.$$

 □

When X is a finite-dimensional space, we get the following essential result.

Proposition 9

Let $f:{\mathbb{R}}^p\to \mathbb{R}$ be a $C^{1,1}$ function at x̄. Assume that f is γ-strongly convex. Then, for any $B\in {\partial }_H^{2}f(\overline{x})$, we have the following inequality:

$$\langle Bd,d\rangle \ge c{\parallel d\parallel }^{\gamma },\mathrm{\forall }d\in {\mathbb{R}}^p,$$ 20

for some $c>0$.

Proof

It is clear that $(\mathrm{\nabla }f(\overline{x}),\frac{1}{2}{\partial }_H^{2}f(\overline{x}))$ is a second-order approximation of f at x̄. Now let $B\in {\partial }_H^{2}f(\overline{x})$, so that there exists a sequence $(x_n)\in dom{\mathrm{\nabla }}^{2}f$ such that $x_n\to \overline{x}$ and ${\mathrm{\nabla }}^{2}f(x_n)\to B$. Since f is γ-strongly convex, there exists $c>0$ such that

$$\langle {\mathrm{\nabla }}^{2}f(x_n)d,d\rangle \ge c{\parallel d\parallel }^{\gamma },\mathrm{\forall }d\in {\mathbb{R}}^p,\mathrm{\forall }n\in \mathbb{N}.$$

Letting $n\to +\mathrm{\infty }$, we have

$$\langle Bd,d\rangle \ge c{\parallel d\parallel }^{\gamma },\mathrm{\forall }d\in {\mathbb{R}}^p.$$

 □

The preceding result shows that γ-strongly convex functions enjoy a very desirable property for generalized Hessian matrices. In fact, in this case, any matrix $B\in {\partial }_H^{2}f(\overline{x})$ is invertible. The next result proves the converse of Proposition 9. Let us first recall the following characterization of l.s.c. γ-strongly convex functions.

Theorem 2

Amahroq et al. [8]

Let f: $X\to \mathbb{R}\cup \{+\mathrm{\infty }\}$ be a proper and l.s.c. function. Then f is γ-strongly convex iff ${\partial }_{c}f$ is γ-strongly monotone, that is, there exists a positive real number c such that, for all $x,y\in X$, $x^{\ast }\in {\partial }_{c}f(x)$, and $y^{\ast }\in {\partial }_{c}f(y)$, we have

$$\langle x^{\ast }-y^{\ast },x-y\rangle \ge c{\parallel x-y\parallel }^{\gamma }.$$

We are now in position to state our main second result.

Theorem 3

Let $f:{\mathbb{R}}^p\to \mathbb{R}$ be a $C^{1,1}$ function. Assume that ${\partial }_H^{2}f(\cdot )$ satisfies relation (20) at any $x\in {\mathbb{R}}^p$. Then f is γ-strongly convex.

Proof

Let $t\in [0,1]$ and $u,v\in {\mathbb{R}}^p$. Define $\phi :\mathbb{R}\to \mathbb{R}$ as

$$\phi (t):=f(u+t(v-u)),$$

so that ${\phi }^{\prime }(t):=\langle \mathrm{\nabla }f(u+t(v-u)),v-u\rangle$. By the Lebourg mean value theorem [22] there exists $t_0\in ]0,1[$ such that

$${\phi }^{\prime }(1)-{\phi }^{\prime }(0)\in {\partial }_c{\phi }^{\prime }(t_0).$$

By using calculus rules it follows that

$${\phi }^{\prime }(1)-{\phi }^{\prime }(0)\in {\partial }_c{\phi }^{\prime }(t_0)\subset {\partial }_H^{2}f(u+t_0(v-u))(v-u)(v-u).$$

Hence, there exists $B_{t_0}\in {\partial }_H^{2}f(u+t_0(v-u))$ such that $\langle \mathrm{\nabla }f(v)-\mathrm{\nabla }f(u),v-u\rangle =\langle B_{t_0}(v-u),v-u\rangle$. The result follows from Theorem 2. □

Hiriart-Urruty et al. [19] have presented many examples of $C^{1,1}$ functions. The next proposition shows another example of a $C^{1,1}$ function.

Theorem 4

Let $f:H\to \mathbb{R}$ be continuous on a Hilbert space H. Suppose that f is convex (or 2-strongly convex) and that −f is 2-paraconvex. Then f is Fréchet differentiable on H, and for some $c>0$, we have that

$$\parallel \mathrm{\nabla }f(x)-\mathrm{\nabla }f(y)\parallel \le c\parallel x-y\parallel \mathit{\text{for all }}x,y\in H.$$ 21

Proof

Let $x_0\in X$. Clearly, f is locally Lipschitzian at $x_0$. Now let $x_1^{\ast }$ and $x_2^{\ast }$ be arbitrary elements of ${\partial }_{c}f(x_0)$ and ${\partial }_c(-f)(x_0)$, respectively. By [20], Thm. 3.4, there exists $c>0$ such that ${\partial }_c(-f)(x_0)={\partial }^{(2,c)}(-f)(x_0)$, and for any $y\in H$ and positive real θ, we have

$$(\mathrm{a})\theta \langle x_2^{\ast },y\rangle \le -f(x_0+\theta y)+f(x_0)+c{\theta }^2{\parallel y\parallel }^2$$

and

$$\left({\mathrm{a}}^{\prime }\right)\theta \langle x_1^{\ast },y\rangle \le f(x_0+\theta y)-f(x_0).$$

Adding (a) and (a′), we get

$$\theta \langle x_1^{\ast }+x_2^{\ast },y\rangle \le c{\theta }^2{\parallel y\parallel }^2,$$

and hence

$$\langle x_1^{\ast }+x_2^{\ast },y\rangle \le c\theta {\parallel y\parallel }^2.$$

Letting $\theta \to 0$, we have $\langle x_1^{\ast }+x_2^{\ast },y\rangle \le 0$, so that $x_1^{\ast }=-x_2^{\ast }$. Since $x_1^{\ast }$ and $x_2^{\ast }$ are arbitrary in ${\partial }_{c}f(x_0)$ and ${\partial }_c(-f)(x_0)$, it follows that ${\partial }_{c}f(x_0)$ is single-valued. Put ${\partial }_{c}f(x_0)=\{p(x_0)\}$. Since (a) and (a′) hold for any $\theta >0$ and $y\in H$, we deduce that, for $\theta =1$,

$$\langle p(x_0),y\rangle \le f(x_0+y)-f(x_0)$$

and

$$f(x_0+y)-f(x_0)-\langle p(x_0),y\rangle \le c{\parallel y\parallel }^2.$$

Hence, for all $y\ne 0$, we obtain

$$\frac{|f(x_0+y)-f(x_0)-\langle p(x_0),y\rangle |}{\parallel y\parallel }\le c\parallel y\parallel .$$ 22

Letting $\parallel y\parallel \to 0$ in (22), we conclude that f is Fréchet differentiable at $x_0$. Now since −f is 2-paraconvex and f is Fréchet differentiable, we may prove that there exists $c>0$ such that

$$-\langle \mathrm{\nabla }f(x),y-x\rangle \le -f(y)+f(x)+c{\parallel x-y\parallel }^2\text{for all }x,y\in H.$$ 23

For every $z\in H$, we have that

$$-f(z)\ge -f(x)+\langle \mathrm{\nabla }f(x),x\rangle -\langle \mathrm{\nabla }f(x),z\rangle -c{\parallel x-z\parallel }^2.$$

Thus

$$-f(z)\ge f^{\ast }(\mathrm{\nabla }f(x))-\langle \mathrm{\nabla }f(x),z\rangle -c{\parallel x-z\parallel }^2,$$

so that

$$\begin{array}{rl}& f^{\ast }(\mathrm{\nabla }f(y))\ge \langle \mathrm{\nabla }f(y),z\rangle -f(z),\\ & f^{\ast }(\mathrm{\nabla }f(y))\ge \langle \mathrm{\nabla }f(y),z\rangle +f^{\ast }(\mathrm{\nabla }f(x))-\langle \mathrm{\nabla }f(x),z\rangle -c{\parallel x-z\parallel }^2,\end{array}$$

and hence

$$\begin{array}{rl}& f^{\ast }(\mathrm{\nabla }f(y))-f^{\ast }(\mathrm{\nabla }f(x))-\langle \mathrm{\nabla }f(y)-\mathrm{\nabla }f(x),x\rangle \\ & \ge \langle \mathrm{\nabla }f(y)-\mathrm{\nabla }f(x),z-x\rangle -c{\parallel x-z\parallel }^2\\ & \ge \underset{z\in H}{sup}\{\langle \mathrm{\nabla }f(y)-\mathrm{\nabla }f(x),z-x\rangle -c{\parallel x-z\parallel }^2\}.\end{array}$$

This means that, for all $x,y\in H$,

$$f^{\ast }(\mathrm{\nabla }f(y))-f^{\ast }(\mathrm{\nabla }f(x))-\langle \mathrm{\nabla }f(y)-\mathrm{\nabla }f(x),x\rangle \ge \frac{1}{2c}{\parallel \mathrm{\nabla }f(y)-\mathrm{\nabla }f(x)\parallel }^2.$$

Changing the roles of x and y, we obtain

$$f^{\ast }(\mathrm{\nabla }f(x))-f^{\ast }(\mathrm{\nabla }f(y))-\langle \mathrm{\nabla }f(x)-\mathrm{\nabla }f(y),y\rangle \ge \frac{1}{2c}{\parallel \mathrm{\nabla }f(x)-\mathrm{\nabla }f(y)\parallel }^2.$$

So by addition we get

$$\langle \mathrm{\nabla }f(x)-\mathrm{\nabla }f(y),x-y\rangle \ge \frac{1}{c}{\parallel \mathrm{\nabla }f(x)-\mathrm{\nabla }f(y)\parallel }^2.$$ 24

Consequently, by the Cauchy-Schwarz inequality we obtain

$$\parallel \mathrm{\nabla }f(x)-\mathrm{\nabla }f(y)\parallel \le c\parallel x-y\parallel \text{for all }x,y\in H.$$

 □

Newton’s method

The aim of this section is to solve the Euler equation

$$\mathrm{\nabla }f(x)=0$$ 25

by Newton’s method. The classic assumption is that $f:{\mathbb{R}}^p\to \mathbb{R}$ a $C^2$ mapping and the Hessian matrix ${\mathrm{\nabla }}^{2}f(x)$ of f at x is nonsingular. Here we prove the convergence of a natural extension of Newton’s method to solve (25) assuming that $\mathrm{\nabla }f(\cdot )$ admits ${\beta }_f(\cdot )$ as a first-order approximation. Clearly, if $f:{\mathbb{R}}^p\to \mathbb{R}$ is a $C^{1,1}$ mapping, then using Corollary 1, we obtain that $\mathrm{\nabla }f(\cdot )$ admits ${\partial }_H^{2}f(\cdot )$ as a first-order approximation.

This algorithm has been proposed by Cominetti et al. [23] with $C^{1,1}$ data. Only some ideas were given, but it remains as an open question to state results on rate of convergence and local convergence of that algorithm. In the sequel, $f:{\mathbb{R}}^p\to \mathbb{R}$ is a Fréchet-differentiable mapping such that its Fréchet derivative admits a first-order approximation, and x̄ is a solution of (25).

Theorem 5

Let $f:{\mathbb{R}}^p\to \mathbb{R}$ be a Fréchet-differentiable function, and x̄ be a solution of (25). Let $\epsilon ,r,K>0$ be such that $\mathrm{\nabla }f(\cdot )$ admits ${\beta }_f(\overline{x})$ as a first-order approximation at x̄ such that, for each $x\in {\mathbb{B}}_{{\mathbb{R}}^p}(\overline{x},r)$, there exists an invertible element $B(x)\in {\mathcal{B}}_f(x)$ satisfying $\parallel B{(x)}^{-1}\parallel \le K$ and $\xi :=\epsilon K<1$. Then the sequence $(x_k)$ generated by Algorithm $(\mathcal{M})$ is well defined for every $x_0\in {\mathbb{B}}_{{\mathbb{R}}^p}(\overline{x},r)$ and converges linearly to x̄ with rate ξ.

Proof

Since $\mathrm{\nabla }f(\overline{x})=0$, we have

$$x_{k+1}-\overline{x}=B{(x_k)}^{-1}(\mathrm{\nabla }f(\overline{x})-\mathrm{\nabla }f(x_k)+B(x_k)(x_k-\overline{x})).$$

We inductively obtain that

$$\parallel x_{k+1}-\overline{x}\parallel \le K\parallel \mathrm{\nabla }f(\overline{x})-\mathrm{\nabla }f(x_k)+B(x_k)(x_k-\overline{x})\parallel .$$

Thus

$$\parallel x_{k+1}-\overline{x}\parallel \le \xi \parallel x_k-\overline{x}\parallel ,$$

which means that $x_{k+1}\in {\mathbb{B}}_{{\mathbb{R}}^p}(\overline{x},r)$, and we have $\parallel x_{k+1}-\overline{x}\parallel \le {\xi }^k\parallel x_0-\overline{x}\parallel$. Therefore the whole sequence $(x_k)$ is well defined and converges to x̄. □

Now let us consider the following algorithm under less assumptions.

Theorem 6

Let U be an open set of ${\mathbb{R}}^p$, $x_0\in U$, and $f:{\mathbb{R}}^p\to \mathbb{R}$ be a Fréchet-differentiable function on U. Let $\epsilon ,r,K>0$ be such that $\mathrm{\nabla }f(\cdot )$ admits ${\beta }_f(x_0)$ as a strict first-order approximation at $x_0$ such that, for each $x\in {\mathbb{B}}_{{\mathbb{R}}^p}(x_0,r)$, there exists a right inverse of $B(x)\in {\beta }_f(x_0)$, denoted by $\tilde{B}(x)$, satisfying $\parallel \tilde{B}(x)(\cdot )\parallel \le K\parallel \cdot \parallel$ and $\xi :=\epsilon K<1$.

If $\parallel \mathrm{\nabla }f(x_0)\parallel \le K^{-1}(1-\xi )r$ and ∇f is continuous, then the sequence $(x_k)$ generated by Algorithm $({\mathcal{M}}^{\prime })$ is well defined and converges to a solution x̄ of (25). Moreover, we have $\parallel x_k-\overline{x}\parallel \le r{\xi }^k$ for all $k\in \mathbb{N}$ and $\parallel \overline{x}-x_0\parallel \le \parallel \mathrm{\nabla }f(x_0)\parallel K{(1-\xi )}^{-1}<r$.

Proof

We prove by induction that $x_k\in x_0+r{\mathbb{B}}_{{\mathbb{R}}^p}$, $\parallel x_{k+1}-x_k\parallel \le K{\xi }^k\parallel \mathrm{\nabla }f(x_0)\parallel$, and $\parallel \mathrm{\nabla }f(x_k)\parallel \le {\xi }^k\parallel \mathrm{\nabla }f(x_0)\parallel$ for all $k\in \mathbb{N}$. For $k=0$, these relations are obvious. Assuming that they are valid for $k<n$, we get

$$\begin{array}{rl}\parallel x_n-x_0\parallel & \le \underset{k=0}{\sum ^{n-1}}\parallel x_{k+1}-x_k\parallel \le K\parallel \mathrm{\nabla }f(x_0)\parallel \underset{k=0}{\sum ^{\mathrm{\infty }}}{\xi }^k\\ & \le K\parallel \mathrm{\nabla }f(x_0)\parallel {(1-\xi )}^{-1}<r.\end{array}$$

Thus $x_n\in x_0+r{\mathbb{B}}_{{\mathbb{R}}^p}$ and since $\mathrm{\nabla }f(x_{n-1})+B(x_{n-1})(x_n-x_{n-1})=0$, from Algorithm $({\mathcal{M}}^{\prime })$ we have

$$\begin{array}{rl}\parallel \mathrm{\nabla }f(x_n)\parallel & \le \parallel \mathrm{\nabla }f(x_n)-\mathrm{\nabla }f(x_{n-1})-B(x_{n-1})(x_n-x_{n-1})\parallel \le \epsilon \parallel x_n-x_{n-1}\parallel \\ & \le {\xi }^n\parallel \mathrm{\nabla }f(x_0)\parallel \end{array}$$

and

$$\parallel x_{n+1}-x_n\parallel \le K{\xi }^n\parallel \mathrm{\nabla }f(x_0)\parallel .$$

Since $\xi <1$, the sequence $(x_n)$ is a Cauchy sequence and hence converges to some $\overline{x}\in {\mathbb{R}}^p$ with $\parallel x_0-\overline{x}\parallel <r$. Since ∇f is a continuous function, we get $\mathrm{\nabla }f(\overline{x})=0$. □

Conclusions

In this paper, we investigate the concept of first- and second-order approximations to generalize some results such as optimality conditions for a subclass of convex functions called strongly convex functions of order γ. We also present an extension of Newton’s method to solve the Euler equation under weak assumptions.