Abstract
Here we perform the univariate quantitative approximation, ordinary and fractional, of Banach space valued continuous functions on a compact interval or all the real line by quasi-interpolation Banach space valued neural network operators. These approximations are derived by establishing Jackson type inequalities involving the modulus of continuity of the engaged function or its Banach space valued high order derivative or fractional derivatives. Our operators are defined by using a density function generated by the Richards curve, which is generalized logistic function. The approximations are pointwise and of the uniform norm. The related Banach space valued feed-forward neural networks are with one hidden layer.
Keywords: Richards curve function, Banach space valued neural network approximation, Banach space valued quasi-interpolation operator, Modulus of continuity, Banach space valued Caputo fractional derivative, Banach space valued fractional approximation
Introduction
The first author in [1, 2], see Chapters 2–5, was the first to establish neural network approximation to continuous functions with rates by very specifically defined neural network operators of Cardaliagnet–Euvrard and “Squashing” types, by employing the modulus of continuity of the engaged function or its high order derivative, and producing very tight Jackson type inequalities. He treats there both the univariate and multivariate cases. The defining these operators “bell-shaped ”and “squashing ”functions are assumed to be of compact suport. Also in [2] he gives the Nth order asymptotic expansion for the error of weak approximation of these two operators to a special natural class of smooth functions, see Chapters 4–5 there.
Again the first author inspired by [14], continued his studies on neural networks approximation by introducing and using the proper quasi-interpolation operators of sigmoidal and hyperbolic tangent type which resulted into [3–7], by treating both the univariate and multivariate cases. He did also the corresponding fractional cases [8, 9, 13].
The authors here perform Richards’s curve activated neural network approximations to continuous functions over compact intervals of the real line or over the whole with valued to an arbitrary Banach space . Finally they treat completely the related X-valued fractional approximation. All convergences here are with rates expressed via the modulus of continuity of the involved function or its X-valued high order derivative, or X-valued fractional derivatives and given by very tight Jackson type inequalities.
Our compact intervals are not necessarily symmetric to the origin. Some of our upper bounds to error quantity are very flexible and general. In preparation to prove our results we establish important properties of the basic density function defining our operators which is induced by the Richards curve, which is a sigmoid function. Richards’s curve among others has been used for modeling COVID-19 infection trajectory [17].
Feed-forward X-valued neural networks (FNNs) with one hidden layer, the only type of networks we deal with in this article, are mathematically expressed as
where for , are the thresholds, are the connection weights, are the coefficients, is the inner product of and x, and is the activation function of the network. About neural networks in general read [15, 18, 20].
Preliminaries
A Richards’s curve is
| 1 |
which is strictly increasing on , and it is a sigmoid function, in particular this is a generalized logistic function [21].
See that
| 2 |
We consider the following activation function
| 3 |
which is , all .
The function has great applications in epidemiology and especially in COVID-19 modeling infection trajectories [17].
We have that
| 4 |
We notice that
| 5 |
So that G is an even function.
We have that
that is
| 6 |
Let , we have that
| 7 |
So for and G(x) is strictly decreasing.
Let now then and then so that again and G(x) is strictly decreasing over
Thus G(x) is strictly decreasing on .
Clearly, G(x) is strictly increasing on , and
We observe that
| 8 |
That is, the x-axis is the horizontal asymptote for G.
Conclusion, G is a bell symmetric function with maximum
| 9 |
We need
Theorem 1
It holds
| 10 |
Proof
We observe that
Furthermore ()
| 11 |
Similarly, it holds
| 12 |
Therefore we derive
| 13 |
and
| 14 |
Adding the last two equations we get
| 15 |
Since
we have that
| 16 |
giving
Remark 2
Because G is even it holds
Hence
and
| 17 |
Theorem 3
It holds
| 18 |
Proof
We observe that
So is a density function.
We need
Theorem 4
Let , and with . It holds
| 19 |
Proof
We have that
Let . That is . Applying the mean value theorem we get
where .
Notice that
Thus, we have
| 20 |
| 21 |
for , We have found that
| 22 |
for ,
Denote by the integral part of the number and by the ceiling of the number.
Theorem 5
Let and so that . It holds
| 23 |
Proof
Let . We see that
| 24 |
.
We can choose such that
Therefore we get that
| 25 |
and
| 26 |
That is
| 27 |
proving the claim.
We make
Remark 6
We also notice that
| 28 |
(call , )
Therefore
| 29 |
Similarly,
(call )
| 30 |
Therefore again
| 31 |
Here we find that
| 32 |
Note 7
Let . For large enough n we always obtain . Also , iff .
In general it holds (by , ) that is
| 33 |
Let be a Banach space.
Definition 8
Let and . We introduce and define the X-valued linear neural network operators
| 34 |
Clearly here .
For convenience we use the same for real valued functions when needed. We study here the pointwise and uniform convergence of to with rates.
For convenience, also we call
| 35 |
(similarly, can be defined for real valued functions) that is
| 36 |
So that
| 37 |
Consequently, we derive that
| 38 |
We will estimate the right hand side of the last quantity.
For that we need, for the first modulus of continuity
| 39 |
Similarly, it is defined for (uniformly continuous and bounded functions from into X), for (continuous and bounded X-valued), and for (uniformly continuous).
The fact or , is equivalent to , see [11].
We make
Definition 9
When , or , we define
| 40 |
, , the X-valued quasi-interpolation neural network operator.
We make
Remark 10
We have that
and
| 41 |
and
and finally
| 42 |
a convergent series in .
So, the series is absolutely convergent in X, hence it is convergent in X and . We denote by , for , similarly it is defined for
Main results
We present a set of X-valued neural network approximations to a function given with rates.
Theorem 11
Let , , , Then
(i)
| 43 |
and
(ii)
| 44 |
We get that , pointwise and uniformly.
Proof
We see that
| 45 |
| 46 |
That is
| 47 |
Using the last equality we derive (43).
Next we give
Theorem 12
Let , , , Then
-
(i)
and48 -
(ii)
For we get , pointwise and uniformly.49
Proof
We observe that
| 50 |
| 51 |
proving the claim.
We need the X-valued Taylor’s formula in an appropiate form:
Theorem 13
[10, 12] Let , and , where and X is a Banach space. Let any . Then
| 52 |
The derivatives , , are defined like the numerical ones, see [22], p. 83. The integral in (52) is of Bochner type, see [19].
By [12, 16] we have that: if , then and
In the next we discuss high order neural network X-valued approximation by using the smoothness of f.
Theorem 14
Let , , , , and . Then
-
(i)

53 -
(ii)assume further , for some , it holds
and54 -
(iii)
Again we obtain , pointwise and uniformly.55
Proof
Next we apply the X-valued Taylor’s formula with Bochner integral remainder (52). We have (here )
| 56 |
Then
| 57 |
Hence
| 58 |
Thus
| 59 |
where
| 60 |
We assume that , which is always the case for large enough , that is when
Thus or
Let
| 61 |
in the case of , we find that
| 62 |
for or
We prove it next.
-
(i)Indeed, for the case of , we have
63 -
(ii)for the case of , we have
We have proved (62).64
We treat again , see (61), but differently:
Notice also for that
| 65 |
Next assume , then
| 66 |
Thus
| 67 |
in the two cases.
Therefore
| 68 |
Hence
| 69 |
That is
| 70 |
We further see that
| 71 |
where is defined similarly for real valued functions.
Therefore
| 72 |
That is
| 73 |
for
Putting things together we have proved
| 74 |
that is establishing the theorem.
All integrals from now on are of Bochner type [19].
We need
Definition 15
[12] Let , X be a Banach space, ; , ( is the ceiling of the number), . We assume that . We call the Caputo–Bochner left fractional derivative of order :
| 75 |
If , we set the ordinary X-valued derivative (defined similar to numerical one, see [22], p. 83), and also set
By [12], exists almost everywhere in and .
If , then by [12], hence
We mention
Lemma 16
[11] Let , , , and . Then .
We mention
Definition 17
[10] Let , X be a Banach space, , . We assume that , where . We call the Caputo–Bochner right fractional derivative of order :
| 76 |
We observe that for , and
By [10], exists almost everywhere on and .
If , and by [10], hence
We need
Lemma 18
([11]) Let , , , , . Then .
We mention the left fractional Taylor formula
Theorem 19
[12] Let and where and X is a Banach space, and let . Then
| 77 |
We also mention the right fractional Taylor formula
Theorem 20
[10] Let , X be a Banach space, , , . Then
| 78 |
Convention 21
We assume that
| 79 |
and
| 80 |
for all
We mention
Proposition 22
[11] Let , , . Then is continuous in .
Proposition 23
[11] Let , , . Then is continuous in .
We also mention
Proposition 24
[11] Let , , , and
| 81 |
for all
Then is continuous in .
Proposition 25
[11] Let , , , and
| 82 |
for all
Then is continuous in .
Corollary 26
[11] Let , , , . Then are jointly continuous functions in from into X, X is a Banach space.
We need
Theorem 27
[11] Let be jointly continuous, X is a Banach space. Consider
| 83 |
,
Then G is continuous on
Theorem 28
[11] Let be jointly continuous, X is a Banach space. Then
| 84 |
, is continuous in , .
We make
Remark 29
[11] Let , , , , . Then
| 85 |
Thus we observe
| 86 |
Consequently
| 87 |
Similarly, let , , , , , then
| 88 |
So for , , , , , we find
| 89 |
and
| 90 |
By [12] we get that , and by [10] we obtain that
We present the following X-valued fractional approximation result by neural networks.
Theorem 30
Let , , , , , , Then
-
(i)


91 -
(ii)if , for , we have

92 -
(iii)
and
93 -
(iv)
Above, when the sum
94
As we see here we obtain X-valued fractionally type pointwise and uniform convergence with rates of the unit operator, as
Proof
Let . We have that
From Theorem 19, we get by the left Caputo fractional Taylor formula that
| 95 |
for all
Also from Theorem 20, using the right Caputo fractional Taylor formula we get
| 96 |
for all
Hence we have
| 97 |
for all , iff , and
| 98 |
for all , iff
Therefore it holds
| 99 |
and
| 100 |
Adding the last two equalities obtain
| 101 |
So we have derived
| 102 |
where
| 103 |
We set
| 104 |
and
| 105 |
i.e.
| 106 |
We assume , , which is always the case for large enough , that is when . It is always true that either or
For , we consider
| 107 |
| 108 |
That is
| 109 |
for
Also we have in case of that
| 110 |
That is when , then
| 111 |
Consequently we obtain
| 112 |
| 113 |
So we have proved that
| 114 |
Next when we consider
| 115 |
| 116 |
Therefore when we get that
That is
| 117 |
In case of we have
| 118 |
So when we derived that
| 119 |
Similarly we have that
| 120 |
| 121 |
So we have proved that
| 122 |
Therefore
| 123 |
From the proof of Theorem 14 we get that
| 124 |
for
Putting things together, we have established
| 125 |
| 126 |
As a result we derive
| 127 |
We further have that
| 128 |
Hence it holds
| 129 |
We observe the following:
We have
| 130 |
and
| 131 |
That is
| 132 |
and
| 133 |
Similarly we have
| 134 |
Thus we get
| 135 |
Hence
| 136 |
and
| 137 |
| 138 |
and
| 139 |
That is
We finally notice that
| 140 |
Therefore we get
| 141 |
The proof of the theorem is now finished.
Next we apply Theorem 30 for
Theorem 31
Let , , , Then
-
(i)
and142 -
(ii)
143
When we derive
Corollary 32
Let , , , Then
-
(i)
and144 -
(ii)
145
We make
Remark 33
Some convergence analysis follows based on Corollary 32.
Let , , , We elaborate on ( 145). Assume that
| 146 |
and
| 147 |
, , where .
Then it holds
| 148 |
where
The other summand of the right hand side of (145), for large enough n, converges to zero at the speed , so it is about , where is a constant.
Then, for large enough , by (145), (148) and the above comment, we obtain that
| 149 |
where , converging to zero at the high speed of
In Theorem 11, for and for large enough , the speed is . So by (149), converges much faster to zero. The last comes because we assumed differentiability of f. Notice that in Corollary 32 no initial condition is assumed.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
George A. Anastassiou, Email: ganastss@memphis.edu
Seda Karateke, Email: sedakarateke@topkapi.edu.tr.
References
- 1.Anastassiou GA. Rate of convergence of some neural network operators to the unit-univariate case. J. Math. Anal. Appl. 1997;212:237–262. doi: 10.1006/jmaa.1997.5494. [DOI] [Google Scholar]
- 2.Anastassiou GA. Quantitative Approximations. Boca Raton, New York: Chapman & Hall/CRC; 2001. [Google Scholar]
- 3.Anastassiou, G.A.: Inteligent Systems: Approximation by Artificial Neural Networks, Intelligent Systems Reference Library, vol. 19. Springer, Heidelberg (2011)
- 4.Anastassiou GA. Univariate hyperbolic tangent neural network approximation. Math. Comput. Model. 2011;53:1111–1132. doi: 10.1016/j.mcm.2010.11.072. [DOI] [Google Scholar]
- 5.Anastassiou GA. Multivariate hyperbolic tangent neural network approximation. Comput. Math. 2011;61:809–821. [Google Scholar]
- 6.Anastassiou GA. Multivariate sigmoidal neural network approximation. Neural Netw. 2011;24:378–386. doi: 10.1016/j.neunet.2011.01.003. [DOI] [PubMed] [Google Scholar]
- 7.Anastassiou GA. Univariate sigmoidal neural network approximation. J. Comput. Anal. Appl. 2012;14(4):659–690. [Google Scholar]
- 8.Anastassiou GA. Fractional neural network approximation. Comput. Math. Appl. 2012;64:1655–1676. doi: 10.1016/j.camwa.2012.01.019. [DOI] [Google Scholar]
- 9.Anastassiou GA. Intelligent Systems II: Complete Approximation by Neural Network Operators. Heidelberg, New York: Springer; 2016. [Google Scholar]
- 10.Anastassiou GA. Strong right fractional calculus for banach space valued functions. Revista Proyecciones. 2017;36(1):149–186. doi: 10.4067/S0716-09172017000100009. [DOI] [Google Scholar]
- 11.Anastassiou GA. Vector fractional Korovkin type Approximations. Dyn. Syst. Appl. 2017;26:81–104. [Google Scholar]
- 12.Anastassiou GA. A strong fractional calculus theory for Banach space valued functions. Nonlinear Funct. Anal. Appl. (Korea) 2017;22(3):495–524. [Google Scholar]
- 13.Anastassiou GA. Nonlinearity: Ordinary and Fractional Approximations by Sublinear and Max-Product Operators. Heidelberg, New York: Springer; 2018. [Google Scholar]
- 14.Chen Z, Cao F. The approximation operators with sigmoidal functions. Comput. Math. Appl. 2009;58:758–765. doi: 10.1016/j.camwa.2009.05.001. [DOI] [Google Scholar]
- 15.Haykin S. Neural Networks: A Comprehensive Foundation. 2. New York: Prentice Hall; 1998. [Google Scholar]
- 16.Kreuter, M.: Sobolev Spaces of Vector-valued functions, Ulm Univ., Master Thesis in Math., Ulm, Germany (2015)
- 17.Lee SY, Lei B, Mallick B. Estimation of COVID-19 spread curves integrating global data and borrowing information. PLoS One. 2020;15(7):e0236860. doi: 10.1371/journal.pone.0236860. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.McCulloch W, Pitts W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943;7:115–133. doi: 10.1007/BF02478259. [DOI] [PubMed] [Google Scholar]
- 19.Mikusinski J. The Bochner Integral. New York: Academic Press; 1978. [Google Scholar]
- 20.Mitchell TM. Machine Learning. New York: WCB-McGraw-Hill; 1997. [Google Scholar]
- 21.Richards FJ. A Flexible Growth Function for Empirical Use. J. Exp. Bot. 1959;10(29):290–300. doi: 10.1093/jxb/10.2.290. [DOI] [Google Scholar]
- 22.Shilov GE. Elementary Functional Analysis. New York: Dover Publications Inc; 1996. [Google Scholar]
