A New Robust Method for Nonlinear Regression

Abstract

Background

When outliers are present, the least squares method of nonlinear regression performs poorly. The main purpose of this paper is to provide a robust alternative technique to the Ordinary Least Squares nonlinear regression method. This new robust nonlinear regression method can provide accurate parameter estimates when outliers and/or influential observations are present.

Method

Real and simulated data for drug concentration and tumor size-metastasis are used to assess the performance of this new estimator. Monte Carlo simulations are performed to evaluate the robustness of our new method in comparison with the Ordinary Least Squares method.

Results

In simulated data with outliers, this new estimator of regression parameters seems to outperform the Ordinary Least Squares with respect to bias, mean squared errors, and mean estimated parameters. Two algorithms have been proposed. Additionally and for the sake of computational ease and illustration, a Mathematica program has been provided in the Appendix.

Conclusion

The accuracy of our robust technique is superior to that of the Ordinary Least Squares. The robustness and simplicity of computations make this new technique more appropriate and useful tool for the analysis of nonlinear regressions.

Background

Nonlinear regression is one of the most popular and widely used models in analyzing the effect of explanatory variables on a response variable and it has many applications in biomedical research. With the presence of outliers or influential observations in the data, the ordinary least squares method can result in misleading values for the parameters of the nonlinear regression and the hypothesis testing, and predictions may no longer be reliable. The main purpose of robust nonlinear regression is to fit a model to the data that gives resilient results in the presence of influential observations, leverage points and/or outliers. Rousseeuw and Leroy [1] defined vertical outliers as those data points with outlying values in the direction of the response variable, while leverage points are outliers in the direction of covariates. An observation may be influential if its removal would significantly alter the parameter estimates. Edgeworth [2] proposed the Least Absolute Deviation as a robust method. Huber [3] introduced the method of M-estimation. Rousseeuw [4] introduced the Least Trimmed Squares-estimates. The S-estimator was introduced by Rousseeuw and Yohai [5]. Yohai and Zammar [6] introduced the τ-estimator of linear regression coefficients. It is a high efficiency estimator and has a high breakdown point. Tabatabai and Argyros [7] extended the τ-estimates to the nonlinear regression models. Stromberg [8] introduced algorithms for Yohai's MM estimator of nonlinear regression and Rousseeuw's least median estimators of nonlinear regression. Tabatabai et al. [9] introduced the TELBS robust linear regression method.

In Medical, biological and pharmaceutical research and development nonlinear regression analysis has been a major tool for investigating the effect of multiple explanatory variables on a response variable when the data follows a nonlinear pattern. When outliers and influential observations are present, nonlinear least squares performs poorly. In this paper we introduce a new robust nonlinear regression method capable of handling such cases. Minn et al. [10] showed that lung tumor size can lead to metastasis. Also, aggressive tumor growth is a marker for cells destined to metastasize. They validated their statement by analyzing the lung metastasis gene-expression signature using a nonlinear model. Arisio et al. [11] study of breast cancer confirmed that the size of tumor is an important predictor of axillary lymph node metastases. Ramaswamy, et al. [12]) found that gene-expression signature is a significant factor associated with metastasis in solid tumors carrying such gene expressions. Maffuz et al. [13] showed that pure ductal carcinoma in situ is not associated with lymphatic metastasis independently of tumor size. Hense et al. [14] found that the occurrence and primary metastases in Ewing tumors is related to tumor size, pelvic site and malignant peripheral neuroectodermal tumors. Umbreit et al. [15] studied a group of patients who had undergone surgical resection for a unilateral, sporadic renal tumor. They concluded that tumor size is significantly associated with metastasis in patients suffering from renal masses. Wu et al. [16] retrospectively analyzed 666 patients with nasopharyngeal carcinoma and concluded that the tumor volume was correlated with cervical lymph node metastasis as well as distant metastasis after radiation therapy. In computer vision, robust regression methods have been used extensively to estimate surface model parameters in small image regions and imaging geometry of multiple cameras. Coras et al. [17] used nonlinear regression and showed that micromolar doses of peroxisome prolefector-activated receptor γ reduce glioma cell proliferation.

Roth [18] applied nonlinear sigmoidal curves to monitor the accumulation of polymerase chain reaction products at the end of each cycle by fluorescence. In human blood samples, Kropf et al. [19] found a nonlinear binding association between transforming growth factor beta1 (TGF -β₁) and α₂-Macroglobulin as well as TGF-β₁ and latency-associated peptide (LAP). Yang and Richmond [20] used nonlinear least squares to estimate the effective concentration of unlabeled human interferon-inducible protein 10 that yields 50% maximal binding of iodinated protein 10 to chemokine receptor CXCR3. Hao et al. [21] examined the significance of Nav1.5 protein in cellular processes by applying a nonlinear regression which relates the gene expression of Nav 1.5 protein and TGF-β₁ as well as Nav 1.5 protein and vimentin. TGF–β families are important factors in regulation of tumor initiation, progression, and metastatic activities, Bierie, et al. [22]. Coras et al. [17] applied nonlinear regression models to show that traglitazone concentration has a tendency to inhibit 1 TGF-β₁ release in glioma cell culture.

This paper introduces a new robust nonlinear regression estimator. This new method for robust nonlinear regression has a bounded influence and high breakdown point and asymptotic efficiency under normal distribution and is able to estimate the parameters of nonlinear regression in such a way that is close to the parameter estimates we would have estimated with the absence of outliers in the data. In addition, this new robust nonlinear regression method is computationally simple enough to be used by practitioners.

Methods and Models

We begin with the introduction of our new robust nonlinear regression model. The introduction of the model is followed by two algorithms describing its implementation. We then apply this new model to a real data set with an outlier present. In addition, we will analyze a problem involving tumor size and metastases with and without outliers. Monte Carlo simulations are also performed to evaluate the robustness of our method, in comparison with the ordinary least squares method.

Robust nonlinear regression model

Consider the general nonlinear model of the form

$$y_i=g(\theta ;x_i)+{\epsilon }_i,i=1,2,\dots ,n$$

Where y₁, y₂,..., y_n is a sample of n observations with k predictor variables in the model and the parameter vector θ=(θ₁, θ₂,..., θ_p ). The errors ε_i's are random variables. In a designed experiment, x_ij's are fixed but when x_ij's are observational, they are random variables. The predictor can be fixed, random, or mixed. The ordinary least squares estimate of the parameter vector θ is given by

$$\widehat{\theta }=\underset{\theta \in R^p}{\mathrm{argmin}}\sum _{i=1}^n{\epsilon }_i^2,$$

and the robust estimate of the parameter vector θ is derived by

$$\widehat{\theta }=\underset{\theta \in R^p}{\arg \min }{\sum }_{i=1}^n\frac{p_{\omega }\left(t_i\right)}{L_i},$$ (1)

where the function ρ_ω(x) is defined as

$${\rho }_{\omega }\left(x\right)=1-\mathrm{Sech}\left(\omega x\right),$$

and the positive real number ω is called the tuning constant. The function Sech(·) is the hyperbolic secant function and t_i's are defined by

$$t_i=\frac{(1-h_{ii})(y_i-g(\theta ;x_i))}{\sigma },$$ (2)

where σ is the error standard deviation and h_ii's are the diagonal elements of the matrix H of the form

$H=Diag\left[G{\left(G^{t}G\right)}^{-1}{G}^t\right]$

where the matrix G is defined as

$$G=\left(\begin{array}{ccc}\hfill \frac{\partial g(\theta ;x_1)}{\partial {\theta }_1}\hfill & \hfill \frac{\partial g(\theta ;x_1)}{\partial {\theta }_2}\dots \hfill & \hfill \frac{\partial g(\theta ;x_1)}{\partial {\theta }_p}\hfill \\ \hfill \frac{\partial g(\theta ;x_2)}{\partial {\theta }_1}\hfill & \hfill \frac{\partial g(\theta ;x_2)}{\partial {\theta }_2}\dots \hfill & \hfill \frac{\partial g(\theta ;x_2)}{\partial {\theta }_p}\hfill \\ \hfill \frac{\partial g(\theta ;x_n)}{\partial {\theta }_1}\hfill & \hfill \frac{\partial g(\theta ;x_n)}{\partial {\theta }_2}\dots \hfill & \hfill \frac{\partial g(\theta ;x_n)}{\partial {\theta }_p}\hfill \end{array}\right)=\left(G_1{G}_2\dots G_p\right).$$

For j =1,2,...,k, we define

$$M_j=Median\{\mid x_{1j}\mid ,\mid x_{2j}\mid ,..,\mid x_{nj}\mid \}$$

and for i =1,2,..., n, we define

$$L_i=\sum _{j=1}^{k}Max\{M_j,\mid x_{ij}\mid \}$$

If σ is unknown, one may use one of the following two estimators of σ which were proposed by Rousseeuw and Croux [23].

$$\begin{array}{cc}\hfill & \widehat{\sigma }=1.1926\underset{\{i:1\le i\le n\}}{Median}\left(\underset{\{j:1\le j\le n\}}{Median}\mid r_i-r_j\mid \right)\hfill \\ \hfill & \text{or}\widehat{\sigma }=2.2219{\{\mid r_i-r_j\mid ;i<j,i,j=1,\dots ,n\}}_{\left(l\right)},\hfill \end{array}$$ (4)

where $r_i=y_i-g(\widehat{\theta };x_i)$ and $l=\left(\begin{array}{c}\hfill \left[\frac{n}{2}\right]+1\hfill \\ \hfill 2\hfill \end{array}\right)$ is the binomial coefficient and {.}_(l) is the l th order statistic.

The above estimators of σ have high breakdown points. Under the normality assumption for error terms, the estimators given in (3) and (4) have higher efficiency than median absolute deviation (MAD). In this paper all of our computations are performed using formula (3).

The function ρ_ω: R→R is a differentiable function satisfying the following properties:

1. ρ_ω(0)=0,

2. ρ_ω is bounded,

3. $\forall x\in \mathrm{R},{\rho }_{\omega }\left(x\right)\ge 0$,

4. $\forall x\in \mathrm{R},{\rho }_{\omega }\left(x\right)={\rho }_{\omega }(-x)$,

5. $\underset{x\to \infty }{\lim }{\rho }_{\omega }\left(x\right)=\underset{x\to -\infty }{\lim }{\rho }_{\omega }\left(x\right)=1$,

6. $\forall a,b\in R,\mid a\mid >\mid b\mid \Rightarrow {\rho }_{\omega }\left(a\right)\ge {\rho }_{\omega }\left(b\right)$,

7. $\underset{\mid x\mid \to \infty }{\lim }\frac{d{\rho }_{\omega }\left(x\right)}{dx}=0$.

8. $\forall \kappa >0,\underset{x\to \infty }{\lim }\frac{{\rho }_{\omega }\left(\kappa x\right)}{{\rho }_{\omega }\left(x\right)}=1$.

Taking the partial derivatives of (1) with respect to parameters and setting them equal to zero results in the following system of equations

$$\sum _{i=1}^n\frac{{\psi }_{\omega }\left(t_i\right)}{L_i}\frac{\partial t_i}{\partial {\beta }_j}=0,j=1,2,\dots ,p,$$ (5)

where ψ_ω is the derivative of ρ_ω which is equal to

$${\psi }_{\omega }\left(x\right)=\omega Sech\left(\omega x\right)Tanh\left(\omega x\right)$$

Define the weights w_i as

$${\mathrm{w}}_{\mathrm{i}}=\frac{\psi \left(t_i\right)(1-h_{ii})}{\sigma (y_i-g(\theta ;x_i))L_i}.$$ (6)

Then for j =1, 2,..., p, the equation (5) can be written as

$$\sum _{i=1}^n{w}_i(y_i-g(\theta ;x_i))\frac{\partial g(\theta ;x_i)}{\partial {\theta }_j}=0.$$

The matrix of weights, W is a diagonal matrix whose elements on the main diagonals are w₁,w₂,...,w_n, and the estimator of the parameter vector θ is given by

$$\widehat{\theta (X,y)}={\left({\mathrm{G}}^{\mathrm{t}}\mathrm{WG}\right)}^{-1}{\mathrm{G}}^{\mathrm{t}}\mathrm{Wy}.$$

If g(θ;x_i) is linear function of parameters, then the above model would be identical to TELBS robust linear regression model. Asymptotically, $\widehat{\theta }$ has a normal distribution with mean θ and variance-covariance matrix of the form

$$V=\frac{{\sigma }^{2}E\left({\psi }_{\omega }^2\left(t\right)\right)}{{\left[E\left({\psi }_{\omega }^{\prime }\left(t\right)\right)\right]}^2}E\left({\left(G^{t}G\right)}^{-1}\right),$$

where

$$E\left[{\psi }_{\omega }^{\prime }\left(t\right)\right]={\int }_{-\infty }^{\infty }{\psi }_{\omega }^{\prime }\left(t\right)\frac{e\frac{t^2}{2}}{\sqrt{2\pi }}dt$$

and

$$E\left[{\psi }_{\omega }^2\left(t\right)\right]={\int }_{-\infty }^{\infty }{\psi }_{\omega }^2\left(t\right)\frac{e\frac{t^2}{2}}{\sqrt{2\pi }}dt.$$

The function ${\psi }_{\omega }^{\prime }\left(t\right)$ is defined as

$${\psi }_{\omega }^{\prime }\left(t\right)={\omega }^2[Sec{h}^3\left(\omega \mathrm{t}\right)-\mathrm{Sech}\left(\omega \mathrm{t}\right){\mathrm{Tanh}}^2\left(\omega \mathrm{t}\right)].$$

Under the assumption of normality for the underlying distribution, the asymptotic efficiency, Aeff, is defined as

$$Aeff=\frac{{\left(E\left[{\psi }_{\omega }^{\prime }\left(t\right)\right]\right)}^2}{E\left[{\psi }_{\omega }^2\left(t\right)\right]}.$$ (7)

The tuning constant ω can be calculated by solving equation (7) for ω.

An estimate for the variance-covariance matrix is derived and given as follows

$$\widehat{V}=\frac{n^2{\widehat{\sigma }}^2\sum _{i=1}^n{\psi }_{\omega }^2\left({\mathrm{t}}_{\mathrm{i}}\right)}{(n-p){\left(\sum _{i=1}^n{\psi }_{\omega }^{\prime }\left({\mathrm{t}}_{\mathrm{i}}\right)\right)}^2}{\left(G^{t}G\right)}^{-1}.$$

The robust deviance is defined as

$$D=2{\widehat{\sigma }}^2\sum _{i=1}^n\frac{1-Sech\left(\omega t_i\right)}{L_i}.$$

The deviance plays a major role in model fitting. A smaller value of deviance is preferred over larger values. Following Akaike Criterion [24] and Ronchetti [25], the robust equivalence of AIC is denoted by AICR, and is given by

$$AICR=\frac{D}{{\widehat{\sigma }}^2}+2p\frac{E\left[{\psi }_{\omega }^2\left(t\right)\right]}{E\left[{\psi }_{\omega }^{\prime }\left(t\right)\right]},$$

and the Robust Schwarz Information Criterion BICR is given by

$$BICR=\frac{D}{{\widehat{\sigma }}^2}+2p\ln \left(n\right).$$

A robust coefficient of determination is given by

$$R^2=1-{\left(\frac{\underset{\{i:1\le i\le n\}}{Median}\mid r_i\mid }{\underset{\{i:1\le i\le n\}}{Median}\mid y_i\mid }\right)}^2.$$

For more details, see Rosseeuw and Leroy [1].

There are numerous variable selection techniques available in the literature. One may use the stepwise procedure that may involve in forward selection or backward elimination. For each set S ⊆{x₁, x₂..., x_p} of explanatory variables, the robust final predicted error of Maronna et al. [26] is denoted by RFPE(S) and is defined as

$$REPE\left(s\right)=\frac{\sum _{i=1}^n{\rho }_{\omega }\left(t_i\right)}{n}+\frac{\#\left(s\right)\sum _{i=1}^n{\psi }_{\omega }^2\left(t_i\right)}{n\sum _{i=1}^n{\psi }_{\omega }^{\prime }\left(t_i\right)},$$

where #(S) it the number of elements in the set S. In the forward selected or backward elimination, choose the one whose inclusion or deletion results in the smallest value of RFPE. To perform hypothesis testing, we let $\Omega \subseteq {\mathrm{R}}^{\mathrm{p}}$ be the parameter space and {θ_j1, θ_j2,...,θ_jq} be a subset of {θ₁,θ₂,...,θ_p}.

Define ${\Omega }_0=\{\theta \in \Omega :{\theta }_{j_1}={\theta }_{j_2}=\dots ={\theta }_{j_q}=0\}$, and the function f(θ) as

$$f\left(\theta \right)=\sum _{i=1}^n\frac{{\rho }_{\omega }\left(t_i\right)}{L_i}.$$

Then a robust likelihood ratio type test statistic for testing the null hypothesis $H_0:\theta \in {\Omega }_0$ against the alternative $H_1:\theta \in {\Omega }_0^c$ is

$$S_n^2=2(\underset{\theta \in {\Omega }_0}{Sup}f\left(\theta \right)-\underset{\theta \in \Omega }{Sup}f\left(\theta \right))∕q$$

For more information, the reader is referred to Hampel et al. [27]. Asymptotically under the null hypothesis $\frac{E\left[{\psi }_{\omega }^{\prime }\left(t\right)\right]}{E\left[{\psi }_{\omega }^2\left(t\right)\right]}{S}_n^2$ has a chi-square distribution with q degrees of freedom, where the Wald type test statistic is defined as $W_n^2=n({\widehat{\theta }}_{j_1},{\widehat{\theta }}_{j_2},\dots ,{\widehat{\theta }}_{j_q})V_q^{-1}{({\widehat{\theta }}_{j_1},{\widehat{\theta }}_{j_2},\dots ,{\widehat{\theta }}_{j_q})}^{\mathrm{t}}$, and $\frac{1}{n}{V}_q$ is the asymptotic variance-covariance matrix for the given vector $({\widehat{\theta }}_{j_1},{\widehat{\theta }}_{j_2},\dots ,{\widehat{\theta }}_{j_q})$. the null distribution of the statistic $W_n^2$ is asymptotically a chi-square distribution with q degrees of freedom.

Any of the following two robust algorithms can be used to estimate the parameter vector θ and standard deviation σ of a nonlinear regression model:

Algorithm I

1. Set j=0 and ${\widehat{\sigma }}^{\left(0\right)}=1$. Calculate the initial estimate ${\widehat{\theta }}^{\left(0\right)}$ of parameter vector θ by minimizing ${\sum }_{i=1}^n\frac{{\rho }_{\omega }(y_i-g(\theta ;x_i))}{L_i}$.

2. Set j=j+1 and calculate the followings:

   1. ${\widehat{\sigma }}^{\left(j\right)}$ using formula (3) or (4).
   2. Use ${\widehat{\theta }}^{(j-1)}$ to evaluate matrix ${\widehat{G}}^{\left(j\right)}$.
   3. Evaluate matrix $H^{\left(j\right)}=Diagonal[\left({\widehat{G}}^{\left(j\right)}{\left({\left({\widehat{G}}^{\left(j\right)}\right)}^t\right)}^{-1}{\widehat{G}}^{\left(j\right)}\right]$.
   4. The ith diagonal element of matrix H^(j) is denoted by $h_i^{\left(j\right)}$.

3. Calculate ${\widehat{\theta }}^{\left(j\right)}$ by minimizing $f\left(\theta \right)={\sum }_{i=1}^n\frac{{\rho }_{\omega }\left(t_i^{\left(j\right)}\right)}{L_i}$, where $t_i^{\left(j\right)}=\frac{(y_i-g(\theta ;x_i))(1-h_i^{\left(j\right)})}{{\widehat{\sigma }}^{\left(j\right)}}$,

4. If convergence occurs, stop. Otherwise go to step 2 and continue the process.

The convergence occurs when $\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}-{\widehat{\theta }}^{(\mathrm{j}-1)}\Vert \cong 0$. The convergence criterion is to stop the algorithm when $\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}-{\widehat{\theta }}^{(\mathrm{j}-1)}\Vert \le \epsilon or\frac{\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}-{\widehat{\theta }}^{(\mathrm{j}-1)}\Vert }{\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}\Vert }\le \epsilon$. One way choose the value of ε=0.00001.

Algorithm II

1. Set j=0 and ${\widehat{\sigma }}^{\left(0\right)}=1$. Calculate the initial estimate ${\widehat{\theta }}^{\left(0\right)}$ of prameter vector θ by minimizing ${\sum }_{i=1}^n\frac{{\rho }_{\omega }(y_i-g(\theta ;x_i))}{L_i}$.

2. Set j=j+1. For 1≤i≤n, calculate ${\widehat{\sigma }}^{\left(j\right)},t_i^{\left(j\right)}$ and weight $w_i^{\left(j\right)}$ where ${\omega }_i^{\left(j\right)}=\frac{\psi \left(t_i^{\left(j\right)}\right)(1-h_i^{\left(j\right)})}{{\widehat{\sigma }}^{\left(j\right)}(y_i-g({\widehat{\theta }}^{\left(j\right)};x_i))L_i}$ and $t_i^{\left(j\right)}=\frac{(y_i-g(\theta ;x_i))(1-h_i^{\left(j\right)})}{{\widehat{\sigma }}^{\left(j\right)}}$, use them to calculate the weights matrix $w^{\left(j\right)}$.

3. Use information from step 2 to calculate. ${\widehat{\theta }}^{\left(j\right)}={\left(G^{\left(j\right)}\right)}^t{W}^{\left(j\right)}{G}^{\left(j\right)-1}{\left(G^{\left(j\right)}\right)}^t{W}^{\left(j\right)}y$

4. If convergence occurs, stop. Otherwise go to step 2 and continue the process.

The convergence occurs when $\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}-{\widehat{\theta }}^{(\mathrm{j}-1)}\Vert \cong 0$. The convergence criterion is to stop the algorithm when $\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}-{\widehat{\theta }}^{(\mathrm{j}-1)}\Vert \le \epsilon or\frac{\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}-{\widehat{\theta }}^{(\mathrm{j}-1)}\Vert }{\Vert {\widehat{\theta }}^{\left(\mathrm{j}\right)}\Vert }\le \epsilon$. One may choose the value of ε=0.00001.

Drug concentration data

Kenakin [28] used a set of responses to the concentration of an agonist in a functional assay. They fit the following model to their data. In this data, observation 5 has an outlier in the response direction,

$$\mathrm{Response}=\mathrm{Basel}+\frac{Max-Basel}{1+{10}^{n\left(\log (E{C}_{50}-\log (A)\right)}}$$ (8)

Table 1 shows their actual and predicted concentrations as well as our results for fitted Hyperbolastic model of type III (H3). For this example, the new robust technique is an effective regression tool in estimating model parameters in the presence of outliers. Figure 1 shows the fitted curve using hyperbolastic model of type III (H3). Figure 2 uses formula (8) and the least squares fitted curve for the concentration data.

Table 1.

Parameter Estimates for the Concentration Data.

OBS	Concentration	Response	Least Square	Calculating using 1	Hyperbolastic H3
1	0.01	2	2	0.8	2
2	0.03	8	8.7	5.2	8
3	0.1	28	27.8	28	27.9
4	0.3	59	59.0	62.6	61.4
5	1	95	84.3	77.9	78.1
6	3	78	84.3	80.4	78.4
7	10	80	84.3	80.8	78.4

Figure 1.

Fitted curve using hyperbolastic model of type III (H3).

Figure 2.

The least squares fitted curve for the concentration data.

Tumor metastasis

The data in Table 2 consist of 12 observations. The response variable is the fraction of breast cancer patients with metastases and the predictor variable is the tumor size. Table 2 is from Michaleson et al. [29]. This data was originally collected by Tabar et al. [30-32] and Tubiana et al. [33,34]. To assess the robustness of our new method with regard to a special class of nonlinear growth models, we utilize this tumor metastasis data that is free of outliers. We first fit a model to the data using the robust method as well as least squares when there is no outlier present. Then we plant outliers in X direction, Y direction and both X and Y direction. In the X direction we change the X value in observation 12 from 90 to 2. In the Y direction, we change the y value in observation 6 from .55 to 3 and in both X and Y direction we change observation 12 in X direction from 90 to 2 and observation 7 in Y from .56 to 3.

Table 2.

Tumor Size Versus Fraction Metastasized Data.

Tumor size x	12	17	17	25	30	39	40	50	60	70	80	90
Fraction Metastasized y	0.13	0.20	0.27	0.45	0.42	0.55	0.56	0.66	0.78	0.83	0.81	0.92

For illustrative purposes, we have fitted hyperbolastic of type II, Gompertz and logistic models. In the past, these models have been used to monitor cancer progression and regression. Each model has three parameters θ₁, θ₂, and θ₃ with θ₁ and θ₂ being positive and ε_i are random errors. The response y_i is the fraction metastasized and x_i is the tumor size for individual i. The left graphs in Figures 3-5 are fitted curves using our proposed robust nonlinear regression technique and the graphs on the right sides of Figures 3-5 have been drawn using the nonlinear least square regression technique by planting outliers in X direction, Y direction and both X and Y direction. As you can see, when there is no outlier in the data all models perform well regardless of using the robust method or Least Squares. But when we plant outliers in the X, Y, and/or XY directions the fits become unacceptable for Least Squares whereas the robust method performs well for all models.

Figure 3.

Hyperbolastic model of type II.

Figure 5.

The logistic model.

The hyperbolastic model of type II or simply H2 has the form $y_i=\frac{{\theta }_1}{1+{\theta }_{2}Arc\sinh \left[\exp [-{\theta }_3{x}_i]\right]}+{\in }_i$.

The Gompertz model is of the form: y_i = θ₁ exp[–θ₂ exp(–θ₃x_i)]+∈_i.

The logistic model is of the form $y_i=\frac{{\theta }_1}{1+{\theta }_2\exp [-{\theta }_3{x}_i]}+{\in }_i$.

Simulation

We perform simulation experiments to evaluate the robustness of our new nonlinear regression method compared to the least squares method. We have simulated the biochemistry model known as Michaelis-Menten kinetics. In biochemistry this model expresses the reaction velocity V as a function of concentration of substrate C as

$$V=\frac{\alpha C}{\beta +C},$$

where the parameter α denotes the maximum reaction velocity and β is the substrate concentration at which the initial velocity V₀ is 50% of the maximum reaction velocity. The larger the parameter β, the lower is the efficiency between the substrate and enzyme. This model has also been used in many biological systems such as gene regulatory system. In order to investigate the robustness of our new method relative to the method of least squares, we considered the nonlinear Michaelis-Menten equation of the form

$$y_i=\frac{{\theta }_1{x}_i}{{\theta }_2+x_i}+{\epsilon }_i,i=1,2,\dots ,n$$

Where the response variable is y_i and x_i is fixed. In our simulations we set x_i=i and ε_i as the standard normal distribution with mean 0 and standard deviation 1. We performed 1000 repetitions using two sample sizes n=20 and n=50. The outliers were randomly chosen in the direction of X, Y and both X and Y. We used contamination levels of 0%, 10%, 20%, 30%, and 40%. In this simulation the parameter values are θ₁=5 and θ₂ =1. The software Mathematica is used in the simulation process. To evaluate the robustness of these estimators, we randomly choose 10%, 20%, 30% and 40% of the simulated observations and contaminate the selected data by magnifying their size by a factor of 100 in the direction of explanatory variable X, response variable Y, and both response Y and explanatory X variables. Finally, we estimate both bias and mean squared errors using the following equations

$$bias=\mid \frac{\sum _{l=1}^m{\widehat{\theta }}_l}{m}-\theta \mid$$

Where m is the number of iterations in the simulation. The mean squared error is estimated by

$$MSE=\frac{\sum _{l=1}^m{({\widehat{\theta }}_l-\theta )}^2}{m}.$$

Tables 3-5 give the summary of our simulation outcome for both small and large sample sizes. The asymptotic efficiency for our simulation studies has been set to 95% level. By examining the simulation tables, we find out that in the absence of contamination in the simulated data, both the least square and the proposed robust method perform well with respect to bias, mean square error and mean estimated parameter values. However, when contamination enters into our simulated data in the direction of explanatory variable X, or response variable Y, or both X and Y, then the new method outperforms the least squares method for both small and large samples. We also observe that the estimated values of parameters θ₁ and θ₂ are in close proximity of the true values of the parameters θ₁ and θ₂. The simulation results clearly indicate the robustness of our new nonlinear regression technique relative to least squares method when outliers or influential observations are present.

Table 3.

Bias, Mean Square Errors (MSE) and Mean Estimated Parameter (MEP) with Percentage Contamination in the X Direction.

n=20		0%		10%		20%		30%		40%
		θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2
Least-Squares	Bias	0.0375	0.1083	25343	47179	8583	6953	2407	5185	1604	4015
	MSE	0.0479	0.4842	5.5E+9	1.5E+10	4.6E+9	2.1E+9	3.2E+8	2.2E+9	1.0E+8	1.4E+9
	MEP	5.0375	1.1083	25348	45180	9324	7329	3456	7733	1934	5234
Robust Method	Bias	0.0901	0.1525	0.0056	0.2242	0.034	0.2421	0.0199	0.2265	0.0176	0.3269
	MSE	0.0534	0.5646	0.2111	0.9423	0.2502	1.2292	0.2578	0.8812	0.2767	1.6324
	MEP	5.0701	1.1327	5.0056	1.2201	5.0134	1.2440	5.0199	1.2263	4.9824	1.3269

n=50		0%		10%		20%		30%		40%
		θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2
Least-Squares	Bias	0.0131	0.1430	5214	4108	7326	5423	2113	4927	1476	3104
	MSE	0.0396	0.2566	6.1E+9	1.7E+9	3.2E+9	1.8E+9	2.5E+8	1.8E+9	7.9E+7	1.2E+9
	MEP	5.0131	1.0414	30123	58972	8589	6954	2412	6186	1609	4016
Robust Method	Bias	0.0701	0.1625	0.0328	0.0914	0.02925	0.1285	0.0494	0.1843	0.0050	0.2876
	MSE	0.2167	0.5221	0.2333	0.6257	0.2718	0.7224	0.3150	0.8543	0.2937	0.9745
	MEP	5.0701	1.1131	5.0328	1.0914	5.0292	1.1285	5.0494	1.1843	4.9734	1.1567

Table 5.

Bias, Mean Square Errors (MSE) and Mean Estimated Parameter (MEP) with Percentage Contamination in the X-Y-Direction.

n=20		0%		10%		20%		30%		40%
		θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2
Least-Squares	Bias	0.0375	0.1083	25342.0	45178	8582	6952	2406	5184	1603	4015
	MSE	0.0479	0.4842	5.5E+9	1.5E+10	4.6E+9	2.1E+9	3.2E+8	2.2E+9	1.0E+8	1.4E+9
	MEP	5.0375	1.1083	25347	45179	8587	6953	2411	5185	1608	4016
Robust Method	Bias	0.0901	0.1525	0.0762	0.0895	0.0081	0.0599	0.1191	0.1685	0.0176	0.0417
	MSE	0.0534	0.5646	0.1676	0.3112	0.1484	0.6847	0.2901	0.7813	0.2539	0.7448
	MEP	5.0701	1.1327	5.0762	1.0895	4.9919	1.0599	5.1191	1.1685	5.0176	0.0417

n=50		0%		10%		20%		30%		40%
		θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2
Least-Squares	Bias	0.0131	0.1430	660	892	583	582	566	478	559	404
	MSE	0.0396	0.2566	455959	1.0E+6	345138	437874	323227	252951	314270	176481
	MEP	5.0131	1.0414	665	893	588	583	571	479	564	405
Robust Method	Bias	0.0701	0.1625	0.0173	0.0736	0.0892	0.1767	0.0399	0.0059	0.01445	0.0256
	MSE	0.2167	0.5221	0.0526	0.3141	0.0746	0.4530	0.0474	0.2435	0.0857	0.4877
	MEP	5.0701	1.1131	4.9827	1.0736	5.0892	1.1767	4.9601	1.0060	4.9856	1.0256

Conclusion

In this paper we introduced a new robust estimator of nonlinear regression parameters. In addition, robust testing for hypothesis about model parameters was introduced. Moreover, two algorithms were developed to perform the robust nonlinear estimation of model parameters. The computer simulation revealed the robustness of our new estimator. This robust method provides a powerful alternative to least squares method. The robust method presented in this paper has influence functions bounded in both the response and the explanatory variable direction. It has high asymptotic breakdown point and efficiency. A Mathematica program is also provided to ease in computations. This program does the necessary calculations to perform the robust nonlinear regression analysis of the drug concentration example given in this paper.

Figure 4.

The Gompertz mode.

Table 4.

Bias, Mean Square Errors (MSE) and Mean Estimated Parameter (MEP) with Percentage Contamination in the Y Direction.

n=20		0%		10%		20%		30%		40%
		θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2
Least-Squares	Bias	0.0375	0.1083	68395	12511	115870	12323	115212	9858	136703	8915
	MSE	0.0479	0.4842	1.50E+10	1.50E+8	1.2E+11	1.2E+9	1.4E+11	9.9E+8	2.4E+11	1.0E+9
	MEP	5.0375	1.1083	68400	12512	115875	12324	115217	9859	136708	8916
Robust Method	Bias	0.0901	0.1525	0.0358	0.0523	0.0362	0.0250	0.0899	0.1605	0.2034	0.3292
	MSE	0.0534	0.5646	0.2296	0.5710	0.2555	0.6212	0.3534	0.8233	0.3928	1.095
	MEP	5.0701	1.1327	4.9641	1.0523	4.9638	0.9751	5.0899	1.1601	5.2034	1.3292

n=50		0%		10%		20%		30%		40%
		θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2	θ 1	θ 2
Least-Squares	Bias	0.0131	0.1430	23650	11692	18843	5605	25855	5462	33790	5170
	MSE	0.0396	0.2566	4.6E+9	1.1E+9	7.6E+9	6.3E+9	1.2E+10	5.1E+08	3.1E+10	7.1E+08
	MEP	5.0131	1.0414	23655	11693	18848	5606	25860	5463	33795	5171
Robust Method	Bias	0.0701	0.1625	0.0365	0.0646	0.0262	0.0143	0.0504	0.0640	0.0022	0.0510
	MSE	0.2167	0.5221	0.0555	0.2067	0.0660	0.3303	0.0673	0.3861	0.0710	0.6139
	MEP	5.0701	1.1131	5.0365	1.0646	4.974	0.9857	5.0505	1.0650	5.0022	1.0510