A new outlier detection method based on convex optimization: application to diagnosis of Parkinson’s disease

ABSTRACT

Neuroscience is a combination of different scientific disciplines which investigate the nervous system for understanding of the biological basis. Recently, applications to the diagnosis of neurodegenerative diseases like Parkinson’s disease have become very promising by considering different statistical regression models. However, well-known statistical regression models may give misleading results for the diagnosis of the neurodegenerative diseases when experimental data contain outlier observations that lie an abnormal distance from the other observation. The main achievements of this study consist of a novel mathematics-supported approach beside statistical regression models to identify and treat the outlier observations without direct elimination for a great and emerging challenge in humankind, such as neurodegenerative diseases. By this approach, a new method named as CM_TMSOM is proposed with the contributions of the powerful convex and continuous optimization techniques referred to as conic quadratic programing. This method, based on the mean-shift outlier regression model, is developed by combining robustness of M-estimation and stability of Tikhonov regularization. We apply our method and other parametric models on Parkinson telemonitoring dataset which is a real-world dataset in Neuroscience. Then, we compare these methods by using well-known method-free performance measures. The results indicate that the CM_TMSOM method performs better than current parametric models.

1. Introduction

Neuroscience [18] consists of a combination of different disciplines that work together cooperatively, for understanding of the structure and function of the normal and abnormal brain. It has a wide range of research, from the molecular biology of nerve cells to the biological basis of normal and disordered behavior, emotion, and cognition, and today, it is one of the most rapidly growing areas of science. Neurodegenerative diseases, such as Parkinson, Alzheimer and epilepsy, are research areas of Neuroscience, affecting deeply the lives of patients and their families. Nowadays, optimization and statistical learning techniques are used to solve problems in many fields of research area [14,32]. Therefore, in our study, we will try to contribute to Neuroscience with statistical regression and convex optimization methods by considering dataset related to Parkinson’s disease (PD), which is the second most common neurodegenerative disorder after Alzheimer’s and affects millions of people around the world adversely. All studies to date suggest that age is the single most important risk factor for the origin of this disease which increases steeply after the age of 50 [8]. However, medication and surgical intervention can hold back the progression of the disease and alleviate some of the symptoms, for which there is no available cure [40,41]. Thus, early diagnosis is very important in order to improve the patient’s quality of life and to prolong it [21].

Overall, to track the progress of symptoms associated with this disease, the unified Parkinson’s disease rating scale (UPDRS) is used for monitoring by trained medical staff. Tsanas et al. [47] dealt with remote replication of UPDRS assessment by using only simple, self-administered, and noninvasive speech tests, since symptom monitoring is costly and logistically inconvenient for patient and clinical staff. They characterized speech with signal processing algorithms, extracting clinically useful features of average Parkinson’s disease progression, suggesting that their assessments are clinically more useful in terms of accuracy than the clinicians’ estimates. Nowadays, since Parkinson’s disease remains in public discussion, even regarded as of growing importance, there is an increased interest in studies on PD that apply various statistical and data mining methods to evaluate data related to PD. Tsanas et al. [47] used linear and nonlinear regression techniques for mapping the selected subset of features to UPDRS, employing least-squares estimation, and nonparametric classification and regression trees. Ene [9] made an application study on a PD dataset by considering some probabilistic neural network variants for classifying between healthy people and those with PD. Sakar et al. [10] developed a generalization of the predictive model for telemonitoring by using well-known machine learning tools based on the PD dataset. Furthermore, several clustering studies were conducted based on datasets including PD and normal human voice. Bhattacharjee and Mukherjee [6] considered a multi-layer perceptron by using best ranked voice feature and associative rule generation for clustering and classification of PD. In their study, they investigated several clustering and classification methods like k-means and filtered clustering.

As it is understood from the above studies, regression and classification models are of great importance in Neuroscience by means of PD datasets. These models are constructed based on the Least-Squares (LS) or Maximum-Likelihood (ML) estimation of model parameters under some specific assumption, such as normality of the error distribution [27,33,34]. However, statistical analysis will not yield useful and accurate results if the dataset covers one or more outlier observations. Therefore, it is important to identify these observations and bound their influence using methods that are less sensitive against outliers, while the most appropriate model fit represents properties of the data, i.e. robust methods [16,17], or removal outliers from the dataset. In our study, we use an alternative for robust methods by providing stability for solution and selecting model features which are necessary to fit the PD data. Therefore, we will combine robustness of M-estimation, which was introduced by Huber [16], with the stability of Tikhonov regularization [46]. Tikhonov regularization addresses a particular form of a penalized least-squares criterion:

$$\begin{array}{r}\underset{\mathit{\beta }}{\text{minimize}}\sum _{i=1}^n{(y_i-{\mathit{x}}_{\mathit{i}}^{\mathit{T}}\mathit{\beta })}^2+\lambda \sum _{j=1}^p{|{\beta }_j|}^r,\end{array}$$ (1)

that is called Bridge estimators [12]. Therefore, we firstly construct a Tikhonov regularization problem for mean-shift outlier model (MSOM) based on the M-estimation with Huber-type function having the following objective function:

$$\begin{array}{r}\text{minimize}\sum _{i=1}^n\rho (r_i),\end{array}$$ (2)

where $r_i=y_i\text{ - }{\mathit{x}}_{\mathit{i}}^T\mathit{\beta }$ is the ith component of the residual vector and $\rho$is a Huber-type function [16,17]. Then, we call the solution for this problem as M_TMSOM. Secondly, we treat our problem by continuous optimization, considered to become a complementary technology and alternative solution technique, which is called $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ for this model. A problem that is closely related to quadratic programing is the second-order cone program (SOCP). SOCP was used in some of our works [42,45,44,52]. In these studies, we saw that SOCP has better performance or is very much competitive with respect to the other methods.

This paper is organized as follows: in Section 2, we give information about Parkinson’s disease dataset, and we review linear regression and some of direct outlier identification methods for it. In Section 3, we consider mean-shift outlier model and construct the Tikhonov regularization problem based on the M-estimation for it. Then, in Section 4, we propose to solve generated estimation problem by one of the continuous optimization methods. Finally, in Section 5, we give an application study to construct a numerical comparison on the performance of linear model (LM), MSOM and our new method, with respect to our PD telemonitoring dataset.

2. Materials and methods for outlier identification

2.1. Data description

The dataset underlying our study was taken from Machine Learning Repository (UCI) database; it was created in 2009 by Athanasios Tsanas of the University of Oxford, in collaboration with 10 medical centers in the US and Intel Corporation, which developed the telemonitoring device to record speech signals. The data were collected by Tsanas et al. [47] by using Intel AHTD, which is a telemonitoring system designed to facilitate remote, internet-enabled measurement of a variety of PD-related motor impairment symptoms. This dataset has 16 features and 5875 observations, and it is composed of a wide range of biomedical voice measurements from 42 people with early-stage Parkinson’s disease, recruited to a six-month trial of a telemonitoring device for remote symptom progression monitoring. The recordings were automatically captured in the patients’ homes. The attributes involved in the dataset are as follows: age, gender (0: male, 1: female), time interval from baseline recruitment date, clinician's motor UPDRS score (motor UPDRS), clinician’s total UPDRS score (total UPDRS), and 16 biomedical voice measures. These can be listed as follows: several measures of variation in fundamental frequency, such as Jitter (%), Jitter (Absolute), Jitter: RAP, Jitter: P, Jitter: DDP, also some measures of variation in amplitude which are Shimmer, Shimmer(dB), Shimmer: A, Shimmer: A, Shimmer: A, Shimmer: DDA, two measures of ratio of noise to tonal components in the voice which are NHR and HNR; and furthermore, a nonlinear dynamical complexity measure named RPDE, also signal fractal scaling exponent which is DFA, and finally a nonlinear measure of fundamental frequency variation named PPE.

2.2. Methods for outlier identification for PD dataset

Our main task is to construct a predictive model that has a maximally accurate result for UPDRS by using the $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ method. There are several methods to obtain predictive models by using features of UPDRS. Linear regression model is one of them and the most famous one. The standard linear model, LM [33,34], with n observations or output variable for UPDRS and $p$independent or input variables for PD dataset, is given by

$$\begin{array}{r}X=W\beta +\epsilon ,\end{array}$$ (3)

where $X$ is an $n\times 1$-data vector for response variable, $W$ is a full-rank $n\times p$-data matrix for independent variables with $p\times 1$-column vectors, $W\beta$ is a $p\times 1$-vector of unknown parameters, and $\epsilon$ is an $n\times 1$-vector of independent, identically distributed $N(0,{\sigma }^2)$ random errors. Here, the purpose is to estimate the $p\times 1$-vector of unknown parameters $\beta$ based on the given large number of input and output values that minimize the error to determine a most accurate predictive model for UPDRS over the entire dataset. However, it would not be possible to obtain a predictive model that provides maximally accurate predictions for UPDRS, if the dataset contains one or more than one outlier observations. Therefore, we should define outlier observations for UPDRS and bound their influence through methods which are less sensitive against outliers or remove them from the dataset.

For LM, there are different approaches for outlier detection [2,31]. These approaches are partitioned into two categories: Direct approaches and Indirect approaches, involving residuals from the robust fit. Hadi and Simonoff [13] presented a method for direct approaches. For these approaches, they partitioned the data into a set of ‘clean’ data points with size $h\text{ = }\text{integer }\text{(}n\text{ + }L\text{ - 1)}/\text{2}$ and a set of points that contain the potential outliers, where L is the number of observations that are most likely candidate outliers. The potential outliers are then tested to see how extreme they are relative to the clean subset, using an appropriate diagnostic measure like the internally residual [38] or Cook distance [7]. According to the study Ruppert [39], the success of the procedure is based on the initial clean subset of data. The procedure works well for low-leverage outliers, but it may fail when the sample contains a set of several high-leverage outliers.

An indirect approach to outlier identification works through a robust regression estimate. The aim of robust regression is to provide resistant (stable) results in the presence of outliers. In order to achieve this stability, robust regression limits the influence of outliers. The most widely used robust methods for multivariate data analysis are M-estimates (ML type estimators) introduced by Huber [16,17], LMS-estimates (Least Median of Squares) [36], LTS estimates (Least Trimmed Squares) [37] and S-estimates [7,39], which are a generalized form of the LMS and LTS estimates. The common feature of all of these methods is to model fit without deleting outlying observations from the dataset. Mean-shift outlier [20] is one of these robust methods for which we proposed a new solution method based on Tikhonov regularization by CQP [45]. For the current study, we aim to obtain Huber’s M-estimator depending on Tikhonov regularization by solving CQP problem.

2.2.1. Mean-shift outlier model (MSOM)

The MSOM is a very important outlier-detection method, and it can be considered for the detection of outlier observation in PD dataset since it provides the same residual sum of squares as the model fitted after omitting the relevant observations. Therefore, that approach is very convenient for studying the regression model in the presence of outliers in the datasets, related not only to the PD, but also to all scientific and technological fields.

The MSOM is given by

$$\begin{array}{r}\mathit{y}=\mathit{X}\mathit{\beta }+{\mathit{e}}_i\delta \text{ + }\mathit{\epsilon }\mathbf{,}\end{array}$$ (4)

where ${\mathit{e}}_i$ is the ith unit vector, i.e. ${\mathit{e}}_i^T=(0,\dots ,1,0,\dots ,0)(i=1,2,\dots ,n)$with 1 at the ith position. In this model, it is assumed that really on exclusive $y_i$ or ${\mathit{x}}_i\mathit{\beta }$ deviates systematically by some value $\delta$ from the model $y_i={\mathit{x}}_i\mathit{\beta }\text{ + }{\epsilon }_i.$ Then, the ith observation $(y_i,{\mathit{x}}_i\mathit{\beta })$ would contain a different bias term from the rest of the observations, and $(y_i,{\mathit{x}}_i\mathit{\beta })$ would hence be considered an outlier. For checking this fact, the hypothesis

$$\begin{array}{r}{H}_0:\delta =0(\text{i}\text{.e}\text{.,}E(y)\mathbf{=}\mathit{X}\mathit{\beta })\end{array}$$ (5)

is tested against the alternative hypothesis

$$\begin{array}{r}{H}_1:\delta \ne 0(\text{i}\text{.e}\text{.,}E(y)\mathbf{=}\mathit{X}\mathit{\beta }\text{ + }{\mathit{e}}_i\delta ),\end{array}$$ (6)

using the likelihood-ratio test statistic:

$$\begin{array}{r}{F}_i=\frac{[SSE(H_{\text{0}})\text{ - }SSE(H_{\text{1}})]/1}{SSE(H_{\text{1}})/(n-p-1)},\end{array}$$ (7)

where $SSE(H_{\text{0}})$ is the residual sum of squares in the model $\mathit{y}=\mathit{X}\mathit{\beta }+\mathit{\epsilon },$ containing all the n observations, and $SSE(H_{\text{1}})$ is the residual sum of squares in the model $\mathit{y}=\mathit{X}\mathit{\beta }+{\mathit{e}}_i\delta +\mathit{\epsilon }$ [33].

3. Parameter estimation for MSOM

3.1. MSOM revisited by M-estimation

In this section, we will use the M-estimation method to assess the parameters in the MSOM for PD dataset. Let us assume that m observations $y_i$, where m < n, deviate systematically from the corresponding model $y_i={\mathit{x}}_i\mathit{\beta }\text{ + }{\epsilon }_i$ by ${\delta }_i$ using one of the direct methods as in Hadi and Simonoff [13], test statistic $F_i$ given by Equation (7), Cook’s distance [7] or Studentized residuals [4]; that is, $m$ observations are outliers. Then, MSOM can be written as

$$\begin{array}{r}\mathit{y}=\mathit{X}\mathit{\beta }+\mathit{E}\mathit{\delta }\text{ + }\mathit{\epsilon }\mathbf{,}\end{array}$$ (8)

where $\mathit{X}$ is a full rank $n\times p$-matrix of independent variables, $\mathit{E}$ is an $n\times m$-matrix with $m$ indicator variables, and $\mathit{\delta }$ is an $m\times 1$ vector of the regression coefficients (intercepts) of the indicator variables. Then, MSOM can be stated as

$$\begin{array}{r}\mathit{y}={\mathit{X}}^{\ast }{\mathit{\beta }}^{\ast }\text{ + }\mathit{\epsilon }\mathbf{,}\end{array}$$ (9)

where ${\mathit{X}}^{\ast }=\left(\begin{array}{cc}\mathit{X}|& \mathit{E}\end{array}\right)$ is an $n\times (p+m)$-block matrix constructed by matrices $\mathit{X}$ and $\mathit{E},$ and ${\mathit{\beta }}^{\ast }={\left(\begin{array}{cc}{\mathit{\beta }}^T|& {\mathit{\delta }}^T\end{array}\right)}^T$ is a $(p+m)\times 1$-vector constructed by vectors $\mathit{\beta }$ and $\mathit{\delta }$. The M-estimator for MSOM is obtained by solving the minimization problem

$$\begin{array}{rl}\underset{{\mathit{\beta }}^{\mathbf{\ast }}}{\text{minimize}}& G({\mathit{r}}^{\ast }),\end{array}$$ (10)

$$\begin{array}{rl}\text{where}G({\mathit{r}}^{\ast })& =\sum _{i=1}^n\rho (r_i^{\ast }).\end{array}$$ (11)

In Equation (11), $r_i^{\ast }$ is the ith residual given as $r_i^{\ast }:=y_i-{\mathit{x}}_i\mathit{\beta }-{\delta }_i$, and $\rho$ is a nonconstant convex function that gives the contribution of each residual to the objective function $G({\mathit{r}}^{\ast })$ and has the following properties:

$$\begin{array}{r}(i)\rho (r_i^{\ast })\ge 0,(\mathrm{ii})\rho (0)=0,(\mathrm{iii})\rho (r_i^{\ast })=\rho (-r_i^{\ast }),(\mathrm{iv})\rho (r_i^{\ast })\ge \rho (r_j^{\ast })\text{for }|r_i^{\ast }|\ge |r_j^{\ast }|.\end{array}$$ (i)

To determine M-estimation for MSOM, several nonconstant convex functions can be considered Tukey’s Bisquare $\rho$ function [3] or the following Huber-type function [16, 17], which will be used in this study:

$$\begin{array}{r}\rho (r_i^{\ast })=\{\begin{array}{l}{\displaystyle \frac{r{{}_i^{\ast }}^2}{2}},,|r_i^{\ast }|\le k\\ k|r_i^{\ast }|-{\displaystyle \frac{k^2}{2}},|r_i^{\ast }|>k\end{array},\end{array}$$ (12)

where $k(k>0)$ is a tuning constant and its value is selected to achieve high efficiency. For Huber-type functions, the value $k\text{ = 1}\text{.345}\sigma$ producing, efficiency at the 95% level, provides a protection against outliers in the case of errors are normally distributed. Here, $\sigma$ is the standard deviation of errors. Preferably, a strong measure of spread is used for the standard deviation of residuals. For this measure, generally, the value

$$\begin{array}{r}\sigma =\frac{\text{Med}|{\mathit{r}}^{\ast }|}{\text{0}\text{.6745}}\end{array}$$ (13)

is chosen, where Med:= middle value [11].

The basic algorithm for computing M-estimator for regression is iteratively least-squares (IRLS) [15]. Weighted least-squares (WLS) fit is carried out inside an iteration loop. For each iteration, a set of weights for the observations are used in the least-squares fit. The weights are constructed by applying a weight function to the current residuals. Initial weights are based on residuals from an initial fit. To solve the problem in Equation (10), two cases are considered:

• (i) Let $|r_i^{\ast }|\le k$. Then, the problem

  $$\begin{array}{r}\underset{{\mathit{\beta }}^{\mathbf{\ast }}}{\text{minimize}}\frac{1}{2}||\mathit{y}-{{\mathit{X}}^{\ast }}^{\mathrm{T}}{\mathit{\beta }}^{\ast }|{|}_2^2\end{array}$$ (14)

  is obtained, where $||\cdot |{|}_2$ is the Euclidean norm. This problem is solved by the least-squares estimation method [33]. When $|r_i^{\ast }|\le k$, the M-estimator of the parameter vector ${\mathit{\beta }}^{\ast }$also is the LS estimator, and this estimator is the ${\hat{\mathit{\beta }}}_{\mathrm{M}}^{\ast }$ vector which is the solution of n equations, as shown below depending on the function $\rho (r_{{}_i}^{\ast })$:

  $$\begin{array}{r}\sum _{i\text{ = 1}}^n\psi (r_i^{\ast })\frac{\mathrm{\partial }{r}_i^{\ast }}{\mathrm{\partial }{\mathit{\beta }}_{{}^i}^{\ast }}=\text{0}.\end{array}$$ (15)

  Then, the least-squares estimate of ${\mathit{\beta }}^{\ast }$ is chosen as the value ${\hat{\mathit{\beta }}}^{\ast }$ which minimizes the residual sum of squares (RSS):

  $$\begin{array}{r}\text{RSS: = }\sum _{i=1}^n{(y_i-\mathit{x}{{}_i^{\ast }}^{\mathrm{T}}{\hat{\mathit{\beta }}}^{\ast })}^2=(\mathit{y}-{\mathit{X}}^{\ast }{\hat{\mathit{\beta }}}^{\ast }{)}^T(\mathit{y}-{\mathit{X}}^{\ast }{\hat{\mathit{\beta }}}^{\ast })=||\mathit{y}-{\mathit{X}}^{\ast }{\hat{\mathit{\beta }}}^{\ast }|{|}_2^2.\end{array}$$ (16)

  This optimal vector of value for ${\hat{\mathit{\beta }}}^{\ast }$and the estimate of $\mathit{y}$ are obtained as follows:

  $$\begin{array}{rl}& {\hat{\mathit{\beta }}}^{\ast }=({{\mathit{X}}^{\ast }}^T{\mathit{X}}^{\ast }{)}^{-1}{{\mathit{X}}^{\ast }}^T\mathit{y},\\ & \hat{\mathit{y}}\mathbf{=}{\mathit{X}}^{\ast }{\hat{\mathit{\beta }}}^{\ast }\mathbf{=}{\mathit{X}}^{\ast }({{\mathit{X}}^{\ast }}^T{\mathit{X}}^{\ast }{)}^{-1}{{\mathit{X}}^{\ast }}^T\mathit{y}.\end{array}$$ (17)
• (ii) Let $|r_i^{\ast }|>k$. Then, the problem in Equation (10) for Huber-type M-estimation turns into the problem

  $$\begin{array}{r}\text{minimize}\sum _{i=1}^n\left(k|y_i-x_i^{{}_{\ast }T}{\mathit{\beta }}^{\ast }|-\frac{1}{2}{k}^2\right).\end{array}$$ (18)

This problem can be solved by the IRLS method. Hence, we get $\psi (r_i^{\ast })=k\cdot \text{sgn}(r_i^{\ast })$ and $w(r_i^{\ast })=k/|r_i^{\ast }|$. This problem can be written as ${{\mathit{X}}^{\mathbf{\ast }}}^{{}^T}\mathit{W}{\mathit{X}}^{\mathbf{\ast }}{\mathit{\beta }}^{\ast }={{\mathit{X}}^{\mathbf{\ast }}}^{{}^T}\mathit{W}y$, where $\mathit{W}$is the weight matrix with diagonal elements $w(r_i^{\ast })$. Hence, we find the estimate of ${\mathit{\beta }}^{\ast }$ as

$${\hat{\mathit{\beta }}}^{\ast }=({{\mathit{X}}^{\mathbf{\ast }}}^{{}^T}\mathit{W}{\mathit{X}}^{\mathbf{\ast }}{)}^{-1}{{\mathit{X}}^{\mathbf{\ast }}}^{{}^T}\mathit{W}\mathit{y}\mathbf{.}$$

3.2. Tikhonov regularization problem for M-estimator of MSOM (M_TMSOM)

The purpose of robust regression is to produce solutions not affected by outliers. M-estimator is one of the most important robust methods, as mentioned in Subsection 3.1. However, one may encounter an ill-posed problem (irregular or unstable) while trying to find the M-estimators for MSOM. Therefore, Tikhonov regularization problem [1,3] is considered to make an ill-posed problem well-posed, and get a more powerful solution against outliers, while searching M-estimators of the parameters in the MSOM regression model based on the related dataset. Furthermore, we should note that a Tikhonov regularization problem considers model features which are necessary to fit the data, whereas model features, which are unnecessary to fit the data, will conversely be removed by the regularization. This feature of Tikhonov regularization allows us to obtain easily interpretable models.

To obtain M-estimator for MSOM by using Huber-type function, one of the options that can be chosen to construct the Tikhonov regularization for the case $|r_i^{\ast }|\le k,$ is the damped least-squares problem given subsequently:

$$\begin{array}{r}\text{minimize}\frac{1}{2}||\mathit{y}-{\mathit{X}}^{\ast }{\mathit{\beta }}^{\ast }|{|}_2^2+{\lambda }^2||{\mathit{\beta }}^{\ast }|{|}_2^2,\end{array}$$ (20)

where ${\lambda }^2$ is considered to be a penalty parameter that establishes a trade-off between both accuracy, i.e. a small sum of error squares, and not too high a complexity. There are different criteria to obtain the value $\lambda$ just if possible: (a) The discrepancy principle [24] between $||{\mathit{\beta }}^{\ast }|{|}_2$ and $||\mathit{y}-{\mathit{X}}^{\ast }{\mathit{\beta }}^{\ast }|{|}_2$, (b) the plotting of $||{\mathit{\beta }}^{\ast }|{|}_2$versus $||\mathit{y}-{\mathit{X}}^{\ast }{\mathit{\beta }}^{\ast }|{|}_2$ on a log–log scale where the curve of optimal values of $||{\mathit{\beta }}^{\ast }|{|}_2$ and $||\mathit{y}-{\mathit{X}}^{\ast }{\mathit{\beta }}^{\ast }|{|}_2$ often takes on a pronounced L-shape [1], and (c) L-curve criterion, in which the value of $\lambda$ that gives the solution closest to the corner of the L-curve, is selected.

As mentioned previously, the purpose of robust regression is to provide results limiting the impact of outliers or resistant (stable) to outliers. Therefore, we shall consider Tikhonov regularization problem to obtain M-estimation for MSOM; then we will solve them by CQP. Tikhonov regularization problem is generated in both cases when $|r_i^{\ast }|>k.$

In the case $r_i^{\ast }\ge 0$: we may obtain $\rho (r_i^{\ast })$ as

$$\begin{array}{r}\rho (r_i^{\ast })={\sqrt{w_i}}^2(y_i^{\ast }-{\mathit{x}}_i^{{\ast }^T}{\mathit{\beta }}_i^{\ast }{)}^2.\end{array}$$ (21)

In the case $r_i^{\ast }<0$: we may obtain $\rho (r_i^{\ast })$ as

$$\begin{array}{r}\rho (r_i^{\ast })=\frac{-k}{y_i^{\ast }-{\mathit{x}}_i^{{\ast }^T}{\mathit{\beta }}_i^{\ast }}(y_i^{\ast }-{\mathit{x}}_i^{{\ast }^T}{\mathit{\beta }}_i^{\ast }{)}^2.\end{array}$$ (22)

If $y_i^{\ast }-x_i^{{\ast }^T}{\beta }_i^{\ast }$ < 0, then Equation (22) appears as follows:

$$\begin{array}{r}\rho (r_i^{\ast })={\sqrt{w_i}}^2(y_i^{\ast }-{\mathit{x}}_i^{{\ast }^T}{\mathit{\beta }}_i^{\ast }{)}^2,\end{array}$$ (23)

where $y_i^{\ast }$ and $w_i$ are given as

$$\begin{array}{r}{y}_i^{\ast }:=\{\begin{array}{l}{y}_i\text{ - }{\displaystyle \frac{k}{2}},r_i^{\ast }\ge 0\\ y_i+{\displaystyle \frac{k}{2}},r_i^{\ast }<0\end{array},w_i:=\{\begin{array}{l}{\displaystyle \frac{k}{y_i^{\ast }-{\mathit{x}}_i^{{\ast }^T}{\mathit{\beta }}_i^{\ast }}},r_i^{\ast }\ge 0\\ {\displaystyle \frac{-k}{y_i^{\ast }-{\mathit{x}}_i^{{\ast }^T}{\mathit{\beta }}_i^{\ast }}},r_i^{\ast }<0\end{array},\end{array}$$ (24)

respectively. Let ${\mathit{W}}^{\mathbf{\ast }}:=\left(diag\sqrt{w_i}\right)$ be a matrix with ith diagonal elements $\sqrt{w_i}$; then we obtain objective function $G({\mathit{r}}_{{}^i}^{\ast })$ as follows:

$$\begin{array}{r}G({\mathit{r}}_{{}^i}^{\ast })=||{\mathit{y}}^{\ast \ast }-\mathit{A}{\mathit{\beta }}^{\ast }|{|}^2,\end{array}$$ (25)

where ${\mathit{y}}^{\ast }$ is an $n\times 1$-vector with ith element connected to $r_i^{\ast }$, expressed as Equation (24), ${\mathit{y}}^{\ast \ast }={\mathit{W}}^{\ast }{\mathit{y}}^{\ast }$ and $\mathit{A}={\mathit{W}}^{\ast }{\mathit{X}}^{\ast }$an $n\times 1$-vector and an $n\times (p+m)$-matrix, respectively. Then, a Tikhonov regularization problem to get an M-estimator for MSOM can be set as

$$\begin{array}{r}\text{minimize}||{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}-\mathit{A}{\mathit{\beta }}^{\ast }|{|}_2^2+{\kappa }^2||{\mathit{\beta }}^{\ast }|{|}_2^2.\end{array}$$ (26)

We regard ${\kappa }^2$ as a penalty parameter mentioned above.

4. On CQP and its application to M-estimation for MSOM ( $\mathbf{C}{\mathbf{M}}_{\mathbf{T}}\mathbf{M}\mathbf{S}\mathbf{O}\mathbf{M}$)

4.1. Conic quadratic programing

Convex programing deals with problems that consist of minimizing a convex function over a convex set. Such programs arise frequently in many different application fields and have many important properties, such as strong duality theory and the fact that any local minimum is a global minimum. These programs are not only computationally tractable but also allow for theoretically efficient solution methods. Convex programing consists of several important specially structured classes of problems, such as semidefinite programing, conic quadratic programing, and geometric programing. Let us give some information about conic quadratic programing by benefiting from [5].

Geometrically, a convex program can be represented in the conic form as

$$\begin{array}{r}\underset{\mathit{x}}{\text{minimize}}{\mathit{c}}^T\mathit{x}\text{subject to}\mathit{A}\mathit{x}-\mathit{b}\in K,\end{array}$$ (27)

where $K\subseteq {\mathbb{R}}^N$ is a cone (closed, pointed, convex and with a nonempty interior), and ${\mathbb{R}}^n\to {\mathbb{R}}^N,\text{defined}\text{by}x\mapsto Ax$, is a linear embedding. A conic quadratic problem is a conic problem,

$$\begin{array}{r}\underset{\mathit{x}}{\text{minimize}}{\mathit{c}}^T\mathit{x}\text{subject to}\mathit{A}\mathit{x}-\mathit{b}\in K,\end{array}$$ (28)

for which the cone K is a direct product of several so-called ice-cream cones or second-order cones or Lorentz cone given as

$$\begin{array}{rl}K& =L^{n_{\text{1}}}\times L^{n_{\text{2}}}\dots \times L^{n_k}\\ & =\{{(y{[\text{1}]}^T,\dots ,y{[k]}^T)}^T|y[i]\in L^{n_i}(i=\text{1,2,}\dots ,k)\},\end{array}$$ (29)

where $L^{n_i}$is an $n_i$-dimensional second-order cone defined by

$$\begin{array}{r}{L}^{n_i}=\left\{\mathit{x}={(x_{\text{1}},x_{\text{2}},\dots ,x_{n_i})}^T\in {\mathbb{R}}^{n_i}|x_{n_i}\ge \sqrt{x_1^2+\dots +x_{n_i-1}^2}\right\}(n_i\ge 2).\end{array}$$ (30)

The optimization problem, based on the cone, can be solved by primal-dual interior point methods.

Equation (29) shows that a conic quadratic program is an optimization problem with a linear objective function and finitely many ice-cream constraints

$$\begin{array}{r}{\mathit{A}}_i\mathit{x}-{\mathit{b}}_i\in L^{n_i}\text{(}i=\text{1}\text{,2,}\dots ,k\text{),}\end{array}$$ (31)

where

$$\begin{array}{r}\text{[}\mathit{A},\mathit{b}\text{]}=[{[{\mathit{A}}_{\text{1}}\text{;}{\mathit{b}}_{\text{1}}]}^T,\dots ,{[{\mathit{A}}_k\text{;}{\mathit{b}}_k]}^T{]}^T\end{array}$$ (32)

is the partition of the data matrix $\text{[}\mathit{A},\mathit{b}\text{]}$ corresponding to the partition of $\mathit{y}$ in Equation (26). Thus, conic quadratic program can be written as

$$\begin{array}{r}\underset{\mathit{x}}{\text{minimize}}{\mathit{c}}^T\mathit{x}\text{subject to }{\mathit{A}}_i\mathit{x}-{\mathit{b}}_i\in L^{n_i}(i=\text{1,2,}\dots ,k\text{)}.\end{array}$$ (33)

Partitioning the data matrix $[{\mathit{A}}_i\text{;}{\mathit{b}}_i]$ by

$$\begin{array}{r}[{\mathit{A}}_i;{\mathit{b}}_i]=\left[\begin{array}{cc}{\mathit{D}}_i& {\mathit{d}}_i\\ {\mathit{p}}_i^T& {\mathit{q}}_i\end{array}\right],\end{array}$$ (34)

with ${\mathit{D}}_i$ being of the type $\text{(}{n}_i-1\text{)}\times (\dim x)$, the problem can be written as

$$\begin{array}{r}\underset{\mathit{x}}{\text{minimize}}{\mathit{c}}^T\mathit{x}\text{subject to }||{\mathit{D}}_i\mathit{x}-{\mathit{d}}_i|{|}_2\le {\mathit{p}}_i^T\mathit{x}-q_i(i=\text{1,2,}\dots ,k).\end{array}$$ (35)

This is a most explicit form of the conic problem and the one which we will use. In this form, ${\mathit{D}}_i$ are matrices of the same row dimension as $\mathit{x}$. Furthermore, the lengths of the column vectors ${\mathit{d}}_i$ are the column dimensions of the matrices ${\mathit{D}}_i$, and ${\mathit{p}}_i$ are column vectors of the same dimension as $\mathit{x}$; finally, $q_i$ are reals [5]

Consequently, the problem dual to Equation (28) is

$$\begin{array}{r}\underset{\mathit{\lambda }}{\text{max}}\text{imize}{\mathit{b}}^T\mathit{\lambda }\text{subject to }{\mathit{A}}^T\mathit{\lambda }=\mathit{c},\mathit{\lambda }\in K.\end{array}$$ (36)

If we write $\mathit{\lambda }:=({{\mathit{\lambda }}_1}^T,{{\mathit{\lambda }}_2}^T,\dots ,{{\mathit{\lambda }}_k}^T{)}^T$ with $n_i$-dimensional blocks ${\mathit{\lambda }}_i$, then the dual problem can be stated as follows:

$$\begin{array}{r}\underset{{\mathit{\lambda }}_1,\dots ,{\mathit{\lambda }}_n}{max\text{imize}}\sum _{i=1}^k{\mathit{b}}_i^T{\mathit{\lambda }}_i\text{ subject to }\sum _{i=1}^k{\mathit{A}}_i^T{\mathit{\lambda }}_i=c\text{and }{\mathit{\lambda }}_i\in L^{n_i}(i=\text{1,2,}\dots ,k).\end{array}$$ (37)

If it is taken ${\mathit{\lambda }}_i=({{\mathit{\kappa }}_i}^T,{\nu }_i{)}^T$ with a scalar component ${\nu }_i$, it can be shown that the following form is the problem dual to Equation (35):

$$\begin{array}{r}\underset{({\mathit{\kappa }}_i),({\nu }_i)}{max\text{imize}}\sum _{i=1}^k[{\mathit{\kappa }}_i^T{\mathit{d}}_i+{\nu }_i{q}_i]\text{ subject to}\sum _{i=1}^k[{\mathit{D}}_i^T{\mathit{\kappa }}_i+{\nu }_i{\mathit{p}}_i]=c,{||{\mathit{\kappa }}_i||}_2\le {\nu }_i\text{(}i=\text{1,2,}\dots ,k).\end{array}$$ (38)

The design variables in Equation (38) are column vectors ${\mathit{\kappa }}_i$, having the same dimensions as the vectors ${\mathit{d}}_i$, and reals ${\nu }_i$(i = 1,2, … , k). The problems in Equations (35) and (38) are standard forms of a conic quadratic problem and of its dual.

4.2. Application of CQP to M-Estimation to MSOM ( $\mathit{C}{\mathit{M}}_{\mathit{T}}\mathit{M}\mathit{S}\mathit{O}\mathit{M}$)

In this subsection, we solve the problem in Equation (26) by using Conic Quadratic Problems with their Interior Point Methods, IPMs, as an alternative to Exterior or Penalty Methods which can be applied to the formulation of Tikhonov Regularization. CQP and IPMs [25,35] are very well understood and numerically very much developed now, since they possess the form of well-structured convex optimization problems [5] with their advanced theory and powerful methods. Furthermore, our more mathematical approach is a model-based one which is an alternative to more model-free approaches from statistics which, e.g. employ generalized cross-validation. In fact, we are on a way toward a unified treatment of our data mining problem with the help of modern mathematical optimization.

Let us deal with the Tikhonov regularization problem in Equation (26) through CQP. We can easily formulate Equation (26) as a CQP problem. Indeed, based on an appropriate choice of a bound S, we state the following optimization problem:

$$\begin{array}{r}\begin{array}{l}\text{minimize }||\mathit{A}{\mathit{\beta }}^{\ast }-{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}|{|}_2^2\\ \text{subject}\text{to}||{\mathit{\beta }}^{\ast }|{|}_2^2\le S.\end{array}\end{array}$$ (39)

Let us underline that this choice of S should be the outcome of a careful learning process, with the help of method-free or method-based methods [14]. In our study, this bound is selected by a trial and error approach as an initial treatment. In future research, S might also be regarded as a decision variable. In Equation (39), we have the least-squares objective function $||\mathit{A}{\mathit{\beta }}^{\ast }-{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}|{|}_2^2$ which is a nonlinear function and the inequality constraint function $-||{\mathit{\beta }}^{\ast }|{|}_2^2+S$, which is requested to be nonnegative for feasibility. Therefore, we can pass from optimization problem in Equation (39) to an equivalent one with a linear objective. To that aim, it will be enough to add a new design or height variable, say, t, and rewrite the problem of Equation (39), equivalently as follows:

$$\begin{array}{r}\begin{array}{l}\underset{t,{\mathit{\beta }}^{\ast }}{\text{minimize}}t\\ \text{subject}\text{to}||\mathit{A}{\mathit{\beta }}^{\ast }-{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}|{|}_2^2\le t^2,t\ge 0,\\ ||{\mathit{\beta }}^{\ast }|{|}_2^2\le S,\end{array}\end{array}$$ (40)

or, equivalently again,

$$\begin{array}{r}\begin{array}{l}\underset{t,{\mathit{\beta }}^{\ast }}{\text{minimize}}t\\ \text{subject}\text{to}||\mathit{A}{\mathit{\beta }}^{\ast }-{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}|{|}_2\le t,\\ ||{\mathit{\beta }}^{\ast }|{|}_2\le \sqrt{S}.\end{array}\end{array}$$ (41)

In fact, the problem in Equation (41) is a CQP problem as in Equation (35) with

$$\begin{array}{rl}\mathit{c}& ={\left(\begin{array}{cc}1|& {\mathbf{0}}_{p+m}^T\end{array}\right)}^T,\mathit{u}=(t,{{\mathit{\beta }}^{\ast }}^T{)}^T,\\ {\mathit{D}}_1& =\left(\begin{array}{cc}{\mathbf{0}}_{n\times \text{1}}|& {\mathit{A}}_{n\times (p+m)}\end{array}\right),{\mathit{d}}_1={\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }},q_1=0,\\ {\mathit{p}}_1& =(1,0,\dots ,0{)}_{1\times (p+m)}^T,{\mathit{D}}_2=\left(\begin{array}{cc}{\mathbf{0}}_{(p+m)\times 1}|& {\mathit{I}}_{\text{(}p+m\text{)}\times (p+m)}\end{array}\right),\\ {\mathit{d}}_2& ={{\text{0}}_{\text{(p}+m\text{)}}}_{\times 1},{\mathit{p}}_2={\mathbf{0}}_{(p+m)\times 1}\text{and}\\ q_2& =-\sqrt{S}.\end{array}$$

In order to state optimality conditions for this problem, we firstly reformulate the problem of Equation (41) as follows:

$$\begin{array}{r}\begin{array}{l}\underset{t,{\mathit{\beta }}^{\ast }}{\text{minimize}}t\\ \text{subject}\text{to}\mathit{\chi }=\left(\begin{array}{cc}{\mathbf{0}}_{n\times 1}& {\mathit{A}}_{n\times (p+m)}\\ \text{1}& {\mathbf{0}}_{(p+m)\times 1}^T\end{array}\right)\left(\begin{array}{c}t\\ {\mathit{\beta }}^{\ast }\end{array}\right)+\left(\begin{array}{c}-{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}\\ 0\end{array}\right),\\ \mathit{\eta }=\left(\begin{array}{cc}{\mathbf{0}}_{(p+m)\times 1}& {\mathit{I}}_{\text{(}p+m\text{)}\times (p+m)}\\ \mathbf{0}& {\mathbf{0}}_{(p+m)\times 1}^T\end{array}\right)\left(\begin{array}{c}t\\ {\mathit{\beta }}^{\ast }\end{array}\right)+\left(\begin{array}{c}{\mathbf{0}}_{(p+m)\times 1}\\ S\end{array}\right),\\ \mathit{\chi }\in L^{n+1},\mathit{\eta }\in L^{p+m+1},\end{array}\end{array}$$ (42)

where $L^{n+1},L^{p+m+1}$are $(n+1)\text{ - }$ and $(p+m+1)\text{ - }$dimensional second-order (or Lorentz) cones. The dual problem to our primal problem Equation (41) is given by

$$\begin{array}{r}\begin{array}{l}\text{maximize}{\left(\begin{array}{c}{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}\\ 0\end{array}\right)}^{\mathrm{T}}{\mathit{\omega }}_{\mathbf{1}}+\left(\begin{array}{c}{\mathbf{0}}_{(p+m)\times 1}\\ -\sqrt{S}\end{array}\right){\mathit{\omega }}_{\mathbf{2}}\\ \text{subject}\text{to}\left(\begin{array}{cc}{\mathbf{0}}_{n\times 1}^T& 1\\ {\mathit{A}}^T& {\mathbf{0}}_{(p+m)\times 1}\end{array}\right){\mathit{\omega }}_{\mathbf{1}}\\ +\left(\begin{array}{cc}{\mathbf{0}}_{(p+m)\times 1}^T& 0\\ {\mathit{I}}_{\text{(}p+m\text{)}\times (p+m)}& {\mathbf{0}}_{(p+m)\times 1}\end{array}\right){\mathit{\omega }}_{\mathbf{2}}=\left(\begin{array}{c}1\\ {\mathbf{0}}_{(p+m)\times 1}\end{array}\right),\\ {\mathit{\omega }}_{\mathbf{1}}\in L^{n+1},{\mathit{\omega }}_{\mathbf{2}}\in L^{p+m+1}.\end{array}\end{array}$$ (43)

Moreover, $(t,{\mathit{\beta }}^{\ast },\mathit{\chi },\mathit{\eta },{\mathit{\omega }}_{\mathbf{1}},{\mathit{\omega }}_{\mathbf{2}})$ is a primal-dual optimal solution if and only if [5,22,52]:

$$\begin{array}{r}\begin{array}{l}\\ \mathit{\chi }=\left(\begin{array}{cc}{\mathbf{0}}_{n\times 1}& \mathit{A}\\ \text{1}& {\mathbf{0}}_{(p+m)\times 1}^T\end{array}\right)\left(\begin{array}{c}t\\ {\mathit{\beta }}^{\ast }\end{array}\right)+\left(\begin{array}{c}-{\mathit{y}}^{\mathbf{\ast }\mathbf{\ast }}\\ 0\end{array}\right),\\ \mathit{\eta }=\left(\begin{array}{cc}{\mathbf{0}}_{(p+m)\times 1}& {\mathit{I}}_{\text{(}p+m+1\text{)}\times (p+m)}\\ \text{0}& {\mathbf{0}}_{{}_{(p+m)\times 1}}^T\end{array}\right)\left(\begin{array}{c}t\\ {\mathit{\beta }}^{\ast }\end{array}\right)+\left(\begin{array}{c}{\mathbf{0}}_{(p+m)\times 1}\\ \sqrt{S}\end{array}\right),\\ \left(\begin{array}{cc}{\mathbf{0}}_{n\times 1}^T& 1\\ {\mathit{A}}^T& {\mathbf{0}}_{(p+m)\times 1}\end{array}\right){\mathit{\omega }}_{\mathbf{1}}+\left(\begin{array}{cc}{\mathbf{0}}_{(p+m+1)\times 1}^T& 0\\ {\mathit{I}}_{\text{(}p+m\text{)}\times (p+m)}& {\mathbf{0}}_{(p+m+1)\times 1}\end{array}\right){\mathit{\omega }}_{\mathbf{2}}=\left(\begin{array}{c}1\\ {\mathbf{0}}_{(p+m)\times 1}\end{array}\right),\\ {\mathit{\omega }}_{\mathbf{1}}^T\mathit{\chi }=0,{\mathit{\omega }}_{\mathbf{2}}^T\mathit{\eta }=0,\\ {\mathit{\omega }}_{\mathbf{1}}\in L^{n+1},{\mathit{\omega }}_{\mathbf{2}}\in L^{p+m+1},\mathit{\chi }\in L^{n+1},\mathit{\eta }\in L^{p+m+1}.\end{array}\end{array}$$ (44)

To solve linear programing, LP, and well-structured convex optimization problems, the classes, such as conic CQP, IPMs, which were introduced by Karmarkar [19], can be used [26,35]. These methods have been applied effectively since 1984 [19], and they are very efficient in solving CQP, semidefinite and geometric optimization problems. IPMs are exhibiting and exploiting different properties of LP and CQP that make them very attractive for large-scale optimization. These algorithms have the advantage of employing the structure of the program, of allowing better complexity bounds and displaying a much better practical performance.

5. Numerical application

The numerical application of the $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ method is conducted by using Parkinson Telemonitoring Dataset from UCI: Machine Learning Repository (available at https://archive.ics.uci.edu/ml/datasets/Parkinsons+Telemonitoring). The attribute lists of this dataset are shown in Table 1, with their names used in the $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ model.

Table 1.

The attributes of Parkinson’s telemonitoring dataset.

Name for CMTMSOM	Variable	Name for CMTMSOM	Variable
Y1	mUPDRS	Shimmer A	X8
Y2	tUPDRS	Shimmer A	X9
X1	Jitter (%)	Shimmer A	X10
X2	Jitter (Abs)	Shimmer DDA	X11
X3	Jitter RAP	NHR	X2
X4	Jitter P	HNR	X13
X5	Jitter DDP	RPDE	X14
X6	Shimmer	DFA	X15
X7	Shimmer (dB)	PPE	X16

Table 2 reports several descriptive statistics for the response (output) and predictor (response) variables. The results show that some attributes have different ranges of values. For that reason, all attributes are normalized in order to make them comparable while doing $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ modeling.

Table 2.

Descriptive statistics of the selected variables.

	Mean	Standard Deviation	Median	Skewness	Kurtosis
Y1	21.3	8.13	20.87	0.08	−0.94
Y2	29.02	10.7	27.58	0.27	−0.36
X1	0.01	0.01	0	6.45	67.41
X 2	0	0	0	3.28	18.13
X 3	0	0	0	6.94	78.44
X 4	0	0	0	7.58	81.47
X 5	0.01	0.01	0.01	6.94	78.44
X 6	0.03	0.03	0.03	3.31	15.22
X 7	0.31	0.23	0.25	3.1	13.07
X 8	0.02	0.01	0.01	3.1	14.7
X 9	0.02	0.02	0.02	3.7	19.22
X 10	0.03	0.02	0.02	3.41	19.14
X 11	0.05	0.04	0.04	3.1	14.71
X 12	0.03	0.06	0.02	6.55	52.54
X 13	21.68	4.29	21.92	−0.81	2.5
X 14	0.54	0.1	0.54	−0.04	−0.07
X 15	0.65	0.07	0.64	0.28	−0.88

In order to investigate the relation between the variables, correlation matrix is obtained for the given dataset. The results show that seven variables $(X_{\text{1}}\text{, }{X}_{\text{4}}\text{, }{X}_{\text{5}}\text{, }{X}_{\text{6}}\text{, }{X}_{\text{7}}\text{, }{X}_{\text{8}}\text{, }{X}_{\text{9}})$ are highly correlated with other variables. All these variables are significantly linked with each other, since they belong to the measures of variation in fundamental frequency and in amplitude. Moreover, the correlation between $Y_{\text{1}}$ and $Y_{\text{2}}$ is very high (0.94), and they are positively correlated with each other. Therefore, we eliminate $(X_{\text{1}}\text{, }{X}_{\text{4}}\text{, }{X}_{\text{5}}\text{, }{X}_{\text{6}}\text{, }{X}_{\text{7}}\text{, }{X}_{\text{8}}\text{, }{X}_{\text{9}})$ and $Y_{\text{2}}$ by conducting a multicollinearity test in this study. After removing the variables that are correlated to each other by more than 0.9, the variables selected to model this dataset are determined, and their correlation matrix is introduced in Table 3. By this table, acceptable results have been obtained according to the statistical modeling theory.

Table 3.

The correlation among the variables selected into to the CM_TMSOM model.

	Y1	X2	X3	X10	X11	X12	X13	X14	X15	X16
Y1	1
X2	0.0508	1
X3	0.0726	0.8446	1
X10	0.1365	0.5899	0.6030	1
X11	0.0842	0.6238	0.6502	0.8856	1
X12	0.0749	0.6999	0.7923	0.7115	0.7327	1
X13	−0.1570	−0.7064	−0.6414	−0.7779	−0.7806	−0.6844	1
X14	0.1286	0.5470	0.3828	0.4807	0.4368	0.4166	0.6590	1
X15	−0.1162	0.3522	0.2148	0.1796	0.1307	0.0220	−0.2905	−0.1920	1
X16	0.1624	0.7878	0.6706	0.6234	0.5767	0.5646	0.7587	−0.5660	0.3946	1

In this study, some of the variables, such as age and gender, will not be used in modeling; instead, they will be employed for interpretation of the final model. In fact, the number of people and age ranges are different according to gender. In this dataset, there are 28 males, and 4008 observations out of total 5876 within this dataset belong to males. The rest are obtained from females (14 people with 1867 observations). A pie chart of given dataset, according to gender, is provided in Figure 1. While the range of males’ age is between 49 and 78 years, the range of age for females is between 36 and 85. Corresponding age and gender patterns are given in Figure 2.

Figure 1.

Pie chart of Parkinson’s dataset according to gender.

Figure 2.

Age and gender patterns for Parkinson’s data set.

In order to find the outliers in this dataset for males and females, the following outlier detection procedure, demonstrated in Figure 3, is applied [13,23]. First of all, a multivariate linear model is constructed and the residuals are obtained. Outlier detection measures, which are studentized residuals, leverage, measures of influence, such as Cooks distance, and model performance measures, are calculated. Detailed information on these measures can be found in Montgomery and Peck [23]. After that, in order to find actual outliers, the potential outliers are removed from the dataset, and the corresponding model fit is checked. This procedure is repeated until all of the outlier observations are detected in the dataset.

Figure 3.

Flowchart for the outlier detection procedure.

For each gender, we obtained the number of potential outliers, as presented in Figure 4. Although the number of females is much lower than the number of males, most of the outlier observations belong to the group of females. After the detection of outliers within the given dataset, we constructed three models, LM, MSOM and $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$, on our normalized dataset. In this study, the applications are performed by using MATLAB, and for the $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ model parameters, a MATLAB code is running along with using the optimization software MOSEK^TM.

Figure 4.

Number of outlier observation according to gender.

To evaluate the model efficiency, the model-free performance measures, such as Mean Absolute Error (MAE), Mean Squared Error (MSE), Correlation Coefficient (r), Multiple Coefficient of Determination $(R^2)$, Adjusted $R^2(\text{Adj - }{R}^2)$, and Mean Absolute Percentage Error (MAPE), are calculated. The results presented are based on improvements of the model performances, as stated in Table 4.

Table 4.

Performances of the models based on improvements in percentage.

	LM $\to$MSOM	LM $\to$ CMTMSOM	MSOM $\to$CMTMSOM
MAE	2% $\downarrow$	4% $\downarrow$	2% $\downarrow$
MAPE	2% $\downarrow$	4% $\downarrow$	2% $\downarrow$
MSE	3% $\downarrow$	5% $\downarrow$	2% $\downarrow$
r	10% $\uparrow$	18% $\uparrow$	6% $\uparrow$
R2	22% $\uparrow$	39% $\uparrow$	13% $\uparrow$
Adj-R2	11% $\uparrow$	28% $\uparrow$	15% $\uparrow$

The results show that the developed $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ model performs fairly well for the given dataset according to all performance measures. This is due to the fact that our new model combines the power of MSOM and M-Estimation in modeling the data with the effective estimation of CQP. The second best model is MSOM. This model can be considered to be in an intermediate position with regard to method-free performance measures. The worse model is LM, when compared to the C_TMSOM and MSOM, since this model does not handle the effects of outlier observations. It should be noted that ‘ $\downarrow$’ and ‘ $\uparrow$’ represent decreasing and increasing behavior of the measure values, respectively. In addition, the symbol ‘ $\to$’ shows the transition based on improvements between both models. For example, $\text{LM}\to \text{MSOM}$ and MAE $(\text{2\%}\downarrow )$mean that MSOM displays a $\text{2\%}$ better performance than LM based on MAE measure. Decrease in the values of MAE, MSE and MAPE shows a better result. On the other hand, $\text{LM}\to \text{C}{\text{M}}_{\text{T}}\text{MSOM}$ and $R^2$ $(\text{39\%}\uparrow )$ means that $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ reveals a $\text{39\%}$ better performance than LM based on $R^2$ measure. Increase in the values of r, $R^2$ and $\text{Adj - }{R}^2$ displays a better result. Moreover, Figure 5 displays the relation between the number of parameters in LM, MSOM, $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ models and some of the performance measures such as MAE and RMSE. Although the number of parameters for MSOM and $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ is much more than LM, MSOM and $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ perform much better than LM.

Figure 5.

Comparison of the number of parameters of LM, MSOM, $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ models in terms of MAE and RMSE.

6. Concluding remarks

This paper has given a contribution to problems of outlier-detection for regression in Neuroscience, supported by mean-shift outlier regression, and by combining modern methods of continuous optimization, especially CQP, with M-estimation, which is a robust method. In the study, firstly, we represented mean-shift outlier regression problem as a CQP problem, based on M-estimates. Secondly, we employed mean-shift outlier model as a predictive model for the diagnosis of PD by means of UPDRS on the basis of PD dataset. We achieved to set up a bridge between methods of statistical learning, inverse problems and the powerful tools prepared for well-structured convex optimization.

In this paper, we studied on a relatively high-dimensional dataset for the setting $p<n$. Since $\text{C}{\text{M}}_{\text{T}}\text{MSOM}$ works effectively on such complex and sensitive datasets, for more complete and high dimensional Neuroscience datasets $(p\gg n)$, the results should indicate a better performance when compared to other models which do not take into account outliers [48,49,50,51].

We plan to conduct a comparative study that will consider our method, generalized partial linear model [25] and conic generalized partial linear model (CGPLM) [43] on the basis of certain datasets, also from other dementia diseases, such as Alzheimer’s disease. Furthermore, we consider to conduct a further robustification by employing RCMARS [28], RMARS [29] and RCGPLM [30].

With this developed method, we have sought to make progress in science, especially through mathematics and statistics, to provide novel and powerful elements and modules in order to allow for better early warning system in our field of Neuroscience. We are using these notions with the highest care, well aware of the complexity of neurodegenerative disease and of any subject with very pronounced human factors. This method can gradually become parts of clinical and medical computer programs. What’s more, we can work out and advance further our codes, graphics and tables toward elements in a Graphical User Interface (GUI) which could be used very conveniently and meaningfully by any practitioner. Eventually, we can employ our whole methodology, modules and future GUIs for other highly emerging inverse problems, especially in medical ones, as well as in biosciences, earth sciences or environmental sciences.

Disclosure statement

No potential conflict of interest was reported by the author(s).