Clustering Sequence Data with Mixture Markov Chains with Covariates Using Multiple Simplex Constrained Optimization Routine (MSiCOR)

Abstract

Mixture Markov Model (MMM) is a widely used tool to cluster sequences of events coming from a finite state-space. However, the MMM likelihood being multi-modal, the challenge remains in its maximization. Although Expectation-Maximization (EM) algorithm remains one of the most popular ways to estimate the MMM parameters, however, convergence of EM algorithm is not always guaranteed. Given the computational challenges in maximizing the mixture likelihood on the constrained parameter space, we develop a pattern search-based global optimization technique which can optimize any objective function on a collection of simplexes, which is eventually used to maximize MMM likelihood. This is shown to outperform other related global optimization techniques. In simulation experiments, the proposed method is shown to outperform the expectation-maximization (EM) algorithm in the context of MMM estimation performance. The proposed method is applied to cluster Multiple sclerosis (MS) patients based on their treatment sequences of disease-modifying therapies (DMTs). We also propose a novel method to cluster people with MS based on DMT prescriptions and associated clinical features (covariates) using MMM with covariates. Based on the analysis, we divided MS patients into three clusters. Further cluster-specific summaries of relevant covariates indicate patient differences among the clusters. Supplementary materials for this article are available online.

1. Introduction

Mixture Markov Model (MMM) is a well-known clustering technique in order to cluster event sequences where each event is coming from a state-space given by a collection of finite events. In literature several statistical applications of MMM and Mixture Hidden Markov Markov Model (MHMM) can be found for clustering event sequence data in various fields (Chi et al. 2007; Melnykov 2016; Gupta, Kumar, and Vassilvitskii 2016; Li et al. 2019). A few applications of MMM can also be found in the field of biostatistics (Haan-Rietdijk et al. 2017; Das et al. 2023). However, one of the challenging aspects of estimating MMM parameters remains in optimizing the mixture likelihood. Due to the multi-modal nature of the mixture likelihood, it is difficult to maximize using derivative-based methods as these tend to converge at local maxima. Although the expectation-maximization (EM) algorithm can be used for parameter estimation for MMM (Helske and Helske 2019), its performance largely relies on the initial point solution (Couvreur 1997). Several articles addressed the convergence issues of using EM algorithm for estimating parameters in mixture model (Archambeau, Lee, and Verleysen 2003) which might pose a challenge in obtaining the maximum likelihood estimate (MLE) in case of MMM as well. Gupta, Kumar, and Vassilvitskii (2016) proposed a faster alternative to the EM algorithm for optimizing the MMM likelihood, but this was later shown to under-perform when compared to the EM algorithm in terms of predictive performance. Therefore, exploring alternative approaches to maximize MMM likelihood would be helpful in improving the estimation performance, resulting in improved inference and related prediction performances as well.

As an alternative to EM algorithm and other convex optimization techniques, evolutionary global optimization techniques (Andrzej and Stanislaw 2006) can be considered as a natural choice in order to address the complex and multi-modal nature of the objective function that is, the MMM likelihood. For optimizing multi-modal functions, several evolutionary global optimization techniques have been proposed in the last few decades, for example, “Genetic Algorithm (GA)” (Fraser 1957; Bethke 1980; Goldberg 1989) and “Simulated Annealing (SA)” (Kirkpatrick, Gelatt, and Vecchi 1983; Granville, Krivanek, and Rasson 1994), which are widely used nowadays. These methods were first developed for unconstrained global optimization, later they were extended for optimizing objective functions with constraints (Smith and Romeijn 1994; Reid 1996). Among other non-deterministic global optimization techniques, Particle Swarm Optimization (PSO) (Kennedy and Eberhart 1995; Eberhart and Kennedy 1995) remains popular for unconstrained global optimization. Torczon (1997) proposed a different evolutionary based global optimization technique called “Generalized Pattern Search” (GPS) which is a more generalized version of the Direct search method earlier proposed by Hooke and Jeeves (1961). In GPS, using an “exploratory moves algorithm” (Torczon 1997), the objective function values at a set of neighboring points around the current solution are evaluated and the point with the best (i.e., the maximum or minimum, based on requirement) objective function value is selected as the updated solution. Later GPS has been extended to solve global optimization problems on several constrained spaces and shown to outperform other related evolutionary based algorithms (Das 2021; Das et al. 2022). In this article, we propose a Pattern search based algorithm to maximize the MMM likelihood, compare the performance of the proposed optimization algorithm to other existing evolutionary global optimization techniques, and eventually demonstrate the estimation performance of the proposed algorithm while maximizing MMM likelihood to outperform the results obtained using EM algorithm based on simulation experiments.

The parameter space of the MMM likelihood for known (finite) number of clusters is composed of a set of transition matrices, initial state probability vectors and the vector of mixture proportions. Now, the parameters of the collection of transition matrices can be also observed as a collection of simplexes since each row of a transition matrix belongs to a simplex space. Also, note that, the mixture proportion vector and each of the initial state probability vectors are simplexes as well. Therefore, the whole parameter space of the MMM likelihood can be expressed as a collection of simplexes. In order to solve the MMM likelihood, we come up with a Pattern search based global optimization technique to solve any objective function whose parameter space is given by a collection of simplexes, called multiple simplex constrained optimization routine (MSiCOR). We also explore some of its theoretical properties.

As an application to MSiCOR in the context of MMM, we cluster Multiple sclerosis (MS) patients based on their disease-modifying treatment (DMT) prescription history. MS is an autoimmune disease of the the central nervous system (CNS) that leads to neurodegeneration in the brain and spinal cord. In MS, the immune system damages the protective sheath (myelin) that covers nerve fibers, causing dysfunctional communication between the CNS and the rest of the body For most people with MS (pwMS), the disease starts with relapses resulting in episodes of new or worsening symptoms due to acute focal inflammation of the CNS. MS is generally diagnosed in early adulthood. Major symptoms of MS include muscle stiffness, paralysis, cognitive impairment, fatigue, depression, visual disturbance, balance and gait difficulty, and problems with bladder, bowel, or sexual function.

Although there is no cure for MS, there are 20 FDA-approved DMTs that reduce inflammatory disease activity and delay disease progression. DMTs can be divided into several mechanistic categories, including interferon-beta, glatiramer acetate, fumarates such as dimethyl fumerate and natalizumab, sphigophine-1-phosphate modulators such as fingolimod, and B-cell depletion agents such as rituximab. Over the course of this chronic disease, pwMS typically receive serial DMTs as monotherapies. Common reasons for switching DMTs include therapeutic failure (e.g., relapse), intolerance or adverse events.

Prior research efforts identified various demographic, clinical, genetic and neuroimaging features associated with disease activity of pwMS (Myhr et al. 2001; Barcellos et al. 2002). Further, several research works focused on the time series analysis of magnetic resonance imaging (MRI) intensity (Meier and Guttman 2003) and volumetric medical image sequences (Thirion and Calmon 1999; Ghribi et al. 2018) of the MS patients. Garcia-Dominguez et al. (2016) showed the dependence of DMT preference on socio-demographic and clinical features. However, to the best of our knowledge, few studies have examined DMT prescription sequence data. These prior studies have focused on either a single best patient-specific DMT option (Grand’Maison et al. 2018), or the criteria for switching or stopping DMTs (Gross and Corboy 2019). Moreover, emphasis has remained mostly on possible treatment options of a patient at a given time point based on response to previous DMTs. To the best of our knowledge, DMT prescription sequence remains an unexplored clinical space in MS.

As an example, it is possible to estimate transition probabilities of moving from one DMT to another DMT based on patient history both at a personal and at a population level under certain statistical model assumptions. We hypothesize that the variation among these transition probabilities across patients is governed by clinical, demographic, and/or other factors. Since the choice of DMT prescription largely depends on patient disease status, the prescribed DMT at any given time can be mapped to a set of clinical conditions and observations. Thus, analysis of DMT transitions and patient clusters based on how transitions across DMT options occur over individual timelines can provide insights on the associations of the clinical features and observations with DMT prescription sequence.

Different statistical models can be used to cluster event sequences in logitudinal data (Scott, Mohan, and Gauthier 2020; Murphy et al. 2021). One potential approach is marked point processes (MPP; Jacobsen 2006). However, there is generally a lag between the actual onset of new neurological symptoms (i.e., relapses indicative of disease activity) and the observed date of DMT prescription. The lag can occur when patients visit their physicians a few days after the onset of symptoms/relapses. Given the typical time that it takes for insurance authorization process, there can be further lags between prescription date and the actual DMT start date. Thus, in the scenario of MS DMTs, incorporating the observed time periods between any two prescription dates into the model might be misleading, making MPPs unsuitable for modeling. However, the order in which a patient has been prescribed DMTs over their observation period is still useful and can be modeled in other ways using state-space models (SSMs). In the literature, hidden Markov chain models (HMMs) have been used to cluster event sequences (Helske and Helske 2019). However in our scenario we do not have any hidden underlying variable like in HMMs.

In this article, we model MS DMT prescription sequences using a mixture of discrete state-space Markov chain models. There are a few notable reasons for clustering pwMS based on DMT sequences. First, over the disease course, pwMS are serially treated with multiple DMTs that might change over time. As such, estimating the transitional probabilities of changing across different pairs of DMTs across MS patients or any sub-populations will have clinical relevance. Second, after the individuals are clustered based on DMT sequences, cluster-specific summary statistics of clinical and demographic features can inform prescription guidance for future patients. Further, patient-specific covariates can also be used for clustering along with DMT sequences. We thus incorporate patient-specific covariates within MMMs to explain cluster memberships as in latent class analysis. Although prior publications reported MMM clustering of sequence data (e.g., Gupta, Kumar, and Vassilvitskii 2016), to the best of our knowledge, none has considered MMM analysis including subject-level covariates. Helske and Helske (2019) describes the outline of the MMM and MHMM clustering, including covariates, but no further extensive simulation studies and case studies were explored using MMM. Bolano (2020) considered a mixture transition distribution-like model to account for covariates in Markovian models with illustration using a 3-state HMM and a covariate with three levels. Of clinical relevance, to the best of our knowledge, no study has clustered MS DMT sequence data using MMMs. In related literature, Altman and Petkau (2005) applied a HMM on MRI lesion count data for MS patients, though this is substantially different from the proposed Markov chain model where the state-space consists of all possible types of DMTs for MS patients and the underlying model is a MMM.

The rest of this article is organized as follows. The mixture Markov chain model is described in Section 2. Section 3 is dedicated to the maximum likelihood estimation process, including its computational aspects. Specifically, MSiCOR is described in Section 3.2. The comparative performance of MSiCOR along with some existing optimization techniques is evaluated in Section 3.3; and some of its theoretical properties are discussed in Section 3.4. A simulation study is conducted in Section 4 to test the performance of MSiCOR, including comparisons with the EM algorithm in Section 4.5, in the context of MMM parameter estimation. Section 5 contains the analysis of MS DMT sequence data along with patient-level clinical data. Finally, Section 6 concludes the article.

2. Mixture Markov Chain Model (MMM)

2.1. MMM Likelihood Without Covariates

Suppose the observed data consist of medication sequences on $N$ treatments from $K$ patients, ${\{{\mathbf{Y}}_{\mathit{i}}=(Y_{i,1},\dots ,Y_{i,h_i})\}}_{i=1}^K$, where $Y_{i,t}\in \{1,\dots ,N\}$ represents the treatment received by patient $i$ at time $t\in \{1,\dots ,h_i\}$, and $h_i$ denotes the length (or the total number of time-points) of the medication sequence for the $i\text{th}$ patient. We assume that the medication sequences are generated from $L$ time-homogeneous Markov chain processes, or clusters. Let $\mathbf{M}=[M_1,\dots ,M_L]$ denote $L$ Markov transition matrices corresponding to the $L$ clusters, and $\mathbf{s}={[s_l(k)]}_{N\times L}=[{\mathbf{s}}_1,\dots ,{\mathbf{s}}_L]$ denote the corresponding initial state probability vectors where ${\mathbf{s}}_l={[s_l(1),\dots ,s_l(N)]}^{⊺}$ for $l\in \{1,\dots ,L\}$. Then

$$\mathbb{P}({\mathbf{Y}}_i\mid z_i=l,{\mathbf{X}}_i)=s_l(Y_{i,1})M_l(Y_{i,1},Y_{i,2})\cdots M_l(Y_{i,h_i-1},Y_{i,h_i}),$$ (1)

where $z_i$ denotes the cluster membership of the $i\text{th}$ patient, $M_l(y_t,y_{t+1})$ denotes the transition probability from $y_t$ to $y_{t+1}$ and $s_l(y_t)$ denotes the initial state probability of the state $y_t$ in the $l\text{th}$ cluster, for $y_t\in \{1,\dots ,N\}$. The likelihood is given by

$$\begin{gathered} L_{\mathbf{M},\mathbf{s}}(Y_1,\dots ,Y_K)=\prod _{i=1}^K\sum _{l=1}^L{w}_l{s}_l(Y_{i,1})M_l(Y_{i,1},Y_{i,2})\cdots \\ {M}_l(Y_{i,h_i-1},Y_{i,h_i}), \end{gathered}$$ (2)

where ${\{w_l\}}_{l=1}^L$ denotes the mixture probability of the $l\text{th}$ cluster such that ${\sum }_{l=1}^L{w}_l=1,w_l\ge 0$.

2.2. MMM Likelihood with Covariates

Consider along with medication sequence data, we also have covariate data for those $K$ patients given by ${\{{\mathbf{X}}_i\}}_{i=1}^K$, where ${\mathbf{X}}_i$ is a $p$-dimensional covariate with 1 being its first element. We assume that the medication sequences are generated from latent class models of $L$ time-homogeneous Markov chain processes with latent class membership $z$ also depending on the covariates ${\mathbf{X}}_i$ but that ${\mathbf{Y}}_i$ is independent of ${\mathbf{X}}_i\mid z$. (1) holds true in this case as well, and additionally we assume

$$w_{il}\equiv \mathbb{P}(z_i=l\mid {\mathbf{X}}_i)=\frac{\text{exp}({\mathbf{X}}_i^{⊺}{\gamma }_l)}{1+{\sum }_{l^{\prime }=2}^L\text{exp}({\mathbf{X}}_i^{⊺}{\gamma }_{l^{\prime }})},l=1,\dots ,L,$$ (3)

where $\Gamma =({\gamma }_1,\dots ,{\gamma }_L)$ denotes the cluster-specific coefficient vectors of the covariates. First cluster is considered as the baseline cluster, so we take ${\gamma }_1\equiv 0$. Here the likelihood is given by

$$\begin{gathered} L_{\mathbf{M},\mathbf{s},\Gamma }(Y_1,\dots ,Y_K)=\prod _{i=1}^K\sum _{l=1}^L{w}_{il}{s}_l(Y_{i,1})M_l(Y_{i,1},Y_{i,2})\cdots \\ {M}_l(Y_{i,h_i-1},Y_{i,h_i}). \end{gathered}$$ (4)

The posterior probability of a patient belonging to cluster $l$ can be obtained as

$$\mathbb{P}(z_i=l\mid Y_i,X_i)=\frac{\mathbb{P}(Y_i\mid z_i=l,X_i)\mathbb{P}(z_i=l\mid X_i)}{{\sum }_{l^{\prime }=1}^L\mathbb{P}(Y_i\mid z_i=l^{\prime },X_i)\mathbb{P}(z_i=l^{\prime }\mid X_i)}.$$ (5)

Once we estimate the cluster-specific parameters, we can use (5) to identify where any given patient belongs. Thus, after estimating the clusters, we may summarize the available statistics for the patients to explore the cluster characteristics and differences. In Section 5, after the clusters are estimated based on real data, we summarize the patient relapse rate and rates of a few other relevant medical codes for each cluster. These include the International Classification of Diseases (ICD) code, Current Procedural Terminology (CPT) code, and the Concept Unique Identifiers (CUIs).

3. Computation

3.1. The Estimation Problem

We maximize (4) in order to estimate the parameters for each Markov chain component. To begin with, the $(d-1)$-dimensional simplex is defined as

$${\Delta }^{d-1}=\left\{(x_1,\dots ,x_d)\in {\mathbb{R}}^d:x_i\ge 0,i=1,\dots ,d,\sum _{i=1}^d{x}_i=1\right\}.$$

In (4), $s_l$ and each row of $M_l$ belongs to $(N-1)$ dimensional simplexes, and $\alpha \in {\Delta }^{L-1}$. Apart from the coefficient vector $\Gamma$, the remainder of the parameter space consists of $L(N+1)$ simplexes of size $(N-1)$ and one simplex of size $(L-1)$.

In order to maximize (4), one may use the EM algorithm (Helske and Helske 2019), however, EM algorithm tends to get stuck at poor local solutions due to the possible multi-modal nature of the mixture likelihood. Moreover, the estimation performance of EM also depends on the initial starting point, as discussed in Section 1. An alternative is to apply a direct numerical maximization procedure on the objective function. Widely used algorithms for constrained optimization include interior point (IP) methods (Potra and Wright 2000), sequential quadratic programming (SQP; Wright 2005), and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. However, these algorithms typically require good initial starting points in order to reduce the risk of being trapped at poor local maxima. In addition, popular global optimization techniques include the genetic algorithm (GA; Fraser 1957) and simulated annealing (SA; Kirkpatrick, Gelatt, and Vecchi 1983). However, a major disadvantage of these algorithms is that they can be computationally slow, and thus there remains a tradeoff between estimation accuracy and computation time.

Among other global optimization methods, a generalized pattern search algorithm was proposed by Torczon (1997), and variations have been proposed for various constrained parameter spaces (Lewis and Torczon 1999, 2000). Variants of PS have been shown to outperform existing GA or SA algorithms (see, e.g., Das 2021). In order to maximize the likelihood, we first develop a Pattern search (Torczon 1997) based algorithm to maximize any objective function whose parameter space is given by a collection of simplexes, called multiple simplex constrained optimization routine (MSiCOR). Then, we further modify it to perform updating steps of the unconstrained coefficient vector $\Gamma$, with an end goal to maximize (4) overall. Thus, we consider a variation of PS that can be used to optimize objective functions over parameter spaces that are a collection of simplexes and of unconstrained parameter spaces, as is the case for the MMM likelihood with covariates based on MS DMT sequences and patients’ clinical data.

3.2. Multiple Simplex Constrained Optimization Routine (MSiCOR)

Consider a non-convex objective function $f:{\Delta }^{n_1-1}\times \cdots \times {\Delta }^{n_B-1}\mapsto \mathbb{R}$ which we wish to minimize. In pattern search (PS), within an iteration while optimizing over a $M$-dimensional space, $2M$ candidate points in the neighborhood of the current solution are explored. These are obtained by changing one coordinate at a time, both in positive or negative direction, keeping other coordinates unchanged. For example, in case we want to optimize a function $f$ over an unconstrained space ${\mathbb{R}}^M$, given the step-size $s$ and current solution $\mathbf{z}=(z_1,\dots ,z_M)$, the objective function $f$ is evaluated at $2M$ new candidate points, given by ${\{{\mathbf{z}}_i^{+}\}}_{i=1}^M$ and ${\{{\mathbf{z}}_i^{-}\}}_{i=1}^M$ where ${\mathbf{z}}_i^{+}=(z_1,\dots ,z_{i-1},z_i+s,z_{i+1},\dots ,z_M)$ and ${\mathbf{z}}_i^{-}=(z_1,\dots ,z_{i-1},z_i-s,z_{i+1},\dots ,z_M)$. At each iteration, the best solution out of these $(2M+1)$ points is selected based on the objective function values at those points. In PS, the step-size $s$ is chosen adaptively. Unlike the case of unconstrained pattern search, in our problem, the parameter space is composed of multiple simplexes, and the general PS needs to be modified for our scenario.

MSiCOR consists of several runs. Iterations are performed within each run until a convergence criteria is met, which is detailed in the sequel. Each run starts from the solution returned by the previous run and attempts to find a better solution, with the initial solution for the first run being user-provided. The algorithm terminates and returns the final solution when the solutions obtained by two consecutive runs are close. Having multiple runs aids in jumping out of local minima.

3.2.1. Tuning Parameters

Each run depends on the following tuning parameters: initial global step-size $s_{\text{initial}}>0$, step decay rate $\rho >1$, step-size threshold $\varphi >0$, and sparsity threshold $\lambda \ge 0$. The values of these tuning parameters are set by the user and are kept unchanged across runs. We consider two additional tuning parameters ${\tau }_1$, ${\tau }_2$ to control the convergence criteria. Finally, the maximum number of iterations within a run and the maximum number of runs can be fixed as $M_{\text{iter}}$ and $M_{\text{run}}$, respectively.

3.2.2. Global and Local Step-Sizes

The parameter space consists of multiple unit-simplex blocks. Suppose there are $B$ unit-simplex blocks, and that the $j\text{th}$ simplex block is $(n_j-1)$-dimensional and denoted by ${\mathbf{P}}_j=(p_{j,1},\dots ,p_{j,n_j})\in {\Delta }^{n_j-1}$ for $j=1,\dots ,B$. The total number of parameters is $M={\sum }_{j=1}^B{n}_j$. Within each run, we consider a global step-size $\eta$ and $2M$ local step-sizes ${\{{\{(s_{j,i}^{+},s_{j,i}^{-})\}}_{i=1}^{n_j}\}}_{j=1}^B$ which are chosen adaptively depending on the tuning parameter values as well as the improvement in the values of objective function. Inside a run, in the first iteration the value of the global step-size is set to be ${\eta }^{(1)}=s_{\text{initial}}$, where ${\eta }^{(h)}$ denotes the value of global step-size in the $h\text{th}$ iteration in a run. The value of $\eta$ remains the same throughout an iteration. At the end of each iteration, its value either remains same or gets divided by $\rho (>1)$ based on a criteria described later in Section 3.2.6. At the beginning of an iteration, the values of the local step-sizes $s_{j,i}^{+}$ and $s_{j,i}^{-}$ are set to be the value of the global step-size $\eta$ of that iteration.

3.2.3. Exploratory Movements

Suppose the current value of the parameters at the beginning of the $h\text{th}$ iteration is $\mathbf{P}={\mathbf{P}}^{(h)}=({\mathbf{P}}_1^{(h)},\dots ,{\mathbf{P}}_B^{(h)})$, where ${\mathbf{P}}_j^{(h)}=(p_{j,1}^{(h)},\dots ,p_{j,n_j}^{(h)})\in {\Delta }^{n_j-1}$ for $j=1,\dots ,B$. During the iteration, the objective function is evaluated at $2M$ feasible points in the neighborhood of ${\mathbf{P}}^{(h)}$. These feasible points are obtained by taking steps around ${\mathbf{P}}^{(h)}$ modulated by the local step-sizes $s_{j,i}^{+}$, $s_{j,i}^{-}$ for $j=1,\dots ,B$, and $i=1,\dots ,n_j$. These can be divided into $M$ “positive” movements $(j,i,+)$ and $M$ “negative” movements $(j,i,-)$ for $j=1,\dots ,B$ and $i=1,\dots ,n_j$. We call a coordinate of a unit-simplex box “significant” if its value is greater than $\lambda$. Suppose that there are $m_j(<n_j)$ significant positions in the $j\text{th}$ simplex block ${\mathbf{P}}_j^{(h)}$ excluding the $i\text{th}$ position $p_{j,i}^{(h)}$. The $(j,i,+)\text{th}$ movement consists of updating $p_{j,i}^{(h)}$ to $(p_{j,i}^{(h)}+s_{j,i}^{+})$ and subtracting $s_{j,i}^{+}∕m_j$ from the $m_j$ significant positions, thus keeping the sum of the values of the $j\text{th}$ simplex block one. We then check whether the updated ${\mathbf{P}}_j^{(h)}$ is in the unit-simplex. If so, we set ${\mathbf{P}}_j^{(h)}(i,+)$ equal to the updated ${\mathbf{P}}_j^{(h)}$, and if not (since it is possible if either $p_{j,i}^{(h)}+s_{j,i}^{+}>1$ or at least one of the updated values at the significant positions is negative), we update the local step-size by setting $s_{j,i}^{+}=s_{j,i}^{+}∕\rho$ and repeat the same step until the updated ${\mathbf{P}}_j^{(h)}$ is in unit-simplex. However, we do not allow $s_{j,i}^{+}$ to be smaller than $\varphi$. In case $s_{j,i}^{+}$ becomes smaller than $\varphi$ (by dividing it multiple times by $\rho$), we set ${\mathbf{P}}_j^{(h)}(i,+)={\mathbf{P}}_j^{(h)}$. The $(j,i,-)\text{th}$ movement is performed in a very similar manner by subtracting $s_{j,i}^{-}$ from $p_{j,i}^{(h)}$ and adding $s_{j,i}^{-}∕m_j$ to the other significant positions, and we refrain from detailing it here.

3.2.4. Sparsity Control

We incorporate a sparsity control step in order to encourage possibly sparse solutions. For each of the obtained modified simplex blocks ${\{{\mathbf{P}}_j^{(h)}(i,+)\}}_{i=1}^{n_j}$ and ${\{{\mathbf{P}}_j^{(h)}(i,-)\}}_{i=1}^{n_j}$ for $j=1,\dots ,B$, we set the values of the “insignificant” coordinates (that is, coordinates less than $\lambda$) to zero. This is adjusted for in the “significant” positions by incrementing each of them by the same amount in order to keep the sum of all the coordinates to be one. After the sparsity control step, if the modified ${\mathbf{P}}_j^{(h)}(i,+)$ (or ${\mathbf{P}}_j^{(h)}(i,-)$) remains in unit-simplex, we denote it by ${\overline{\mathbf{p}}}_j^{(h)}(i,+)$ (or ${\overline{\mathbf{p}}}_j^{(h)}(i,-)$). If not, we set ${\overline{\mathbf{p}}}_j^{(h)}(i,+)={\mathbf{P}}_j^{(h)}(i,+)$ (or ${\overline{\mathbf{p}}}_j^{(h)}(i,-)={\mathbf{P}}_j^{(h)}(i,-)$).

Remark 1. $\lambda$ should be taken relatively large in case of prior knowledge that the final solution is sparse. If not, it can be chosen relatively small or zero.

3.2.5. Selecting the Best Candidate Solution

Corresponding to the modified simplex block ${\overline{\mathbf{p}}}_j^{(h)}(i,+)$, the candidate solution is given by ${\mathbf{P}}^{(h)}(j,i,+)=({\mathbf{P}}_1^{(h)},\dots ,{\mathbf{P}}_{j-1}^{(h)},{\overline{\mathbf{p}}}_j^{(h)}(i,+),{\mathbf{P}}_{j+1}^{(h)},\dots ,{\mathbf{P}}_B^{(h)})$. Similarly, corresponding to the modified simplex block ${\overline{\mathbf{p}}}_j^{(h)}(i,-)$, the candidate solution is given by ${\mathbf{P}}^{(h)}(j,i,+)=({\mathbf{P}}_1^{(h)},\dots ,{\mathbf{P}}_{j-1}^{(h)},{\overline{\mathbf{p}}}_j^{(h)}(i,-),{\mathbf{P}}_{j+1}^{(h)},\dots ,{\mathbf{P}}_B^{(h)})$. Note that ${\{{\overline{\mathbf{p}}}_j^{(h)}(i,+)\}}_{j=1}^{n_j}$ and ${\{{\overline{\mathbf{p}}}_j^{(h)}(i,-)\}}_{j=1}^{n_j}$ belong to the unit-simplex. Thus we obtain $2M$ candidate solution points ${\mathbf{P}}^{(h)}(j,i,+)$ and ${\mathbf{P}}^{(h)}(j,i,-)$ for $j=1,\dots ,B$ and $i=1,\dots ,n_j$. The objective function is evaluated at these $2M$ candidate solutions and the best solution point (i.e., the solution point where the value of the objective function is the lowest) out of the $(2M+1)$ points including current solution ${\mathbf{P}}^{(h)}$ is set as the updated solution ${\mathbf{P}}^{(h+1)}$.

3.2.6. Loop Termination Criteria

As mentioned in Section 3.2.2, at the end of each iteration, the value of $\eta$ either remains the same or gets divided by $\rho$. If $\mid f({\mathbf{P}}^{(h+1)})-f({\mathbf{P}}^{(h)})\mid <{\tau }_1$ at the end of the $(h+1)\text{th}$ iteration, we set $\eta =\eta ∕\rho$, and leave it unchanged otherwise. Moreover, we set the minimum allowable value of $\eta$ to be $\varphi$, and terminate a run if $\eta$ becomes smaller than $\varphi$. For example, consider a situation where $\eta >\varphi$ at the start of the $h\text{th}$ iteration within the $R\text{th}$ run. However, at the end of $h\text{th}$ iteration, $\mid f({\mathbf{P}}^{(h)})-f({\mathbf{P}}^{(h-1)})\mid <{\tau }_1$, and so we set $\eta =\eta ∕\rho$, but it happens to be that $\eta ∕\rho <\varphi$. We then terminate the $R\text{th}$ run at the $h\text{th}$ iteration and return the solution obtained at the end of $h\text{th}$ iteration as the solution of the $R\text{th}$ run, which then serves as the starting point for the $(R+1)\text{th}$ run. Recall that we set $\eta =s_{\text{initial}}$ at the beginning of each run. Suppose ${\widehat{\mathbf{P}}}^{(R)}$ denotes the solution returned by the $R\text{th}$ run. The algorithm terminates when $\mid f({\widehat{\mathbf{P}}}^{(R)})-f({\widehat{\mathbf{P}}}^{(R-1)})\mid <{\tau }_2$ and returns ${\widehat{\mathbf{P}}}^{(R)}$ as the final solution. In Figure 1 we provide a flowchart of the runs executed within MSiCOR. Pseudo-code for MSiCOR is provided in Algorithm 1.

Figure 1.

Flowchart of the MSiCOR: $gs$ (same as $\eta$) denotes the global step-size, phi $(\varphi )$ denotes the step-size threshold, $P^{(h)}$ denotes the solution returned at the end of $h\text{th}$ run, tol_fun_2 (same as ${\tau }_2$) is a cutoff value.

3.3. Comparative Performance Using Benchmark Functions

In order to evaluate the comparative performance of MSiCOR, we consider the minimization problem of four benchmark functions (Jamil and Yang 2013) namely Rastrigin function, Ackley’s function, sphere function and Griewank function whose parameter spaces are modified to be a collection of simplexes (see Section C of the supplementary material for function expressions). The benchmark functions are quite challenging to minimize because of their typical complex structure along with presence of multiple minimums (generally) which pose challenges in finding the global minimum. It is impossible to design an optimization algorithm which can always find the global minimum for any black-box function. Although some global optimization algorithms are backed up with theoretical evidence on global convergence (e.g., Schmitt 2001 on genetic algorithm), however, those convergence properties are derived based on a lot of assumptions regarding the properties of the objective function. First those properties might not hold true in practice. Second there is no way to verify if those properties hold true for a black-box function since the function is already “black-box” by nature. Therefore, to comprehend the performance of any heuristic, global or black-box optimization technique, a common practice is to conduct experiments based on minimizing benchmark functions; and eventually based on average computation time and quality of the obtained solutions, one may get an idea about the relative performance of an algorithm. In this context, along with MSiCOR we also consider Genetic Algorithm (GA), Sequential Quadratic Programming (SQP) and Interior Point (IP) optimization techniques to minimize the aforementioned benchmark functions (see Section C of the supplementary material for details). IP and SQP algorithms search for local minimum while optimizing any function and in general they are less time consuming. On the other hand GA tries to find global minimum being more time consuming. These above-mentioned well-known algorithms are available in Matlab via the Optimization Toolbox functions fmincon (for IP and SQP algorithm) and ga (for GA). While using IP and SQP algorithms, the upper bound for maximum number of iterations and function evaluations have been set to be infinity each. For GA, the default options of “ga” function in Matlab has been considered. MSiCOR is implemented in Matlab. The comparative study has been performed for the cases n = 5, d = 5 and n = 5, d = 10 for all the above-mentioned algorithms, where $n$ is the number of simplexes and $(d-1)$ is the dimension of each simplex blocks. All the computations have been performed in a computer with 64-Bit Windows 8.1, Intel i7 3.6GHz processor, 32GB RAM. For each case, all the algorithms have been initialized from 100 randomly generated starting points. The true minimum of all considered modified benchmark functions is 0. The average computation time (in seconds, also standard deviation in parenthesis), obtained minimum value of the objective functions and standard deviations of obtained objective function values for each cases have been noted down in Table 1. It is observed that MSiCOR provides better solution for considered benchmark functions compared to other methods, except for modified sphere function. The sphere function being convex, it is expected for IP and SQP to perform well while optimizing it. But MSiCOR outperforms GA for all considered benchmark functions. Although MSiCOR took more time on average to converge compared to IP and SQP, it took much less time compared to GA. It is also noted that the standard deviation of the objective function values at the obtained solutions is the least for MSiCOR, which implies that MSiCOR algorithm is more robust and less dependent on starting points compared to other algorithms. To check the performance of the proposed algorithm in higher dimensional problems, additional simulation studies have been performed. In Table 2 we explore the performance of MSiCOR for higher dimensional simplex blocks (n = 20, 50, d = 10, 20).

Table 1.

Comparative study of MSiCOR, IP, SQP, and GA for optimizing modified Rastrigin, Ackley, Sphere, and Griewank function starting from 100 randomly generated initial points.

Functions	Algorithms	n = 5, d = 5			n = 5, d = 10
		min. value	sd of solutions	mean time (sd)	min. value	sd of solutions	mean time (sd)
Modified Rastrigin	MSiCOR	1.39e + 01	8.20	6.83 (2.17)	2.30e + 01	19.59	9.12 (3.26)
	IP	4.01e + 02	276.27	0.12 (0.04)	9.14e + 02	666.64	4.35 (17.09)
	SQP	1.62e + 02	150.51	0.05 (0.01)	8.78e + 02	244.09	0.21 (0.02)
	GA	1.51e + 01	56.72	20.26 (2.26)	2.32e + 02	1157.85	22.08 (4.40)
Modified Ackley’s	MSiCOR	3.48e - 02	1.06	6.50 (2.53)	6.51e - 02	0.23	9.81 (3.62)
	IP	4.86e + 01	6.87	0.13 (0.09)	6.86e + 01	4.50	0.36 (0.07)
	SQP	7.75e + 00	14.62	0.06 (0.01)	2.17e + 01	14.77	0.18 (0.04)
	GA	6.63e + 00	11.94	19.93 (2.60)	1.60e + 01	7.32	24.33 (3.26)
Modified Sphere	MSiCOR	7.42e - 05	<1e-06	5.86 (0.59)	5.13e - 04	5.7e - 06	9.92 (1.37)
	IP	1.63e −15	<1e-06	0.07 (0.03)	8.90e - 16	< 1e - 06	0.12 (0.04)
	SQP	5.60e - 12	< 1e - 06	0.04 (0.00)	2.72e - 11	< 1e - 06	0.17 (0.02)
	GA	5.80e - 01	45.25	20.25 (1.89)	1.34e + 02	633.55	22.76 (5.43)
Modified Griewank	MSiCOR	4.94e - 01	0.40	4.29 (1.42)	1.08e + 00	0.55	7.75 (1.46)
	IP	1.07e + 01	28.54	0.17 (0.05)	2.19e - 01	10.74	0.31 (0.07)
	SQP	1.44e + 01	13.20	0.07 (0.01)	6.97e - 01	0.99	0.28 (0.03)
	GA	1.04e + 01	3.57	18.53 (1.44)	2.73e + 02	12.39	26.67 (3.16)

NOTE: Objective function values at the best (or minimum) solution out of 100 obtained solutions using the aforementioned algorithms are noted down for each case (under “min. value” column); along with standard deviation (SD) of objective function values at the solutions (under “sd of solutions” column). Also mean and s.d. (within parenthesis) of computation times (in seconds) are noted for each case (under “mean time (sd)” column).

Table 2.

Performance of MSiCOR in higher dimensional modified Rastrigin, Ackley, Sphere, and Griewank function starting from 100 randomly generated initial points.

Functions	Measures	n = 20, d = 10	n = 20, d = 20	n = 50, d = 10	n = 50, d = 20
Modified Rastrigin	min. value	1.03e - 02	2.04e - 02	2.58e - 02	5.07e - 02
	sd of solutions	0.41	0.32	0.51	0.39
	mean time (SD)	53.28 (10.05)	80.27 (25.21)	438.23 (78.37)	689.45 (129.72)
Modified Ackley’s	min. value	4.01e - 02	4.01e - 02	1.00e - 01	1.00e - 01
	sd of solutions	0.92	0.56	0.96	1.02
	mean time (SD)	110.94 (11.29)	150.83 (14.85)	611.41 (49.88)	1348.45 (131.55)
Modified Sphere	min. value	5.24e - 05	1.02e - 04	1.30e - 04	2.59e - 04
	sd of solutions	<0.01	<0.01	<0.01	<0.01
	mean time (SD)	41.99 (2.04)	87.83 (5.72)	308.74 (10.65)	577.34 (17.90)
Modified Griewank	min. value	1.04e - 01	1.27e - 01	9.25e - 02	2.43e - 01
	sd of solutions	0.58	0.28	1.15	0.79
	mean time (SD)	58.48 (2.98)	201.70 (10.59)	741.82 (20.01)	3291.46 (210.35)

NOTE: Objective function values at the best (or minimum) solution out of 100 obtained solutions using MSiCOR are noted down for each case (“min. value” row); along with standard deviation (SD) of objective function values at the solutions (“sd of solutions” row). Also mean and SD (within parenthesis) of computation times (in seconds) are noted for each case (“mean time (SD)” row).

Algorithm 1 MSiCOR

$$\begin{array}{cc}\hfill & 1:R\leftarrow 1.\hfill \\ \hfill & 2:top:\hfill \\ \hfill & 3:h\leftarrow 1\text{and}{\eta }^{(0)},{\eta }^{(1)}\leftarrow s_{\text{initial}}.\hfill \\ \hfill & 4:\mathbf{if}R=1\mathbf{then}\hfill \\ \hfill & 5:{\mathbf{P}}^{(0)}\leftarrow \text{Initial guess}(\text{Cell}∕\text{list of simplexes of dimensions}{n}_1,\dots ,n_B,\text{respectively}).\hfill \\ \hfill & 6:\mathbf{else}\hfill \\ \hfill & 7:{\mathbf{P}}^{(0)}\leftarrow {\widehat{\mathbf{P}}}^{(R-1)}.\hfill \\ \hfill & 8:\mathbf{while}(h\le M_{\text{iter}}\text{and}{\eta }^{(h)}>\varphi )\mathbf{do}\hfill \\ \hfill & 9:F_1\leftarrow f({\mathbf{P}}^{(h-1)})\text{and}\eta \leftarrow {\eta }^{(h-1)}.\hfill \\ \hfill & 10:\mathbf{for}j=1,\dots ,B\mathbf{do}\hfill \\ \hfill & 11:\mathbf{for}v=1,\dots ,2n_j\mathbf{do}\hfill \\ \hfill & 12:i\leftarrow [(v+1)∕2].\hfill \\ \hfill & 13:{\mathbf{q}}_{j,v}\leftarrow {\mathbf{P}}_j^{(h-1)}\text{and}{\mathbf{q}}_v^{\text{temp}}\leftarrow {\mathbf{P}}_j^{(h-1)}.\hfill \\ \hfill & 14:\Lambda \leftarrow \text{which}(p_{j,l}^{(h-1)}<\lambda ),l\in \{1,\dots ,n_j\}\setminus \{i\}.\hfill \\ \hfill & 15:\Gamma \leftarrow \text{which}(p_{j,l}^{(h-1)}\ge \lambda ),l\in \{1,\dots ,n_j\}\setminus \{i\}.\hfill \\ \hfill & 16:s_{j,v}\leftarrow {(-1)}^v{\eta }^{(h)}.\hfill \\ \hfill & 17:\mathbf{if}(n(\Gamma )>0)\mathbf{then}\hfill \\ \hfill & 18:garbage\leftarrow \Sigma ({\mathbf{q}}_v^{\text{temp}}(\Lambda )).\hfill \\ \hfill & 19:{\mathbf{q}}_{j,v}(i)\leftarrow {\mathbf{q}}_v^{\text{temp}}(i)+s_{j,v}\text{and}{\mathbf{q}}_{j,v}(\Gamma )\leftarrow {\mathbf{q}}_v^{\text{temp}}(i)-s_{j,v}∕n(\Gamma )+garbage∕n(\Gamma )\text{and}{\mathbf{q}}_{j,v}(\Lambda )\leftarrow 0.\hfill \\ \hfill & 20:\mathbf{while}({\mathbf{q}}_{j,v}\notin {\Delta }^{n_j-1}\text{and}\mid s_{j,v}\mid >\varphi )\mathbf{do}\hfill \\ \hfill & 21:s_{j,v}\leftarrow s_{j,v}∕\rho .\hfill \\ \hfill & 22:{\mathbf{q}}_{j,v}(i)\leftarrow {\mathbf{q}}_v^{\text{temp}}(i)+s_{j,v}\text{and}{\mathbf{q}}_{j,v}(\Gamma )\leftarrow {\mathbf{q}}_v^{\text{temp}}(i)-s_{j,v}∕n(\Gamma )+garbage∕n(\Gamma ).\hfill \\ \hfill & 23:\mathbf{else}\hfill \\ \hfill & 24:{\mathbf{q}}_{j,v}(i)\leftarrow 1\text{and}{\mathbf{q}}_{j,v}(\Lambda )\leftarrow 0.\hfill \\ \hfill & 25:\mathbf{if}({\mathbf{q}}_{j,v}\in {\Delta }^{n_j-1})\mathbf{then}{f}_{j,v}\leftarrow f({\mathbf{P}}_1^{(h-1)},\dots ,{\mathbf{P}}_{i-1}^{(h-1)},{\mathbf{q}}_{j,v},{\mathbf{P}}_{i+1}^{(h-1)},\dots ,{\mathbf{P}}_{n_j}^{(h-1)})\mathbf{else}{f}_{j,v}\leftarrow F_1.\hfill \\ \hfill & 26:(j_{\text{best}},v_{\text{best}})\leftarrow {\text{arg min}}_{j,v}{f}_{j,v},\text{over}j=1,\dots ,B,v=1,\dots ,n_j.\hfill \\ \hfill & 27:F_2\leftarrow f_{j_{\text{best}},v_{\text{best}}}.\hfill \\ \hfill & 28:{\mathbf{P}}^{(h)}\leftarrow {\mathbf{P}}^{(h-1)}.\hfill \\ \hfill & 29:\mathbf{if}(F_2<F_1)\mathbf{then}{\mathbf{P}}_{j_{\text{best}}}^{(h)}\leftarrow {\mathbf{q}}_{j_{\text{best}},v_{\text{best}}}.\hfill \\ \hfill & 30:\mathbf{if}(h>1)\mathbf{then}\hfill \\ \hfill & 31:\mathbf{if}(\mid F_1-F_2\mid <{\tau }_1\text{and}\eta >\varphi )\mathbf{then}\eta \leftarrow \eta ∕\rho .\hfill \\ \hfill & 32:{\eta }^{(h)}\leftarrow \eta \text{and}h\leftarrow h+1.\hfill \\ \hfill & 33:{\widehat{\mathbf{P}}}^{(R)}\leftarrow {\mathbf{P}}^{(h)}.\hfill \\ \hfill & 34:\mathbf{if}\mid f({\widehat{\mathbf{P}}}^{(R)})-f({\widehat{\mathbf{P}}}^{(R-1)})\mid <{\tau }_2\mathbf{then}\hfill \\ \hfill & 35:\mathbf{return}\widehat{\mathbf{P}}={\widehat{\mathbf{P}}}^{(R)}\text{as}\mathbf{final}\mathbf{solution}.\hfill \\ \hfill & 36:\mathbf{exit}\hfill \\ \hfill & 37:\mathbf{else}\hfill \\ \hfill & 38:R\leftarrow R+1.\hfill \\ \hfill & 39:\mathbf{goto}top.\hfill \end{array}$$

3.4. Theoretical Properties

The greatest challenge of solving a non-convex optimization problem is that in general algorithms cannot be designed to guarantee reaching the global optima. However, it is a desirable property of any algorithm that it should reach a global minimum when the function is convex. In this section, via Theorem 1, it is shown that taking the values of the parameters $\varphi$, ${\tau }_1$ and ${\tau }_2$ significantly small, the stopping criteria of the proposed algorithm ensures that the solution obtained is a global minimum in case the objective function is convex. The detailed proof of Theorem 1 is provided in Section B of the Supplementary material.

Theorem 1. Suppose $\mathbf{s}={\Delta }^{n_1-1}\times \cdots \times {\Delta }^{n_B-1}$ and $f$ is convex, continuous and differentiable on $\mathbf{s}$. Suppose $\mathbf{u}=({\mathbf{u}}_1,\dots ,{\mathbf{u}}_B)\in \mathbf{s}$ and ${\mathbf{u}}_j=(u_{j,1},\dots ,u_{j,n_j})\in {\Delta }^{n_j-1}$ for $j=1,\dots ,B$, and each of its coordinates are nonzero. Consider a sequence ${\delta }_{j,k}=s_j∕{\rho }^k$ for $k\in \mathbb{N}$, $s_j>0$, $\rho >1$ for all $j=1,\dots ,B$. Define ${\mathbf{u}}_{j,k}^{(i+)}=(u_{j,1}-{\delta }_{j,k}∕(n-1),\dots ,u_{j,i-1}-{\delta }_{j,k}∕(n-1),u_{j,i}+{\delta }_{j,k},u_{j,i+1}-{\delta }_{j,k}∕(n-1),\dots ,u_{j,n_j}-{\delta }_{j,k}∕(n-1))$ and ${\mathbf{u}}_{j,k}^{(i-)}=(u_{j,1}+{\delta }_{j,k}∕(n-1),\dots ,u_{j,i-1}+{\delta }_{j,k}∕(n-1),u_{j,i}-{\delta }_{j,k},u_{j,i+1}+{\delta }_{j,k}∕(n-1),\dots ,u_{j,n_j}+{\delta }_{j,k}∕(n-1))$ for $j=1,\dots ,B$ and $i=1,\dots ,n_j$. If for all $k\in \mathbb{N}$, $f(\mathbf{u})\le f({\mathbf{u}}_1,\dots ,{\mathbf{u}}_{j-1},{\mathbf{u}}_{j,k}^{(i+)},{\mathbf{u}}_{j+1},\dots ,{\mathbf{u}}_B)$ and $f(\mathbf{u})\le f({\mathbf{u}}_1,\dots ,{\mathbf{u}}_{j-1},{\mathbf{u}}_{j,k}^{(i-)},{\mathbf{u}}_{j+1},\dots ,{\mathbf{u}}_B)$ (whenever ${\mathbf{u}}_{j,k}^{(i+)},{\mathbf{u}}_{j,k}^{(i-)}\in {\Delta }^{n_k-1}$) for $j=1,\dots ,B$ and $i=1,\dots ,n_j$, $\mathbf{u}$ is a point of global minimum of $f$.

It should be noted that taking step-size threshold $\varphi$ small enough, the allowable values of local step-sizes $s_{j,i}^{+}$ and $s_{j,i}^{-}$ can be taken as close to zero as required. Also note that in Theorem 1, the role of ${\delta }_{j,k}$ is analogous to that of $s_{j,i}^{+}$ and $s_{j,i}^{-}$ in Section 3.2. In other words, in the proposed algorithm if we take ${\tau }_1=0$ and $M_{\text{iter}}=\infty$, the iterations within a run stops when for very small value of $s_{j,i}^{+}$ and $s_{j,i}^{-}$ for $j=1,\dots ,B$ and $i=1,\dots ,n_j$, corresponding movements in the neighborhood do not yield better solution than the current solution. Hence, it that scenario, the obtained solution by the proposed algorithm is a global minimum if the objective function follows the regularity conditions considered in Theorem 1. Note that, for a convex function satisfying the regularity conditions, the convergence criteria ensures that at the end of any run the solution obtained is a global minimum. Hence, in this case, evaluation of only one run will be enough to find the global minimum.

4. Simulation Study

4.1. Setup

We consider $K=1000$ patients belonging to $L=3$ clusters. The number of states is chosen to be n = 10. For each cluster, we simulate rows of each 10 × 10 transition matrix $M_l$ from an uniform Dirichlet distribution. Initial state distribution vectors corresponding to clusters $s_1,\dots ,s_L$ are also generated from an uniform Dirichlet distribution. We choose $p=4$ covariates (including an intercept term), which are randomly generated from a $\mathbf{N}(0,I_4)$ distribution. For each patient, we generate the length of their observed Markov chain (i.e., $h_i$ for the $i\text{th}$ patient) from a discrete uniform distribution on $\{6,\dots ,12\}$. The length of the chain of the same subject may vary across different simulations. To generate the observations, we first compute the probabilities of each patient belonging to the different clusters $w_{kl}$ using (4). Then, using multinomial draws corresponding to prior probabilities, cluster memberships of each patient is evaluated. For each patient, based on corresponding cluster membership, initial state is generated using multinomial draws from initial state distribution vector. The following states of the Markov chain are generated using multinomial draws from corresponding cluster-specific transition matrix.

4.2. Parameter Estimation

In order to estimate the parameters, we first look for a warm starting point for cluster-specific initial state distribution vectors and transition matrices. We first maximize $L_{\mathbf{M},\mathbf{s},\Gamma }(Y_1,\dots ,Y_K)$ given by (4) taking $w_{kl}=\frac{1}{L}$, (i.e., equal weight for each cluster) using MSiCOR and obtain initial estimates for $\mathbf{M}$ and $\mathbf{s}$. Then for given initial values of $\mathbf{M}$ and $\mathbf{s}$, we maximize $L_{\mathbf{M},\mathbf{s},\Gamma }(Y_1,\dots ,Y_K)$ given by (4) and thus we obtain an initial estimate for $\Gamma$. For maximizing the likelihood as a function of $\Gamma$ for given ($\mathbf{M}$, $\mathbf{s}$), we use patternsearch function in MATLAB which performs a global maximization of the likelihood as a function of $\Gamma$. Once we obtain the initial estimates for ($\mathbf{M}$, $\mathbf{s}$, $\Gamma$), we then maximize the likelihood updating ($\mathbf{M}$, $\mathbf{s}$, $\Gamma$) simultaneously within each iteration of the proposed algorithm. Note that the parameters $\mathbf{M}$, $\mathbf{s}$ are a collection of simplexes and hence can be directly estimated using MSiCOR, however, $\Gamma$ being unconstrained, we adopt another optimization technique for updating $\Gamma$ at each iteration within MSiCOR. Das (2023) proposed global optimization technique Recursive Modified Pattern Search (RMPS) for optimizing hyper-rectangular parameter space. RMPS can be easily modified for optimizing unconstrained parameter space (see Algorithm 1 in the supplementary file). Using MSiCOR and modified RMPS for unconstrained optimization, at each iteration, we update ($\mathbf{M}$, $\mathbf{s}$, $\Gamma$) simultaneously This joint algorithm is provided in Algorithm 2. Here in the algorithm, ($\mathbf{M}$, $\mathbf{s}$) being collection of simplexes, we denote it by $\mathbf{P}$ and unconstrained parameter $\Gamma$ is denoted by $\mathbf{l}$.

Algorithm 2 Algorithm for jointly updating multiple simplexes (using MSiCOR) and unconstrained parameter vector (using RMPS).

$$\begin{array}{cc}\hfill & 1:R\leftarrow 1\hfill \\ \hfill & 2:top:\hfill \\ \hfill & 3:h\leftarrow 1\hfill \\ \hfill & 4:{\eta }^{(0)},{\eta }^{(1)}\leftarrow s_{\text{initial}}\hfill \\ \hfill & 5:\mathbf{if}R=1\mathbf{then}\hfill \\ \hfill & 6:{\mathbf{P}}^{(0)}\leftarrow \text{Initial guess}(\text{Cell}∕\text{list of simplexes of dimensions}{n}_1,n_2,\dots ,n_B\text{respectively})\hfill \\ \hfill & 7:{\mathbf{l}}^{(0)}\leftarrow \text{Initial guess (unconstrained parameter vector)}\hfill \\ \hfill & 8:\mathbf{else}\hfill \\ \hfill & 9:{\mathbf{P}}^{(0)}\leftarrow {\widehat{\mathbf{P}}}^{(R-1)}\hfill \\ \hfill & 10:{\mathbf{l}}^{(0)}\leftarrow {\widehat{\mathbf{l}}}^{(R-1)}\hfill \\ \hfill & 11:\mathbf{while}(h\le M_{\text{iter}}\text{and}{\eta }^{(h)}>\varphi )\mathbf{do}\hfill \\ \hfill & 12:F\leftarrow f({\mathbf{P}}^{(h-1)})\hfill \\ \hfill & 13:\eta \leftarrow {\eta }^{(h-1)}\hfill \\ \hfill & 14:\text{Find}{f}_{j_{\text{best}},v_{\text{best}}}\text{and}\text{set}{F}_1\leftarrow f_{j_{\text{best}},v_{\text{best}}}\text{using steps}12-36\text{of Algorithm}1.\hfill \\ \hfill & 15:\text{Find}{g}_{k_{\text{best}}}\text{and}\text{set}{F}_2\leftarrow g_{k_{\text{best}}}\text{using steps}12-19\text{of Algorithm 1 of the supplementary file}.\hfill \\ \hfill & 16:{\mathbf{P}}^{(h)}\leftarrow {\mathbf{P}}^{(h-1)}\hfill \\ \hfill & 17:{\mathbf{l}}^{(h)}\leftarrow {\mathbf{l}}^{(h-1)}\hfill \\ \hfill & 18:F^{\prime }\leftarrow \text{min}(F_1,F_2)\hfill \\ \hfill & 19:\mathbf{if}(F^{\prime }<F)\mathbf{then}\hfill \\ \hfill & 20:\mathbf{if}(F_1<F_2)\mathbf{then}\hfill \\ \hfill & 21:{\mathbf{P}}_{j_{\text{best}}}^{(h)}\leftarrow {\mathbf{q}}_{j_{\text{best}},v_{\text{best}}}\hfill \\ \hfill & 22:\mathbf{else}\hfill \\ \hfill & 23:{\mathbf{l}}^{(h)}\leftarrow {\mathbf{l}}_{k_{\text{best}}}\hfill \\ \hfill & 24:\mathbf{if}(h>1)\mathbf{then}\hfill \\ \hfill & 25:\mathbf{if}(\mid F-\text{min}(F,F^{\prime })\mid <tol\_fun\text{and}\eta >\varphi )\mathbf{then}\hfill \\ \hfill & 26:\eta \leftarrow \eta ∕\rho \hfill \\ \hfill & 27:{\eta }^{(h)}\leftarrow \eta \hfill \\ \hfill & 28:h\leftarrow h+1\hfill \\ \hfill & 29:({\widehat{\mathbf{P}}}^{(R)},{\widehat{\mathbf{l}}}^{(R)})\leftarrow ({\mathbf{P}}^{(h)},{\mathbf{l}}^{(h)}),\hfill \\ \hfill & 30:\mathbf{if}\mid f({\widehat{\mathbf{P}}}^{(R)},{\widehat{\mathbf{l}}}^{(R)})-f({\widehat{\mathbf{P}}}^{(R-1)},{\widehat{\mathbf{l}}}^{(R-1)})\mid <{\tau }_2\mathbf{then}\hfill \\ \hfill & 31:\mathbf{return}\widehat{\mathbf{P}}={\widehat{\mathbf{P}}}^{(R)},\widehat{\mathbf{l}}={\widehat{\mathbf{l}}}^{(R)}\text{as}\mathbf{final}\mathbf{solution}\hfill \\ \hfill & 32:\mathbf{exit}\hfill \\ \hfill & 33:\mathbf{else}\hfill \\ \hfill & 34:R\leftarrow R+1\hfill \\ \hfill & 35:\mathbf{goto}top.\hfill \end{array}$$

4.3. Mapping True Clusters to Estimated Clusters

One of the main concerns regarding parameter estimation in mixture models or mixture of experts (MoE) (Masoudnia and Ebrahimpour 2014) is to ensure identifiability. MSiCOR (for MMM without covariates) and MSiCOR-RMPS hybrid algorithm (for MMM with covariates) are both designed to find the point of maxima of the mixture likelihood; subsequently we obtain the estimated parameter values corresponding to each cluster. In case we perform MMM clustering to any new dataset, we can divide the sample population in desired number of clusters and identifiability might not be a concern in absence of reference clusters. However, to assess the estimation performance based on simulation study, it is crucial to map the estimated clusters to the true reference clusters. For any given cluster, let us denote the initial state distribution vector and the transition matrices by $s_{n\times 1}$ and $M_{n\times n}$, respectively. From $s$ and $M$, we construct the appended matrix $A={[s;M]}_{(n+1)\times n}$. Given two appended matrices $B$ and $C$, we define the total variation distance $(\text{TVD})$ as

$$\mathbf{TVD}(B,C)=\sum _{i=1}^{n+1}\sum _{j=1}^n\mid B(i,j)-C(i,j)\mid .$$

Suppose we denote the true appended matrices (constructed using the initial state distribution vector and the transition matrices) by $A_1$, $A_2,\dots ,A_L$ and the estimated appended matrices by $B_1,\dots ,B_L$. Then we find the permutation $\sigma$ of $\{1,\dots ,L\}$ for which ${\sum }_{l=1}^L\mathbf{TV}(A_l,B_{\sigma (l)})$ is minimized. Thus corresponding permutation helps us identifying the mapping across the true clusters to the estimated clusters.

4.4. Results

After we obtain the estimates of ($\mathbf{M}$, $\mathbf{s}$, $\Gamma$), using (5), we find the membership probabilities of each subject to three estimated clusters. We consider each subject to belong to the cluster with corresponding highest membership probabilities. To compare the true and the estimated cluster membership of the patients, first we map three true clusters to three estimated clusters. Then we calculate the true positive rate (TPR) of membership of the subjects. In order to calculate the standard error of the parameter estimates, we perform 20 simulation iterations. The true and estimated parameter values are provided in Table 3. It is noted that the true and the estimated cluster covariate coefficients are close.

Table 3.

True and estimated coefficients of the covariates in the simulation study.

Clusters		Intercept	Var 1	Var 2	Var 3
Cluster 2	True	−0.75	0.64	−0.19	−0.13
	Est.	−0.738 (0.0212)	0.614 (0.0229)	−0.185 (0.0239)	−0.149 (0.0162)
Cluster 3	True	−0.07	1.12	−0.25	−0.4
	Est.	−0.086 (0.0199)	1.123 (0.0192)	−0.270 (0.0210)	−0.414 (0.0213)

NOTE: Standard errors are provided in the parenthesis.

As mentioned earlier, based on posterior membership probabilities from estimated parameters, the membership of the subjects can also be estimated. In Table 4 we note down the true and estimated proportions of subjects belonging to each cluster.

Table 4.

True and estimated cluster proportions (empirical standard errors shown in parentheses).

	Cluster 1	Cluster 2	Cluster 3
True proportion	0.422	0.174	0.404
Estimated proportion	0.422 (3.4 × 10−3)	0.178 (3.2 × 10−3)	0.400 (3.9 × 10−3)

4.5. Comparison with the EM Algorithm

In order to compare the performance of MSiCOR with the EM algorithm in the MMM (without covariates), we consider another simulation study. Here again we consider $K=1000$ patients where each patient belongs to one of the $L=3$ clusters. The dimension of the state-space is taken to be n = 10. Corresponding to each cluster, initial state distribution vectors are generated from uniform Dirichlet distribution. The transition matrices corresponding to the first, second and the third clusters are taken to be 30%, 50%, and 70% sparse respectively. The nonzero elements of each row of the transition matrices corresponding to each cluster are generated from uniform Dirichlet distribution. For each patient, the length of their observed Markov chain (i.e., $h_i$ for the $i\text{th}$ patient) is chosen following a discrete uniform distribution on $\{6,\dots ,12\}$. Then based on the obtained true transition matrices and the initial state probability vectors, the patient-specific state-space sequences are generated. To fit EM, we use the fit_model function in tseqHMM R package (Helske and Helske 2019). The simulation experiment is repeated 50 times and corresponding TVD is measured for both MSiCOR and EM methods. Using TVD, estimated clusters are mapped to the true clusters. After mapping, the misclassification rate (MR) is calculated. In Table 5 it is observed that MSiCOR performs better than the EM algorithm in terms of TVD and MR measures. It is noted that MSiCOR performs better than the EM algorithm yielding lower TVD and MR compared to the EM algorithm. It is also noted that unlike MSiCOR, the EM algorithm seems not to converge on a few occasions yielding very high TPR and MR values (Figure 2).

Table 5.

Comparison of MSiCOR and EM based on total variation distance (TVD) and misclassification rate (MR) based on 50 simulation experiments; mean TVD and TPR values are noted down in the table.

Sample size	Method	TVD	Max TVD	MR	Max MR
n = 500	MSiCOR	4.57 (0.13)	5.27	0.000 (0.000)	0.004
	EM	5.41 (0.66)	26.42	0.019 (0.013)	0.474
n = 1000	MSiCOR	3.18 (0.04)	3.63	0.000 (0.000)	0.003
	EM	3.66 (0.48)	26.34	0.010 (0.010)	0.434

NOTE: Standard errors are provided in parentheses.

Figure 2.

Boxplot of log of total variation distance (TVD) for MSiCOR and EM based on 50 simulation experiments with sample size 500.

5. Application to Clustering of Medication-Sequence Data of Multiple Sclerosis Patients in EHR Cohort

As a test case, we use MS disease-modifying therapy (DMT) sequence data from an electronic health record (EHR) cohort based at the Massachusetts General and Brigham hospital system (Boston, US) which includes the Comprehensive Longitudinal Investigation of Multiple Sclerosis at Brigham and Women’s Hospital (CLIMB) cohort (Xia et al. 2013; Zhang et al. 2020; Ahuja et al. 2021; Hou et al. 2021; Liang et al. 2022). The EHR cohort contains patient-level data, including DMTs as well as a number of clinical and demographic variables. In this dataset, there are twelve available DMTs for MS patients: alemtuzumab, cyclophosphamide, daclizumab, dimethyl fumarate, fingolimod, glatiramer acetate, interferon-beta, mitoxantrone, natalizumab, ocrelizumab, rituximab and teriflunomide. Of these, daclizumab has been withdrawn from the market and only a few patients received this DMT. Therefore, we exclude daclizumab from our analysis by omitting the corresponding encounters from the MS DMT sequences. We combine rituximab and ocrelizumab under the same mechanistic category (that is, B-cell Depletion). We thus have ten DMT categories in total, which form the state-space of our Markov model. Further, we only consider patients who started on MS DMTs on or after January 1, 2006, because the Mass General Brigham system began implementation of the electronic prescriptions during 2005.

To avoid over-counting given that consecutive visits that are only a few days apart can sometimes list the same DMT prescription, we combine the MS DMT observations into three-month period clusters, starting from DMT start date. Within any three-month period, the consecutive same DMTs are counted as one observation. If a patient takes one DMT during a given three-month period, we count it as one observation representing the given three-month period. For example, during any 3 month period, if the encounters are $A\to A\to A\to A\to A\to A\to A$ or $A\to A\to A\to A$ or $A$ (as long as all encounters in that three-month period are the same DMT $A$), we take the observation as only $A$ for that 3 month period. On the other hand, consider a scenario where a patient has been on DMT sequence $A\to A\to A\to A\to B\to B\to B\to A\to A\to A\to C\to C$ during a three-month period. By including the unique consecutive DMTs into one observation, we then get $A\to B\to A\to C$ as the representative observation for that three-month period.

In the EHR cohort, for clustering, we only consider the patients for which clinical and demographic data are available along with MS DMT sequnce data; also those patients must start on MS DMT on or after 2006. After applying the aforementioned filters, we finally cluster 822 patients. In the mixture Markov model analysis, we consider the following covariates that are routine in MS research: age at diagnosis, disease duration, gender, race (white, black, and others). The disease duration is the time elapsed from the year of first neurological symptom to the DMT start year. The parameters are estimated using the joint algorithm (using MSiCOR and modified RMPS for unconstrained optimization) as described in Section 4.2.

To identify the number of clusters, we fit the MMM with covariates for number of clusters $3,\dots ,6$. Using Bayesian Information Criterion (BIC) (Schwarz 1978), we identify the optimal number of clusters to be 3 (Table 6). After we cluster the 822 patients into 3 DMT sequence clusters, we identify the corresponding cluster for any patient by calculating the posterior cluster membership probabilities given by (5). Once the cluster membership probabilities are calculated for a patient, we assign that patient to the cluster corresponding to the highest membership probability. Among the 822 patients, 445 patients belong to the first cluster, 161 patients belong to the second cluster and the 216 patients belong to the third cluster. In Figure 3, we plot the MS DMT sequences of the patients corresponding to each cluster, up to the first 10 DMTs that they have received in their disease course. In case a patient has received fewer than 10 DMTs, the rest of the slots are kept blank. In the first cluster, most patients have been on interferon-beta, while a small number of patients have been on rituximab-ocrelizumab (B-cell Depletion), fingolimod and dimethyl fumarate. In the second cluster, patient-specific lengths of DMT sequences are smaller than that of the first and the third cluster. In this cluster, patients are mostly treated with glatiramer acetate, interferon-beta, teriflunomide, cyclophosphamide and dimethyl fumarate. In the third cluster, the majority of the patients have been on natalizumab. In summary, patients in the first and the third cluster received predominantly interferon-beta and natalizumab, respectively, whereas patients in cluster 2 received multiple DMTs without any predominant DMT. We also estimate the cluster-specific coefficient values corresponding to the clinical and demographic covariates considered which is provided in Table 7.

Table 6.

Summary of Bayesian Information Criterion (BIC) values after fitting mixture Markov model with covariates for number of clusters $L=3,\dots ,6$.

Number of clusters	BIC
3	13072
4	13556
5	14237
6	15065

Figure 3.

Upto first 10 MS DMTs are plotted for the patients from each cluster. B-cell Depletion DMT group consists of Rituximab and Ocrelizumab.

Table 7.

Regression coefficient for clinical and demographical variables for cluster 2 and 3 taking cluster 1 as the reference cluster.

Clusters	Intercept	Age at diagnosis	Disease duration	Gender (female = 1)	Race White	Race Black
Cluster 2	−2.48 (0.04)	0.04 (0.02)	−0.0 (0.01)	0.29 (0.06)	0.34 (0.05)	−0.09 (0.05)
Cluster 3	−0.17 (0.06)	−0.02 (0.01)	−0.04 (0.03)	0.40 (0.11)	0.75 (0.09)	1.11 (0.08)

NOTE: Coefficients are estimated within proposed mixture Markov model with covariates analysis. Standard errors (calculated based on 50 bootstrap samples), is provided in the parenthesis.

Among the 822 patients, a subset of 488 patients belong to the CLIMB cohort. We compare clinical characteristics across the 3 clusters for the patients belonging to the CLIMB cohort which is noted down in Table 8. CLIMB patients have more detailed clinical information. Among CLIMB cohort patients, Cluster 1, 2, and 3 account for 56%, 20%, and 24% of the patients, respectively. When comparing demographics, Cluster 2 patients from CLIMB cohort (pfCc) have the oldest mean age at diagnosis. Cluster 1 pfCc have the lowest whereas Cluster 3 pfCc have the highest proportion of women. When comparing MS outcomes during patient follow-up, Cluster 3 pfCc have the highest annualized relapse rate or ARR (registry data) as well as the highest annualized counts of MS ICD codes, MS-related MRI CPT codes, and MS-relevant CUIs (e.g., “multiple sclerosis.”, “physical therapy”). Cluster 1 and 2 pfCc share similar mean ARR as well as the mean annualized counts of MS ICD codes and MS-related MRI CPT codes. When assessing the yearly incident relapse rate over time, all three clusters decline over time which is consistent with our prior finding (Liang et al. 2022), but Cluster 3 pfCc show consistently higher yearly relapse rate than the other two clusters (Figure 4). Cluster 3 pfCc also have the highest mean annualized counts of total ICD codes and total CPT codes, suggesting the highest healthcare utilization and comorbidity burden. Cluster 2 pfCc contains larger proportion of DMT “cyclers” (who experienced high frequency of switches to different DMTs) than Cluster 1 and 3 pfCc. For example, patients in Cluster 2 switched off standard-efficacy DMTs (for example, interferon-beta, glatiramer acetate), while patients in Cluster 3 switched to natalizumab, a higher-efficacy DMT which was approved in 2004. Cluster membership according to DMT prescription sequences correlate with key clinical outcomes such as yearly relapse rate ( Figure 4).

Table 8.

Characteristics of patient clusters based on DMT prescription sequence: % or Mean (SD).

Feature	Cluster 1	Cluster 2	Cluster 3
Number of patients	271	100	117
Age at diagnosis (year)	36.5 (10.1)	41.9 (10.7)	34.1 (9.5)
Duration from diagnosis to 1st DMT (year)	0.9 (2.4)	1.4 (2.9)	1.5 (3.1)
Race-White	90%	94%	93%
Race-Black	4%	3%	7%
Race-Other	6%	3%	0%
Gender-Male	32%	27%	19%
Gender-Female	68%	73%	81%
Annualized relapse rate	0.16 (0.2)	0.18 (0.3)	0.33 (0.5)
ICD : Total (count/year)	19.2 (20.0)	26.2 (25.6)	40.3 (42.2)
ICD : MS (count/year)	11.8 (7.9)	13.9 (10.4)	24.2 (16.1)
CPT : Total (count/year)	20.0 (16.0)	25.5 (23.4)	36.2 (41.1)
CPT : MS MRI (count/year)	3.7 (1.9)	3.4 (2.0)	4.6 (2.3)
CPT : ED visit (count/year)	0.7 (0.5)	0.6 (0.6)	0.8 (0.7)
CUI : Communicable disease (count/year)	1.3 (2.5)	1.7 (2.4)	4.2 (4.1)
CUI : Double vision (count/year)	0.6 (0.8)	0.7 (1.0)	0.6 (1.0)
CUI : Gadolinium (count/year)	1.8 (1.0)	1.9 (1.1)	2.5 (1.5)
CUI : MRI (count/year)	5.6 (2.3)	5.9 (3.3)	7.4 (3.8)
CUI : Methylprednisolone (count/year)	1.3 (3.0)	1.6 (3.0)	2.6 (5.1)
CUI : Multiple sclerosis (count/year)	7.6 (5.4)	9.4 (7.5)	15.0 (12.0)
CUI : Sensation loss (count/year)	6.1 (3.5)	6.7 (5.0)	8.5 (6.3)
CUI : Nystagmus (count/year)	1.0 (0.8)	1.4 (1.1)	1.3 (1.1)
CUI : Optic neuritis (count/year)	0.4 (1.1)	0.6 (1.6)	0.6 (1.6)
CUI : Sense of pain (count/year)	3.3 (4.1)	4.7 (5.3)	7.5 (9.3)
CUI : PET scan (count/year)	3.0 (3.5)	4.7 (5.5)	7.9 (10.2)
CUI : Steroid (count/year)	1.4 (1.5)	1.6 (1.9)	2.3 (2.9)
CUI : Recurrent disease (count/year)	0.7 (0.8)	0.9 (1.1)	1.3 (1.9)
CUI : Physical therapy (count/year)	3.1 (3.6)	4.8 (5.4)	8.1 (10.8)
CUI : Has difficulty doing (count/year)	1.9 (1.5)	2.4 (1.9)	3.4 (4.6)
CUI : Migraine disorders (count/year)	0.8 (0.8)	0.9 (0.8)	1.3 (1.6)
CUI : Flare (count/year)	0.3 (0.6)	0.4 (0.7)	0.7 (1.6)
CUI : Tingling sensation (count/year)	0.6 (0.9)	0.8 (1.0)	0.9 (1.3)

Figure 4.

Yearly incident relapse rate over time for patients from CLIMB cohort belonging to all three clusters.

6. Conclusion

In the context of estimating the parameters in the proposed model, we propose a novel Blackbox optimization technique MSiCOR based on Pattern Search (PS) for optimizing any function over the collection of unit-simplexes, where the simplexes can be of different dimensions. We also show that for convex functions, under some regularity conditions, MSiCOR converges to the global optimum point. Based on comparative study using several benchmark functions, MSiCOR is shown to outperform Genetic Algorithm (GA), Sequential Quadratic Programming (SQP) and Interior Point (IP) in terms of performance, in general. To maximize the likelihood of the proposed MMM with covariates, along with MSiCOR, we further use modified Recursive Modified Pattern Search (RMPS) for unconstrained optimization (provided in Section D of the supplementary material). Instead of updating the simplex constrained parameters and unconstrained parameters alternatively, we update all those parameters simultaneously using a joint algorithm and the iterative update steps of MSiCOR and updated RMPS for unconstrained optimization.

Further, we proposed a novel MMM with covariates technique to cluster subjects based on their state sequence data along with clinically relevant patient-specific covariates. Using EHR data on treatment prescription sequence, the proposed method can cluster patients into clinically meaningful subgroups. The proposed model is useful for identifying the differential treatment sequences and trajectories in the patient population. Further, it informs how patient-specific covariates influence the treatment sequence and trajectory. Once the clusters are identified, membership probabilities for different clusters could be computed for future patients based on the estimated parameters.

The simulation study shows that the estimated coefficients of the covariates for different clusters in the proposed MMM with covariates are close to the true values. Further for general MMM (without covariates), using MSiCOR, we obtain better results when compared to the EM algorithm.

As a critical test case, we deploy the proposed method to cluster and analyze MS patients with available DMT and clinical and demographic covariates as a part of a well-characterized research cohort study. We identify 3 clusters of MS patients based on membership on the basis of DMT prescription sequence, where the first and the third cluster are enriched for interferon-beta and natalizumab, respectively, while the patients in the second cluster received multiple DMTs without a single predominant DMTs. Patients in different DMT sequence clusters exhibited different demographic and clinical characteristics. Notably, DMT sequence cluster informed differential clinical outcomes. In the future, the proposed algorithm could be applied to other chronic diseases where medication sequence data and patient-specific covariate values are available. The proposed global optimization technique (combining MSiCOR and modified RMPS for unconstrained optimization) can also be applied for mixture Hidden Markov model analysis.