A nonstationary Markov transition model for computing the relative risk of dementia before death

Abstract

This paper investigates the long-term behavior of the k-step transition probability matrix for a nonstationary discrete time Markov chain in the context of modeling transitions from intact cognition to dementia with mild cognitive impairment (MCI) and global impairment (GI) as intervening cognitive states. The authors derive formulas for the following absorption statistics: (1) the relative risk of absorption between competing absorbing states, and (2) the mean and variance of the number of visits among the transient states before absorption. Since absorption is not guaranteed, sufficient conditions are discussed to ensure that the substochastic matrix associated with transitions among transient states converges to zero in limit. Results are illustrated with an application to the Nun Study, a cohort of 678 participants, 75 to 107 years of age, followed longitudinally with up to ten cognitive assessments over a fifteen-year period.

1. Introduction

The natural development of a chronic disease is often expressed in terms of distinct health stages and a Markov chain is a simple yet powerful tool for modeling the progression of individuals through these stages [1, 2]. Muenz and Rubinstein [3] modeled the transitions of a two-state ergodic chain using two logistic regressions where covariates are permitted to change with time. Kay [4] proposed a stochastic process to investigate the relationship between survival time and the health states with death the only absorbing state. Chen et al. [5] introduced a non-homogeneous exponential regression stochastic model to estimate the transition parameters for multi-state disease progression with no backward transition. Li and Chan [6] used a continuous Markov chain to examine the transition rate among multinomial responses all of which are transient. These approaches do not apply to the situation where the chain involves multiple competing absorbing states. Salazar et al. [7] proposed a stationary multi-state Markov model with shared random effects and multiple absorbing states. Incorporating the transition modeling strategy into a generalized linear mixed model (GLMM) facilitates the identification of risk factors associated with transitions among states and provides an expression for the joint distribution of the response vector for a subject while accounting for the dependency among observations on the same subject. Yu et al. [8] continued the investigation by incorporating the baseline distribution into the followup likelihood. The purpose of this paper is to investigate the role of time dependent risk factors such as age in Salazar’s model. Briefly, the emphasis is on the long run behavior of the underlying Markov chain which is no longer assumed to be stationary. We show that in this case the fundamental matrix of the chain needs to be replaced by an infinite matrix series whose convergence status needs a closer examination before absorption statistics can be computed.

2. Motivation for Absorption Statistics

Salazar et al. [7] emphasized estimation of the one-step transition matrix because in that article we were among the first to introduce a shared random effect to correlate the observations in the vector of responses for an individual and because in the application to the BRAiNS cohort data fewer than 10% of the participants transitioned to dementia. To estimate the one-step transition matrix we wrote our own program to approximate the likelihood function, which is defined by an integral. A simulation study [7] showed that the approximation to this integral produced reasonable estimates of the unknown model parameters in the one-step transition matrix, and that these parameter estimates are robust across a spectrum of distributions for the shared random effect. Since then, several developments have occurred. The first is that with the advent of new statistical procedures, including NLMIXED in SAS [9] and GLLAMM in STATA [10], the programming involved in evaluating the one-step transition matrix by approximating an integral has been greatly simplified. In addition, recent research findings by Litiere et al. [11] validated the findings in the simulation study.

In this paper, we apply our methodology to the well-known Nun Study data. The two datasets, BRAiNS and the Nun Study, are similar in that serial cognitive assessments of cohort participants define a Markov chain having three transient states and two absorbing states; the transient states are defined differently in each cohort but the absorbing states are the same: dementia and death. In the application of our methodology to these two data sets, however, we discovered important differences that motivate the current investigation. The Nun Study cohort is older; the mean baseline age of a nun is 83, ten years older than the mean baseline age in BRAiNS. Consequently, the event rate in the Nun Study is significantly higher than that in the BRAiNS cohort, as shown in Table 1. This is true even accounting for the different follow-up periods between these two studies, reflecting the strong association of age with dementia and death.

Table 1.

Comparison of events between two cohorts: BRAiNS and the Nun Study

		Baseline	Follow-up
Cohort	N	Dementia	Dementia	Death*	Events
		n (%)	n (%)	n (%)	n (%)
BRAiNS	553	0 (0%)	55 (10.0%)	144 (26.0%)	199 (36.0%)
Nun Study	501	77 (15.4%)	153 (30.5%)	184 (36.7%)	414 (82.6%)

^*died before conversion to dementia

The analysis of the Nun Study data focuses on this disparity. Recently Yu et al. [8] noted that the likelihood function proposed by Salazar et al. [7] conditions on the baseline status of the individual but that, under a shared random effects model, separating the baseline distribution from the overall model likelihood can lead to underestimation of the effects of risk factors on the one-step transitions. In the Nun Study, the attenuation of effects is substantial due to the presence of 77 participants who were demented at baseline and heterogeneous baseline diagnostics for the rest participants.

In Section 3, we review the definition of this amended likelihood and report its fit to the Nun Study data. Since events occur relatively quickly in that data set, we are able to focus on the computation of the relative risk of dementia before death, a result of more relevance to the field of dementia than the one-step transition probabilities. We end that section by reviewing the definition of this relative risk in a stationary finite Markov chain with absorbing states. Since all transitions depend heavily on age, the Markov chain for the Nun Study is nonstationary. To our knowledge there is virtually no literature for computing the absorption statistics in this case. In Section 4, we state formulae for computing the relative risk as well as the moments of the number of transitions until absorption as functions of the elements of the one-step transition matrices in the nonstationary case. Those readers more interested in the application than in the derivation of this methodology are directed to Section 5, where these absorption statistics are computed and discussed for the Nun Study data.

3. The Extended Multi-state Markov Model and its Application

The extended multi-state Markov model introduced in Yu et al. [8] provides a suitable framework for identifying the risk factors associated with the progression of healthy individuals to a chronic disease with death treated as a competing event. Let Y_i = (y_i1, ···, y_ini) denote the vector of observations for subject i where y_ij denotes the cognitive state of the subject at visit j. Assuming the Markov property the joint distribution of this vector can be written as

$$f(y_{i1},\dots ,y_{i{n}_i})=f(y_{i1})f(y_{i2}\mid y_{i1})f(y_{i3}\mid y_{i2})\dots f(y_{i{n}_i}\mid y_{i{n}_i-1})$$ (3.1)

Each conditional probability f (y_ik|y_ik−1) can be interpreted as a particular element inside a one-step transition matrix for the Markov process. More specifically, suppose y_ik−1 = s and y_ik = v. Then f (y_ik = v|y_ik−1 = s), denoted by P_sv (X_k, γ), is simply the probability of transition for subject i from state y_ik−1 = s at k−1th visit to y_ik = v at k th visit, where s and v are elements of finite transition states within a particular multi-state system. Assume a finite stochastic process consisting of five transition states with three transient and two competing absorbing states. Given observed covariate vector X_k and unobserved random effect γ, we let

$$P_{sv}(\mathit{\theta }\mid {\mathit{X}}_k,\gamma )=\{\begin{array}{l}\frac{1}{1+{\displaystyle \sum _{h=2}^5}exp({\alpha }_h+{\mathit{X}}_k^{\prime }{\mathit{\beta }}_h+{\mathit{\xi }}_h(s)+\gamma )}\hfill \\ \frac{exp({\alpha }_v+{\mathit{X}}_k^{\prime }{\mathit{\beta }}_v+{\mathit{\xi }}_v(s)+\gamma )}{1+{\displaystyle \sum _{h=2}^5}exp({\alpha }_h+{\mathit{X}}_k^{\prime }{\mathit{\beta }}_h+{\mathit{\xi }}_h(s)+\gamma )}\hfill \end{array}$$ (3.2)

The first equation applies for v=1, and the second for v = 2, ··· 5. Here Θ represents the vector of all unknown parameters (α||β||ξ(s)), in which α is the vector of intercepts; β is the vector of unknown fixed effects for covariates of interest; and ξ (s) is the vector of unknown fixed effects for the prior state s.

In addition, let π_j = P(y_i1 = j) represent the probability that subject i is in some state j at the baseline. The probability of the baseline status is similarly parameterized by a separate multinomial logistic regression, which gives

$${\pi }_j({\mathit{\varphi }}_j\mid {\mathit{X}}_B,\gamma )=\{\begin{array}{l}\frac{1}{1+{\displaystyle \sum _{h=2}^4}exp({\tau }_h+{\mathit{X}}_B^{\prime }{\mathit{\delta }}_h+\gamma )}\hfill \\ \frac{exp({\tau }_j+{\mathit{X}}_B^{\prime }{\mathit{\delta }}_j+\gamma )}{1+{\displaystyle \sum _{h=2}^4}exp({\tau }_h+{\mathit{X}}_B^{\prime }{\mathit{\delta }}_h+\gamma )}\hfill \end{array}$$ (3.3)

Again the first equation applies for j=1, and the second for j = 2, ··· 4. The vector ϕ_j ≡ (τ_j || δ_j) represents the vector of unknown parameters determining the baseline probabilities. X_B refers to the observed covariate vector at baseline.

If N is the total number of the subjects in the study, let Y = (Y₁,…Y_N) denote the entire data set. The following marginal likelihood will yield maximum likelihood estimates (MLEs) of the unknown parameters, assuming a random intercept γ distributed N (0, σ²)

$$L(\mathit{\Theta },\mathit{\varphi },{\sigma }^2\mid \mathit{Y})=\int \prod _{i=1}^N\prod _{k=2}^{n_i}\prod _{\begin{array}{l}s=1\dots 3,\\ v=1\dots 5\end{array}}{(P_{sv}(\mathit{\Theta }\mid {\mathit{X}}_{k,}\gamma ))}^{{\delta }_{y_{i,k-1},s}{\delta }_{y_{i,k},v}}{\pi }_{y_{i1}}(\mathit{\varphi }\mid {\mathit{X}}_B,\gamma )dF(\gamma )$$ (3.4)

where δ_{yik−1, s} and δ_{yik, v} are indicator functions valued at 1 if y_ik−1= s and y_ik = v and 0 otherwise. Notice that the likelihood function includes two components, with π_yi1 (ϕ | X_B, γ) representing the baseline status and ${\displaystyle \prod _{k=2}^{n_i}}{\displaystyle \prod _{\begin{array}{l}s=1\dots 3,\\ v=1\dots 5\end{array}}}{(P_{sv}(\mathit{\Theta }\mid {\mathit{X}}_k,\gamma ))}^{{\delta }_{y_{i,k-1},s}{\delta }_{y_{i,k},v}}$ representing the sequence of ensuing one-step transitions, and the two likelihood components sharing the same random effect.

We fit this likelihood function to the Nun Study data, a longitudinal study of aging and Alzheimer’s disease. The cohort consists of 678 members of the School Sisters of Notre Dame religious congregation [12]. Each participant agreed to allow investigators complete access to their convent archives, participate in near-annual assessments of cognitive and physical function, and donate their brain at death. Transitions among states are modeled by a discrete-time Markov chain having three transient (intact cognition, mild cognitive impairment or MCI, and globally impairment or GI) and two competing absorbing states (death and dementia). The flowing diagram of the transitions is presented in Figure 1 [13].

Figure 1.

Flow diagram of one-step transitions on progression to dementia

The first 10 waves consisting of cognitive assessments and measures of activities of daily living are used in this investigation. The transitions among the states of the process are listed in Table 2. We let transition probabilities depend on three covariates: age, the presence/absence of an apolipoprotein E-ε4 allele (APOE4, a common genetic risk factor for Alzheimer’s disease), and levels of education. The dependency among repeated follow-up waves within the same subject is captured using a random intercept. There were 177 participants being excluded from the analysis due to missing covariates or consent withdrawal.

Table 2.

Transitions in cognitive status in the Nun Study

	Current Visit
Prior Visit	Intact Cognition	MCI	GI	Dementia	Death
	n (%)	n (%)	n (%)	n (%)	n (%)
Intact Cognition	520 (65.8%)	179 (22.7 %)	52 (6.6 %)	5 (0.6 %)	34 (4.3 %)
MCI	159 (15.0 %)	629 (59.2 %)	123 (11.6 %)	81 (7.6 %)	71 (6.7 %)
GI	15 (4.2 %)	36 (10.0 %)	162 (45.1 %)	67 (18.7 %)	79 (22.0 %)
Dementia	0 (0%)	0 (0%)	0 (0%)	77 (27.9%)	199 (72.1%)

MCI = mild cognitive impairment; GI=global impairment

The analysis was implemented in SAS using the NLMIXED procedure [14]. The integral is approximated numerically and the dual quasi-Newton algorithm is used to optimize the log-likelihood function. Table 3 lists the MLEs and standard errors for the model. Transitions are significantly affected by age, level of education and the genetic factor APOE4.

Table 3.

Parameter estimates for transition probabilities in the Nun Study (Initial state: intact cognition)

Risk Factor	MCI	GI	Dementia	Death
	MLE (SE)	MLE (SE)	MLE (SE)	MLE (SE)
Age (centered)	0.1502* (0.022)	0.2289* (0.024)	0.2190* (0.027)	0.2439* (0.026)
APOE4	1.6231* (0.391)	2.0029* (0.411)	2.1163* (0.433)	1.7837* (0.434)
Educ. = 16	−1.3889* (0.584)	−1.5188* (0.607)	−1.4378* (0.637)	−1.2003 (0.643)
Educ. > 16	−2.1096* (0.582)	−2.2948* (0.605)	−2.1097* (0.636)	−1.6550* (0.641)
Prior States:
Intact Cog.	−0.4728 (0.371)	−2.9604* (0.362)	−4.4713* (0.568)	−2.7569* (0.386)
MCI	0.2512 (0.369)	−2.7764* (0.350)	−2.3354* (0.368)	−2.6330* (0.367)

^*Significant at α being 0.05; estimate of sigma =1.99* (0.192)

MCI=mild cognitive impairment; GI=global impairment

Given that events occur quickly in this cohort, the absorption statistics are of particular interest in this multi-state Markov model. By modeling the transition among states with a Markov structure, the one-step transition probability matrices constructed based on the parameter estimates provide additional information with respect to the probability of absorption into dementia or death (before dementia) as well as the moments associated with the number of transitions until absorption.

Following the notation in Bhat and Miller [15], consider a finite Markov chain with S states, the first T of which are transient and the remainder absorbing. Denote the k-step transition matrix of the chain by ${\mathit{P}}^{(m,m+k)}={\mathit{P}}^{(m,m+1)}\times \cdots \times {\mathit{P}}^{(m+k-1,m+k)}={\displaystyle \prod _{i=1}^k}{\mathit{P}}^{(m+i-1,m+i)}$, where m is the initial starting time. Let T be the set of transient states and $N_{ij}^{(m)}$, i, j∈T be the random variable representing the number of times the process visits j before it eventually enters an absorbing state, having initially started from state i at time m.

In the stationary case, the above transition matrix does not depend on m, and we have P^(m,m+k) = P^k. Also, in this case absorption is a certain event, it occurs geometrically fast, and the mean and variance of the number of visits before absorption are always finite [15]. The key component in calculating absorption statistics in that case is the fundamental matrix (I−Q)⁻¹, where Q is the substochastic matrix describing transitions among the transient states and I is the T by T identity matrix. Let F = (I−Q)⁻¹ R where R is the T by (S-T) submatrix containing the one-step transition probabilities from a transient to an absorbing state. F contains the probability of eventual absorption into a given absorbing state from an initial transient state. If there are only two absorbing states, the ratio of the element in the first column to the corresponding element in the second column of F defines the relative risk of absorption into dementia before death for that given row or initial transient state.

These simple formulas do not apply to the Nun Study data because the one-step transition probabilities depend on age, making the underlying chain nonstationary. In the next section we provide formula for computing F^(m) (which now depends on m) and the first two moments of $N_{ij}^{(m)}$ in the nonstationary case. These new formulas are applied to the Nun Study data in Section 5.

4. Absorption Statistics for a Nonhomogeneous Chain

Stationarity fails in the nonhomogeneous Markov chain. The transition probabilities among states change with time and as a result the time-dependent transition diagram makes it difficult to formulate the moments associated with $N_{ij}^{(m)}$, i, j∈T. Moreover, the absorption of transient states is not certain. Unless otherwise noted, the nonhomogeneous discrete time Markov chain considered here consists of either absorbing states or transient states at any time, with each state defined as follows.

Definition 1

A state j is absorbing if and only if ∀ m, m ∈ N^*, $P_{jj}^{(m,m+1)}\equiv 1$; with N^* being the set of non-negative integers.

Definition 2

A state i is transient if ∀ m ∈ N^*, ∃ a finite number of transitions that allows the process to go from i to an absorbing state with a positive probability.

Further assume that ∀ m ∈ N^*, ∃ k_m ∈ N such that $P_{(i,j)}^{(m,m+k_m)}>0$ for any i ∈ T and some j∈T^c. Writing this positive probability as h_m(i), it follows that there exists a q(i) such that $q(i)=1-\underset{m}{inf}{h}_m(i)$. Since the chain has only a finite number of states, $q=\underset{i\in \mathit{T}}{max}(q(i))$ exists.

Define the norm of a real vector v= (v₁, v₂, v₃, ···) by $\left|\right|\mathit{v}\left|\right|={\displaystyle \sum _j}\mid v_j\mid$, and the norm of a square matrix A= [a_ij] by $\left|\right|\mathit{A}\left|\right|=\underset{i}{sup}\sum _j\mid a_{ij}\mid$ [16].

Similar to a homogeneous chain, renumber the states so that P^(m,m+k) can be expressed in the following canonical form:

$${\mathit{P}}^{(m,m+k)}=\left[\begin{array}{cc}{\mathit{Q}}^{(m,m+1)}& {\mathit{R}}^{(m,m+1)}\\ \mathit{0}& \mathit{I}\end{array}\right]\times \cdots \times \left[\begin{array}{cc}{\mathit{Q}}^{(m+k-1,m+k)}& {\mathit{R}}^{(m+k-1,m+k)}\\ \mathit{0}& \mathit{I}\end{array}\right]=\left[\begin{array}{cc}{\mathit{Q}}^{(m,m+k)}& {\mathit{R}}^{\ast \ast }\\ \mathit{0}& \mathit{I}\end{array}\right]$$ (4.1)

Here

$${\mathit{R}}^{\ast \ast }={\mathit{R}}^{(m,m+1)}+{\mathit{R}}^{(m+1,m+2)}{\mathit{Q}}^{(m,m+1)}+\cdots +{\mathit{R}}^{(m+k-1,m+k)}{\mathit{Q}}^{(m,m+k-1)}.$$ (4.2)

Notice that the norm of the substochastic matrix $\left|\right|{\mathit{Q}}^{(m,m+k)}\left|\right|=\left|\right|{\displaystyle \prod _{i=1}^k}{\mathit{Q}}^{(m+i-1,m+i)}\left|\right|$ is a non-increasing sequence in k for any m. Moreover if 0 < q < 1, then ||Q^{(m, n)}|| converges to zero as n → ∞ [17].

In limit the elements of the submatrix R^** describe the probabilities of transition from transient states to absorbing states. If we let it denoted by F^{( m)}, we have

$${\mathit{F}}^{(m)}=\sum _{i=0}^{\infty }\left[{\mathit{Q}}^{(m,m+i)}\right]{\mathit{R}}^{(m+i,m+1+i)}$$ (4.3)

Relative risk of absorption between competing absorbing states can be derived directly from the matrix F^(m) by taking the ratio of the row elements.

The theorems below give M^(m) and V^(m), the matrices containing $E(N_{ij}^{(m)})$ and $\mathit{var}(N_{ij}^{(m)})$.

Theorem 1

If m denotes the initial starting time, then

$${\mathit{M}}^{(m)}=[E(N_{ij}^{(m)})]=\mathit{I}+\sum _{k=1}^{\infty }{\mathit{Q}}^{(m,m+k)}$$ (4.4)

Theorem 2

The nonstationary variance, given the initial time being m, is

$${\mathit{V}}^{(m)}={\mathit{M}}^{(m)}+2\left\{{\left(\sum _{i=1}^{\infty }{\mathit{Q}}^{(m,m+k)}\right)}_D+\left(\sum _{i=1}^{\infty }{\mathit{Q}}^{(m,m+k)}\right){\left(\sum _{j=i+1}^{\infty }{\mathit{Q}}^{(m+i,m+j)}\right)}_D\right\}-{\mathit{M}}^{(m)}\#{\mathit{M}}^{(m)}$$ (4.5)

Here the symbol # refers to the elementwise multiplication. These moments exist if 0 < q < 1 where $q=\underset{i\in \mathit{T}}{max}(q(i))$ as in definition 2. Proofs are sketched in appendix.

The transition probabilities in the Nun Study are age-dependent and the process can therefore be treated as a nonhomogeneous Markov chain. We show that the Nun Study data meet the sufficient condition specified earlier. Based on the model MLEs Θ̂, given a nun at age m, each row of the Q^(m,m+1) matrix can be estimated as follows:

$$\left[\frac{1}{1+{\displaystyle \sum _{h=2}^5}exp({\widehat{C}}_h(s)+m{\widehat{\beta }}_h^{\mathit{age}})}\frac{exp({\widehat{C}}_2(s)+m{\widehat{\beta }}_2^{\mathit{age}})}{1+{\displaystyle \sum _{h=2}^5}exp({\widehat{C}}_h(s)+m{\widehat{\beta }}_h^{\mathit{age}})}\frac{exp({\widehat{C}}_3(s)+m{\widehat{\beta }}_3^{\mathit{age}})}{1+{\displaystyle \sum _{h=2}^5}exp({\widehat{C}}_h(s)+m{\widehat{\beta }}_h^{\mathit{age}})}\right]$$

Here Ĉ_h (s) depends on the prior state s but is invariant to age, s=1,2,3. ${\widehat{\mathbf{\beta }}}_h^{\mathit{age}}$ are the MLEs associated with the covariate age from the Nun Study data and are positive. We have

$$\underset{s}{\mathit{Max}}\left|\frac{1+{\displaystyle \sum _{h=2}^3}exp({\widehat{C}}_h(s)+(m+i){\widehat{\beta }}_h^{\mathit{age}})}{1+{\displaystyle \sum _{h=2}^5}exp({\widehat{C}}_h(s)+(m+i){\widehat{\beta }}_h^{\mathit{age}})}\right|\le \underset{s}{\mathit{Max}}\left|\frac{1+{\displaystyle \sum _{h=2}^3}exp({\widehat{C}}_h(s)+m{\widehat{\beta }}_h^{\mathit{age}})}{1+{\displaystyle \sum _{h=2}^5}exp({\widehat{C}}_h(s)+m{\widehat{\beta }}_h^{\mathit{age}})}\right|=q$$

For any m there always exists an immediate transition such that any state in the transient class will be absorbed with a positive probability at least as large as 1 − q. As 0< q<1, absorption is assured and the first and second moments of the number of transitions before absorption are finite in the Nun Study data.

5. Application: The Nun Study

Using the Nun Study data, the relative risk of dementia before death can be calculated for each initial age m by first computing the 3 by 2 matrix F^(m) from Equation (4.3) and then by taking the ratio of the element in the first column of F^{( m)} to the corresponding element in the second column. This yields three relative risks, one for each of the three possible initial transient states. In estimating F^(m) we substitute the MLEs for the unknown betas into the elements of P^(m,m+k) using Equations (3.2) and (4.1). We note that while F^(m) is defined as a sum of an infinite series, it converges relatively quickly due to the involvement of exponential terms of age. Therefore the partial sum of the first few terms in this series is sufficient to approximate the final F^(m).

We adjust for the variability of both random effects as well as the estimated fixed effects in constructing approximate 90% confidence intervals for the relative risks. A smoothed bootstrap technique is adopted based on 1000 iterations [18]. Each iteration involves a random number generated from N (0, σ̂²) and a random vector from MVN (β̂, Σ̂) with σ̂² being the MLE for the variance of random effect γ and Σ̂ the asymptotic variance covariance matrix for the fixed effects β̂. These estimates are then incorporated into Equation (3.2) to compute the estimated transition probabilities and accompanying F^(m). The 5th and 95th percentiles of the iterated relative risks are then used as the lower and upper bounds for the said confidence interval. The resulting estimated relative risks and confidence intervals are plotted in Figure 2.

Figure 2.

Relative Risks and 90% confidence interval of dementia before death

Initial state: intact cognition; age 75 (circle), age 80 (dot) age 85 (square) age 90 (diamond), and age 95 (triangle)

The figure displays the estimated relative risks of developing dementia before death by APOE4 status, level of education, and the ages 75, 80, 85, 90, and 95, assuming the initial state is intact cognition. The figure shows that the relative risk for incurring dementia before death decreases with advancing age regardless of the nun’s genetic status or educational level; this is most pronounced for nuns with positive APOE 4. Nuns with a negative APOE 4 and over 16 years of education have relative risks less than one. In contrast, nuns with positive APOE 4 and 12 or fewer years of education have relative risks greater than one. The estimated confidence intervals especially for this latter subgroup are wider. As suggested by one of the reviewers this could be due to the bootstrapping procedure since this procedure leads to poor estimation whenever the sample size for a subgroup is small; hence, caution is advised in interpreting these confidence intervals.

The following matrices list the mean estimates of the number of visits among transient states before absorption into death or dementia for two sets of covariate pairs: no genetic risk and high education and then genetic risk and low education. Initial age is set at 80. The means were computed using the result of Theorems 1 of the previous section; again only a few terms of the infinite series defining these matrices need to be computed. In addition, elementwise standard errors are bootstrapped in order to evaluate the variability of the mean estimates; these bootstrapped standard errors account for the variability in the beta estimates.

The first example represents a typical subject with an APOE4 allele, no more than 12 years of education and initial age being 80;

$${\mathit{M}}^{(80)}=\left[\begin{array}{ccc}1.06(0.06)& 2.71(0.62)& 1.38(0.42)\\ 0.04(0.04)& 3.78(0.62)& 1.24(0.40)\\ 0.01(0.02)& 0.94(0.37)& 2.40(0.44)\end{array}\right]$$

In contrast, a typical subject with no APOE4 allele, over 16 years of education and initial age being 80 has the following mean estimates for the random variable $N_{ij}^{(80)}$

$${\mathit{M}}^{(80)}=\left[\begin{array}{ccc}3.29(0.39)& 3.04(0.28)& 0.90(0.12)\\ 2.06(0.37)& 4.16(0.28)& 0.87(0.11)\\ 1.05(0.28)& 1.60(0.26)& 1.92(0.14)\end{array}\right]$$

The lead elements in these matrices refer to an 80 year old nun who is cognitively intact at baseline. According to these matrices, a highly educated nun with no genetic risk can expect to have an average of 3.29 transitions into the cognitively intact state before absorption, while a low educated nun with a genetic risk can expect to have 1.06 transitions into this same state before absorption. If we add the elements in the first row of M⁽⁸⁰⁾, then we obtain the number of visits an 80-year-old nun will make before absorption if she is cognitively intact at baseline (7.2 for a highly educated nun with no genetic risk and 5.2 for a low educated nun with a genetic risk). These numbers of visits translate into 9 years and 6.5 years, respectively, after accounting for the time interval between visits, which averaged 15 months. These latter figures are about the same for a nun who has mild cognitive impairment at baseline. However, the duration before absorption is much shorter if the 80-year-old nun has a global impairment at baseline (5.8 years with high education and no genetic risk, 4.3 years with low education and genetic risk).

6. Conclusion and Discussion

This paper examines the absorption statistics of the k-step transition probability matrix for a nonhomogeneous discrete-time Markov chain by focusing on (1) the relative risks of absorption between competing absorbing states and (2) the first and second moments of number of visits among transient states before absorption. These absorption statistics are important in studying the effects of risk factors associated with the progression of healthy individuals through pre-illness states to a chronic disease, with death treated as a competing event. It has been well established that these statistics always exist in a time homogeneous chain and they are directly related to the fundamental matrix. In the nonhomogeneous case, the absorption statistics are not guaranteed to hold since the fundamental matrix is replaced by an infinite matrix series. Proper assumptions about the transition matrix are required to ensure the convergence of this series. One such condition is the existence of a positive supremum bounded between 0 and 1 for the transition probabilities among transient states, independent of when (initial time m) and where (initial state i ∈T) the process starts. Provided this condition, formulas are derived in computing the absorption statistics but partial sums can be used to evaluate the infinite series.

An alternative is to ignore baseline cases of dementia and to ignore the non stationarity. This would simplify the arithmetic since the likelihood would no longer account for baseline and would account for age only through its baseline value when estimating the long run behavior of the chain. The resulting beta estimates could be seriously biased downward in the presence of prevalent as well as incident cases of dementia implying the likelihood yields incorrect beta estimates [8]. In addition, even if the likelihood accounts for the baseline cases failure to account for the non-stationarity of the chain can lead to inaccurate statistics for the long run behavior of the chain. To illustrate consider M⁽⁸⁰⁾ for the case no APOE 4 allele and education greater than 16 years computed in Section 5. If stationarity is assumed, then its elements are inflated by a median value of 82% (range 2.6% to 352%) yielding much larger estimates of these first moments than in the non-stationary case.

The results are applied to the Nun Study. Subjects in the Nun Study do not share the same baseline state. Among 501 subjects used for analysis, 128 had intact cognition at baseline, 249 had MCI, 47 had GI and 77 were demented. The heterogeneous states at baseline for subjects in the study make it important to incorporate the baseline outcomes into the likelihood construction. A multi-state Markov transition model with multiple likelihood components sharing the same random effect is developed for this purpose. The results show that age, education and APOE4 are significant predictors of transition probabilities. The risk of dementia before death declines as age and the level of education increase, while the presence of APOE4 increases such risk. In addition, the expected number of visits among transient states before absorption is also affected by APOE4 status.

There are other approaches for analyzing data involving competing risk events. Gooley et al. [19] proposed using cumulative incidence (CI) estimates for the probability of failure for an event subject to competing risks. Commenges et al. [20, 21] adopt an intensity function approach and investigate an illness-death model to estimate the incidence and prevalence of dementia with competing risk of death. As an alternative, our extended multi-state Markov model provides a framework to specify explicitly the one-step transition matrices of the process. Thus, we can model the observations collected at successive cognitive assessments in the cohort visits and at the same time study their long-term behavior, while accounting for both observed risk factors and latent random effects. Unlike all other approaches to this problem, our approach accounts for visiting multiple transient states and for back transitions among these transient states before absorption into one of the competing absorbing states. Both of these features are characteristic of the real-life trajectories of cognitive decline.

The methodology presented here does have some limitations since it relies on a discrete time model. On the one hand a discrete time model is most sensible for a longitudinal study of aging which consists of serial assessments of cognition since it accounts for the competing risk of death when following individuals from cognitively intact to dementia with transitions to and from intervening impaired states. The reason is that all events except death are interval censored. Use of the Markov model presented here has the advantage that reverse transitions from impaired states to cognitively intact can be accounted for. But the model does assume that serial cognitive assessments are equally spaced such as in the BRAiNS cohort studied by Salazar et al [7]. In the Nun study presented here the retired nuns were located in seven different states and the same team of cognitive assessors had to make the rounds among these retirement centers. Allowing for training of the team and funding gaps the time interval between assessments was not always equal and averaged 1.3 ± 0.3 years (range 1 – 1.9 years). The effect of this lack of equality of successive assessments is unknown.