Associations between Variability of Risk Factors and Health Outcomes in Longitudinal Studies

Abstract

Many statistical methods have been developed that treat within-subject correlation that accompanies the clustering of subjects in longitudinal data settings as a nuisance parameter, with the focus of analytic interest being on mean outcome or profiles over time. However, there is evidence that in certain settings (Elliott 2007; Harlow et al. 2000; Sammel et al. 2001 Kikuya et al. 2008) underlying variability in subject measures may also be important in predicting future health outcomes of interest. Here we develop a method for combining information from mean profiles and residual variance to assess associations with categorical outcomes in a joint modeling framework. We consider an application to relating word recall measures obtained over time to dementia onset from the Health and Retirement Survey.

1 Introduction

Summary statistics such as the sample mean often describe central tendencies in observed risk factors measured at a particular point in time. However, there are many applications in which characteristics of the risk factors over time are of primary scientific interest for predicting disease. These characteristics can include both changes in the mean function, such as a slope, and measures of variability about the mean. While variances are sometimes modeled to accommodate heteroscedaticity or a hierarchical covariance structure [1], methods that treat variances as being of primary interest and the mean or trend as a nuisance are far less common than the converse [2]. Examples of the latter include Harlow et al. [3], where within-woman variability in menstrual cycle length at earlier ages was demonstrated to be an important predictor of abnormal uterine bleeding at later ages; Sammel et al. [4], who used a two-stage model to show that high levels variability in reproductive hormone levels were associated with increased prevalence of menopausal symptoms such as severe hot flashes; Elliott [5], who used a penalized spline model to detrend affect data in a sample of recovering myocardial infarction patients and showed that both low levels and high levels of the residual variance were associated with increased risk of depression; and Kikuya et al. [6], who found that day-to-day variability in blood pressure was associated with increased cardiovascular and stroke mortality risk, while day-to-day variability in heart rate was associated with increased cardiac and stroke mortality risk. Outside of the medical and public health arenas, economists have considered price volatility as an important predictor of risk and returns in financial markets [7], but in general methods to assess associations between variability and risk are underdeveloped in the biostatistics literature.

Methods for longitudinal data have usually treated within-subject variability as a nuisance parameter, with the focus being on central tendency measures to relate the risk factor to an outcome of interest. For example, Muthén et al. [8] used growth mixture models to classify subjects based on underlying mean trends in childhood aggression measures and related these classes to juvenile delinquency risk. Ye et al. [9] used a measurement error model to relate profiles of PSA levels to time to prostate cancer recurrence. These methods focus on error which is non-differential, such that it does not influence the outcome of interest directly. This manuscript develops methods to model the joint distribution of within-subject mean trends and variability in repeated measurements of risk factors as predictors of categorical health outcomes, with the goal of maximizing the predictive power that can be gleaned from longitudinal risk factor data. We use these methods to determine how trends and variability in memory tests are associated transition to dementia using data from the Health and Retirement Study. In particular, we jointly model subject-level slope-intercepts and residual variances of a memory recall test as predictors of onset of dementia during the follow-up period. Although we use a fully Bayesian approach, we provide a simulation study to consider the repeated-sampling properties of our proposed method.

1.1 Memory and Cognition Testing in the The Health and Retirement Study (HRS)

Many studies have shown a positive relationship between older age and variability in performance on sensory, motor, and cognitive tasks [10–11]. There is evidence that intraindividual cognitive variability is a significant source of this variability in performance between groups, especially among older adults [12–13]. Until recently, age-related increases in intra-individual variability have usually been attributed to lack of instrument reliability [14]. While variability arising from this type of measurement error may not be meaningful in and of itself, in many cases intraindividual variability may arise from origins other than measurement error and may provide insight into underlying psychological processes and lead to several possible theoretical interpretations [14]. Intraindividual variability may provide support to existing theories of cognitive aging including the differentiation-dedifferentiation theory and the common cause hypothesis [11]. Intraindividual variability may also be an early marker of cognitive deficits [15]. More specifically, this type of variability might reflect an adaptive response to cognitive decline. For example, it may require high levels of attentional capacity to maintain low levels of variability over repeated trials of a task or across cognitive tasks [14].

Current methods of detection of dementia often rely on information on a person‘s level of performance from a single assessment. A main limitation of this approach is the inability to distinguish between poor performance due to low baseline intellectual ability, preclinical stages of dementia, or fluctuations in cognitive performance due to mood, sensory stimulation, or other state-based differences [16–17]. Because performance on tests such as the Mini-Mental State Examination (MMSE) are known to vary even over relatively short time intervals, our ability to detect early stages of dementia using these methods is not reliable [18]. However, interpreting lack of reliability in this case is difficult, as it may reflect characteristics of both the specific cognitive test and the individual being tested, that is to say inconsistent classification arises from variability due to classification error as well as intraindividual variability in performance. In spite of the difficulties in measurement, investigators have hypothesized that variable cognitive performance would precede consistently poor performance in individuals with mild cognitive impairment (MCI) and those in the very early stages of dementia and thus provide information that predicts over and above what can be achieved by predicting from level information alone [13,17]. We assess this hypothesis using data from the Health and Retirement Study (HRS).

The HRS is a nationally representative, prospective panel study of community-dwelling US adults born between 1890 and 1959 with oversampling of minorities and Florida residents [19]. The HRS include five cohorts: the Asset and Health Dynamics Among the Oldest Old Study (AHEAD) cohort of persons born between 1890–1923; the Children of the Depression Age (CODA) cohort of those born between 1924–1930; the original HRS cohort of those born between 1931–1941; the War Babies (WB) cohort of those born between 1942–1947; the Early Baby Boomer (EBB) cohort of those born between 1948–1953, and the Middle Baby Boomer (MBB) cohort of those born between 1954–1959 [19]. Our focus will be on the AHEAD cohort, consisting of 8,222 subjects born between 1890 and 1923, who have had data collected in 1993, and, if they survived, 1995, 1998, 2000, 2002, 2004, 2006, and 2008. Interviews are conducted by telephone for most respondents under 80 years of age and face-to-face for persons 80 years of age or older. Baseline and re-interview rates have been consistently high, with baseline rates ranging from 70%–81% across the cohorts. Follow-up response rates are on average in the low to mid-90% range; subjects who fail to respond at a given wave are attempted at the next wave unless they have died; hence some missingness is intermittent.

In the HRS, cognitive function is assessed through several questions asked at every wave. For these analyses, only performance on the episodic memory tasks are considered. In particular, we focus on total recall, a measure of episodic memory that consists of immediate and delayed recall of a 10-word list which is asked in the HRS survey. Data on dementia diagnosis come from Medicare claims records linked to HRS respondents. Longitudinal HRS survey data are matched to administrative Medicare records for HRS respondents who have previously consented to have their Medicare data released; over 80% consent to do so. Of those who provided an identification number, 98% have been successfully matched to Medicare files. Dementia is defined using the ICD-9 codes listed in the Chronic Condition Data Warehouse (CCW) definition of Alzheimer’s Disease and Related Disorders or Senile Dementia at http://www.ccwdata.org/cs/groups/public/documents/document/ccw_conditioncategories.pdf. Respondents are classified as having received a dementia diagnosis if they had at least one dementia diagnosis code in any of the Medicare claims files, including: inpatient, outpatient, part B physician supplier, Skilled Nursing Facility (SNF), hospice, and durable medical equipment files. This claims-based diagnostic measure has reasonable sensitivity and specificity for dementia (0.85 and 0.89; see Taylor et al. [20]). The key question of interest is the degree to which trends and variability in cognitive and memory tests are associated with transition to dementia, after adjustment for educational level, race/ethnicity, and gender.

2 A Model to Relate the First Two Moments of Subject-Level Risk Factors to a Health Outcome

Sammel et al. [4] obtained subject-level growth curves and residual variances and used them to predict outcomes in a two-stage model. Here we extend this idea as a shared parameter model linking mean and variance parameters governing the continuous subject-level longitudinal risk factor measures Y with the binary outcome of interest W:

$$Y_{it}|{\beta }_i,{\sigma }_i^2~N(f({\beta }_i;t),{\sigma }_i^2)$$ (1)

$$W_i|{\beta }_i,{\sigma }_i^2,{\mathit{Z}}_i,\mathrm{\gamma }~\mathit{\text{BER}}({\pi }_i),\text{log}\left(\frac{{\pi }_i}{1-{\pi }_i}\right)=g(\mathrm{\gamma };{\beta }_i,{\sigma }_i^2,{\mathit{Z}}_i)$$

$${\beta }_i|\beta ,\mathrm{\Sigma }~N(\beta ,\mathrm{\Sigma })$$

$$\text{log}({\sigma }_i^2)|\sigma ,{\mathrm{\Psi }}^2~N(\sigma ,{\mathrm{\Psi }}^2).$$

We assume that the longitudinal risk factor measures are normally distributed with mean f(β_i; t) that may be a linear or non-linear (polynomial or spline) function of t parameterized by the subject-level parameters β_i, and subject-level residual variance ${\sigma }_i^2$. Similarly, we assume that the log-odds of the outcome is given by a function $g(\mathrm{\gamma };{\beta }_i,{\sigma }_i^2,{\mathit{Z}}_i)$ that allows for linear or non-linear relationships between the subject-level mean profile and residual variance parameters as well as other subject-level covariates, parameterized by the population-level parameters γ. For a fully Bayesian model, we ensure a proper posterior by proposing the following conjugate independent hyperpriors:

$$\beta ~N({\beta }_0,V_{\beta 0})$$

$$\sigma ~N({\sigma }_0,V_{\sigma 0})$$

$${\mathrm{\Sigma }}^{-1}~\mathit{\text{Wishart}}(k,S_0)$$

$${\mathrm{\Psi }}^{-2}~\mathit{\text{Gamma}}(a_{\sigma },b_{\sigma })$$

$$\mathrm{\gamma }~N({\mathrm{\gamma }}_0,V_{\mathrm{\gamma }}).$$

Very weakly informative hyperprior parameters were used to avoid unduely influencing the information provided by the data.

The posterior distribution is obtained using a Markov Chain Monte Carlo (MCMC) approach that combines Gibbs sampling with Metropolois Hastings draws [21–22]. In brief, Gibbs sampling obtains draws from a joint distribution of p(θ | data) for θ = {θ₁, …, θ_q} by initializing θ at some reasonable θ⁽⁰⁾ and drawing ${\theta }_1^{(1)}\text{from}p({\theta }_1|{\theta }_2^{(0)},\dots ,{\theta }_q^{(0)},\text{data}),{\theta }_2^{(1)}\text{from}p({\theta }_2|{\theta }_1^{(1)},{\theta }_3^{(0)},\dots ,{\theta }_q^{(0)},\text{data})$, and so forth. As T→∞, θ^(T) ~˙ p(θ₁,…,θ_n|data). The conditional draws are obtained using adaptive rejection sampling [23] in WinBugs software (WinBugs V1.4.3, Imperial College and MRC, UK, 2007).

3 Associations between Dementia and Baseline Level, Change, and Variability in Total Recall

3.1 Preliminary Analyses

In order to obtain accurate information about dementia onset, we restrict our analysis to the AHEAD cohort subjects who are fee-for-service Medicare beneficiaries and were not diagnosed with dementia at the time of the baseline (1993) interview (4,983 of 8,222 subjects). In order to estimate stable subject-level intercepts, slopes, and variances, we further restrict the analysis to the 2,372 subjects who had at least four interviews during the follow-up period; an additional 20 were excluded for lacking age data, yielding a total of 2,352 for analysis.

Table 1 shows the distribution of total recall by year of follow-up, along with age, gender, education, and race/ethnicity of participating survivors. An interview was completed at a given follow-up with 69–99% of survivors with 4 or more total interviews. Mean recall in 1993 was 8.5 words out of 20 (10 for immediate recall and 10 for delayed recall), declining to 6.5 in 2008. Subjects’ mean age at baseline was 75.4 years, increasing to 88.4 years among participating survivors in 2008. At baseline, 65% of subjects were female; 33% had less than a high school education and 15% had more than a high school education; and 83% were white, 12% African-American, and 4% were Hispanic. Participating survivors became increasingly female and more highly educated through the follow-up period.

Table 1.

Total recall, age, gender, education, and race/ethnicity by year of follow-up among participating survivors. HS=High school. (Standard deviations in parentheses).

Year	1993	1995	1998	2000	2002	2004	2006	2008
n survived	2352	2318	2317	2312	2261	2042	1657	1000
n interviewed	2292	2286	2269	2247	1843	1481	1146	852
Recall	8.5(3.7)	8.8(3.6)	8.2(3.6)	7.5(3.5)	7.2(3.5)	6.8(3.3)	6.5(3.3)	6.5(3.3)
Age	75.4(4.6)	77.4(4.6)	79.7(4.6)	81.8(4.6)	84.0(4.6)	85.7(4.4)	87.3(4.1)	88.4(3.5)
% Female	65.2	64.9	64.6	64.4	66.0	65.8	67.2	69.0
% <HS	33.2	33.0	32.8	30.7	29.4	29.0	29.0	27.8
% HS	52.0	52.2	52.4	52.6	53.7	54.2	53.7	53.6
% >HS	14.7	14.7	14.8	14.8	15.6	16.3	17.3	18.5
% White	83.1	83.2	83.4	83.5	83.9	83.7	83.2	83.3
% Black	12.1	11.8	11.9	11.7	11.6	11.7	11.8	11.6
% Hispanic	3.8	4.0	3.7	3.8	3.7	4.0	4.2	4.5
% Other	0.9	1.0	1.1	1.1	0.8	0.7	0.8	0.6

A dementia diagnosis was obtained among 605 subjects (25.7%) by 2008. Figure 1 plots total recall by age among a subsample of subjects who did not develop dementia and a subsample of subjects who did. There are no clear associations between the observed recall trends and the development of dementia.

Figure 1.

Total recall by age among (a) 10 subjects without a dementia diagnosis, and (b) 10 subjects with a dementia diagnosis.

3.2 Joint Modeling of Recall Mean and Variance to Predict Onset of Dementia During Follow-Up

We fit the model proposed in Section 2 to the total recall and dementia outcome data, letting the risk factor measures y_it be the total recall score raised to the power .7325 to improve the approximation to normality. The outcome w_it is an indicator for whether or not dementia was diagnosed at any time during the follow-up period. Recall measures are obtained from all subjects regardless of dementia diagnosis. Because of the small number of recall observations per subject (4–8), we only considered low-degree polynomials for f(β_i, t); based on preliminary mixed-model analysis we chose a quadratic trend: $f({\beta }_i,t)={\beta }_{0i}+{\beta }_{1i}{\tilde{a}}_{\mathit{\text{it}}}+{\beta }_{2i}{\tilde{a}}_{\mathit{\text{it}}}^2$, where ã_it is the age of the ith person at the t follow-up visit standardized for numerical stability reasons by subtracting the global mean age and dividing by the global standard deviation of age: ã_it = (a_it −81.0496)/5.8487 for age a_it. Thus β_0i corresponds to a subject-level mean, β_1i to a subject-level slope, and β_2i to a subject-level curvature in (transformed) recall scores. For the dementia model, we assume $g(\mathrm{\gamma },{\beta }_i,{\sigma }_i^2,{\mathit{z}}_i)={\mathrm{\gamma }}_0+{\mathrm{\gamma }}_1{\beta }_{0i}+10{\mathrm{\gamma }}_2{\beta }_{1i}+100{\mathrm{\gamma }}_3{\beta }_{2i}+{\mathrm{\gamma }}_4{\sigma }_i^2+{\mathrm{\gamma }}_{5}S({\sigma }_i^2)+{\mathrm{\gamma }}_6^T{\mathit{z}}_i$. (We inflated the random effects associated with slope and curvature to bring the components of γ to be on the same scale and thus improve convergence of the MCMC algorithm.) The function $S(x)={(x-x_1)}_{+}^3-\frac{x_3-x_1}{x_3-x_2}{(x-x_2)}_{+}^3+\frac{x_2-x_1}{x_3-x_2}{(x-x_3)}_{+}^3$ contains the non-linear component of a restricted cubic spline [24] with knots at x₁, x₂, and x₃, termed “restricted” because S(x) is constrained to be linear in its tails (x < x₁ and x > x₃), thus avoiding overfitting while still accommodating any non-linearities in the relationship between risk of dementia onset and subject-level residual variances. The knot values were chosen based on a visual inspection of histograms of the posterior means of the residual variances of a transformed recall scores-only model, and were fixed as (1,3,5). The covariate vector z_i includes dummy variables for education, race/ethnicity, gender, and baseline age categories (65–70, 71–75, 76–80, and 80+). Finally, we assume relatively flat hyperpriors β₀ ≡ 0, V_β0 = diag(1000), σ₀ = 0, V_σ0 = 100, k = 3, S₀ = diag(.1), a_σ = b_σ = .01. Having brought the predictors of dementia to approximately a unit scale, we use priors of the form γ₀ ≡ 0, and V_γ = diag(100). Four chains of 20,000 draws were obtained after a burn-in of of 1,000 draws. Convergence of the Markov Chain Monte Carlo algorithm was assessed for each parameter using the Gelman-Rubin statistic $\widehat{\sqrt{R}}$ [25], which is (approximately) the square root of the total variance of the draws of the parameter divided by the within-chain variance. The maximum value was 1.03 across all population parameters, considered sufficient for convergence.

The posterior mean of β and Σ were $\left(\begin{array}{c}\hfill 4.38\hfill \\ \hfill -.59\hfill \\ \hfill -.09\hfill \end{array}\right)\text{and}\left(\begin{array}{ccc}\hfill 1.190\hfill & \hfill .078\hfill & \hfill -.091\hfill \\ \hfill .078\hfill & \hfill .185\hfill & \hfill -.012\hfill \\ \hfill -.091\hfill & \hfill -.012\hfill & \hfill .015\hfill \end{array}\right)$ respectively, indicating an overall accelerating decline in recall, with considerable between-subject variability. Figure 2 shows the observed and predicted values of y_it for 4 randomly-chosen subjects without and with dementia diagnosis, where the predicted values are given by ${\widehat{y}}_{\mathit{\text{it}}}={\widehat{\beta }}_{0i}+10{\widehat{\beta }}_{1i}{\tilde{a}}_{\mathit{\text{it}}}+100{\widehat{\beta }}_{2i}{\tilde{a}}_{\mathit{\text{it}}}^2\text{for}{\widehat{\beta }}_{\mathit{\text{pi}}}=E({\beta }_{\mathit{\text{pi}}}|y)$. Some subjects had approximately flat trends, while others decreased with varying degrees of rapidity, with no obvious differences between those who developed dementia and those who did not.

Figure 2.

Observed and predicted recall scores with .7325 power transformation, among a subsample of those without a dementia diagnosis and with a dementia diagnosis.

Table 2 provides the posterior means and 95% credible intervals for the predictors of dementia onset during follow-up. There is moderate evidence that higher intercepts and positive or less pronounced negative curvatures are associated with increased risk of dementia, and somewhat stronger evidence that increasing variability in cognitive scores is positively associated with risk of dementia, up to a threshold level in the upper tail of the individual recall score variability distribution. To better visualize these relationship, we plot in Figure 3 the log-odds of dementia for a given subject-level mean, slope, curvature, and residual variance relative to the posterior mean of these population-level quantities (in the case of variance, this is e^σ+Ψ2/2 = .93), holding baseline covariates constant. (The tick marks at the bottom of the plot denote the posterior means a random samples of the posterior means of β_0i, β_1i, β_2i, and ${\sigma }_i^2$, and provide a way to interpret the range of risks associated with the subject-level residual variance in the population.) Subjects with variances below the population mean are at reduced risk, while subjects above this mean are at increased risk up to a threshold of approximately 2.5–3. As a specific example, subjects with a variance of 2.5 are predicted to have an odds ratio of 2.48 (95% CI 1.14,7.85) for development of dementia relative to subjects with a variance of 0.5. Similarly, subjects with less negative curvature are at a higher risk of dementia than subjects with more negative curvature: for example, subjects with a curvature of 0 (linear trend) are predicted to have an odds ratio of 5.42 (95% CI 1.01,947.19) for development of dementia relative to those with a curvature of −.15. Among the baseline covariates, only gender showed any evidence of being associated with risk of dementia onset, with males have an odds of 0.76 relative to females (95% CI .52–1.01).

Table 2.

Log OR of dementia as a function of trends and variances of total recall score with power transformation a, adjusted for education, race/ethnicity, and gender. SD=standard deviation (estimated empirically in two-stage model and parametrically in joint model); HS=High school. Boldface denotes CIs that exclude 0.

		Joint Model	Joint Model	Joint Model
	Two-Stage Model	a=.7325	a=2/3	a=3/4
Subject-Level Intercept βi0	−.003(−.040,.034)	1.001(−.028,2.957)	.607(−.023,2.283)	.925(.004,2.615)
Subject-Level Slope βi1	−.001(−.025,.025)	.400(−.615,2.344)	−.077(−1.043,.747)	.442(−.398,2.057)
Subject-Level Curvature βi2	.065(.018,.112)	14.81(.05,45.69)	9.26(.21,38.44)	13.53(.44,39.53)
Subject-Level Linear Variance ${\sigma }_i^2$	.050(−.081,.182)	.639(.084,1.524)	.766(.195,1.490)	.478(.007,1.088)
Subject-Level Non-linear Variance S(${\sigma }_i^2$ )	−.002(−.012,.008)	−.092(−.336,−.008)	−.777(−2.293,−.061)	−.042(−.117,−.001)
< HS (vs. >HS)	−.096(−.314,.122)	−.140(−.478,.156)	−.123(−.397,.139)	−.136(−.465,.152)
HS (vs. >HS)	.027(−.251,.305)	.012(−.375,.407)	.017(−.304,.343)	.011(−.383,.377)
Black (vs. White)	.003(−.294,.299)	.024(−.394,.452)	.011(−.332,.353)	.015(−.370,.412)
Hispanic( vs. White)	−.036(−.530,.457)	−.095(−.782,.539)	−.084(−.654,.485)	−.082(−.752,.527)
Other (vs. White)	.013(−.921,.946)	.041(−1.105,1.175)	.001(−1.042,.947)	.003(−1.209,1.114)
Male (vs. Female)	−.217(−.417,−.017)	−.273(−.645,.005)	−.234(−.493,−.006)	−.263(−.613,−.002)
Baseline age 70–74 (vs. 65–70)	−.152(−.468,.163)	−.226(−.715,.191)	−.186(−.556,.154)	−.210(−.699,.189)
Baseline age 75–79 (vs. 65–70)	−.033(−.375,.308)	−.081(−.568,.357)	−.064(−.473,.314)	−.069(−.507,.365)
Baseline age 80+ (vs. 65–70)	−.028(−.406,.351)	−.128(−.684,.358)	−.096(−.551,.317)	−.126(−.668,.358)

Figure 3.

log-OR for dementia as a function of subject-level intercept, slope, curvature, and variance versus posterior mean of the population-level mean of intercept, slope, curvature, and variance. Tick marks show random sample of posterior means of 200 individual level variances ${\sigma }_i^2$.

For comparison, a two stage model was fit to these data. First, a separate Gaussian regression model was used to relate recall data to a quadratic association of age (standardized to have a mean of 0 and variance of 1 across all subjects) was fit for each subject. Next, the estimated intercepts, slopes, curvature, and residual variance obtained were used to predict the probability of dementia onset in a logistic regression model, adjusting for education, race/ethnicity, gender, and baseline age. Table 2 shows that, as in the joint model, curvature and gender are associated with risk of dementia, and in the same direction (positive or less pronounced negative curvatures associated with increased risk; males with decreased risk).

3.3 Model Checking

We use posterior predictive distribution (PPD) model checking [25] to assess whether the proposed model provides a reasonable approximation to the true data. The PPD “p-value” represents the probability that an observed statistic (which can be a function of both the data y and the parameter θ) is more extreme than replicated statistic, conditional on the observed data: P(T(y^obs, θ) ≤ T(y^rep, θ) | y), where y^rep is drawn from the posterior predictive distribution f(y^rep | y) = ⌠f(y^rep | θ, y)p(θ | y)dθ. Although PPD p-values are not true p-values in that they do not have a uniform distribution, values close to 0 or 1 give evidence of poor model fit. For the predictor data Y_it (transformed recall scores), we computed for each subject a chi-square discrepancy statistics of the form $T_i({\mathit{y}}_i;{\beta }_i,{\sigma }^2)={\mathrm{\Sigma }}_t\frac{{(y_{\mathit{\text{it}}}-f({\beta }_i,t))}^2}{{\sigma }_i^2}$. We compute $P(T_i({\mathit{y}}_i^{\mathit{\text{obs}}};{\beta }_i,{\sigma }^2)<T({\mathit{y}}_i^{\mathit{\text{rep}}};{\beta }_i,{\sigma }^2)|({\mathit{y}}_i^{\mathit{\text{obs}}}))$ by keeping y_i fixed at its observed values and computing 200 values of $T({\mathit{y}}_i^{\mathit{\text{obs}}};{\beta }_i,{\sigma }_i^2)$ from 200 draws from the posterior of β_i, ${\sigma }_i^2$, and comparing these with 200 draws from $T({\mathit{y}}_i^{\mathit{\text{rep}}};{\beta }_i,{\sigma }_i^2)$, which has a ${\chi }_{n_i}^2$ distribution. Figure 4 shows the resulting histogram of the 2,352 PPD p-values for each subject’s recall score trajectory. The median PPD-value was .48; the range was .20 to .78, indicating a reasonable degree of model fit for all subjects. The largest p-value was for a subject whose recall scores was extremely low across all follow-ups, indicating “floor effects” that were not entirely captured by the normality assumption.

Figure 4.

Histogram of PPD p-values for subject-level recall score trajectory discrepancy statistic: (a) Power transformation of .7325, (b) power transformation of 2/3, (c) power transformation of 3/4.

Some preliminary transformations indicated a rather poor fit: Figure 4 shows the equivalent histograms for recall score power transformations of 2/3 and 3/4. Roughly speaking, power transformations less that .7325 led to overdispersed data, and power transformations greater than .7325 lead to underdispersed data, although the integer nature of the underlying recall scores plays a role as well. Because of this sensitivity we report in Table 2 the association between dementia onset and the recall score mean profiles and residual variances for the two alternative power transformations. We found that there was a modest degree of sensitivity to the choice of transformation, but that the basic finding remained that overall mean, curvature, residual variance, and gender are associated dementia onset, with the function form of the residual variance association being similar. The 2/3rd power transformation suggested a somewhat stronger quadratic association between risk of dementia and residual variance, as well as somewhat narrower intervals for the baseline covariate effects, possibly due to somewhat less predictive information being captured from the recall scores.

We also considered the predictive distribution of the dementia outcome W_i. We considered the total count T = Σ_i w_i, which was T^obs = 605 in the observed data; comparing this with $T^{\mathit{\text{rep}}}={\mathrm{\Sigma }}_i{w}_i^{\mathit{\text{rep}}}$ yields a the posterior predictive “p-value” of .50, where $w_i^{\mathit{\text{rep}}}$ is drawn from a Bernoulli distribution with probability ${\pi }_i^{\mathit{\text{rep}}}=\frac{{\text{exp}}^{g({\mathrm{\gamma }}^{\mathit{\text{rep}}},{\beta }_i^{\mathit{\text{rep}}},{\sigma }_i^{2\mathit{\text{rep}}},{\mathit{z}}_i)}}{1+{\text{exp}}^{g({\mathrm{\gamma }}^{\mathit{\text{rep}}},{\beta }_i^{\mathit{\text{rep}}},{\sigma }_i^{2\mathit{\text{rep}}},{\mathit{z}}_i)}}\text{and}{\mathrm{\gamma }}^{\mathit{\text{rep}}},{\beta }_i^{\mathit{\text{rep}}},{\sigma }_i^{2\mathit{\text{rep}}}$ and drawn from their posterior distributions in the MCMC chain. To assess whether the predicted probabilities are reasonable at the tails of their distribution as well as overall, we considered a Hosmer-Lemeshow-type fit statistic T_k = Σ_i∈k w_i, where k = 1, …, 10 indexes the deciles of the probability of dementia π_i: we compute $T_k^{\text{obs}}={\mathrm{\Sigma }}_{i\in k}{w}_i$ where k is based on the posterior draws of ${\pi }_i^{\text{rep}}$, and compare with the distribution $T_k^{\mathit{\text{rep}}}={\mathrm{\Sigma }}_{i\in k}{w}_i^{\mathit{\text{rep}}}$, based on both the posterior draws of ${\pi }_i^{\mathit{\text{rep}}}$ and the posterior predictive draws of $w_i^{\mathit{\text{rep}}}$. This yielded PPD p-values across the deciles of (.32,.38,.34,.47,.44,.43,.56,.52,.52,.55), indicating reasonable fit for the second stage of the model over the range of π̂_i, with a very modest tendency to overestimate risk of dementia in the lowest-probability subjects and underestimate risk of dementia in the highest-probability subjects.

4 Simulation Study

We conducted a simulation study as follows. We consider 1,000 observations with 5 repeated measures per subject, with the continuous predictors generated under a Gaussian random effects linear model and the dichotomous outcome generated under a logistic model that is a function of the random effects that govern the predictors:

$$Y_{\mathit{\text{it}}}|{\beta }_i,{\sigma }_i^2~N({\beta }_{0i}+{\beta }_{1i}t,{\sigma }_i^2),t=0,\dots ,4,i=1,\dots ,1000$$

$$W_i|{\beta }_i,{\sigma }_i^2,\mathrm{\gamma }~\mathit{\text{BER}}({\pi }_i),\text{log}\left(\frac{{\pi }_i}{1-{\pi }_i}\right)={\mathrm{\gamma }}_0+{\mathrm{\gamma }}_1{\beta }_{0i}+{\mathrm{\gamma }}_2{\beta }_{1i}+{\mathrm{\gamma }}_3{\sigma }_i^2$$

$${\beta }_i~N(\beta ,\mathrm{\Sigma })$$

$$\text{log}({\sigma }_i^2)~N(\sigma ,{\mathrm{\Psi }}^2)$$

We have $\beta ={(4-.1)}^T,\mathrm{\Sigma }=\left(\begin{array}{cc}.7\hfill & .07\hfill \\ .07\hfill & .12\hfill \end{array}\right),\sigma =-.4,{\mathrm{\Psi }}^2=.3,\text{and}\mathrm{\gamma }={(-\mathrm{6\; .75\; 0\; 2})}^T$. This corresponds to a 1 standard deviation increase in the subject-level intercept being associated with a 69% increase in the odds of the outcome, and a 1 standard deviation increase in the subject-level variance being associated with a 310% increase in the odds of the outcome. The probability of the outcome is approximately 19% when the random effects are fixed at their population means.

Two chains of 2,500 draws were obtained after a burn-in of 2,500 draws. The bias (based on the posterior mean), nominal 95% credible interval coverage, and power for the population-level regression parameters estimated from 100 simulated datasets are given in Table 4. Bias is negligible for the population slope and intercept for the predictors, and coverage is approximately correct. There is a modest degree of bias for the logistic regression parameters, on the order of 10–15%, but again coverage is approximately correct. For comparison, we include the results of a two-stage analysis for the logistic regression parameters. Bias toward the null is severe, with nominal 95% coverage being essentially 0 for three of the four parameters.

5 Discussion

Despite the great proliferation of longitudinal health data during the past three decades, relatively little attention has been paid to the role that variability in such data might play in predicting outcomes of interest. This manuscript attempts to fill this gap by developing a method to combine information about both mean trends and variances in longitudinal data to predict categorical outcomes of interest. The method is applied to predict onset of dementia in elderly adults over a fourteen-year time period using recall data measured every two years. Residual variability is found to be associated with dementia risk, with subjects with low variability being less likely to develop dementia by the end of the follow-up period of fourteen years than subjects with moderate to high variability. Overall mean level and curvature (quadratic trend of recall) were marginally associated with dementia risk, with increase mean level and increased quadratic trend found to be associated with increased risk of dementia onset. Little predictive power was found for linear trend or in the baseline measures of education, race/ethnicity, or age. The cognitive performance trends had associations that were reversed from those hypothesized with the diagnostic outcome (higher intercepts associated with decreased risk, accelerating declines associated with increased risk), possibly due to the fact that intercepts and curvatures had a strong negative correlation (posterior mean of -.68), which would lead to some instability in estimation of associated effects. Increased within-person variability had a significant prognostic relationship in the hypothesized direction.

The importance of using a joint model to assess the relationship between the individual parameters governing the trends and variability of recall and risk of dementia is seen in the fact that simple two-stage models that used the results from individual linear regression fits of recall data either had had little relationship with dementia risk, or it was in the reverse direction from that hypothesized (negative curvature, i.e. an increasingly rapid reduction in memory performance, was associated with a decreased risk of dementia onset). This is likely due to very substantial bias toward the null induced by measurement error, and/or spurious relationships induced by floor or ceiling effects, in the first-stage model. A two-stage procedure that used, e.g., empirical Bayes estimates from random effects for both the means and variances to stabilize first stage estimation of the mean trends and residual errors would likely have improved performance, but standard sofware for fitting random effects models does not typically allow for estimation of subject-level random effects for the variance components.

The authors also attempted to fit the shared parameter model in (1) using a fully likelihood-based method. However, integrating out the subject-level variance random effects proved virtually intractable using adaptive Gaussian quadrature methods available for the PROC NLMIXED procedure available SAS V 9.1 (SAS Institute, Cary, NC). Our simulation study suggests that the Bayesian approach with weakly informative priors has reasonable repeated sampling properties, and is vastly superior to estimates obtained from a two-stage method.

Use of a low-degree polynomial to model the longitudinal recall data was dictated by both the nature of the question at hand – specifically the desire to link slopes and curvatures of recall trends to dementia risk – as well as the relative paucity of follow-up visits for each individual. Even if sufficient longitudinal data were available, higher order terms in non-linear growth profiles for the longitudinal risk factors yield predictors of outcome that are difficult to interpret. Elliott [5] used a penalized spline model to detrend subject-level daily affect data consisting of up to 35 follow-up measures, relating the risk of depression to latent clusters of subject-level variability. An extension that would incorporate information from both means and variances could assign subjects into latent classes of profiles and variances, and link these classes to a categorical outcome of interest via a log-linear model [26–27]. Such an approach requires some clustering of the profiles and the residual variability of longitudinal predictors; in the application of interest here no such clustering was evident.

As Table 1 shows, most of the missing data in the longitudinal predictors was structurally missing (due to death, rather than loss-to-follow up). Use of a linear mixed model for the longitudinal data makes a missing at random (MAR) assumption [28]. When the missingness is intermittent, the MAR assumption seems reasonable: violation would require that we systematically miss low- or high-mean or highly variable observations within a subject. The MAR assumption is stronger in the small fraction of dropout data, in that it requires that our modeling assumptions of the mean or variance be correct. Future work could consider selection models that would provide sensitivity analysis for violations of the MAR assumptions.

Finally, some discussion of the limitations of our analysis is in order. First, restricting our analysis to those subjects with four or more recall scores, while necessary to provide information about quadratic trends and residual variances, may also lead to selection bias, since subjects with three or fewer follow-ups may be at higher risk of death, which could have a variety of impacts on risk of dementia during follow-up. Indeed, subjects with three or fewer followups were older (79.3 years vs. 75.4 years) and had lower recall scores (6.3 vs. 8.5) at baseline, and were more likely to be male (42.5% vs. 35.3%), lack a high school degree (52.5% vs. 33.3%), and be African-American (16.3% vs. 12.1%) (all p < .001 by t-test or χ² test). This limitation might be addressed in part by treating the outcome as a time-to-event measure rather than a dichotomous outcome over the whole follow-up period. This would have the advantage of accounting for administrative or competing risk censoring in cases of drop-out or death, as well as increased power and a more nuanced understanding of the associations of interest. However, the problem of insufficient information to estimate trends and residual variance in subjects with few follow-up measures will remain, and highlights the requirement for sufficient follow-up data to implement the proposed methods. A second major issue results from the numerical “fragility” of the models, at least as fit in Winbugs. For example, attempts to include non-linear effects for mean trends in the prediction of dementia onset parallel to those used for variance lead to numerical overflow (“trap”) errors, as did use of flatter hyperprior parameters in S₀ for p(Σ⁻¹). Direct computation of the relevant conditional distributions for the Gibbs algorithm, though more time-consuming, may allow for direct control of overflow errors; alternatively model approaches such as use of growth mixture models to classify mean trends into categorical predictors may provide more robust numercial results.

Table 3.

Simulation study results: Population-level regression parameters. Bias for posterior mean; coverage for 95% credible intervals.

	Two-Stage Model			Joint Model
Parmameter	Nominal 95%			Nominal 95%
(True Value)	Bias	Coverage	Power	Bias	Coverage	Power
β0 (5)	—	—	—	.00	94	—
β1 (.1)	—	—	—	−.00	96	—
γ0 (−4)	2.75	0	—	−.20	99	—
γ1 (.75)	−.34	1	100	.01	94	99
γ2 (0)	.43	35	65	.01	93	7
γ3 (2)	−1.53	0	100	.17	95	100