The MMSE Can Yield Biased and Imprecise Estimates of Change: A Novel IRT Analysis of Latent Change Scores from the A4 Clinical Trial

Abstract

The measurement precision of change scores has previously been investigated from the perspective of classical test theory (CTT). However, the measurement precision of change scores has not been thoroughly explored from an item response theory (IRT) perspective. In this study, we provide, to our knowledge, one of the first direct investigations of change score precision within an IRT framework. Specifically, using archival data from the Anti-Amyloid Treatment in Asymptomatic Alzheimer’s (A4) trial, we examined standard error of estimate for change scores on the Mini–Mental State Examination (MMSE), one component of the Preclinical Alzheimer Cognitive Composite used to measure change between intervention arms. Multidimensional two-parameter IRT models were fitted to the MMSE item data with one latent dimension reflecting baseline ability and a second reflecting change in ability over time (i.e., latent change scores). Results showed that standard error depended on change magnitude and that change scores were expected to be biased towards zero when baseline performance scores were near ceiling. The results demonstrate why measures with pronounced ceiling effects should not be used to assess change in clinical trials or other longitudinal studies, and should be used cautiously in clinical settings. This study also demonstrates how IRT can be used to evaluate change score precision.

Change scores are the typical outcomes in most longitudinal studies and clinical trials, whether calculated as observed (difference) scores, modeled as latent change scores, or predicted as time-by-outcome interaction effects. Although comprehensive methodologies have been developed for estimating latent change (McArdle, J. J. & Nesselroade, J. R., 2014), relatively little attention has been given to the measurement precision of latent change scores. Measurement imprecision can significantly impact longitudinal research and clinical trials—leading to biased estimates and reduced statistical power—particularly when tools with pronounced floor effects (i.e., where most examinees show very poor performance) or ceiling effects (i.e., where most examinees show very good performance) are used as outcome measures.

In classical test theory (Cronbach & Furby, 1970; Trafimow, 2015) the reliability coefficient for difference scores (${\rho }_{D{D}^{\prime }}$) is defined as a function of the reliability of the score at the first time point (${\rho }_{X{X}^{\prime }}$), the reliability of the score at the second time point (${\rho }_{Y{Y}^{\prime }}$), and the correlation between scores over time ($2{\rho }_{XY}$):

$${\rho }_{D{D}^{\prime }}=\frac{{\rho }_{X{X}^{\prime }}+{\rho }_{Y{Y}^{\prime }}-2{\rho }_{XY}}{2-2{\rho }_{XY}}$$ (1)

Another approach to characterizing the reliability of change scores within the CTT framework is to estimate the Reliable Change Index (RCI) for individual subjects (Duff, 2012). The RCI is mostly used in the context of applied clinical contexts for detecting change in outcomes over time, but can also be used in research settings. Although various RCI-like indices have been proposed (typically accounting for practice effects), the formula generally follows the form:

$$\text{RCI}=\frac{Y-X}{{\mathit{\text{SE}}}_D}$$ (2)

The RCI quantifies reliable change as the difference between two observations divided by the standard error of the difference (${\mathit{\text{SE}}}_D$). In turn, the ${\mathit{\text{SE}}}_D$ is typically estimated using the correlation between scores over time:

$${\mathit{\text{SE}}}_D={SD}_X\sqrt{1-{\rho }_{XY}}$$ (3)

where ${SD}_X$ is the standard deviation of the baseline score. Once the RCI is calculated, an examinee’s RCI can be evaluated against an inferential threshold to determine whether the change is large enough in magnitude to be considered statistically meaningful.

Note that, based on these equations, higher correlations between scores over time (${\rho }_{XY}$) increase the RCI but decrease the ${\rho }_{D{D}^{\prime }}$. Some have taken the decrease in ${\rho }_{D{D}^{\prime }}$ to imply that change scores, especially those based on highly stable measures, should not be used to study change in pre-post clinical trials and other types of longitudinal studies; however, this conclusion is categorically false (D. R. Thomas & Zumbo, 2012; Trafimow, 2015). Higher correlations between measures lead to reduced variance in difference scores, and reduced variance in difference scores produces larger effect size and greater statistical power. For example, Cohen’s $d$ for comparing difference scores between two groups can be defined as:

$$\text{Cohen’s}d=\frac{D_2{-D}_1}{{SD}_D}$$ (4)

where $D_2$ is the mean difference score for Group 2, $D_1$ is the mean difference score for Group 1, and ${SD}_D$ is the standard deviation of the difference scores. Because the variability of difference scores appears in the denominator of Equation 4, higher correlations between scores—which reduces the variability of difference scores—results in larger effect sizes and, consequently, greater statistical power (note that while some approaches use ${SD}_X$ in the denominator, this version of Cohen’s $d$ is directly relevant to inferential comparison of mean differences).

Why do higher correlations between scores over time reduce the measurement precision of difference scores, yet increase precision for measures of reliable change? This apparent contradiction arises, in part, from how CTT defines reliability: as a ratio of true score variance to total variance. A measure cannot demonstrate high reliability if there is little variability in observed scores. As a result, CTT-based difference score reliability metrics can be misleading indicators of measurement precision in the context of intervention research and other types of longitudinal studies.

A better approach to evaluating the precision of change scores would be to focus on the estimation error for the individual change score estimates themselves. The RCI attempts to do this, but because a single value of reliability is used to derive ${\mathit{\text{SE}}}_D$, the estimation errors are not permitted to vary within the population (as they should). The more modern psychometric framework known as item response theory (IRT) allows us to do this. The reader is referred to several reviews and books that compare CTT and IRT (Embretson & Reise, 2000; M. L. Thomas, 2011); here, we focus specifically on their ability to characterize the precision of change score estimates. The concept of reliability is not required or directly used in IRT, although it is possible to approximate CTT reliability coefficients using IRT-based metrics. Instead, IRT emphasizes the standard error of parameter estimates. Unlike the CTT concept of standard error of measurement, which is a single value applied uniformly across all subjects, the standard error of the estimated parameters (hereafter standard error of estimate) in IRT can vary across individuals depending on their performance level.

IRT models are generalized linear latent factor models (Skrondal, A. & Rabe-Hesketh, 2004), meaning they characterize latent (unobserved) variables using generalized linear model (GLM) equations. For example, when item responses are binary, as is often the case in cognitive assessments, the models typically use logit or probit link functions along with binomial variance functions. In this context, the binomial variance function is particularly important, as it implies that the variability of item scores is tied directly to the probability of a correct response (i.e., the transformed linear predictor). More precisely, the standard error of an individual’s estimated ability parameter, ${SE}_{\theta }$, is (Reckase, 2009):

$${SE}_{\theta }=\frac{1}{\sqrt{{\alpha }^{2}P\left(\theta \right)\left(1-P\left(\theta \right)\right)}}$$ (5)

where $\alpha$ is the item discrimination parameter, $\theta$ is the examinee’s latent ability score, and $P\left(\theta \right)$ is the probability of correct response given $\theta$ defined as

$$P\left(\theta \right)=g^{-1}\left(\nu +\alpha \theta \right)$$ (6)

$g^{-1}$ is the inverse link function (logistic or normal cumulative distribution function) and $\nu$ is the item intercept parameter (inversely related to difficulty). Note that in Equation 5 the denominator is largest, and thus the standard error is smallest, when $P\left(\theta \right)=0.50$. Conversely, the denominator is smallest, and the standard error is largest, when $P\left(\theta \right)\to 1.00$ or $P\left(\theta \right)\to 0.00$.

${SE}_{\theta }$ is a function of both item properties ($\nu$ and $\alpha$) and the examinee’s ability level ($\theta$). The consequences of mismatched item/test difficulty and examinee ability on the estimation of difference scores have been previously discussed from an IRT perspective (Jabrayilov et al., 2016; Reise & Haviland, 2005; Tang et al., 2023), though not directly in terms of the latent change score itself (but see Ho et al., 2023 for a conceptually similar discussion). This is likely because most IRT models are not structured to directly estimate latent change scores over time. Instead, they typically estimate latent ability at each time point and then compute a difference score post hoc. This approach is suboptimal, as it relies on error-prone estimates of ability at each time point and does not yield a direct estimate of the standard error for the change score.

In contrast, Thomas et al. (2021) presented a novel modeling approach that is capable of directly quantifying error in latent change scores using IRT methods. The original application of this work focused on differences in performance between experimental conditions in a cross-sectional framework; however, the structural components of the model are equally applicable when assessing change between time points. In this application, we extend that framework to longitudinal data. The key point to understand is that a change score assessment is a type of multidimensional assessment where the observed follow-up scores derive from both the latent baseline scores (${\theta }_{\mathit{\text{baseline}}}$) and the latent change scores (${\theta }_{\mathit{\text{change}}}$); in other words, the follow-up score is a function of the sum of the ${\theta }_{\mathit{\text{baseline}}}$ score and the ${\theta }_{\mathit{\text{change}}}$ score. And, as a result, Equation 5 is generalized to include both dimensions of latent ability (Segall, 2009):

$${\mathit{\text{SE}}}_{\mathbf{\theta }}={\left[{\mathbf{\alpha }\mathbf{\alpha }}^{\mathit{T}}P\left(\mathbf{\theta }\right)\left(1-P\left(\mathbf{\theta }\right)\right)\right]}^{-1}$$ (7)

where $\mathbf{\alpha }$ and $\mathbf{\theta }$ are vectors of discrimination and ability parameters, respectively. Like the unidimensional standard error function (Equation 5), the multidimensional standard error function in Equation 7 indicates that standard error is smallest when predicted accuracy is near 0.50, and largest when predicted accuracy approaches the floor (0.00) or ceiling (1.00) of performance. For the multidimensional model, $P\left(\mathbf{\theta }\right)$ is defined as:

$$P\left(\mathbf{\theta }\right)=g^{-1}\left(\nu +{\alpha }_{\text{baseline}}{\theta }_{\mathit{\text{baseline}}}+{\alpha }_{\text{change}}{\theta }_{\mathit{\text{change}}}\right)$$ (8)

To accommodate change in the weightings of ability, each value of $\alpha$ is defined as:

$$\alpha =\gamma \lambda$$ (9)

where $\gamma$ is a quantiative code that determines whether the latent change score impacts a given response (i.e., baseline ability affects both the observed baseline scores and observed change scores, but change in latent ability only affects the observed change scores) and $\lambda$ is the discriminate weight for the item (M. L. Thomas et al., 2021). $\gamma$ is key to the modeling approach. It is a quantitative code identical to those used in general linear models. In the context of measuring performance, $\gamma$ defines the contrast of interest, whether that contrast is between experimental conditions or between time points. For example, in a pre-post design, $\gamma$ is coded as 0 for the baseline assessment and 1 for the follow-up assessment. This coding ensures that ${\theta }_{\mathit{\text{change}}}$ is accurately interpreted as the difference in ability between assessments. The approach follows the same logic as contrast coding of time-point data in latent growth models.

Framing change score assessment as a multidimensional problem clarifies that ${SE}_{\mathbf{\theta }}$ is influenced by the examinee’s baseline ability (${\theta }_{\mathit{\text{baseline}}}$), the examinee’s change in ability (${\theta }_{\mathit{\text{change}}}$), and the item properties ($\nu$ and $\alpha$). To accurately measure change, a test must be well suited to measure both baseline scores and the follow-up scores after change has occurred (i.e., the test must avoid performance ceilings and floors at both time points). Thus, depending on the properties of a test, estimates of change can be precise at both time points, imprecise at both time points, or precise at one time point but imprecise at another.

Imprecision in score estimates is expected to fluctuate most for measures with pronounced floor effects or ceiling effects. Floor and ceiling effects arise from an interaction between the characteristics of the population being assessed and both the overall difficulty and the range of difficulty of the task being administered. For example, in dementia research, tools such as the Mini–Mental State Examination (MMSE; Folstein et al., 2014) are commonly used to classify participants as cognitively normal, preclinical (e.g., mild cognitive impairment), or demented. For most cognitively normal individuals, MMSE items are too easy, and most examinees earn perfect or near-perfect scores (i.e., are at the performance ceiling). In contrast, for individuals with dementia, MMSE items are more challenging, resulting in lower total scores and greater variability. The MMSE’s simplicity and brief administration time have led to its widespread use as an outcome measure in clinical trials. In fact, the MMSE is one component of the Preclinical Alzheimer Cognitive Composite (PACC; Donohue et al., 2014), which is a commonly used primary cognitive outcome composite in Alzheimer’s clinical trials. Whether the MMSE provides precise estimates of cognitive change within the PACC is expected to depend on participants’ baseline ability levels (${\theta }_{\mathit{\text{baseline}}}$) and their change in ability over time (${\theta }_{\mathit{\text{change}}}$).

In this study, we conducted a secondary data analysis of MMSE outcome data from the Anti-Amyloid Treatment in Asymptomatic Alzheimer’s (A4) trial (Sperling et al., 2023). The A4 trial recruited cognitively normal older adults (Clinical Dementia Rating score of 0 and MMSE ≥ 25) with elevated brain amyloid levels to participate in a randomized clinical trial comparing solanezumab—a monoclonal antibody intended to clear amyloid from the brain—to placebo. This dataset provides an ideal opportunity to examine how measurement precision for change scores is affected by a mismatch between examinee ability and test difficulty. Specifically, because the A4 trial recruited cognitively normal individuals, we hypothesized that the MMSE would perform poorly as a measure of change in this population.

Methods

Outcome data from the A4 trial were obtained from a public online repository that requires registration but is otherwise freely accessible (www.a4studydata.org; Alzheimer’s Clinical Trials Consortium, 2025). The A4 trial was preregistered, but this study’s design and its analysis were not preregistered. The A4 trial screened thousands of potential participants over a five-visit pre-baseline period, ultimately enrolling a subset who met eligibility criteria and randomizing them to receive either placebo or solanezumab. In the dataset, week 0 (visit 6) marks the start of the trial, and week 240 (visit 66) marks the end (although an open-label extension phase followed the initial trial period).

The MMSE was administered during the screening phase and subsequently every 12 weeks for enrolled participants, including at baseline and at the end of the trial. Because our interest in these data is methodological, we focused specifically on the baseline (week 0; visit 6) and study conclusion (week 240; visit 66) time points. Analyses were limited to the 1,138 participants who had complete MMSE data at both time points. Demographic and clinical characteristics of this full sample are reported in Sperling et al. (2023). For the restricted sample included in the current study, the mean age was 71.14 years (range: 65–85.26) and mean years of education was 16.63. Sixty percent were female, 3% identified as Hispanic or Latino, less than 1% identified as American Indian or Alaskan Native, 2% identified as Asian, 2% identified as Black or African American, 95% identified as White, and all other participants identified with multiple racial categories or did not report. The current study was reviewed by a local institutional review board and was determined not to involve human subjects, as defined by DHHS and FDA regulations.

The MMSE is a 30-point/item screening measure of cognitive status. Items cover domains including orientation to time (5 points), orientation to place (5 points), registration (3 points), attention (5 points), recall (3 points), and language (9 points). Each item is scored correct versus incorrect. The A4 MMSE dataset labels items as “Correct” versus “Incorrect”, with the exception of the 5 attention questions which are unscored. These items, which require the examinee to spell a word backwards, were therefore manually scored in R prior to analysis where one point was awarded for each correct letter in the correct position, in reverse order (this approach, the traditional scoring method, is slightly more stringent than the method used in the A4 trial; as implied by the results discussed below, the more lenient scoring approach likely exaggerates the ceiling effects observed on the MMSE.).

Prior research has suggested that the MMSE is best represented by a bifactor model, consisting of one general cognitive ability factor and several domain-specific subfactors (Rubright et al., 2016). Additional information about this modeling approach can be found in the Supplemental Information. Analyses were conducted using R version 4.4.2. To estimate item parameters, a bifactor model was first fitted to the data using the bfactor function in the mirt package version 1.42 (Chalmers, 2012). The factor loadings from this bifactor model were then converted into IRT discrimination parameters and applied to the combined week 0 and week 240 MMSE data using the cogirt package version 1.0.0 (M. L. Thomas, 2024). The cogirt package supports the estimation of latent baseline and change scores based on a multidimensional two-parameter logistic (2PL) IRT model using a Metropolis-Hastings Robbins-Monro algorithm. In our analysis, discrimination parameters were fixed to the values derived from the bifactor model, while item intercepts were freely estimated. To distinguish between baseline and latent change, treatment (i.e., dummy) contrast coding was used in the experimental design matrix (i.e., the γ value; M. L. Thomas et al., 2021). Corresponding item parameters were constrained to be equal across the week 0 and week 240 assessments. Latent ability scales were assigned a multivariate normal prior to identify the model. Code from these analyses will be shared upon request.

Results

MMSE scores at baseline were near the maximum score of 30, showing low variability, moderate negative skew, and a leptokurtic distribution (M = 28.81, SD = 1.32, skew = −1.46, kurtosis = 2.70). At follow-up, performance declined slightly, variability increased, and the distribution became even more negatively skewed and highly leptokurtic (M = 28.54, SD = 2.15, skew = −3.36, kurtosis = 18.62). Histograms of performances are reported in Supplemental Figure 1. Reliability (Cronbach’s alpha) was poor at the first time point (0.41) but improved at follow-up (0.77).

Parameter estimates for the MMSE items are reported in Supplemental Table 2. The average estimate of latent baseline ability was high (M = 2.728, SD = 0.735), reflecting strong MMSE performance at baseline among the A4 trial participants. The average estimate of latent change in ability was negative and close to zero (M = −0.032, SD = 0.535), suggesting that, on average, participants’ cognitive abilities did not change substantially over the course of the trial (as reported in the original study).

Figure 1 plots each participant’s latent change score estimate on the x-axis against the standard error of that estimate (${\mathit{\text{SE}}}_{\mathbf{\theta }}$) on the y-axis. Each data point is color-coded by baseline MMSE score: green triangles for participants with scores of 30 (n = 431), blue triangles for scores of 29 (n = 343), orange circles for scores of 28 (n = 200), and black squares for scores of ≤ 27 (n = 164). The figure reveals a clear pattern: participants with higher baseline MMSE scores (i.e., higher baseline cognitive ability) tended to have less precisely estimated latent change scores, as indicated by higher ${SE}_{\theta }$ values. In contrast, those with lower baseline scores had more precise estimates of change. Additionally, change scores were more precise for participants with more negative latent change than for those with more positive latent change.

Figure 1.

Standard Error of Estimate for A4 Trial Participants with Different Baseline Mini–Mental State Examination (MMSE) Scores

To better understand the consequences of differential measurement precision, we used the cogirt package to simulate MMSE change score data based on parameter estimates derived from the A4 trial. We simulated two subgroups: participants with baseline latent ability scores at the A4 trial mean ($M_{\theta \mathit{\text{baseline}}}=2.728$) and participants with baseline latent ability scores two standard deviations below the mean ($M_{\theta \mathit{\text{baseline}}}=1.258$). In both groups, we simulated a standard deviation of negative latent change (i.e., based on variability estimates from the A4 trial data; $M_{\theta \mathit{\text{change}}}=-0.768$). The results are shown in Figure 2, along with LOESS fit lines relating latent to observed change in each group. Even though both groups were simulated to have the same true latent change in MMSE ability, the average observed MMSE change score was −1.444 in the high baseline ability group but −4.338 in the low baseline ability group; a significant difference of nearly 3 MMSE points between groups (t(1136) = 14.255, p < 0.001). Cohen’s d for the single-group observed change effect was 0.551 for the high baseline ability group but 1.066 for the low baseline ability group. Based on these effect sizes, a total sample size of N = 106 would be needed to detect the change effect in the high baseline ability group, whereas only N = 30 would be needed to detect the change effect in the low baseline ability group.

Figure 2. Simulated Mini–Mental State Examination (MMSE) Latent Change Plotted Against Observed Change.

Note: Individual data points are plotted along with LOESS fit lines for each group.

Discussion

This study presents one of the first investigations of change score precision from the perspective of IRT. Specifically, using archival MMSE data from the A4 trial, we confirmed our hypothesis that the MMSE performs poorly as a measure of change in this population due to mismatched task difficulty and examinee ability. The average error in latent change score estimates was high and varied systematically across participants with different baseline ability and change scores. Change estimates for individuals who scored near the ceiling of performance on the MMSE at either baseline or follow-up were much less precise than those for participants with lower MMSE scores at either time point. Additionally, in a simulation based on the A4 data, we demonstrated that both observed and latent change scores are attenuated (biased towards zero) among participants with higher versus lower baseline ability scores. Interestingly, we found that MMSE change scores can be attenuated by as much as 3 points on average, the same magnitude of change that has is considered reliable based on the RCI (Hensel et al., 2007)

A reasonable question to ask is whether the psychometric limitations of the MMSE contributed to the null findings of the A4 trial for cognitive outcomes (and possibly other studies that used the measure). The MMSE is one of four scaled and normed tests contributing to the PACC (the primary outcome for the A4 trial). While the MMSE likely added some error and bias to the PACC scores, without contributing much reliable variance, this error and bias would not have preferentially impacted participants randomized to solanezumab. Moreover, the A4 trial had a large sample and was well powered. Thus, it is very unlikely that psychometric problems with MMSE changed the main conclusions of the A4 study. Instead, the cautionary notes of this analysis apply more for smaller studies, or studies that use the MMSE alone as a primary outcome. Our results clearly highlight that the MMSE and other low ceiling measures are not optimally designed to measure change within cognitively normal populations.

The MMSE is more appropriate in studies involving participants with baseline scores far below ceiling performance, and who tend to show more substantial declines. For participants who score near the ceiling at baseline, the MMSE is effectively blind to cognitive change unless that change is both large and negative (and even then, that change is underestimated). Therefore, researchers using measures with low performance ceilings like the MMSE in clinical trials or other longitudinal studies should carefully consider the characteristics of their sample before selecting such instruments. While the MMSE’s short administration time and ease of use are appealing, these advantages come at a cost: measures with low performance ceilings can significantly attenuate or even bias effect size estimates, leading to reduced statistical power and, potentially, inaccurate inferential conclusions.

To our knowledge, this study is the first to use IRT to directly examine the measurement precision of multidimensional latent change scores. However, in a conceptually similar approach, Ho et al. (2023) used unidimensional IRT models to simulate plausible values—values drawn from the posterior distribution of latent trait estimates—for NIH PROMIS Physical Function scores at two time points. They then calculated differences between these simulated scores across multiple draws to generate a sampling distribution of change. Like the present study, they found that the precision of change estimates varied systematically across the latent trait continuum. Previous research has also examined RCIs from the perspective of CTT versus IRT (Jabrayilov et al., 2016; Reise & Haviland, 2005; Tang et al., 2023). However, these applications did not directly quantify the latent change score or its standard error. Instead, latent change scores are typically estimated separately at each time point, the difference scores are computed, and the standard error of the differences is approximated by summing the squared standard errors of each score. We argue that this approach is suboptimal for assessing reliable change. Instead, change statistics should be derived directly from the estimated latent change scores and their associated standard errors (or from propensity distributions as in Ho et al.). Doing so in the present study revealed directly why the MMSE provides poor-quality estimates of change for cognitively normal individuals and is generally ill-advised for use in clinical trials or other types of longitudinal research that do not focus on impaired populations.

It should be noted that several factors limit the generalizability of these findings, including characteristics of the sample and the context in which the assessments were administered within a clinical trial. While item parameters are typically assumed to be invariant across populations, numerous studies have demonstrated that demographic, social, cultural, and clinical characteristics can influence item behavior (e.g., by altering item difficulty or discrimination). Context also plays an important role. The MMSE is widely used in research settings but even more commonly administered in everyday clinical practice, and these differing contexts can affect how participants interpret and respond to items. While the core arguments of this paper are broadly applicable, the degree of match, or mismatch, between a test and the individual being assessed will inevitably vary across groups and individuals. Additionally, as mentioned above, this study is the first to use IRT to directly examine the measurement precision of multidimensional latent change scores. Although the principles of the model are based on a previous cross-sectional application that examined changes in performance between test conditions, the extension to longitudinal data is novel, and additional work is needed to understand the strengths and limitations of this approach.

Despite these concerns, this study raises serious questions about the use of screening measures as outcomes in clinical trials and other longitudinal studies. Whether the concern is ameliorated by screening measures with less pronounced ceiling effects, such as the Montreal Cognitive Assessment (MoCA) and the Saint Louis University Mental Status Exam (SLUMS), is unclear, but this is an important point for future investigation. Although the focus of this study is on research applications, our findings also have implications for clinical practice. Screening measures like the MMSE are often used by clinicians to monitor patients over time. Our results suggest that such measures may fail to detect, or imprecisely capture, early signs of cognitive decline in higher-functioning patients. Clinicians should therefore be cautious when interpreting changes, or lack thereof, over time in patients with near-ceiling performance, as these may reflect measurement error rather than true cognitive change. Additionally, while the RCI can be used at the individual patient level to quantify confidence in change, the commonly used formula relies on a single reliability value. It therefore does not account for the fact that patients with higher baseline cognitive functioning are measured less precisely than those with lower baseline ability. Caution is warranted when interpreting measures poorly matched to a patient’s ability level.

Public significance statement:

This study shows that the MMSE produces imprecise and biased estimates of change and should not be used to measure outcomes in clinical trials and other longitudinal studies involving higher-functioning persons. It also suggests that screening measures should be used cautiously in clinical care. Finally, the study presents a novel IRT-based approach for evaluating the precision of change scores, which can be applied to other measures.