The influence of cerebrospinal fluid (CSF) biomarkers on clinical dementia evaluations

Abstract

Background

Cerebrospinal fluid (CSF) proteins have become accepted biomarkers for Alzheimer’s disease (AD) in research settings. The extent of their use, perceived utility, and influence on decision making in clinical settings, however, is less well studied.

Methods

Clinicians who evaluate older adults (N = 193) were randomized to view normal, borderline, AD-consistent, or no CSF information in two vignettes portraying patients with borderline and mild AD symptoms. Clinicians also reported on the use and perceived utility of CSF biomarkers.

Results

Even though clinicians reported infrequent use and low perceived utility of CSF biomarkers, viewing AD-consistent CSF values made clinicians more likely to make an AD-related diagnosis, increased diagnostic confidence, and led clinicians to initiate treatment more often than clinicians who had no CSF information.

Conclusions

CSF biomarkers influence decision making depending on the extent to which biomarkers reflect AD pathology, consistency between clinical-pathological information, and the ambiguity of protein values.

1. Introduction

Alzheimer’s disease (AD) is the most common cause of dementia in the United States, expected to affect nearly 14 million people by the year 2050.¹ AD is characterized clinically by cognitive impairment and functional decline and pathologically by the presence of neurofibrillary tangles and amyloid plaques.² Patients with memory complaints suggestive of AD are assessed for a history of intraindividual cognitive decline, preferably with observations from a collateral source, and for objective evidence of cognitive impairment by mental status or neuropsychological testing.³ In the research setting, cerebrospinal fluid (CSF) biomarkers have been used to identify the molecular pathology of AD. Reduced levels of the protein amyloid-beta₄₂ (Aβ₄₂) and elevated levels of the protein tau or its phosphorylated isoform (ptau₁₈₁) are suggestive of underlying AD pathology.⁴ Levels of these two proteins and their ratios to one another have been used to discriminate among different neurodegenerative dementia etiologies,⁵ to predict rate of AD progression,⁶ and to track pathological changes in clinical trials.⁷ However, although recent guidelines have suggested that CSF biomarkers may be used in support of the clinical diagnosis of AD in practice settings,³ there is limited information regarding how clinicians actually use CSF biomarkers to make clinical decisions.⁸ We sought to evaluate the influence of CSF biomarkers on diagnostic and clinical decision making.

The role of CSF information in clinical dementia assessment is evolving. Previous studies have found that CSF biomarkers correlate with AD-related structural brain changes⁹ and are reliable predictors of risk of developing AD dementia among individuals with few overt symptoms (preclinical)¹⁰ and with mild cognitive impairment (MCI).¹¹ However, CSF biomarkers have been largely confined to research settings for at least three reasons. First, CSF protein values have been shown to vary due to laboratory standardization issues, making biomarker information potentially unreliable. To address this problem, the Alzheimer’s Association quality control program, established to examine ways to increase protein measurement reliability, recently published a study suggesting that locally standardized procedures could increase the utility of CSF measures.¹² Second, CSF biomarkers potentially identify pathological processes in individuals with presumptive preclinical AD, hypothesized to be present for 10–20 years before the manifestation of cognitive symptoms,¹³ thus raising ethical concerns about the disclosure of biomarker information given the lack of treatment options to prevent symptom onset. At the same time, early reliable diagnosis of AD pathology will be necessary for preventive treatment and could give comfort to patients and families who are eager to understand possible causes of cognitive decline. Finally, given that CSF protein measurements are continuous measures that require some interpretation, it is unknown how clinicians might use this information outside of controlled research settings.

While previous studies have examined the predictive power of CSF biomarkers in clinical settings,^14–16 little research has been done to evaluate how clinicians use CSF biomarkers to make decisions in clinical practice. Kester and colleagues¹⁷ studied physician diagnosis and diagnostic confidence in a small sample of patients with suspected AD in a hospital memory clinic in the Netherlands, finding that AD-consistent CSF biomarkers increased diagnostic confidence but only led to a change in diagnosis in 10% of cases. In contrast, a recent multicenter study in France of physician diagnosis before and after viewing CSF biomarker results found that clinicians tended to alter diagnosis according to CSF information.¹⁸ Given limited and mixed information in the literature, we sought to examine the current use and influence of CSF biomarkers in combination with other clinical details on clinicians’ diagnostic and treatment decisions for patients with memory complaints. In this study, we presented clinicians with two clinical vignettes to illustrate an ambiguous and a mild AD dementia presentation in a clinic. Vignettes are a valuable, cost-effective research method for understanding professional judgment of multiple clinical factors while mirroring plausible, real-world scenarios.¹⁹ Given previous studies about the utility of CSF information in research settings^{4–7,9–11,18}, we hypothesized that biomarker values would influence dementia diagnosis, diagnostic confidence, and treatment recommendations.

2. Method

2.1 Participants

Recruitment

Physicians (MD and DO), nurse practitioners and advanced practice nurses (NP and APRN), and physician assistants (PA) were eligible for the study if they evaluate patients over age 65. We targeted academic and nonacademic licensed clinicians in the United States and recruited from primary care, internal medicine, neurology, geriatrics, and geriatric psychiatry. Potential participants were contacted between January and July, 2013, using publicly available e-mail addresses on university and medical center websites and professional organization e-mail lists. Although several hundred e-mails were originally sent, because initial contacts potentially did not meet inclusion criteria, it is not known how many eligible potential participants were contacted. Informed consent was obtained from all participants in the study, which was approved by the Washington University Human Research Protection Office.

Sample size

Based on a desired power of .95, alpha set at .05, and a conventional medium effect size of .3,²⁰ G*Power²¹ suggested a required sample size of 191. Out of 291 individuals who began the questionnaire, 248 respondents were eligible to participate and were randomized into conditions in the study. A total of 193 participants completed the entire questionnaire, while 55 individuals partially completed the questionnaire. Completers and partial completers were statistically similar in terms of age, years in clinical practice, approximate percentage of patients seen over the age of 65, practice specialty, and practice setting.

2.2 Materials

The study design and materials were developed by a team of investigators representing neurology, nursing, clinical psychology, and social work. We pilot tested the questionnaire with 10 clinicians at the Knight Alzheimer’s Disease Research Center and revised it for clarity based on their feedback. The questionnaire took approximately 10 minutes to complete, and was hosted by Qualtrics, a secure online survey platform.

Demographic and practice questions and randomization

At the start of the questionnaire, participants completed a series of demographic and professional background questions. Participants also responded to clinical practice questions (i.e., how often they collect and how useful they find various diagnostic tests for cognitive impairment). If they met inclusion criteria, participants were randomized into one of four experimental groups according to the type of CSF information they viewed with each of two clinical vignettes, consistent with a 2 × 4 within-subjects factorial design. Participants in groups 1, 2, and 3 were shown normal, borderline, or AD-consistent CSF values, respectively, with each vignette, while participants in group 4 were not shown any CSF information.

CSF education

We developed a one-page education sheet outlining the clinical use of CSF biomarkers in identifying AD pathology (see Supplement A). This information was presented directly before the two clinical vignettes and contained sensitivity and specificity information for Aβ₄₂, tau, ptau₁₈₁, and the ratio between Aβ₄₂ and tau, which usually provides the best classification information for individuals with and without AD pathology.¹⁰ The education sheet cited limitations of these data, specifically noting that CSF values might reflect underlying AD pathology even in cognitively normal individuals.^9,16

Clinical vignettes

The two vignettes included information about patient age, gender, memory, functional status, mood, subjective complaints, an informant report, and CSF information for groups 1–3 (see Supplement B). One vignette described a borderline or unclear case with ambiguous presenting symptoms, and the second described a patient with symptoms consistent with mild AD dementia. Embedded in the vignettes were Aβ₄₂, tau, ptau₁₈₁, and ratio values consistent with normal, borderline, or AD presentations for participants in each of the three CSF conditions. The order of the vignettes was counterbalanced across participants.

After each vignette, participants chose a diagnosis from a list of six options (normal/no diagnosis, MCI due to unknown causes, MCI due to AD, AD dementia, memory loss due to uncertain causes, or depression); rated their diagnostic confidence on a scale from 1 (not at all confident), 3 (moderately confident), to 5 (very confident); and indicated their recommendation for treatment, if any, in an open-ended response. Next, participants were asked whether each clinical detail in the vignette (i.e., age, gender, memory, functional status, mood, informant report, CSF values) made them less or more confident in their diagnosis on a 5-point scale: 1 (less confident), 3 (neither less nor more confident), 5 (more confident). Finally, participants were asked in an open-ended question what additional clinical details they would have liked in order to evaluate each case. While answering these questions, participants were able to view the vignette and CSF education sheet.

2.3 Data analysis

Statistical analyses were conducted in three phases using SPSS version 21 (IBM, Armonk, NY). First, descriptive statistics of demographic and practice information were calculated to characterize the sample. Second, chi-squares, t-tests, and analyses of variance were conducted to evaluate whether CSF information was related to diagnostic choices, diagnostic confidence, and treatment plan. Finally, a series of logistic regressions was conducted to model multivariate associations between clinician diagnosis and CSF group assignment, demographic and practice variables, and clinical detail ratings.

3. Results

3.1 Demographic and practice information

Table 1 summarizes demographic information for the sample. Respondents were mostly physicians (90%), although the non-MD clinicians did not differ from the physicians on any demographic or practice characteristics and were therefore included in the final sample to represent the diversity of clinicians treating AD. CSF biomarkers were rated low in terms of their use and perceived utility, relative to other diagnostic tools (see Table 2).

Table 1.

Participant Characteristics (N =193)

Characteristic	M/n	SD/%
Sex
Male	102	53
Female	84	44
Unspecified	7	3
Age	50.98	10.80
Race/Ethnicity
White	153	79
Asian	22	11
Hispanic, Latino, Spanish	6	3
Black or African American	4	2
Other	8	6
Degree
Medical Doctor (MD)	175	90
Doctor of Osteopathy (DO)	7	4
Nurse Practitioner (NP)	7	4
Advanced Practice Nurse (APRN)	2	1
Physician Assistant (PA)	2	1
Practice specialty
Neurology	93	48
Geriatrics	59	31
Internal medicine	15	8
Primary care	14	7
Psychiatry	10	5
Unspecified	2	1
Practice setting
University/Academic medical center	147	76
Veterans Administration Hospital	19	10
Clinic	13	7
Solo private practice	6	3
Hospital	3	2
Nursing home/Long term care	3	2
Unspecified	2	1
Percentage of patients >65 years (%)	73.13	23.02
Years in clinical practice	19.42	11.56

Note. The four experimental groups do not differ on any participant characteristic variables except for physician age (Normal CSF, M = 54.51 vs. no CSF, M = 48.36, p = .02) and years in clinical practice (Normal CSF, M = 23.35 vs. no CSF, M = 17.75, p = .037).

Table 2.

Frequency of Use and Utility of Diagnostic Tests (N = 193)

Test	How often used M (SD)	How useful M (SD)
Cognitive screeninga	4.59 (0.81)	4.10 (0.94)
Comprehensive cognitive testing	3.17 (1.29)	4.00 (1.17)
Structural neuroimagingb	3.87 (1.23)	3.24 (1.16)
Metabolic neuroimagingc	1.64 (0.73)	2.45 (1.30)
CSF analysis	1.56 (0.76)	2.16 (1.25)

Note. All values are on a 1–5 Likert-type scale, 1 (Not at all), 3 (Moderately), 5 (Very).

^aFor example, Mini-Mental Status Examination (MMSE).

^bFor example, Magnetic Resonance Imaging (MRI).

^cFor example, Positron Emission Tomography (FDG PET).

3.2 Do particular CSF protein values influence clinical decisions?

In order to examine the effect of CSF information on diagnostic decisions, we consolidated diagnostic categories to eliminate small cell sizes and to reflect our interest in clinician choice of underlying etiology (AD-related or unknown etiology) rather than in diagnostic labels that may vary across practice specialty or setting.

For both vignettes, the diagnosis chosen by clinicians depended, in part, on the CSF values presented (see Figure 1). When given normal CSF values, few clinicians made an AD-related diagnosis (19.4% for the borderline vignette, 27.3% for the AD dementia vignette). The proportion of clinicians assigning a diagnosis related to AD was higher when they received borderline CSF values (41% for the borderline vignette, 47.7% for the AD dementia vignette), and higher still when they received AD-consistent CSF values (77.5% for the borderline vignette, 86% for the AD dementia vignette). Clinicians who did not receive CSF information assigned diagnoses in equal numbers for the AD dementia vignette, while most clinicians (78%) diagnosed unknown etiology for the borderline vignette. The results of a binary logistic regression revealed that for both vignettes, viewing AD-consistent CSF values led to significantly increased odds of choosing an AD-related diagnosis, while for the AD dementia vignette only, viewing normal CSF values led clinicians to make fewer AD-related diagnoses compared to clinicians who did not have any CSF information (see Table 3).

Figure 1.

Percentage of clinicians choosing diagnostic categories by CSF condition for the borderline and mild AD dementia vignettes. Adjusted standardized residuals, signifying difference from expected chi-square distribution, are significant at the ***p < 001 level.

Table 3.

Logistic Regression of Significant Predictors of Diagnostic Choice (AD Etiology versus Unknown Etiology)

Predictor	B	SE B	Exp(B)	95% C.I. Exp(B)
Borderline vignette
Normal CSF group	−0.16	0.54	0.86	0.30 – 2.48
Borderline CSF group	0.90	0.47	2.47	0.98 – 6.22
AD CSF group	2.50	0.51	12.21***	4.50 – 33.17
AD dementia vignette
Normal CSF group	−1.16	0.46	0.31*	0.13 – 0.78
Borderline CSF group	−0.10	0.43	0.91	0.39 – 2.12
AD CSF group	1.53	0.54	4.60**	1.61 – 13.14
Vignette informant report	1.16	0.29	3.17***	1.81 – 5.56

Note. Nonsignificant predictors removed from logistic regression: Years in clinical practice, percentage of patients over the age of 65, practice specialty, vignette diagnostic confidence.

^*p < .05,

^**p < .01,

^***p < .001

Regarding diagnostic confidence, there was a significant effect of CSF group on confidence ratings for both vignettes (for the borderline vignette, F(3,188) = 4.05, p = .008; for the AD dementia vignette, F(3,189) = 5.66, p = .001). On the borderline vignette, clinicians who received AD-consistent CSF values were significantly more confident in their diagnosis compared to clinicians who did not view CSF information (see Table 4). On the AD dementia vignette, clinicians who received AD-consistent CSF values were more confident in their diagnosis than clinicians who viewed borderline CSF values or no CSF information.

Table 4.

Diagnostic Confidence Ratings (N = 193)

	Vignette
CSF Condition	Borderline M (SD)	AD M (SD)
No CSF	3.16a (0.72)	3.24cd (0.80)
Normal CSF	3.28 (0.83)	3.55 (0.78)
Borderline CSF	3.25 (0.81)	3.38c (0.72)
AD CSF	3.67a (0.75)	3.86d (0.80)

Note. All values are on a 1–5 Likert-type scale of diagnostic confidence, 1 (Not at all), 3 (Moderately), 5 (Very). Values that share subscripts are significantly different at the p < .05 level.

There was a significant effect of CSF values on the decision to treat for the borderline vignette, χ²(3, N = 161) = 13.44, p = .004, but not for the AD dementia vignette, χ²(3, N = 161) = 6.61, p = .086. Clinicians who received AD-consistent CSF were significantly more likely to recommend treatment than those who received borderline or normal CSF values (48.5% versus 35% and 27.3%, respectively; z = 2.7, p < .01).

3.3 Confidence ratings of clinical details and preference for additional tests

Post-vignette ratings of CSF utility differed across CSF groups (for the borderline vignette, F(2,132) = 5.99, p = .003; for the AD dementia vignette, F(2,130) = 8.37, p < .001). Tukey post-hoc tests showed that clinicians viewing AD-consistent CSF values rated that information more useful than clinicians who received borderline or normal CSF values. Taken together, these results indicate that viewing AD-consistent CSF values was associated with a similar level of diagnostic confidence compared to the presence of other common clinical tools and that AD-consistent CSF values were rated as more helpful than borderline or normal CSF values.

Finally, clinicians indicated which additional clinical tools they would have found useful in formulating a diagnosis. For both vignettes, 25–30% of clinicians requested neuroimaging, neuropsychological testing, and laboratory information such as blood analysis. Fewer clinicians (approximately 15%) indicated that brief neuropsychological screening or a depression evaluation would have been useful. Clinicians also described a wide variety of additional information they would have collected. Some clinicians indicated that additional testing was required to arrive at a diagnosis (e.g., “Really must have cognitive testing as well.”). Others wrote that they would like longitudinal follow-up information as well as medical history.

4. Discussion

This study is among the first to examine how clinicians use CSF biomarkers in combination with other clinical information to diagnose cognitive impairment. Clinicians responded to clinical practice questions and evaluated two case vignettes describing a patient with ambiguous clinical symptoms and a patient with a more typical yet still mild AD dementia phenotype. While clinicians reported infrequent use and limited utility of CSF biomarkers in their current clinical practice, viewing CSF information in the study vignettes nonetheless influenced their diagnosis, diagnostic confidence, and decision to treat. Taken together, these results highlight the influence of CSF biomarkers on clinical decisions, even when that information is weighed alongside other clinical details routinely reported as more useful in clinical practice. Examining these findings more closely, a number of trends are apparent.

First, as CSF values changed to reflect AD pathology, clinicians made AD-related diagnoses with increasing frequency and with greater confidence (see Figure 1), even while other clinical details remained constant. Clinicians who viewed normal CSF values tended to make diagnoses with unknown etiology, while most clinicians who viewed AD-consistent CSF values made AD-related diagnoses. Furthermore, the proportion of clinicians choosing to initiate treatment on the borderline vignette rose as CSF values increasingly pointed to AD pathology (27.3%, 35%, and 48.5% for normal, borderline, and AD CSF values, respectively). Finally, clinicians who viewed AD-consistent CSF values reported the highest diagnostic confidence compared to clinicians who viewed other types of CSF data. As such, AD-consistent CSF biomarkers may exert a confirmatory effect relative to more familiar, established clinical details with which clinicians currently have more experience.

Second, while current practice guidelines do not endorse the use of CSF biomarkers, in part because of the potential for indeterminate protein values,³ our results show that ambiguous CSF values (i.e., borderline values) have little effect on clinical decisions. Zetterberg and colleagues⁸ cite indeterminate biomarkers as a challenge to clinicians who may view CSF information as a definitive and objective diagnostic test, without considering the entire clinical picture. The results of this study suggest that when presented with ambiguous or indeterminate CSF values, clinicians rely on other clinical details to make diagnostic and treatment decisions.

Finally, inconsistency between pathological and clinical information affects clinical decisions. Clinicians who viewed normal CSF values in a vignette that included other clinical details consistent with mild AD dementia most often chose a diagnosis with unknown etiology. In contrast, clinicians who viewed AD-consistent CSF values in a vignette that included other ambiguous clinical details were swayed by those CSF values and made an AD-related diagnosis. This finding is consistent with a recent study by Mouton-Liger and colleagues, who report that physicians in specialty memory clinics relied on CSF biomarkers when clinical and pathological information was mismatched.¹⁸ Taken together, in the context of borderline clinical details, AD-consistent CSF values are likely to hold more weight for diagnosis and treatment planning, whereas the same values appear to have less effect on clinical decisions in cases with other signs pointing to mild AD dementia.

These findings suggest that clinicians do not view CSF values in a vacuum, but instead consider their utility in combination with other clinical details to make decisions. Clinician reports on the use and the utility of clinical tools shed light on how evaluations are currently conducted, and how they might evolve as CSF testing becomes more widespread in clinical settings. Overall, reported use of neuropsychological screening and testing, neuroimaging, and CSF results were consistent with current practice guidelines for evaluating patients with memory complaints.³ However, there were variations in ratings of the use and utility of each tool and significant differences between practice specialties, suggesting the lack of standardization in dementia assessment. A recent study examining Alzheimer’s Disease Neuroimaging Initiative (ADNI) investigator attitudes about the clinical value of research results for AD suggests that clinicians find amyloid imaging and CSF biomarker data as equally useful as more widely used clinical details such as psychometric testing and structural brain imaging.²² In contrast, low clinician ratings of CSF biomarker utility in the current study may reflect real-world financial concerns, including cost of CSF biomarker analysis and questionable insurance coverage. While the current study offers an initial examination of decision making biases when considering clinical and pathological information, future research should incorporate cost and insurability issues, as well as explore attitudes and practices in a wider variety of settings and disciplines.

Given that this study is among the first to examine how clinicians use CSF information with other clinical details to diagnose dementia, a number of limitations should be acknowledged. First, because we tried to reach a large diverse sample of clinicians working in a broad range of settings, it is unclear whether our results generalize to specific clinician populations. Furthermore, the majority of participants in our sample work in academic or research based settings. Future studies should examine knowledge and use of biomarkers among community practitioners who may have systematically different understandings of biomarker utility for AD diagnosis. Second, the vignettes were brief in order to encourage participation by busy clinicians and lacked the more extensive detail that clinicians might access in practice. While more detailed vignettes might more closely reflect real-world clinical scenarios, ambiguous vignettes allowed for the expression of variability in decision making that may be present in clinical settings.¹⁹ Future studies could include additional test results, examine how various combinations of results influence clinical decision making, and explore how the same clinician values different CSF information in various clinical contexts. More extensive vignettes comprised of formal diagnostic criteria could allow for more complete understanding of the function of CSF information within current practice guidelines. Furthermore, future studies could incorporate factors likely to be present in deciding to order CSF testing, such as cost and patient willingness. Finally, we were not able to examine the effect of CSF values on specific diagnoses due to small cell sizes, nor did this study assess the utility of biomarkers in differential diagnosis across the dementias.

Despite these limitations, this study suggests that CSF values influence clinical decisions, even while clinicians do not view them as especially useful. CSF information is likely to become more widely used in clinical practice in the years ahead, particularly should clinicians attempt to identify preclinical cases if opportunities for early intervention become available. More research is clearly needed to understand how biomarkers are used by different clinicians, and whether changes could be made to clinician training or clinical practice guidelines to offer more specific guidance on how biomarkers should be used to improve diagnostic reliability. This study represents a first step in exploring the potential role of CSF biomarkers in clinical evaluations, and in understanding how clinicians integrate clinical and pathological information to make clinical decisions.