A comparison of tau and 14-3-3 protein in the diagnosis of Creutzfeldt-Jakob disease

Abstract

Objective:

To compare the respective efficiency of CSF tau (quantitative) and CSF 14-3-3 protein (qualitative) in the diagnosis of prion disease.

Methods:

We made measurements on 420 live subjects, who subsequently underwent a postmortem neuropathology examination, including protein chemistry, immunohistochemistry, and histology. We performed tau by ELISA. We detected 14-3-3 protein by Western blot. Both assays were optimized for maximum efficiency (accuracy).

Results:

We found tau and 14-3-3 proteins to be closely correlated, but tau had a significantly better ability to predict disease status than 14-3-3 protein. Also, tau distinguished disease status at least as well as when both assays' results are combined in a variety of ways. Importantly, the area under the receiver operating characteristic curve for tau (0.82) was significantly larger than that for 14-3-3 protein (0.68) (p < 0.001). Diagnostic test statistics are provided for the study subjects with 58.3% prevalence, and for a more typical, nonselected, 7.5% prevalence as received by our center.

Conclusion:

In this study, tau is superior to 14-3-3 protein as a marker in the diagnosis of Creutzfeldt-Jakob disease, and is as efficient singly compared to a variety of combinations with 14-3-3 protein. This is the first study of this magnitude to examine prion disease diagnostic tests in a carefully characterized patient population with detailed statistical evaluation.

Accurate and possibly early markers are increasingly important in prion diseases which often present a rapid course and, because of their infectivity, require special biosafety precautions during clinical management and tissue examinations.^1,2 The CSF 14-3-3 protein is widely used as a surrogate marker in the premortem diagnosis of Creutzfeldt-Jakob disease (CJD) and related prion illnesses.^3–7 The test is usually performed by immunoblot, but is not quantitative. This leads to a significant percentage of the 14-3-3 tests being excluded as ambiguous. ELISA has been proposed for the quantitative determination of the 14-3-3,⁷ although the lack of a commercial kit has limited this approach.

Most recent studies report 14-3-3 protein sensitivity in the range of 43%–100%.^7–11 The specificity is stated to range from 47% to 97%.^7,12,13

Other CSF markers have been proposed along with 14-3-3 protein, including S-100 protein, neuron-specific enolase, or tau.^12,14 Unfortunately, the first 2 of these do not have sufficient sensitivity or specificity.^14–16 In contrast, total tau protein is reported to be clinically useful.^12,17

The purpose of this study was to establish the diagnostic test statistics for tau, including the Bayesian concept of diagnosticity given by the likelihood ratio for comparison with 14-3-3, with a defined patient population and sized to be statistically valid. Also, we intended to establish the most efficient combination of tau and 14-3-3 protein for the diagnosis of prion disease.

METHODS

CSF specimens were referred to the National Prion Disease Pathology Surveillance Center (NPDPSC) from US medical institutions for patients with CJD or other prion diseases in the differential diagnosis. Over a 27-month time period, tau and 14-3-3 protein were measured on 5,496 samples (tau: 812 positive, 4,684 negative; protein 14-3-3: 1,157 positive, 2,048 negative, 2,291 ambiguous), but tau values were not reported. Both assays were optimized for maximum accuracy (efficiency). During an additional 21 months, deaths followed by neuropathology completed our study population. Genetic analysis was performed to assess the polymorphism at codon 129 and detection of pathogenic mutations in the open reading frame of the prion protein gene (PRNP).^18,19 The decision to request an autopsy was made by the decedent's legal next of kin as recommended by the decedent's clinican and was usually preceded by an extensive neurologic evaluation that included 14-3-3 protein.

We measured 14-3-3 protein by Western blot with a detection polyclonal antibody (sc-629, Santa Cruz Biotechnology, CA). We judged blots negative, ambiguous (weakly positive), or positive according to the density of the 14-3-3 immunoblot bands. We measured total tau by ELISA following an initial 1:10 dilution (Biosource; now Invitrogen, CA).

The prevalence of prion diseases in our population of 420 autopsy-verified patients was 58.3%. We estimated the prevalence of confirmed prion disease among all patients tested during the same time period to be 7.5%.

More details of the methods, statistical analyses, and results are presented in appendices e-1 to e-3 on the Neurology® Web site at www.neurology.org.

Standard protocol approvals, registrations, and patient consents.

Case Western Reserve University institutional review board approved the use of human subjects and discarded tissues for this study via protocol 01-95-01.

RESULTS

Tissue-verified prion positive cases.

A total of 420 of the 5,496 patients underwent autopsy, which definitely confirmed the diagnosis of prion disease in 245 (58.3%). None of the prion-positive cases had a history of prion exposure during medical or surgical treatments, making the possibility of iatrogenic prion disease unlikely; none had the features of variant CJD.²⁰

Individual CSF examinations: 14-3-3 protein.

We conducted 14-3-3 tests according to our standard procedure, which identifies the 3 diagnostic categories of positive, negative, and ambiguous. Semiquantification was extended by splitting the ambiguous category based on the median density (∼300 ng/mL), thereby providing 4 categories.

Individual CSF examinations: tau protein.

The amount of tau (pg/mL) determined in the 420 cases, which varied from 0 to 140,000 pg/mL, required the choice of an appropriate decision point. Plotting the diagnostic capabilities of each test as receiver operating characteristic (ROC) curves indicated that 900–1,200 pg/mL was an optimal range to separate negative from positive cases for tau, determined by the region of maximum change of curve slope (figure 1). We also analyzed the tau test with the histogram of the distribution of positive and negative prion disease cases as a function of the amounts of tau detected in the individual CSF samples (figure 2). This histogram pointed to a maximally efficient decision point at 1,150 pg/mL. The suitability of the 1,150 pg/mL decision point was further confirmed by comparing the tau test predictive values at this decision point with those at 900 pg/mL, 1,000 pg/mL, and 1,400 pg/mL (table 1). The ROC curve of the tau tests was also compared with that of the 14-3-3 tests. The area under the curve (AUC) ± SE of the ROC curve for tau was 0.819 ± 0.020 vs 0.672 ± 0.022 for the 14-3-3 test (p < 0.001) when the 4 categories (ambiguous tests assigned to the negative or positive category) were adopted (figure 1). At 7.5% prevalence and a 1,150 cutoff, the diagnosticity given by the likelihood ratio equal to sensitivity/(1 −specificity) for tau is 2.67, while for 14-3-3 protein, with ambiguous considered negative, it is 1.78.

Figure 1. Receiver operating characteristic curves for tau and 14-3-3 protein.

Four categories (negative, ambiguous negative, ambiguous positive, and positive) are used for 14-3-3 protein. The area under the curve ± SE for tau was 0.819 ± 0.020 vs 0.672 ± 0.022 for the 14-3-3 test. The thin green line represents a test that does not change predictive ability.

Figure 2. Histogram of distribution of tau values (log) per 0.1 log unit.

Positive Creutzfeldt-Jakob disease (CJD) cases (blue triangles) and negative cases (red squares) are shown. The log scale is used for clarity with tau values from 10 to 140,800 pg/mL. The most accurate cutoff of tau between cases positive and negative for CJD is 1,150 pg/mL. The prion disease-positive subjects were distributed as follows: 247 with tau at or above 1,150 pg/mL, 24 between 1,149 and 800 pg/mL, and 149 below 800 pg/mL. BN represents the level below which 10% of positive cases fall. BP represents the level above which 10% of negative cases fall.

Table 1.

Tau test statistics at different decision points

Abbreviations: NPV = negative predictive value; PPV = positive predictive value.

Effect of disease progression on tau test.

The mean amount of tau detected decreased progressively at increasing time intervals between CSF withdrawal and disease onset spanning from over 14,600 pg/mL, at an interval of less than 43 days from clinical onset (highest quintile), to 2,030 pg/mL at more than 200 days (lowest quintile).

Neuropathology of the false-positive and false-negative cases with the tau test.

Conditions among the 57 false-positive cases with adequate neuropathologic examination included neurodegenerative diseases such as Alzheimer disease, which was the most common diagnosis, multiple infarcts in 11 cases, and brain neoplasms in 4 cases (table 2). Moreover, in 5 of these cases the CSF was contaminated by blood, which in itself is likely to lead to a false-positive result.

Table 2.

Neuropathologic diagnoses of tau false-positive cases

^aAll cases with multiple ischemic lesions more than 1/2 cm except for 2 with diffuse ischemia.

^bTwelve cases of Alzheimer disease, 2 cases of cerebellar degeneration, 1 case of thalamic degeneration with mild cortical involvement, and 1 case of frontotemporal dementia.

^cOne case of granulomatous encephalitis, one case of Cryptococcus meningoencephalitis (both with multiple areas of necrosis), one case with large and destructive perivascular lymphocyte infiltrates, and one case of unspecified encephalitis with infarcts.

No special cause for the 32 false-negative cases emerged except for the prion disease being familial (3 cases) and blood contamination (1 case). Technical reasons such as long storage before testing could not be verified because collection date of specimen was not always provided by the sender. However, both tau and 14-3-3 protein tend to increase with disease progression. Of the 32 false-negative cases, 9 were associated with a second specimen. Had the last specimen been used, there would have been 3 fewer false-negative cases.

Effect of blood presence in CSF.

Blood is a contaminant sometimes found in received CSF samples. Blood was detected in the CSF from 13 confirmed CJD and 20 non prion cases. Incorrect results were 19 for 14-3-3 protein (18 FP/1 FN) and 12 for tau (11 FP/1 FN). Therefore, although the number of blood contaminated samples is small, the presence of blood in the CSF increases the number of false-positive cases in both tests.

Accuracy of tau and 14-3-3 tests (ambiguous cases excluded) according to prion disease molecular characteristics and prion disease form and subtype.

When the positive cases were examined according to the PrP genotype at codon 129 and the PrP^Sc type, valine homozygosity (VV) and PrP^Sc type 1 had the highest sensitivity with the tau test (100% and 95.5%) while methionine/valine (MV) heterozygosity and the presence of both PrP^Sc types (type 1–2) had the lowest with 82% and 72%, respectively. Furthermore, VV homozygosity and presence of PrP^Sc type 1 were associated with the highest amounts of CSF tau (10,431 pg/mL and 14,515 pg/mL). Sensitivity of the 14-3-3 test was consistently higher than that of the tau test (see appendix e-2).

The diagnostic accuracy of tau was greater for “typical” than “atypical” cases (figure 3).

Figure 3. Diagnostic reliability of tau for “typical” and “atypical” cases.

The 2 receiver operating characteristic curves (typical and atypical) show greater area under the curve (AUC) for typical (0.886) vs atypical (0.750) cases (95% confidence interval for AUC for atypical is 0.667 to 0.833, which is lower than, and has no overlap with, the 95% confidence interval for typical, which is 0.840 to 0.932).

Combined use of the tau and 14-3-3 tests.

Our separate analysis of the 2 tests indicates that although both are useful tau is overall better than the 14-3-3 test. Next, we tried to combine the 2 tests in different ways to assess whether the combinations resulted in better predictive values than the tau test alone. Various ways, described in brief below, of combining tau protein and 14-3-3 protein were statically performed. A complete descriptive analysis is provided in appendix e-3.

An ambiguous 14-3-3 protein was linked to ambiguous tau (between 307 pg/mL and various higher decision points). Ambiguous 14-3-3 protein was taken to be dominant and additive to ambiguous tau, or replaced by numeric or ambiguous tau. Tau was next taken as dominant, except between 900 and 1,150 pg/mL, when the single decision point for 14-3-3 protein was used. This was followed by either test being required positive, with ambiguous 14-3-3 protein taken as positive, negative, or substituted by tau. Next, both tests were required to be positive, with ambiguous 14-3-3 protein taken as positive, negative, or substituted by tau.

None of the combinations is clearly superior to the tau protein test alone.

DISCUSSION

In 1998, the CSF 14-3-3 diagnostic test for prion diseases was adopted by the WHO.⁴ This led to increased scrutiny of the CSF 14-3-3 protein test with widely reported Bayesian statistics, and studies suggesting use of other CSF surrogate markers.^4–17 Many of these studies were conducted with relatively small numbers, because prion diseases are uncommon. Additionally, a substantial proportion of the cases were not subject to neuropathology. Our experience has shown that some cases meeting WHO criteria are shown not to have prion disease following neuropathologic examination.²¹ Conversely, others believed not to have prion disease are found to have it upon tissue examination.

A large multicenter study, considered to be one of the more useful, compared 14-3-3, tau, and other brain-derived proteins.²² The findings of this study differ from ours, perhaps for the following reasons: in our study case selection was based solely on 14-3-3 and tau tests having been performed during a 27-month period, followed by neuropathology on all study subjects during the same time period plus an additional 21 months. No cases were excluded and none were rejected provided the stated criteria were met. Only a single set of CSF tests were included, and were always performed on the first CSF specimen received. Some selection bias may exist in our study owing to 14-3-3 protein, but not tau, having been reported and available for inclusion in the decision process leading to neuropathology. This could have led to an overestimation of 14-3-3 sensitivity. As in the multicenter study, it is noted that a second lumbar test in negative cases would be of value.

The tested population for tau and 14-3-3 tests was representative of the general tissue-confirmed population examined at the NPDPSC judged by the percentage of prion positive and negative cases, familial and sporadic cases of prion disease, and the 6 subtypes of the sCJD.

The tau test was less sensitive (87%) than the 14-3-3 test (90%) but had a significantly higher specificity (67/40%), and overall offered better accuracy with less ambiguous reports. The superiority of tau is supported by comparing the ROC curves and the diagnosticity, i.e., the likelihood ratios of the 2 tests. Although the sensitivity of our tau test as well as the decision point that we selected are similar to those reported in previous studies, the specificity was lower than that reported in the literature.^{12,13,23–25} The lower specificity values of our 14-3-3 and tau may be due to the relative representation in our patient population of the sCJD 1–2 cases (20.3%) which are associated with lower sensitivities, the relative high number of negative cases (41.7%), and the frequent lack of advanced diagnostic selection of our patient population which included many subjects often examined in nonspecialized centers where prion disease was one of several possible clinical diagnoses.

Unexpectedly, we found none of the combinations of tau and 14-3-3 protein was clearly superior to the tau protein test alone. However, equivalent performance was obtained with the combination requiring both tests to be positive and with tau replacing ambiguous 14-3-3 protein. The combination with 14-3-3 protein dominant and ambiguous replaced by tau or ambiguous tau became statistically superior at higher upper limits for ambiguous tau. This is achieved only with an unacceptably large number of excluded ambiguous tau cases.

Separately from the analyses based on the logic statements, the ability of the 2 assays to predict disease status, individually and jointly, was examined statistically using logistic regression modeling. It was found that tau protein alone has significantly better ability to predict disease status than the 14-3-3 protein alone. Further details can be found in appendix e-3.

The sensitivity associated with each of the 3 129 genotypes, PrP^Sc types 1, 2, or both types together (1–2), as well as in the individual subtypes of sCJD, consistently shows higher values for the 14-3-3 test (ambiguous values excluded) than tau. However, the variation of the values from one subtype to the other is similar with the exception of the 129 MV cases in which the 14-3-3 sensitivity is higher than in 129 MM cases, while the opposite is true for tau. Overall, the present findings are similar to those of previous European studies.^5,7,22 A study on the correlation between type and topography of brain pathology and predictability of CSF surrogate markers in sCJD has reported that 14-3-3 and tau tests correlate negatively with the severity of the spongiform degeneration and other lesions present in the cerebral cortex, and the degree of neuronal loss present in the thalamus. In contrast, positive correlations were observed with the severity of histologic lesions in the cerebellum and the degree of glial reaction in the basal ganglia.²⁶

Histopathologic examination led to a definitive diagnosis in 41 out of the 57 cases which received a false-positive diagnosis by the tau test (table 2). In at least 23 of these cases, including cases with multiple infarcts, leukodystrophy, cerebellar degeneration, encephalitis, and brain neoplasms, the brain lesions were detectable by MRI study. Were these 23 cases reassigned from false-positive to true-negative on the basis of MRI, the specificity and positive predictive value (PPV) of the tau test alone would have improved from 67.43% and 78.89% to, with combined tau and MRI examination, 80.57% and 86.23%, respectively. Similar findings were found with 14-3-3 protein. This suggests that the MRI examination might be carried out before or after requesting the tau and 14-3-3 tests, depending on availability and relative cost.

Tau and 14-3-3 protein testing and other surrogate tests are included along with clinical evaluation, EEG evaluation, and MRI testing during disease progression. None of these tests or procedures perfectly separate prion disease from other neurologic diseases. Only carefully performed neuropathology is believed capable of this segregation. However, each test or procedure should increase the clinical impression that prion disease is present when it is, and seem less likely when not present. The diagnosticity evaluates the ability of a test and is calculated as the likelihood ratio. In this study, the likelihood ratios for the tau test are better than those for the 14-3-3 protein test. Although diagnosticity values would change with prevalence and disease progression, the superiority of tau would not. Similar to the likelihood ratio is the ratio of the post-test PPV to the pretest PPV. After performing the 14-3-3 protein test, a positive test increases the probability of prion disease by a factor of 1.3–1.5. Tau, on the other hand, increases the probability of prion disease by a factor of 2.4 at 1,150 pg/mL and larger with higher tau values. More detail is presented in appendix e-4.

The cutpoint of an assay affects sensitivity, specificity, and predictive values. As the cutpoint is raised sensitivity, negative predictive value, and false-positives all decrease. Previous studies have used a higher tau cutpoint than we have, leading to a lower sensitivity.²² Our assay used a tau cutpoint of 1,150 pg/mL for maximum efficiency, but a lower cutpoint in a surveillance setting may be desired to minimize falsely rejecting patients with prion disease at a cost of more false-positives.

On the basis of this study, the value of tau as a diagnostic test for prion disease may have been underestimated. In this study, by ROC curve and diagnostic test statistical analysis, including Bayesian considerations, the tau test appears superior to the 14-3-3 protein test.

GLOSSARY

AUC

area under the curve

CJD

Creutzfeldt-Jakob disease

NPDPSC

National Prion Disease Pathology Surveillance Center

PPV

positive predictive value

ROC

receiver operating characteristic.

AUTHOR CONTRIBUTIONS

C. Hamlin serves as primary corresponding author and was responsible for the overview and writing of the manuscript. G. Puoti provided analysis of the supplementary data. S. Berri redesigned the database utilized for the study, provided oversight for case data and manuscript content, as well as contributed to data analysis. E. Sting redesigned the database, provided statistical and data analysis, and contributed to all final table and figure designs and content. C. Harris contributed to the original establishment of the database used for the study and the study design parameters and contributed to the data analysis. M. Cohen provided analysis of histopathological diagnostic evaluations. C. Spear assisted Dr. Cohen in preparation of histopathologic staining needed for diagnostic evaluations. A. Bizzi was the neuroradiologist who reviewed all relevant case MRIs of the study and provided input that was utilized in the Discussion. S. Debanne provided statistical analysis over case data and content of the manuscript. D. Rowland provided statistical analysis over case data and content of the manuscript.

DISCLOSURE

The authors report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.