Validation of MicroRNA Biomarkers for Alzheimer’s Disease in Human Cerebrospinal Fluid

Abstract

We previously discovered microRNAs (miRNAs) in cerebrospinal fluid (CSF) that differentiate Alzheimer’s disease (AD) patients from Controls. Here we examined the performance of 37 candidate AD miRNA biomarkers in a new and independent cohort of CSF from 47 AD patients and 71 Controls on custom TaqMan® arrays. We employed a consensus ranking approach to provide an overall priority score for each miRNA, then used multimarker models to assess the relative contributions of the top-ranking miRNAs to differentiate AD from Controls. We assessed classification performance of the top-ranking miRNAs when combined with apolipoprotein E4 (APOE4) genotype status or CSF amyloid-β₄₂ (Aβ₄₂):total tau (T-tau) measures. We also assessed whether miRNAs that ranked higher as AD markers correlate with Mini-Mental State Examination (MMSE) scores. We show that of 37 miRNAs brought forth from the discovery study, 26 miRNAs remained viable as candidate biomarkers for AD in the validation study. We found that combinations of 6–7 miRNAs work better to identify AD than subsets of fewer miRNAs. Of 26 miRNAs that contribute most to the multimarker models, 14 have higher potential than the others to predict AD. Addition of these 14 miRNAs to APOE4 status or CSF Aβ₄₂:T-tau measures significantly improved classification performance for AD. We further show that individual miRNAs that ranked higher as AD markers correlate more strongly with changes in MMSE scores. Our studies validate that a set of CSF miRNAs serve as biomarkers for AD, and support their advancement toward development as biomarkers in the clinical setting.

INTRODUCTION

Alzheimer’s disease (AD) is the most common form of dementia and the sixth-leading cause of death in the United States [1]. Total costs for health care, long-term care, and hospice for people with AD and other dementias were ~$259 billion in 2017 [2]. There is tremendous effort by many investigators to discover preventative therapies for AD, and to develop biomarkers that identify presymptomatic or preclinical cases of AD and monitor disease progression. Cerebrospinal fluid (CSF) serves as an excellent biofluid for biomarker studies in neuropathological diseases [3]. The most extensively studied CSF biomarkers include amyloid-β₄₂ (Aβ₄₂), total tau (T-tau), and phospho-tau, which are diagnostically useful, but do not track progression in the context of clinical trials [4]. However, the existence of extracellular RNAs (exRNAs) in virtually all biofluids has offered new potential for identifying diagnostic and/or prognostic markers for multiple human diseases [5].

ExRNAs have been described in the literature for over 40 years. In 1978, Stroun et al. showed that both DNA- and a pure RNA-nucleoprotein complex were released by human and frog cultured cells, and the exRNA release was an active mechanism that is unrelated to cell death [6]. The authors commented “whether exRNA is involved in intercellular transfer of specific information or has only an unspecific stimulating function cannot be answered at this stage”[6]. A decade later Benner proposed that “RNA as a short distance-short time messenger seems to be a good match of chemistry and biological function” [7]. Benner’s hypothesis arose from studies showing biological effects on extracellular actions by certain ribonuclease homologs, which “implied that the substrate for extracellular RNases, exRNA, must play a biological role in angiogenesis, neurological development, and other biological processes” [7]. In 1999, Kopreski et al. found tyrosinase mRNA in the serum of patients with malignant melanoma, even after the serum was filtered, indicating that the mRNA was extracellular and that “exRNA in plasma from cancer patients associates with or is protected in a multiparticle complex” [8]. Thus, exRNAs have great promise as biomarkers for diseases, including neurodegenerative diseases [9].

We previously reported that miRNAs in CSF from living donors can serve as candidate biomarkers for AD [10]. We identified a novel subset of 37 CSF miRNAs that were able to distinguish AD patients from Controls in a discovery cohort, based on n = 1 technical replicate/probe on the miRNA array. The 37 miRNAs include 20 that were verified by array in our laboratory and 17 additional miRNAs that were brought forth as candidate markers. The validation studies were performed using a custom TaqMan Low Density qPCR array comprised of n = 3 technical replicates/probe, in order to assess miRNA performance in a new and independent cohort of CSF donors. We generated miRNA profiles for all cohort samples, then used rigorous statistical approaches to rank the AD miRNA candidates. Our studies validated that 26 of the 37 CSF miRNAs identified in the discovery studies can differentiate AD patients from Controls, and combinations of miR-NAs increases classification performance for AD. We identified 14 of the 26 miRNAs as high-ranking markers. We then examined classification performance when the 14 miRNAs are added to current AD markers, apolipoprotein E4 (APOE4) genotype status and CSF Aβ₄₂:T-tau measures. The validation of these CSF miRNAs in a new and independent cohort now advances their consideration for development as biomarkers in the clinical setting, and for their use in bioinformatic studies to identify potential novel gene targets relevant to AD.

MATERIALS AND METHODS

Analytic pipeline

Figure 1 illustrates the analytic pipeline flow for the AD miRNA biomarker validation studies. The analytic pipeline included quality control processing of the miRNA array qPCR data, followed by statistical analysis of the miRNAs to evaluate their ability to correctly identify AD CSF in miRNA multimarker models, in combination with APOE4 genotype status, and in combination with CSF Aβ₄₂:T-tau measures. Further, we evaluated how the miRNA markers correlate with disease severity as represented by Mini-Mental State Examination (MMSE) scores.

Fig. 1.

Analytic pipeline for the AD miRNA biomarker validation studies. The analytic pipeline included quality control processing of the miRNA qPCR data, followed by statistical analysis of the miRNAs. First, we aligned and normalized the data, and then evaluated the classification performance of miRNA expression values in multimarker models, in combination with APOE4 genotype status and CSF Aβ₄₂:T-tau measures. Further, we examined the correlation of MMSE with miRNA expression values by miRNA rank.

Participants

The CSF samples used for the validation studies were obtained from the University of California, San Diego (UCSD) Alzheimer’s Disease Research Center (ADRC). All donor procedures were approved by the UCSD Institutional Review Board (IRB 80012). All participants provided written informed consent and underwent detailed evaluations consisting of medical history, physical and neurological examinations, laboratory tests, and neuropsychological assessments. Healthy Control subjects were recruited from the community through public lectures, newsletters, and word of mouth. Some participants were motivated by a family or spouse history of dementia, but others were simply motivated to contribute to research, and we have previously reported that research lumbar punctures are well tolerated and accepted even among healthy adults [11]. Cognitive health of the healthy volunteers was ascertained with MMSE [12] and clinical interview, and the absence of neurologic disease was confirmed by history and neurologic examination by a board-certified neurologist. The donors were matched by age and sex, to the extent possible.

CSF collection

CSF was collected from donors using protocols established by ADRCs. Lumbar punctures were done in the morning under fasting conditions, in the lateral decubitus position with a 24-gauge Sprotte spinal needle. The first 2 mL of CSF collected was used for clinical tests; samples with > 500 Red Blood Cells/microliter were excluded from the study. Subsequent 10–20 mL of CSF is collected from each donor and gently mixed. The CSF samples were centrifuged at 2000 g for 10 min at room temperature, then aliquoted into polypropylene tubes that include a subject number, but no other identifying information. The CSF aliquots were flash frozen on dry ice and stored at −80°C.

APOE genotyping

APOE genotyping was performed at the UCSD ADRC using PCR restriction fragment length polymorphism analysis, as described [13]. Genomic DNA was extracted and amplified using forward primer: 5′-ACGCGGGCACGGCTGTCCAAGGA-3′; and reverse primer: 5′-GCGGGCCCCGGCCTGGTACAC-3′. PCR products were Hha1 digested, ethidium bromide stained, electrophoresed, and visualized by UV illumination.

CSF Aβ₄₂ and T-tau measures

Measurements of CSF Aβ₄₂ and T-tau levels were performed at the UCSD ADRC using enzyme-linked immunosorbent assays (ELISAs) as previously reported [14]. CSF Aβ 1−42 was measured using the Euroimmun ELISA kit (EQ 6521–9601-L, ADx Neurosciences, Ghent, Belgium). CSF T-tau was measured using the ELISA kit (EQ 6531–9601-L, ADx Neurosciences).

RNA isolation and qPCR

We instituted safeguards to improve quality control of both the RNA isolations and the qPCR arrays based on our AD miRNA discovery studies [10]. First, we included a pooled CSF reference sample (RefStd) as a constant throughout the entire period of the qPCR studies. The RefStd was comprised of CSF donated from healthy community volunteers that was collected, pooled, and stored in 0.5 mL aliquots, as per the CSF collection protocol. RefStds were included in batches of RNA isolations and run together with patient CSF samples on the qPCR arrays to estimate and eliminate variance across processing batches and individual array cards. Cost considerations precluded placement of a RefStd on every card, so 13 RefStd samples were staggered at approximately even intervals throughout the 66-card study. Second, we included multiple miRNA controls on the arrays. Positive controls (miRNAs unchanged between AD and Controls) were combined to form a complex normalizer, while negative controls (miRNAs not found in CSF) were used to check the validity of each array card. In our initial AD discovery studies, we used only U6 small nuclear RNA (U6 snRNA) as a normalizer for the qPCR arrays [10]. For the validation studies presented here, each custom array card contained 51 RNA probes: 37 candidate AD miRNA biomarkers, 9 positive miRNA controls, U6 snRNA, and 4 miR-NAs not detected in CSF (Supplementary Table 1A) at n = 3 technical replicates/miRNA probe in order to add robustness to this study. Third, we imposed strict uniformity over the reagent manufacturing lots: all kit and reagent lots were matched, with only two exceptions: one change in a lot of RNA Clean & Concentrator kit™-5 (R1016, Zymo Research, Irvine, CA), and one change in a lot of Reverse Transcriptase enzyme.

Total RNA was extracted from 0.5 mL of CSF using the mirVana™ PARIS™ Kit (AM1556, (Thermo Fisher Scientific, Waltham, MA) as described [10]. The RNA samples were concentrated using RNA Clean & Concentrator kit™-5 (R1016, Zymo Research). The concentrated RNA samples were reverse transcribed using a custom MiRNA RT pool (4459652, Thermo Fisher Scientific) and TaqMan® MicroRNA Reverse Transcription Kit (4366596, Thermo Fisher Scientific), then preamplified using a custom MiRNA PreAmp pool (4459660, Thermo Fisher Scientific) and TaqMan® PreAmp Master Mix w/QRC (4391128, Thermo Fisher Scientific). The pre-amplification products were diluted 1:4 in RNase/DNase-free water, then 18 μL of diluted samples were mixed with TaqMan® Universal PCR Master Mix II, no UNG (4440040, Thermo Fisher Scientific) and RNase/DNase-free water to a final volume of 450 μL, loaded onto a custom TaqMan® Array Card (4449140, Thermo Fisher Scientific) and amplified on a QuantStudio™ 12K Flex Real-Time PCR instrument (4471089, Thermo Fisher Scientific) using QuantStudio™ 12K Flex Software v1.2.2 (Thermo Fisher Scientific).

Preprocessing of Ct values

The miRNA amplification data was then subjected to quality control filtering of the Ct values using ExpressionSuite Software v.1.0.3 (Thermo Fisher Scientific). All further processing and statistical analyses were conducted using Stata version 15.1 (StataCorp LLC, College Station, Texas) and R version 3.4.1 (R Foundation for Statistical Computing; http://www.r-project.org) software tools. We implemented biomarker acceptability rules to enable go or no-go decisions for each candidate miRNA before assessing its predictive performance in samples. First, we excluded miRNAs that did not amplify in at least 20% of the samples to ensure that candidates had at least some biomarker potential. The 20% cutoff was chosenbecause80%censoringwitha1:1case:control ratio means specificity can never exceed 40%, even with perfect sensitivity. We believed this would be a lower bound of usefulness for a potential biomarker. Second, 2 of the 3 technical replicates/miRNA probe included on the array needed to successfully amplify in the sample. Third, the data was filtered to ensure good quality detection and to avoid false positives. Thus, we included amplifications with median Ct values < 34, a cutoff chosen based on high replicate standard deviations for Cts > 34. Further, the amplifications had to have an AmpScore ≥1.0 and a CqConf ≥ 0.9. The AmpScore indicates, for a given well, the rate of amplification in the linear region of the response curve. The CqConf indicates the calculated confidence (between 0 and 1) for the Cq/Ct value of the well. Thus, miRNAs that met these quality control standards were considered for further analysis. We also required at least 90% attestation (the fraction of samples that show evidence of expression for a given miRNA via their Ct values) across the positive control miRNAs since these were used for normalization. Under these rules, we retained 7 of the 10 positive controls, and 26 of the 37 possible miRNA biomarker candidates (Fig. 1 and Table 2) identified in the discovery study for further analysis in this validation study. All of the miRNA qPCR array data and donor-specific metadata have been reposited in the exRNA Atlas [15] dataset Validation Study for Candidate AD miRNA Biomarkers in Human CSF, #EXR-JSAUG1UH3001-ST that can be accessed via the Datasets link at http://exrna-atlas.org/.

Table 2.

Quantitative and statistical measures of the 37 candidate AD miRNA biomarkers tested in the validation study. The list shows the MIMAT accession number for each miRNA. The 26 miRNAs that remained viable as candidate AD biomarkers are indicated by “Yes” in the ‘Viable’ column. The ‘Rank’ columns indicate the judges used to rank-sort the miRNAs, and include the ranking of each miRNA in the discovery (Disc) study. Two judges (LogRank, ROC) assessed performance of individual miRNA markers, while the remainder assessed classification performance of miRNAs in combination. The % of miRNAs detected in each diagnostic group and the fold change for each miRNA are also listed

MIMAT	miRBase	Viable	Rank							% Detected		Fold Change
	Name		Disc	Log-Rank	ROC	CART	CFOREST	CHAID	BOOST	Control	AD	Log2	95% CI
				p-value	AUC
0000423	miR-125b-5p	Yes	9.5	0.89	0.52	13	22	4	13	69	68	1.17	(0.81,1.67)
0005881	miR-1291	No
0000431	miR-140–5p	Yes	14	0.18	0.57	15	10	21	14	36	23	1.17	(0.89,1.54)
0000434	miR-142–3p	Yes	1	0.93	0.50	22	23	7	18	81	81	1.02	(0.72,1.44)
0000435	miR-143–3p	Yes	7	0.80	0.55	20	13	8	20	36	32	1.20	(0.96,1.50)
0004601	miR-145–3p	No
0000437	miR-145–5p	Yes	17	0.72	0.51	16	24	23	16	71	74	1.10	(0.84,1.43)
0000449	miR-146a-5p	Yes	14	0.24	0.56	10	12	9	6	84	94	0.86	(0.60,1.24)
0002809	miR-146b-5p	Yes	17	0.83	0.57	11	19	18	17	79	72	1.27	(0.91,1.79)
0000417	miR-15b-5p	Yes	21	0.30	0.55	7	9	24	7	31	23	0.96	(0.76,1.22)
0000069	miR-16–5p	Yes	11.5	0.53	0.52	24	26	25	19	84	89	0.96	(0.61,1.51)
0004614	miR-193a-5p	Yes	4	0.22	0.62	4	2	3	3	71	74	1.39	(1.03,1.87)
0000461	miR-195–5p	Yes	20	0.78	0.53	12	14	17	25	31	32	1.06	(0.83,1.36)
0000074	miR-19b-3p	Yes	3	0.12	0.59	3	7	12	8	93	96	1.23	(0.86,1.77)
0002811	miR-202–3p	No
0000280	miR-223–3p	Yes	19	0.06	0.55	17	8	5	4	99	100	1.15	(0.90,1.48)
0000080	miR-24–3p	Yes	8	1.00	0.53	25	20	19	24	83	81	1.06	(0.75,1.49)
0000083	miR-26b-5p	No
0000419	miR-27b-3p	No
0004502	miR-28–3p	Yes	17	0.58	0.52	14	17	10	9	39	43	1.21	(0.98,1.51)
0000086	miR-29a-3p	Yes	11.5	0.91	0.50	8	21	13	22	43	43	1.15	(0.86,1.56)
0000088	miR-30a-3p	Yes	9.5	0.55	0.55	21	11	11	10	50	53	1.28	(0.96,1.70)
0000245	miR-30d-5p	No
0000752	miR-328–3p	Yes	14	0.72	0.55	26	25	15	11	60	66	0.95	(0.69,1.32)
0000760	miR-331–3p	Yes	22	0.71	0.53	18	15	22	26	23	19	1.20	(0.93,1.54)
0004692	miR-340–5p	No
0000710	miR-365a-3p	Yes	24.5	0.69	0.56	23	16	26	21	56	55	0.90	(0.69,1.16)
0003379	miR-378a-3p*	Yes	5	0.11	0.57	2	5	14	5	67	81	0.97	(0.68,1.37)
0000732	miR-378a-3p**	Yes	6	0.31	0.58	9	6	16	15	87	89	1.31	(0.97,1.79)
0002174	miR-484	Yes	24.5	0.16	0.60	6	3	6	12	89	91	1.21	(0.89,1.64)
0002837	miR-519b-3p	No
0002843	miR-520b-3p	No
0002888	miR-532–5p	Yes	24.5	0.85	0.54	19	18	20	23	26	28	1.13	(0.89,1.43)
0003249	miR-584–5p	Yes	24.5	0.22	0.59	1	1	2	2	77	83	1.23	(0.93,1.64)
0003258	miR-590–5p	No
0003265	miR-597–5p	Yes	2	0.56	0.62	5	4	1	1	97	94	1.45	(0.98,2.15)
0003271	miR-603	No

^*Probe designed to nt 1–21 of miR-378a-3p,

^**Probe designed to nt 1–22 of miR-378a-3p.

Batch correction and normalization

Since the CSF samples were measured over the course of many weeks, variation in ambient temperature, machine calibration, and other minor, but uncontrolled and imperceptible factors may arise. Thus, there is a need to ensure that card-to-card batch variations are removed prior to analysis. To accomplish this, within each card, the Ct values for a sample were taken as the median of the miRNA values in the triplicate wells (n = 3 technical replicates/miRNA probe); miRNAs with median value of 34 or larger were censored at 34, which means that any miRNA with a Ct > 34 was considered to have expression too low for reliable detection in the assay. These miRNAs were included in the analysis with an assigned expression value of zero. The median Ct values were then corrected for day-to-day variance in technical processing. Positive controls for the RefStds were treated as anchors and used to align the array cards. With 13 RefStd samples included over the processing of 66 array cards, and not appearing on every array card, we had to interpolate batch corrections for cards that did not include a RefStd. These plausible card corrections were made by averaging predictions from 5 models of batch differences: 1) the median of all probes; 2) the mean of all positive control probes; 3) the mean and variance of the distribution of all probes; 4) a linear trend to connect one RefStd anchor to the next in run-order sequence; and 5) a card median representing a random deflection from the sample mean of all card medians. We then averaged these prediction values to align Ct values across all 66 cards.

Following batch correction, there is a need to normalize the Ct values by the overall miRNA content of the sample, in order to ensure comparability of expression measurements. The aligned Ct values were normalized relative to the mean of 7 “non-changing” positive controls included on the arrays: 6 miRNAs (miR-1290, −204–5p, −30e-3p, −574–3p, −638, −92a-3p), plus U6 snRNA (Supplementary Table 1). The normalization was done by calculating a CSF sample-specific Ct value offset from a grand mean (i.e., the “subject effect”) using a crossed random effects mixed model [16], and then subtracting this offset from the measured Ct values for the sample. Similar to our discovery studies, we then transformed the normalized Ct values onto an expression scale so that higher values indicate relatively greater quantities of miRNA expression (the transformation is expression = Ctnorm_max –Ctnorm, where Ctnorm_max is the largest non-censored normalized Ct value in the data, rounded up to the nearest integer). Censored Ct values were assigned a value of zero on this expression scale and included in the final analysis [10].

Biomarker relevance

We assessed the global relevance of the biomarker candidates by computing the multivariate distance between observations using Mahalanobis distance. Mahalanobis distance is a way of measuring the separation between a data point and the center of a group of data points with respect to many variables, while accounting for their mutual correlations [17]. Thus, samples similar in overall expression lie as close points in the space defined by the 26 miRNA values, while samples dissimilar in expression are widely separated in this space. We calculated the distances of each of the AD and Control samples from the center of the Controls using the covariance of the Controls as the scale in order to measure each sample’s similarity to a typical Control sample. This same approach was used to verify appropriateness of the positive control miRNAs as normalizers.

Biomarker importance ranking

The decision that a biomarker is important should be robust, so for greater assurance it is essential to judge their importance in many different ways. To assess each candidate miRNA as an AD biomarker, we examined: 1) the association between miRNA expression and AD status; 2) clear separation of miRNA expression between the AD and Control CSF; and 3) complementarity of information when combined with other AD markers. Importantly, (1) and (2) are complementary information about an individual marker, and (3) evaluates miRNA value as members of larger biomarker groups that may provide synergistic information regarding the disease state. Biomarker importance was assessed by 1) log-rank tests [18] to account for censoring, 2) receiver operating characteristic (ROC) curves [19], and 3) variable importance in random-forest classifiers using 4 different decision-tree generation rules (CART [20], CFOREST [21], CHAID [22], BOOST [23]) to mitigate bias in the importance estimate due to a particular decision rule.

Prioritization of AD miRNA biomarkers

We designed our analytical approach to elicit consensus across statistical methods and prioritize candidate miRNAs as potential AD biomarkers. Each testing procedure generated a ranking of the 26 validated miRNAs: different “judging” methods sorted the candidates from best (1) to worst (26). Moreover, we incorporated information from our discovery study in order to give due weight to our prior knowledge that certain candidates were likely to fare better than others in validation testing. We did this by ranking the complete set of candidates prior to doing any testing in the validation cohort, and we included this prior ranking as a separate judge. Each judge independently ranks the miRNA markers, and then the ranks for each miRNA marker are summed. This rank sum reflects our statistical consensus opinion of a miRNA marker’s value. Significance of differences in rank sum were assessed via permutation testing of the Skillings-Mack statistic [24].

Multimarker classification performance

Although ranking and prioritization of individual miRNAs is an important first step in understanding how the miRNAs may relate to AD status and progression, our ultimate goal is to develop an AD classifier using the best available miRNA information. To be useful this classifier needs to incorporate several different miRNAs into a “multimarker” model because no single miRNA contains enough information about AD to enable reliable prediction. Thus, assessing which miRNAs work well together and in what combinations is of key importance. Multimarker classification performance was assessed by evaluating linear combinations of all possible 1-, 2-, 3-, and 4-marker subsets of the 26 validated miRNAs. We also carefully examined selected combinations of 5-, 6-, and 7-markers based on performance in smaller sized subsets and/or rank sum. To benchmark classification performance, we used a stepwise Bayesian model-averaging [23] procedure on the full set of 26 markers and selected 9 markers demonstrating robust contribution to all of the multimarker models. The model-averaged area under the ROC curve (AUC) of this 9-marker set (0.716) was used as a benchmark that any proposed model must beat. The subsets described above yielded 76,867 unique models, whose classification (AUC) and model fit (Akaike information criterion [AIC]) were calculated using logistic regression. We plotted AIC versus AUC, denoting number of markers per model, and then superimposed a nonlinear regression of AUC on AIC onto this plot to select top-performing combinations in terms of classification, calibration, and parsimony. The combinations with highest AUC and lowest AIC that rose above the benchmark were selected. The individual contribution to the set of 93 top-performing models (which equated to the top 0.12% of the 76,867 models tested) was assessed for each miRNA by calculating the fraction of these models that the miRNA was included in, and the size of the miRNA’s model-averaged coefficient.

Performance of AD miRNAs plus APOE4 genotype or CSF Aβ₄₂:T-tau measures

Performance of the AD miRNA biomarkers was compared to the performance of the APOE4 genotype status and to the performance CSF Aβ₄₂:T-tau measurements in the validation cohort. Donors who were missing APOE4 genotype data (n = 5 Control, n = 2 AD) or Aβ₄₂:T-tau measures (n = 4 Control, n = 1 AD) were not included in this analysis so that all models could be directly comparable on the same donor cohort (n = 60 Control, n = 41 AD). We compared miRNA classification performance alone to miRNA performance after combining APOE4 genotype or Aβ₄₂:T-tau ratios with the miRNAs. We formed a k-nearest-neighbor classifier (k = 3) so that all of the miRNA information would be used without imposing any assumptions about either the relationships among the miRNAs or how they contribute to AD classification. The k-nearest-neighbor classifier (k = 3) was based on Canberra distance [25] between Mahalanobis-scaled miRNA expression values, setting prior probabilities proportional to the AD and Control group sizes and breaking ties randomly.

Correlation of MMSE with higher-ranked miRNAs

We examined whether the correlation between MMSE scores and individual miRNA expression levels would be larger for miRNAs that ranked higher as AD markers. Correlation with MMSE was measured as the partial R² statistic from a linear regression of the individual miRNA expression on the MMSE score, adjusting for age and sex. The partial R² values were then compared to miRNA ranks using Spearman correlation.

RESULTS

Donor characteristics

The characteristics for the 71 Controls and 47 AD CSF donors evaluated in this validation study are shown in Table 1. The donors were matched by sex to the extent possible, but there was a somewhat higher percentage of females in the Control group (24 males:47 females; 33.8% males:66.2% females) compared to the AD patients (26 males:21 females; 55.3% males:44.7% females). The donors were matched by age: mean age of healthy Controls was 72.72 ± 5.91, mean age of the AD patients was 73.13 ± 9.22. Controls were in good health with a mean MMSE score of 29.13 ± 1.31, Clinical Dementia Rating (CDR) [26, 27] scores of 0, and no evidence or history of cognitive or functional decline. AD patients were diagnosed with probable AD according to the National Institute of Neurological Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association (NINDS-ADRDA) criteria [28, 29], with a mean MMSE score of 22.06 ± 3.47, and CDR scores of 1–2. Of note, the mean MMSE of the AD patients in the validation study (22.06 ± 3.47) was 4 points higher than the mean MMSE in the discovery study (18.28 ± 6.4) [10], indicating that the validation cohort reported here had milder dementia than the previously reported discovery cohort. APOE genotyping was available for 111/118 donors (66 Controls and 45 AD): the Control group had 62.12% with 0, 30.30% with 1, and 7.58% with 2 APOE4 alleles, while the AD group had 35.56% with 0, 42.22% with 1, and 22.22% with 2 APOE4 alleles. As expected, APOE4 genotype was over-represented in the AD group [30]. Most donors in the 0 or 1 category of the APOE4 alleles had an APOE3 genotype, while only three donors had an APOE 2,4 genotype (2 normal control males, 1 AD male). CSF Aβ₄₂ and T-tau measures were available for 106 of the 118 donors (64 Controls and 42 AD). The ratio of Aβ₄₂:T-tau in the Control group is 1.5 ± 0.8, while the AD group had a ratio of 0.6 ± 0.5. Thus, as expected, the Aβ₄₂:T-tau ratio decreased in the AD group [31].

Table 1.

Donor characteristics. The table includes the number, sex, age, and MMSE for the 118 Control and AD patients evaluated in this validation study. APOE4 genotype status was available for 111 of the CSF donors, while Aβ₄₂ and T-Tau measures was available for 106 of the CSF donors evaluated in the study

	CONTROL		AD		All
SEX
Male	24 (33.8%)		26 (55.3%)		50 (42.4%)
Female	47 (66.2%)		21 (44.7%)		68 (57.6%)
Total	71		47		118
AGE*	Mean ± SD		Mean ± SD		Mean ± SD
Male	74.75 ± 6.24		72.65 ± 8.67		73.66 ± 7.60
Female	71.68 ± 5.52		73.71 ± 10.05		72.31 ± 7.21
Total	72.72 ± 5.91		73.13 ± 9.22		72.88 ± 7.37
MMSE*	Mean ± SD		Mean ± SD		Difference ± SDboot†
Male	28.88 ± 1.62		22.88 ± 3.37		−5.99 ± 3.68
Female	29.26 ± 1.11		21.05 ± 3.38		−8.22 ± 3.48
Total	29.13 ± 1.31		22.06 ± 3.47		−7.06 ± 3.67
APOE4 ALLELES	Count	%	Count	%	Count	%
0	41	62.12	16	35.56	57	51.35
1	20	30.30	19	42.22	39	35.14
2	5	7.58	10	22.22	15	13.51
Total^	66	100.00	45	100.00	111	100.00
Aβ42:TAU	Mean ± SD		Mean ± SD		Difference ± SDboot†
Aβ42	479.0 ± 210.1		305.5 ± 154.8		−173.5 ± 258.1
T-tau	392.2 ± 201.1		689.4 ± 335.1		297.3 ± 385.7
Aβ42:T-tau	1.5 ± 0.8		0.6 ± 0.5		−0.9 ± 0.9
mean(Aβ42):mean(T-tau)	1.22 ± 0.10		0.44 ± 0.05		−0.78 ± 0.11
Total#	64		42		106

^*Age and MMSE values represent data at the time of CSF collection.

^†SD_boot, standard deviation of differences between randomly selected AD and CONTROL individuals based on 100,000 bootstrap samples.

^{^}For available donor samples; genotyping data for n = 5 Control and n = 2 AD were not available.

^#For available donor samples; Aβ₄₂ and T-tau measures for n = 7 Control and n = 5 AD were not available.

Measures for validated AD miRNAs

Table 2 lists the 37 candidate AD miRNA biomarkers tested in this validation study, and shows the quantitative and statistical measures of the 26 miR-NAs that remained viable as candidate biomarkers under our acceptability rules (Methods), as indicated by “Yes” in the ‘Viable’ column. Our analytic strategies focused on assessing these 26 viable miRNAs in multimarker combinations, and we demonstrate that these miRNAs work together very well to differentiate AD from Controls. First, we determined how well all 26 miRNAs can jointly separate/classify AD from Control samples. Figure 2A plots the Mahalanobis distances of AD and Control samples, showing how far each is from the center of the Controls in the miRNA expression space. The individual Control samples (gray points) fall mostly at small distances from the center (bottom of plot), while the AD samples (black points) fall mostly at large distances (higher up the plot). Few Control samples are above the 80th percentile (dashed) line, and few AD samples are below it. Using the Mahalanobis distance as a classification index we attain AUC of 0.84, a value that serves as an estimate of the maximum performance of the 26 viable miRNAs in this cohort. In contrast, the classification potential of the positive control miRNAs in this cohort (Fig. 2B) shows that Control (gray) and AD (black) samples are intermixed, and samples at larger distances come from both groups. The 99.9th percentile (dashed) line contains nearly all samples and the AUC for the positive control miRNAs is 0.54. This highlights the overall informativeness of the 26 miRNA candidates as AD biomarkers, as compared to steady-state positive control miRNAs.

Fig. 2.

A) Overall classification potential of biomarker candidate miRNAs. Mahalanobis distances of AD and non-AD (Control) samples from the center of the Controls in the miRNA expression space show that the Control (gray) points fall mostly at small distances from the center (bottom of the plot area), while the AD (black) points fall mostly at large distances (higher up in the plot). Using the distances to classify samples gives an AUC of 0.84. B) Overall classification potential of positive control miRNAs. Mahalanobis distances calculated from the center of the Control group with respect to the positive control miRNAs show that the Control (gray) and AD (black) samples are randomly intermixed, and samples at larger distances come from both groups. Using these distances to classify samples gives an AUC of only 0.54. For the Mahalanobis distance calculations, we imputed values > 34 Ct for censored observations using predictions from a tobit regression model [47] in order to jitter the censored values appropriately. %ile, percentile.

Rankings of validated miRNAs

We used seven independent “judges”, or statistical assessment criteria (described in Materials and Methods) to characterize how well each of the 26 miRNAs work alone or in combinations to correctly identify a CSF sample as AD or Control. Figure 3 shows the ranks and rank sums of each of the 26 miRNAs according to these statistical assessments, including the ranking based on prioritization in the discovery study [10]. Higher ranks are indicated by smaller numbers, so that more highly ranked markers across all criteria have lower rank sums. Two of the criteria (LogRank, ROC) assessed performance of individual markers, while the remainder assessed classification performance of miRNAs in combination. As shown via color-coding (red), miR-193a-5p, −597–5p, −195–5p, and −378a-3p performed best and were also among the top discovery study performers. In contrast, miR-484 and −584–5p were lower in the discovery prioritization, but performed well here. However, our top discovery study performer, miR-142–3p, did not perform well here. The latter two outcomes may reflect the difference in the average MMSE score between the discovery and validation cohorts (4 points lower in the discovery cohort). For example, miR-142–3p may preferentially signal laterstage AD and be an indicator of disease progression.

Fig. 3.

Rank plot of the 26 validated AD miRNA biomarkers. The plot shows the ranks and rank sums of each of the 26 miRNAs according to seven independent “judges”: six statistical assessment criteria, plus the rank of the miRNA in the prior discovery study [10]. The ranks of the individual table cells are color-coded along a red-blue color ramp to visually assess consistency of rankings and to identify higher ranking “hot” (red) and lower ranking “cool” (blue) miRNA candidates.

MiRNA combinations increase classification performance

Most biomarkers reveal complementary information, and even modest miRNA combinations showed greatly improved classification accuracy for AD compared to single miRNAs in our discovery study [10]. As the best miRNA-based classifiers will necessarily be multimarker combinations, we fit 76,867 multimarker models (described above), and selected 93 top-performing models for pairing good predictive power (high AUC) with good calibration (low AIC). Figure 4 presents the ROC curves for all 93 top models, color-coded by size of the model. Averages of ROC curves within each color are presented in bold. Combinations consisting of 7 miRNAs (mean AUC = 0.796) are incrementally better at identifying AD samples than combinations that consist of fewer miRNAs. Further, examples that contain as few as 4 miRNAs attain a mean AUC > 0.72, the value we predicted in our prior study [10]. Some combinations of 6 and 7 miRNAs attain AUCs exceeding 0.80 (faint blue and orange lines). Recalling that the classifier based on Mahalanobis distance in the complete 26-marker space showed an AUC of 0.84 (Fig. 1A), we note that models comprising as few as one-fourth of the markers approach that performance level. These values far exceed the best performing individual miRNAs, underscoring the need for a multimarker-based approach to AD prediction using miRNAs.

Fig. 4.

Classification accuracy for miRNA combinations. The plot depicts the average receiver operating characteristics (ROC) for the 93 best multimarker models of the validated miRNAs. The faint lines are the individual ROC curves for the top-multimarker models (color coded based on the number of miRNAs that contributed to the model). Bold solid lines are the empirical averages of the individual ROC curves. Dashed curves are binormal estimates of the average. The mean area under the curve (AUC) for the averaged ROC curves are presented. Note that some of the individual ROC curves generated from 6 miRNAs (faint orange lines) or 7 miRNAs (faint blue lines) have AUCs that exceed 0.80.

Top contributors to miRNA-based AD classifiers

After we identified the top-performing multimarker models, we next sought to determine which of the 26 miRNAs contribute the most to performance in these models. In Fig. 5, we plotted the percent contribution to the top 93 multimarker models (from Fig. 4) against the consensus rank of each of the 26 validated miRNAs in the screening assessments (shown in Fig. 3). Marker size is proportional to the magnitude of the model-averaged coefficient estimate from the Bayesian model-averaging procedure (Materials and Methods, Multimarker classification performance). The highest-ranked miRNAs markers tend to be, but are not always, the most important contributors to the multimarker models. Our exhaustive assessment of classification performance revealed that 14 of the 26 AD biomarker candidates contributed strongly and in a mutually complementary and additive manner to AD prediction across a broad range of model scenarios. These top 14 contributors to miRNA-based classifiers are listed in Table 4.

Fig. 5.

MiRNA percent contribution to the top multimarker models against the consensus rank. The figure shows how our statistical consensus ranking relates to marker contribution in multimarker models. For each of the 26 miRNAs the percent contribution to a multimarker model is plotted against the consensus rank in the screening assessments. Contribution percentage is based on presence in the top 0.12% (93/76867) of multimarker models. Marker size is proportional to the magnitude of the marker’s model-averaged coefficient estimate.

Table 4.

MiRNAs that contributed strongly and in a mutually complementary and additive manner to AD prediction based on multimarker modeling, sorted based on their percent (%) contribution to multimarker models

MIMAT	miRBase Name Name	% Contribution to Multimarker Models Models	Model-Averaged Logit Coefficient	Bayesian Posterior Model Inclusion Probability	Overall Rank in Screening Assessments
0000417	miR-15b-5p	97.8	−0.14	0.59	12
0000760	miR-331–3p	75.3	−0.01	0.26	24
0000074	miR-19b-3p	64.5	0.07	0.42	3
0003249	miR-584–5p	62.4	0.08	0.46	5
0000449	miR-146a-5p	49.5	0.05	0.39	9
0003265	miR-597–5p	48.4	0.03	0.28	2
0000088	miR-30a-3p	46.2	0.06	0.44	10
0003379	miR-378a-3p*	45.2	0.08	0.56	4
0000710	miR-365a-3p	32.3	−0.03	0.33	21
0002174	miR-484	22.6	0.02	0.29	6
0004614	miR-193a-5p	14.0	0.03	0.34	1
0000431	miR-140–5p	12.9	−0.04	0.36	11
0002809	miR-146b-5p	9.7	−0.01	0.27	16
0000280	miR-223–3p	9.7	0.18	0.45	8

APOE4 status improves miRNA classification performance

The APOE4 allele is a risk-factor gene as it increases a person’s risk of developing AD; however, having an APOE4 allele does not guarantee that one will develop AD. Thus, many researchers believe that APOE testing is useful for studying AD in large groups, but not for determining an individual’s risk for AD. That said, given the known strong association between APOE genotype and AD risk, we examined whether the genotype status was redundant with the miRNA expression values, or whether it could add power to a general miRNA-based classifier. We found that addition of APOE4 genotype status to the best 14 miRNAs (Table 4) increased the classification performance of the nearest-neighbor AD classifier, particularly in the high-specificity range (Fig. 6). The AUC for the best 14 miRNAs independent of APOE4 is 0.820, but when APOE4 was added the AUC increased to 0.856, a 4 point increase similar to that observed for parsimonious models in our discovery study [10]. It is worth noting that the AUC for APOE4 only is 0.637 in this cohort, which is lower than the reported APOE4 performance (~0.67 [32]) and considerably lower than performance in our discovery cohort (0.73). This is indicative of potential latent AD cases among the current cohort Controls. Yet even in this low-powered setting, adding APOE4 to the classifier yielded the same level of improvement, suggesting that the miRNA expression phenotypes are not simply reflecting genetic risk; they also provide independent power to differentiate AD from Controls.

Fig. 6.

APOE4 plus miRNAs improves classification performance over the best-possible miRNAs-only model. The plot depicts the receiver operating characteristics (ROC) curves for a multimarker model that includes the best miRNAs (dashed line), APOE4 only (dark gray line), and all 14 miRNAs plus APOE4 (solid line). The area under the curve (AUC) for the best 14 miRNAs increases from 0.820 without APOE4 to 0.856 with APOE4 (ΔAUC=0.04). The AUC estimates are based on k-nearest-neighbor nonparametric classifiers that incorporate information from the top 14 miRNAs found to contribute to the best multimarker models (Table 4).

Aβ₄₂:T-tau improves miRNA classification performance

The CSF markers Aβ₄₂ and tau (total and phospho) reflect AD pathology, and have high accuracy to diagnose AD with dementia and prodromal AD in mild cognitive impairment cases [33]. Therefore, we examined the performance of Aβ₄₂ and T-tau, alone and in combination with the miRNA biomarkers. Table 3 shows the diagnostic performance of CSF Aβ₄₂ and T-tau for 106 donors with available data, as well as the odds ratio for APOE4 positive status given a 1-standard-deviation change in the CSF marker (in the direction of higher probability of AD) for the donors where both the Aβ₄₂ and T-tau measures, and APOE4 genotype status, were known. As expected [31], we found that both Aβ₄₂ and T-tau are good AD predictors of AD (AUC = 0.78, on par with any of the miRNA models), but the Aβ₄₂:T-tau ratio is dramatically better than either marker alone (AUC = 0.86) (Table 3). We assessed the AUC of a full-information classifier using the top-contributing 14 miRNAs (Table 4), which increased from 0.820 without Aβ₄₂:T-tau to 0.903 with Aβ₄₂:T-tau (ΔAUC=0.08) (Fig. 7). Together, these data show that similar to the APOE4 results, the miRNA information is not redundant with the Aβ₄₂:T-tau ratio, but that it provides independent power to differentiate AD from Controls. This finding importantly reveals that CSF protein and miRNA measures provide complementary information about AD status.

Table 3.

CSF Ab₄₂ and T-tau measures. The table lists the diagnostic performance of CSF Aβ₄₂ and T-tau for the 106 donors with available measures. The table also lists the odds ratio for APOE4 positive status given a 1-standard-deviation change in the CSF marker (in the direction of higher probability of AD) for the 101 donors where both the Aβ₄₂:T-tau and the APOE4 genotype status were known. (Odds ratios were calculated as the exponentiated coefficient from a logistic regression of APOE4 genotype status on the standardized values of the marker)

Marker	AUC*	Best Cutoff	Sensitivity	Specificity	SDΔ odds ratio for APOE4*
Aβ42	0.782	<360	78.6%	64.1%	1.62
T-tau	0.782	>836	73.8%	73.4%	3.97
Aβ42:T-tau	0.861	<0.83	81.0%	79.7%	2.81

^*For reduced cohort not missing APOE genotyping: 60 Control and 41 AD.

Fig. 7.

Aβ₄₂:T-tau plus miRNAs dramatically improves classification performance over the best-possible miRNAs-only model. The plot depicts the receiver operating characteristics (ROC) curves for a multimarker model that includes the “best” (i.e., top-contributing) 14 miRNAs (dashed line), Aβ₄₂:T-tau only (dark gray line), and the best 14 miRNAs plus Aβ₄₂:T-tau (solid line). Area under the curve (AUC) increases from 0.820 for the miR-NAs without Aβ₄₂:T-tau to 0.903 with Aβ₄₂:T-tau (ΔAUC = 0.08). The AUC estimates are based on k-nearest-neighbor nonparametric classifiers that incorporate information from the top 14 miRNAs found to contribute to the best multimarker models (Table 4).

CSF miRNAs and AD severity

Given that this validation cohort had diminished severity of dementia and somewhat reduced classification performance of the miRNA panel compared to the discovery cohort, we evaluated how the miRNA markers correlate with disease severity as represented by the MMSE scores (Table 1). We found that the association (as measured by partial R²) between MMSE scores and miRNA expression became stronger in higher-ranked miRNAs. For example, our higher-ranked miRNAs, such as miR-193a-5p, have expression levels that correlate more strongly with MMSE scores than our lower-ranked miRNAs, such as miR-331–3p, which tend to show only weak correlation with MMSE at best. As shown in Fig. 8, in general, the association between MMSE score and miRNA expression levels is increasingly strengthened by an improvement in rank order, suggesting that these miRNAs may also be able to signal disease progression in AD patients.

Fig. 8.

Correlation of MMSE with top 14 miRNAs found to contribute to the best multimarker models. The X-axis is the rank order of the 14 higher-ranked miRNAs, and their respective single-marker AUCs are shown in the labels. The Y-axis is the squared partial correlation (i.e., partial R²) of MMSE with the miRNA expression in patients, adjusted for age and sex. The figure shows that individual miRNAs that ranked higher as AD markers (Table 4) correlate more strongly with changes in MMSE score. The Spearman correlation between partial R² and miRNA rank was 0.82.

DISCUSSION

We previously discovered that CSF miRNAs can differentiate AD from Controls and potentially serve as new biomarkers for AD [10]. Here we report results of validation studies performed on 37 miRNAs identified in the discovery study, based on n = 1 technical replicate/probe on the miRNA array. For the validation studies we assessed 37 miRNAs; 20 that were verified by array in our laboratory, 17 additional that were brought forth as candidate markers. The validation studies were performed using a custom array comprised of n = 3 technical replicates/probe on the miRNA array, in order to assess miRNA performance in a new and independent cohort of CSF donors. Our studies validate that 26 of the previously identified CSF miRNAs continue to differentiate AD patients from Controls. Further, combinations of miRNAs increases classification performance for AD. Based on our multimarker modeling we identified 14 miRNAs that contributed strongly and in a mutually complementary and additive manner to AD prediction (Table 4). In addition, we confirm that a combination of new (miRNA) and existing (APOE4, Aβ₄₂:T-tau) markers increased classification performance, as we previously observed for APOE4 in our discovery study [10]. In line with our results, hsalet-7b has been shown to work in combination with either Aβ₄₀ and Aβ₄₂, or T-tau and p-tau, to increase the AUC relative to the independent markers [34]. These findings strongly support that CSF miRNAs are not redundant with APOE4 and Aβ₄₂:T-tau, but instead offer additional diagnostic information. We also demonstrate that our highest-ranked AD miR-NAs correlate with MMSE scores in AD, which suggests these miRNAs have the potential to track disease progression and consequently be of use in clinical trials.

Previous multi-center AD biomarker studies have identified confounding factors including sex, age, center of origin, and sample centrifugation status, which negated the utility of the miRNAs to differentiate AD from controls [35]. To mitigate these effects, we selected sex- and age-matched CSF samples and intentionally locked in the parameters for the validation studies. The decision to maintain one technological methodology throughout our biomarker studies was based on evidence of diminished reproducibility of results between expression platforms, even when using the identical RNA for each platform [36]. This finding was also consistent with our CSF analytic studies [37]. Thus, to maximize consistency within these studies, we used the same vendor and the same miRNA probes throughout the discovery and validation phases. Further, we enforced the use of identical manufacturing lots for kits and reagents used throughout the experiments to minimize variation from sources unrelated to the question of differentiating AD from controls. In doing so, we safeguard against spurious associations due to measurement errors and batch effects, which is a strength of our study that adds validity to the results.

Here we have validated that parsimonious combinations of 26 of the discovery miRNAs continue to differentiate AD patients from controls. However, the performance of each miRNA is not an exact match between the discovery and validation studies. There are two experimental parameters that may account for this difference. First, U6 snRNA was used as a normalizer for the discovery studies, but a combination of non-changing miRNAs (including U6) served as a better normalizer for the validation studies. Second, donors in the discovery cohort had more advanced dementia (MMSE: 18) relative to the validation cohort (MMSE: 22), with a mean MMSE score 4 points lower in the discovery cohort. Thus, some miRNAs may signal better in more advanced stages of AD.

Seven out of these 26 miRNAs (miR-125b-5p, −146a-5p, −146b-5p, −15b-5p, −195–5p, −30a-3p, −328–3p) were previously identified by others as candidate biomarkers for AD [38–41]. A previous study identified miR-27a-3p as a candidate biomarker that is decreased in the CSF of AD patients [42]; however, the miR-27a-3p finding was not replicated in our studies. This difference is likely due to the use of different vendors and platforms (TaqMan™ TLDAs versus Exiqon SYBR Green miRCURY LNA arrays) which can lead to inconsistencies in results even when profiling an identical pool of RNA [36].

Our multimarker modeling identified certain miR-NAs that on their own are not interesting as AD biomarkers and would likely have been excluded from earlier studies, but in the proper context they strengthen a combined biomarker classifier (e.g., miR-331). In line with our results, other studies show that multimarker modeling has strengths over single markers. For example, a recent study of miR-NAs isolated from CSF extracellular vesicles found that linear combinations of a subset of differentiating miRNAs (miR-16–5p, −125–5p, −451a, −605–5p) increased classification performance between Controls and either young- or late-onset AD [43]. Two of these four miRNAs (miR-16–5p, −125–5p) were identified in our discovery phase [10] and validated here (Table 2), supporting our observations that measurable differences in AD patient miRNAs have utility as clinical biomarkers, and miRNA combinations increase sensitivity and specificity compared to single miRNAs.

Our initial experiments with AD miRNA biomarkers focused on CSF, which directly bathes the brain. We recognize that plasma is a more accessible biofluid, and we are encouraged by pilot studies indicating that AD miRNA biomarkers are robustly detected in plasma, but extensive plasma studies are beyond the scope of this report. We are currently designing experiments to assess the performance of the AD miRNAs in a statistically powered study in plasma samples that match this validation cohort.

In summary, the validation studies presented here provide further, confirmatory evidence that miRNA expression in CSF from living donors can distinguish AD patients from Controls. Of 37 miRNAs from the discovery study, 26 miRNAs continue to differentiate AD patients from Controls. Our exhaustive classification performance revealed that 14 of the 26 AD biomarker candidates contribute strongly and in a mutually complementary and additive manner to AD prediction across a broad range of model scenarios. The miRNAs validated in this study form a robust set of biomarkers that will now be further evaluated for use as clinical biomarkers for AD. It is still premature to recommend a final list of miRNAs for clinical practice. The miRNAs need to be evaluated for their performance in classifying mild cognitive impairment and their presence in plasma (both discussed above); and we are currently undertaking those studies. They will also be evaluated as to their specificity for AD versus related neurodegenerative disorders and non-degenerative dementia, and examined in longitudinal studies in individual patients to determine their efficacy as prognostic indicators of AD. Moreover, we are studying the functional relationships between miRNAs and AD, along the lines of recent reports investigating the role of miR-146a in inflammatory pathways in brain [44] human brain cells [45], and AD transgenic mouse models [46]. The validated miRNAs can serve to identify novel proteins and pathways linked to AD, and may reveal novel targets for the diagnosis and/or treatment of AD. Thus, our validation of the CSF miRNAs as biomarkers for AD in a new and independent cohort now supports their advancement toward development and refinement as biomarkers in the clinical setting.