Comparison of multiple quantitative strategies for neuropathologic image analyses

Abstract

The traditional semiquantitative (SQ) scoring system for neuropathologic assessment, although widely used, is prone to variability among assessors and does not capture the full spectrum of pathological changes. To address these limitations, digital pathology-based strategies like positive pixel quantitation or advanced artificial intelligence (AI) techniques have been developed. However, a comprehensive comparison of these measures has never been performed. Using 1412 cases from Boston University brain banks, human-driven SQ scoring was compared with computer-driven percent area-stained measures and AI-driven cellular density quantitation of tau pathology in the dorsolateral frontal cortex. When comparing each measure directly in all cases, we observed general agreement between measures. Because the full dataset included a large range of different neuropathologies, to reduce noise we performed a subanalysis in cases with the neurodegenerative disease chronic traumatic encephalopathy (CTE) and examined correlations with clinical and neuropathologic variables. While all methods demonstrated significant ability to predict CTE neuropathology, inconsistent background, noncellular elements, and artifacts increased variability for the positive pixel method. Thus, the AI-driven method was better at identifying pathological changes associated with sparse pathology. Overall, our results demonstrate important differences among neuropathologic assessment techniques and highlight the need for careful consideration when selecting analysis methods.

INTRODUCTION

Over the past several decades, assessments of pathologic protein aggregation have become essential for the study of neuropathological diseases. Numerous analytic techniques applied across various neuropathologies have generated a wealth of data that have significantly enhanced our understanding of disease type, progression, phenotype, and their connections with clinically meaningful outcomes. Traditionally, the most widely used method for assessing disease burden has been expert neuropathologist scoring, that is utilizing a semiquantitative (SQ) scale categorizing findings as none, mild, moderate, or severe.¹ This straightforward scoring system allows a trained observer to efficiently examine large tissue sections and assign a tangible value to the extent of pathology. Such an approach is appealing, as neuropathologists often review hundreds of slides across multiple cases, sometimes in a single session. These SQ scales have been employed for many years across various diseases and are commonly used to provide meaningful results on neuropathology.^2–5 For example, SQ Consortium to Establish a Registry for Alzheimer’s Disease (CERAD), Thal, and Braak scores are staple of Alzheimer disease neuropathologic diagnostic criteria.^6–8

However, this kind of assessment has 3 major flaws: (1) it is subject to biases and errors inherent in human quantitation, (2) it lacks the ability to capture the full spectrum of pathology and discriminate subtle changes, and (3) it can only progress at the speed of the neuropathologist conducting the assessment. Consequently, over the past 2 decades, newer, computer-driven techniques have emerged to capture disease progression more efficiently and address the limitations of previous SQ systems.

Innovations in histologic analysis techniques have been advanced by the development of scanners that can digitize glass slides and generate digital whole slide images (WSIs). These WSIs can then be used for a variety of computational analyses. One simple yet effective computational method that has emerged alongside WSI analyses is the positive pixel method, in which each pixel is classified as either background or positive based on predetermined color thresholds. The number of positive-classified pixels can then be divided by the total number of pixels to generate a percentage of the total area stained. The simplicity, straightforward results, and ease of use have led to its widespread adoptions in research.^9–15

In contrast to simpler positive pixel or area-stained methods, a rapidly evolving aspect of WSI analysis is artificial intelligence (AI)-driven method. Interest in AI stems from its models, trained on thousands of images with matching features to a specific paradigm, showing high levels of accuracy in classifying and detecting image features.¹⁶ The potential of AI classification networks as valuable neuropathologic tools has been demonstrated recently with the successful detection of neurodegenerative features such as neurofibrillary tangles and analyses of cerebral amyloid angiopathy (CAA), and gliomas.^17–19 Although full clinical implementation has been limited by variations in protocols between labs, regulatory requirements, and a lack of comprehensive networks suitable for clinical practice, research is advancing rapidly and shows promising results.^19–24

In our practice, we routinely use a combination of SQ scoring, positive pixel area-stained analyses, and AI-driven analysis techniques to better characterize neurodegenerative diseases, including chronic traumatic encephalopathy (CTE).^25–27 Chronic traumatic encephalopathy is a progressive tauopathy found in individuals with a history of repetitive head injury (RHI) from playing contact sports such as American football, hockey, soccer, and boxing, as well as from military service and intimate partner violence. Quantitation and analyses of hyperphosphorylated tau (ptau) in CTE is critical to our understanding of the pathogenesis and disease propagation throughout the brain.²⁸ We have utilized both positive pixel percent area-stained analyses and AI-driven cell counts to characterize and validate neuropathologic staging criteria and correlate disease severity with clinical symptoms.^17,26,29

Despite utilizing a wide range of quantitative metrics, however, a comprehensive comparison involving an SQ method alongside multiple digital methods has never been conducted to determine whether 1 technique might be more sensitive or accurate for studying human neuropathology.

In this study, using 1412 cases from multiple brain banks from Boston University, we performed a comprehensive comparison of ptau measures derived from pathologist SQ scores, simple percent area-stained measurements, and more complex AI driven cell counts. We analyzed the correlation between each measure with an individual case to assess consistency among related data. Additionally, we conducted a subanalysis on cases with CTE to examine the strengths and weaknesses of each technique to characterize neurodegenerative pathology and compare to clinical features. This work provides a detailed assessment of different types of neuropathologic analytic metrics and explores the benefits and limitations of each approach concerning neurodegenerative disease research.

METHODS

Subjects

Subjects were used from 3 Boston University brain banks: the Understanding Neurologic Injury and Traumatic Encephalopathy (UNITE), Framingham Heart Study (FHS), and Alzheimer’s Disease Research Center (ADRC) brain banks. Donation collection, processing, histology, and evaluation have been harmonized across all 3 banks. Each donation was predicated on written consent from next of kin. Institutional Review Board approval was obtained through Boston University School of Medicine, VA Bedford Healthcare System, and VA Boston Healthcare System. Approval for the clinical evaluation protocol was obtained through Boston University School of Medicine. Cases were assessed for neurodegenerative diseases using well-established criteria for Alzheimer disease, neocortical Lewy body disease, CTE, frontotemporal lobar degeneration, limbic age-related TDP-43 encephalopathy neuropathological change, and motor neuron disease.^28,30–34 Neuropathologic evaluations occurred blinded to clinical evaluations and were reviewed by neuropathologists (V.E.A., T.D.S., A.C.M.). Upon completion of evaluation, cases were digitized and used for downstream analyses (Figure 1), and 1412 cases had available digital histologic images. Demographics, scanner used, and neuropathology are presented in Table 1. For the CTE analysis, we used inclusion criteria selecting for cases that had a neuropathologic diagnosis of CTE and excluded cases that had a neuropathologic diagnosis of any other significant comorbid disease, resulting in 429 cases. Cases with CTE were divided into low CTE (n = 208) or high CTE (n = 221) based on McKee staging criteria.³⁴ We also included 269 control cases that did not receive a diagnosis of CTE or other neurodegenerative disease, and had a CERAD score less than or equal to 1. Demographics of the subanalysis are presented in Table 2.

Figure 1.

Overview of analysis pipeline. Black arrows represent the sequence of events. (A) Stained slides were reviewed by an expert neuropathologist to generate a semiquantitative score of ptau pathology. (B) Glass slides were then digitized to generate whole slide images (WSIs). (C) Using HALO, gray matter (green), white matter (yellow), and glass (red) were identified with a trained Densenet AI. Gray matter annotations were converted to segmentations for downstream AT8% area and AT8+ cell density analyses. (D-F) Representative images of the analytic masks showing the type of pathology analyzed. (D) Nonmarked up, raw WSI. (E) Red overlay marks all AT8-positively stained pixels. (F) Blue overlay highlights AT8+ stained cells. Scale bar: 100 µm.

Table 1.

Demographics for full cohort.

	FHS	ADRC	UNITE
N	189	151	1072
Age at death	87 ± 10	85 ± 11	60 ± 20
Gender (male/female)	87/102	78/73	1037/35
Scanner (AT Turbo/GT450)	100/89	105/46	829/243
CTE (%)	1 (0.5%)	3 (1.9%)	712 (66.4%)
AD (%)	68 (35.9%)	81 (53.6%)	193 (18.0%)
LBD (%)	37 (19.5%)	35 (23.1%)	162 (15.1%
FTLD (%)	27 (14.2%)	24 (15.8%)	67 (6.2%)
MND (%)	2 (1.0%)	0 (0.0%)	28 (2.6%)

Full cohort demographics separated by brain bank. AD, Alzheimer disease; ADRC, Alzheimer’s Disease Research Center; CTE, chronic traumatic encephalopathy; FHS, Framingham Heart Study; FTLD, frontotemporal lobar degeneration; LBD, Lewy body disease; MND, motor neuron disease; UNITE, Understanding Neurologic Injury and Traumatic Encephalopathy.

Table 2.

Demographics for CTE subanalyses.

	No CTE	CTE low	CTE high
n	269	208	221
Age at death	45 ± 21	46 ± 18	67 ± 15
Gender (male/female)	244/25	207/1	221/0
Scanner (AT Turbo/GT450)	210/59	160/48	170/51
CERAD	0.06 ± 0.23	0.05 ± 0.21	0.24 ± 0.43

CERAD, Consortium to Establish a Registry for Alzheimer’s disease; CTE, chronic traumatic encephalopathy.

Clinical histories

Retrospective clinical evaluations and athletic history review were performed with next of kin as previously described.^35–37 Briefly, histories were assessed with telephone interviews and online questionnaire given to next of kin informants. For athletic history measures, informants were asked to describe sport played, level, position, age of first exposure to football, and duration. Researchers conducting evaluations were blinded to neuropathologic findings and informants were interviewed before receiving results of neuropathologic examination. Clinicians assessed a detailed history of participation in contact sport in addition to symptoms associated with mental illness, cognitive, behavior, and motor symptoms. Interviews are qualitatively summarized and a dementia diagnosis is determined using a modified Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision Criteria.³⁸ To assist with dementia assessment, the Functional Activities Questionnaire (FAQ) was used to determine the presence and severity of impairments.³⁹ Higher scores reflect more severe impairments. The cognitive difficulties scale (CDS) was also used to assess cognitive decline where higher scores are related to a greater degree of memory impairment.⁴⁰ Behavior and impulsivity were measures using the Barratt impulsivity scale-11 (BIS-11).⁴¹ Higher scores are also related to greater impulsiveness.

Histology and digital pathology

Tissue blocks were harvested from dorsolateral frontal cortex (DLFC) (Brodmann area 8/9) processed, paraffin-embedded, and cut to 10 µm thickness. The DLFC was selected because it is one of the first areas in the brain where ptau accumulates in CTE; thus, it is ideal to identify more subtle, early changes. Tissue blocks were taken from identical regions from all cases, as previously described.³⁴ Sections were then immunostained with the antibody AT8 (Invitrogen MN1020, 1:1000) to label ptau and ant-Iba1 (Wako 1:500) to label microglia, as previously described.²⁶ After staining was finalized, slides were digitized using an Aperio GT450 or AT Turbo (Leica Biosystems) slide scanner. The GT450 scans slides at 40× magnification with a final resolution of 0.263659 µm pixel⁻¹; the AT Turbo scans slides at 20× magnification for a final resolution of 0.5031 µm pixel⁻¹. Resolution differences were accounted for during the analytic step by setting a standard resolution for all images. WSIs are stored as .svs files. No preprocessing or postprocessing of images occurred to better examine the same visual elements that neuropathologists observe under a microscope for optimal comparison of measures. Slides were then examined with Indica Laboratories HALO.

Neuropathologic scoring and analyses

SQ neuropathologic scoring

The analytic pipeline is outlined in Figure 1. Histologic glass slides were first reviewed by neuropathologists to provide SQ scoring assessment of AT8 ptau pathology in the DLFC images (Figure 1A). Neuropathologists (T.D.S., A.C.M., V.E.A.) independently assessed the degree of ptau pathology using SQ rating scales of 0-3, where 0 = none, 1 = mild, 2 = moderate, and 3 = severe, as previously described.²⁵ All analyses were performed on the same slide.

Digital analysis

After SQ scoring, slides were digitized and imported into HALO for digital analyses (Figure 1B).²⁷ As tissue from each brain bank was processed the same way, that is, with identical methods and personnel, we did not divide samples between banks for training. For training, we pulled cases from the full 1412 dataset irrespective of pathological status. The training dataset was initially collected through random sampling. After samples were randomly selected, we reviewed all images and supplemented additional cases that had features not represented in the initial training set such as higher levels of background, positive staining cell morphologies not already captured, or novel debris elements that could flag false positive events. This method allowed us to have good representation across the full cohort that contains a variety of distinct cell shapes and background levels. All annotations were done on the full WSI. As our analysis was focused on the cortical gray matter, first we designed a DenseNet AI algorithm to automatically identify and select gray matter from white matter and glass (Figure 1C). Gray matter segmentation algorithm was trained using 21 unique images with 212 910 iterations to a cross-entropy of 0.137 using a resolution of 1.44 µm pixel⁻¹. The minimum object size was set to 0.5 mm². Annotations were reviewed by a human observer for accuracy together with a board-certified neuropathologist (T.D.S., A.C.M., V.E.A.) to verify proper selection of the intended structure or cell type. Gray and white matter annotation took approximately 2 days; model training took ∼48 h; and analysis of all images took approximately 2 days. Both AT8% area-stained analyses and AT8 cell density analyses were performed in the same annotation on the same image.

Computational resources

Analyses were performed using an AMD Ryzen Threadripper PRO 5955WX 16-Core 4.00 GHz processor with 64 GB of RAM. Artificial intelligence jobs were processed with a NVIDIA RTX 5000 Ada Generation and a NVIDIA Quadro p6000 graphics card.

AT8% area-stained analysis

After cortical gray matter was annotated, we tuned the HALO Area Quantification Algorithm v2.4.9 to recognize positive AT8 staining compared to nonstained background areas (Figure 1E). On images captured with a GT450 slide scanner, the color settings were 0.179, 0.313, 0.415 for AT8 and 0.211, 0.225, 0.146 for hematoxylin. The optical density (OD) settings were set to a minimum of 0.14 for AT8 and 0.15 for hematoxylin. For the images captured on the AT Turbo slide scanner, the color settings were 0.447, 1.026, 1.176 for AT8 and 0.211, 0.225, 0.146 for hematoxylin. The OD settings were set to a minimum of 0.27 for AT8 and 0.15 for hematoxylin. Thresholds were determined based on a trained observer visual setting the minimums, so the dimmest positive AT8+ cell in a WSI would get captured. They would then repeat this across multiple images with different levels of staining to verify the selected settings would accurately capture all pathology across all cases. The observer would then coordinate with a secondary observer to verify their settings were sufficient and acceptable to capture all pathology of interest. As images were captured over the course of several years, 2 different scanners were used. To verify selected pixel detection was consistent between both scanners, a select group of slides were scanned in on both scanners and OD settings were normalized to achieve similar positive pixel values between the two. Total area of positively stained pixels was divided by the total area measured to determine the AT8% area.

AT8+ cell density analysis

Analyses of AT8+ cells was carried out using HALO AI v4.0. First, we used the nuclear segmenter v2 AI algorithm to identify AT8+ objects with a cellular shape. This method is superior to other non-AI-driven cell shape algorithms as it can cover a wide variety of different types of abnormal shapes. Using a digital zoom between 20× and 40× magnification, annotations were manually drawn, using the pen tool in HALO, around the boarder of AT8+ cells while avoiding any hematoxylin background stain. This annotation is added to the class “Positive cell.” Any AT8+ cell was captured. This included neurons and astrocytes. We then used the rectangle selection tool to draw a second separate box around the new AT8 annotation that also captured the background hematoxylin. This annotation is added to the class “Background.” Using both the AT8 specific annotation and the box including the selected AT8 cell and immediately surrounding environment/background is how HALO learns to separate positive annotations from a nonpositive background. Additionally, as human neuropathology is rarely uniform, we also trained the algorithm to recognize AT8+ cell shapes in the context of various types and levels of background. The algorithm was trained with 72 unique images over 404 891 iterations at a resolution of 0.84 µm pixel⁻¹ to a final cross-entropy of 0.004. The minimum object size was set to 40 µm² to filter out small objects. After the nuclear segmenter was trained, it was applied to the HALO object phenotyper AI algorithm to create a classifier that could identify AT8+ cells. To separate out potential false-positive cells such as hemosiderin-laden macrophages or noncellular debris/artifacts (Figure 1F), we created 2 additional classes called “debris” and “hemosiderin macrophages” and included examples of each into the classifier. This allowed us to create an algorithm that could successfully identify AT8+ cells from background, debris, and hemosiderin-laden macrophages. This algorithm was also trained with 72 unique images over 162 985 iterations at a resolution of 0.45 µm pixel⁻¹ to a final cross-entropy less than 0.001. The number of AT8+ cells was divided by the area measured for a final density value. A human observer reviewed the final density measurements and spot checked 50% of cases to verify proper labeling and classification of phenotypes. Annotation of AT8+ cells took approximately 2 weeks, model training took ∼72 h to complete; analyses of all images took approximately 4 days.

Iba1+ cell density

Analyses of total number of Iba1+ microglia were performed using HALO AI segmenter algorithm. Twenty-three images were trained over 177 419 iterations at a resolution of 0.65 µm pixel⁻¹ for a final cross-entropy of 0.036. The minimum object size was set to 100 µm². Numbers of microglia were divided by the total area measured to determine the density.

Statistical analysis

AT8% area and cell density were normalized with log and square root transformations, respectively. To determine whether quantitative measures of percent area and cell density were mediated by SQ categorization and CTE stage, a one-way ANOVA was used. Linear regression was used to correlate analytic measures to linear continuous variable years of exposure to RHI, Iba1+ cell density, and clinical measures. A binary logistic regression was used for dementia history as it is a binary variable. AT8+ cell density vs AT8% area quartiles were separated by ranking the cases based on AT8+ cell density values and making a cutoff every 25% so that there was the same amount of samples in each group. Receiver operator characteristic (ROC) curves were used to assess predictive ability of tau measures on CTE yes/no status. An independent-sample t-test was used to compare no CTE vs CTE low. Statistical analyses were generated using SPSS (v24, IBM) and Prism (v8, GraphPad Software). Graphical representations were generated using GraphPad Prism and Adobe Illustrator (v29.1, Adobe).

RESULTS

Quantitative measures correlate with neuropathologist-derived SQ pathology scores

First, we investigated whether computer-derived quantitative measures of AT8% area-stained and AI-driven AT8+ cell densities were comparable to current gold standard human neuropathologist SQ scoring. Both quantitative measures correlated with neuropathologist SQ scoring (P < .001) (Figure 2A and B). In a subanalysis, excluding cases used for AT8+ cell density model training did not significantly affect or alter these results (P < .001). Interestingly, the observed overall sample distribution in each SQ category highlighted the variation that can occur within more subjective, human analyses. We next compared the agreement between each quantitative measure within the same sample (Figure 2C). We observed a general correlation between AT8+ cells and AT8% area but there were several outliers that appeared to be inconsistent with the overall trends (Figure 2C, arrows). When examining those outliers, we found that cases that skewed toward increased AT8% area compared to AT8+ cells had higher levels of noncellular tau such as neuropil threads and dot-like grains (Figure 2E and F). Additionally, AT8% area picked up on nonspecific staining artifacts (Figure 2D). In cases that had higher AT8+ cells compared to AT8% area, the majority of pathology was tau restricted to cell bodies (Figure 2G).

Figure 2.

Comparison of 3 analytic strategies demonstrates general agreement between methods and identifies outlier examples. Quantitative comparison of AT8% area (A) and AT8+ cell density (B) to neuropathologic SQ scoring. A one-way ANOVA showed quantification significantly increased following SQ severity ^***P < .001. Each dot represents 1 case. Horizontal black lines within each group represent the mean. (C) Direct comparison of AT8% area and AT8+ cell density in each case from the same tissue section. Color represents SQ pathologic severity based on neuropathologist autopsy assessment. Each dot represents 1 case. Examples of outlier samples that fell outside of the general correlation are annotated. (D-G) Representative images of samples demonstrated outlier cases that showed increased relative AT8% area compared to AT8+ cell density had frequent instances of artifact staining (D) or noncellular tau such as neuropil threads and dot-like grains (E and F). (G) In cases that had higher AT8+ cell density compared to AT8% area, the majority of staining was mainly restricted to cell bodies.

Quantitative measures can increase sensitivity to identify correlations between neuropathologic features

Next, we compared each of the 3 measures on their ability and sensitivity to correlate with neuropathologic features and clinical scales. As the full dataset was comprised of multiple types of neurodegenerative diseases, which have a spectrum of clinical and pathologic features, we performed a subanalysis in cases with CTE only to reduce noise in the data. Semiquantitative correlations with AT8% area and AT8+ cell density were similar in CTE-specific analyses (Figure 3A and B). When examining how well the SQ score, AT8% area, and AT8+ cell density correlated with CTE stage, all measures significantly correlated with disease severity (Figure 3C-E). Receiver operator characteristic curve analyses also demonstrated that all 3 measures were significantly able to predict CTE (P < .001). However, AT8+ cell density had a slightly higher AOC (0.904), compared to AT8% area (0.847) and SQ scores (0.892) (Figure 3F).

Figure 3.

Comparison of AT8 analytic measures using CTE cases. (A and B) Quantitative analysis of AT8% area (A) and AT8+ cell density (B) compared to neuropathologist SQ score within our CTE cohort. One-way ANOVA was used to generate statistics. (C-E) We then compared SQ scores (C), AT8% area (D), and AT8+ cell density (E) to CTE severity. Statistics was generated using a one-way ANOVA. Each dot represents 1 case, horizontal black lines within each group represents the mean. ^***P < .001. (F) ROC curves were then used to determine how well each of the 3 measures could predict a diagnosis of CTE. (G and H) Finally, we compared the 3 analytic measures to continuous variable of environmental factors (duration of contact sports exposure [years]) (G) and local cellular pathology (Iba1+ cell mm⁻²) (H). Linear regression was used to generate statistics. (I) Representative image of AT8 and Iba1 immunostains for each group. Scale bar represents 200 µm.

Both duration of contact sports exposure and microglial density are associated with tau pathology in CTE.^29,42 Therefore, we compared duration of play and Iba1+ cell density to our 3 pathologic measures of tau pathology. When comparing the 3 measures to the environmental stimulus of years of exposure to RHI, all 3 scores were highly significant (P < .0001) (Figure 3G). For the degree of neuroinflammatory changes (as measured by Iba1+ cell density), we observed that while all 3 measures demonstrated significance, AT8+ cell density had the highest significance (SQ: P = .028, AT8% area: P = .022, AT8+ cell density: P = .003) (Figure 3H and I).

Finally, we examined how well each of the pathologic ptau measures correlated with clinical symptom progression (Table 3). AT8% area, AT8+ cell density, and SQ scores significantly correlated with worse scores of every clinical measure other than BIS-11 (P < .05). The fully quantitative measures (percent area, cell density) demonstrated better significance values relative to the SQ measure for the CDS and FAQ, indicating lower estimated error. The quantitative measures also resulted in higher Beta values for the CDS and FAQ, indicating that they were stronger, more robust correlates. The odds ratio for dementia history was also stronger with the quantitative methods compared to the SQ method.

Table 3.

Correlation of neuropathologic measures with clinical symptom progression.

	Semi-quantitative	AT8% area	AT8+ cell density
CDS	β  =  0.080, p = 0.040a	β = 0.153, p < 0.001b	β  =  0.136, p = 0.001b
BIS-11	β  =  0.023, p = 0.600	β = -0.012, p = 0.807	β = -0.044, p = 0.351
FAQ	β  =  0.084, p = 0.011a	β = 0.142, p < 0.001b	β = 0.160, p < 0.001b
Dementia history	OR = 1.43, p < 0.001c	OR = 1.70, p < 0.001b	OR = 1.53, p < 0.001b

^a P < .05.

^b P < .001.

^c P < .01.

BIS-11, Barratt impulsivity scale-11; CDS, cognitive difficulty score; FAQ, functional activities questionnaire.

Agreement between AT8% area and AT8+ cell density is worse in cases with low pathology and driven by off target staining

Although AT8% area and AT8+ cell density were generally correlated, disagreement between measures (primarily associated with off target staining) was still observed (Figure 2C-G). To that end, we wanted to better understand how this could translate to experimental errors. Using our CTE cohort, we again correlated AT8% area and AT8+ cell density (Figure 4A). As with previous results, although there was general consensus between measures, there was not complete overlap. Therefore, we subdivided cases into 4 quartiles each containing 25% of the total dataset arranged in ascending order of AT8% area (Figure 4B-E). We then examined the overall correlation coefficients for all quartiles. The correlation coefficient for first (Figure 4B) and second quartiles (Figure 4C) was below 0.1 (low correlation). However, the third quartile had a moderate correlation of 0.53 (Figure 4D), and the fourth quartile had a high correlation of 0.79 (strong correlation) (Figure 4E).⁴³ The increase in correlation, along with the visual trend of increasing linearity, demonstrates that the methods agree more as both measures increase. A notable characteristic of the first 2 quartiles is the persistence of low AT8+ cell density values even as the AT8% area increases. We observed that common features such as staining artifacts, noncellular debris, and off target cells with pigmentation such as hemosiderin-laden macrophages contributed to the false-positive signal in the AT8% area analyses (Figure 4F). Additionally, subtle variation in background could also contribute to false-positive pixels.

Figure 4.

The severity of pathology impacts the relative strength of each quantitative measure. (A) To further explore the lack of total agreement between the AT8% area and cell density methods, we divided the CTE dataset into quartiles. (B-E) Each quartile was then plotted and analyzed independently. Each dot represents 1 case and the red lines represent the line of best fit. (F) Representative images of nonreal staining artifacts and pigmented cells such as hemosiderin-laden macrophages that were the main source of disagreement in the first and second quartile. (G and H) Quantitative analyses of cases with low or no CTE pathology demonstrated AT8% area (G) had more sample range overlap due to off target staining, while AT8+ cell density demonstrated more discrete group clustering (H). Independent sample t-test was used to generate statistics. Each dot represents a single case. Horizontal line represents mean. ^***P < .001, ^****P < .0001. (I) Finally, an ROC curve demonstrated that AT8+ cell density was a stronger predictor of CTE when only looking at no CTE vs low CTE cases.

Last, to examine the possible consequences of this low-end pathology discrepancy in neuropathologic studies, we performed a subanalysis comparing cases with no CTE and low-stage CTE. Although both AT8% area (Figure 4G) and AT8+ cell density (Figure 4H) were significantly different between no CTE and low CTE, the P-value for AT8+ cell density was smaller than that of the AT8% area by a factor of 10, indicating less predicted error in the distinction (P < .001, pP< .0001). This indicated a higher degree of separation between each group and suggests that the AT8+ cell density was a more sensitive analysis in the context of very low pathology. Finally, using an ROC curve analyses, both measures could predict no CTE vs low CTE (P < .001). However, AT8 cell density had a higher AOC (0.825) compared to AT8% area (0.729), demonstrated stronger ability to discriminate between CTE status (Figure 4I).

DISCUSSION

In this study, we provide a comprehensive analysis comparing 3 common methods for neuropathology analysis, that is, expert-driven SQ scoring, percent area-stained measurements and AI-driven cell density quantitation. Using CTE as our model system, we demonstrated that all 3 are effective in identifying and quantifying neuropathology. Furthermore, each method showed strong correlations between ptau pathology and clinical manifestations.

Although all 3 methods showed statistically significant findings in most cases, we identified important benefits and limitations for each approach. This suggests that there is no single best method for neuropathology image analyses, emphasizing the need for careful consideration when selecting the appropriate analysis technique. Based on these findings, we have developed a pro and cons list to assist researchers in choosing the best image analysis tool or process that aligns with their unique datasets and research objectives (Table 4).

Table 4.

Benefits, limitations, and recommended use strategies for neuropathologic image analyses.

	Description of method	Pros	Cons	Recommended use case
Semiquantitative scoring	An expert looks at tissue and obtains a subjective assessment based on established criteria.	Quickest to implement No computational equipment required Results are easiest to directly interpret	Ordinal data restricts analysis types Most subjective measure Lower reproducibility and variation between experts Lower ability to identify subtle changes	Researchers looking for quicker analyses with less necessity for fine detail. Also, useful when no computational analysis software is available
AT8% area	A color threshold is chosen that identifies positively stained pixels compared to background. The percent area is calculated from positive pixels divided by total pixels	Least subjective of all measures Lower computational burden compared to other quantitative methods Easy to implement	Positive area is nonspecific. Any other source of that color can introduce error or noise to model Prone to false positives	Cases that have uniform background and no nonspecific staining elements Useful for cross group collaborations as positive threshold can easily be shared Limited resources and computational tools are available
AT8+ cell density	An AI feature detection network is trained from a human-annotated dataset. The network can then identify the trained features in novel images	Most sensitive measure in identifying pathologic burden differences with low pathology More customizable, can be fine-tuned for particular cell types Can be trained to recognize positive cells in the context of different levels of background and staining intensity	Needs an appropriately large training dataset Important pathologic features with shape/texture not specifically included in training set are excluded from count Updating the model with new features requires retraining Higher computational requirements	Experiment contains diverse samples with varying levels of background, staining artifacts, and quality Cell type of interest can be found in multiple diverse/nonstandard shapes Have access to more powerful computational equipment

The SQ method is the most commonly used technique among the 3 evaluated and is used widely recognized in both clinical and pathologic settings. We and other have published numerous studies utilizing this method.^1,3,4,25,44 Compared to the quantitative methods, the SQ approach is simpler because it does not require slide digitization; instead, it relies primarily on an expert visually examining tissue samples and providing a grade based on their experience and preexisting criteria.^11,45 The benefits of this approach allow for quicker evaluation because information is taken directly from the physical slide rather than relying on additional layers of digitizing and filtering, as in the percent area and cell density models. However, this analysis paradigm also has several limitations. Since SQ scoring relies mainly on human observers, the resulting data are often more susceptible to higher levels of intraobserver and interobserver variability.^1,45 This variation stems from the subjective nature of the assessment, the inherent errors in human interpretation, and the broad definition of SQ categories.² Our findings align with previous reports indicating that human-driven SQ scores exhibit greater variance when compared to other quantitative measures, indicating that even trained experts might disagree on the subtle distinctions between “mild” vs “moderate” scores.^46–49 Additionally, because SQ scores are considered ordinal data, the difference between each category is not clearly defined, which can limit comparative analysis. For instance, the difference between a “severe” and a “moderate” might not be the same as that between a “moderate” and a “mild,” making a proper comparison challenging. Despite these limitations, that is, lower correlation with clinical measures, greater variance within groups, and restrictions on statistical analyses, the SQ method still yields significantly meaningful data and can serve as a valid tool for investigating neuropathology. Therefore, the SQ method is recommended for quicker analyses where less emphasis on fine details is necessary.

The percent area method, also known as the positive pixel method, is the most common fully quantitative technique and has been used in a wide range of studies across various scientific fields.^{1,9,10,49–52} At its core, the method evaluates individual pixels to determine whether they are positive or negative based on a color threshold established by a human user. This approach has several advantages. It is straightforward to implement and can yield rapid results. Additionally, it is less subjective than SQ measures because a defined threshold is consistently applied across all cases. The percent area method also offers a higher degree of reproducibility compared to SQ measures because the specific metrics are clearly defined. However, this method has notable limitations, as indicated by our data. First, user input is still necessary to determine the exact color threshold. Subtle changes in the threshold can significantly impact the results. Furthermore, the percent area method is highly prone to false-positive results because the positive pixel metrics are nonspecific. Any source that matches the target color, whether it be artifact or diagnostically irrelevant pathology, will be counted as positive. As demonstrated by our findings, artifacts such as dust, folded tissue, marker residue, background, already pigmented cells such as hemosiderin-laden macrophages, and even dopaminergic neurons can be falsely identified with percent area measurements. These factors become even more complex in postmortem human tissue in which each case might have varying backgrounds and levels of staining success. The inconsistency in staining and background across different slides introduces additional noise in the data, as the color threshold set for analysis may not identify the same level of pathology equally among all subjects. Nonetheless, the consistency, speed, and lower requirements of the percent area method have contributed to its frequent use in the field. Overall, the percent area measure is recommended for analyses where more pronounced differences are anticipated, as the noise from nonspecific elements is likely to obscure more subtle changes at the low end of pathology. Additionally, caution should be exercised when applying this staining analysis in the presence of variable backgrounds and staining across sample set.

The cell density analysis method is the most complex measure out of the 3 approaches used in the current study and has successfully identified ptau-positive cells in several of our past studies.^9,17,29 This method relies on machine learning algorithms trained to recognize specific cell types, offering numerous advantages.^53,54 Although it is complex, due to many of its advantages, it is becoming more popular in the field, as shown in several recent studies in AD, CTE, progressive supranuclear palsy, and primary age-related tauopathy.^17,55–57 The most significant benefit is its ability to detect a cell type regardless of variations in shape, background levels, or staining intensity. This capability allows for highly automated quantitation of the desired target. Such advantages are particularly valuable in human postmortem research in which cases typically exhibit varied postmortem intervals and tissue quality. Moreover, the significant morphological difference observed between neuronal pretangles, mature neurofibrillary tangles, and glial tangles can make it challenging for traditional cell recognition algorithms to identify all cell types accurately.^56,58 As highlighted by Koga et al, distinguishing between thorny astrocytes and subpial tau containing astrocytes can be difficult but important for disease classifications.⁵⁸ Therefore, an approach that allows for training on a broad range of cells for targeted identification is optimal for achieving specificity. However, the highly specialized training required for the cell density method has some drawbacks. One major limitation of this AI approach is that it excludes cells or features with shapes not included in the training set.⁵⁹ For example, in our dataset, we observed that the accuracy of the density method for AT8-positive cells did not capture neuropathologic features with dot and thread shape. These features were better identified by the percent area method. If the pathology of interest is dots and threads, this would be a good example of a specific use case where the percent area method would be the most optimal strategy. Additionally, diseases such as argyrophilic grain disease, which are characterized by “dots” or grains of tau, would benefit more from the percent area method. Similarly, dot and thread pathology might be early markers and herald neurofibrillary tangle aggregation, so using just the density method might miss all features. These examples highlight that there is no “best” or “one-size-fits-all” approach to images analyses and careful selection of appropriate methods is needed.

Another disadvantage of the cell density method is its reliance on a properly constructed training dataset. An image-detection network is only as reliable as the data that are trained on; poorly chosen or inaccurately annotated data can lead to misleading results.¹¹ The steps of data selection and human annotation are subjective and critical to the process but introduce potential errors.^60–63 Moreover, training and developing an AI network demands significantly more computing power compared to simpler digital methods, such as the percent area approach.^64,65 However, for studies involving a wide range of cellular pathology in conjunction with varied background conditions, frequent debris, or other artifacts, the AI-driven cell density method might prove to be especially beneficial.

The limitations of the study include its sampling methods and the AI tools utilized. Our study cohort consists of individuals whose brains were donated by their families, which does not reflect the general population and suggests a selection bias. Additionally, the clinical metrics and RHI exposure data were gathered retroactively from informants, introducing potential bias. Additionally, while our use of HALO AI offered a more advanced analytic approach, HALO is a predesigned commercial software with ready-to-use features. This partially limits the customizability and ability to fully utilize or understand the AI algorithms. Furthermore, although no workflow limitations were found in this study, other groups have reported crashes and slowdowns using HALO, which could limit its usefulness.⁶⁴ More customizable approaches, with tool kits built from the ground up, and with full transparency into algorithm development, will likely increase our ability to identify neuropathology and associated neurodegenerative changes. The field of digital image analysis is rapidly evolving, with the current standard involving multiple convolutional layers that evaluate every feature of an image.¹⁰ Therefore, future work will be necessary to optimize and validate these new techniques for practical use in neuropathology. Additionally, future studies will need to incorporate the use of holdout sets and comparisons with other distinct brain bank datasets to verify accuracy and effectiveness of models.^54,66,67 Because the primary goal of the current study was to compare and contrast the different analytic strategies within individual WSI, there was not a need to control for batch or other demographic variables. However, future studies that utilize these techniques to identify disease-specific features will need to incorporate elements such as batch, scanner, copathology, institute of origin, and even staining date to help better filter out variability and identify subtle disease changes. Although the current study did not use any postimage processing or color correction methods, these tools might be useful to increase the sensitivity of future studies.

Finally, the current study only focused on a single brain region with a single stain, the DLFC, and AT8. Although this region was chosen due to its early involvement in CTE and was well suited to identify early, more subtle changes, it is possible that model performance will differ in other brain regions. Areas like the hippocampal subfields, amygdala, and even brain stem are equally important to capture pathologic accumulation. These regions have a different composition of cells and different tissue architecture. Therefore, analysis techniques that were developed on 1 brain region are not likely to be easily transferred across the brain. However, simpler techniques like positive pixel measures, which rely just on color gradient, could be more readily applied to a unique tissue type with careful adjustment. The more complex AI techniques will likely perform worse on novel tissue types. Therefore, either AI algorithms will need to be initially trained on a diverse set of tissue regions or individual algorithms could be created for each brain region to avoid inconsistencies.

In conclusion, this report compares 3 common methods of histologic analyses applied to the same set of cases, highlighting the benefits and limitations of each methodology. Our results indicate that all 3 methods effectively differentiate between varying levels of disease severity and CTE stages, establishing their viability as analytic tools. However, we identified specific scenarios where 1 technique might outperform the others. From our comparisons, we developed a set of recommendations to inform researchers about the advantages and limitations of each method (Table 4). The human expert-driven SQ method proved ideal for more rapid analyses. In contrast, the percent area/positive pixel analysis method excelled in disease models characterized by clear pathologic differences and stable backgrounds, minimizing the risk of false positives due to artifacts. On the other hand, though the cell density method exhibited the lowest error rate and highest sensitivity, its reliance on a predetermined training set restricts analyses to specific trained elements, limiting flexibility. While each method has its strengths, it may be beneficial to combine all 3 approaches for a more comprehensive identification of changes, as suggested by previous research.⁵⁰ Nonetheless, this combined approach requires significantly more resources, making it impractical for most researchers. Therefore, it is essential for researchers to carefully select the most appropriate method for analyzing neuropathology data. We hope that the insights presented in this work will serve as a guide for future researchers, enabling them to select a methodology that best aligns with the specific needs of their studies for optimal results.

FUNDING

The work was supported by grant funding from: National Institute of Aging (P30AG072978 and AG090553), National Institute of Neurological Disorders and Stroke (U19-AG068753 and R01NS142076), Department of Veterans Affairs Biorepository (BX002466), and the Department of Veterans Affairs Career Development Award (BX004349). The views, opinions, and/or findings contained in this article are those of the authors and should not be construed as an official Veterans Affairs or Department of Defense position, policy, or decision unless so designated by other official documentation. Funders did not have a role in the design and conduct of the study; collection, management, analyses, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

CONFLICTS OF INTEREST

None declared.