A multi-resolution textural approach to diagnostic neuropathology reporting

Abstract

We present a computer aided diagnostic workflow focusing on two diagnostic branch points in neuropathology (intraoperative consultation and p53 status in tumor biopsy specimens) by means of texture analysis via discrete wavelet frames decomposition. For intraoperative consultation, our methodology is capable of classifying glioblastoma versus metastatic cancer by extracting textural features from the non-nuclei region of cytologic preparations based on the imaging characteristics of glial processes, which appear as anisotropic thin linear structures. For metastasis, these are homogeneous in appearance, thus suitable and extractable texture features distinguish the two tissue types. Experiments on 53 images (29 glioblastomas and 24 metastases) resulted in average accuracy as high as 89.7 % for glioblastoma, 87.5 % for metastasis and 88.7 % overall. For p53 interpretation, we detect and classify p53 status by classifying staining intensity into strong, moderate, weak and negative subclasses. We achieved this by developing a novel adaptive thresholding for detection, a two-step rule based on weighted color and intensity for the classification of positively and negatively stained nuclei, followed by texture classification to classify the positively stained nuclei into the strong, moderate and weak intensity sub-classes. Our detection method is able to correctly locate and distinguish the four types of cells, at 85 % average precision and 88 % average sensitivity rate. These classification methods on the other hand recorded 81 % accuracy in classifying the positive and negative cells, and 60 % accuracy in further classifying the positive cells into the three intensity groups, which is comparable with neuropathologists’ markings.

Introduction

Although we have advanced our understanding of the molecular underpinnings of glioblastoma pathobiology, the median survival for GB patients is still ~15 months. Even amongst experts, diagnostic interpretations represent subjective analyses by pathologists, show highly variability, and such subjectivity confounds clinical decision-making, ultimately affecting patient outcomes. The initial notion that distinct morphological patterns may predict patient outcome was supported by the discovery that GB patients showing microvascular proliferation, but no necrosis, have a slightly prolonged survival [1]. However, reliance on subjective morphologic interpretations is fraught with caveats. For example, patients showing a morphological component of oligodendroglioma-like cells in GB were reported to have a median survival of 26 months [2], but other studies have been unable to duplicate this finding [3,4]. Indeed, the nosology of such mixed gliomas is highly debated amongst experts [5]. The disparity between groups may result from GBs known intra-tumoral heterogeneity [6], but subjectivity of diagnostic and prognostic tests are known barriers to progress in clinical trials research for glioma patients (discussed by Kim et al.) [7].

Although the advent of novel biomarkers in glioma diagnostics may ultimately diminish classification challenges for neuropathologists [5, 8, 9], objective quantitative measurements of biomarker expression remain a challenge in routine diagnostic practice [10]. As GB cells permeate the brain parenchyma as individual cells, GB tissue specimens often represent mixtures of neoplastic and non-neoplastic cells. Thus, modern molecular tests capable of providing prognostic information and management decisions must be interpreted in the context of tumor morphology. Otherwise, total cell homogenates would be composed of a mixture of malignant and non-malignant components. Prognostic tests affected by this barrier include ki67-labeling indexes [11], p53 analysis [12], EGFR analysis [13], and detection of genomic alterations by fluorescent in situ hybridizations (reviewed by Horbinski et al. [14]). A significant motivation of this work was to generate a simple image analysis algorithm that could facilitate objective diagnostic and prognostic reporting for neuropathologists. We developed our analysis to focus on two branch points in diagnostic neuropathology workflows: intraoperative consultation (i.e., “frozen section”) and prognostic reporting of glioma. From an image analysis perspective, although these images represent distinct visual challenges for neuropathologists, we were able to utilize similar mathematical approaches.

The current status quo workflow in diagnostic neuropathology begins with an intraoperative consultation. If this test is requested, a cytologic prep (smear) and/or frozen section is performed. These procedures take ~20 min to complete, requires specialized training, and can be utilized to identify viable neoplasm in samples. Additional tissue, if available, would then be submitted for formalin fixation and paraffin embedding (FFPE) where pathologists report the tumor type, WHO grade, and additional prognostic markers. Standard immunohistochemistry markers currently utilized in clinical practice carrying prognostic value include ki67, p53, IDH1R132H, and ATRX. Although the advent of whole genome sequencing of tumors will ultimately improve medical decision-making for these patients [15], traditional diagnostic interpretation of these samples is still needed for at least two circumstances. First, cytologic preparations represent a high-yield methodology to determine tissue type and therefore are an optimal and low-cost methodology to triage tissues for molecular testing. Second, whole genome sequencing methodologies represent whole cell homogenates, and therefore such metrics represent averages of the whole tissue. Obtaining expression data from individual tumor cells in tissue preparations would provide an invaluable adjunct to genomic tests that utilize whole cell homogenates.

Within this context we generated digitized image analysis workflows aimed at aiding/supplementing pathological interpretation. We focused on two diagnostic branch points in clinical decision-making: intraoperative consultation and prognostic reporting with p53 immunohistochemistry. The p53 tumor suppressor gene is frequently mutated or lost early in gliomagenesis. Normal p53 has a short half-life resulting in poor immunohistochemical detection; in contrast, TP53 mutation leads to elevated and detectable p53 protein levels [16]. TP53 mutations correlate with worse survival in glioma patients [12]. Studies in other tumor paradigms have shown that the staining intensity correlates with TP53 mutation status [17]. Nevertheless, reporting p53 expression as a proxy for TP53 mutation status is highly subjective. Furthermore, gliomas show inter-tumoral heterogeneity in p53 mutation status [18]. Thus, p53 immunohistochemistry is an optimal paradigm to develop image analysis algorithms. Digital histopathological analysis by computer-aided image analysis algorithms has already been shown to increase diagnostic accuracy in follicular lymphoma and neuroblastoma [19–34]. We were able to address these two decision branch points (intraoperative consultation and p53 immunohistochemistry evaluation) by implementing identical image analysis methodologies through texture analysis by means of discrete wavelet frames features and k-nearest neighbor clustering.

Materials and methods

Patient selection, demographics, and study design

Retrospective cases were evaluated from the Pathology Tissue Archives of The Ohio State University Wexner Medical Center under an IRB approved protocol. Demographic information for the two analyses is presented in Supplemental Tables 1A and 1B. For the intraoperative consultation study design, the inclusion criteria required that patients undergoing intraoperative consultation during neurosurgery have received an unequivocal diagnosis of a neoplastic tissue, metastatic neoplasm or glioma at the time of intraoperative consultation. The final pathology report served as the ground truth for the designation of metastasis versus primary high grade glioma for evaluation. A total of 53 cytological smear images were used in the experiment, 29 of which belong to glioblastoma and 24 of which belong to carcinoma metastatic to brain. Image resolution is 1040 × 1388 pixels. Due to the limited number of samples, a leave-one-out strategy is used in the classification stage. Using this approach, all 53 images are used as test images. When a particular image is being used as test image, the remaining 52 images are used as training. The inclusion criteria included one case of chondrosarcoma, and one case of lymphoma; these for simplicity were grouped in the metastatic neoplasm category as the texture of the non-nuclei regions were more similar to the metastatic brain cancer group.

For the p53 analyses, 48 cases of high grade glioma retrospective cases were evaluated. Inclusion criteria included the need for an unequivocal diagnosis and grade (confirmed by neuropathologist JO) that had undergone p53 immunohistochemistry. To determine the ground truth for p53 staining, two board-certified neuropathologists and one pathology research fellow reviewed six scanned/digitized slides immunostained with p53. First, to determine inter-observer variability, the three pathologists scored separately the status of each cell in a high-power field (HPF), at ×40 magnification, as “negative stain,” “weak intensity,” “moderate intensity,” or “strong intensity.” Three pathologists viewing the specimen concurrently were needed to establish the ground truth as intraoberserver variability was too high for our algorithm development if only one pathologist was used as the ground truth. Then, using a similar grading scheme, each pathologist proceeded to mark another five HPFs from five different cases.

Immunohistochemistry, photomicroscopy, and slide scanning

The intraoperative cytologic preps were prepared during the normal course of OSU’s clinical service work and stained with H&E. After the cases were obtained from the retrospective search, 10× and 40× images were captured on a Zeiss Imager.M2 microscope using the following objectives: 10×/0,3 EC-PLANNEOFLUAR, and 40×/0,75 EC-PLANNEOFLUAR. Stereoinvestigator^™ was used to generate a white balanced image, and the raw images were captured and saved in.tiff format. In general, hospital policies require rendering an intraoperative consultation within 20–30 min from specimen receipt by pathology. Therefore, for this approach, we did not perform digitized slide scanning on these samples.

p53 immunohistochemistry was performed in the clinical labs of the OSU Department of Pathology using Dako clone D0-7 (diluted 1:1500). The specimens with p53 immunohistochemical staining were separate from the specimens utilized for the cytological preparations. Slides were subjected to antigen retrieval for 30 s in a pressure cooker in citrate buffer, pH 6.0. DAB precipitation was performed using Novolink polymer (Leica). For the p53 immunohistochemical staining analysis, slides were scanned at ×40 magnification using a high-resolution whole slide scanner Aperio (Vista, CA) ScanScope^™ at the resolution of 0.23 micrometer/pixel. The digitized slides were then transferred to a server where a web-based software is in place for the pathologists to independently mark the cells to determine the ground truth.

Mathematical models derived for image analysis

For both the intra-operative consultation and prognosis applications, discrete wavelet frames (DWF) [35–42] and k-nearest neighbor classification were used, and are included in the supporting materials.

Results

Successful classification of glioblastoma and metastatic neoplasm samples by automated image analysis

Using the mathematical models, we sought to classify two types of brain tumors, glioblastoma and metastatic neoplasms to brain, which are the most common differential diagnosis for an intracranial space occupying lesion during intraoperative consultation for neurosurgery (see Fig. 1a). Since the nuclei represent irregular images in both groups, we developed an innovative approach of modeling the background and foreground structures separately, which builds upon our previously developed model-based intermediate representation approach [43, 44]. We focused on cytological preparations because they are more sensitive than frozen H&E section [45], and they can be prepared with less tissue consumption than a traditional frozen section.

Fig. 1.

Segmentation of microscopic images. Rows 1 and 2 are examples of glioblastoma segmentation, whereas rows 3 and 4 show the same for metastatic carcinoma. Left column is the original .tiff Bright field photomicrograph, middle column is nuclear segmentation, and right column is the non-nuclear segmentation. Removal of the non-nuclei regions exposes the diagnostically relevant region of interest where glial processes are noted

The input images to our system are H&E-stained tissue slides digitized at ×10 magnification. Figure 1 shows two examples each from the glioblastoma and metastasis tissue samples, and their segmentation results. Visually, after nuclear segmentation, the regions of interest consist of either the anisotropic thin line segments of the glioblastoma or the homogeneous region of the metastatic tissue. We focused the design of the algorithm to interrogate these inter-nuclear regions so that our algorithm would be capable of broadly being utilized in other tumor comparisons. Although the cytologic morphologies of metastatic carcinoma differ from other entities, such as lymphoma, inflammatory, or infectious processes, all of these entities share in common a lack of anisotropic glial filaments and it is likely that the process outlined here has the potential for broad application to these entities as well. Specifically, our novel workflow showed improved diagnostic accuracy relative to DWF without segmentation (88.7 % compared to 83 %, respectively).

Table 1 shows the classification accuracy for glioblastoma and metastasis cases against several k-nearest neighbor values. Our method achieves correct classification as high as 88.7 % (only six misclassifications out of 53) for k = 1. Supplemental Fig. 1 shows the six wrongly classified images, three from each glioblastoma and metastasis classes. These images show technical artifacts during the cytological smear preparations, such as air drying (Supplemental Fig. 1a) and excessive pressure (Supplemental Fig. 1d and e). Table 1 shows performance of our methodology without segmentation, i.e. applying the DWF directly on the whole image, which shows worse performance relative to the segmentation strategy. Our method’s performance shows more consistency across different values of k, where the accuracies are all more than 83 %. This suggests that the features extracted from the segmented regions of interest are better clustered in the feature space.

Table 1.

Percentage of correct classification

K-nn	Proposed method			Without segmentation
	Overall (%)	Glioblastoma (%)	Metastasis (%)	Overall (%)	Glioblastoma (%)	Metastasis (%)
1-nn	88.7	89.7	87.5	83.0	82.8	83.3
3-nn	86.8	86.2	87.5	86.8	86.2	87.5
5-nn	86.8	89.7	83.3	84.9	86.2	83.3
7-nn	83.0	82.8	83.3	77.4	82.8	70.8

Improved p53 immunohistochemistry reporting by digitized image analysis

Immunohistochemistry of p53 shows a gradation between no staining and dark brown straining on DAB-based methodologies. It is generally accepted that diffuse, strong immunoreactivity to p53 in tumor cells is indicative of a TP53 missense mutation resulting in abnormally long p53 half-life [46–48]. In addition, some non-CNS tumor paradigms have also found lack of p53 expression to correlate with other TP53 mutation variants [49]. With this in mind, we generated a four tiered grading scheme comprising strong, intermediate, weak, and negative p53 immunohistochemistry staining. Classification of p53 immunohistochemistry intensity into these categories is not utilized in clinical practice due to its subjectivity, but it has been utilized in the scientific literature [50]. Supplemental Fig. 2 shows 50 examples of strong, moderate, weak, and negatively stained cells of a p53 image as marked by the pathologists. Tumor genetic heterogeneity is a highly relevant emergent area in clinical and research settings and assessment of staining intensities such as for p53, which correlates with TP53 mutations, may hence become very relevant in these settings. Thus, methods capable of determining staining intensities through objective and unbiased modalities for immunohistochemistry are highly relevant. While our current iteration of image analysis approach is not intended to be used clinically, it provides proof of principle of the utility of the proposed methods. The 50 samples for the strong, moderate, and weak intensity cells were used as the training samples in the k-nearest neighbor classification of discrete wavelet frames features in classifying the positive cells into the three positive cell classes. The cells’ characteristics are subjective, especially between the three sub-classes of the positively stained cells. One of the challenges in classifying positive cells is the staining intensity differences between images. Because of this, cells with similar intensities may belong to different strength classes of different tissue, and the moderate intensity cells especially suffer as they can be easily classified into the other two classes. The inter-reader agreement also suffered as a result, underlining the very difficult task faced by the pathologists in prognosis reporting of p53-stained tissues.

Inter-reader variability analysis

Table 2 shows the summary of cell marking between the three pathologists. Overall the three pathologists’ classifications into positive and negative cells differs quite considerably. Table 3 further examines the pathologist agreement by looking at their classification of each cell. Each column gives the number of cells where the pathologists are in agreement with each other. For example, in the first row of Table 3, pathologists A and B gave the same classification for 966 cells (207 for positive cells, and 759 for negative cells), which is equivalent to 83.8 % of the 1153 total cell marked. Note that the agreements between any two pathologists are between 78.8 and 83.8 %, which drops to 70.6 % when all three pathologists are considered. Table 4 separates the positive nuclei agreement into strong, moderate, and weak positive cells, and the percentage of agreement drops even further (between 55.4 and 59.2 % for any two pathologists, and a very low 39.5 % agreement when all three pathologists are considered). These analyses show that even expert pathologists differ quite significantly in classifying positive and negative cells in digitized p53-stained images, and considerably when further classifying the positive cells into the three tiers of staining strength. We conclude that traditional methodologies to report staining intensity of p53 result in high inter-observer variability which can be improved by automated image analysis.

Table 2.

Positive and negative cell counts by different pathologists

	Strong	Moderate	Weak	Total positive	Negative	Total
Pathologist A	148	271	362	781	325	1106
Pathologist B	24	372	472	868	285	1153
Pathologist C	119	196	405	720	378	1098

Table 3.

Marking agreement between pathologists (positive/negative)

	Positive	Negative	Total	Pctg
A & B	759	207	966	83.8
A & C	673	235	908	78.8
B & C	691	220	911	79.0
A & B & C	641	173	814	70.6

Table 4.

Marking agreement between pathologists (strong/moderate/weak/negative)

	Strong	Moderate	Weak	Negative	Total	Pctg
A & B	24	185	267	207	683	59.2
A & C	102	134	211	235	682	59.2
B & C	24	144	251	220	639	55.4
A & B & C	24	101	157	173	455	39.5

Computer detection and classification of strong–moderate–weak-negative cells

Supplemental Fig. 3 shows the flowcharts of our detection and classification approaches. The details of the methods are described in the supporting materials. Table 5 summarizes the performance of the detection technique. ‘Cases’ refers to the different digitized tissue, ‘GT’ is the ground truth by pathologist A, B or C, ‘TP’ is the true positive (correctly detected cells), ‘FP’ is the false positive (incorrectly detected cells) and ‘FN’ is the false negative (missing or undetected cells). Equations utilized for sensitivity, precision, and accuracy are delineated in the supplemental materials and methods.

Table 5.

Performance of the proposed cell detection technique

Cases	GT	TP	FP	FN	Precision	Sensitivity
1	A	844	188	32	0.96	0.82
1	B	878	200	49	0.95	0.81
1	C	852	187	49	0.95	0.82
2	A	420	77	43	0.91	0.85
3	A	221	12	73	0.75	0.95
4	A	257	15	30	0.90	0.94
5	A	185	9	82	0.69	0.95
6	A	172	23	69	0.71	0.88
Average					0.85	0.88

Our detection method showed 85 % precision and 88 % sensitivity rate. The sensitivity rate in particular is very promising as all cases recorded more than 80 % rate. This suggests that our system is able to detect as many cells as identified by the pathologists. The precision rate on the other hand is slightly lower, suffering from slightly high false positive cases. Some of this is also caused by the pathologists missing some cells when marking the ground truth. Note that there could be more than 1000 cells in one viewed field causing the pathologists to unintentionally miss some of the cells.

Table 6 shows the positive–negative classification accuracy for the six images. As seen from the table, the accuracy varies between 0.76 and 0.95, with an average of 0.81. Considering the agreement between any two pathologists is between 78.8 and 83.8 %, and between all three pathologists is 70.6 %, the results obtained are very promising, suggesting that our system measures up well with the given task. For case 1, we can also compare how the system measures against the different ground truth of the three pathologists, and in this particular case, it tends to agree more with pathologist C. Figure 2 shows one example (Case 2) of the detection and classification against the available ground truth. The strong, moderate, weak and negative cells identified by the system are marked with red, yellow, green and blue squares respectively. For reference purposes, the cells’ classes identified by the pathologists are marked with asterisks following the same color scheme.

Table 6.

Summary of positive–negative classification accuracy

Cases	GT	Correctly classified	Incorrectly classified	Accuracy
1	A	652	192	0.77
1	B	681	197	0.78
1	C	691	161	0.81
2	A	224	96	0.77
3	A	193	28	0.87
4	A	209	48	0.81
5	A	175	10	0.95
6	A	130	42	0.76
Average				0.81

Fig. 2.

Sample result. Example of detection and classification by the system (case 2) against the available ground truth

Table 7 shows the strong-moderate-weak classification accuracy for the six images. Although the overall accuracy of 60.1 % is not very high for a 3-class problem, it is comparable to the agreement between any two pathologists (55.4–59.2 %), and much higher compared to the agreement between all three pathologists (39.5 %). The low agreement between the pathologists suggests that classifying the positive cells into strong, moderate, or weak cells is a very challenging task; and that our classification algorithm is able to perform the task quite reliably. Note that the average accuracy is dragged down by the three classifications on Case 1 where the inter-reader agreement analysis were carried out, which reports very low agreement between the three pathologists. It is impossible to get high accuracies for this image against all three ground truths when they vary rather significantly.

Table 7.

Summary of strong–moderate–weak classification accuracy

Cases	GT	Correctly classified	Incorrectly classified	Accuracy
1	A	237	237	50.0
1	B	291	208	58.3
1	C	226	236	48.9
2	A	108	69	61.0
3	A	124	69	64.2
4	A	101	71	58.7
5	A	115	60	65.7
6	A	73	26	73.7
Average				60.1

Discussion and conclusion

In this study, we generated digitized image analysis workflows aimed at aiding/supplementing pathological interpretation focusing on two important diagnostic neuropathology branch points in clinical decision-making: intraoperative consultation and prognostic reporting of p53 in glioblastoma patients. Reporting of p53 immunohistochemistry was utilized as a paradigm to develop a workflow capable of providing objective staining intensity interpretation of nuclear epitopes.

For the intraoperative consultation, we developed a methodology for the classification of cytologic preparations from glioblastoma and metastasis sample tissues in the setting of intraoperative neuropathology by means of foreground and background modeling. Even though only a small quantity of the tissue was imaged, our method, which is based on the segmentation of a region of interest by means of visually meaningful decomposition, discrete wavelet frames, and k-nearest neighbor classification, was able to correctly differentiate glioblastoma and metastasis tissues in 47 out of 53 cases (88.7 % overall accuracy relative to neuropathologists who had the luxury of evaluating the entire slide and frozen section). The ability to improve diagnostic accuracy from minute sample preparations has significant implications. First, computer aided imaging can become a very useful adjunct on diagnostically difficult cases, such as those obtained from eloquent cerebral cortical or brainstem sites. Second, tissue exhaustion during the intraoperative consultation can preclude downstream testing of small samples. Therefore, any modalities capable of preserving tissue for downstream preparations are needed. The promising performance of our method relies on the extraction of textural features from the segmented region of interest instead of the whole image, which is proven through the classification experiment. Some of the incorrectly classified cases were found to be influenced by external factors such as air drying and excessive pressure during cytological smear preparations.

For the prognosis applications, we have developed a methodology for the detection and classification of p53-stained cells into 4 classes by means of novel thresholding for the detection, two-step rule based on weighted color and intensity for the classification for the positive–negative classification, and texture analysis for strong-moderate-weak classification. Such expression profile reporting is inherently subjective and prone to high intraobserver variability. Thus, our first challenge was to implement “consensus ground truth” generated by three pathologists viewing the data concurrently. This consensus ground truth was used to train the algorithm. These methods were able to achieve 85 % average precision and 88 % average sensitivity rate for cell’s detection, 81 % accuracy in classifying them into positively or negatively stained cells, and 60 % accuracy in classifying the positive cells into three strength classes. Considering the subjective nature in differentiating between the positive cells, as well as between the weakly stained positive cells from certain negative cells and even non-cells, and the fact that inter-observer variability is rather high (only 70 % agreement among three pathologist for the positive–negative classification, and 40 % for the strong–moderate–weak classification), our system provides promising results for both the detection and classification tasks. We conclude that this workflow classifies immunohistochemistry intensity data in a more reliable and objective fashion compared to traditional reporting. We further conclude that similar consensus ground truth implementations will be needed to develop similarly unbiased/computer aided prognostic marker reporting mechanisms. We envision that these approaches could be implemented as diagnostic adjuncts in settings where pathologists utilize whole slide imaging for routine diagnostic use.