Abstract.
The role of Ki-67 index in determining the prognosis and management of gastroenteropancreatic neuroendocrine tumors (GEP-NETs) has become more important yet presents a challenging assessment dilemma. Although the precise method of Ki-67 index evaluation has not been standardized, several methods have been proposed, and each has its pros and cons. Our study proposes an imaging semiautomated informatics framework [semiautomated counting (SAC)] using the popular biomedical imaging tool “ImageJ” to quantify Ki-67 index of the GEP-NETs using camera-captured images of tumor hotspots. It aims to assist pathologists in achieving an accurate and rapid interpretation of Ki-67 index and better reproducibility of the results with minimal human interaction and calibration. Twenty cases of resected GEP-NETs with Ki-67 staining that had been done for diagnostic purposes have been randomly selected from the pathology archive. All of these cases were reviewed in a multidisciplinary cancer center between 2012 and 2019. For each case, the Ki-67 immunostained slide was evaluated and five camera-captured images at magnification were taken. Prints of images were used by three pathologists to manually count the tumor cells. The digital versions of the images were used for the semiautomated cell counting using ImageJ. Statistical analysis of the Ki-67 index correlation between the proposed method and the MC revealed strong agreement on all the cases evaluates (), with an intraclass correlation coefficient of 0.993, “95% CI: 0.984 to 0.997.” The results obtained from the SAC are promising and demonstrate the capability of this methodology for the development of reproducible and accurate semiautomated quantitative pathological assessments. ImageJ features are investigated carefully and accurately fine-tuned to obtain the optimal sequence of steps that will accurately calculate Ki-67 index. SAC is able to accurately grade all the cases evaluated perfectly mating histopathologists’ manual grading, providing reliable and efficient solution for Ki-67 index assessment.
Keywords: gastroenteropancreatic neuroendocrine tumors, histopathology, ImageJ, Ki-67 index, manual counting, semiautomated method
1. Introduction
Neuroendocrine neoplasms are epithelial neoplasms with predominant neuroendocrine differentiation. Although they can arise throughout the body, those that arise in the pancreas or the tubular gastrointestinal tract are named gastroenteropancreatic neuroendocrine tumors (GEP-NETs).1–4 They are separated into two major categories including well-differentiated neuroendocrine tumors (NETs) and poorly differentiated neuroendocrine carcinomas. Although the latter group is associated with a rapid clinical course and poor prognosis, the former is a heterogeneous group that shows a spectrum of aggressiveness and unpredictable biologic behavior in relation to the morphologic features.5–7
It is recommended to grade the well-differentiated GEP-NETs using a three-tiered grading system to provide better reflection of the tumor aggressiveness and better guidance for chemotherapy.8 This grading system is endorsed by the American Joint Committee on Cancer,9 the European Neuroendocrine Tumor Society (ENTS),10 and the World Health Organization (WHO).11 GEP-NET grading relies on assessment of tumor proliferative rate by both mitotic count and Ki-67 labeling index, as summarized in Table 1.
Table 1.
Recommended grading system for well-differentiated GEP-NETs.11
| Grade (G) | Mitotic rate (per ) | Ki-67 index (%) |
|---|---|---|
| Well-differentiated NET, G1 | ||
| Well-differentiated NET, G2 | 2 to 20 | 3 to 20 |
| Well-differentiated NET, G3 |
Mitotic rate should be reported as the number of mitoses per . At least should be evaluated in the most mitotically active part of the tumor.11 Ki-67 index is reported as percentage of positive tumor cells—which are the cells showing nuclear immunoreactivity—in areas of highest nuclear labeling or hotspots.11 Grade assignment based on Ki-67 index is typically higher than that of mitotic count. If the case is evaluated by both methods, then the case is assigned according to the higher value.11 Both methods are recommended to grade well-differentiated GEP-NETs, yet mitotic rate is less preferable due to high interobserver variability; therefore, Ki-67 index is considered a better measure.
In order to assess the expression of the proliferation index marker Ki-67, a number of methods have been used, including eyeballing estimate (EE), manual counting (MC), and automated/semiautomated counting (SAC).12–14 Eyeballing can be used for most tumors; however, for tumors with Ki-67 index close to grade cut-offs (Table 1), it is recommended to manually count the print of a camera-captured image of the hotspot. In the MC, it has been recommended that a minimum of 500 tumor cells be counted to determine the Ki-67 index, and a notation is made if fewer cells are available.11 Automated counting is considered reliable, reproducible, and accurate; however, it is not widely applied as it requires careful and accurate manipulation of the software parameters used to find the best fine-tuned settings to avoid imprecisions.12–14 Tang et al.12 mention that the SAC method (Aperio immunohistochemistry nuclear quantitative image analysis algorithm) that was used requires manual calibration of the threshold for the size and shape of tumor cells in order to exclude stromal cells and lymphocytes. In addition, pathologists also need to outline tumor regions of interests before running the algorithm. Other commercial image analysis software such as automated cellular image cytometer III (DAKO, Carpinteria, California) can produce good results;13 however, their performance has a major impact on turnaround time (limited by machine and personnel accessibility), and sometimes they tend to over-count the stromal cells and lymphocytes as cells encountered.
Currently, manual quantification of Ki-67 index is considered the gold standard of Ki-67 index quantification, in spite of this, it is time-consuming and subject to intra- and interobserver variability, especially if performed by readers with various expertise. To overcome these limitations, more attention to digitally automated and (semi-) automated techniques has led to the development of rapid, reliable, and reproducible tools that are mainly based on medical image processing techniques, such as computer aided diagnosis (CAD) tools. CAD combines elements of computer vision and artificial intelligence with pathologic image processing.15–17 It is an interdisciplinary technology that is designed to assist clinicians and physicians to obtain a more accurate interpretation of medical images, and thus improve the accuracy of case diagnosis and treatment decisions.15–17 Therefore, it is vital to accurately evaluate the performance of these tools to ensure the reliability and the precision of their outcomes. The aim of this study is to assess the performance of one of the most widely used Java-based CAD tools for biomedical imaging “ImageJ”18,19 in quantifying Ki-67 index, and to evaluate its accuracy and concordance with the MC method.
2. Materials and Methods
2.1. Cases Collection
This is a retrospective analysis study of 20 patients (9 females and 11 males) with age range 14 to 79 (mean and median: 47 each), with confirmed diagnosis of well-differentiated NETs of the pancreas (three cases), stomach (four cases), duodenum and ampulla (two cases), jejunum and ileum (seven cases), appendix (two cases), and colon (two cases). These cases were randomly selected from resected surgical specimens reviewed at the King Hussein Cancer Center in the past seven years (Institutional Review Board approval obtained). Patient data were collected from the hospital electronic medical reports. All stained slides including hematoxylin and eosin (H&E) and Ki-67 immunohistochemical (IHC) stained slides relevant to the selected cases were retrieved from the pathology archive. This study investigates only Ki-67 IHC stained slides. Three expert pathologists from the same institution with more than five years of experience in Ki-67 assessment were involved in this study. Also a qualified pathologist reviewed the morphology of each tissue sample utilizing an H&E stained section before interpreting the IHC result.
2.2. Immunohistochemistry Slides Preparation
Formalin-fixed, paraffin-embedded tissue blocks were used for IHC staining. The anti-Ki-67 (Ventana, 30-9) rabbit monoclonal primary antibody was used on Ventana BenchMark ultra-automated slide stainer platform. Tumor cells that showed Ki-67 nuclear protein expression of any intensity were counted as positive cells. The Ki-67 proliferative index was calculated by dividing the number of Ki-67 positive tumor cells by the total number of counted tumor cells. A qualified pathologist experienced in immunohistochemistry procedures had evaluated the positive and negative controls before interpreting the results.
2.3. Images Acquisition
The immunostained slides for Ki-67 proliferation index (for the 20 cases) were examined to assess the adequacy and quality of tissues. A minimum of 1000 tumor cells was used to overcome tumor heterogeneity, although the recommended minimum number of cells is 500.11 Five nonoverlapping camera-captured images at magnification were taken for each case (total of 100 images) from the tumor’s hotspots, such that areas of lower proliferative activity were not included and saved as Tagged Image Format Files. Slides were examined using an Olympus BX 43 trinuclear light microscope, and images were captured using Olympus LC30 digital camera with 3.1-megapixel CMOS chip. It has with a pixel size of digitized with 10-bits resolution. The camera supports various binning modes that can be used to increase sensitivity, as well as frame rate, and offers exposure times ranging from just up to 750 ms. Prints of the 100 images were provided for three pathologists for independent Ki-67 index manual cell counting.
2.4. Quantification of Ki-67 Nuclear Immunoreactivity
2.4.1. Manual counting
For each of the 20 cases, the Ki-67 index was counted manually by three pathologists using print hard copies of five camera-captured images from the tumor hotspots that included a minimum of 1000 tumor cells. As a reflection of the tissue nature, every captured image would display a mixture of tumor and nontumor cells, such as lymphocytes, endothelial cells, and stromal cells. However, the tumor cells are predominant in any image. For each case, the total number of tumor cells with positive (stained with brown color) and negative (stained with blue color) nuclear stain were calculated separately by adding the counts of all five images, nontumor cells were excluded based on their morphologic features that differ from that of the tumor cells. The Ki-67 index for each case was determined by dividing the total number of positive tumor cells over the total number of tumor cells (positive and negative).
2.4.2. Semiautomated counting methodology
In order to count the positive and negative cells automatically and hence get an objective quantification of the Ki-67 index, publicly available tools can be utilized and tuned to determine the best steps combination that will produce optimal results. ImageJ is one of the most popular and most widely used open source software tools for biomedical imaging in biological microscopy community.18,19 It performs complex image processing and quantification tasks to transform medical images into measurable and quantifiable units. It has many features to visualize, manipulate, and analyze medical images. Its popularity is mainly attributed to the ease of use, specifically for nontechnical experts, platform independency, and support to different image formats.20,21 Figure 1 demonstrates the procedure that was followed for feature extraction and processing using ImageJ to perform cell nuclei counting.
Fig. 1.
SAC method steps for Ki-67 index quantitation.
2.5. Statistical Analyses
The concordance of the natural log-transformed Ki-67 index obtained by the MC and SAC methods was assessed by intraclass correlation coefficients (ICC: with two-way mixed effect and with absolute agreement) and by Lin’s concordance correlation coefficients (CCC). Similarly, the concordance between the MC of the three readers was assessed. Correlations range between 0 and 1. Perfect correlation is indicated by 1 and the absence of any correlation is considered 0. ICC values were reported along with their corresponding 95% confidence intervals (CIs). All analysis was performed using SPSS version 21 (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, version 21.0. Armonk, New York: IBM Corp.) and MedCalc Software for Windows version 19.1 (MedCalc Software, Ostend, Belgium). Correlations and concordance were assessed with Bland–Altman plots22,23 and summary descriptive statistics presented in Secs. 3.2 and 3.3.
3. Results
3.1. Fine-Tuning of SAC Method
This section discusses the different fine-tuning steps required for the semiautomated cell nuclei counting method. The focus is on providing more accurate segmentation while reducing irrelevant background noise in the image samples.
3.1.1. Nuclei texture smoothing
Gaussian low-pass filtering is used to smooth out the inhomogeneity of the nuclear staining. This filter has a parameter, which defines the effective spread of the function. The larger the value of is, the greater the smoothing effect is (Fig. 2). The optimal size of the Gaussian filter is dependent upon the scale of the nuclei in the image and the size of the digital image. Pathologist might require to calibrate the value of once before analyzing the image. The aim is to smooth the image until its irrelevant features are dissolved and the texture of the nuclei becomes smooth, but at the same time retains the weighted average intensity across the image. For the image shown in Fig. 2, smaller values of the parameter still show the inner of the nuclei [Figs. 2(a) and 2(b)], which might cause the oversegmentation of one nucleus into multiple nuclei, whereas the larger values of do not preserve distinguished nuclei intensities [Figs. 2(d) and 2(e)]. However, an intermediate value would obtain the desired effect [Fig. 2(c)].
Fig. 2.
(a) A subregion from a Ki-67 IHC stained image and (b)–(e) the corresponding effect of the convolution (objective magnification, ) with a Gaussian filter of increasing standard deviations. From left to right, the values of the Gaussian filters used are 0, 3, 6, 9, and 12 pixels, respectively.
3.1.2. Color space selection
After blurring the image, the goal is to have all nuclei appear with comparable intensity and at the same time to be distinct from the background. Therefore, the color channel that achieves the maximum nuclei intensity homogeneity and discrimination from the background was selected. It was noticed that the red channel is suitable for sparse nuclei distribution [Fig. 3(a)], whereas the blue channel was used when the distribution is dense [Fig. 3(b)].
Fig. 3.
(a) Ki-67 IHC stained image with sparse nuclei distribution, in which the red color channel separates the nuclei from the background while maintaining a good nuclei intensity homogeneity and thus will be selected. (b) A similar image with dense nuclei distribution, in which the blue color channel will be selected.
3.1.3. Contrast adjustment
Distinguishing the nuclei from the surrounding cytoplasm and tissues (background) is vital. Therefore, contrast adjustment is performed using image intensity distributions (histogram equalization) and applied on the selected blurred color channel [Figs. 4(a) and 4(b)]. This step is only required when total signals are being counted, as the positive signals are well-differentiated from the background. And it is performed only once for the first image of the first case and afterward generalizes to the rest of the images of all cases. This adjustment enhances the contrast of the input image so that the gradient of the image is strong enough to properly segment the image later by the watershed technique [Fig. 4(c)].
Fig. 4.
Different steps for the SAC cell nuclei segmentation and counting of Ki-67 immunostained images representing (a) original image, (b) smoothed red channel image using Gaussian low-pass filter, (c) contrast enhanced image using histogram equalization, (d) thresholded image, (e) binary image, (f) inverted image to show the nuclei in white and the background in black, (g) segmented image using watershed algorithm, and (h) cell nuclei counting image.
3.1.4. Image thresholding
It is important to unify the intensities of the nuclei and this is accomplished by applying a threshold value to the image, in which the intensity values fall above the threshold is considered foreground and the ones fall below it is part of the background [Fig. 4(d)]. Automatic cell nuclei intensity thresholding was performed using Otsu’s method in ImageJ. It has the advantage of minimizing intraclass intensity variance through maximizing interclass variance.24 A manual adjustment of the automatically calculated threshold might be required for improved background–foreground separability in order to check that only cell nuclei are included. As in the previous method, this parameter calibration is performed for the first image of the first case and afterward generalizes to the rest of the images of all cases.
3.1.5. Image binarization
The thresholding will separate the image into foreground and background, which will then need to be converted into black and white in a process known as binarization such that the foreground appears in black, while the background is in white [Fig. 4(e)].
3.1.6. Image postprocessing
In image processing, the foreground should appear in white while the background is in black; therefore, image inversion is performed. A hole is defined as an area of dark pixels surrounded by lighter pixels. The isolate dark spots or holes are filled using a flood-fill operation on background pixels of the input binary image if nuclei appear to have many gaps [Fig. 4(f)]. Also fragments joining—related to a single cell nucleus–might be required by the image dilation operation.
3.1.7. Watershed nuclei segmentation
The watershed is a region-based segmentation method of mathematical morphology,25,26 which can be used for separating/segmenting different objects in an image. The watershed transform finds “catchment basins” in an image by treating it as a surface where bright pixels represent high elevations and dark pixels represent low elevations. The watershed transform can be used to segment contiguous regions of interest into distinct objects (Fig. 5). Challenging conditions related to cell nuclei boarder touching, overlapping and occlusion have been resolved using this technique [Fig. 4(g)]. This will facilitate for subsequent cell nuclei counting.
Fig. 5.
Watershed algorithm steps: (a) binary image containing two overlapping circular objects resembling cell nuclei, (b) computed distance transform of the complement of image in (a), and (c) watershed transformed image displayed as an RGB image.
3.1.8. Count segmented objects
Instead of counting all the objects in the segmented image (watershed transformed image), only the regular and large enough objects are being counted and the rest are filtered out (empirically setting the circularity parameter and size to 0.5 and , respectively). Cell nuclei counting is performed [Fig. 4(h)] after applying the mask to each nucleus according to the prespecified size and circularity parameters.
3.2. Concordance of the Ki-67 Index Quantification Obtained by MC and SAC
A total of 20 well-differentiated GEP-NET cases were analyzed, each case is represented by five nonoverlapping captured images, and each image contains between 170 and 800 cells with a mean of . Therefore, each case is represented by a minimum of 1300 cells and up to 3700 cells with a mean of . The concordance of the Ki-67 proliferation index (applied on log-transformed values) between MC and SAC was excellent in all the cases analyzed with an ICC of 0.993 (95% CI: 0.984 to 0.997) and a CCC of 0.986 (95% CI: 0.966 to 0.994). The absolute difference of the untransformed “raw” indices obtained by the two methods ranged between 0% and 1.59%, except for a single case (case 16, with a difference of 3.4%) having a mean of and quartiles of 0.12% and 0.67%. The counting difference between the two methods was assessed by a Bland–Altman plot22 (Fig. 6), which plots the difference against the mean. It also shows the differences trend, if any exists, and identifies proportional biases, if any exists. This figure shows that the two methods have a good level of agreement without any detectable proportional biases.
Fig. 6.
Bland–Altman plot for assessing the agreement between the two methods MC* and SAC. Log-transformed Ki-67 scores are used. MC* is the average of the MC of three expert pathologists. All the cases were used in this analysis (). The mean of scores is shown on the axis, whereas the actual differences are shown in the axis. The mean of differences line is shown as solid line (). The 95% agreement lower and upper limits () are shown as dashed lines at and , respectively. The equality line () is also shown as small dashed line. The 95% CI around the mean difference and the agreement limits are shown as vertical bars.
Using the SAC method, the percentage of untransformed Ki-67 index ranged from 0.5% to 56.37% with a mean of and quartiles of 1.67% and 5.19%. The gold standard is considered the average of the MC of three readers (MC*), in which the percentage of untransformed Ki-67 index ranged from 0.30% to 57% with a mean of and quartiles of 1.72% and 5.96% (Table 2). Both methods obtained very close results. Using the gold standard (MC*), this collection of cases is classified as 50% in G1 (; ), 40% in G2 (; index 3% to 20%), and 10% in G3 (; ), which is identical to the classification obtained by SAC (100% concordance, Fig. 7).
Table 2.
Summary of untransformed Ki-67 proliferation index statistics obtained using MC and SAC.
| MC reader 1 | MC reader 2 | MC reader 3 | MCa | SAC | |
|---|---|---|---|---|---|
| Minimum | 0.42 | 0.18 | 0.29 | 0.30 | 0.50 |
| 25th perc | 1.94 | 1.65 | 1.59 | 1.72 | 1.67 |
| 50th perc (median) | 3.32 | 3.21 | 3.24 | 3.23 | 2.97 |
| Mean | 8.49 | 7.72 | 8.35 | 8.19 | 8.08 |
| 75th perc | 5.96 | 5.79 | 5.93 | 5.96 | 5.19 |
| Maximum | 57.56 | 56.78 | 56.69 | 57.01 | 56.37 |
| STD | 14.31 | 13.10 | 14.57 | 13.95 | 14.31 |
Note: perc, percentile and STD, standard deviation.
The average of the counts obtained through MC by three readers and it is considered as the gold standard in this study.
Fig. 7.
Summary of cases grading () according to the WHO grading system based on the Ki-67 index obtained using MC and SAC, which shows 100% concordance. G1 (; ), G2 (; index 3% to 20%), and G3 (; ). MC* is the average of the counts obtained through MC by three readers and it is considered as the gold standard in this study.
3.3. Interobserver Variability Assessment
The Ki-67 proliferation index was manually and independently calculated by three readers through counting the brown and blue cells separately, then dividing the number of brown cells by the total number of cells. The three readers participated in the MC are all expert pathologists who are experienced in evaluating Ki-67 labeling, therefore, correlation of the log-transformed Ki-67 proliferation index between the three readers was excellent in all the cases analyzed with an ICC of 0.997 (95% CI: 0.992 to 0.999). From this index and according to the WHO grading system, each case can be classified into one of the grades to indicate the prognosis of the disease. There was a 100% concordance in cases’ classification between the three readers; however, the actual counts vary (Table 3). For the majority of the cases, reader 2 obtained higher counts than readers 1 and 3, specifically for case 16 (, G3).
Table 3.
Summary of brown and blue cells count statistics obtained by the three readers for the MC method.
| Brown cells | Blue cells | Total cells | |||||||
|---|---|---|---|---|---|---|---|---|---|
| MC1 | MC2 | MC3 | MC1 | MC2 | MC3 | MC1 | MC2 | MC3 | |
| Minimum | 9 | 4 | 7 | 1080 | 1397 | 853 | 1318 | 1484 | 1400 |
| 25th perc | 43.5 | 42.5 | 43.8 | 1488.0 | 1675.5 | 1485.8 | 1725.8 | 1777.5 | 1733.5 |
| 50th perc (median) | 67.0 | 69.0 | 66 | 1819.0 | 1941.5 | 1900.5 | 2072.0 | 2240.5 | 2165 |
| Mean | 203.7 | 212.3 | 192.7 | 2019.5 | 2176.4 | 2090.3 | 2223.2 | 2388.6 | 2282.9 |
| 75th perc | 142.5 | 145.0 | 133.5 | 2468.8 | 2576.3 | 2576.5 | 2644.8 | 2859.5 | 2774.8 |
| Maximum | 1835 | 1881 | 1759 | 3613 | 3841 | 3571 | 3713 | 3941 | 3659 |
| STD | 409.1 | 427.5 | 388.3 | 677.5 | 679.3 | 717.0 | 668.6 | 706.9 | 681.4 |
Another method was applied to assess the concordance between the three readers: the Bland–Altman plot (Fig. 8). It is an alternative analysis that is based on analyzing the mean difference and constructing limits of agreement. Although ICC and CCC can show a high correlation, the Bland–Altman plot can detect the existence of proportional biases. This is indicated by the location of the equality line ( in Fig. 8) that falls outside the CI, although most of the differences located within the 95% limit of agreement ().22,23 Nevertheless, these differences, on the tested cases, did not cause any disagreement between the three readers when classifying the cases into their corresponding grades, as stated above.
Fig. 8.
Bland–Altman plot for assessing the agreement between the three readers against the MC*. Log-transformed Ki-67 scores are used. The MC* is shown on the axis, whereas the actual differences are shown on the axis. The mean of differences line is shown as solid line. The 95% agreement lower and upper limits () are shown as dashed lines. The equality line () is also shown as small dashed line. 95% CI around the mean difference and the agreement limits are shown as vertical bars.
The relative difference of the Ki-67 index between the MC of the readers and the average (MC*) is illustrated in Fig. 9. All the cases except for one case showed a relative difference of . For case 16, the difference ranged from 2.5% to 7.9%. The difference, however, of the actual counts between the readers was more pronounced than that for the Ki-67 index and it increases as the number of cells in the image increases, specifically when counting blue cells and for higher grades (Fig. 10).
Fig. 9.
The relative difference of the Ki-67 index between the MC of each reader (reader 1 in blue, reader 2 in red, and reader 3 in black) and the gold standard (MC*) across all the cases, except for case 16 in which the difference between MC of the readers and MC* is around 2.5%, 7.9%, and 5.4% for readers 1, 2, and 3, respectively.
Fig. 10.
Distribution of (a) brown and (b) blue cells counts by reader 1, reader 2, and reader 3 for all the cases () separated by their WHO grading (G1, G2, and G3).
4. Discussion
NETs are rare and heterogeneous group of neoplasms that mostly arise from specialized body cells called endocrine cells and with varied and confusing histology.27–29 Approximately two-thirds of NETs arise in the gastrointestinal tract, and therefore, they are referred to as GEP-NETs. GEP-NETs are a group of heterogeneous neoplasm that are mostly malignant, well-differentiated, and with minimal variations in their morphology. The stage of the tumor and its primary location predict its prognosis and outcome. However, classifying the tumors into corresponding accurate stage is not an easy task.30–32
There are plenty of potential prognostic parameters, histomorphological and biological,33,34 yet the proliferative rate based on Ki-67 index (a nuclear antigen that is expressed in proliferating cells, as it is present in all phases of the cell cycle except for G035) is the strongest and most informative along with mitotic count.36,37 Mitotic count has high interobserver variability,38 and therefore, calculating the proliferative rate, specifically using the Ki-67 IHC staining, is considered an essential part in determining the grade of GEP-NETs and potentially reflect their prognosis and influence their therapeutic decisions.39–49
Due to its pivotal clinical role in tumor assessment and to reduce the interobserver variability50,51 and to improve the reproducibility of Ki-67 index interpretation, there were attempts to standardize the utility of Ki-67 in diagnostic histopathology.52 Furthermore, many recognized organizations such as, the ENTS and WHO incorporated a grading system for NETs that is based on Ki-67 index (grade 1, G1: ; grade 2, G2: 3%–20%; and grade 3, G3: ).53–56 Tumor type and its grading level are correlated, for example, the management of G1 tumor level is largely dependent on the histologic grade, especially for those with low-grade nonresectable tumors. In general, the higher the grade is, the more aggressive the clinical behavior and the poorer the prognosis are.57
Ki-67 index can be quantified by various methods, such as EE, MC, and automated and semiautomated approaches. Although EE is considered the fastest method, since it determines the index without formal accurate counting of individual cells, it has the most inter- and intraobserver variability as it highly depends on the expertise of the pathologists to be able to give a close estimate.58–61 This variability is more pronounced when the tumor grade is low (G1 or G2; index ). MC is very accurate, specifically if it was performed by expert pathologist; however, it is time-consuming where a minimum of 500 cells should be counted according to standards.58–61 Recently and with the emergent of computer aided tools for diagnosis and analysis, several attempts were performed to utilize the benefits of faster, reproducible, and accurate tumor assessment and grading.62–65
In this study, an accurate, efficient, reliable, and reproducible Ki-67 index quantification is proposed based on one of the most popular and publicly available software tools for biomedical imaging in biological microscopy community, ImageJ.19,20 The primary objective of this study is to determine the optimal sequence of ImageJ steps and fine-tune their parameters with as minimal human interaction as possible (Fig. 1), then assess its accuracy and measure the concordance between the gold standard methods of Ki-67 scoring, MC, and the proposed semiautomated method.
ImageJ provides numerous features for complex image analysis and statistical filtering and assessment. In GEP-NET Ki-67 IHC stained images, the nuclei sometimes appear grainy, which might cause oversegmentation when processed via CAD system. To overcome this issue, it is recommended to locally blur (smooth) the Ki67-immunostained images prior to digital image analysis to assist in the automated segmentation and detection of tumor cells. ImageJ has many built-in image processing filters implemented, the one that was used in this work is the Gaussian low-pass filter. The filters suitable standard deviation () parameter was empirically selected, as choosing the suitable value may affect the performance of the subsequent steps (Fig. 2). After smoothing, it is preferred to select the image color channel that shows the maximum nuclei intensity homogeneity and at the same time is distinguishable from the background. Empirically, the red channel obtained the best results for sparse nuclei distribution [Fig. 3(a)], whereas the blue channel is better for dense distributions [Fig. 3(b)]. It is important then to adjust the contrast of the image to differentiate the nuclei from the surrounding tissues. This enhances the segmentation of the nuclei at later steps. Through this adjustment, the intensities can be better distributed on the histogram, which allows for areas of lower local contrast to gain a higher contrast. Histogram equalization accomplishes this by effectively spreading out the most frequent intensity values [Fig. 4(c)]. This step is only applied when counting total signals, not just the positive signals (brown cells). To obtain a uniform nuclei color that is different from the background, image thresholding is applied [Fig. 4(d)]. ImageJ applies Otsu’s thresholding method that exhaustively searches for the threshold that minimizes the intraclass variance, defined as a weighted sum of variances of the two classes (foreground and background), binary (black and white) image is the result of this step [Fig. 4(e)]. Image segmentation to foreground (tumor cells) and background (nontumor cells) is performed automatically, no manual selection for the scoring area or for the region of interest is required. Subsequently, image postprocessing is applied to enhance nuclei segmentation and counting; several postprocessing techniques can be applied, such as inversion, hole filling, and dilation [Fig. 4(f)].
Cells in tissue specimens can be tightly packed and are composed mostly of nuclei with very little cytoplasm separating them. The nuclei often seem to touch each other, although in fact there should be separating membranes, but an optical microscope cannot resolve this. Also, the image is quite noisy, because it was made with a fast scan on a confocal laser scanning microscope, which inherently has a low signal-to-noise ratio. This gives the objects fuzzy edges and adds uncertainty to the intensity values of each pixel, making it harder to segment properly. A region-based algorithm, like watershed is applied to deal with cell nuclei fuzzy edges and for separating/segmenting apparently touching nuclei [Figs. 4(g) and 5]. The algorithm can as well reduce the influence of background noise, which could take the form of statistical (Poisson) noise, thermal noise, or readout noise (imperfect operation of physical electronic devices). Other suboptimal situations that can affect the signal-to-noise ratio are quite common, such as possible optical aberrations (i.e., lens causing light spread out away from the focal point) and uneven illumination due to surrounding environment; especially when reaching the full resolution limits. The segmented image is ready for nuclei counting, which can be performed in ImageJ using the feature “analyze particles” [Fig. 4(h)]. To improve the results and increase the accuracy, it is important to filter out the overly small objects that appear to be cell nuclei. In addition, extremely irregular-shaped objects should also be excluded. A range from 300 to 700 pixels was chosen as the size of the objects to keep, and a range from 0.50 to 1 object circularity was applied to keep the regular objects. The manual adjustment of the ImageJ parameters is required only once, at the beginning, when either choosing to count the total signals (blue and brown color cells) or positive signals (cells stained with brown color). In particular, parameter pertaining to contrast adjustment or dynamic Otsu’s thresholding is only required when counting total signals and is not required for the other case of the well-differentiated positive signals. This parameter calibration is performed for the first image of the first case and afterward generalizes to the rest of the images of all cases. Once calibrated, the procedure can be easily integrated with routine diagnostic work.
After determining the optimal ImageJ steps for quantitating the Ki-67 index (Fig. 1), all the 20 cases, which are represented by a total of 100 images with a mean of , were analyzed by both the SAC and MC methods to assess the accuracy of the SAC method and measure its concordance with the gold standards. The results demonstrate comparable accuracy with excellent ICC of 0.993 (95% CI: 0.984 to 0.997) and a CCC of 0.986 (95% CI: 0.966 to 0.994) between the two methods and error-free histopathologist-level cases grading (100% concordance, Figs. 6 and 7).
MC is preferred by most pathologists over any CAD tool since it provides the opportunity of manually evaluating each tumor cell and therefore obtains accurate assessment. In addition, machine analysis, diagnosis, and interpretation of medical images are still questionable and unreliable. However, with the recent advancement of artificial intelligence, specifically with machine and deep learning, astonishing expert-level results are obtained.66 Similar surprising results can also be obtained through implementing and developing tools that are based on specific feature extraction, analysis, and manipulation, yet without complex learning approaches, such as ImageJ and similar CAD tools.
The results obtained by the proposed semiautomated tool with human calibration of two or three parameters are comparable, in terms of accuracy, the results obtained by expert pathologist yet with much less time and effort (Table 4). Noting that the SAC time does not include the time required for selecting the scoring area, since selecting the region of interest “scoring area” is not required by the SAC method—as it takes the whole image as input and automatically detects the tumor cells as foreground and excludes the nontumor cells, such as lymphocytes and endothelial cells. Several previous studies assess the performance of EE method and most of them agreed on the high inter- and intraobserver variability of this method and the weak interclass agreement with MC gold standard.11,58–61 In this study, the focus was on proposing a semiautomated method with comparable results with MC. However, the objectivity, reproducibility, reliability, efficiency, and easy to apply by pathologists of the SAC method makes it surpass the MC method.
Table 4.
Comparison of different Ki-67 proliferation index quantitation methodologies.
| Methodology | Average time | Practicality | Accuracy | Prone to interobserver variability | Prone to intraobserver variability |
|---|---|---|---|---|---|
| EE13 | Highest | Very low | Very likely | Likely | |
| MC | 8 to 10 min | Very high | Very high | Likely | Unlikely |
| SAC | a | Very high | Very high | Unlikely | None |
| b |
For counting the brown cells.
For counting the total number of cells (brown and blue), parameters calibration is only required once and only for the first image, so no further calibration time is needed nor included in this time.
The practicality of both methods (MC and SAC) is similar since MC needs the accessibility of a camera/printer setup, which is somewhat comparable to the accessibility of ImageJ software tool, both can be easily provided; however, ImageJ is open-source without any additional costs. The time needed to perform counting using MC is higher than that needed by SAC, with similar accuracy and with low interobserver variability—high interobserver variability is clearly pronounced if it was performed by a mixed of expert pathologists and nonexperts.12 However, in this study, three highly experienced pathologists perform the MC, and this is the reason behind obtaining a very high correlation between their results, with an ICC of 0.997 (95% CI: 0.992 to 0.999) but without any intraobserver variability.
5. Conclusion
The Ki-67 proliferative index is a surrogate marker for prognostic stratification of GEP-NETs. Accurate scoring of this marker is essential for precise pathologic grading that will implicate on the treatment. MC on the print of camera-captured image of the tumor hotspot with scoring a minimum of 500 tumor cells is highly recommended and considered the gold standard method to determine the Ki-67 index. However, this method is time-consuming and impractical in the clinical setting. Therefore, developing an automated counting method using digital image analysis and CAD tools would enhance the scoring process, but requires careful modification of the software to avoid imprecisions. ImageJ features were investigated carefully and accurately fine-tuned to obtain the optimal sequence of steps that will accurately calculate Ki-67 index. The correlation of the results between the two methods revealed strong concordance, which suggests the SAC method as a reliable and accurate method for counting. SAC can also be a good choice as an assessment tool for training junior histopathologists for improving their nuclei detection accuracy.
Acknowledgments
The authors thank Yazan Al-Othman and Duha Saadeh for the data collection and manuscript formatting, and the statisticians of KHCC (Dalia Al-Rimawi and Hadeel Abdelkhaleq) for their assistance in statistical analysis.
Biographies
Heba Saadeh is currently an assistant professor in epigenetics and bioinformatics at the Computer Science Department, University of Jordan, Amman, Jordan. She worked as a research associate at the Babraham Institute (in the bioinformatics and epigenetics groups), Cambridge, United Kingdom. She obtained her PhD in bioinformatics from King’s College London, United Kingdom, and her MSc and BSc degrees in computer science from the University of Jordan. Her research interests are in data science, specifically in the medical/biology fields.
Niveen Abdullah is a consultant of cellular pathology, Milton Keynes University Hospital, United Kingdom. Previously she was a consultant of histopathology and molecular pathology, and deputy medical director of the blood bank at King Hussein Cancer Center, Jordan. She is a fully registered medical practitioner with specialist registration and license to practice in histopathology in the United Kingdom. Her training included a fellowship in molecular pathology and transfusion medicine at Leeds Teaching Hospitals-NHS Trust, United Kingdom, and a fellowship with the Royal College of Pathologists, United Kingdom.
Madiha Erashdi is a fellow of the Royal College of Pathologists (FRCPath) in histopathology, Department of Pathology and Laboratory Medicine, King Hussein Cancer Center, Amman, Jordan. Certified by the Arab Board of Pathology, she was previously a resident in histopathology and cytopathology at King Abdullah University Hospital, Jordan. She obtained higher specialty in medicine (histopathology) from Jordan University of Science and Technology, with a background in general medicine and surgery (MBBS) from the University of Tripoli, Libya.
Maher Sughayer is a full member and chair of the Department of Pathology and Laboratory Medicine at King Hussein Cancer Center (KHCC), and an associate professor of pathology. A medical doctor (MD), he is certified by the American Board of Pathology, and has vast clinical experience in many fields of pathology.
Omar Al-Kadi received his PhD from the University of Sussex, then was a visiting researcher at the Center for Vision, Speech and Signal Processing at the University of Surrey. He was also a research fellow at the Institute of Biomedical Engineering at the University of Oxford. Currently he is an associate professor at the University of Jordan. His research interest is mainly concerned with improving tissue characterization and understanding of tumor behavior.
Disclosures
Authors declare that they have no conflicts of interest.
Contributor Information
Heba Saadeh, Email: heba.saadeh@ju.edu.jo.
Niveen Abdullah, Email: niveen.abdullah@mkuh.nhs.uk.
Madiha Erashdi, Email: ME.13475@khcc.jo.
Maher Sughayer, Email: msyghayer@khcc.jo.
Omar Al-Kadi, Email: alkadi@ieee.org.
References
- 1.Graeme-Cook F., Neuroendocrine Tumors of the GI Tract and Appendix, Odze R. D., Goldblum J. R., Crawford J. M., Eds., 2nd ed., pp. 483–503, Elsevier, Surgical Pathology of the GI Tract, Liver, Biliary Tract, and Pancreas, WB Saunders, Philadelphia, Pennsylvania: (2004). [Google Scholar]
- 2.Chetty R., “An overview of practical issues in the diagnosis of gastroenteropancreatic neuroendocrine pathology,” Arch. Pathol. Lab. Med. 132, 1285–1289 (2008). 10.1043/1543-2165(2008)132[1285:AOOPII]2.0.CO;2 [DOI] [PubMed] [Google Scholar]
- 3.Modlin I. M., et al. , “Gastroenteropancreatic neuroendocrine tumours,” Lancet Oncol. 9(1), 61–72 (2008). 10.1016/S1470-2045(07)70410-2 [DOI] [PubMed] [Google Scholar]
- 4.Tang L. H., Klimstra D. S., “Conundrums and caveats in neuroendocrine tumors of the pancreas,” Surg. Pathol. Clin. 4(2), 589–624 (2011). 10.1016/j.path.2011.03.003 [DOI] [PubMed] [Google Scholar]
- 5.Garcia-Carbonero R., et al. , “Incidence, patterns of care and prognostic factors for outcome of gastroenteropancreatic neuroendocrine tumors (GEP-NETs): results from the National Cancer Registry of Spain (RGETNE),” Ann. Oncol. 21(9), 1794–1803 (2010). 10.1093/annonc/mdq022 [DOI] [PubMed] [Google Scholar]
- 6.Yao J. C., et al. , “One hundred years after ‘carcinoid’: epidemiology of and prognostic factors for neuroendocrine tumors in 35,825 cases in the United States,” J. Clin. Oncol. 26(18), 3063–3072 (2008). 10.1200/JCO.2007.15.4377 [DOI] [PubMed] [Google Scholar]
- 7.Klimstra D. S., et al. , “Pathology reporting of neuroendocrine tumors: application of the delphic consensus process to the development of a minimum pathology data set,” Am. J. Surg. Pathol. 34(3), 300–313 (2010). 10.1097/PAS.0b013e3181ce1447 [DOI] [PubMed] [Google Scholar]
- 8.Kos-Kudła B., et al. , “Current treatment options for gastroenteropancreatic neuroendocrine tumors with a focus on the role of lanreotide,” Wspolczesna Onkol. 21(2), 115–122 (2017). 10.5114/wo.2017.68619 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Edge S. B., Compton C. C., “The American Joint Committee on cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM,” Ann. Surg. Oncol. 17(6), 1471–1474 (2010). 10.1245/s10434-010-0985-4 [DOI] [PubMed] [Google Scholar]
- 10.Klöppel G., et al. , “ENETS consensus guidelines for the standards of care in neuroendocrine tumors: towards a standardized approach to the diagnosis of gastroenteropancreatic neuroendocrine tumors and their prognostic stratification,” Neuroendocrinology 90(2), 162–166 (2009). 10.1159/000182196 [DOI] [PubMed] [Google Scholar]
- 11.Bosman F., et al. , WHO Classification of Tumours of the Digestive System, WHO Press, Geneva, Switzerland: (2010). [Google Scholar]
- 12.Tang L. H., et al. , “Objective quantification of the Ki67 proliferative index in neuroendocrine tumors of the gastroenteropancreatic system: a comparison of digital image analysis with manual methods,” Am. J. Surg. Pathol. 36(12), 1761–1770 (2012). 10.1097/PAS.0b013e318263207c [DOI] [PubMed] [Google Scholar]
- 13.Reid M. D., et al. , “Calculation of the Ki67 index in pancreatic neuroendocrine tumors: a comparative analysis of four counting methodologies,” Mod. Pathol. 28(5), 686–694 (2015). 10.1038/modpathol.2014.156 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Schmechel S. C., et al. , “Quantitative comparison of immunohistochemical staining measured by digital image analysis versus pathologist visual scoring,” Diagn. Pathol. 7(1), 42 (2012). 10.1186/1746-1596-7-42 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Castellino R. A., “Computer aided detection (CAD): an overview,” Cancer Imaging 5(1), 17–19 (2005). 10.1102/1470-7330.2005.0018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Doi K., “Computer-aided diagnosis in medical imaging: historical review, current status and future potential,” Comput. Med. Imaging Graphics 31(4–5), 198–211 (2007). 10.1016/j.compmedimag.2007.02.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Shiraishi J., et al. , “Computer-aided diagnosis and artificial intelligence in clinical imaging,” Semin. Nucl. Med. 41(6), 449–462 (2011). 10.1053/j.semnuclmed.2011.06.004 [DOI] [PubMed] [Google Scholar]
- 18.Schneider C. A., Rasband W. S., Eliceiri K. W., “NIH Image to ImageJ: 25 years of image analysis,” Nat. Methods 9(7), 671–675 (2012). 10.1038/nmeth.2089 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rueden C. T., et al. , “ImageJ2: ImageJ for the next generation of scientific image data,” BMC Bioinf. 18(1), 1–26 (2017). 10.1186/s12859-017-1934-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Chang Y. H., et al. , “Primer for image informatics in personalized medicine,” Procedia Eng. 159, 58–65 (2016). 10.1016/j.proeng.2016.08.064 [DOI] [Google Scholar]
- 21.Lind R., Open Source Software for Image Processing and Analysis: Picture This with ImageJ, Woodhead Publishing Limited, Abington, Cambridge: (2012). [Google Scholar]
- 22.Bland J. M., Altman D. G., “Statistical methods for assessing agreement between two methods of clinical measurement,” Lancet 327, 307–310 (1986). 10.1016/S0140-6736(86)90837-8 [DOI] [PubMed] [Google Scholar]
- 23.Krouwer J. S., “Why Bland–Altman plots should use X, not (Y+X)/2 when X is a reference method,” Stat. Med. 27, 778–780 (2008). 10.1002/(ISSN)1097-0258 [DOI] [PubMed] [Google Scholar]
- 24.Huang D. Y., Wang C. H., “Optimal multi-level thresholding using a two-stage Otsu optimization approach,” Pattern Recognit. Lett. 30(3), 275–284 (2009). 10.1016/j.patrec.2008.10.003 [DOI] [Google Scholar]
- 25.Serra J. P., Image Analysis and Mathematical Morphology, Vol. 1, Academic Press, London: (1982). [Google Scholar]
- 26.Beucher S., “Watershed, hierarchical segmentation and waterfall algorithm,” in Mathematical Morphology and Its Applications to Image Processing, Computational Imaging and Vision, Serra J., Soille P., Eds., Vol. 2, pp. 69–76, Springer, Dordrecht: (1994). [Google Scholar]
- 27.Barakat M. T., Meeran K., Bloom S. R., “Neuroendocrine tumours,” Endocr.-Relat. Cancer 11(1), 1–18 (2004). 10.1677/erc.0.0110001 [DOI] [PubMed] [Google Scholar]
- 28.Oronsky B., et al. , “Nothing but NET: a review of neuroendocrine tumors and carcinomas,” Neoplasia 19(12), 991–1002 (2017). 10.1016/j.neo.2017.09.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Dasari A., et al. , “Trends in the incidence, prevalence, and survival outcomes in patients with neuroendocrine tumors in the United States,” JAMA Oncol. 3(10), 1335–1342 (2017). 10.1001/jamaoncol.2017.0589 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Grozinsky-Glasberg S., Grossman A. B., Korbonits M., “The role of somatostatin analogues in the treatment of neuroendocrine tumours,” Mol. Cell Endocrinol. 286(1–2), 238–250 (2008). 10.1016/j.mce.2007.10.006 [DOI] [PubMed] [Google Scholar]
- 31.Modlin I. M., Lye K. D., Kidd M., “A 5-decade analysis of 13,715 carcinoid tumors,” Cancer 97(4), 934–959 (2003). 10.1002/(ISSN)1097-0142 [DOI] [PubMed] [Google Scholar]
- 32.Estrozi B., Bacchi C. E., “Neuroendocrine tumors involving the gastroenteropancreatic tract: a clinicopathological evaluation of 773 cases,” Clinics 66(10), 1671–1675 (2011). 10.1590/S1807-59322011001000002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Oladejo A. O., “Gastroenteropancreatic neuroendocrine tumors (GEP-NETs)—approach to diagnosis and management,” Ann. Ibadan Postgrad. Med. 7(2), 29–33 (2009). 10.4314/aipm.v7i2.64085 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Freis P., et al. , “Prognostic factors in neuroendocrine carcinoma: biological markers are more useful than histomorphological markers,” Sci. Rep. 7, 40609 (2017). 10.1038/srep40609 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Gerdes J., et al. , “Cell cycle analysis of a cell proliferation-associated human nuclear antigen defined by the monoclonal antibody Ki-67,” J. Immunol. 133(4), 1710–1715 (1984). [PubMed] [Google Scholar]
- 36.Miller H. C., et al. , “Role of Ki-67 proliferation index in the assessment of patients with neuroendocrine neoplasms regarding the stage of disease,” World J. Surg. 38(6), 1353–1361 (2014). 10.1007/s00268-014-2451-0 [DOI] [PubMed] [Google Scholar]
- 37.Özaslan E., et al. , “Evaluation of the concordance between the stage of the disease and Ki-67 proliferation index in gastroenteropancreatic neuroendocrine tumors,” Eur. J. Gastroenterol. Hepatol. 28(7), 836–841 (2016). 10.1097/MEG.0000000000000619 [DOI] [PubMed] [Google Scholar]
- 38.Foltyn W., et al. , “The value of the Ki-67 proliferation marker as a prognostic factor in gastroenteropancreatic neuroendocrine tumours,” Endokrynol. Pol. 63(5), 362–366 (2012). [PubMed] [Google Scholar]
- 39.Khan M. S., et al. , “A comparison of Ki-67 and mitotic count as prognostic markers for metastatic pancreatic and midgut neuroendocrine neoplasms,” Br. J. Cancer 108(9), 1838–1845 (2013). 10.1038/bjc.2013.156 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Bubendorf L., et al. , “Ki67 labelling index: an independent predictor of progression in prostate cancer treated by radical prostatectomy,” J. Pathol. 178(4), 437–441 (1996). 10.1002/(ISSN)1096-9896 [DOI] [PubMed] [Google Scholar]
- 41.Vilar E., et al. , “Chemotherapy and the role of the proliferation marker Ki-67 in digestive neuroendocrine tumors,” Endocr. Relat. Cancer 14(2), 221–232 (2007). 10.1677/ERC-06-0074 [DOI] [PubMed] [Google Scholar]
- 42.Luukkaa H., et al. , “Prognostic significance of Ki-67 and p53 as tumor markers in salivary gland malignancies in Finland: an evaluation of 212 cases,” Acta Oncol. 45(6), 669–675 (2006). 10.1080/02841860500543208 [DOI] [PubMed] [Google Scholar]
- 43.Suri V. S., et al. , “Histological spectrum of ependymomas and correlation of p53 and Ki-67 expression with ependymoma grade and subtype,” Indian J. Cancer 41, 66–71 (2004). [PubMed] [Google Scholar]
- 44.Karamitopoulou E., et al. , “Ki-67 immunoreactivity in human central nervous system tumors: a study with MIB 1 monoclonal antibody on archival material,” Acta Neuropathol. 87(1), 47–54 (1994). 10.1007/BF00386253 [DOI] [PubMed] [Google Scholar]
- 45.Allegranza A., et al. , “Proliferating cell nuclear antigen expression in central nervous system neoplasms,” Virchows Arch. A 419, 417–423 (1991). 10.1007/BF01605076 [DOI] [PubMed] [Google Scholar]
- 46.Molino A., et al. , “Ki-67 immunostaining in 322 primary breast cancers: association with clinical and pathological variables and prognosis,” Int. J. Cancer 74(4), 433–437 (1997). 10.1002/(ISSN)1097-0215 [DOI] [PubMed] [Google Scholar]
- 47.Gunia S., et al. , “Ki67 staining index and neuroendocrine differentiation aggravate adverse prognostic parameters in prostate cancer and are characterized by negligible inter-observer variability,” World J. Urol. 26(3), 243–250 (2008). 10.1007/s00345-008-0257-0 [DOI] [PubMed] [Google Scholar]
- 48.Preusser M., et al. , “Ki67 index in intracranial ependymoma: a promising histopathological candidate biomarker,” Histopathology 53(1), 39–47 (2008). 10.1111/his.2008.53.issue-1 [DOI] [PubMed] [Google Scholar]
- 49.Karamitopoulou E., et al. , “Proliferating cell nuclear antigen immunoreactivity in human central nervous system neoplasms,” Acta Neuropathol. 85(3), 316–322 (1993). 10.1007/BF00227728 [DOI] [PubMed] [Google Scholar]
- 50.Grzybicki D. M., et al. , “Interobserver variability associated with the MIB-1 labeling index: high levels suggest limited prognostic usefulness for patients with primary brain tumors,” Cancer 92(10), 2720–2726 (2001). 10.1002/(ISSN)1097-0142 [DOI] [PubMed] [Google Scholar]
- 51.Mengel M., et al. , “Inter-laboratory and inter-observer reproductibility of immunohistochemical assessment of Ki-67 labelling index in a large multi-centre trial,” J. Pathol. 198(3), 292–299 (2002). 10.1002/path.1218 [DOI] [PubMed] [Google Scholar]
- 52.Lloyd R. V., “Utility of Ki-67 as a prognostic marker in pancreatic endocrine neoplasms,” Am. J. Clin. Pathol. 109(3), 245–247 (1998). 10.1093/ajcp/109.3.245 [DOI] [PubMed] [Google Scholar]
- 53.Plöckinger U., et al. , “Guidelines for the diagnosis and treatment of neuroendocrine gastrointestinal tumours: a consensus statement on behalf of the European Neuroendocrine Tumour Society (ENETS),” Neuroendocrinology 80(6), 394–424 (2004). 10.1159/000085237 [DOI] [PubMed] [Google Scholar]
- 54.Rindi G., et al. , “TNM staging of foregut (neuro)endocrine tumors: a consensus proposal including a grading system,” Virchows Arch. 449(4), 395–401 (2006). 10.1007/s00428-006-0250-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Klöppel G., et al. , “Site-specific biology and pathology of gastroenteropancreatic neuroendocrine tumors,” Virchows Arch. 451, 9–27 (2007). 10.1007/s00428-007-0461-0 [DOI] [PubMed] [Google Scholar]
- 56.Kim B. S., et al. , “Comparison of the prognostic values of the 2010 WHO classification, AJCC 7th edition, and ENETS classification of gastric neuroendocrine tumors,” Medicine 95(30), e3977 (2016). 10.1097/MD.0000000000003977 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Herrera-Martínez A. D., et al. , “Targeted systemic treatment of neuroendocrine tumors: current options and future perspectives,” Drugs 79, 21–42 (2019). 10.1007/s40265-018-1033-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Leung S. C. Y., et al. , “Analytical validation of a standardized scoring protocol for Ki67: phase 3 of an international multicenter collaboration,” NPJ Breast Cancer 2(1), 1–9 (2016). 10.1038/npjbcancer.2016.14 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Polley M. Y. C., et al. , “An international Ki67 reproducibility study,” J. Natl. Cancer Inst. 105(24), 1897–1906 (2013). 10.1093/jnci/djt306 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Varga Z., et al. , “How reliable is Ki-67 immunohistochemistry in grade 2 breast carcinomas? A QA study of the Swiss Working Group of Breast- and Gynecopathologists,” PLoS One 7(5), e37379 (2012). 10.1371/journal.pone.0037379 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Polley M. Y., et al. , “An international study to increase concordance in Ki67 scoring,” Mod. Pathol. 28(6), 778–786 (2015). 10.1038/modpathol.2015.38 [DOI] [PubMed] [Google Scholar]
- 62.Laurinavicius A., et al. , “A methodology to ensure and improve accuracy of Ki67 labelling index estimation by automated digital image analysis in breast cancer tissue,” Breast Cancer Res. 16(2), 1–13 (2014). 10.1186/bcr3639 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Koopman T., et al. , “Digital image analysis of Ki67 proliferation index in breast cancer using virtual dual staining on whole tissue sections: clinical validation and inter-platform agreement,” Breast Cancer Res. Treat. 169(1), 33–42 (2018). 10.1007/s10549-018-4669-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Aeffner F., et al. , “Introduction to digital image analysis in whole-slide imaging: a white paper from the digital pathology association,” J. Pathol. Inf. 10, 9 (2019). 10.4103/jpi.jpi_82_18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Kwon A. Y., et al. , “Practical approaches to automated digital image analysis of Ki-67 labeling index in 997 breast carcinomas and causes of discordance with visual assessment,” PLoS One 14(2), e0212309 (2019). 10.1371/journal.pone.0212309 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Esteva A., et al. , “Dermatologist-level classification of skin cancer with deep neural networks,” Nature 542(7639), 115–118 (2017). 10.1038/nature21056 [DOI] [PMC free article] [PubMed] [Google Scholar]










