Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2013 Dec 30;8(12):e82576. doi: 10.1371/journal.pone.0082576

Evaluation of Mitotic Activity Index in Breast Cancer Using Whole Slide Digital Images

Shaimaa Al-Janabi 1, Henk-Jan van Slooten 1, Mike Visser 2, Tjeerd van der Ploeg 3, Paul J van Diest 4, Mehdi Jiwa 1,*
Editor: Zoran Culig5
PMCID: PMC3875418  PMID: 24386102

Abstract

Introduction

Mitotic Activity Index (MAI) is an important independent prognostic factor and an integral part of the breast cancer grading system. Thus, correct estimation of this prognostically relevant feature is essential for guiding treatment decision and assessing patient prognosis.

The aim of this study was to validate the use of high resolution Whole Slide Images (WSI) in estimating MAI in breast cancer specimens.

Methods

MAI was evaluated in 100 consecutive breast cancer specimens by three observers on two occasions, microscopically and on WSI with a wash out period of 4 months. MAI was also translated to mitotic scores as in grading. Inter- and intra-observer agreement between microscopic and digital MAI counts and scores was measured.

Results

Almost perfect inter-observer agreements were obtained from counting MAI using a conventional microscope (intra-class correlation coefficient (ICCC) 0.879) as well as on WSI (ICCC 0.924). K coefficients reflected good inter-observer agreements among observers' microscopic mitotic scores (average kappa 0.642). Comparable results were also observed among digital mitotic scores (average kappa 0.635). There was strong to perfect intra-observer agreements between MAI counts and mitotic scores for the two diagnostic modalities (ICCC 0.716–0.863, kappa 0.506–0.617). There were no significant differences in mitotic scores using both diagnostic modalities.

Conclusion

Scoring mitoses using WSI in breast cancer seems to be just as reliable and reproducible as when using a microscope. Further development of software and image quality will definitely encourage the use of WSI in routine pathology practice.

Introduction

More than a decade ago, the practice of pathology began changing, with the introduction of slide scanners which enable the acquisition of pathology information from glass slides and translate it into a digital form commonly known as digital slides or Whole Slide Images (WSI). WSI provide the possibility of viewing and manipulating pathology samples on a computer screen in a way comparable to a conventional microscope [1]. Moreover, WSI boast many advantages over glass slides and a conventional microscope; including easy image accessibility, sharing, annotating and amenability to automated image analysis which is believed to improve the objectivity and productivity within pathology practice. These features facilitated WSI integration in different pathology applications, mainly used for education, consultation, frozen section diagnosis, quality assurance, clinico-pathological conferences and research [1]–[4]. Despite the fact that several validation studies have shown that the diagnostic performance using WSI is comparable to that of a conventional microscope [5]–[12], implementing WSI in primary diagnostics is still in its infancy. However, WSI have been used for this purpose in some pathology laboratories after carrying out their own local validation studies [13], [14]. One of the possible factors hindering WSI integration in routine pathology practice is that they have yet to be approved for primary diagnostics by the Food and Drug Administration (FDA) [15]. Additionally, the FDA has classified whole slide scanners as Class III medical devices (Slide scanner classification) necessitating extensive systematic validation studies and premarket approval before WSI can become a platform for primary diagnostics [16].

From our previous studies concerning the validation of WSI for primary diagnostics of different body systems [6]–[9], we concluded that WSI contain sufficient information for rendering most of the diagnostics within pathology. Nevertheless, we would expect that examining fine cellular details such as cellular division (mitosis) on WSI, scanned at one focal plane could pose some diagnostic difficulties. Thus, testing the validity of WSI in assessing this theoretically difficult but clinically relevant feature is crucial.

In breast cancer, tumor proliferation is one of the most important independent prognostic factors and is an integral part of the breast tumor grading system [17], [18] which has also an impact on the determination of patient treatment [19]. Different techniques may be used to estimate proliferation [20]–[22]; the most widely applicable method used in the common practice is the estimation of the mitotic activity index (MAI). MAI is defined as the numbers of mitotic figures in a given area of tumor [23]. Traditionally, MAI is scored on glass slides using light microscopy where mitosis is counted in 10 high power fields (40× magnification) or per unit area (2 mm2) in the most active part of the tumor [21], [23], [24]. Scoring MAI under a microscope requires the differentiation of true mitoses from similar figures such as apoptotic bodies, dark nuclei and tissue artifacts, for which a three-dimensional view and a fine microscopic focusing is required. Missing the z-axis and the ability of fine microscopic focusing on WSI scanned at one focal plane, may lead to under or overestimating MAI scores on WSI. To our knowledge, this is the first multi-observer study concerned with validating the scoring of MAI in breast cancer on the bases of WSI and digital microscope.

Materials and Methods

This study was performed at Symbiant Pathology Expert Center in The Netherlands, consisting of pathology laboratories at three different locations serving 6 hospitals in the province of North Holland with a population of about one million people. For this study no ethics committee approval or patient consent were required. Additionally, all samples were properly coded and anonymized (Federal Guidelines).

One hundred consecutive breast cancer cases which have been previously assessed for their proliferative activity were included in this study. These concerned 6 biopsies in cases undergoing neo-adjuvant chemotherapy and 94 resections from two laboratories. From each case, one representative slide was selected by two pathologists to be used for evaluating the Mitotic Activity Index (MAI). In addition, the regions for mitosis counting were marked beforehand. This study was performed in two phases. First, MAI was scored by three observers on the same marked area on the selected glass slides using light microscopy. Thereafter, the glass slides were scanned and after a wash out period of at least 4 months WSI were presented to the same observers to recount mitosis. Table 1 details the cases included in the study.

Table 1. Overview of cases included for comparing mitoses counts on glass slides and whole slide images.

Diagnostic entity Specimen type
Biopsies Resections Total
Invasive ductal carcinoma 6 76 82
Invasive lobular carcinoma 13 13
Mucinous carcinoma 1 1
Papillary carcinoma 3 3
Tubular carcinoma 1 1
Total 6 94 100
Open in a new tab

Microscopically, only cells with very evident morphology of mitosis were counted as defined before [17], [25] by absence of the nuclear membrane, clearly visible hairy extension of nuclear material (condensed chromosome), either clotted (beginning metaphase), in plane (metaphase/anaphase), or in separate clots (telophase). Doubtful cells with a hyperchromatic nucleus or cells suspected of apoptosis were excluded. The above mentioned criteria have been adopted in counting mitoses using a conventional microscope as well as WSI.

The time needed for scoring mitosis was recorded for the first ten cases in this study. Additionally, tissue quality (as poor, acceptable or good) and scan quality (as hazy, acceptable with some indistinct regions, acceptable or good) were assessed by all observers.

MAI assessment on glass slides by conventional microscopy

Two pathologists marked the regions for mitoses counting on the H&E slides. These regions were selected at the most cellular area of the tumor, mostly located at the peripheral invasive part of the tumor as before [26]. Areas with necrosis or Ductal Carcinoma In Situ were excluded. Counting mitoses was performed at 400× magnification using a Leica light microscope equipped with 10× ocular and 40× (0.85 N/A) objective (having a field diameter of about 540 µm) in 9 consecutive fields with a total surface area of 2.06 mm2. The total number of mitoses in those 9 fields was taken as the MAI.

MAI assessment on WSI

Glass slides were scanned using a Leica Scanner SCN400 at 40×. The standard image viewer for Leica Scanner “Digital Image Hub” was used for annotating and exploring WSI. WSI were displayed on high resolution 30″ Barco Pathology Displays (Barco, Brussels, Belgium) having a resolution of 6 MP. Examining WSI on 40×, in an area of 2 mm2, about 7 screen fields fitted into the same 2 mm2 area annotated before on the glass slides. Each observer was asked to annotate all the mitotic figures that he could detect within this area. Afterwards mitoses annotations were counted for each observer separately. Figure 1 is a snapshot from a WSI of an invasive breast cancer showing the selected areas for counting mitosis digitally and microscopically in addition to the digitally annotated mitotic figures within a 2 mm2 area.

Figure 1. Snapshot of WSI showing the annotated areas for scoring mitosis microscopically and digitally.

Figure 1

Open in a new tab

1- An overview image of the WSI, 2- The defined area for counting mitosis microscopically, 3- The defined area for counting mitosis digitally, 4a- Microscopic snapshots of different mitotic figures, 4b- Digital snapshots of different mitotic figures.

Direct comparison of mitoses on glass slides and WSI

Perception of mitotic figures might be more difficult on WSI scanned at one focal plane than in conventional slides where one can perceive the 3-D structure by focusing. For this purpose and in order to gain insight into the differences in appearance using the two diagnostic modalities, mitotic figures from 15 cases were identified under a microscope and compared instantaneously with the corresponding object on WSI. Digitally, these mitoses appeared as dark nuclei with very fine projections (early metaphase), in plane with irregular margins (late metaphase), mitoses with individually dispersed chromosomes and dark ring-like shapes (anaphase) or as separated parallel dark clots (telophase). Figure 2 shows snapshots from several WSI showing these different forms of actual mitotic figures. Figure 3 shows snapshots from WSI showing different mitosis-like figures.

Figure 2. Snapshots from several WSI of several breast resections diagnosed previously as an infiltrative ductal carcinoma using a light microscopy.

Figure 2

Open in a new tab

These snapshots showing different appearances of mitotic figures encircled by green circles. Panels A–C show cells in early metaphase. Panels D–G show different forms of mitotic division in late metaphase. Panel H–L shows different forms of anaphase. Panels M–P show cells in telophase.

Figure 3. Snapshots of WSI showing the possible appearance of mitosis-like figures surrounded by green circles.

Figure 3

Open in a new tab

Data evaluation

MAI values were transferred into mitotic scores as for grading as follows: Score 1: 0–6 mitosis/2 mm2, Score 2: 7–12 mitosis/2 mm2, Score 3: 13 mitosis or more/2 mm2.

Agreement was assessed between observers using the same diagnostic modality, and for each observer using the two different modalities. Intra- and inter-observers agreement for the continuous MAI was assessed using Intra Class Correlation Coefficient (ICCC), scores 0–0.2, 0.3–0.4, 0.5–0.6, 0.7–0.8, >0.8 indicating poor, fair, moderate, strong and almost perfect agreement, respectively. For mitotic scores, kappa statistics (K) were calculated to estimate inter- and intra-observer agreement [27]–[30], kappas <0.20, 0.21–0.40, 0.41–0.60, 0.61–0.80, and 0.81–1.00 indicating poor, fair, moderate, good and perfect agreement, respectively.

The level of significance was calculated using the Wilcoxon signed-rank test. Systematic differences between microscopic and digital MAI values and mitotic scores were read from the Wilcoxon signed rank test and scatter plots.

The possible effects of tissue and scan quality on differences between the conventional and digital MAI assessments were evaluated using the Mann-Whitney test.

Results

For all observers, tissue quality did not have a significant effect on the differences between conventional and digital mitotic scores (P = 0.836, 0.187 and 0.225 for observers 1, 2, and 3, respectively).

Per observer, there was no significant effect of scan quality on the differences in scoring mitosis conventionally and on WSI (P = 0,328, 0,275 and 0.266 for observer 1, 2 and 3 respectively.

Counting mitoses on WSI was more time consuming than on glass slides. The average amount of time needed to count mitoses on glass slides ranged from 3–5 minutes versus 10–12 minutes for WSI.

Inter-observer agreement for the same diagnostic modality

There was almost perfect inter-observer agreement among all observers in assessing MAI using a conventional microscope (ICCC 0.879) and on WSI (ICCC 0.924).

Mitotic scores again yielded a good inter-observer agreements among all observers using a conventional microscope (average kappa 0.642 (K1 = 0.645, K2 = 0.667, K3 = 0.615)), and WSI (average kappa 0.635 (K1 = 0.756, K2 = 0.584, K3 = 0.565)). Figure 4 gives an overview of stepwise kappa statistics between observers.

Figure 4. Overview of stepwise kappa statistics between observers.

Figure 4

Open in a new tab

Intra-observer agreement for microscopic vs. WSI based mitoses counting

There was strong to perfect intra-observer agreement in counting mitoses when comparing both diagnostic modalities with ICCC of 0.863, 0.716, and 0.773 for observers 1, 2 and 3, respectively. In general, there was a noticeable trend towards underestimating mitotic counts on WSI if compared to microscopic mitotic counts as shown in table 2 and figure 5 per observer. Additionally, there was a better correlation between high (digital/microscopic) mitotic counts than between lower mitotic counts assessed by the two modalities. Pearson correlation coefficients between (digital and microscopic) mitotic counts from all observers sorted per mitotic scores categories are 0.455, 0.179 and 0.617 for scores 1,2 and 3 respectively.

Table 2. Overview of mitotic counts showing the trends towards underestimating mitotic counts when performed digitally.

Observer 1 Observer 2 Observer 3 Total
Mitotic counts estimation Digitally underestimated* 47 55 54 156
Digitally overestimated* 37 27 35 99
Same counts digitally and microscopically 16 18 11 45
Total 100 100 100 300
Agreement of digital and microscopic mitotic counts using ICCC 0.863 0.716 0.773 0.779
Open in a new tab

Figure 5. Scatter plots of microscopic versus digital MAI counts per observer indicating the tendency to underestimate of mitotic counts on WSI.

Figure 5

Open in a new tab

Moderate to good intra-observer agreement was observed between mitotic scores using both methods with kappa values of 0.617, 0.617, and 0.506 for observers 1, 2 and 3, respectively. Digital MAI scores were lower in 43/300 pairs of scores (microscopic and digital) and higher in 30/300 pairs (P = 0.683, 0.086 0.590 for the three observers respectively). Table 3 details per observer the intra-observer agreement of microscopic versus digital mitotic scores.

Table 3. Concordance rate of each observer for scoring Mitotic Activity Index (MAI) using WSI and conventional microscopy.

Observer 1 Observer 2 Observer 3 Total
Mitotic scores estimation Digitally underestimated* 11 15 17 43
Digitally overestimated* 10 6 14 30
Same scores digitally and microscopically 79 79 69 227
Open in a new tab

Per observer, digital mitotic scores have been compared with the gold standard (the conventional mitotic scores) and assessed as under- or overestimated. This assessment is general and includes all scores categories, thus scores1, 2 and 3.

Discussion

The aim of this study was to validate the use of WSI in evaluating the MAI in breast cancer cases. MAI is an integral part of the breast cancer grading system and eventually gives an estimation of the degree of aggressiveness of the tumor and guides treatment protocols [31]. Correct evaluation of this prognostically relevant criterion is crucial since under- or overestimating mitosis scores could have important clinical implications for the patient [32].

100 breast cancer biopsies and resections were subjected to mitosis counting by three observers on two occasions; first using a conventional microscope and then after a wash-out period of at least 4 months on WSI scanned at 40×. There was almost perfect inter-observer agreement in assessing the MAI on the bases of the conventional microscope (ICCC 0.879) and on WSI (ICCC 0.924). There was also a good inter-observer agreement among three observers in scoring MAI using either a conventional microscope or WSI with average kappa values of 0.642 and 0.635, respectively. The results of this study are comparable to other studies that examined inter-observer agreement of scoring mitoses and grading of breast cancer cases by conventional microscopy only [17], [26], [33], [34].

There was a tendency to slightly underestimating the number of mitoses on WSI (table 2), but when transferring mitoses counts to mitotic scores as in grading, WSI based scores did not significantly differ from scoring mitosis using glass slides and a conventional microscope (Table 3). This indicates that scoring mitosis in breast cancer cases can be reliably done on WSI scanned at 40× magnifications and at one focal plane without influencing prognostic impact of mitotic counts.

Inter-observer agreement of the digital mitotic counts (ICCC 0.924) was slightly higher than microscopic mitotic counts (ICCC 0.879). This might be due to the fact that digital mitotic counts were performed precisely in the same annotated area of 2 mm2 whereas this was not the case for microscopic counting. Selection of different areas for estimating MAI and tumor heterogeneity [32] might explain the slightly lower observer agreement in counting mitosis microscopically.

Fine microscopic focusing can be helpful for differentiation of actual mitoses from mitotic-like bodies. Losing the ability of fine focusing on WSI scanned at one focal plane may theoretically impede mitosis identification. Scanning glass slides on multiple focal planes providing a z-axis to WSI may facilitate the digital evaluation of mitotic figures but increases scanning time and storage requirements which is yet impractical for routine pathology work. With the continuous improvement of scanning speed and reduction in storage cost, we expect that such limitations will be solved in the near future. Improving inter- and intra-observer reproducibility in counting mitoses can possibly be achieved by following a strict scoring protocol [17] as well as practicing more digital MAI scoring [2], [5].

Counting mitoses on WSI turned out to be more time consuming than its conventional counterpart, mainly due to cumbersome software that requires 5–6 mouse clicks to annotate one mitotic figure, and counting the total number of annotations at the end. However, annotating each mitotic figure was important for the context of this study but might not be necessary in routine practice. Adjusting the next versions of the software for research purposes to include more features such as one click annotation, an option for an automatic mitotic annotation counter and applying a 2 mm2 grid has been discussed with the vender. These additional features will definitely decrease scoring time and risks of error in counting mitoses and may eventually increase reproducibility. Furthermore, running automated MAI scoring on WSI would be a step forward and will assist in the objective determination of mitotic activity and hence tumor grading. Automatic detection of cancerous epithelial cells on imprint cytology slides created from breast cancer specimens [35], automated measurement of nuclear size in breast cancer [36], has already been tried with acceptable results.

The quality of WSI was generally good and adequate for use in estimating the MAI. The most frequently used level of magnification for mitosis perception was 80× digital magnification since this level of magnification offers the observer a field of view most comparable to 40× under a microscope. Also, keyboard shortcuts, which provide a more user friendly and optimal navigation within WSI, were used to move WSI in order to explore a 2 mm2 surface area.

Tissue quality was not optimal for every case included in this study and this might be the reason behind the discrepancies in mitotic scores. This leads to extra difficulty in counting mitosis either microscopically or digitally. Such cases were not excluded from this study as they reflect the routine mix in this pathology centre. Since poor tissue morphology can have an effect on counting mitoses [17], [32] and can rapidly compromise the quality of WSI for MAI scoring, further studies testing the effect of proper tissue morphology on digital mitosis scoring are important. However, the quality of the tissue sections included in this study did not have significant effect on the differences in scoring mitosis microscopically and digitally.

Despite of the fact that the quality of the currently produced WSI is sufficient to perform most of the diagnostics within pathology as has been approved by several validation studies, primary diagnostics based solely on WSI requires improvement of many issues such as scanning speed, image quality, software solutions and navigation interface which will definitely guarantee the successful integration of WSI routine pathology.

In conclusion, counting mitoses in breast cancer can reliably be done on high resolution WSI scanned at one focal plane. Further improvement in the software characteristics, scanning speed, and image quality will definitely encourage the use of WSI in routine practice.

Acknowledgments

We would like to thank Alice van Berne for her quick editing work. We would also like to thank Andre Slappendel for his help in calibrating the microscopic work station and his explanation of the work process in the morphometry department.

Funding Statement

These authors have no support or funding to report.

References

  • 1. Al Janabi S, Huisman A, van Diest PJ (2012) Digital pathology: current status and future perspectives. Histopathology 61 (1) 1–9. [DOI] [PubMed] [Google Scholar]
  • 2. Ho J, Parwani AV, Jukic DM, Yagi Y, Anthony L, et al. (2006) Use of whole slide imaging in surgical pathology quality assurance: design and pilot validation studies. Hum Pathol 37 (3) 322–331. [DOI] [PubMed] [Google Scholar]
  • 3. Dee FR (2009) Virtual microscopy in pathology education. Hum Pathol 40 (8) 1112–1121. [DOI] [PubMed] [Google Scholar]
  • 4. Rocha R, Vassallo J, Soares F, Miller K, Gobbi H (2009) Digital slides: present status of a tool for consultation, teaching, and quality control in pathology. Pathol Res Pract 205 (11) 735–741. [DOI] [PubMed] [Google Scholar]
  • 5. Nielsen PS, Lindebjerg J, Rasmussen J, Starklint H, Waldstrom M, et al. (2010) Virtual microscopy: an evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. Hum Pathol 41 (12) 1770–1776. [DOI] [PubMed] [Google Scholar]
  • 6. Al Janabi S, Huisman A, Vink A, Leguit RJ, Offerhaus GJ, et al. (2012) Whole slide images for primary diagnostics of gastrointestinal tract pathology: a feasibility study. Hum Pathol 43 (5) 702–707. [DOI] [PubMed] [Google Scholar]
  • 7. Al Janabi S, Huisman A, Vink A, Leguit RJ, Offerhaus GJ, et al. (2012) Whole slide images for primary diagnostics in dermatopathology: a feasibility study. J Clin Pathol 65 (2) 152–158. [DOI] [PubMed] [Google Scholar]
  • 8. Al Janabi S, Huisman A, Willems SM, van Diest PJ (2012) Digital slide images for primary diagnostics in breast pathology: a feasibility study. Hum Pathol 43 (12) 2318–2325. [DOI] [PubMed] [Google Scholar]
  • 9. Al Janabi S, Huisman A, Nikkels PG, Ten Kate FJ, van Diest PJ (2013) Whole slide images for primary diagnostics of paediatric pathology specimens: a feasibility study. J Clin Pathol 66 (3) 218–223. [DOI] [PubMed] [Google Scholar]
  • 10. Molnar B, Berczi L, Diczhazy C, Tagscherer A, Varga SV, et al. (2003) Digital slide and virtual microscopy based routine and telepathology evaluation of routine gastrointestinal biopsy specimens. J Clin Pathol 56 (6) 433–438. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Gilbertson JR, Ho J, Anthony L, Jukic DM, Yagi Y, et al. (2006) Primary histologic diagnosis using automated whole slide imaging: a validation study. BMC Clin Pathol 6: 4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12. Li X, Liu J, Xu H, Gong E, McNutt MA, et al. (2007) A feasibility study of virtual slides in surgical pathology in China. Hum Pathol 38 (12) 1842–1848. [DOI] [PubMed] [Google Scholar]
  • 13. Al Janabi S, Huisman A, Nap M, Clarijs R, van Diest PJ (2012) Whole slide images as a platform for initial diagnostics in histopathology in a medium-sized routine laboratory. J Clin Pathol 65 (12) 1107–1111. [DOI] [PubMed] [Google Scholar]
  • 14. Pantanowitz L, Valenstein PN, Evans AJ, Kaplan KJ, Pfeifer JD, et al. (2011) Review of the current state of whole slide imaging in pathology. J Pathol Inform 2: 36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Lange H (2011) Digital Pathology: A Regulatory Overview. Lab Med 42: 587–591. [Google Scholar]
  • 16. Cornish TC, Swapp RE, Kaplan KJ (2012) Whole-slide imaging: routine pathologic diagnosis. Adv Anat Pathol 19 (3) 152–159. [DOI] [PubMed] [Google Scholar]
  • 17. van Diest PJ, Baak JP, Matze-Cok P, Wisse-Brekelmans EC, van Galen CM, et al. (1992) Reproducibility of mitosis counting in 2,469 breast cancer specimens: results from the Multicenter Morphometric Mammary Carcinoma Project. Hum Pathol 23 (6) 603–607. [DOI] [PubMed] [Google Scholar]
  • 18. Lester SC, Bose S, Chen YY, Connolly JL, de Baca ME, et al. (2009) Protocol for the examination of specimens from patients with invasive carcinoma of the breast. Arch Pathol Lab Med 133 (10) 1515–1538. [DOI] [PubMed] [Google Scholar]
  • 19. Baak JP, van Diest PJ, Voorhorst FJ, van der WE, Beex LV, et al. (2005) Prospective multicenter validation of the independent prognostic value of the mitotic activity index in lymph node-negative breast cancer patients younger than 55 years. J Clin Oncol 23 (25) 5993–6001. [DOI] [PubMed] [Google Scholar]
  • 20. Pietilainen T, Lipponen P, Aaltomaa S, Eskelinen M, Kosma VM, et al. (1996) The important prognostic value of Ki-67 expression as determined by image analysis in breast cancer. J Cancer Res Clin Oncol 122 (11) 687–692. [DOI] [PubMed] [Google Scholar]
  • 21. van Diest PJ, Brugal G, Baak JP (1998) Proliferation markers in tumours: interpretation and clinical value. J Clin Pathol 51 (10) 716–724. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Medri L, Volpi A, Nanni O, Vecci AM, Mangia A, et al. (2003) Prognostic relevance of mitotic activity in patients with node-negative breast cancer. Mod Pathol 16 (11) 1067–1075. [DOI] [PubMed] [Google Scholar]
  • 23. Fitzgibbons PL, Page DL, Weaver D, Thor AD, Allred DC, et al. (2000) Prognostic factors in breast cancer. College of American Pathologists Consensus Statement 1999. Arch Pathol Lab Med 124 (7) 966–978. [DOI] [PubMed] [Google Scholar]
  • 24. van Diest PJ, van der WE, Baak JP (2004) Prognostic value of proliferation in invasive breast cancer: a review. J Clin Pathol 57 (7) 675–681. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Baak JP, van Diest PJ, Benraadt T, Matze-Cok E, Brugghe J, et al. (1993) The Multi-Center Morphometric Mammary Carcinoma Project (MMMCP) in The Netherlands: value of morphometrically assessed proliferation and differentiation. J Cell Biochem Suppl 17G: 220–225. [DOI] [PubMed] [Google Scholar]
  • 26. Tsuda H, Akiyama F, Kurosumi M, Sakamoto G, Yamashiro K, et al. (2000) Evaluation of the interobserver agreement in the number of mitotic figures of breast carcinoma as simulation of quality monitoring in the Japan National Surgical Adjuvant Study of Breast Cancer (NSAS-BC) protocol. Jpn J Cancer Res 91 (4) 451–457. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33 (1) 159–174. [PubMed] [Google Scholar]
  • 28. Viera AJ, Garrett JM (2005) Understanding interobserver agreement: the kappa statistic. Fam Med 37 (5) 360–363. [PubMed] [Google Scholar]
  • 29. Meyer JS, Alvarez C, Milikowski C, Olson N, Russo I, et al. (2005) Breast carcinoma malignancy grading by Bloom-Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index. Mod Pathol 18 (8) 1067–1078. [DOI] [PubMed] [Google Scholar]
  • 30. Beck JS (1985) Observer variability in reporting of breast lesions. J Clin Pathol 38 (12) 1358–1365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Clayton F (1991) Pathologic correlates of survival in 378 lymph node-negative infiltrating ductal breast carcinomas. Mitotic count is the best single predictor. Cancer 68 (6) 1309–1317. [DOI] [PubMed] [Google Scholar]
  • 32. Beresford MJ, Wilson GD, Makris A (2006) Measuring proliferation in breast cancer: practicalities and applications. Breast Cancer Res 8 (6) 216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Longacre TA, Ennis M, Quenneville LA, Bane AL, Bleiweiss IJ, et al. (2006) Interobserver agreement and reproducibility in classification of invasive breast carcinoma: an NCI breast cancer family registry study. Mod Pathol 19 (2) 195–207. [DOI] [PubMed] [Google Scholar]
  • 34. Robbins P, Pinder S, de Klerk N, Dawkins H, Harvey J, et al. (1995) Histological grading of breast carcinomas: a study of interobserver agreement. Hum Patho 26 (8) 873–879. [DOI] [PubMed] [Google Scholar]
  • 35. Martin DT, Sandoval S, Ta CN, Ruidiaz ME, Cortes-Mateos MJ, et al. (2011) Quantitative automated image analysis system with automated debris filtering for the detection of breast carcinoma cells. Acta Cytol 55 (3) 271–280. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Veta M, Kornegoor R, Huisman A, Verschuur-Maes AH, Viergever MA, et al. (2012) Prognostic value of automatically extracted nuclear morphometric features in whole slide images of male breast cancer. Mod Pathol 25 (12) 1559–65. [DOI] [PubMed] [Google Scholar]

Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES