Abstract
CanAssist Breast (CAB) is a prognostic test for early-stage hormone receptor-positive invasive breast cancer. The test involves performing immunohistochemical (IHC) analysis for five biomarkers, namely CD44, ABCC4, ABCC11, N-cadherin, and pan-cadherin. In addition to IHC grading information, three clinical features, i.e., tumor size, grade, and lymph node status, serve as input into the machine learning-based algorithm to generate the CAB risk score. CAB was developed and initially validated using manual IHC. This study’s objectives included: i) automate CAB IHC on an autostainer and establish its performance equivalence with manual IHC ii) validate CAB test using samples in Tissue MicroArray (TMA) format. IHC for CAB biomarkers was standardized on Ventana BenchMark XT autostainer. Two IHC methods were compared for IHC gradings and corresponding CAB risk scores/risk categories. A concordance analysis was done using MedCalcTM software. The manual and automated IHC staining methods exhibited a high level of concordance on IHC gradings for 40 cases with an Intra-class Correlation Coefficient (ICC) of >0.85 for 4 of 5 biomarkers. 100% concordance was achieved in risk categorization (low- or high-risk), with very good agreement between the risk scores demonstrated by a kappa statistic of 0.83. TMA versus whole tissue section concordance was analyzed using 45 samples on an autostainer, and the data showed 92% concordance in terms of risk category. The results confirm the equivalence between manual and automated staining methods and demonstrate the utility of TMA as an acceptable format for CanAssist Breast testing.
Keywords: Breast cancer, prognosis, machine learning, risk classifier, method comparison
Introduction
Immunohistochemistry (IHC) is considered a gold standard technique used to localize a specific antigen or protein of interest, based on the principle of antigen-antibody interactions [1,2]. Since the inception of the technique, numerous modifications and advancements have been incorporated to make the test more reliable and robust as a diagnostic tool [3]. IHC performed by the manual method is a complex and multistep technique requiring considerable hands-on time, typically entrusted to skilled staff. Reliance on manpower makes this technique susceptible to variations brought on by operators, observers, runs and reagents. Automation of this technique helps mitigate these gaps. Considering the impact of pre-analytical, analytical and post-analytical factors (fixation, choice of primary antibody and detection system, and accurate interpretation of staining), any change or improvement to an IHC assay must be validated before patient testing [4-7]. Also, IHC analysis on a large number of tumor tissue samples has become mandatory to understand the clinical utility of any diagnostic, prognostic, and predictive marker. Tissue MicroArray (TMA), a recent innovation in histopathology, has been an invaluable addition in discovering and validating new biomarkers [8]. A tissue microarray contains many small representative tissue samples from numerous tumor tissue blocks assembled on a single paraffin block. Therefore, studies conducted on TMA allow the high throughput analysis of tumor biology with the highest experimental uniformity possible. Furthermore, it helps improve the utilization of valuable tissue resources, which acts as a limiting factor in multiple clinical studies [9,10].
The advent of automation in IHC and the use of tissue microarray technology have had a beneficial impact on biomarker validation by helping in the acquisition of large datasets [11]. CanAssist Breast (CAB) is an IHC-based test that uses a machine learning-based risk classifier that aids in the risk stratification of patients with early-stage hormone receptor-positive breast cancer [12]. CAB stratifies patients into two distinct risk groups, low- or high-risk, for distant recurrence based on the ‘CAB risk score’ derived using IHC data from a panel of biomarkers (CD44, ABCC4, ABCC11, N-cadherin, and pan-cadherin), in association with clinical prognostic factors (tumor size, grade, and lymph node status). The CAB risk score is a numerical value on a scale of 0 to 100 with a pre-defined cut-off of 15.5. The patient is classified either as low-risk (≤15.5) or high-risk (≥15.6) for distant recurrence based on the individual CAB risk score [12]. CAB was clinically validated in a retrospective cohort of 857 patients [13]. The analytical performance of CAB is established across several repeatability and reproducibility variables, making it a robust prognostic test [14].
In this manuscript, we describe the standardization and validation of the IHC staining of CAB biomarkers on the Ventana BenchMark XT IHC autostainer. Equivalent performance of CAB by manual and automated IHC staining methods is established by comparing CAB risk scores and risk categories. Additionally, we also showcase comparable performance of CAB performed on whole tissue sections versus in tissue microarray format using an autostainer.
Materials and methods
Sample selection
Forty FFPE blocks with more than 30% tumor content from primary surgically resected specimens of early-stage (AJCC staging-I/II), hormone receptor-positive, HER2 negative invasive breast cancer patients were included in this study; inclusion and exclusion criteria were followed as detailed earlier [12]. Out of the 40 FFPE sample-set, 21 samples belonged to the low-risk for recurrence category, and the remaining 19 samples belonged to the high-risk for recurrence category as per manual IHC analysis [12]. About 22.5% (9/40) of them were from the CAB risk scores 11.7 to 18.3, representing the C5-C95 range around the clinical decision cut-off point of 15.5 [15].
For TMA construction, numerous breast carcinoma samples were screened, and 45 FFPE tumor blocks were selected for TMA construction. The factors like tissue thickness, structural integrity of the block and tumor availability at discrete places in a block enabled extraction of multiple cores. Cores representative of the tumor heterogeneity were evaluated during the screening.
Materials
IHC staining, grading, and risk score generation, were performed as detailed earlier in the manual IHC method [12]. UltraView universal DAB IHC kit and OptiView DAB IHC kit detection systems; Cell Conditioning1 (CC1), Cell Conditioning2 (CC2), Protease2 retrieval solutions; Hematoxylin and Bluing counterstain reagents; and ancillary reagents like EZPrep, Reaction buffer, and LCS required to carry out IHC on Ventana BenchMark XT were procured from Roche, Ventana.
IHC standardization on the autostainer
‘One factor at a time’ approach was followed to standardize the IHC protocol on the automated IHC platform. Optimal staining was defined as specific staining of each marker at the expected location in positive control blocks (with no or minimum background staining) similar to the staining pattern obtained using manual IHC. Retrieval conditions, primary antibody concentration, detection system, and reaction times were tweaked individually or in combination, to obtain the intended staining results.
Automated IHC & CanAssist Breast workflow
Five, three-micron tissue sections on charged slides were taken from the 40 FFPE tumor blocks analyzed previously by the manual IHC method. Before loading slides onto the autostainer, slides were baked in a hot air oven at 60°C for an hour. Then, appropriately labeled slides were loaded onto slide placeholders in the instrument, along with all required reagents like detection system, wash buffer, and other ancillary reagents. Primary antibodies were dispensed manually during the antibody titration step. Next, the slides were counterstained with hematoxylin and bluing reagents. Post-run, the stained slides were sequentially dehydrated with graded alcohols 70%, 95% and 100%, then rinsed in xylene, dried, and mounted with DPX.
The expression levels of CAB biomarkers on the IHC slides at specific locations (i.e., membrane or cytoplasm) were graded by trained onco-pathologists independently. The IHC gradings and clinical data serve as input into the machine learning-based algorithm to generate the CAB risk score.
TMA construction and concordance analysis between whole sections and TMA
Quick-Ray Manual Tissue Microarrayer and the one mm core size premade recipient block with 120 core (12×10) capacity was used to prepare the TMA blocks. The TMA blocks were prepared as per the user manual of the UNITMA-Quick Ray® Manual Tissue Microarrayer by St. Johns Health Innovation Foundation, Bangalore [http://unitma.com/product/manual-tissue-microarrayer/]. The cores located at the edges were fortified with placental tissue cores (asymmetric core) to avoid possible edge effects and core loss during the staining process. 20 of the 45 samples were embedded in TMA block-1, and the remaining 25 samples were embedded in TMA block-2. Images of the prepared TMA blocks are presented in Figure 1. Each sample was represented in triplicate placed adjacent to each other and interspersed with a few normal tissue asymmetric cores to aid in orientation. The CanAssist Breast test was performed on the whole sections of the FFPE blocks and the TMA blocks, followed by computation and comparison of CAB risk scores.
Figure 1.
Representative images of (A) Donor block with cores, (B) Recipient block sampled with tumor tissue & asymmetric tissues, and (C) H&E staining of a TMA block.
Statistical analysis
Concordance was established between the automated IHC and Manual IHC-based CAB testing at the level of IHC gradings as well as for the predicted CAB risk scores and risk categories using appropriate statistical methods detailed below.
Dot plots-IHC gradings of each marker from the two methods are plotted as individual dots. The Intra-class Correlation Coefficient (ICC) and systematic mean differences were calculated using MedCalcTM software, version 18.10.2 [16,17]. Bland-Altman plots were the differences between the two methods plotted against the manual IHC method as the reference method using MedCalc. Inter-rater agreement (kappa) was calculated using MedCalc, and strength of agreement interpreted as per standard [18]. Each sample’s CanAssist Breast risk category derived from the automated IHC method was compared with the corresponding risk group obtained by the manual IHC method.
Results
IHC standardization on the autostainer
Staining obtained by the automated IHC method was similar to that obtained by manual IHC staining, as shown in Figure 2. IHC standardization on BenchMark XT autostainer was done using UltraView DAB detection system and the retrieval buffer, CC1 for both membrane and cytosolic biomarker staining. These conditions resulted in the desired intense membrane staining of CD44 (Figure 2A1, 2B1). For ABCC4 and ABCC11 antibodies, OptiView DAB was utilized to achieve membrane staining (Figure 2B2, 2B3 respectively) similar to manual staining (Figure 2A2, 2A3 respectively). For cytosolic markers, N-cadherin and pan-cadherin antibodies use of CC2 retrieval buffer, OptiView DAB for N-cadherin (Figure 2B4) and UltraView DAB kit for pan-cadherin antibodies (Figure 2B5) produced cytoplasmic staining comparable to manual staining (Figure 2A4, 2A5 respectively). The details of the final optimized protocols are provided in Table 1.
Figure 2.
CanAssist Breast biomarkers IHC Staining patterns across the Manual IHC staining and Automated IHC staining, photomicrographs at 40× magnification. A1-A5: Manually stained IHC images of CD44, ABCC4, ABCC11, N-cadherin, and pan-cadherin. Similarly, B1-B5: Automated IHC images of CD44, ABCC4, ABCC11, N-cadherin, and pan-cadherin.
Table 1.
CanAssist Breast biomarkers and their optimized protocols on the Ventana BenchMark XT autostainer
| S. No. | Biomarker | Protocol | ||
|---|---|---|---|---|
|
| ||||
| Detection System | Retrieval buffer | Primary Antibody Incubation time | ||
| 1 | CD44 | UltraView DAB kit | CC1 | 32 mins |
| 2 | ABCC4 | OptiView DAB kit | Protease2 | 32 mins |
| 3 | ABCC11 | OptiView DAB kit | Protease2 | 32 mins |
| 4 | N-Cadherin | OptiView DAB kit | CC2 | 48 mins |
| 5 | pan-Cadherin | UltraView DAB kit | CC2 | 60 mins |
CC, Cell Conditioning.
Comparison of IHC grading across the manual and automated methods
After standardization of staining protocols, the IHC gradings of all the 5 biomarkers by two methods were compared using Intra-class Correlation Coefficient (ICC) and dot plots. Figure 3A-E shows the dot plots depicting expression ranges of all biomarkers similar in both methods. The ICC for each of the CAB biomarkers are shown in Table 2 indicates high ICC ranging from 0.87-0.97 for four biomarkers CD44, ABCC4, ABCC11, and pan-cadherin. In contrast, N-cadherin showed a low ICC (0.05) despite a lack of variation in expression levels across the two methods, as noted in the dot blot in Figure 3E. Further, the systematic mean differences were calculated to substantiate the equivalence between the two staining methods, as shown in Table 2. For N-cadherin, the mean difference between the two methods was -0.25 (95% CI: -0.58 to 0.08). Thus, the low systematic mean difference and narrow 95% CI strongly indicate that the low ICC obtained is due to the limited range of protein expression of N-cadherin, unlike the other biomarkers.
Figure 3.
Dot plots for IHC scores of CanAssist Breast marker suite (manual vs. automated IHC). A. CD44 marker expression profile, % of membrane staining across 40 cases. B. Expression profile of ABCC4 marker, % of membrane staining across 40 cases. C. Percentage of ABCC11 staining across 40 cases. D. Pan-cadherin marker expression profile of percentage of Histoscore (% of staining X Intensity) cytoplasm staining across 40 cases. E. N-cadherin marker expression profile, % of cytoplasm staining across 40 cases.
Table 2.
Agreement between manual and automated IHC (Ventana BenchMark XT) staining methods, the Intra-class Correlation Coefficient (ICC), and systematic mean differences for IHC grading of CanAssist Breast biomarkers
| S. No. | Biomarker of the CAB Panel | Factors evaluated | Agreement between Manual IHC and Automated IHC | Systematic Difference between Manual IHC and Automated IHC | ||
|---|---|---|---|---|---|---|
|
|
|
|||||
| Intra-class Correlation Coefficient (ICC) | 95% CI | Mean | 95% CI | |||
| 1 | CD44 | % of Membrane staining (0-100%) | 0.97 | 0.9383 to 0.9831 | -2.9 | -6.1576 to 0.4136 |
| 2 | ABCC4 | % of Membrane staining (0-100%) | 0.87 | 0.7395 to 0.9355 | 1.84 | 0.4404 to 3.2331 |
| 3 | ABCC11 | % of Membrane staining (0-100%) | 0.94 | 0.8865 to 0.9681 | -0.41 | -1.4436 to 0.6156 |
| 4 | N-Cadherin | % of Cytoplasm staining (0-100%) | 0.05 | -0.7482 to 0.4898 | -0.25 | -0.5800 to 0.0825 |
| 5 | pan-Cadherin | Histo score (0-300) (% of Cytoplasm staining x Intensity of Cytoplasm) | 0.88 | 0.7693 to 0.9357 | 0.06 | -3.0454 to 3.1798 |
Comparison of CAB risk scores and risk categories between manual and automated methods
The risk score and risk category across 40 samples for both manual and automated methods were assessed. These samples were representative of the spectrum of risk scores observed in CAB testing of patient clinical samples. Minor differences were observed in risk scores for the same sample across the two staining methods. However, as shown in Figure 4A, 100% concordance between the risk categories predicted using IHC gradings from both methods. In addition, the linear regression analysis, shown in Figure 4B, further confirms good concordance with the high Pearson correlation coefficient (r) 0.9755 with a significant P value of <0.0001. Next, differences in the CAB risk scores generated using the two protocols were assessed by the Bland-Altman plot by using the CAB risk scores generated by the manual IHC method as the reference [17,18]. The analysis was specifically carried out on cases with risk scores in the C5-C95 range (CAB risk scores of 11.7 to 18.3) around the clinical decision cut-off point of 15.5. The mean difference of CAB risk scores across the two methods closer to the cut-off point was also within the tighter 95% of the CIs, as depicted in Figure 4C. Finally, the Kappa statistic was used to determine the degree of agreement between the methods (measuring the extent to which both methods arrive at the same risk score). Kappa statistic was 0.83, indicating a very good agreement in the CAB risk score prediction using both methods.
Figure 4.
A. Risk category concordance between the two methods. The risk categories proportion of low-risk and high-risk cases remained the same by prediction through both IHC methods, and there were no cases observed shifting from one risk category to another risk category. B. Scatter plot of 40 cases, CAB risk score by automated IHC method plotted against respective risk scores by manual IHC method. Pearson Correlation coefficient (r) between the two methods found to be 0.9755 with a significance level P<0.0001; the dotted lines represent 95% CI. C. Bland-Altman plot for the differences in CanAssist Breast risk scores between Manual to Automated IHC methods, around the clinical decision cut-off point, C5-C95 range (CAB risk scores from 11.7 to 18.3).
Analysis of CAB on whole sections versus on tissue microarray
Individual cores that were available for all the 5 CAB markers were selected. Thus, 84% of samples (38/45) had gradings of all five CAB markers by three pathologists. The pool of analyzable cases (38) is presented in Table 3, of which 26% of the cases had all three cores intact, 42% of the cases had 2 of the 3 cores intact, and 32% of cases had only one of 3 cores available to grade.
Table 3.
Summary of the cases and cores presented in TMA analysis
| Number of cases | Number of cores available to grade | Number of cores folded/floated/could not be graded | Analyzable |
|---|---|---|---|
| 10 | 3 | 0 | Yes |
| 16 | 2 | 1 | Yes |
| 12 | 1 | 2 | Yes |
| 7 | 0 | 3 | No |
| Total number of cases | 45 | ||
| Total number of cases analyzable | 38 | ||
| Total number of cores available to grade & analyze | 74 | ||
| Total number of cores floated/folded/unable to grade & analyze | 61 | ||
There was 92% concordance between the CAB risk categories derived from the TMA cores and whole tissue section IHC analysis, presented in Figure 5. The concordant group has 100% concordance for CAB risk categories between the individual cores and the whole tissue section. The number of cores available to grade did not influence the concordance in an individual case. Concordance between the methods was seen even when only one core was available to grade and analyze. In addition, four cases from the TMA block-1 were repeated in TMA block-2 preparation to assess the reproducibility of CAB risk categories using TMA, and the data showed 100% reproducibility.
Figure 5.
CanAssist Breast risk category concordance compared between the TMA and the whole sections with automated IHC. The risk categories of 35 out of 38 cases of TMAs matches that of CAB risk categories of whole section CAB analysis.
Discussion
Automation of IHC removes most variables (operator and run) associated with the manual technique and produces consistent and reproducible results [19]. It aids in the optimal utilization of reagents and resources, ensuring consistent quality. Standardization of IHC for CAB biomarkers on Ventana BenchMark XT autostainer was optimal as the IHC gradings obtained were comparable to manual IHC (Figure 3). The Ventana BenchMark XT autostainer could replicate the retrieval conditions of pressure-induced heat-mediated retrieval methods like in MERS-Multiple Epitope Retrieval System, PathnSitu, in retrieving epitopes and in turn detection of target antigens. OptiView DAB detection system with two-step linker and multimer system led to improved sensitivity with enhanced signal amplification. The most crucial advantage of IHC on the Ventana BenchMark XT over the manual method was the reduced background staining while maintaining tissue architecture intact.
As shown in Figure 4A and 4B, there was 100% concordance in terms of risk category and a very good agreement in risk scores generated by both the methods, demonstrating that the change in methodology did not influence the performance of CAB. There was good agreement in IHC gradings by ICC for four biomarkers out of five except N-cadherin which showed a lower ICC value. Unlike the other 4 markers, the N-cadherin expression shows limited spread on the grading scale of 0-100. ICC is used to measure the degree of relatedness of values from different methods structured as groups. Since N-cadherin expression has a very narrow range (78-82%), ICC was unable to classify the data into different groups, resulting in lower ICC. Nevertheless, the systematic differences value, denoting the variability between the methods, is not significant and proves the equivalence between both staining methods. A similar narrow expression range of N-cadherin was also reported in analytical validation of CAB which indicates the inherent nature of the expression of this biomarker [14]. The magnitude of difference between the CAB risk scores by manual and automated methods was found to be within ±1.96 SD in C5 to C95 range as shown in a Bland Altman plot (Figure 4C). This demonstrates good concordance even near the cut-off point.
IHC automation can help in meeting the requirements of high reproducibility and repeatability concomitant with increased demand [4,20]. Consistent with this, a recent study on the transition from manual to automated IHC staining reported staining protocols for 78 primary antibodies on the Ventana BenchMark XT platform [20]. Automation of CanAssist Breast IHC also opens up the possibility of decentralizing the testing allowing CAB to be performed in laboratories equipped with Ventana BenchMark XT autostainer.
In the process of evaluating the applicability and acceptability of TMAs for CAB testing, a 92% concordance was noted between risk categories using the conventional whole tissue sections and the TMA samples. The study results unravel the possibility of exploring the clinical utility of the CAB test across different populations by testing the specific study cohorts, especially where the tissue samples are preserved in the form of a TMA. Various studies confirm that 2-3 sampled cores are adequate to represent the whole section tumor heterogeneity. In this subset of CAB test TMA validation, the data prove that the CAB risk category did not get influenced by the number of cores sampled compared to the whole section, possibly due to appropriate targeted tumor sampling [21,22]. IHC marker validation using TMA is more efficient. It avoids using whole sections in terms of reagent consumption, tissue resource utilization and has better utilization of skilled technician’s time involved in the process of IHC staining; therefore use of TMA for the clinical and analytical validation of IHC biomarkers is rapidly growing [10,23].
Conclusion
The study results showcase the equivalent performance of CAB by manual IHC & automated IHC (Ventana BenchMark XT) methods with respect to risk scores and risk categories. Further, the CAB test showed acceptable performance using TMA, which opens the door to using archived TMA samples from important completed trials to enhance the validation of CAB.
Acknowledgements
We thank Dr. Satish Sankaran and Dr. Aparna Gunda for their help with the critical reading of the manuscript. Also, we extend our thanks to hospital administrators and the team for the support of this study.
Disclosure of conflict of interest
The study was privately funded. The funding agency has no role in study design and execution. All authors are employees/consultants at OncoStem Diagnostics Private Limited. Authors have no other competing interests to declare.
References
- 1.Taylor CR, Burns J. The demonstration of plasma cells and other immunoglobulin-containing cells in formalin-fixed, paraffin-embedded tissues using peroxidase-labelled antibody. J Clin Pathol. 1974;27:14–20. doi: 10.1136/jcp.27.1.14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schacht V, Kern JS. Basics of immunohistochemistry. J Invest Dermatol. 2015;135:1–4. doi: 10.1038/jid.2014.541. [DOI] [PubMed] [Google Scholar]
- 3.Prichard JW. Overview of automated immunohistochemistry. Arch Pathol Lab Med. 2014;138:1578–1582. doi: 10.5858/arpa.2014-0083-RA. [DOI] [PubMed] [Google Scholar]
- 4.Goldstein NS, Hewitt SM, Taylor CR, Yaziji H, Hicks DG Members of Ad-Hoc Committee On Immunohistochemistry Standardization. Recommendations for improved standardization of immunohistochemistry. Appl Immunohistochem Mol Morphol. 2007;15:124–133. doi: 10.1097/PAI.0b013e31804c7283. [DOI] [PubMed] [Google Scholar]
- 5.Engel KB, Moore HM. Effects of pre-analytical variables on the detection of proteins by immunohistochemistry in formalin-fixed, paraffin-embedded tissue. Arch Pathol Lab Med. 2011;135:537–543. doi: 10.5858/2010-0702-RAIR.1. [DOI] [PubMed] [Google Scholar]
- 6.Khoury T, Sait S, Hwang H, Chandrasekhar R, Wilding G, Tan D, Kulkarni S. Delay to formalin fixation effect on breast biomarkers. Mod Pathol. 2009;22:1457–1467. doi: 10.1038/modpathol.2009.117. [DOI] [PubMed] [Google Scholar]
- 7.Anagnostou VK, Welsh AW, Giltnane JM, Siddiqui S, Liceaga C, Gustavson M, Syrigos KN, Reiter JL, Rimm DL. Analytic variability in immunohistochemistry biomarker studies. Cancer Epidemiol Biomarkers Prev. 2010;19:982–991. doi: 10.1158/1055-9965.EPI-10-0097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hewitt SM. Tissue microarrays as a tool in the discovery and validation of predictive biomarkers. Methods Mol Biol. 2012;823:201–14. doi: 10.1007/978-1-60327-216-2_13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Voduc D, Kenney C, Nielsen TO. Tissue microarrays in clinical oncology. Semin Radiat Oncol. 2008;18:89–97. doi: 10.1016/j.semradonc.2007.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Jawhar NM. Tissue microarray: a rapidly evolving diagnostic and research tool. Ann Saudi Med. 2009;29:123–127. doi: 10.4103/0256-4947.51806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Gustavson MD, Bourke-Martin B, Reilly D, Cregger M, Williams C, Mayotte J, Zerkowski M, Tedeschi G, Pinard R, Christiansen J. Standardization of HER2 immunohistochemistry in breast cancer by automated quantitative analysis. Arch Pathol Lab Med. 2009;133:1413–1419. doi: 10.5858/133.9.1413. [DOI] [PubMed] [Google Scholar]
- 12.Ramkumar C, Buturovic L, Malpani S, Kumar Attuluri A, Basavaraj C, Prakash C, Madhav L, Doval DC, Mehta A, Bakre MM. Development of a novel proteomic risk-classifier for prognostication of patients with early-stage hormone receptor-positive breast cancer. Biomark Insights. 2018;13:1–9. doi: 10.1177/1177271918789100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Bakre MM, Ramkumar C, Attuluri AK, Basavaraj C, Prakash C, Buturovic L, Madhav L, Naidu N, Somashekhar SP, Gupta S, Doval DC. Clinical validation of an immunohistochemistry-based CanAssist-Breast test for distant recurrence prediction in hormone receptor-positive breast cancer patients. Cancer Med. 2019;8:1755–1764. doi: 10.1002/cam4.2049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Attuluri AK, Serkad CP, Gunda A, Ramkumar C, Basavaraj C, Buturovic L, Madhav L, Naidu N, Krishnamurthy N, Prathima R, Kanaldekar S. Analytical validation of CanAssist-Breast: an immunohistochemistry based prognostic test for hormone receptor positive breast cancer patients. BMC Cancer. 2019;19:1–10. doi: 10.1186/s12885-019-5443-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Garrett PE, Lasky FD, Meier KL. User protocol for evaluation of qualitative test performance; approved guideline. CLSI; 2008. [Google Scholar]
- 16.Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6:284–290. [Google Scholar]
- 17.Bland JM, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1996;327:307–310. [PubMed] [Google Scholar]
- 18.Altman DG. Inter-rater agreement. Practical statistics for medical research. Stat Med. 1991;10:1635–1636. [Google Scholar]
- 19.Magaki S, Hojat SA, Wei B, So A, Yong WH. An introduction to the performance of immunohistochemistry. Methods Mol Biol. 2019;1897:289–298. doi: 10.1007/978-1-4939-8935-5_25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Pilar EF, Zettler CG. Standardization of concentrated antibodies for use in automated immunohistochemistry. J Histol Histopathol. 2020;7:4. [Google Scholar]
- 21.Pinder SE, Brown JP, Gillett C, Purdie CA, Speirs V, Thompson AM, Shaaban AM. The manufacture and assessment of tissue microarrays: suggestions and criteria for analysis, with breast cancer as an example. J Clin Pathol. 2013;66:169–177. doi: 10.1136/jclinpath-2012-201091. [DOI] [PubMed] [Google Scholar]
- 22.Eckel-Passow JE, Lohse CM, Sheinin Y, Crispen PL, Krco CJ, Kwon ED. Tissue microarrays: one size does not fit all. Diagn Pathol. 2010;5:1–10. doi: 10.1186/1746-1596-5-48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Wakeman K, Sonawane S, Al Rasheed MR, Setty S. Utility of tissue microarray for validation of immunohistochemistry: the price is right! Am J Clin Pathol. 2016;146:19. [Google Scholar]