Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Jul 9.
Published in final edited form as: Am J Clin Pathol. 2012 Jan;137(1):75–85. doi: 10.1309/AJCPMMLQ67YOMGEW

Automated analysis of multidimensional flow cytometry data improves diagnostic accuracy between mantle cell lymphoma and small lymphocytic lymphoma

Habil Zare 1,2,★, Ali Bashashati 3,★, Robert Kridel 4,5, Nima Aghaeepour 1, Gholamreza Haffari 6,3, Joseph M Connors 4,7, Randy D Gascoyne 4,5, Arvind Gupta 2, Ryan R Brinkman 1,8,*, Andrew P Weng 1,5,*
PMCID: PMC4090220  NIHMSID: NIHMS595110  PMID: 22180480

Abstract

Background

Mantle cell lymphoma (MCL) and small lymphocytic lymphoma (SLL) exhibit similar, but distinct immunophenotypic profiles. While many cases can be diagnosed with high confidence based on flow cytometry (FCM) results alone, ambiguous cases are frequently encountered and necessitate additional studies including immunohistochemistry for cyclinD1 and fluorescence in-situ hybridization (FISH) analysis for t(11;14) translocation.

Design and Methods

In order to determine if greater diagnostic accuracy could be achieved from flow cytometry data alone, we developed an unbiased, machine-based algorithm and used it to automatically identify those features within the multidimensional space that best distinguish between the two disease types.

Results

Data from 44 MCL cases and 70 SLL cases were analyzed. Using conventional diagnostic criteria, we were able to accurately assign only 64% of MCL and 69% of SLL cases. Using features identified by our automated approach, we were able to assign 100% of MCL and 97% of SLL cases correctly. The most discriminating feature was the ratio of mean fluorescence intensities (MFI) between CD20 and CD23. Unexpectedly, we also observed that inclusion of FMC7 expression in the diagnostic algorithm reduced its accuracy.

Conclusion

Computational methods allow objective assessment of the relative contribution of component data features to overall diagnostic accuracy, and reveal some conventional criteria can actually compromise this accuracy. Furthermore, computational approaches enable exploiting the full dimensionality of FCM data and can potentially lead to discovery of novel biomarkers relevant for clinical outcome.

Introduction

Mantle cell lymphoma (MCL) and small lymphocytic lymphoma (SLL) are both mature B-cell neoplasms [1–3]. MCL is characterized by a proliferation of monomorphous small to medium-sized B lymphocytes, with slightly irregular nuclear contours and typically presents with advanced stage lymphadenopathy, hepatosplenomegaly, and bone marrow involvement [4]. SLL on the other hand is composed mostly of small cells with round nuclei and clumped chromatin with an admixture of larger nucleolated forms called prolymphocytes and paraimmunoblasts. SLL represents the predominantly lymphomatous version of chronic lymphocytic leukemia (CLL), and the spectrum of SLL/CLL typically involves lymph nodes, spleen, liver, bone marrow, and peripheral blood [4–6].

Unlike SLL, which generally shows an indolent course justifying a watch and wait approach in asymptomatic patients [3], MCL is an aggressive lymphoma that is usually treated at diagnosis. Therefore, accurate distinction between these two diagnosis is crucial. The hallmarks of MCL are the t(11;14)(q13;q32) translocation, present in the vast majority of cases, and the resulting overexpression of cyclin D1 [7, 8]. While fluorescence in situ hybridization (FISH) and immunohistochemistry (IHC) are excellent ancillary tests for these features, performing and interpreting them requires resources which may not be available in all laboratory settings [9]. Flow cytometry is frequently utilized in evaluation of lymphoproliferative disorders, and is especially useful in the differential diagnosis between SLL and MCL since they usually exhibit distinct immunophenotypes [10, 11]. While both lymphomas are CD5+, MCL is generally CD23− and FMC7+, whereas SLL/CLL is usually CD23+ and FMC7−. However, a significant proportion of SLL and MCL (e.g., more than 15% [12]) present conflicting flow cytometry signatures and are prone to misclassification [13]. Several groups have attempted to address this challenge by closer analysis of flow cytometry data [10–12,14–25], but most resulting diagnostic algorithms compromise sensitivity for specificity or vise versa [13]. For instance, the approach suggested by Morice et al. [11] was reported to have 82% sensitivity to CLL/SLL and 56% sensitivity to MCL for 175 studied cases that were CD5+. An other example is Matutes score that can be computed based on monoclonal light chain immunoglobin, CD5, FMC7, CD23, and CD22 [26, 27]. This approach relies upon a subjective assessment of positive vs. negative and moderate/strong vs weak staining for each marker, however, and thus is highly sensitive to interobserver variation. Additional markers such as CD54 [27] and CD200 [28] have improved upon SLL/MCL discrimination, but their routine use at present is not widespread.

It is widely recognized that data analysis is by far one of the most challenging and time-consuming aspects of flow cytometry (FCM) experiments as well as being a primary source of variation in clinical tests [29–37]. Investigators have traditionally relied on intuition rather than on standardized statistical inference in the analysis of FCM data [38]. Our hypothesis is that the accuracy of diagnosis can be significantly improved by utilizing the information that already exists in FCM data, but is missed by traditional data analysis approaches. The goal of this study was to discover more sensitive and more specific FCM features to reduce diagnostic errors, time and effort required for data analysis, and unnecessary utilization of ancillary tests. Our approach was to use an unbiased algorithm to analyze retrospectively multidimensional FCM data in order to identify the most informative features. We report here that the CD20/CD23 ratio is the single most powerful FCM feature in discriminating between SLL and MCL, and can improve diagnostic accuracy over conventional approaches involving binary (i.e. positive vs negative) decision criteria applied to each marker individually. Additionally, surface immunoglobulin light chain (sIg) intensity and CD11c expression are useful in classifying cases with borderline CD20/CD23 ratios. Unexpectedly, we observed that while FMC7 expression generally correlates with MCL cases overall, it can actually confound accurate classification of cases with borderline CD20/CD23 ratios.

Design and Methods

Patient Samples

One hundred fourteen lymph node biopsy specimens with final pathologic diagnoses of either mantle cell lymphoma (MCL; 44 cases, 39%) or small lymphocytic lymphoma (SLL; 70 cases, 61%) and for which flow cytometric analysis was performed at the BC Cancer Agency between 1997 and 2010 were identified. Patient characteristics for SLL were 27% female/73% male with median age of 65 years. Patient characteristics for MCL were 32% female/68% male with median age of 65.5 years.

Pathologic Classification

Final pathologic diagnoses for all cases were determined by expert staff hematopathologists at the BC Cancer Agency after integration of findings from biopsy histology, immunohistochemistry/FISH analysis where indicated, flow cytometry, and clinical history. All cases with MCL diagnoses were confirmed positive for cyclin D1 by immunohistochemistry, and 9/10 with available FISH results were positive for IGH-CCND1 translocation. Cyclin D1 immunohistochemistry and/or IGH-CCND1 FISH analysis was also performed on 35/70 (50%) and 6/70 (9%) cases of SLL, respectively, all of which were negative.

Flow Cytometry Data Acquisition

A total of 88 cases were analyzed on a single laser, three color Beckman Coulter FC500 cy-tometer with surface immunophenotyping for CD19/CD3/CD5, CD19/CD23/FMC7, CD19/-kappa/lambda, CD20/CD10/CD11c, CD19/CD45/CD14, and CD7/CD4/CD8. The remaining 26 cases were analyzed on a three laser, eight color Becton Dickinson CantoII cytometer with surface immunophenotyping for CD19/CD20/CD3/CD5/CD10/CD11c/kappa/lambda, CD19/CD3/CD5/CD23/FMC7/CD38/CD25/CD103, and CD45/CD2/CD3/CD5/CD7/CD4/-CD8/CD56.

Photomultiplier (PMT) voltage settings were substantially modified twice during the 3-color data acquisition period and once during the 8-color data acquisition period as reported in [39]. Because changes in PMT voltage settings of the instrument alters the mean fluorescent intensity (MFI) of the corresponding markers and can mask biological information [39, 40], we segregated the data into five distinct time periods such that PMT voltage settings were essentially constant within each time period. The five time periods included 23, 21, 44, cases during 3-color era, and 19 and 7 cases during 8-color era.

Definitional Criteria for Positive/Negative Marker Expression

According to World Health Organization (WHO) immunophenotypic classification, typical MCL is CD5+, CD23−, FMC7+ whereas typical SLL is CD5+, CD23+, FMC7− [41]. In routine clinical practice, the distinction between “positive” vs “negative” expression for a given marker is often based on absolute threshold values as defined by either an internal negative control population within the same staining tube or parallel analysis of unstained patient cells, patient cells stained with isotype control antibodies, or staining of cells from a “normal” control sample. While this approach gives clear results when cell populations of interest exhibit uniform and bright expression of a given marker, it is much less informative when expression is variable or dim. In order to compare our automated approach to conventional practice, we applied the t-test as a statistical measure to define positive vs negative marker expression rather than subjective interpretation. Internal control populations were used to define “negative” expression for each marker. For example, CD19− cells were defined as the negative control population for CD11c, CD23, and FMC-7 markers, whereas normal CD19+ CD5− B cells served as negative control for CD5 expression on malignant B cells.

Computational Methodology for Data Processing

We previously developed SamSPECTRAL methodology (publicly available from Bioconductor, www.bioconductor.org/packages/devel/bioc/html/SamSPECTRAL.html) to cluster individual flow data points and thus define cell populations automatically [42]. As a preprocessing step to exclude dead cells/cell debris, data from each case was clustered in FSC and SSC log-scale dimensions using SamSPECTRAL with parameters σnormal = 2000 and separation factor = 0.8. Cell clusters with mean FSC less than 200 were considered as dead cells/cell debris and excluded from further analysis. Data were then clustered again with SamSPECTRAL (σnormal = 200 and separation factor = 0.9 for 3-color data; σnormal = 100 and separation factor = 0.8 for 8-color data) using all available fluorescence channels plus forward and side scatter parameters. The algorithm typically identified 3–8 clusters within each sample tube. Feature values for each identified cluster included the size of each cluster (as a fraction of total live events), MFIs in each fluorescence channel, and mean FSC and SSC values. Next, we applied our FeaLect methodology (publicly available from www.cran.rproject.org/web/packages/FeaLect/)), a novel feature selection technique similar to the Bolasso algorithm [43], to identify flow cytometric features which were most useful in discriminating between MCL and SLL diagnoses.

Setting of Thresholds

For each feature under study, we used the standard R density method, which uses a Gaussian kernel [44, 45], to estimate the densities of MCL vs SLL cases within each of the five time frames. We then computed the Bayes error [46,47] for 1000 points uniformly distributed in the range of values for each feature, and selected the optimal threshold which minimized the Bayes error (i.e. the probability of misclassification between MCL and SLL was minimized).

Online Supplementary Material

Supplementary materials for this paper can be found at: www.cs.ubc.ca/~zare/MCL_SLL.html that include a figure presenting an overview to our automatic methodology, a figure showing four examples of typical/atypical SLL and MCL cases, and a table of comparative discriminative values based on all 114 studies cases. Also, a computational package is provided to reproduce the data analyses performed in this study.

Results

Identifying Discriminative FCM Data Features

We applied our computational approach to the problem of SLL/MCL discrimination in two steps. First, we used the SamSPECTRAL clustering algorithm to identify cell populations, then we employed our FeaLect methodology to identify FCM data features of these populations which differentiated best between SLL and MCL.

Given that the 114 cases included in this study spanned five distinct time periods during which cytometer platform and/or voltage settings had changed significantly, we first analyzed data from the 3rd time period since it contained the largest number of cases at 44/114 (39%), and defined this set of cases as the “training” set.

When all 15 available markers were analyzed for their discriminative value, our unbiased, automated approach revealed that only a subset of markers was useful. Consistent with expectations from clinical practice, these included CD19, CD20, CD5, CD23, FMC7, CD11c, and surface Ig light chain. It was apparent from the data that the features selected by FeaLect were coming from B cell populations. Though this observation may seem somewhat trivial given that SLL and MCL are B cell lymphomas, the unbiased nature of the feature selection approach provided strong reassurance that there were indeed no subtle T cell phenotypes which could contribute to SLL/MCL discrimination.

In order to compare the performance of our selected features to conventional practice, we objectively identified B cell-containing populations by applying two consecutive t-tests. The first t-test was applied to each cluster with the null hypothesis that the CD19 MFI of the cluster was greater than the CD19 MFI of all live cells. Clusters that passed this test with p value of 0.001 were designated CD19+. As the rest of the events included both CD19dim and CD19− cells (in non-CD19+ clusters), we applied another t-test to identify dim from negative clusters, this time using the null hypothesis that the cluster expressed CD19 at higher levels than the rest of the “non-CD19+” events. Clusters that passed this test with p value of 0.001 were designated as CD19dim. All remaining clusters were designated as CD19−. Both CD19+ and CD19dim clusters were included, and CD19− clusters excluded from subsequent analyses. We then examined MFI values for each marker individually as well as all pairwise combinations of MFI ratios for their ability to discriminate between MCL and SLL. We observed CD23, FMC7, CD20, immunoglobulin light chain, and CD5 (typically regarded as most useful in MCL vs. SLL diagnosis [10, 13–24, 48–51]) to give variable results when considered individually (Figure 1). Interestingly, three pairwise ratios (CD20/CD23, FMC7/CD23, and CD20/CD11c) showed considerable improvement in discriminating between MCL and SLL (Figure 2).

Figure 1.

Figure 1

Open in a new tab

Discriminative value of individual markers typically utilized for diagnosis of MCL vs. SLL. “Typical” MCL cases were defined as having a CD5+, CD23−, and FMC7+ B cell signature, whereas “typical” SLL cases were defined as having a CD5+, CD23+, and FMC7− B cell signature. Any cases without a “typical” MCL or SLL immunophenotype were designated “atypical”. All MCL cases were confirmed as cyclin D1 positive by IHC. Cases left/right of gray lines are confirmed SLL/MCL cases, accordingly, and the green cutoffs show the Bayes decision boundary (see Methods). The blue/pink curves are estimated densities for SLL/MCL, accordingly. For clarity purposes, only cases from the third time frame (14 MCL shown in pink, 30 SLL shown in blue) are depicted.

Figure 2.

Figure 2

Open in a new tab

Most discriminative fluorescence ratios for MCL vs. SLL. Symbols are as depicted in Figure 1.

Developing a Diagnostic Predictor

We next sought to combine these three discriminative ratios (CD20/CD23, FMC7/CD23, and CD20/CD11c) into a composite diagnostic predictor, which we have termed the “Combined Ratio Score” or CRS. The CRS is derived for each case by counting the number of ratios which are above an empirically defined threshold (see Methods). For example, if all three ratios are above their corresponding thresholds, then the CRS will have a value of 3, which represents the strongest prediction for MCL. Conversely, if all ratios are below their corresponding thresholds, then the CRS will have a value of 0, which represents the strongest prediction for SLL.

Since the CRS was initially defined using the 44-sample training set, we tested its performance on an independent, or “validation” set of 70 samples comprising the remaining cases identified in this study. The performance of the CRS on this 70-sample test set is summarized in Table 1, with comparison to each of the component pairwise ratios as well as to selected individual markers. Although the CRS achieved 100% sensitivity, 92% specificity, and 96% accuracy in diagnosing MCL in this 70 sample validation set, we noted rather unexpectedly that the CRS performed less reliably than did the best pairwise feature, the CD20/CD23 ratio, alone. In examining further the underlying cause for poorer performance of the CRS, it became apparent that FMC7 was the primary confounding variable in that it was expressed at relatively high levels (and scored as positive by t-test comparison) in confirmed SLL cases with borderline high CD20/CD23 ratios (Figure 4). In contrast, sIg light chain intensity appeared to improve upon the discriminative value of the CD20/CD23 ratio in that most cases with borderline ratios would have been assigned correctly if light chain intensity were considered (i.e. dim light chains favoring SLL). The incremental value of considering CD11c expression with CD20/CD23 ratio appeared variable in that some cases would have been assigned correctly, but others would not.

Table 1. Discriminative values on test set.

MCL is considered as positive and SLL as negative.

TPa TNb FPc FNd Sensitivitye Specificityf Accuracyg
graphic file with name nihms595110t1.jpg CD20/CD23 ratio 30 40 0 0 100%h 100% 100%
FMC7/CD23 ratio 30 37 3 0 100% 92% 96%
CD20/CD11c ratio 24 33 7 6 80% 82% 81%
Combined Ratio Score 30 37 3 0 100% 92% 96%
graphic file with name nihms595110t2.jpg CD23 intensity 30 35 5 0 100% 88% 93%
FMC7 intensity 24 32 8 6 80% 80% 80%
CD20 intensity 24 33 7 6 80% 82% 81%
Light chain 18 38 2 12 60% 95% 80%
CD5 intensity 19 22 18 11 63% 55% 59%
Open in a new tab
a

TP (true positive) = the number of MCL cases which are diagnosed correctly

b

TN (true negative) = the number of SLL cases which are diagnosed correctly

c

FP (false positive) = the number of SLL cases which are diagnosed as MCL wrongly

d

FN (false negative) = the number of MCL cases which are diagnosed as SLL wrongly

e

Sensitivity to MCL is the portion of MCL cases that are correctly diagnosed by a feature (TP/(TP+FN)).

f

Specificity is the portion of SLL cases that are diagnosed correctly (TN/(TN+FP)).

g

Accuracy is the portion of all correctly diagnosed cases.

h

Performance higher than 95% is shown in boldface.

Figure 4.

Figure 4

Open in a new tab

Incorporating CD20/CD23 ratio with a third marker to diagnose borderline cases. Each gray line shows the optimum threshold which provides best discrimination between MCL and SLL, solely based on the third marker. The dashed lines determine borderline cases, i.e., the probability of observing a MCL case with CD20/CD23 ratio smaller than the pink threshold is less than 0.01. (a) FMC7 can be misleading for SLL cases with borderline high CD20/CD23 ratios. (b) Immunoglobulin light chain expression may improve diagnostic accuracy. (c) CD11c gives variable results.

Despite these potential limitations of the CRS, it is worth noting that the superior performance of the CD20/CD23 ratio critically depends upon optimized setting of the threshold value between MCL and SLL. In fact, while cases with CD20/CD23 ratios distant from the threshold value may be regarded with confidence, cases with CD20/CD23 ratios close to the borderline are less clear. It is in this context that perhaps the CRS holds value. In fact, when the CRS is applied to the entire dataset of 114 cases, 36/44 MCL cases (82%) received a score of 3, whereas 67/70 (96%) of SLL cases received a score of either 0 or 1. Only 11/114 cases (10%) received an ambiguous score of 2, and included the remaining 8/44 (18%) MCL cases and 3/70 (4%) SLL cases (Figure 3). In comparison, using conventional criteria of FMC7+/CD23− for MCL and FMC7−/CD23+ for SLL, only 76/114 (67%) of cases were correctly assigned, leaving 38/114 (33%) cases with ambiguous FMC7− CD23− or FMC7+ CD23+ phenotypes (16 MCL, 22 SLL). While most experienced flow cytometrists would incorporate other features such as CD20 and sIg intensity into their diagnostic assessment (albeit in a highly subjective manner), in practice these ambiguous FMC7− CD23− and FMC7+ CD23+ phenotypes often elicit ordering of ancillary IHC and/or FISH tests. Thus, utilization of the CRS in clinical practice may help to reduce the number of additional tests required to establish the diagnosis.

Figure 3.

Figure 3

Open in a new tab

Combined Ratio Scores obtained for 114 (44 MCL and 70 SLL) cases. The calculated combined scores are 2 or 3 for all MCL cases leading to 100% sensitivity, whereas, the scores of most (96%) SLL cases are 0 or 1.

Sensitivity to Thresholds

A common approach to assess the value and robustness of a diagnostic test is through receiver operating characteristic (ROC) analysis which is performed by plotting the true positive rate (TPR, or sensitivity) vs. false positive rate (FPR, or 1-specificity) while varying the test threshold from the minimum observed value to the maximum observed value [52]. The area under the ROC curve is a measure of overall test performance and if a curve for a given test lies above that for an alternate test, the former is considered to be more robust. We thus performed ROC analysis to compare the performance of the CRS against each of the component pairwise ratios, as well as select individual markers (Figure 5). ROC analysis confirmed that the CD20/CD23 ratio was indeed the most robust FCM data feature in discrimination between MCL and SLL.

Figure 5.

Figure 5

Open in a new tab

ROC curves obtained for all 114 (44 MCL and 70 SLL) cases. CD20 to CD23 ratio (brown) is the most accurate (100%) and most robust (AUC=1) FCM feature.

Discussion

Using an automated, unbiased approach to examine multidimensional flow cytometry data, we have attempted in this study to improve upon diagnostic accuracy in distinguishing between two immunophenotypically related lymphomas, SLL and MCL. The use of CD19 or CD20, CD5, CD23, and FMC7 as “positive” vs. “negative” markers with consideration of features such as intensity of CD20 and surface immunoglobulin (sIg) light chain expression is widespread [22, 53]. However, some cases cannot be confidently diagnosed by applying these conventional approaches to flow cytometry data analysis. Our automated algorithm has identified that the CD20/CD23 ratio is the most robust FCM feature for discriminating SLL from MCL. Unexpectedly, inclusion of additional features such as FMC7 and CD11c expression actually results in a greater likelihood of misdiagnosis, most specifically in SLL cases with borderline high CD20/CD23 ratios. In contrast, consideration of immunoglobulin light chain expression may improve diagnostic accuracy for these borderline cases. Our findings provide new insight into the relative contribution of each FCM data feature to the overall diagnostic algorithm, and by improving the diagnostic accuracy of FCM data analysis, can potentially reduce the amount of confirmatory ancillary testing required.

Because most clinical flow cytometry labs already routinely acquire both CD20 and CD23 expression data in their standard diagnostic panels for assessment of lymphoproliferative disorders, calculation of the CD20/CD23 ratio should be very easy to implement. The optimal cutoff value for CD20/CD23 ratio to discriminate between SLL and MCL, however, will be sensitive to interlaboratory variables such as staining protocols, choice of antibody clones and fluorochrome conjugates, and instrumentation settings and sensitivity. Therefore, our method currently may require an initial calibration step by each lab using its own set of training samples to determine the optimal cutoff.

While some studies have shown that the biological information of FMC7 can also be captured by CD20 intensity [54–56], it remains common practice to consider both FMC7 expression and CD20 intensity in evaluating FCM data. Interestingly, analysis of the discriminative value of each marker individually suggested FMC7 is superior to CD20 (Figure 1); however, when taken in the context of the CD20/CD23 ratio, FMC7 actually compromised diagnostic accuracy. This apparent contradiction may be due to the fact that while most MCL/SLL cases tend to be positive/negative for FMC7, respectively, this correlation appears to be less meaningful in the particular context of cases with borderline phenotypes.

The current study was limited to examination of FCM data from lymph node samples to avoid other potential variables from confounding this initial analysis. Further studies are warranted to determine if our approach yields similar diagnostic accuracy in peripheral blood and bone marrow samples. Also, as CD5+ marginal zone lymphoma (MZL) and CD5+ diffuse large B-cell lymphoma (DLBCL) are very rare and our approach requires sufficient numbers of cases to perform the automated clustering/feature discovery and statistical analyses, we were not able to include these lymphoma types in the current study.

Existing high dimensional FCM data may potentially contain valuable biological information that is hidden from conventional analyses because such approaches rely on bivariate plots and manual gating. Our identification of CD20/CD23 ratio is an example of revealing novel features that capture inapparent, but meaningful diagnostic information that already resides within existing FCM data. While our goal for the current study was to develop and test the automated algorithm for improving diagnostic accuracy, this approach is capable of identifying novel multidimensional features that could provide valuable prognostic information, or aid in recognition and defining of novel biologic subtypes that are currently subsumed under a single diagnostic heading. Unsupervised clustering of lymphoma samples facilitated by our unbiased automatic approach will be a focus of future work and can potentially lead to discovery of such novel subtypes.

Acknowledgments

We would like to acknowledge the Hematopathology staff at BCCA (Drs. D. Banerjee, M. Chhanabhai, M. Hayes, A. Karsan, B. Skinnider, and G. Slack) for expert pathologic diagnoses, and the BCCA Clinical Flow Cytometry Laboratory staff for their excellent technical work. This study was supported by funding from NSERC, MITACS, Canadian Cancer Society grant 700374, NIH/NIBIB grant EB008400, CIHR grant 94132, the Terry Fox Foundation, and the Terry Fox Research Institute. BCCA received an unrestricted grant from F Hoffmann-LaRoche that was used to support research on the integration of PET scanning into lymphoma management. RBB and APW are MSFHR Scholars.

References

  • 1.Chan W, Armitage J, Gascoyne R, Connors J, Close P, Jacobs P, et al. A clinical evaluation of the International Lymphoma Study Group classification of non-Hodgkins lymphoma. Blood. 1997;89(11):3909–3918. [PubMed] [Google Scholar]
  • 2.Hallek M, Cheson BD, Catovsky D, Caligaris-Cappio F, Dighiero G, Dohner H, et al. Guidelines for the diagnosis and treatment of chronic lymphocytic leukemia: a report from the International Workshop on Chronic Lymphocytic Leukemia updating the National Cancer Institute-Working Group 1996 guidelines. Blood. 2008;111(12):5446. doi: 10.1182/blood-2007-06-093906. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Eichhorst B, Hallek M, Dreyling M. Chronic lymphocytic leukemia: ESMO Clinical Recommendations for diagnosis, treatment and follow-up. Ann Oncol. 2008;19(suppl 2):60–62. doi: 10.1093/annonc/mdn090. [DOI] [PubMed] [Google Scholar]
  • 4.Mason D, Harris N, Delsol G, et al. WHO Classification of Tumours of Hematopoietic and Lymphoid Tissues. International Agency for Research on Cancer. 2008;(1):317–319. [Google Scholar]
  • 5.Autio K, Aalto Y, Franssila K, Elonen E, Joensuu H, Knuutila S. Low number of DNA copy number changes in small lymphocytic lymphoma. Haematologica. 1998;83(8):690. [PubMed] [Google Scholar]
  • 6.Gascoyne RD. The Science and Value of Lymphoma Classification. Orbital diseases present status and future challenges. 2005;12:65. [Google Scholar]
  • 7.Miranda RN, Briggs RC, Kinney MC, Veno PA, Hammer RD, Cousar JB. Immunohistochemical Detection of Cyclin D1 Using Optimized Conditions Is Highly Specific for Mantle Cell Lymphoma and Hairy Cell Leukemia. Modern Pathology. 2000;13(12):1308–1314. doi: 10.1038/modpathol.3880239. [DOI] [PubMed] [Google Scholar]
  • 8.Yatabe Y, Suzuki R, Tobinai K, Matsuno Y, Ichinohasama R, Okamoto M, et al. Significance of cyclin D1 overexpression for the diagnosis of mantle cell lymphoma: a clinicopathologic comparison of cyclin D1-positive MCL and cyclin D1-negative MCL-like B-cell lymphoma. Blood. 2000;95(7):2253. [PubMed] [Google Scholar]
  • 9.Schlette E, Fu K, Medeiros L. CD23 expression in mantle cell lymphoma: clinicopathologic features of 18 cases. American Journal of Clinical Pathology. 2003;120(5):760–6. doi: 10.1309/XV4A-G7EM-WQU7-ER67. [DOI] [PubMed] [Google Scholar]
  • 10.Schlette E, Fu K, Medeiros LJ. CD23 expression in mantle cell lymphoma: clinicopathologic features of 18 cases. American journal of clinical pathology. 2003;120(5):760. doi: 10.1309/XV4A-G7EM-WQU7-ER67. [DOI] [PubMed] [Google Scholar]
  • 11.Morice WG, Kurtin PJ, Hodnefield JM, Shanafelt TD, Hoyer JD, Remstein ED, et al. Predictive value of blood and bone marrow flow cytometry in B-cell lymphoma classification: comparative analysis of flow cytometry and tissue biopsy in 252 patients. Mayo Clinic Proceedings. 2008;83(7):776. doi: 10.4065/83.7.776. [DOI] [PubMed] [Google Scholar]
  • 12.Costa E, Pedreira C, Barrena S, Lecrevisse Q, Flores J, Quijano S, et al. Automated pattern-guided principal component analysis vs expert-based immunophenotypic classification of B-cell chronic lymphoproliferative disorders: a step forward in the standardization of clinical immunophenotyping. Leukemia. 2011;25(2):385–385. doi: 10.1038/leu.2010.160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Craig FE, Foon KA. Flow cytometric immunophenotyping for hematologic neoplasms. Blood. 2008;111(8):3941. doi: 10.1182/blood-2007-11-120535. [DOI] [PubMed] [Google Scholar]
  • 14.DiRaimondo F, Albitar M, Huh Y, O’Brien S, Montillo M, Tedeschi A, et al. The clinical and diagnostic relevance of CD23 expression in the chronic lymphoproliferative disease. Cancer. 2002;94(6):1721–1730. doi: 10.1002/cncr.10401. [DOI] [PubMed] [Google Scholar]
  • 15.Geisler CH, Larsen JK, Hansen NE, Hansen MM, Christensen BE, Lund B, et al. Prognostic importance of flow cytometric immunophenotyping of 540 consecutive patients with B-cell chronic lymphocytic leukemia. Blood. 1991;78(7):1795. [PubMed] [Google Scholar]
  • 16.Matutes E, Owusu-Ankomah K, Morilla R, Garcia MJ, Houlihan A, Que T, et al. The immunological profile of B-cell disorders and proposal of a scoring system for the diagnosis of CLL. Leukemia official journal of the Leukemia Society of America. 1994;8(10):1640. [PubMed] [Google Scholar]
  • 17.Ahmad E, Garcia D, Davis BH. Clinical utility of CD23 and FMC7 antigen coexistent expression in B-cell lymphoproliferative disorder subclassification. Cytometry. 2002;50(1):1–7. [PubMed] [Google Scholar]
  • 18.Garcia DP, Rooney MT, Ahmad E, Davis BH. Diagnostic usefulness of CD23 and FMC7 antigen expression patterns in B-cell lymphoma classification. American journal of clinical pathology. 2001;115(2):258. doi: 10.1309/VWTK-XYT5-D0DK-06HQ. [DOI] [PubMed] [Google Scholar]
  • 19.Molica S, Mannella A, Crispino G, Dattilo A, Levato D. Comparative flow cytometric evaluation of bcl-2 oncoprotein in CD5+ and CD5-B-cell lymphoid chronic leukemias. Haematologica. 1997;82(5):555. [PubMed] [Google Scholar]
  • 20.Hubmann R, Schwarzmeier JD, Shehata M, Hilgarth M, Duechler M, Dettke M, et al. Notch2 is involved in the overexpression of CD23 in B-cell chronic lymphocytic leukemia. Blood. 2002;99(10):3742. doi: 10.1182/blood.v99.10.3742. [DOI] [PubMed] [Google Scholar]
  • 21.Ocqueteau M, San Miguel JF, Gonzalez M, Almeida J, Orfao A. Do myelomatous plasma cells really express surface immunoglobulins? Haematologica. 1996;81(5):460. [PubMed] [Google Scholar]
  • 22.Tefferi A, Bartholmai BJ, Witzig TE, Li CY, Hanson CA, Phyliky RL. Heterogeneity and clinical relevance of the intensity of CD20 and immunoglobulin light-chain expression in B-cell chronic lymphocytic leukemia. American journal of clinical pathology. 1996;106(4):457–461. doi: 10.1093/ajcp/106.4.457. [DOI] [PubMed] [Google Scholar]
  • 23.Molot R, Meeker T, Wittwer C, Perkins S, Segal G, Masih A, et al. Antigen expression and polymerase chain reaction amplification of mantle cell lymphomas. Blood. 1994;83(6):1626. [PubMed] [Google Scholar]
  • 24.Delgado J, Matutes E, Morilla AM, Morilla RM, Owusu-Ankomah KA, Rafiq-Mohammed F, et al. Diagnostic significance of CD20 and FMC7 expression in B-cell disorders. American journal of clinical pathology. 2003;120(5):754. doi: 10.1309/FNGC-YEMJ-E3MA-E5L2. [DOI] [PubMed] [Google Scholar]
  • 25.Molica S, Dattilo A, Mannella A, Levato D. CD11c expression in B-cell chronic lymphocytic leukemia. A comparison of results obtained with different monoclonal antibodies. Haematologica. 1994;79(5):452. [PubMed] [Google Scholar]
  • 26.Matutes E, Owusu-Ankomah K, Morilla R, Garcia MJ, Houlihan A, Que T, et al. The immunological profile of B-cell disorders and proposal of a scoring system for the diagnosis of CLL. Leukemia: official journal of the Leukemia Society of America, Leukemia Research Fund, UK. 1994;8(10):1640. [PubMed] [Google Scholar]
  • 27.Deneys V, Michaux L, Leveugle P, Mazzon A, Gillis E, Ferrant A, et al. Atypical lymphocytic leukemia and mantle cell lymphoma immunologically very close: flow cytometric distinction by the use of CD20 and CD54 expression. Leukemia. 2001;15(9):1458–1465. doi: 10.1038/sj.leu.2402200. [DOI] [PubMed] [Google Scholar]
  • 28.Palumbo GA, Parrinello N, Fargione G, Cardillo K, Chiarenza A, Berretta S, et al. CD200 expression may help in differential diagnosis between mantle cell lymphoma and B-cell chronic lymphocytic leukemia. Leukemia research. 2009;33(9):1212–1216. doi: 10.1016/j.leukres.2009.01.017. [DOI] [PubMed] [Google Scholar]
  • 29.Krutzik PO, Irish JM, Nolan GP, Perez OD. Analysis of protein phosphorylation and cellular signaling events by flow cytometry: techniques and clinical applications. Clinical Immunology. 2004;110(3):206–221. doi: 10.1016/j.clim.2003.11.009. [DOI] [PubMed] [Google Scholar]
  • 30.Lazar T. Manual of Clinical Laboratory Immunology. Laboratory Hematology. 2003;9(1):52–52. [Google Scholar]
  • 31.Roederer M, Hardy RR. Frequency difference gating: a multivariate method for identifying subsets that differ between samples. Cytometry. 2001;45(1):56–64. doi: 10.1002/1097-0320(20010901)45:1<56::aid-cyto1144>3.0.co;2-9. [DOI] [PubMed] [Google Scholar]
  • 32.Suni MA, Dunn HS, Orr PL, Laat R, Sinclair E, Ghanekar SA, et al. Performance of plate-based cytokine flow cytometry with automated data analysis. BMC immunology. 2003;4(1):9. doi: 10.1186/1471-2172-4-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Herzenberg LA, Parks D, Sahaf B, Perez O, Roederer M, Herzenberg LA. The history and future of the fluorescence activated cell sorter and flow cytometry: a view from Stanford. Clinical chemistry. 2002;48(10):1819. [PubMed] [Google Scholar]
  • 34.Overton WR. Modified histogram subtraction technique for analysis of flow cytometry data. Cytometry. 1988;9(6):619–626. doi: 10.1002/cyto.990090617. [DOI] [PubMed] [Google Scholar]
  • 35.Aghaeepour N, Nikolic R, Hoos HH, Brinkman RR. Rapid cell population identification in flow cytometry data. Cytometry Part A. 2011;79(1):6–13. doi: 10.1002/cyto.a.21007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Roederer M. Spectral compensation for flow cytometry: visualization artifacts, limitations, and caveats. Cytometry. 2001;45(3):194–205. doi: 10.1002/1097-0320(20011101)45:3<194::aid-cyto1163>3.0.co;2-c. [DOI] [PubMed] [Google Scholar]
  • 37.Perfetto SP, Chattopadhyay PK, Roederer M. Seventeen-colour flow cytometry: unravelling the immune system. Nature Reviews Immunology. 2004;4(8):648–655. doi: 10.1038/nri1416. [DOI] [PubMed] [Google Scholar]
  • 38.Bashashati A, Brinkman R. A survey of flow cytometry data analysis methods. Advances in Bioinformatics. 2009:1–19. doi: 10.1155/2009/584603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Johnson NA, Boyle M, Bashashati A, Leach S, Brooks-Wilson A, Sehn LH, et al. Diffuse large B-cell lymphoma: reduced CD20 expression is associated with an inferior survival. Blood. 2009;113(16):3773. doi: 10.1182/blood-2008-09-177469. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Lenkei R, Gratama J, Rothe G, Schmitz G, D’hautcourt J, Årekrans A, et al. Performance of calibration standards for antigen quantitation with flow cytometry. Cytometry. 1998;33(2):188–196. doi: 10.1002/(sici)1097-0320(19981001)33:2<188::aid-cyto13>3.0.co;2-q. [DOI] [PubMed] [Google Scholar]
  • 41.Criel A, Verhoef G, Vlietinck R, Mecucci C, Billiet J, Michaux L, et al. Further characterization of morphologically defined typical and atypical CLL: a clinical, immunophenotypic, cytogenetic and prognostic study on 390 cases. British Journal of Haematology. 1997;97(2):383–391. doi: 10.1046/j.1365-2141.1997.402686.x. [DOI] [PubMed] [Google Scholar]
  • 42.Zare H, Shooshtari P, Gupta A, Brinkman R. Data reduction for spectral clustering to analyze high throughput flow cytometry data. BMC Bioinformatics. 2010;11(1):403. doi: 10.1186/1471-2105-11-403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Bach F. Model-consistent sparse estimation through the bootstrap. 2009 HAL-00354771. [Google Scholar]
  • 44.Sheather SJ, Jones MC. A reliable data-based bandwidth selection method for kernel density estimation. Journal of the Royal Statistical Society Series B (Methodological) 1991;53(3):683–690. [Google Scholar]
  • 45.Scott DW. Multivariate density estimation: theory, practice, and visualization. Wiley-Interscience; 1992. [Google Scholar]
  • 46.Toussaint G. Bibliography on estimation of misclassification. Information Theory, IEEE Transactions on. 2002;20(4):472–479. [Google Scholar]
  • 47.Brun M, Sabbagh D, Kim S, Dougherty ER. Corrected small-sample estimation of the Bayes error. Bioinformatics. 2003;19(8):944. doi: 10.1093/bioinformatics/btg144. [DOI] [PubMed] [Google Scholar]
  • 48.Asplund SL, McKenna RW, Doolittle JE, Kroft SH. CD5-positive B-cell neoplasms of indeterminate immunophenotype: a clinicopathologic analysis of 26 cases. Applied Immunohistochemistry & Molecular Morphology. 2005;13(4):311. doi: 10.1097/01.pai.0000137363.36091.7e. [DOI] [PubMed] [Google Scholar]
  • 49.Ginaldi L, De Martinis M, Matutes E, Farahat N, Morilla R, Catovsky D. Levels of expression of CD19 and CD20 in chronic B cell leukaemias. Journal of clinical pathology. 1998;51(5):364. doi: 10.1136/jcp.51.5.364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Wang L, Abbasi F, Gaigalas AK, Vogt RF, Marti GE. Comparison of fluorescein and phycoerythrin conjugates for quantifying CD20 expression on normal and leukemic B-cells. Cytometry Part B: Clinical Cytometry. 2006;70(6):410–415. doi: 10.1002/cyto.b.20140. [DOI] [PubMed] [Google Scholar]
  • 51.Meyerson HJ, MacLennan G, Husel W, Tse W, Lazarus HM, Kaplan D. D cyclins in CD5+ B-cell lymphoproliferative disorders. American Journal of Clinical Pathology. 2006;125(2):241. doi: 10.1309/7C2V-V961-P60R-MLHD. [DOI] [PubMed] [Google Scholar]
  • 52.Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry. 1993;39(4):561–577. [PubMed] [Google Scholar]
  • 53.Marti G, Faguet G, Bertin P, Agee J, Washington G, Ruiz S, et al. CD20 and CD5 Expression in B-Chronic Lymphocytic Leukemia. Annals of the New York Academy of Sciences. 1992;651(1):480–483. doi: 10.1111/j.1749-6632.1992.tb24651.x. [DOI] [PubMed] [Google Scholar]
  • 54.Deans JP, Polyak MJ. FMC7 is an epitope of CD20. Blood. 2008;111(4):2492. doi: 10.1182/blood-2007-11-126243. [DOI] [PubMed] [Google Scholar]
  • 55.Hubl W, Iturraspe J, Braylan RC. FMC7 antigen expression on normal and malignant B-cells can be predicted by expression of CD20. Cytometry. 1998;34(2):71–74. doi: 10.1002/(sici)1097-0320(19980415)34:2<71::aid-cyto2>3.0.co;2-f. [DOI] [PubMed] [Google Scholar]
  • 56.Lyons AB, Parish CR. Determination of lymphocyte division by flow cytometry. Journal of immunological methods. 1994;171(1):131–137. doi: 10.1016/0022-1759(94)90236-4. [DOI] [PubMed] [Google Scholar]

RESOURCES