Abstract
Five laboratories in the Chronic Lymphocytic Leukemia (CLL) Research Consortium (CRC) investigated standardizing and pooling of fluorescence in situ hybridization (FISH) results as a collaborative research project. This investigation used fixed bone marrow and blood cells available from previous conventional cytogenetic or FISH studies in two pilot studies, a one-day workshop, and proficiency test. Multiple FISH probe strategies were used to detect 6q-, 11q-, +12, 13q-, 17p-, and IGH rearrangements. Ten specimens were studied by participants who used their own probes (pilot study 1). Of 312 FISH interpretations, 224 (72%) were true-negative, 74 (24%) true-positive, 6 (2%) false-negative, and 8 (3%) false-positive. In pilot study no. 2, each participant studied two specimens using identical FISH probe sets to control for variation due to probe sets and probe strategies. Of 80 FISH interpretations, no false interpretations were identified. At a subsequent workshop, discussions produced agreement on scoring criteria. The proficiency test that followed produced no false-negative results and 4% (3/68) false-positive interpretations. Interpretation disagreements among laboratories were primarily attributable to inadequate normal cutoffs, inconsistent scoring criteria, and the use of different FISH probe strategies. Collaborative organizations that use pooled FISH results may wish to impose more conservative empiric normal cutoff values or use an equivocal range between the normal cutoff and the abnormal reference range to eliminate false-positive interpretations. False-negative results will still occur, and would be expected in low-percentage positive cases; these would likely have less clinical significance than false positive results. Individual laboratories can help by closely following rigorous quality assurance guidelines to ensure accurate and consistent FISH studies in their clinical practice and research.
1. Introduction
Studies of interphase nuclei using fluorescence in situ hybridization (FISH) are an essential part of the clinical evaluation of patients with B-cell chronic lymphocytic leukemia (CLL) [1–5]. FISH methods and DNA probes used to analyze cells from patients with CLL vary among cytogenetic laboratories. This is at least in part because national standards established for clinical studies generally are left to the discretion of the laboratory director — which FISH probes to use for CLL, definition of analytic details such as scoring criteria, and how to define the normal cutoff.
National guidelines to validate and use FISH assays in clinical practice have been published provided by the American College of Medical Genetics and the National Committee for Clinical Laboratory Standards [6,7]. However, not every laboratory follows these guidelines in the same way. FISH methods are accurate and reproducible when they are validated appropriately and continuous quality assurance procedures are used [8]. Multiple laboratories that work together to validate specific FISH probes can achieve excellent results following such guidelines [9–11].
The CLL Research Consortium (CRC) involves multiple institutions that work together to investigate the biology of CLL and develop treatments for CLL. The CRC FISH database currently includes results of more than 3,800 diagnostic (and many follow-up) FISH studies. Lack of FISH standardization can be problematic for cooperative groups when FISH data are pooled for clinical correlative studies. Differences among laboratories in validation procedures, FISH probes, scoring criteria, and statistical methods to define normal and abnormal results can be unintended sources of variation. This can complicate data analysis and reduce the validity of conclusions from correlative studies. To further investigate these important issues in a consortium dedicated to the study of CLL, five participating laboratories in the CRC designed and executed a joint FISH study to test for scoring variation and to identify common methods and scoring techniques that would ultimately generate more concordant FISH results.
2. Materials and methods
The selection of specimens, slide preparations, and data coding in this study were accomplished with approval of the Mayo Clinic Institutional Review Board, and informed consent was obtained in accordance with the Declaration of Helsinki. The FISH processing and analysis of coded slide preparations by each participant were performed with approval of the Institutional Review Board at each participating site.
Initially, a detailed survey questionnaire was sent to each laboratory to assess equipment, methods, and experience with FISH for CLL. Participants identified as A, B, C, D, and E listed features of their fluorescence microscopes, including filters, wattages, manufacturers, models, lenses, and digital capture systems. Each laboratory reported their clinical experience scoring FISH for CLL, including number of samples per year, types of samples (blood or bone marrow), FISH probes used, and time points of patient samples (diagnostic or follow-up). Slide preparation, pretreatment, washing techniques, and scoring practices were also compared among the participating sites.
FISH strategies used by participants in this investigation included enumeration, ND-FISH, and D-FISH (numeric and deletion FISH and double-fusion FISH, respectively) [12]. The enumeration probe strategy uses one probe per chromosome and is generally used to establish the number of chromosomes present in the interphase nucleus. The ND-FISH strategy to detect aneuploidy and chromosome deletions uses a probe of one color for a control site and a probe of another color for an interstitial target site on the same chromosome. The D-FISH strategy is used to detect a reciprocal translocation or inversion, using probes of different colors at the expected rearrangement breakpoints to produce two fused signals in the event of a rearrangement.
The FISH probes used by all five laboratories (Table 1) were designed by commercial or local institution laboratories to detect 11q, 13q, and 17p deletions, as well as trisomy 12 and translocations involving the IGH locus at 14q32.33. Participants A, B, C, and E used a commercially available set of locus-specific FISH probe sets that hybridize to loci on 11q and 17p together, and the chromosome 12 centromere region and 13q together [13]. Participants A, B, C, and E employed a D-FISH method to detect translocations involving the IGH locus at 14q32.33 (Table 1) [14]. Participant D used an ND-FISH strategy with five sets of probes that hybridize to 6q, 11q, +12, 13q, and 17p, and a D-FISH method to detect translocations involving the IGH locus [3,14–16]. For the proficiency study, four of the five laboratories used a FISH probe to detect 6q deletion (Abbott Molecular, Des Plaines, IL or home brew). Hereafter abnormalities in this report are referred to as 6q-, 11q-, +12, 13q-, 17p-, and t(14;?) or t(11;14). Heterozygous and homozygous loss of the D13S319 probe is referred to as 13q–x1 and 13q–x2, respectively. Gains of the MDM2 or LSI13q34 probe without gain of the chromosome 12 centromere or D13S319 (13q14), respectively, are referred to as +MDM2 and +LSI13q34, respectively.
Table 1.
FISH probes initially used by CRC participants in this investigationa
| Participant |
||
|---|---|---|
| FISH anomaly | A, B, C, Eb | Dc |
| 6q- | Not done | CEP6 (chromosome 6 centromere) cMYB (6q23) |
| 11q- | ATM (11q22) | CEP11 (chromosome 11 centromere) ATM (11q22) |
| +12 | CEP12 (chromosome 12 centromere) | CEP12 (chromosome 12 centromere) MDM2 (12q15) |
| 13q- | D13S319 (13q14.3) | D13S319 (13q14.3) |
| LSI13q34 (13q34) | LSI13q34 (13q34) | |
| 17p- | P53 (17p13) | CEP17 (chromosome 17 centromere) P53 (17p13) |
| t(14;?) or t(11;14)d | IGH-CCND1 (14q32 and 11q13) | IGH-CCND1 (14q32 and 11q13) |
All probes were made by Abbott Molecular (Des Plaines, IL) except for cMYB and MDM2, which were homebrew probes made at Mayo Clinic.
These probes are used in sets to simultaneously detect 11q- and 17p-; +12 and 13q-.
These probe sets were used in an ND-FISH strategy to detect 6q-, 11q-, +12, 13q- and 17p-.
A D-FISH method with CCND1 and IGH were used to detect translocations involving the IGH locus on chromosome 14.
The number of cells analyzed and the normal cutoff values varied among the participating laboratories (Table 2). In participant laboratory A, one technologist scored 100 nuclei per hybridization site, and in participant laboratories B, C, D, and E, two technologists each scored 100 nuclei per hybridization site. To set their normal cutoff for each FISH pattern, participant A used the maximum percentage of nuclei with each abnormal signal pattern from 10 normal control samples, and participant B used the standard deviation and standard errore—based evaluation of five normal controls. Participant C used the mean plus three standard deviations to calculate their normal cutoff, and based their normal range on more than 132 normal samples. Participants D and E used the Excel beta-inverse function that uses the binomial expansion formula to calculate the upper 95% confidence limit as the normal cutoff for each FISH pattern based on studies of 20 normal controls [8].
Table 2.
Normal cutoff values for CRC Laboratories
| Participant |
|||||||
|---|---|---|---|---|---|---|---|
| A | B | C | D | E | Min | Max | |
| Nuclei scored |
100 | 200 | 200 | 200 | 200 | 100 | 200 |
| 6q- | NA | NA | NA | 6.0 | NA | 6.0 | 6.0 |
| 11q- | 2.0 | 2.0 | 2.5 | 4.5 | 5.0 | 2.0 | 5.0 |
| +12 | 2.0 | 1.0 | 0.5 | 1.0 | 1.5 | 0.5 | 2.0 |
| +MDM2 | NA | NA | NA | 10.0 | NA | 10.0 | 10.0 |
| 13q–x1 | 2.0 | 2.0 | 6.0 | 6.5 | 7.0 | 2.0 | 7.0 |
| 13q–x2 | 2.0 | 2.0 | 2.5 | 1.0 | 1.5 | 1.0 | 2.5 |
| + LSI13q34 | NA | NA | NA | 10.0 | NA | 10.0 | 10.0 |
| 17p- | 5.0 | 5.0 | 6.0 | 8.0 | 5.0 | 5.0 | 8.0 |
| t(11;14) | 1.0 | 1.0 | 0.5 | 1.0 | 2.0 | 0.5 | 2.0 |
| t(14;?) | 2.0 | NA | NA | 8.0 | NA | 2.0 | 8.0 |
Abbreviations: CRC, CLL Research Consortium; Max, maximum; Min, minimum; NA, not available.
Normal cutoff values represent the upper limit of the normal range for each participant and are expressed as percentage of nuclei scored.
Residual fixed-cell bone marrow and blood cells from previous conventional cytogenetic or previous FISH studies were used to perform two pilot studies, a one-day workshop, and proficiency test. One specimen was from a normal bone marrow transplant donor and 11 were from CLL patients who were referred for clinical testing at diagnosis or follow-up, with chromosome abnormalities detected at Mayo Clinic by conventional cytogenetic studies, FISH analyses, or both.
Both pilot studies were performed between February 10, 2006 and May 8, 2007. Pilot study 1 was done to compare participant FISH probes and analytic performance to score patient samples. Microscope slides from 10 patients were prepared and coded for blinded studies at Mayo Clinic, and then sent to each participating laboratory to be hybridized with their clinical CLL FISH probes and analyzed according to their local protocol. At the Mayo Clinic, the pilot and proficiency samples were scored by technologists who had not prepared the coded slides.
In pilot study 2, the Mayo Clinic Cytogenetics Laboratory provided each laboratory with six separate probe sets that hybridized to target and control loci on 6q, 11q, +12, 13q, 17p, and IGH-CCND1 to detect IGH rearrangements (Table 1, participant D) [3,15]. The purpose of this experiment was to compare the CRC laboratories’ scoring while controlling for variation due to probe sets and probe strategies. To accomplish this experiment, coded slides were prepared from two pilot study no. 1 patient samples and sent to each participating laboratory. Each participant used their own FISH hybridization methods, but they used the same published analytic protocol for these FISH probes [3].
Next, technologists and directors from each participating laboratory attended a one-day workshop to discuss ways to improve the group’s performance with FISH studies for CLL. The first portion of the day was devoted to discussion and presentations regarding the results of the two pilot studies. Then the technologists worked in pairs at microscopes to directly compare their FISH scoring criteria. Finally, shared scoring criteria were established by consensus.
In a proficiency test 9 months after the workshop, coded slides from two new patient samples previously found to be abnormal by previous clinical FISH testing were provided for each laboratory. Participants were requested to use their own FISH probes and local protocols to analyze these slide preparations (Tables 1 and 2).
All results from the pilot studies and proficiency survey were submitted to SAS, DLVD, and GWD to summarize and evaluate. The summary results were then shared with the participant laboratory directors for review and comment.
3. Results
The survey results confirmed that each participating CRC cytogenetic laboratory performed FISH for CLL as a clinical service (approximately 80–2,000 specimens per year), participated in College of American Pathologists proficiency surveys, and met American College of Medical Genetics (ACMG) and National Committee for Clinical Laboratory Standards (NCCLS) guidelines in most respects. Each laboratory employed microscopes and other equipment known to be sufficient to carry out FISH studies. Microscope slide preparation, pretreatment, washing techniques, and scoring practices were similar among the participant laboratories.
3.1. Pilot study 1
This study was done to compare the CRC laboratories’ ability to detect 11q-, +12, 13q–x1, 13q–x2, t(11;14), t(14;?), and 17p-. Participant results for the 10 specimens in pilot study 1 for each laboratory were compared with those of other participants (Table 3) and with the original conventional cytogenetic and FISH studies. Participant D used 6 FISH probe sets to detect each of the chromosome abnormalities for a total of 60 FISH assays (including tests for 6q deletion). Participants A, B, and C used 5 FISH probes, for a total of 150 FISH assays. Participant E did not test for t(11;14) or t(14;?), and used 4 FISH-probe-sets to detect the other chromosome abnormalities, for a total of 40 FISH assays. Counting results for 13q–x1 and 13q–x2 separately, and not counting the 10 6q results from participant D, the participants performed 240 FISH assays that resulted in 290 FISH interpretations (Table 3).
Table 3.
Pilot study 1: percent of abnormal nuclei for each participant for 10 patients with CLL and a known karyotype or FISH patterna
| Percent 11q- nuclei |
Percent +12 nuclei |
Percent 13q–x1 nuclei |
||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | PB | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 1 | 81.0 | 85.0 | 71.0 | 93.0 | 83.0 | 1 | 12.0 |  |
9.0 | 8.0 | 12.0 |
| 2 | PB | 99.0 | 91.0 | 94.0 | 94.0 | 95.5 | 2 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 2 | 96.0 | 98.0 | 96.5 | 92.0 | 97.0 |
| 3 | BM |  |
0.0 | 0.0 | 0.5 | 0.5 | 3 | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 3 |  |
0.0 | 0.0 | 1.5 | 1.0 |
| 4 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 4 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 4 | 1.0 | 0.0 | 0.0 | 0.5 | 0.0 |
| 5 | PB | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 5 | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 5 | 62.0 | 67.0 | 70.0 | 76.0 | 74.5 |
| 6 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.5 | 6 | 84.0 | 68.5 | 78.0 | 87.5 | 73.5 | 6 | 1.0 | 0.0 | 0.0 | 2.0 | 0.0 |
| 7 | BM | 96.0 | 96.0 | 98.5 | 96.0 | 94.0 | 7 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 7 | 1.0 | 0.0 |  |
1.0 | 1.0 |
| 8 | BM |  |
 |
8.5 | 7.5 |  |
8 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 8 | 1.0 | 0.0 | 4.0 | 4.5 | 3.0 |
| 9 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 9 | 96.0 | 95.5 | 92.5 | 97.0 | 96.0 |
| 10 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 10 | 79.0 | 72.0 | 73.0 | 94.0 | 71.0 | 10 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Percent 13q–x2 nuclei |
Percent 17p- nuclei |
Percent t(14;?) nuclei |
||||||||||||||||
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | PB | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1 | 6.0 | 0.0 | 0.0 | 1.0 | 0.0 | 1 | 73.0 | 83.0 | 74.0 | 68.0 | ND |
| 2 | PB | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 2 | 0.0 | 0.0 | 0.0 | 0.5 | 1.0 | 2 | 0.0 |  |
0.0 | 0.0 | ND |
| 3 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 3 | 3.0 | 0.0 | 0.0 | 0.0 | 4.0 | 3 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
| 4 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 4 | 1.0 | 0.0 | 0.0 | 0.5 | 2.0 | 4 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
| 5 | PB | 35.5 | 30.0 | 28.5 | 21.5 | 22.0 | 5 |  |
0.0 | 0.0 | 1.0 | 3.0 | 5 |  |
 |
0.0 | 0.0 | ND |
| 6 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 6 | 1.0 | 0.0 | 0.0 | 1.5 | 2.5 | 6 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
| 7 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 7 |  |
0.0 | 0.0 | 0.5 | 0.0 | 7 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
| 8 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 8 | 19.0 |  |
17.0 |  |
10.0 | 8 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
| 9 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 9 | 71.0 | 59.5 | 48.5 | 64.0 | 42.0 | 9 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
| 10 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 10 | 96.0 | 94.5 | 94.0 | 96.0 | 97.0 | 10 | 0.0 | 0.0 | 0.0 | 0.0 | ND |
False positive
False negative
Abbreviations: BM, bone marrow; ND, not done; PB, peripheral blood; Pt, patient.
True positive and true negative results are defined as obtaining the same result as was obtained on the original conventional cytogenetic studies, previous FISH studies, or the consensus of four or five participant laboratories.
Participants A, B, C, D, and E used their own FISH probes and methods to obtain these results.1
Participant D found 6q- in 16% nuclei in patient 8 and +MDM2 in 16.5% of nuclei in patient 2. These findings required the ND FISH strategy, and only participant D studied pilot study 1 patients for these chromosome abnormalities. Participant E did not score for IGH rearrangements in pilot study 1.
For each participant, the FISH results of patient 4, the control subject, were normal. The consensus FISH results for patient 3 were normal and consistent with that patient’s sole abnormality of gain of MDM2 without gain of the chromosome 12 centromere. Each of the remaining specimens had one, two, or three FISH anomalies. Of 290 possible FISH interpretations, 208 (72%) were true-negative, 68 (23%) true-positive, 6 (2%) false-negative, and 8 (3%) false-positive.
Of 75 opportunities to identify an abnormal FISH pattern (15 abnormalities × 5 participants), there were 68 correct interpretations, 6 false negative results, and 1 not tested (Table 3). Each of the five participating laboratories detected the same FISH anomaly in 13 incidents of true-positive results. Each participant detected an 11q- in two specimens with a range of 91–99% of nuclei for patient 2 and 94–98.5% for patient 7. For patient 8, who had 12.5% 11q–in the clinical study, only participants C and D identified sufficient abnormal nuclei to exceed their normal cutoff. Each participant detected trisomy 12 in three specimens, with a range of 71–93% for patient 1, 68.5–87.5% for patient 6, and 71–94% for patient 10. Each participant detected a 13q–x1 in three specimens with a range of 92–98% for patient 2, 62–76% for patient 5, and 92.5–97% for patient 9. For patient 1, four participants identified a low percentage of 13qx1 nuclei, ranging from 8 to 12%, whereas participant B had a false-negative result. Each participant detected 13q–x2 in 21.5–35.5% of nuclei of patient 5. Each participant detected a 17p- in two bone marrow specimens with a range of 42–71% for patient 9 and 94–97% for patient 10. For patient 8, who had 13.5% 17p- nuclei in the clinical study, each participant identified some nuclei with 17p-, with a range of 5–17%, but the result was below the normal cutoff for participants B and D. All four participants that tested for IGH translocations in patient 1 observed a t(14;?) in 68–83% of nuclei.
In the first pilot study, each laboratory had one or more false-positive or false-negative findings: participant A had 6; B had 5; and C, D, and E each had 1. The 14 false interpretations were not related to any specific kind of chromosome abnormality. The six false-negative interpretations involved 11q- in 3, 13q–x1 in 1, and 17p- in 2 patients. The eight false-positive interpretations involved 11q- in 1, 13q–x1 in 2, 17p- in 2, and t(14;?) in 3 patients. These false interpretations were not related to specimen tissue type; each of the three blood specimens and three of the seven bone marrow specimens had one or more false-interpretations. Five of the six false-negative interpretations involved a bone marrow specimen from patient 8, which was likely related to the low percentage of abnormal cells with abnormalities of chromosome 11 and 17. The 11q FISH results for patient 8 were interpreted as abnormal in the original clinical study (12.5% 11q deletion) and by participants C and D, thus the interpretations of participants A, B, and E were regarded as false-negative.
3.2. Pilot study 2
In pilot study 2, Mayo Clinic provided each laboratory with coded microscope slides, FISH probe sets, and analytic procedures (Table 1, participant D). Patients 1 and 2 in pilot study 2 were the same as patients 3 and 10, respectively, in pilot study 1. Participant results for these were compared with those of other participants, with the original conventional cytogenetic and FISH studies, and with the results from pilot study 1. None of the 80 results led to a false-negative or false-positive interpretation (Table 4). The two false-negative interpretations provided by participant A in pilot study 1 were not repeated in pilot study 2. The results for patient 2 agreed with the interpretations for patient 10 in pilot study 1. The percentage of +MDM2 results for patient 1 were relatively low, ranging from 10 to 20%, yet all participants interpreted this region as a duplicated genetic defect.
Table 4.
Percent abnormal nuclei reported by each CRC participant for two patients with CLL and a known karyotype or FISH pattern in pilot study 2
| Percent 6q- nuclei |
Percent 11q- nuclei |
Percent +12 nuclei |
||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | BM | 2.0 | 0.0 | 0.0 | 1.0 | 2.0 | 1 | 0.0 | 0.0 | 0.0 | 0.5 | 0.0 | 1 | 0.0 | 0.5 | 0.0 | 0.0 | 0.0 |
| 2 | BM | 0.0 | 0.0 | 0.0 | 1.0 | 0.5 | 2 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 2 | 89.0 | 86.0 | 84.0 | 93.5 | 96.0 |
| Percent +MDM2 nuclei |
Percent 13q–x1 nuclei |
Percent 13q–x2 nuclei |
||||||||||||||||
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | BM | 18.0 | 15.5 | 10.0 | 16.5 | 20.0 | 1 | 0.0 | 0.0 | 0.0 | 1.5 | 0.5 | 1 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 2 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 2 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| Percent + LSI13q34 nuclei |
Percent 17p- nuclei |
Percent t(14;?) nuclei |
||||||||||||||||
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | BM | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 1 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| 2 | BM | 70.0 | 70.5 | 61.0 | 65.5 | 69.5 | 2 | 95.0 | 92.0 | 92.0 | 96.0 | 93.5 | 2 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
Abbreviations: BM, bone marrow; Pt, patient.
Participants used the same ND-FISH probes to obtain these results.
Patient 1 is the same as patient 3 in pilot study 1 (Table 3); patient 2 is the same as patient 10 in pilot study1 (Table 3).
3.3. One-day workshop
Discussions at the one-day workshop led to agreement on a consistent set of scoring criteria including the following: (1) score only nonoverlapping cells with intact borders; (2) two signals of the same color separated by a distance less than a signal width will be counted as one signal; and (3) for IGH-CCND1 a red and a green signal would be considered “fused” if they were located less than one signal width apart, or if they overlap to create a yellow fusion signal.
3.4. Proficiency test
A proficiency test followed 9 months after the workshop. Participant results for the proficiency test were compared with those of other participants and with the original conventional cytogenetic and FISH studies. There were no false-negative interpretations, although 3 (4%) of 68 results led to false-positive interpretations (Table 5). Participants B and C each interpreted patient 1 as having a low percentage of trisomy 12 nuclei; both of their results fell within 2.5% of their normal cutoff for this probe. Participant A interpreted patient 1 as having 13q–x1 in 5% of nuclei, which exceeded their 2% normal cutoff.
Table 5.
Percent of abnormal nuclei reported by each CRC participant for two patients with CLL and a known karyotype or FISH pattern in a proficiency test
| Percent 6q- nuclei |
Percent 11q- nuclei |
Percent +12 nuclei |
||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | PB | nd | 1.5 | 0.0 | 0.0 | 0.0 | 1 | 0.0 | 2.0 | 0.0 | 0.0 | 0.5 | 1 | 0.0 |  |
 |
0.0 | 1.0 |
| 2 | PB | nd | 0.0 | 0.0 | 0.0 | 0.5 | 2 | 1.5 | 0.0 | 0.0 | 0.0 | 1.0 | 2 | 66.5 | 75.5 | 68.0 | 70.5 | 70.0 |
| Percent 13q–x1 nuclei |
Percent 13q–x2 nuclei |
Percent 17p- nuclei |
||||||||||||||||
| Pt | Specimen | A | B | C | D | E | Pt | A | B | C | D | E | Pt | A | B | C | D | E |
| 1 | PB |  |
1.5 | 0.0 | 0.0 | 5.0 | 1 | 0.0 | 2.0 | 0.0 | 0.0 | 0.0 | 1 | 67.0 | 57.0 | 64.0 | 38.0 | 56.5 |
| 2 | PB | 54.0 | 50.0 | 56.0 | 56.0 | 45.5 | 2 | 28.0 | 18.5 | 23.0 | 18.5 | 26.0 | 2 | 4.0 | 0.0 | 0.0 | 0.0 | 1.5 |
| Percent t(14;?) nuclei | ||||||||||||||||||
| Pt | Specimen | A | B | C | D | E | ||||||||||||
| 1 | PB | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ||||||||||||
| 2 | PB | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ||||||||||||
False positive
False negative
Abbreviations: ND, not done; PB, peripheral blood; Pt, patient.
Participants used their own FISH probes and methods to obtain these results.
4. Discussion
An important goal of the CRC is to pool FISH data from the clinical practice of member cytogenetic laboratories that study patients with CLL. In addition, the CRC-led clinical trials are in need of a coherent FISH panel for data analysis and association studies with clinical outcome. The results of this study reveal some of the challenges faced by the CRC and other organizations that pool FISH results for CLL where the laboratories work independently of one another. The results of this study also provide information on how collaborating laboratories can deal with these issues.
Pilot study 1 revealed that false-positive and false-negative interpretations occurred in 5% of FISH assays. These false interpretations occurred primarily when FISH results were near the normal cutoff. We observed multiple variables that may have introduced methodologic variation among these laboratories. For example, we observed variations in probe strategy, reporting of anomalies, scoring criteria, number of nuclei analyzed, and statistical methods used to establish normal cutoffs.
In pilot study 2, the same state-of-the-art probe sets, methods, and normal cutoffs were used for CLL by all the laboratories. We wished to test whether using the same probes would reduce variation due to differences in FISH probes. Indeed, for patient 1 in pilot study 2 (patient 3 in pilot study 1) the differences in percentage abnormal nuclei were the same or less in pilot study 2 than in pilot study 1. A similar result was obtained for patient 2 in pilot study 2 (patient 10 in pilot study 1), including considerably less variability in the scores for trisomy 12. Likewise, no false-positive or false-negative interpretations were reported in pilot study 2 (Table 4).
The discussions at the workshop lead to a consensus on scoring criteria for CRC FISH testing in CLL, as detailed in Results. However, the proficiency test that occurred 9 months after the workshop demonstrated that false positive interpretations (4%) persisted. That no false-negative interpretations were reported would be perhaps expected because the only low percentage abnormality was gain of MDM2 in 10–20% of nuclei for patient 2. The false-positive trisomy 12 results for patient 1 merit additional comment. If, for example, patient 1 did indeed have a low percentage trisomy 12 clone despite the original clinical testing and three participants finding no evidence of trisomy 12, then the percentage of abnormal nuclei would be very low (average 2.5% for the two laboratories reporting a trisomy 12 and an average of 1.3% trisomy 12 if the scores from all five participants are combined).
The distribution of false interpretations is also informative. Whereas the false negative interpretations were distributed more or less randomly among the participant laboratories, all 11 false-positive interpretations were confined to participants A, B, and C, with six of them reported by participant A. It seems likely that many of the false-positive interpretations were due to inadequate normal cutoffs, and indeed the normal cutoffs of participants A and B were generally the lowest of all participants in this investigation. As is standard clinical practice, each participant laboratory established their own normal cutoff based on their own experiments to identify the incidence of false-positive cells in normal controls. At least four different statistical methods were employed by the laboratories in this investigation to determine FISH results, but only participants D and E used appropriate non-parametric methods to establish their normal cutoff values [8]. Laboratories A, B and C did not appear to have established their normal cutoffs in accordance with the criteria of the American College of Medical Genetics [6] or the National Committee for Clinical Laboratory Standards [7]. It is uncertain whether the 100 nuclei count at laboratory A had a further impact on their false-positive rate. Had all 5 participants employed the highest reported cutoff (Table 2), 5/11 of the false-positive interpretations from pilot study 1 and the proficiency study would have been reported as normal.
The normal cutoffs used among the CRC laboratories varied by as much as 5% (Table 2). For example, the normal cutoff for 13q-x1 varied from 2 to 7%, and for 11q- it varied from 2 to 5%. As just noted, this caused discrepant interpretations for specimens with a possible low level of abnormal nuclei. Setting a normal cutoff too low could be a particular problem for CLL patients, who are often diagnosed early in their disease when the percentage of abnormal nuclei can be quite low. A possible remedy might be for a collaborative group to define an empiric shared abnormal reference range so that scores just above the normal cutoff are not automatically interpreted as abnormal. For example, had 10% been set as the lower threshold for the abnormal reference range for deletions, any result below the cutoff would be interpreted as normal, any result between the cutoff and 10% could be classified as equivocal, and any result over 10% would be regarded as abnormal. If a 10% normal cutoff were applied to the pilot and proficiency tests, then only 1/11 of the false-positive results would have been interpreted as abnormal (12% for 17p- from participant A in pilot study 1). For trisomy 12 and for the D-FISH analysis, separate empiric abnormal reference ranges might be agreed upon; e.g., 5% for trisomy 12 and 10% for a t(14;?) result.
Given the importance of determining FISH status in CLL, it is imperative to consider the impact of false-negative versus false-positive FISH results. A growing body of evidence indicates that a low percentage of abnormal nuclei in CLL FISH panels does not portend as negative a prognosis as does a higher percentage for a given genetic defect [5,17,18]. Further evaluation of this is one of the long-term goals of the CRC Cytogenetic studies. For specimens that involve low percentages of abnormal nuclei, it would seem better to report false-negative interpretations for CLL FISH probes than it would be to report false-positive interpretations. Physicians may use false-positive interpretations to incorrectly estimate the prognosis of their patients with CLL and unintentionally counsel and treat them inappropriately. Eliminating all false-positive interpretations in this investigation would require adopting a shared universal cutoff of 12%. The false-positive finding for IGH in patient 2 by participant B is most likely related to a probe issue rather than to an erroneous normal cutoff. This laboratory reported that 9.5% of nuclei had an IGHx3 FISH pattern for the IGH-CCND1 probe, but this specimen did not produce evidence of t(14;?) in previous conventional cytogenetic studies, FISH assays, or by other participants. In contrast to the Abbott Molecular IGH-CCND1 XT dual-color, dual-fusion translocation probe used by participants A, D, and E, the Abbott Molecular IGH probe used by participants B and C in the first pilot study contains a 450-kb gap in the IGH variable region, which can create a false-positive result if cutoffs are not set sufficiently high and the scoring strategy is not altered to accommodate this larger space.
This study also revealed occasional participant discordance in the percent of abnormal nuclei observed in the same specimens (Table 3). In pilot study 1, the percent of abnormal nuclei that was reported among the five FISH assays in which all participants correctly identified an abnormality varied by as much as 29% (patient 9, 17p-results). This problem most likely resulted from inconsistent scoring criteria, possible sampling error due to the smaller number of cells examined by participant A, and possible sample distribution differences between the microscope slides prepared from the cell pellet. The workshop reduced the variability seen in our subsequent testing, but some participant discordance in the proficiency test samples remained, most notably again for the 17p- scoring of patient 1 (Table 5). Peculiarities of the Abbott Molecular chromosome 17 probe may account for some of that variability. This probe, used by all five participants, is known to exhibit a smaller signal than the other deletion probes in the CLL FISH panel, has variable hybridization efficiency, and tends to overlap more often due to colocalization of the chromosome 17 short arms.
What, then, is the most prudent and economical approach for organizations such as the CRC, which wish to pool their FISH data from different laboratories in their research program? To reduce problems with the normal cutoff for each FISH test, it seems reasonable to collect CLL data as percentages of abnormal nuclei (as the CRC does) rather than simply the interpretations (normal or abnormal). The availability of actual percentage of observed abnormal nuclei would permit the statisticians to perform their research studies only on specimens with percentages of abnormal nuclei clearly within a set abnormal reference range. Because determinations of percentage of abnormal nuclei in CLL FISH studies can be misleading due to inconsistent scoring criteria, shared validation procedures and quality assurance programs should be considered for interla-boratory correlative studies to assure accuracy, reproducibility, and comparability of their results [8].
In summary, organizations that pool results of various laboratories, such as the CRC, require accurate and consistent laboratory testing for their research studies of CLL. This collaborative study emphasizes the need for all FISH laboratories to (1) closely follow rigorous national standards and guidelines for their procedures, (2) analyze adequate number of cells to achieve accurate results, (3) apply strict scoring criteria to obtain consistent interpretations, and (4) use reliable nonparametric statistical methods to establish their normal cutoff for each chromosome abnormality. Strict standardization of FISH methods among all clinical cytogenetics laboratories enhances the confidence in FISH studies for both routine clinical applications and in clinical trial research settings such as the CRC or other cooperative intergroup clinical research.
Acknowledgments
This work was partially supported National Insitutes of Health (NIH) grant PO1-CA81534 of the CRC and NIH grant CA95241 of PI Neil Kay. We acknowledge the CRC members and thank the staff of the CRC Cytogenetics Laboratories for their expert technical support.
References
- 1.Shanafelt TD. Predicting clinical outcome in CLL: how and why. Hematology Am Soc Hematol Educ Program. 2009:421–429. doi: 10.1182/asheducation-2009.1.421. [DOI] [PubMed] [Google Scholar]
- 2.Dohner H, Stilgenbauer S, Benner A, Leupolt E, Krober A, Bullinger L, et al. Genomic aberrations and survival in chronic lymphocytic leukemia. N Engl J Med. 2000;343:1910–1916. doi: 10.1056/NEJM200012283432602. [DOI] [PubMed] [Google Scholar]
- 3.Dewald GW, Brockman SR, Paternoster SF, Bone ND, O’Fallon JR, Allmer C, et al. Chromosome anomalies detected by interphase fluorescence in situ hybridization: correlation with significant biological features of B-cell chronic lymphocytic leukaemia. Br J Haematol. 2003;121:287–295. doi: 10.1046/j.1365-2141.2003.04265.x. [DOI] [PubMed] [Google Scholar]
- 4.Fink SR, Paternoster SF, Smoley SA, Flynn HC, Geyer SM, Shanafelt TD, et al. Fluorescent-labeled DNA probes applied to novel biological aspects of B-cell chronic lymphocytic leukemia. Leuk Res. 2005;29:253–262. doi: 10.1016/j.leukres.2004.07.012. [DOI] [PubMed] [Google Scholar]
- 5.Van Dyke DL, Shanafelt TD, Call TG, Zent CS, Smoley SA, Rabe KG, et al. A comprehensive evaluation of the prognostic significance of 13q deletions in patients with B-chronic lymphocytic leukaemia. Br J Haematol. 2010;148:544–550. doi: 10.1111/j.1365-2141.2009.07982.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.American College of Medical Genetics. [Accessed on May 1, 2010];Standards and guidelines for clinical genetics laboratories. Available at: http://www.acmg.net/AM/Template.cfm?Section=Laboratory_Standards_and_Guidelines&Template=/CM/HTMLDisplay.cfm&ContentID=5308.
- 7.Enns R, Dewald G, Barker P, Rasmussen D, Watson M, Wood G, Wyatt P. NCCLS document MM7-A [ISBN 1-56238-524-0] Wayne, PA: 2004. Fluorescence in situ hybridization (FISH) methods for medical genetics: approved guideline. [Google Scholar]
- 8.Wiktor AE, Van Dyke DL, Stupca PJ, Ketterling RP, Thorland EC, Shearer BM, et al. Preclinical validation of fluorescence in situ hybridization assays for clinical practice. Genet Med. 2006;8:16–23. doi: 10.1097/01.gim.0000195645.00446.61. [DOI] [PubMed] [Google Scholar]
- 9.Dewald GW, Stallard R, Bader PI, Chen K, Zenger-Hain J, Harris CJ, et al. Toward quality assurance for metaphase FISH: a multicenter experience. Am J Med Genet. 1996;65:190–196. doi: 10.1002/(SICI)1096-8628(19961028)65:3<190::AID-AJMG4>3.0.CO;2-U. [DOI] [PubMed] [Google Scholar]
- 10.Dewald G, Stallard R, Al Saadi A, Arnold S, Bader PI, Blough R, et al. A multicenter investigation with interphase fluorescence in situ hybridization using X- and Y-chromosome probes. Am J Med Genet. 1998;76:318–326. [PubMed] [Google Scholar]
- 11.Dewald G, Stallard R, Al Saadi A, Arnold S, Blough R, Ceperich TM, et al. A multicenter investigation with D-FISH BCR/ ABL1 probes. Cancer Genet Cytogenet. 2000;116:97–104. doi: 10.1016/s0165-4608(99)00120-x. [DOI] [PubMed] [Google Scholar]
- 12.Dewald G, Brockman SR, Paternoster SF. Molecular cytogenetic studies in hematological malignancies. In: Finn W, Peterson L, editors. Hematopathology in Oncology. Norwell, MA: Kluwer Academic Publications; 2004. pp. 69–112. [Google Scholar]
- 13.Goorha S, Glenn MJ, Drozd-Borysiuk E, Chen Z. A set of commercially available fluorescent in-situ hybridization probes efficiently detects cytogenetic abnormalities in patients with chronic lymphocytic leukemia. Genet Med. 2004;6:48–53. doi: 10.1097/01.gim.0000105741.57923.08. [DOI] [PubMed] [Google Scholar]
- 14.Nowakowski GS, Dewald GW, Hoyer JD, Paternoster SF, Stockero KJ, Fink SR, et al. Interphase fluorescence in situ hybridization (FISH) with an IGH probe is important in the management of patients with a clinical diagnosis of chronic lymphocytic leukemia (CLL) Brit J Hematol. 2005;130:36–42. doi: 10.1111/j.1365-2141.2005.05548.x. [DOI] [PubMed] [Google Scholar]
- 15.Dewald GW, Therneau T, Larson D, Lee YK, Fink S, Smoley S, et al. Relationship of patient survival and chromosome anomalies detected in metaphase and/or interphase cells at diagnosis of myeloma. Blood. 2005;106:3553–3558. doi: 10.1182/blood-2005-05-1981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dewald GW, Wyatt WA, Juneau AL, Carlson RO, Zinsmeister AR, Jalal SM, et al. Highly sensitive fluorescence in situ hybridization method to detect double BCR/ABL fusion and monitor response to therapy in chronic myeloid leukemia. Blood. 1998;91:3357–3365. [PubMed] [Google Scholar]
- 17.Catovsky D, Richards S, Matute E, Oscier D, Dye MJS, et al. on behalf of the UK National Cancer Research Institute (NCRI) Haematological Oncology Clinial Studies Group and NCRI Chronic Lymphocytic Leukaemia Working Group. Assessment of fludarabine plus cyclophosphamide for patients with chronic lymphocytic leukaemia (the LRF CLL4 Trial): a randomised controlled trial. The Lancet. 2007;370:230–239. doi: 10.1016/S0140-6736(07)61125-8. [DOI] [PubMed] [Google Scholar]
- 18.Tam CS, Shanafelt TD, Wierda WG, Abruzzo L, Van Dyke DL, O’Brien S, et al. De novo deletion 17p13.1 chronic lymphocytic leukemia shows significant clinical heterogeneity: the MD Anderson and Mayo Clinic experience. Blood. 2009;114:957–964. doi: 10.1182/blood-2009-03-210591. [DOI] [PMC free article] [PubMed] [Google Scholar]