Abstract
Background
This article was aimed to test the use of validation rules for blood smear review after automated hematological testing using Mindray CAL‐8000 (two hematological analyzers and one autoslider).
Methods
This study was based on 1013 peripheral blood samples (PB) referred for routine hematological testing. Results of testing on CAL‐8000 were analyzed using both locally derived and International Consensus Group for Hematology (ICGH) validation rules, and then compared with data obtained by optical microscopy (OM). A workflow analysis was also completed.
Results
The overall agreement with locally derived and ICGH criteria was 91% and 85%, but a higher sensitivity was observed for locally derived criteria (0.97 vs 0.95). The percentage of false negative and false positive samples was 2.1% and 7.1% using ICGH criteria, and was 1.4% and 14% using locally defined rules. The throughput of CAL‐8000 system was 208 samples/h, with a percentage of OM analysis comprised between 14% and 17%, and sensitivity of 0.97. As regards personnel activity, we estimated 0.8 full‐time equivalent (FTE) of technical staff and 0.7 FTE of personnel for clinical validation of data and blood smear review.
Conclusion
These results show that customization of validation rules is necessary for enhancing the quality of hematological testing and optimizing workflow.
Keywords: automated count, hematological testing, optical microscopy, rules
1. INTRODUCTION
The complete blood cell count (CBC), also including the leukocyte differential (DIFF‐profile), is one of the most requested laboratory test in routine clinical practice. Due to the introduction of a new generation of automated hemocytometers, a kaleidoscope of quantitative and qualitative information has become available along with the traditional parameter of the CBC, mostly in form of flags or scattergrams.1, 2, 3 Morphological flags play a crucial role not only for identifying cellular abnormalities, but also for reliably detecting preanalytical or analytical pitfalls such as platelet clumps or erythrocytes agglutination.4, 5, 6
The generation of morphological flags is mainly aimed at improving the dissection between normal and pathological blood samples, thus enhancing the appropriateness of optical microscopy (OM) examination. Nonetheless, several lines of evidence now attest that the diagnostic performance of different morphological flags, in terms of sensitivity and specificity, varies widely among the various hematological analyzers which are currently available in the market.4, 5, 6
The presence of real or spurious abnormalities of many laboratory investigations can be further investigated by adding new tests to existing requests either automatically, on the basis of predefined algorithms (i.e., reflex testing), or manually by laboratory professionals (i.e., reflective testing). In automated hematology, reflective/reflex testing is mainly finalized to identify those specimens needing OM analysis, and can both be triggered by the integrated evaluation of many instrument data, such as numerical results, scattergrams, and morphological flags. The appropriate setting of criteria and rules for activating this type of reflective/reflex testing was found to be effective to greatly limit the number of specimens really necessitating OM evaluation.7, 8, 9, 10
Due to the constant increasing of automation and shortage of skilled personnel, the OM review of blood smears has become a challenging issue for the organization of modern clinical laboratories, which are forced to analyze large volumes of samples in short time, with high accuracy and minimizing the risk of producing false negative results.10, 11, 12, 13 The appropriate use of automated hematology hence requires the accurate knowledge and evaluation of both sensitivity and specificity of the technology, as well as of the algorithms and rules for blood smear review. The need to establish a uniform behavior between laboratories has led to the definition of a panel of 41‐rules from International Consensus Group for Hematology10 (41‐ICGH rules), which specifically entails the activation of additional tests according to a logical pathway (i.e., “and” and/or “or”) during first‐time sample analysis or repeated testing. Nevertheless, the percentage of samples needing OM analysis depends on various factors, such as the laboratory organization, the skill of the personnel and the local patient population. Each single laboratory should hence establish, and systematically verify, the efficiency of local rules for analysis of blood smears.11, 12, 13 Therefore, the aim of this article was to test the practical use of locally defined criteria and the “41‐ICGH rules” issued by the international consensus group for hematology review of the International Society of Hematology (ISLH)10 and the optimization a workflow analysis using the new Mindray CAL‐8000 automated hematology system.
2. MATERIALS AND METHODS
2.1. Blood samples
This study was based on 1013 peripheral blood samples (PB) collected in K3EDTA tubes (Becton Dickinson, Franklin Lakes, NJ, USA) and referred to the clinical laboratory of Papa Giovanni XXIII Hospital in Bergamo for routine hematological testing. The routine samples used in this study were identified using a sequential criterion until attaining a number of 1000 specimens, which has been arbitrarily fixed as our minimum target. More specifically, the first 20 samples received in the laboratory between 9.00 and 11.00 am were selected on the first day, the first 20 samples received in the laboratory between 11.00 and 12.00 am were selected on the second consecutive day, whereas the first 20 samples received in the laboratory between 1.00 and 2.00 pm were selected on the third consecutive day. On the fourth day, the protocol was replicated for three other consecutive days. The protocol was repeated (for overall 18 days) until obtaining the minimum target of samples that has been originally planned (i.e., >1000).
A workflow analysis (i.e., the time necessary for completing testing, the percentage of results directly released to the clinics, the percentage of reflex testing activation and blood smear review) was also carried out during this investigation for three consecutive days, by analyzing a total number of 2595 samples, corresponding to the routine activity of the local laboratory between 8 and 12 am, and entailing 853 samples on the first day, 891 on samples on the second day, and 851 samples on the third day, respectively.
The PB analysis was carried out using CAL‐8000 (Mindray, Shenzhen‐China), the largest core model produced by Mindray, and composed by three analytical modules integrated within a single system (i.e., two BC‐6800 analyzers and one automatic Autoslider instrumentation SC‐120). The analysis of all samples was performed according to manufacturer's instructions, and the quality of data was evaluated by internal quality control assessment using a proprietary material on three different levels.14 A peripheral blood smear was also automatically prepared for all samples, using Autoslider SC‐120 and May‐Grunwald‐Giemsa staining (Carlo Erba, Italy).
2.2. Morphological analysis of blood smears
All samples included in this study underwent morphological analysis by OM according to the morphological criteria defined by the International Council for Standardization in Haematology (ICSH) recommendations.14, 15 Briefly, OM analysis was carried out by two experienced hematologists (third opinion was asked for results displaying more that 5% disagreement) on 200 cells at 400× and 1000× magnification, as for CLSI document H20‐A216 and ICSH guidelines.17 After OM analysis, each sample was classified as positive or negative according to the criteria established by the International Consensus Group for Hematology (ICGH).10
2.3. Local validation of rules for blood smear review
Two panels of rules were used and validated for selecting samples needing OM analysis (i.e., classified as positive) after automated hematological testing. The former panel, known as “41‐ICGH rules” is that proposed by the ICGH,10 whereas the latter has been locally defined after critical review of the literature and the analytical performance of CAL‐8000 (Table 1). The outcome of the application of the validation rules was evaluated for all the samples included in this study by direct comparison with OM data, using the Cohen's kappa test (K≥0.6 indicates substantial agreement). Therefore, according to the outcome of the application of validation rules and OM, the samples were classified as true positive (TP; sample positive for both OM analysis and validation rules), true negative (TN; sample negative for both OM analysis and validation rules), false negative (FN; sample positive for OM analysis but not selected for blood smear analysis after automated hematological testing according to validation rules), and false positive (FP; sample negative for OM analysis but still selected for blood smear analysis after automated hematological testing according to validation rules).
Table 1.
Locally defined validation rules for optical microscopy analysis after automated hematological testing
| Parameters | Cut‐off |
|---|---|
| Leukocytes (*109/L) | ≤3.0 |
| ≥17.0 | |
| Platelets (*109/L) | ≤50 |
| ≥600 | |
| Hemoglobin (g/L) | ≤70 |
| ≥220 | |
| Red blood cell mean corpuscular volume (fL) | ≤75 |
| ≥105 | |
| RDW | ≥22 |
| Neutrophils (*109/L) | ≤1.0 |
| ≥8.0 | |
| Lymphocytes (*109/L) | ≤1.0 |
| ≥5.0 (Adult) | |
| ≤1.5 | |
| ≥7.0 (≤12 years old) | |
| Monocytes (*109/L) | =0 |
| ≥1.0 | |
| Basophils (*109/L) | ≥0.15 |
| Eosinophils (*109/L) | ≥2.0 |
| Nucleated reed blood cells (*109/L) | Any value |
| MCHC (g/L) | ≤300 |
| ≥370 | |
| MCH (pg) | ≤26 |
| Morphological flags | Any Morphological Flag |
2.4. Workflow analysis
The CAL‐8000 system is equipped with the middleware LabExpert for workflow management, through which a number of rules can be defined, including (a) automatic release of results; (b) stop releasing data needing operator's evaluation; (c) stop releasing data and implementation of automated reflex testing (e.g., nucleated red blood cells counting using a separate channel, reticulocytes enumeration, platelets counting using optical technique); (d) stop releasing data and automatic preparation of blood smears. The rules were also tested using delta check for comparing data of the same patient at 7 days for the following parameters: platelets (delta check, ±30%), hemoglobin (delta check, ±20%), leukocytes (delta check, ±50%), and automated reflex test for NRBC count with red cell distribution width (RDW) >22, and/or morphology flag: “NRBC present”.
2.5. Statistical analysis
The statistical analysis was carried out using Analyse‐it (software version 3.90.1; Analyse‐it software Ltd; Leeds, UK). The statistical significance was set at P<.05. The study was performed according to the Declaration of Helsinki, under the terms of all relevant local legislation and with approval of the local ethical Committee. The study was also based on routine samples, which were used for the study protocol in an anonymous form, only once routine analyses were completed.
3. RESULTS
Out of the 1013 samples included in the study, 39% were from women and 61% from men, respectively. The mean age of the patients was 56 years (95% Confidential Interval [CI], 53‐57 years), with no statistically significant difference between genders. More specifically, 25.3% of the samples were collected from outpatients, 10.3% from blood donors, 12.4% were drawn from patients hospitalized in the hematology ward, 12.1% from patients hospitalized in the oncology ward and 2.7% from children hospitalized in pediatrics ward, whereas the remaining 37.2% samples were collected from surgical and/or medical units.
According to the “41‐ICGH rules” proposed by the ICGH,10 518 (51%) of the 1013 analyzed samples were found to be positive for OM analysis (Table 2), showing 91% agreement (Cohen's kappa, 0.81; 95% CI, 0.78‐0.85), 0.95 sensitivity and 0.87 specificity with the outcome of blood smear review. Overall, 21 samples (i.e., 2.1%) were classified as FN according to the “41‐ICGH rules” proposed by the ICGH (i.e., samples positive for OM analysis but not selected for blood smear analysis after automated hematological testing according to validation rules). Specifically, OM analysis revealed that 12 samples contained myelocytes and metamyelocytes, three contained nucleated red blood cells, one displayed morphological platelet abnormalities and four were found to have significant neutrophil dysplasia (Table 3). As regards FP samples (sample negative for OM analysis but still selected for blood smear analysis after automated hematological testing according to validation rules), the percentage was relatively modest (72 samples; 7.1%), and mostly attributable to the cut‐offs proposed by the ICGH (45% FP for platelet count and 38% FP for leukocyte count) (Table 4).
Table 2.
Results obtained with the use of the two validation rules panels
| Sensitivity (95% CI) | Specificity (95% CI) | Diagnostics agreements (95% CI) | Cohen K test (95% CI) (K ≥ 0.6 P value) | Number of samples and % | ||||
|---|---|---|---|---|---|---|---|---|
| FP (%) | FN (%) | TP (%) | TN (%) | |||||
| 41‐ICGH rules | 95% (93‐97) | 87% (83‐89) | 91% (88‐92) | 0.81 (0.78‐0.85) P<.001 | 73 (7.2) | 21 (2.1) | 445 (43.9) | 474 (46.8) |
| Locally defined validation rules | 97% (95‐98) | 74% (70‐77) | 85% (82‐87) | 0.70 (0.65‐0.74) P<.001 | 142 (14.0) | 14 (1.4) | 452 (44.6) | 405 (40.0) |
FN: False negative FP: False positive; TN: True negative; TP: True positives.
Table 3.
Description false negatives samples with application of the two rules panels
| Morphological abnormalities | False negative 41‐ICGH rules (%) | False negativeLocally defined validation rules (%) |
|---|---|---|
| Metamyelocyte, myelocyte | 12 (58) | 8 (58) |
| Nucleated reed blood cells | 3 (14) | 2 (14) |
| Reed blood cells morphology alteration | 1 (4.5) | 0 (0) |
| Platelet morphology alteration | 1 (4.5) | 1 (7) |
| Dysplastic neutrophils | 4 (19) | 3 (21) |
| Total of false negative samples | 21 (2.1) | 14 (1.4) |
Table 4.
Description of major cause of false positive samples with application of the two rules panels
| Rules panels | False positive samples (%) | Rules gives false positive samples | % |
|---|---|---|---|
| 41‐ICGH rules | 73 (7.2) | Platelets <100*109/L or PLT count >1000*109/L | 45 |
| Leucocytes <4.0 *109/L or WBC count >30.0*109/L | 38 | ||
| Neutrophils <1.0*109/L or neutrophils >20.0*109/L | 22 | ||
| Morphology flag: IG present | 18 | ||
| Morphology flag: Abn. lymph/blast | 16 | ||
| Locally defined validation rules | 142 (14.0) | Lymph count <1.0*109/L or lymph count >5.0*109/L | 44 |
| Leucocytes <3.0*109/L or WBC count >17.0*109/L | 27 | ||
| Neutrophils <1.0*109/L or neutrophils >8.0*109/L | 25 | ||
| Platelets <50*109/L or platelets >600*109/L | 15 | ||
| MCV <75 fL or MCV >105 fL | 10 | ||
| Morphology flag: IG present | 9 |
MCV, mean corpuscular volume; IG, Immature Granulocytes.
As regards the locally defined validation rules (Table 1), the agreement with the outcome of blood smear review was 85% (Cohen's kappa, 0.70; 95% CI, 0.65‐0.74), whereas the sensitivity was 0.97 and the specificity was 0.74, respectively. Overall, 14 samples (i.e., 1.4%) were classified as FN according to the locally defined validation rules (i.e., samples positive for OM analysis but not selected for blood smear analysis after automated hematological testing according to validation rules), eight for the presence of myelocytes and metamyelocytes, two for the presence of nucleated red blood cells, one displayed morphological platelet abnormalities and three were found to have significant neutrophil dysplasia (Table 3). The number of FP samples (sample negative for OM analysis but still selected for blood smear analysis after automated hematological testing according to validation rules) was instead higher (142;14%), and mostly attributable to the cut‐off used for flagging the lymphocyte (i.e., 44% FP) and leukocyte (i.e., 27% FP) counts (Table 4).
Due to the greater sensitivity of the locally developed validation rules compared to the “41‐ICGH rules” proposed by the ICGH (0.97 vs 0.95), we evaluated the practical use of the former approach on CAL‐8000 for three consecutive days, using delta check criteria (Table 5). Interestingly, the throughput of the system composed by two hematological analyzers and one autoslider was 208 samples/h, with a percentage of OM analysis comprised between 14% and 17%, and a sensitivity of 0.97. As regards personnel activity, we estimated 0.8 full‐time equivalent (FTE) of technical staff and 0.7 FTE of personnel for clinical validation of data and blood smear review.
Table 5.
Performance panel suggested rules applied to the CAL‐8000 system
| Number of samples and type | Number of samples with results blocked by LabExpert (%) | Number and percentage of smear | Number other reflex test performed | Data released by operator without add any reflex test (%) | Time required to complete the activity including microscopic review | |||
|---|---|---|---|---|---|---|---|---|
| Add by rules as a automated reflex (%) | Add by operator (%) | Morphological review (%) | ||||||
| Day 1 | 853 OP=409 (48%) IP=448 (52%) | 159 (18) | 108 (12.6) | 14 (1.6) | 122 (14.2) | 11 (1 RET and 10 NRBC) | 26 (3) | 4 h |
| Day 2 | 891 OP=338 (38%)IP=553 (62%) | 240 (27) | 146 (16.4) | 10 (1.1) | 156 (17.5) | 8 (3 RET and 5 NRBC) | 76 (8.5) | 4 h e 30 m |
| Day 3 | 851 OP=307 (36%) IP=554 (64%) IP | 246 (29) | 110 (12.9) | 13 (1.5) | 123 (14.4) | 10 (3 RET and 7 NRBC) | 113 (13) | 4 h |
h, hours; IP, In patients; m, minutes; NRBC, Nucleated red blood cells; N, Number; OP, Out patients; RET, Reticulocytes.
4. DISCUSSION
It is now undeniable that each single clinical laboratory should locally define and then validate a panel of rules for automated hematological testing, aimed at identifying those samples really needing OM analysis.11, 18 Therefore, we designed a study to investigate the performance and impact of the “41‐ICGH rules” proposed by the ICGH,10 as well as the performance of a panel of locally defined criteria (Table 1), on the routine activity of the CAL‐8000 system. The first data emerged from our study is that both panels were characterized by excellent performance, displaying an overall agreement with OM analysis comprised between 85% and 91%. Interestingly, the locally defined criteria were found to have greater sensitivity than the “41‐ICGH rules” (0.97 vs 0.95), whereas the percentage of FN samples was very modest in either case (1.4% and 2.1%) (Table 3), in accord with previous recommendations issued by the ICCH. 11 The percentage of FP data was nearly double using locally defined criteria compared to the “41‐ICGH rules” (14% vs 7%), but this value was still comprised within the 18.6% limit set by the consensus group.
The comparison of our data with those previously published shows that the percentage of FP and FN samples is comparable or even better than what has been earlier obtained, wherein these percentages typically ranged between 11.3% and 17.3% for FP samples and between 2.2% and 14.3% for FN samples, respectively.9, 11, 18, 19 The sensitivity and specificity data are also rather similar to those previously reported, which were comprised between 0.41 and 0.86, and 0.77 and 0.94, respectively.9, 11, 19 Interestingly, the implementation of the panel of locally defined criteria in routine activity of the CAL‐8000 system was associated with a percentage of blood samples needing OM analysis comprised between 14% and 17%. These modest variations are obviously attributable to the different type of samples and patients' case mix across the 3 days of evaluation.
In conclusion, the results of our study show that customization of validation rules is a necessary step for enhancing the quality of hematological testing and optimizing the workflow. Our preliminary model was found to be suitable for routine use on the Mindray CAL‐8000 technology, but necessitates confirm in other laboratories with differing case mix and instrumentation.
FUNDING
The authors received no financial support for the research, authorship, and/or publication of this article.
ACKNOWLEDGMENTS
None.
REFERENCES
- 1. Buttarello M, Plebani M. Automated blood cell counts state of the art. Am J Clin Pathol. 2008;130:104‐116. [DOI] [PubMed] [Google Scholar]
- 2. Briggs C. Quality counts: new parameters in blood cell counting. Int Jnl Lab Hem. 2009;31:277‐297. [DOI] [PubMed] [Google Scholar]
- 3. Buttarello M. Quality specification in haematology: the automated blood cell count. Clin Chim Acta. 2004;346:45‐54. [DOI] [PubMed] [Google Scholar]
- 4. Hotton J, Broothaers J, Swaelens C, et al. Performance and abnormal cell flagging comparisons of three automated blood cell counters: Cell‐Dyn Sapphire, DxH‐800, and XN‐2000. Am J Clin Pathol 2013;140:845‐852. [DOI] [PubMed] [Google Scholar]
- 5. Kang SH, Kim HK, Ham CK, et al. Comparison of four hematology analyzers, CELL‐DYN Sapphire, ADVIA 120, Coulter LH 750, and Sysmex XE‐2100, in terms of clinical usefulness. Int Jnl Lab Hem. 2008;30:480‐486. [PubMed] [Google Scholar]
- 6. Lippi G, Plebani M. EDTA‐dependent pseudothrombocytopenia: further insights and recommendations for prevention of a clinically threatening artifact. Clin Chem Lab Med. 2012;50:1281‐1285. [DOI] [PubMed] [Google Scholar]
- 7. Tan BT, Nava AJ, George TI. Evaluation of the Beckman Coulter UniCel DxH 800, Beckman Coulter LH 780, and Abbott Diagnostics Cell‐Dyn Sapphire hematology analyzers on adult specimens in a tertiary care hospital. Am J Clin Pathol. 2011;135:939‐951. [DOI] [PubMed] [Google Scholar]
- 8. Meintker L, Ringwald J, Rauh M, et al. Comparison of automated differential blood cell counts from Abbott Sapphire, Siemens Advia 120, Beckman Coulter DxH 800, and Sysmex XE‐2100 in normal and pathologic samples. Am J Clin Pathol. 2013;139:641‐650. [DOI] [PubMed] [Google Scholar]
- 9. Hur M, Cho JH, Kim H, et al. Optimization of laboratory workflow in clinical hematology laboratory with reduced manual slide review: comparison between Sysmex XE‐2100 and ABX Pentra DX120 Int. Jnl Lab Hem. 2011;33:434‐440. [DOI] [PubMed] [Google Scholar]
- 10. Barnes PW, McFadden L, Machin SJ, et al. The International Consensus Group for Hematology Review: suggested criteria for action following automated CBC and WBC differential analysis. Lab Hematol. 2005;11:83‐90. [PubMed] [Google Scholar]
- 11. Pratumvinit B, Wongkrajang P, Reesukumal K, et al. Validation and optimization of criteria for manual smear review following automated blood cell analysis in a large university hospital. Arch Pathol Lab Med. 2013;137:408‐414. [DOI] [PubMed] [Google Scholar]
- 12. Novis DA, Walsh M, Wilkinson D, et al. Laboratory productivity and the rate of manual peripheral blood smear review A College of American Pathologists Q‐Probes Study of 95 141 complete blood count determinations performed in 263 institutions. Arch Pathol Lab Med. 2006;130:596‐601. [DOI] [PubMed] [Google Scholar]
- 13. Gulati G, Song J, Dulau Florea A, et al. Purpose and criteria for blood smear scan, blood smear examination, and blood smear review. Ann Lab Med. 2013;33:1‐7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Ford J. Red blood cell morphology. Int J Lab Hematol. 2013;35:351‐357. [DOI] [PubMed] [Google Scholar]
- 15. Palmer L, Briggs C, McFadden S, et al. ICSH recommendations for the standardization of nomenclature and grading of peripheral blood cell morphological features. Int J Lab Hematol. 2015;37:287‐303. [DOI] [PubMed] [Google Scholar]
- 16. Clinical Laboratory Standards Institute . Reference Leukocyte (WBC) Differential Count (Proportional) and Evaluation of instrumental Methods; approved guideline ‐ second edition. CLSI document H20‐A2. Wayne, PA: Clinical Laboratory Standards Institute; 2010. [Google Scholar]
- 17. Briggs C, Culp N, Davis B, et al. ICSH guidelines for the evaluation of blood cell analysers including those used for differential leucocyte and reticulocyte counting. Int J Lab Hematol. 2014;36:613‐627. [DOI] [PubMed] [Google Scholar]
- 18. Tseng V, Morgan AS, Leith P, et al. Efficient assessment of peripheral blood lymphocytosis in adults: developing new thresholds for blood smear review by pathologists. Clin Chem Lab Med. 2014;52:1763‐1770. [DOI] [PubMed] [Google Scholar]
- 19. Froom P, Havis R, Barak M. The rate of manual peripheral blood smear reviews in outpatients. Clin Chem Lab Med. 2009;47:1401‐1405. [DOI] [PubMed] [Google Scholar]