Novel TB smear microscopy automation system in detecting acid-fast bacilli for tuberculosis – A multi-center double blind study

Abstract

Due to COVID-19 pandemic, there is a large global drop in the number of newly diagnosed cases with tuberculosis (TB) worldwide. Actions to mitigate and reverse the impact of the COVID-19 pandemic on TB are urgently needed. Recent development of TB smear microscopy automation systems using artificial intelligence may increase the sensitivity of TB smear microscopy. The objective is to evaluate the performance of an automation system (μ-Scan 2.0, Wellgen Medical) over manual smear microscopy in a multi-center, double-blind trial. Total of 1726 smears were enrolled. Referee medical technician and culture served as primary and secondary gold standards for result discrepancy. Results showed that, compared to manual microscopy, the μ-Scan 2.0's performance of accuracy, sensitivity and specificity were 95.7% (1651/1726), 87.7% (57/65), and 96.0% (1594/1661), respectively. The negative predictive value was 97.8% at prevalence of 8.2%. Manual smear microscopy remains the primary diagnosis of pulmonary tuberculosis (TB). Use of automation system could achieve higher TB smear sensitivity and laboratory efficiency. It can also serve as a screening tool that complements molecular methods to reduce the total cost for TB diagnosis and control. Furthermore, such automation system is capable of remote access by internet connection and can be deployed in area with limited medical resources.

1. Background

Tuberculosis is a preventable, treatable, and curable disease. However, due to COVID-19 pandemic, there is a large global drop in the number of newly diagnosed cases with TB (a fall of 18% from 7.1 million in 2019 to 5.8 million in 2020 [1]. A similar pattern of a sharp fall in 2020 is also observed with particularly large absolute and relative reductions in the regions of South-East Asia and the Western Pacific. In combination, these two regions accounted for most (84%) of the global reduction of TB cases between 2019 and 2020 [1]. The countries that contributed most to the global drop between 2019 and 2020 were India (41%), Indonesia (14%), Philippines (12%) and China (8%); these and 12 other countries accounted for 93% of the total global drop of 1.3 million. This drop may reverse our efforts in STOP TB and takes the world back to 2012 [1]. It is foreseeable that more cases of TB will submerge in the next few years, which could be a challenge to our public health system [2].

Given the above-mentioned statistics, mathematical projections suggest that the impact of disruptions caused by the pandemic on the number of people developing TB and dying from the disease could be much worse in 2021 and 2022 [3]. Actions to mitigate and reverse the impact of the COVID-19 pandemic on TB are urgently needed. One solution to recover TB patients is to improve the diagnostic method. The most readily available and economical diagnostic tests for TB is sputum smear microscopy, especially in resource-limited countries [4]. Although molecular methods (eg. Xpert MTB/RIF) have been proposed to be more sensitive, WHO still suggest that such methods may be used as a follow-on test to TB smear microscopy in adults suspected of having TB but not at risk of MDR-TB or HIV-associated TB [5]. Molecular methods did not translate into lower tuberculosis-related morbidity and was unlikely to be affordable in many high TB burden countries without subsidy [6]. However, the current smear microscopy sensitivity is only 50–60% in pulmonary TB [1,7]. The fluorescence microscopy is more sensitive than conventional microscopy; however, the specificity of fluorescence microscopy for detection of acid-fast bacilli (AFB) in sputum is similar to that of conventional microscopy [8]. Despite these limitations, sputum smear microscopy remains a standard technique for clinical diagnosis of pulmonary TB and remains an integral part of the global TB control strategy [1]. Current WHO TB guidelines recommend immediate initiation of anti-TB treatment and follow-up monitoring after smear-positive results.

Recently some TB smear microscopy automation systems are being developed that take advantages of mechanical automation and artificial intelligence (AI) which may increase the sensitivity of TB smear microscopy [[9], [10], [11], [12], [13], [14]]. Such systems work similar to digital pathology which TB smear slides are digitally scanned and image recognition algorithm is deployed to detect AFB. Although most studies reported better performance than human examination, most are still in development using laboratory-spiked samples or “proof-of-concept” systems. Tu et al. first documented the efficacy (overall accuracy of 93.0% (2096/2254)) using a commercial model (μ-Scan 1.0, Wellgen Medical, Kaohsiung) [14]. The authors concluded that the automation system is to increase the test performance. However, experienced technicians are still needed to verify the test results and further improve of the system [14]. This current study is a follow up on Tu et al. using an upgrade system (μ-Scan 2.0). The objective of this study is to determine whether the performance of TB smear microscopy can be improved in combination of laboratory technicians and the automation system.

2. Materials and methods

• A.Study Design: This study was a multi-center, double-blind trial. TB specimens were collected from 43 hospitals and clinics in northern Taiwan from March to June 2020. Specimen collection, processing, transportation, and smear microscopy of AFB by Ziehl-Neelsen (ZN) were performed according to the national guideline. Mycobacterial culture was also conducted using Mycobacteria Growth Indicator Tube (MGIT) and Löwenstein -Jensen (LJ) culture media (BD Microbiology Systems, Franklin Lakes, NJ, USA).
• B.Evaluation Procedures: The evaluation flow of μ-Scan 2.0 vs. manual smear microscopy is illustrated in Fig. 1 . The TB smears were first processed through manual microscopy by medical technicians on duty. After the laboratory reports were issued as routine, all slides were transported to another laboratory for automation microscopy. A smear microscopy automation system (“automation system”) (μ-Scan 2.0, Wellgen Medical Co. LTD, Kaohsiung) was used (Fig. 2 ). The system consists of three components: (1) microscopic imaging acquiring hardware with auto-focusing and 8-slide/batch scanning mechanism. The images are stored in 2,448 × 2,048 JPG format for open access; (2) an internal computer that equipped with high-end graphic card for real time imaging processing without connection to the internet. This edge computing workflow eliminates the need for internet and clouding computing, and maximizes cybersecurity and patient data safety; (3) image recognition software for detection and classification of positive AFB in images. In the detection phase, candidate AFBs were tagged and differentiated from other background substances in smear based on color and morphological features. In the classification phase, the feature parameters (eg. shape of bacillus) were extracted from AFB candidates as the input parameters to a proprietary classifier. One hundred microscopic images (in equivalent to 400 manual microscopy fields of view) were analyzed and suspect AFBs were tagged by the image recognition algorithm (Figs. 3–1, Figs. 3–2 ). The technicians reviewed all images tagged as “positive” and either confirm or eliminate the image tags, and issued the results (defined as “automation system results”). A licensed laboratory technician served as “Referee Technician” in evaluating the system performance. The referee received independent results from manual microscopy and automation system, compared the discrepancy, and issued the final smear results. (Fig. 1).
• C.System Performance Evaluation and Quality Control: Positive smears detected by the automation system were re-examined by technicians using a microscope (Olympus CX-21, Olympus Corporation, Tokyo) under 1,000× oil lens as primary gold standard to resolved the discrepancy. Culture results were used as secondary gold standard to compare the sensitivity between smear microscopy and culture.
• D.Data Interpretation: The evaluation of test performance is based on overall accuracy, sensitivity, specificity, and negative predictive value. Statistical analysis (IBM SPSS, version 26, Armonk, NY) was performed using a 2-tailed 5% significance level for all analyses.

Fig. 1.

Study flow for comparison of performance between manual smear microscopy and automation system.

Fig. 2.

The automated microscope system in operation.

Figs. 3–1.

Examples of Tagged AFBs by the AI software.

Figs. 3–2.

Examples of Tagged AFBs by the AI software.

3. Results

1726 TB smears were used for evaluation. Based on the manual microscopy results, there were 65 AFB positive smears and 1661 AFB negative smears. Based on automation system results, there were 124 AFB positive smears and 1602 AFB negative smears. After referee technician's review, the system performance of overall accuracy, sensitivity, and specificity were 95.7% (1651/1726), 87.7% (57/65), and 96.0% (1594/1661), respectively (Table 1 ).

Table 1.

Performance of automation system and manual smear microscopy to detect AFBs.

Test Performance		Manual smear microscopy
		Positive	Negative
Automation System	Positive	57	67
	Negative	8	1594

The culture results of the 1726 specimens by MGIT and LJ media showed 142 specimens positive for M. tuberculosis complex. If compare the smear microscopy results with culture, the sensitivity for human technicians was only 33.8% (38/142); however, the automation system can achieve the sensitive of 74.6% (106/142) which is significantly higher than human technicians (Table 2 , p < 0.05). The negative predictive value is 97.8% (1566/1602) at a prevalence of 8.2% (142/1726).

Table 2.

Performance of automation system and culture to detect AFBs.

Test Performance		Culture
		Positive	Negative
Automation System	Positive	106	18
	Negative	36	1566

4. Discussion

The most economical, rapid, and readily available method for laboratory diagnosis of TB is smear microscopy by detecting acid-fast bacilli (AFB) in patient's specimens. However, the sensitivity of smear microscopy is highly variable [8] due to less experienced or trained staff, long hours of workload, and no presence of quality assurance. New technologies, such as the GeneXpert, based on molecular method are becoming available, it is unlikely that these technologies will be the affordable replacements of smear microscopy in many high burden countries without subsidy from governments or NGOs. Thus, if automation, AI and big data can be applied into TB smear microscopy, such systems may significantly increase the sensitivity of TB smear microscopy.

In this multi-center, double blind trial, the automation system (μ-Scan 2.0) achieved accuracy of 95.7% (1651/1726), sensitivity of 87.7% (57/65), specificity of 96.0% (1594/1661), respectively. When compare the smear microscopy results with culture, the sensitivity of human technicians was only 33.8% (38/142); however, the automation system can achieve the sensitivity of 74.6% (106/142) which is significantly higher than human technicians (Table 2, p < 0.05). Furthermore, the negative predictive value, an important indicator for a screening tool, is 97.8% (1566/1602) at the study prevalence of 8.2%. The performance of the automation system reported in this study exceeded the performance of manual microscopy reported in literatures [1]. Thanks to the special design (optic magnification and enhancement), the automation system can scan the smear that covers the area in equivalent of 400 fields @1,000× oil lens (WHO recommends >300 fields @1,000× oil lens) in 100 digital images. In addition, due to recent graphic processing and AI technology, the detection and recognition of AFBs is more accurate.

However, there is an issue to be resolved as whether culture should be used as gold standard in performance comparison with such smear microscopy automation system. There was a long discussion about the discrepancy of TB smear-positive but culture-negative results [4]. Often time nontuberculous mycobacteria (NTM) are present in patient's specimens, in forms of infection, colonization, or contamination during sampling, and result in smear positive for AFB but culture negative for M. tuberculosis complex. Current guidelines and recommendations stated that M. tuberculosis complex and NTM cannot be differentiated under smear microscopy. Thus, we support the hypothesis that the smear microscopy automation system should use referee technician and microscopy as gold standard in comparison with human technician during the performance evaluation.

During the study, we found several issues when applying smear microscopy automation system in clinical laboratories. First of all, the area of smear on slide needs to be standardized. The purpose of having the smear area set at 1 cm × 2 cm recommended by WHO is not to dilute the smear (if smear > 1 cm × 2 cm) to prevent the AFBs omitted by manual microscopy. Second, the technique of applying smear onto slide is also important. Smear too thick or too thin may affect the performance of the automation system. Third, current automation systems may be limited to pulmonary specimens because the extra-pulmonary specimens (eg. fluids) is too thin for the system's optic auto-focus and yield blurry images. Lastly, in our system design, the laboratory technicians are required to verify the results from the automation system before issuing the test report because the automation system serves as “clinical decision support” system. Its purpose is to reveal as many “AFB-like” images as possible for laboratory technicians to verify. Therefore, the design of image recognition algorithm is to maximize sensitivity, and false positive images may occur and technicians are needed to rule out the false positive results.

In conclusion, microscopic examination by human is the last mile of the laboratory automation. We believe the use of the TB smear microscopy automation systems could achieve higher TB smear sensitivity and laboratory efficiency. It can also serve as a screening tool that complements molecular methods (eg. GeneXpert) to reduce the total cost for TB diagnosis and control. Furthermore, such automation systems may have potential to expand to other microbiological application such as gram stains, parasite smears, and other smears that require professional technicians and labor-intensive works.

Author contributions statement

Ms. Hsiao-Chuan Huang designed and conducted the study and drafted the manuscript.

Mr. King-Lung Kuo provided smears of this study and quality control of the smears, and reviewed manuscript.

Ms. Mei-Hsin Lo and Ms. Hsiao-Yun Chou participated in data collection and statistics and manuscript preparation.

Yusen E. Lin co-designed and conducted the study, and co-drafted and finalized the manuscript.

Declaration of competing interest

This study was financially sponsored by Department of Health, Taipei City Government. The authors rented and used μ-Scan 2.0 smear microscopy automation system (Wellgen Medical, Kaohsiung) for the study.