Abstract
Tuberculosis, which primarily affects developing countries, remains a significant global health concern. Since the 2010s, the role of chest radiography has expanded in tuberculosis triage and screening beyond its traditional complementary role in the diagnosis of tuberculosis. Computer-aided diagnosis (CAD) systems for tuberculosis detection on chest radiographs have recently made substantial progress in diagnostic performance, thanks to deep learning technologies. The current performance of CAD systems for tuberculosis has approximated that of human experts, presenting a potential solution to the shortage of human readers to interpret chest radiographs in low- or middle-income, high-tuberculosis-burden countries. This article provides a critical appraisal of developmental process reporting in extant CAD software for tuberculosis, based on the Checklist for Artificial Intelligence in Medical Imaging. It also explores several considerations to scale up CAD solutions, encompassing manufacturer-independent CAD validation, economic and political aspects, and ethical concerns, as well as the potential for broadening radiography-based diagnosis to other nontuberculosis diseases. Collectively, CAD for tuberculosis will emerge as a representative deep learning application, catalyzing advances in global health and health equity.
Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning
Supplemental material is available for this article.
© RSNA, 2024
Keywords: Computer-aided Diagnosis (CAD), Conventional Radiography, Thorax, Lung, Machine Learning
Summary
The global implementation of artificial intelligence–based computer-aided diagnosis tools, which approach human-level accuracy on chest radiographs in tuberculosis triage and screening, requires a transparent development process; rapid manufacturer-independent validation; and comprehensive economic, political, and ethical considerations by all relevant stakeholders.
Introduction
Tuberculosis was the leading cause of death among infectious diseases globally until the advent of the COVID-19 pandemic (1). The World Health Organization (WHO)'s “End TB Strategy” targets a 95% reduction in the number of tuberculosis deaths and a 90% reduction in tuberculosis incidence by 2035, relative to 2015 (2). The first milestones for 2020 were 35% and 20% reductions in tuberculosis deaths and incidence, respectively; however, the achieved reductions were only 5.9% and 10%, respectively, in 2021 (1,2), underscoring the challenges in implementing effective strategies to decrease the global tuberculosis burden.
Because most patients with drug-sensitive tuberculosis can be treated successfully with standardized regimens (2), prompt tuberculosis detection and systematic screening of high-risk groups for tuberculosis are integral components of the WHO's strategies (2). Chest radiography can play a pivotal role in tuberculosis detection in low-income, resource-constrained countries bearing a major tuberculosis burden (3), but the limited availability of expert readers is a major hurdle hampering the effective application of chest radiography (4,5). Artificial intelligence (AI) for computer-aided diagnosis (CAD) of tuberculosis on chest radiographs could enable a breakthrough in tackling constrained human resources (3,6,7). This article describes the role of chest radiography in tuberculosis management and discusses the current status and future perspectives of AI CAD, focusing on global health (Fig 1).
Overview of the Current Status of Tuberculosis Worldwide
The number of estimated global tuberculosis deaths rose to 1.6 million from 1.4 million between 2019 and 2021, reversing the declining trend since 2000 (1). Meanwhile, the number of new tuberculosis diagnoses dropped sharply to 5.8 million from 7.1 million between 2019 and 2020, partially rising to 6.4 million in 2021 (1). These changes, which likely reflect the negative impact of the COVID-19 pandemic, underscore the critical importance of identifying undiagnosed tuberculosis cases to reduce tuberculosis deaths.
The tuberculosis burden disproportionately affects low-income countries, particularly in Africa and Southeast Asia, where tuberculosis mortality rates are much higher (approximately 40 deaths per 100 000 population in 2021) than in the United States and Europe (three deaths per 100 000 population) (1). This global disparity can be attributed to several factors, including limited health care infrastructure and human resources, a shortage of diagnostic and therapeutic tools, a higher prevalence of comorbidities (such as HIV infection and malnutrition), a lack of adequate policies or funding, and other socioeconomic health determinants. Effectively addressing the global tuberculosis burden requires comprehensive strategies with international collaboration (1).
The Expanding Role of Chest Radiography in Tuberculosis
Thoracic tuberculosis can be seen on chest radiographs even in its early stages, before symptoms appear (8). However, radiographic manifestations of tuberculosis can overlap with those of other respiratory diseases, and the interreader variability, chest radiography availability, operation costs, and scarcity of high-quality readers (4) are significant limitations of chest radiography. Consequently, the traditional role of chest radiography in the WHO guidelines was limited to a complementary tool at the last phase of the diagnostic algorithm for tuberculosis, when result of a sputum test is negative (3).
Technical advances have recently enabled portable digital chest radiography systems with lower costs and better image quality (9). National tuberculosis prevalence surveys in high-burden countries have shown that chest radiography is the most sensitive screening tool for detecting pulmonary tuberculosis, surpassing symptom-based approaches (10). Recognizing this potential, WHO endorsed using chest radiography in adults for tuberculosis triage and screening (Fig 2) (3).
Triage refers to the process by which health care professionals determine the diagnostic pathways for individuals who present with symptoms (3,6). The low sensitivity of symptom-based triage (eg, with only 35% of patients having chronic cough) may hinder confirmatory examination for many patients with tuberculosis. Combining symptoms (eg, hemoptysis, fever, night sweats, or weight loss) may increase the sensitivity, but doing so also increases the number of false-positive results and the cost of confirmatory examinations, such as rapid molecular diagnostic tests (RMDTs) (3,11). Incorporating chest radiography into the triage process for symptomatic patients can reduce false-positive rates and costs of confirmatory examinations (3).
Systematic screening for tuberculosis—that is, targeting high-risk individuals who do not seek health care (3,6,11)—can minimize the gap between incident tuberculosis cases and tuberculosis detection and reduce the risk of community transmission (11). Because screening involves a large number of examinations, the WHO recommends tuberculosis screening in targeted high-risk groups (11), such as recent contacts of patients with tuberculosis and incarcerated populations, considering that these high-risk groups provide the lowest number of screening examinations needed to identify one tuberculosis case (11,12).
CAD with Deep Learning for Tuberculosis
Despite the improved availability and operability of chest radiography hardware, expert readers remain limited, especially in resource-constrained countries (4–7). This limitation would be amplified for systematic screening, which involves many chest radiographs and often takes place outside health care institutions. In this regard, automated chest radiograph interpretation using a CAD tool has emerged as an attractive option to scale up chest radiography–based systematic screening or triage (3,6,7,12). Despite active research, CAD exhibited limited performance for screening or triage (13) until the rise of deep learning (DL), and the WHO recommended using CAD for research only (3).
The DL technology has led to remarkable advances in the automated detection of tuberculosis at chest radiography (14). Early studies showed accurate differentiation between chest radiographs depicting tuberculosis and normal chest radiographs (15–17). Subsequent studies reported that DL algorithms showed performance similar or superior to that of expert radiologists (18–20). DL algorithms also successfully distinguished active pulmonary tuberculosis from posttreatment changes (20,21) and active pulmonary tuberculosis from latent tuberculosis infection (22), and also depicted various imaging features of pulmonary tuberculosis (19,23). Some studies have found that the diagnostic accuracy of human readers can be enhanced when assisted by DL algorithms (18,22).
Diagnostic Performance in Real-World Validation
Multiple DL-based CAD tools have been approved for clinical use and have entered the market (Table S1). However, excellent performance in early validation does not necessarily guarantee real-world performance, and several factors may contribute to this discrepancy: (a) the lower prevalence of tuberculosis in real-world settings, (b) subtler chest radiography findings due to earlier presentation, and (c) different image quality and domain shift. Therefore, CAD performance needs to be validated in real-world scenarios (6). A previous systematic review and meta-analysis (14) evaluated the performance of AI-based CAD tools in 36 studies (13 developmental studies and 23 clinical studies). However, the meta-analysis included diagnostic case-control design studies, which did not appropriately reflect the prevalence and spectrum of real-world situation. Furthermore, a substantial proportion of clinical studies were conducted in settings other than triage or systematic screening. Therefore, the systematic review may not sufficiently describe the diagnostic performance of CAD tools in real-world triage and systematic screening settings.
To describe the landscape of CAD performance in real-world scenarios of triage and systematic screening, we searched PubMed, Web of Science, and SCOPUS with the following query: tuberculosis AND (deep learning OR artificial intelligence OR computer-assisted OR computer-aided) AND (chest radiograph OR chest x-ray). From 354 searched reports published through September 2023, we identified 19 studies reporting the performance of CAD for tuberculosis diagnosis in triage (n = 9) (24–32) or screening (n = 10) settings (33–42) (Table 1). Among them, 10 studies (six in the triage setting and four in the screening setting) compared the performance of multiple CAD tools using real-world data (24–26,30,31,36,40–42).
Table 1:
The minimum requirements of tests for triage or systematic screening of tuberculosis as described in the WHO target product profile (TPP) (43) are a sensitivity of 90% and a specificity of 70%. The optimal requirements in the WHO TPP are a sensitivity of 95% and a specificity of 80%. Various studies have investigated whether CAD tools meet the TPP using real-world data (Table 1).
For tuberculosis triage, two CAD tools showed noninferior performance to the WHO TPP in symptomatic adults in Pakistan (27). Another study found that five CAD tools outperformed radiologists and two of them met the WHO TPP in individuals aged 15 years or older in Bangladesh (26). In an individual patient data meta-analysis evaluating three CAD tools from four studies (30), the CAD tools did not meet the WHO TPP, while exhibiting similar accuracy to readers. In particular, performance decreased in HIV-infected patients and those with smear-negative tuberculosis.
For tuberculosis screening, the reported CAD performances were slightly lower than those for triage, possibly because of the much lower prevalence and earlier tuberculosis presentation. In screening U.S.-bound migrants, three CAD tools did not satisfy the WHO TPP but exhibited similar performance to radiologists (41). In a community-based screening in Vietnam (36), none of 12 CAD tools met the WHO TPP and six of them exhibited performance similar to that of human readers. In screening male prisoners in Brazil (40), only one of three CAD tools met the WHO TPP. In an individual-patient meta-analysis evaluating three CAD tools from six studies (42), none met the WHO TPP, although the tools approximated experts’ performance.
Despite the limited degree to which CAD tools have satisfied the TPP, the WHO has stated that CAD tools may be used in place of human readers for interpreting chest radiographs for tuberculosis screening and triage, given their similar accuracy to human readers (12).
Critical Appraisal of Developmental Process Reporting in CAD Tools
The Stop TB Partnership and FIND, a global alliance for diagnostics, surveyed CAD tools to identify those that evaluate chest radiographs for tuberculosis (44). They solicited information from developers of both market-ready and in-progress CAD tools through questionnaires. The profiles of these CAD tools are posted and regularly updated on the AI4HLTH website (https://www.ai4hlth.org/). As of September 2023, 20 CAD tools (13 market-ready, three certification-pending, and four under development) were listed (Tables S1, S2). Seven CAD tools target only adults (age, ≥18 years), and the other 13 can be applied to adolescents (age, 10–17 years) as well. Four CAD tools can be used in children (age, 0–6 years). Nineteen CAD tools can depict various diseases or abnormal findings other than tuberculosis. However, no CAD tool has shown whether it can differentiate tuberculosis and nontuberculosis diseases using chest radiography.
To enhance the reliability of CAD, transparently declaring the developmental processes and training datasets is critical. We reviewed the literature on the development of the CAD tools listed on the AI4HLTH website and identified 12 reports (nine in peer-reviewed journals, one in conference proceedings, and two in preprint archives) for nine CAD tools (18,20,45–54). Literature was absent for 11 CAD tools. We also included two reports on the development of CAD tools (19,55) not listed on the website for comparison. Of the 12 reports, seven focused on the identification of tuberculosis at chest radiography (18–20,46,47,49,50), and the remaining five focused on CAD tools for detecting various abnormalities (45,51,52,55) or diseases (48). All reports were evaluated by a thoracic radiologist (W.G.J., 4 years of experience as an attending thoracic radiologist), using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) (56).
The median number of CLAIM items satisfied was 32 of 42 (range, 10–39) (Table 2, Table S3). All reports described the backgrounds, objectives, and hypotheses of the study and the structures and initial parameters of the models. However, only one study described the method of sample size determination, and two described the measurement of inter- and intrarater variability and analyses regarding incorrect classifications. At least half of the reports did not clearly describe (a) the eligibility criteria, (b) the method of de-identification, (c) how missing data were handled, (d) the rationale for reference standard selection, (e) the technique of model ensembling, (f) the conduction of external validation, (g) a flow diagram for inclusion and exclusion of participants or data, or (h) the research registration on a platform.
Table 2:
It is a limitation of this analysis that the appraisal of literature was performed by a single thoracic radiologist, which can be influenced by the subjective judgment of the radiologist.
Strengths and Weaknesses of Current CAD Tools and Practical Considerations for Use
Compared with traditional human reading–dependent systems, current CAD tools exhibit noteworthy strengths, primarily their scalability across different scenarios: offline service using a hardware system versus online service using a cloud server and integration with radiography machines versus integration with an institutional picture archiving and communication system (PACS). Offline service would be beneficial for systematic screening outside a health care institution or in resource-constrained settings with unstable network access. Meanwhile, the strength of online service would be reduced cost for hardware devices and ease of maintenance and update. In terms of integration with legacy system, integration with radiography machines would be beneficial in systematic screening outside a health care institution, whereas integration with a PACS would be beneficial in a large-institution setting where multiple radiography machines are in use. This scalability of CAD tools makes tuberculosis triage and screening feasible in resource-limited settings or beyond health care institutions where human interpretation is challenging. Reproducible interpretations devoid of inter- and intrareader variability with preserved accuracy are also crucial benefits of CAD.
However, several important weaknesses in current CAD tools must be addressed. First, the optimal method to integrate CAD with existing human reading systems remains unclear. WHO has proposed four scenarios to integrate CAD into tuberculosis screening or triage: (a) initial CAD screening followed by the reader's final interpretation for abnormal results; (b) initial CAD screening with verification of results by readers for a portion of selected results; (c) stand-alone CAD in place of a human reader, with abnormal results referred for diagnostic evaluation; and (d) parallel interpretation by CAD and reader, with abnormal result from either reading referred for diagnostic evaluation (12). In a substantially resource-limited setting where a human reader's interpretation is unavailable, use of stand-alone CAD to replace human readers would be the only practical option. However, because false-positive results of CAD are not uncommon, confirmative interpretation for abnormal CAD results by a human reader can improve the specificity of interpretations, thereby reducing the number of chest radiographs to be read and the corresponding cost (22). In situations with sufficient human readers to interpret all chest radiographs, using CAD as a second reader can enhance the sensitivity for tuberculosis detection (22). The method of integrating CAD into tuberculosis screening or triage should be chosen after consideration of the availability of human readers and related costs. Further investigation is required to confirm the efficacy and cost-effectiveness of various methods of integration.
Second, the sensitivity and specificity of a CAD system may vary depending on the characteristics of target population, such as the prevalence of tuberculosis and other pulmonary diseases and the proportion of people living with HIV. Because most CAD tools generate continuous scores rather than binary positive or negative results, appropriate calibration of the operating threshold before implementation in a new setting would be necessary to minimize the fluctuation of CAD performance (12). The WHO Global Tuberculosis Programme and the Special Program for Research and Training in Tropical Diseases developed a toolkit for calibration of CAD before implementation. The toolkit provides a generic protocol for CAD calibration studies, including detailed methods, as well as an online tool for the analysis of CAD performance to determine the optimal threshold (57).
Uncertainty regarding the legal liability for errors in interpretation is another important hurdle for the wide implementation of CAD tools. If a CAD tool is used as an assistant tool for the radiologist, the radiologist who made a final decision would typically bear the liability risk (58–60). Conversely, if a CAD tool is used as an independent reader, the issue would become more complex. AI itself may not bear liability because AI cannot be conferred a legal personality (58,60). In such cases, the institution or operator overseeing tuberculosis screening and triage may bear vicarious liability. Theoretically, the manufacturer of a CAD tool also could be held liable under products liability (58,59).
Limited evidence for the effectiveness in children and coverage for only frontal chest radiography are also weaknesses of CAD tools. Finally, the scarcity of education on how to use CAD tools is another limitation to overcome.
Manufacturer-independent Evaluation Platform of CAD Tools
FIND, the global alliance for diagnostics (https://www.finddx.org), is a global nonprofit organization working to accelerate equitable access to reliable diagnosis around the world. As a WHO Collaborating Center for laboratory strengthening and diagnostic technology evaluation, FIND collaborates with more than 40 countries and 210 locations (Table S4). One aspect of FIND's work in digital health focuses on improving equitable access to appropriate CAD tools for use in low- and middle-income countries. This work increases evidence generation for use of chest radiography CAD tools in marginalized populations and facilitates the procurement, optimization, and implementation of these tools in low- and middle-income country populations. This work includes independent evaluations of the CAD performance and supported a recent guidance on the use of CAD software to evaluate chest radiography for tuberculosis by the WHO through independent performance evaluations (12,41,42).
FIND has developed a validation platform (VP) enabling independent iterative evaluations of CAD tools for tuberculosis (41,42,61,62). The VP has been built to evaluate CAD as described below and in Figure 3.
Identification of fit-for-use CAD tools: FIND identifies available CAD tools for tuberculosis through iterative landscapes of this field and works with our partner network. FIND engages with developers of promising CAD products and, in collaboration with the Stop TB Partnership, maintains a repository that describes the product maturity and market penetration (https://www.ai4hlth.org/) (44). The goal is to identify all promising CAD tools with tuberculosis use cases, which are trained on data from low- and middle-income countries and may benefit populations from these settings.
Onboarding of CAD tools for evaluation: After identifying a promising CAD tool, FIND signs a contractual agreement with the developer, allowing independent evaluations to be publicly reported. Then, the VP team, with systems engineering support from the eSHIFT Partner Network (the VP technical partner) works with the CAD developers to install their software in a secure virtual environment, verifying performance against a reference set of chest radiographs. Once the operational capability is mutually confirmed, the FIND team then removes developer access to the VP environment.
Curation of high-quality datasets for validation: The International Telecommunication Union (ITU)–WHO Focus Group on Artificial Intelligence for Health for evaluations of AI-based products (the open code initiative) did not contribute to the evaluation of commercially available chest radiography CAD products for the WHO tuberculosis guidelines. FIND was a member and contributor to the focus group and has supported its recent transition into the WHO– ITU–World Intellectual Property Organization (WIPO)–led global initiative on AI for Health as a founding member. FIND is exploring partnerships with the WHO–ITU–WIPO global initiative and with the open code initiative in future evaluations of chest radiography CAD for WHO and other stakeholders. FIND collaborates with multiple global partners to collect high-quality chest radiographs and associated metadata. This includes clinical and demographic data, laboratory testing results for tuberculosis as reference diagnosis, and chest radiographs, adhering to international standards for ethical research. The data are drawn from published cohorts to maximize transparency and are never shared with developers (41). Current datasets encompass almost 20 000 chest radiographs from multiple countries in five WHO regions.
Independent CAD evaluations: FIND evaluates CAD product performance using these datasets for specific populations and use cases. The evaluation considers accuracy against both radiologist and laboratory reference diagnoses, including subgroup analysis by age, sex, location, and comorbidities. Additional information regarding CAD performance is also collected by sites, image types (computed vs digital radiography), and chest radiography devices. For developers, these evaluations may generate evidence for performance comparisons against preexisting, established CAD tools and influence procurement or policy decisions, incentivizing involvement.
FIND's independent analyses can offer significant value for decision-makers. These analyses can also assist in evaluating new versions and products of CAD tools, guaranteeing comparability with existing policies. This supports country adoption and optimization efforts. In instances without established policies, such as using CAD for tuberculosis evaluation in children or distinguishing respiratory infections (tuberculosis, COVID-19, and pneumonia), the VP generates evidence highlighting software benefits and limitations in specific contexts.
Economic, Political, and Ethical Considerations When Scaling Up CAD for Tuberculosis
The potential impact of CAD on global tuberculosis is immense. However, the effect of social inequalities on global health should be considered (63). Indeed, the concept of “global health” carries presuppositions about epistemology and intervention methods that, despite well-intentioned aims, may inadvertently perpetuate these inequalities (64,65). Overemphasis on technical solutions and service privatization can distract from addressing real issues of social determinants of health (66), which are especially pertinent in tuberculosis. Given that CAD tools would be deployed primarily in resource-limited countries yet developed in wealthier countries, several considerations must extend beyond the effectiveness of CAD (67).
The WHO's endorsement of CAD has catalyzed evolution of the market (12). However, the WHO has yet to establish prequalification processes, entailing intricate regulatory requirements encompassing technical, economic, and political factors. Furthermore, given the swift pace of software development and update, robust regulation is particularly important for CAD. With the release of more accurate versions, health equity could be threatened if these improvements are not accessible to all users. Regulations need to tackle the technical hurdles that hinder CAD adoption. Achieving consensus on CAD specifications and regulations is also crucial.
Calibrating CAD also involves technical, ethical, and political challenges. Although the WHO has suggested large-scale epidemiologic studies to calibrate CAD thresholds (12), the resources for calibration processes can be affected by resource disparities, particularly for RMDTs. Although FIND has established a VP based on low- and middle-income multinational data (61), the issue of local support for CAD calibration remains unresolved and could be a crucial factor in successful CAD use.
Ethical concerns regarding CAD use have largely involved the prevention of data misuse. The authority of states and patients over their own data has become a matter of serious concern. Many CAD solutions offer analysis via data stored in the cloud, prompting concerns about the security of potentially sensitive health information. However, beyond issues of data security and confidentiality, CAD technologies carry important economic and political implications. The political stakes associated with CAD are substantial, especially regarding the sovereignty of states and patients over their data. This becomes even more critical as data sovereignty is increasingly central to the evaluation and legitimacy of global health care programs.
Finally, commercial aspects must also be considered in relation to the infrastructural challenges posed by the introduction of digital health care tools. Data infrastructure may be privatized, particularly in countries lacking the capacity to manage cloud systems themselves. Leaving such data infrastructure in the hands of private corporations and profit-seeking shareholders would pose a significant political problem. Ultimately, CAD systems must meet numerous technical, economic, and political conditions to realize their potential (68). It is important to balance the potential of CAD with due consideration of the health care systems in which they will be deployed and a critical assessment of the benefits for each potential stakeholder.
Expanding CAD Applications beyond Tuberculosis Detection
To date, most AI investigations have concentrated on detecting tuberculosis findings on chest radiographs. However, AI can also extract other valuable information from chest radiographs. For example, AI could help predict drug resistance (69) and monitor response to antituberculosis medication (20). The prediction of high-risk patients requiring intensive treatment would also be valuable, and such predictions have been feasible for other infectious diseases (70,71).
Appropriate logical explanation for the backgrounds and factors of AI's prediction is also important, especially for these novel AI applications. For AI depicting tuberculosis findings on chest radiographs, indicating the location of the finding can substantially provide the background of AI's output (72,73). A physician can decide whether to accept or reject the AI's prediction, after assessing whether the abnormality detected with the AI is consistent with findings of tuberculosis. However, regarding AI predicting outcomes beyond the tuberculosis diagnosis, the problem would be more complex. Because outcome prediction is much less intuitive than detection or diagnosis prediction, simply highlighting the area that contributed to the AI's prediction would be insufficient for the explanation of logical background of AI's prediction, and a physician would be not able to decide whether to accept or reject the AI's prediction. This limited explainability may undermine the reliability of AI, and even can be an important obstacle for the clinical application of AI (74,75).
Conclusion
CAD technology based on DL has major potential to boost the role of chest radiography for tuberculosis detection in resource-constrained countries. This potential success will be achieved by the following fulfillment processes (Fig 1): (a) transparent declaration of development processes and data; (b) publicly available information for users on indications, costs, and regulatory approvals; (c) rapid and manufacturer-independent evaluation of CAD tools using real-world datasets; and (d) incorporation of technical, political, economic, and ethical considerations of CAD deployment, initiating educational activities for potential users, and monitoring the performance and effectiveness of CAD after deployment. Active collaboration among developers, health care institutions, authorities, and international organizations will make it possible to harness the full potential of CAD and reduce tuberculosis globally.
Acknowledgments
Acknowledgments
The authors would like to acknowledge Nikhil Jagtiani, MS, DEng, and Rigveda Kadam, MBA, BTech, from the FIND digital health team, for their contributions to editing of the FIND section of the manuscript.
Data collection and development of the FIND validation platform was supported with funding provided by the German Ministry for Education and Research (BMBF) through KfW.
Data sharing: All data generated or analyzed during the study are included in the published paper.
Disclosures of conflicts of interest: E.J.H. Research grant from Lunit. W.G.J. No relevant relationships. P.M.D. Author funded by the OBVIA (Observatoire international sur les impacts societaux de l'IA et du numerique, from the Fonds de Recherche de Quebec) (project number: FRQSC,268938). M.A. Full-time consultant at FIND; support for travel from FIND. M.R. No relevant relationships. S.H.Y. Stock options in MEDICAL IP.
Abbreviations:
- AI
- artificial intelligence
- CAD
- computer-aided diagnosis
- CLAIM
- Checklist for Artificial Intelligence in Medical Imaging
- DL
- deep learning
- ITU
- International Telecommunication Union
- PACS
- picture archiving and communication system
- RMDT
- rapid molecular diagnostic test
- TPP
- target product profile
- VP
- validation platform
- WHO
- World Health Organization
- WIPO
- World Intellectual Property Organization
References
- 1. World Health Organization . Global tuberculosis report 2022 . Geneva: : World Health Organization; . https://www.who.int/teams/global-tuberculosis-programme/tb-reports/global-tuberculosis-report-2022. Published 2022. Accessed August 5, 2023. [Google Scholar]
- 2. World Health Organization . Implementing the end TB strategy: the essentials, 2022 update . Geneva: : World Health Organization; . https://www.who.int/publications/i/item/9789240065093. Published 2022. Accessed August 5, 2023. [Google Scholar]
- 3. World Health Organization . Chest radiography in tuberculosis detection: summary of current WHO recommendations and guidance on programmatic approaches . Geneva: : World Health Organization; . https://www.who.int/publications/i/item/9789241511506. Published 2016. Accessed August 5, 2023. [Google Scholar]
- 4. Pedrazzoli D , Lalli M , Boccia D , Houben R , Kranzer K . Can tuberculosis patients in resource-constrained settings afford chest radiography? Eur Respir J 2017. ; 49 ( 3 ): 1601877 . [DOI] [PubMed] [Google Scholar]
- 5. Frija G , Blažić I , Frush DP , et al . How to improve access to medical imaging in low- and middle-income countries? EClinicalMedicine 2021. ; 38 : 101034 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ahmad Khan F , Pande T , Tessema B , et al . Computer-aided reading of tuberculosis chest radiography: moving the research agenda forward to inform policy . Eur Respir J 2017. ; 50 ( 1 ): 1700953 . [DOI] [PubMed] [Google Scholar]
- 7. Geric C , Qin ZZ , Denkinger CM , et al . The rise of artificial intelligence reading of chest X-rays for enhanced TB diagnosis and elimination . Int J Tuberc Lung Dis 2023. ; 27 ( 5 ): 367 – 372 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Nachiappan AC , Rahbar K , Shi X , et al . Pulmonary tuberculosis: role of radiology in diagnosis and management . RadioGraphics 2017. ; 37 ( 1 ): 52 – 72 . [DOI] [PubMed] [Google Scholar]
- 9. Story A , Aldridge RW , Abubakar I , et al . Active case finding for pulmonary tuberculosis using mobile digital chest radiography: an observational study . Int J Tuberc Lung Dis 2012. ; 16 ( 11 ): 1461 – 1467 . [DOI] [PubMed] [Google Scholar]
- 10. World Health Organization . National tuberculosis prevalence surveys 2007-2016 . Geneva: : World Health Organization; . https://www.who.int/publications/i/item/9789240022430. Published 2021. Accessed August 5, 2023. [Google Scholar]
- 11. World Health Organization . Systematic screening for active tuberculosis: principles and recommendations . Geneva: : World Health Organization; . https://iris.who.int/handle/10665/84971?show=full. Published 2013. Accessed August 5, 2023. [PubMed] [Google Scholar]
- 12. World Health Organization . WHO consolidated guidelines on tuberculosis. Module 2: screening – systematic screening for tuberculosis disease . Geneva: : World Health Organization; . https://www.who.int/publications/i/item/9789240022676. Published 2022. Accessed August 5, 2023. [PubMed] [Google Scholar]
- 13. Pande T , Cohen C , Pai M , Ahmad Khan F . Computer-aided detection of pulmonary tuberculosis on digital chest radiographs: a systematic review . Int J Tuberc Lung Dis 2016. ; 20 ( 9 ): 1226 – 1230 . [DOI] [PubMed] [Google Scholar]
- 14. Zhan Y , Wang Y , Zhang W , Ying B , Wang C . Diagnostic accuracy of the artificial intelligence methods in medical imaging for pulmonary tuberculosis: a systematic review and meta-analysis . J Clin Med 2022. ; 12 ( 1 ): 303 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Hwang S , Kim HE , Jeong J , Kim HJ . A novel approach for tuberculosis screening based on deep convolutional neural networks . In: Proc SPIE 9785, Medical Imaging 2016: Computer-Aided Diagnosis, 97852W (24 March 2016) . SPIE; , 2016. ; 750 – 757 . [Google Scholar]
- 16. Lakhani P , Sundaram B . Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks . Radiology 2017. ; 284 ( 2 ): 574 – 582 . [DOI] [PubMed] [Google Scholar]
- 17. Nash M , Kadavigere R , Andrade J , et al . Deep learning, computer-aided radiography reading for tuberculosis: a diagnostic accuracy study from a tertiary hospital in India . Sci Rep 2020. ; 10 ( 1 ): 210 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Hwang EJ , Park S , Jin KN , et al . Development and validation of a deep learning-based automatic detection algorithm for active pulmonary tuberculosis on chest radiographs . Clin Infect Dis 2019. ; 69 ( 5 ): 739 – 747 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Kazemzadeh S , Yu J , Jamshy S , et al . Deep learning detection of active pulmonary tuberculosis at chest radiography matched the clinical performance of radiologists . Radiology 2023. ; 306 ( 1 ): 124 – 137 . [DOI] [PubMed] [Google Scholar]
- 20. Lee S , Yim JJ , Kwak N , et al . Deep learning to determine the activity of pulmonary tuberculosis on chest radiographs . Radiology 2021. ; 301 ( 2 ): 435 – 442 . [DOI] [PubMed] [Google Scholar]
- 21. Choi YR , Yoon SH , Kim J , Yoo JY , Kim H , Jin KN . Chest radiography of tuberculosis: determination of activity using deep learning algorithm . Tuberc Respir Dis (Seoul) 2023. ; 86 ( 3 ): 226 – 233 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Park J , Hwang EJ , Lee JH , et al . Identification of active pulmonary tuberculosis among patients with positive interferon-gamma release assay results: value of a deep learning-based computer-aided detection system in different scenarios of implementation . J Thorac Imaging 2023. ; 38 ( 3 ): 145 – 153 . [DOI] [PubMed] [Google Scholar]
- 23. Engle E , Gabrielian A , Long A , Hurt DE , Rosenthal A . Performance of Qure.ai automatic classifiers against a large annotated database of patients with diverse forms of tuberculosis . PLoS One 2020. ; 15 ( 1 ): e0224445 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Qin ZZ , Sander MS , Rai B , et al . Using artificial intelligence to read chest radiographs for tuberculosis detection: a multi-site evaluation of the diagnostic accuracy of three deep learning systems . Sci Rep 2019. ; 9 ( 1 ): 15000 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Qin ZZ , Barrett R , Del Mar Castro M , et al . Early user experience and lessons learned using ultra-portable digital X-ray with computer-aided detection (DXR-CAD) products: a qualitative study from the perspective of healthcare providers . PLoS One 2023. ; 18 ( 2 ): e0277843 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Qin ZZ , Ahmed S , Sarker MS , et al . Tuberculosis detection from chest x-rays for triaging in a high tuberculosis-burden setting: an evaluation of five artificial intelligence algorithms . Lancet Digit Health 2021. ; 3 ( 9 ): e543 – e554 . [DOI] [PubMed] [Google Scholar]
- 27. Khan FA , Majidulla A , Tavaziva G , et al . Chest x-ray analysis with deep learning-based software as a triage test for pulmonary tuberculosis: a prospective study of diagnostic accuracy for culture-confirmed disease . Lancet Digit Health 2020. ; 2 ( 11 ): e573 – e581 . [DOI] [PubMed] [Google Scholar]
- 28. Murphy K , Habib SS , Zaidi SMA , et al . Computer aided detection of tuberculosis on chest radiographs: an evaluation of the CAD4TB v6 system . Sci Rep 2020. ; 10 ( 1 ): 5492 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Tavaziva G , Majidulla A , Nazish A , et al . Diagnostic accuracy of a commercially available, deep learning-based chest X-ray interpretation software for detecting culture-confirmed pulmonary tuberculosis . Int J Infect Dis 2022. ; 122 : 15 – 20 . [DOI] [PubMed] [Google Scholar]
- 30. Tavaziva G , Harris M , Abidi SK , et al . Chest X-ray analysis with deep learning-based software as a triage test for pulmonary tuberculosis: an individual patient data meta-analysis of diagnostic accuracy . Clin Infect Dis 2022. ; 74 ( 8 ): 1390 – 1400 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Kagujje M , Kerkhoff AD , Nteeni M , Dunn I , Mateyo K , Muyoyeta M . The performance of computer-aided detection digital chest x-ray reading technologies for triage of active tuberculosis among persons with a history of previous tuberculosis . Clin Infect Dis 2023. ; 76 ( 3 ): e894 – e901 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Biewer A , Tzelios C , Tintaya K , et al . Accuracy of digital chest x-ray analysis with artificial intelligence software as a triage and screening tool in hospitalized patients being evaluated for tuberculosis in Lima, Peru . medRxiv 2023.05.17.23290110 [preprint] https://www.medrxiv.org/content/10.1101/2023.05.17.23290110. Published December 7, 2023. Accessed August 5, 2023. [DOI] [PMC free article] [PubMed]
- 33. Koesoemadinata RC , Kranzer K , Livia R , et al . Computer-assisted chest radiography reading for tuberculosis screening in people living with diabetes mellitus . Int J Tuberc Lung Dis 2018. ; 22 ( 9 ): 1088 – 1094 . [DOI] [PubMed] [Google Scholar]
- 34. Habib SS , Rafiq S , Zaidi SMA , et al . Evaluation of computer aided detection of tuberculosis on chest radiography among people with diabetes in Karachi Pakistan . Sci Rep 2020. ; 10 ( 1 ): 6276 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Lee JH , Park S , Hwang EJ , et al . Deep learning-based automated detection algorithm for active pulmonary tuberculosis on chest radiographs: diagnostic performance in systematic screening of asymptomatic individuals . Eur Radiol 2021. ; 31 ( 2 ): 1069 – 1080 . [DOI] [PubMed] [Google Scholar]
- 36. Codlin AJ , Dao TP , Vo LNQ , et al . Independent evaluation of 12 artificial intelligence solutions for the detection of tuberculosis . Sci Rep 2021. ; 11 ( 1 ): 23895 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Mungai B , Ong'angò J , Ku CC , et al . Accuracy of computer-aided chest X-ray in community-based tuberculosis screening: lessons from the 2016 Kenya National Tuberculosis Prevalence Survey . PLOS Glob Public Health 2022. ; 2 ( 11 ): e0001272 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Fehr J , Konigorski S , Olivier S , et al . Computer-aided interpretation of chest radiography reveals the spectrum of tuberculosis in rural South Africa . NPJ Digit Med 2021. ; 4 ( 1 ): 106 . [Published correction appears in NPJ Digit Med 2021;4(1):115.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Liao Q , Feng H , Li Y , et al . Evaluation of an artificial intelligence (AI) system to detect tuberculosis on chest X-ray at a pilot active screening project in Guangdong, China in 2019 . J XRay Sci Technol 2022. ; 30 ( 2 ): 221 – 230 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Soares TR , Oliveira RD , Liu YE , et al . Evaluation of chest x-ray with automated interpretation algorithms for mass tuberculosis screening in prisons: a cross-sectional study . Lancet Reg Health Am 2022. ; 17 : 100388 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Gelaw SM , Kik SV , Ruhwald M , et al . Diagnostic accuracy of three computer-aided detection systems for detecting pulmonary tuberculosis on chest radiography when used for screening: analysis of an international, multicenter migrants screening study . PLOS Glob Public Health 2023. ; 3 ( 7 ): e0000402 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Kik SV , Gelaw SM , Ruhwald M , et al . Diagnostic accuracy of chest x-ray interpretation for tuberculosis by three artificial intelligence-based software in a screening use-case: an individual patient meta-analysis of global data . medRxiv 2022.01.24.22269730 [preprint] https://www.medrxiv.org/content/10.1101/2022.01.24.22269730. Published January 27, 2022. Accessed August 5, 2023.
- 43. World Health Organization . High priority target product profiles for new tuberculosis diagnostics: report of a consensus meeting . Geneva: : World Health Organization; . https://www.who.int/publications/i/item/WHO-HTM-TB-2014.18. Published 2014. Accessed August 5, 2023. [Google Scholar]
- 44. Qin ZZ , Naheyan T , Ruhwald M , et al . A new resource on artificial intelligence powered computer automated detection software products for tuberculosis programmes and implementers . Tuberculosis (Edinb) 2021. ; 127 : 102049 . [DOI] [PubMed] [Google Scholar]
- 45. Ajmera P , Onkar P , Desai S , et al . Validation of a deep learning model for detecting chest pathologies from digital chest radiographs . Diagnostics (Basel) 2023. ; 13 ( 3 ): 557 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Hogeweg L , Mol C , de Jong PA , Dawson R , Ayles H , van Ginneken B . Fusion of local and global detection systems to detect tuberculosis in chest radiographs . Med Image Comput Comput Assist Interv 2010. ; 13 ( Pt 3 ): 650 – 657 . [DOI] [PubMed] [Google Scholar]
- 47. Hogeweg L , Sánchez CI , Maduskar P , et al . Automatic detection of tuberculosis in chest radiographs using a combination of textural, focal, and shape abnormality analysis . IEEE Trans Med Imaging 2015. ; 34 ( 12 ): 2429 – 2442 . [DOI] [PubMed] [Google Scholar]
- 48. Kwon T , Lee SP , Kim D , et al . Diagnostic performance of artificial intelligence model for pneumonia from chest radiography . PLoS One 2021. ; 16 ( 4 ): e0249399 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Melendez J , van Ginneken B , Maduskar P , Philipsen RH , Ayles H , Sanchez CI . On combining multiple-instance learning and active learning for computer-aided detection of tuberculosis . IEEE Trans Med Imaging 2016. ; 35 ( 4 ): 1013 – 1024 . [DOI] [PubMed] [Google Scholar]
- 50. Melendez J , van Ginneken B , Maduskar P , et al . A novel multiple-instance learning-based approach to computer-aided detection of tuberculosis on chest x-rays . IEEE Trans Med Imaging 2015. ; 34 ( 1 ): 179 – 192 . [DOI] [PubMed] [Google Scholar]
- 51. Park S , Lee SM , Lee KH , et al . Deep learning-based detection system for multiclass lesions on chest radiographs: comparison with observer readings . Eur Radiol 2020. ; 30 ( 3 ): 1359 – 1368 . [DOI] [PubMed] [Google Scholar]
- 52. Putha P , Tadepalli M , Reddy B , et al . Can artificial intelligence reliably report chest x-rays? Radiologist validation of an algorithm trained on 2.3 million x-rays . arXiv 1807.07455 [preprint] https://arxiv.org/abs/1807.07455. Published July 19, 2018. Accessed August 5, 2023. [Google Scholar]
- 53. Saporta A , Gui X , Agrawal A , et al . Benchmarking saliency methods for chest x-ray interpretation . Nat Mach Intell 2022. ; 4 ( 10 ): 867 – 878 . [Google Scholar]
- 54. Chopade R , Stanam A , Patil A , Pawar S . K-fold semi-supervised self-learning technique for image disease localization . In: Smys S , Tavares JMRS , Shi F , eds. Computational Vision and Bio-Inspired Computing . Advances in Intelligent Systems and Computing, vol 1439 . Springer; , 2023. ; 691 – 696 . [Google Scholar]
- 55. Seah JCY , Tang CHM , Buchlak QD , et al . Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study . Lancet Digit Health 2021. ; 3 ( 8 ): e496 – e506 . [DOI] [PubMed] [Google Scholar]
- 56. Mongan J , Moy L , Kahn CE Jr . Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers . Radiol Artif Intell 2020. ; 2 ( 2 ): e200029 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57. World Health Organization . Determining the local calibration of computer-assisted detection (CAD) thresholds and other parameters: a toolkit to support the effective use of CAD for TB screening . Geneva: : World Health Organization; . https://iris.who.int/handle/10665/345925?show=full. Published 2021. Accessed August 5, 2023. [Google Scholar]
- 58. Mezrich JL . Is artificial intelligence (AI) a pipe dream? Why legal issues present significant hurdles to AI autonomy . AJR Am J Roentgenol 2022. ; 219 ( 1 ): 152 – 156 . [DOI] [PubMed] [Google Scholar]
- 59. Jaremko JL , Azar M , Bromwich R , et al . Canadian Association of Radiologists white paper on ethical and legal issues related to artificial intelligence in radiology . Can Assoc Radiol J 2019. ; 70 ( 2 ): 107 – 118 . [DOI] [PubMed] [Google Scholar]
- 60. Hwang EJ , Goo JM , Yoon SH , et al . Use of artificial intelligence-based software as medical devices for chest radiography: a position paper from the Korean Society of Thoracic Radiology . Korean J Radiol 2021. ; 22 ( 11 ): 1743 – 1748 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61. Arentz M , Jagtiani N , Kik S , Ruhwald M , Kadam R . AI-CAD for tuberculosis and other global high-burden diseases . Lancet Digit Health 2023. ; 5 ( 3 ): e115 . [DOI] [PubMed] [Google Scholar]
- 62. FIND . Report from the FIND meeting on March 8, 2023: WHO Recommendations and Developments in the Prequalification of Chest Radiography and Computer-Aided Detection (CXR-CAD) for Tuberculosis . Geneva, Switzerland: : FIND; , 2023. . https://www.finddx.org/wp-content/uploads/2023/03/20220324_rep_crx_cad_tb_FV_EN.pdf. Accessed August 5, 2023. [Google Scholar]
- 63. Biehl J , Petryna A . When People Come First: Critical Studies in Global Health . Princeton University Press; , 2013. . [Google Scholar]
- 64. Farmer P , Kim JY , Kleinman A , Basilico M . Reimagining Global Health: An Introduction . University of California Press; , 2013. . [Google Scholar]
- 65. Lock MM , Nguyen VK . An Anthropology of Biomedicine . Wiley; , 2018. . [Google Scholar]
- 66. Storeng KT , Prince RJ , Mishra A . The politics of health systems strengthening . In: Routledge Handbook on the Politics of Global Health . Routledge; , 2018. ; 114 – 121 . [Google Scholar]
- 67. Onno J , Ahmad Khan F , Daftary A , David PM . Artificial intelligence-based computer aided detection (AI-CAD) in the fight against tuberculosis: effects of moving health technologies in global health . Soc Sci Med 2023. ; 327 : 115949 . [DOI] [PubMed] [Google Scholar]
- 68. David PM , Onno J , Keshavjee S , Ahmad Khan F . Conditions required for the artificial-intelligence-based computer-aided detection of tuberculosis to attain its global health potential . Lancet Digit Health 2022. ; 4 ( 10 ): e702 – e704 . [DOI] [PubMed] [Google Scholar]
- 69. Sethanan K , Pitakaso R , Srichok T , et al . Computer-aided diagnosis using embedded ensemble deep learning for multiclass drug-resistant tuberculosis classification . Front Med (Lausanne) 2023. ; 10 : 1122222 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70. Kim C , Hwang EJ , Choi YR , et al . A deep learning model using chest radiographs for prediction of 30-day mortality in patients with community-acquired pneumonia: development and external validation . AJR Am J Roentgenol 2023. ; 221 ( 5 ): 586 – 598 . [DOI] [PubMed] [Google Scholar]
- 71. Pyrros A , Rodriguez Fernandez J , Borstelmann SM , et al . Validation of a deep learning, value-based care model to predict mortality and comorbidities from chest radiographs in COVID-19 . PLOS Digit Health 2022. ; 1 ( 8 ): e0000057 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72. Selvaraju RR , Cogswell M , Das A , Vedantam R , Parikh D , Batra D . Grad-CAM: visual explanations from deep networks via gradient-based localization . In: 2017 IEEE International Conference on Computer Vision (ICCV) . IEEE; , 2017. ; 618 – 626 . [Google Scholar]
- 73. Redmon J , Divvala S , Girshick R , Farhadi A . You only look once: unified, real-time object detection . In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE; , 2016. ; 779 – 788 . [Google Scholar]
- 74. Ploug T , Sundby A , Moeslund TB , Holm S . population preferences for performance and explainability of artificial intelligence in health care: choice-based conjoint survey . J Med Internet Res 2021. ; 23 ( 12 ): e26611 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75. Ghassemi M , Oakden-Rayner L , Beam AL . The false hope of current approaches to explainable artificial intelligence in health care . Lancet Digit Health 2021. ; 3 ( 11 ): e745 – e750 . [DOI] [PubMed] [Google Scholar]