CT scan AI-aided triage for patients with COVID-19 in China

In The Lancet Digital Health, Minghuan Wang and colleagues¹ describe the development of an artificial intelligence (AI) algorithm using CT scans for patient triage at fever clinics in China. The authors trained a U-Net-based model on a dataset obtained from Tongji Hospital (Wuhan, China), using a development set containing CT scans for 2447 patients (1647 patients with RT-PCR-confirmed COVID-19 and 800 patients without COVID-19). After internal validation, the authors externally validated their model on datasets from three fever clinics consisting of 2120 patients. The model had high accuracy for the detection of radiological changes compared with radiological reports (reference standard). AI-aided triage achieved a sensitivity of 0·923 (95% CI 0·914–0·932), a specificity of 0·851 (0·842–0·860), a positive predictive value of 0·790 (0·777–0·803), a negative predictive value of 0·948 (0·941–0·954), and an area under the curve of 0·953 (0·949–0·959). Two additional external testing sets were used, which included 761 CT scans from 722 patients with RT-PCR-confirmed COVID-19 admitted to Guanggu Fangcang Hospital (Wuhan, China) and a retrospective dataset, which included 686 scans from 651 patients who visited Tianyou Hospital (Wuhan, China) or The Third People's Hospital of Shenzhen (Shenzhen, China) for respiratory diseases before the COVID-19 outbreak. These datasets were used to assess the performance of the AI algorithm in patients with mild COVID-19 symptoms (using RT-PCR as the reference standard), and in non-COVID-19 cases (to address the hypothesis that CT changes are not specific for COVID-19 [ie, similar CT findings for other respiratory infection or diseases]). Another interesting aspect of the study was the assessment of AI's ability to identify changes in CT opacity. AI also achieved a sensitivity of 0·962 and specificity of 0·875 for the identification of increases in lesion burden, with high agreement between the radiologist panel and AI (Cohen's kappa coefficient 0·839, 95% CI 0·718–0·940).

This study had several strengths. First, the development and internal validation sets included a large number of patients, a prerequisite for training an AI model. Second, several external validation sets were used, and the model was tested in three regions of China in populations with varied COVID-19 prevalence, including cohorts that predated the emergence of the virus. Third, the three external validation sets included CT scans for consecutive patients collected over a 2-week period. Fourth, the model was trained and validated on scans obtained using a number of different CT scanners made by different manufacturers.

The accuracy of AI for triaging was compared against a radiologist panel, rather than RT-PCR, which showed that the model accurately identified changes that had been identified by humans. When RT-PCR was used as the reference standard for identifying patients with COVID-19, the sensitivities and specificities of AI decreased. In settings with radiologist staffing shortages, AI could reduce reporting times, enabling expedited management of COVID-19 cases. The timeliness of patient management is crucial and would form part of wider preventative measures, especially in populations with high disease prevalence where health-care resources might be scarce. Such a model of CT-triaging has been effective in China, where new hospitals equipped with CT scanners were available or could be built quickly during the early stages of local COVID-19 outbreaks, when RT-PCR testing and rapid turnaround times were unavailable.

The use of CT in a more general context, as knowledge about the disease increases, is likely to change. For example, new evidence suggests that more than 50% of patients have no changes on CT during the initial phase of symptom onset,² and other studies have shown that RT-PCR might yield negative results despite typical CT findings.3, 4 This study was done at the beginning of the pandemic, when less information about the disease and associated CT changes was available. The utility of such systems also depends on the infrastructure of each country. The rapid construction of fever clinics with dedicated CT resources might not be feasible in other countries. Reliance on CT systems might increase the potential risk of cross infection between patients and health-care staff, although some automated scanning solutions have emerged that might mitigate risks to technicians and health-care workers.⁵ As knowledge and understanding of COVID-19 continues to evolve, CT scans might have additional uses. For example, identification and quantification of lung changes could prove useful since temporal changes have been shown to be associated with disease severity.⁶ CT scans might also help identify individuals at high risk of having COVID-19, and possibly having worse outcomes.7, 8 Volumetric quantification, although outside the scope of this Article, could potentially be useful for severity assessment and for serial monitoring of patients.⁹ However, future studies to investigate these clinically relevant outcomes are still needed.