Artificial Intelligence and Chest Imaging. Will Deep Learning Make Us Smarter?

In 2015, approximately 800 million multislice imaging studies were performed in the United States alone (1). With data of this magnitude comes large but as yet largely untapped potential. Chest computed tomography (CT) scans are increasingly ordered in smokers with or at risk for chronic obstructive pulmonary disease (COPD) to screen for lung cancer, evaluate pulmonary nodules detected on chest radiographs, assess for concurrent interstitial lung disease, and plan for lung transplantation and lung volume reduction surgery (2). However, despite the growing use and widespread availability of chest CTs, the wealth of data they contain is not consistently used in routine practice and has not yet been incorporated into clinical guidelines for COPD diagnosis, prognosis, or management.

Traditional analytic approaches to chest CT imaging remain primarily limited to the research setting and have focused on highly detailed and specific structural assessments. They include the quantification of emphysema by automated densitometry methods (3), measurement of airway wall thickness and lumen size (4), indirect assessment of nonemphysematous air trapping thought to relate to small airway abnormality (5), and characterization of pulmonary blood volume and flow (6). Through such approaches, important clinical associations have been identified between respiratory symptoms, exacerbations, and disease progression on one hand and both emphysema and airway abnormality on the other (7, 8). But there is another way.

Programs such as IBM Watson have a different vision for how such data could be used. Simply put, Watson is an artificial intelligence platform. One of its goals (beyond winning in popular television game shows such as Jeopardy!) is to improve medical care by using big data to optimize health system performance, engage patients, and manage population health. For example, in demonstration projects, Watson has been used to identify personalized treatments for lung cancer.

In this issue of the Journal, González and colleagues (pp. 193–203) use a deep machine learning approach called convolutional neural network (CNN) analysis and apply it to chest CTs from the COPDGene and ECLIPSE cohorts to determine whether this methodology could be used to detect and stage COPD, as well as predict acute respiratory events and mortality among smokers (9). More specifically, CNN analysis is a form of machine learning whose design is inspired by vision processing in living organisms. It incorporates all the available data, rather than focus on specific features of interest, and has been successfully used in image and video recognition (think Facebook’s automatic tagging algorithms and Google photo search), as well as natural language processing (10). Promising medical applications of CNN include the detection of diabetic retinopathy in retinal fundus photographs (11), the identification of tuberculosis on chest radiographs (12), and the classification of skin cancers based on clinical images (13). However, a key feature of CNN analysis is the requirement of a training process that teaches the computational model how to label the patterns it sees.

In this analysis, the authors used a montage of four imaging slices from the chest CTs of 7,983 COPDGene subjects to “train” the model and then applied the resulting algorithm to scans from another 1,000 COPDGene subjects as well as 1,672 ECLIPSE subjects. The authors found that the index of concordance (c-statistic) for detection of COPD based on the CNN prediction (defined as a predicted probability of >50%) was 0.856 in the COPDGene cohort. Furthermore, with respect to GOLD spirometric severity, about half of COPDGene participants were correctly staged (based on the GOLD stage with the highest predicted probability by CNN), and 75% were classified within one stage. Applying this algorithm to the ECLIPSE cohort, the performance was a bit poorer, as 29% of individuals were correctly staged and 75% were within one stage.

The ability of the CNN to predict acute respiratory events was also tested. COPDGene participants predicted by the model to be at risk for an event had 2.15 higher odds of having one compared with those predicted not to be at risk (c-statistic 0.64), but the model performed poorly in ECLIPSE, as it was unable to identify those at greater risk. With respect to mortality analysis, the CNN prediction was good in COPDGene when all subjects were considered (c-statistic 0.72), but performed less well when those with a diagnosis of COPD were specifically examined in both the COPDGene and ECLIPSE cohorts.

The authors noted several technical challenges that could ultimately affect the implementation of such an analytic approach on a large scale. Because of the substantial amount of data to be processed, it was impossible to examine all the slices from any one CT scan, and therefore a composite image of four representative cuts was made. The model improved with larger numbers of scans, and therefore several thousand images would be required for adequate training. One must also consider the potential effect of varying imaging protocols. The COPDGene study was performed more recently, using later-generation scanners and a different protocol than ECLIPSE (14, 15), which may explain some of the variance in predictive ability. However, the authors did perform additional analyses using two reconstruction algorithms performed in COPDGene and demonstrated comparable results. In addition, the effect of using “real life” scans on the performance of such deep learning algorithms is unclear. Although virtually all the chest CTs in patient cohort studies are obtained during clinical stability, many of the CTs in everyday practice are ordered for further evaluation of acute symptoms that could be related to infection, pulmonary embolism, or pulmonary edema among others.

The authors also attempted to make a comparison between deep learning analysis and percentage low attenuation area, a traditional emphysema quantification method, and concluded that the CNN performs better for respiratory event and mortality prediction. However, it is known that other potential valuable imaging features such as airway disease, air trapping, and pulmonary vascular abnormalities are also associated with clinical outcomes. Furthermore, if deep learning were to be employed for the purposes of predictive analytics at an individual patient level, it is likely that outcome prediction would be enhanced by the inclusion of other data in the medical record, such as demographic information, laboratory values, and known medical problems. Nevertheless, as the authors pointed out, there is also potential value for such analyses at the population level.

Imagine the ability to provide an estimate of COPD burden, exacerbation risk, and mortality within a given geographic unit such as a county or state. In this analysis, the authors demonstrate some of the immense potential of artificial intelligence when applied to chest CT images. Adoption of such methodologies at the practice level will ultimately depend both on the ability for such algorithms to be incorporated into standardized workflows, as well as their perceived value.