Chest Radiographs: A New Form of Identification?

See also the article by Macpherson et al in this issue.

Vineet K. Raghu, PhD, is an instructor of radiology at Massachusetts General Hospital and Harvard Medical School. He is interested in developing artificial intelligence tools to identify individuals at high risk of age-related chronic diseases and in combining multi-omics data with medical imaging to better understand disease etiology and biomarkers of risk.

Michael T. Lu, MD, MPH, is director of AI and co-director of the Massachusetts General Hospital (MGH) Cardiovascular Imaging Research Center (CIRC), associate chair of imaging science for the MGH Department of Radiology, and associate professor of radiology at Harvard Medical School.

Deep learning continues to achieve remarkable results in medical image interpretation (1), even approaching the performance of radiologists on some narrowly defined tasks (2). These tools have had particular success with chest radiographs, in part because chest radiography is the most common diagnostic imaging test (3) and because of the availability of large public chest radiograph datasets for model development and testing (1). More recently, deep learning has been applied to chest radiographs for tasks not typically performed by radiologists, such as measuring cardiac function (4) and long-term disease risk prediction (5,6).

In this issue of Radiology: Artificial Intelligence, Macpherson and colleagues build on the work of Packhäuser et al (7) by using deep learning to address another nonstandard task, identifying chest radiographs that belong to the same patient, which they call patient reidentification (8). Their model works by learning to “compress” the original chest radiograph into a small set of numbers called a latent representation in such a way that radiographs from the same patient have similar representations, but radiographs from different patients have dissimilar representations. They built the reidentification model using more than 1 million radiographs from more than 250 000 patients seen at six UK medical centers. To support reproducibility, the authors evaluated the model on three publicly available chest radiograph datasets and found the following:

1. Given an input radiograph, the most similar radiograph according to the model belonged to the same patient 87%–95% of the time.

2. Given two radiographs as input, the model could discern whether the radiographs were from the same patient with greater than 90% sensitivity and specificity.

3. A change in the predicted representation over time for a single patient was associated with the presence of at least one acute abnormality on the follow-up radiograph. The area under the receiver operating characteristic curve of 0.73 was greater than baseline approaches.

Similar to the model reported in Packhäuser et al, the most immediate application of this technology may be in data harmonization to ensure that the same patient does not appear in several different datasets, especially in anonymized, public datasets such as those used in this study. These results also have major implications for anonymization itself: Anonymized chest radiographs could be reidentified by a deep learning model with access to reference radiographs.

An additional novelty of this study is that longitudinal changes in the model-predicted representation is associated with the presence of acute findings—such as lung nodules, hyperexpansion, calcification, and bone fractures—better than models trained to predict patient age. However, the poor performance of age predictors in this context is unsurprising given that these models are trained to be robust to acute, transient changes independent of age. More interestingly, the authors show that the longitudinal change in the model-predicted representation has added value beyond a single time-point representation to predict abnormalities. This result may suggest that the current paradigm of applying deep learning models to a single image may be improved by incorporating a patient’s prior imaging studies, similar to how a radiologist might use growth of a lung nodule in addition to its current size to determine malignancy risk. However, the authors found highly variable performance in using the longitudinal change to predict specific findings, and this approach is limited in that it can only be applied to patients with multiple radiographs. This requirement may bias the applicable population toward sicker patients.

A major challenge in applying deep learning to health care is its “black box” nature (9), where it is unclear which aspects of the image are being used to make the final prediction. Here, the authors incorporated a generative artificial intelligence method to give insight into what their reidentification model “sees” on the radiograph. The identified features seem reasonable, including sex-related findings, aortic knob size, lung volumes, and body habitus. For broad tasks such as patient reidentification and long-term outcome prediction, this type of approach may be more useful than heat map-based interpretability approaches, which may be better suited for localized pathologic conditions such as lung nodules (10). It will be interesting to see whether these types of interpretability techniques ultimately improve trust and understanding of artificial intelligence–based medical image interpretation systems.

The authors’ findings suggest several avenues for further research. Here the authors used the reidentification model’s features to predict acute abnormalities. Can these features be repurposed to predict disease risk or patient prognosis? The authors compared their approach with a model trained to predict patient age and a contrastive learning method, but comparisons with other self- and semisupervised approaches (11) on a variety of tasks could help guide practitioners to effectively leverage large publicly available chest radiograph databases with a limited set of associated data. Last, the radiographs used in this study could be any number of years apart; incorporating the elapsed time between studies may better contextualize the importance of longitudinal changes in the model-predicted representation.

Overall, this interesting addition to the literature validates a model to reidentify patients from chest radiographs and shows that this approach can be useful as a self-supervised learning technique to predict acute abnormalities. This study adds to the body of evidence showing that deep learning can identify interesting features from chest radiographs, even without labeled outcomes during training.