Concerns in the use of adversarial learning for image synthesis in cardiovascular intervention

We read with great interest the article by Olender et al.¹ reporting the novel platform to synthesize realistic intravascular images. In this article, the authors depict the various benefits of medical image synthesis, by showing actual optical coherence tomography (OCT) images generated from conditional generative adversarial networks (GANs). We agree in many respects with the perspective presented by Dr Olender and colleagues, because we have also reported on the generation of cardiac intravascular images using GAN.² However, based on the difficulties we have experienced, we would like to pose some questions to the authors.

The first question is how we should evaluate the validity of generated images. As GANs are unsupervised learning algorithms, there is no absolute measure to evaluate the quality of image synthesis.³ The visual Turing test can be used to evaluate the realism of an image,⁴ but it does not guarantee that the image is pathologically correct. If the purpose of the artificial intelligence (AI) model is to see how the image changes according to the addition of auxiliary variables, supervised image synthesis, like we have previously reported,⁵ should be a more straightforward approach.

Next question is on the scalability of the authors’ method. Considering the differences in physical properties and image resolution between intravascular ultrasound and OCT, image translation between them is not an easy task and will require an extremely large amount of data to ensure pathological consistency. Nevertheless, given the similarity of the scanning methods of the two modalities, it may be relatively feasible as described by the authors. Therefore, it would be interesting for us to see if translation is possible among more different modalities such as coronary angioscopy. Finally, we would like to ask the authors’ opinion about the risk of misdiagnosis caused by AI-generated fake images. In generative models using adversarial loss, there is a concern that the features of the generated image are greatly affected by inherent bias of the training data.⁶

Like many researchers, we also believe that the use of deep learning will be essential in the cardiovascular field. However, before such technology can be used in clinical practice, the above concerns need to be addressed.

Conflict of interest: none declared.

Data availability

The data underlying this article are available in the article.