Skip to main content
. Author manuscript; available in PMC: 2021 Oct 1.
Published in final edited form as: Med Image Comput Comput Assist Interv. 2020 Sep 29;12262:529–539. doi: 10.1007/978-3-030-59713-9_51

Fig. 2:

Fig. 2:

Joint model visualization. Top to bottoni: (Level 1) The lhghlight of the Grad-CAM image is centered around the right hilar region, which is consistent with findings in pulmonary vascular congestion as shown in the report. (Level 2) The lhghlight of the Grad-CAM image is centered around the left hilar region which shows radiating interstitial markings as confirmed by the report heatmap. (Level 3) Grad-CAM liighlights bilateral alveolar opacities radiating out from the hila and sparing the outer lungs. This pattern is classically described as “batwing” pulmonary edema mentioned in the report. The report text is presented in the form of sub-word tokenization performed by the BERT model, starting the report with a [CLS] token and ending with a [SEP].