Figure 1.

Overview of the proposed S‐XAI. a) Our framework for extracting common traits from a dataset, taking the category of cats as an example. Left: the original samples and discovered best combinations of superpixels. Middle: extracting feature maps for all samples from a pretrained CNN. Right: the obtained principal components (PCs) from the row‐centered sample compression on the feature maps and visualization of common traits. b) Our framework for extracting understandable semantic space, taking the semantic space of cats’ eyes as an example. Left: samples with unmasked and masked semantic concept. Middle: extraction of common traits for both kinds of samples. Right: discovered semantically sensitive neurons (SSNs) and the visualization of the semantic space. The big bright eyes are vividly illustrated, which proves that an understandable semantic space is found. c) The workflow of CNN and S‐XAI. The blue part is the prediction process of the CNN. The red part is the process of S‐XAI, in which the semantic probabilities are calculated from the extracted semantic spaces that can be visualized and recognized by humans for trustworthiness assessment and semantic sample searching. The dashed box refers to an optional step.