Algorithm 2. Feature selection for providing explanations

Input: userSelectedFeatures: user-selected features,

shapValue: SHAP values for all features (see Algorithm 1),

features: list of all features

Output: relevantFeatures: relevant feature list

1: $\mathtt{avgShapVal}\leftarrow \overline{\mathtt{shapValue}}$

2: $\mathtt{shapFeatures}\leftarrow \left\{\right\}$

3: for  $\mathtt{feature}\in \mathtt{features}$  do

4:    if $\mathtt{shapValue}\left[\mathtt{feature}\right]>\mathtt{avgShapVal}$ then

5:     $\mathtt{shapFeatures}\leftarrow \mathtt{feature}$

6:    end if

7: end for

8: $\mathtt{relevantFeatures}\leftarrow \mathtt{shapFeatures}\cup \mathtt{userSelectedFeatures}$

9: return relevantFeatures