Table 1.

Studies that describe methods for prediction explanation

Author (year)	Title	Description of method
Lundberg and Lee (2017)	A unified approach to interpreting model predictions30	Presents a unified framework for six prediction explanation methods. Also, proposes a new explanation method that outperforms prior methods in terms of computational complexity and reliability.
Krause et al (2016)	Interacting with predictions: visual inspection of black-box machine learning models31	Describes an interactive environment that enables the user to inspect a model’s prediction by tweaking feature values and observing the effect on the model’s behavior.
Luo (2016)	Automatically explaining machine learning prediction results: a demonstration on type 2 diabetes risk prediction18	Develops a rule-based model to explain the decision made by the prediction model.
Ribeiro et al (2016)	“Why should 1 trust you?”: Explaining the predictions of any classifier15	Proposes a post-hoc explanation method that generates data samples that are similar to the predicted sample, labels the samples by the predictive model, and fits a local linear model to the samples. Uses the weights in the local model to identify the influential features.
Baehrens et al (2009)	How to explain individual classification decisions32	Proposes a prediction explanation method that uses the gradient vector of the predictive model at the point of the predicted sample for measuring feature importance.
Sikonja and Kononenko (2008)	Explaining classifications for individual instances33	Explains a sample by assigning an importance factor to each sample’s feature. The importance factor of a feature is defined as the change in the model’s prediction on removal of the feature from the sample.
Štrumbelj and Kononenko (2008)	Toward a model independent method for explaining classification for individual instances34	Describes a model-independent explanation method for probabilistic classifiers. Calculates an importance weight for each feature by measuring the change in the class probability on removal of the feature from the conditional probability of the class given the sample features.
Lemaire et al (2008)	Contact personalization using a score understanding method35	Computes the influence of a feature by measuring the effect of changing the feature’s value on the model’s prediction.
Poulin et al (2006)	Visual explanation of evidence in additive classifiers36	Describes a framework to visualize each feature’s contribution to a prediction. Provides the capability to analyze the effect of changing feature values on a classifier’s decision. The method is applicable to additive models such as naive Bayes, and support vector machines.
Szafron et al (2003)	Explaining naïve Bayes classifications37	Provides a graphical explanation framework for naive Bayes predictions. For a sample, the framework visualizes each feature’s contribution to the decision made by the classifier.
Reggia and Perricone (1985)	Answer justification in medical decision support systems based on Bayesian classification11	Proposes an explanation method for Bayesian classifiers by using prior and likelihood values to determine important features responsible for the posterior probability of the outcome.