Explainable AI using SHAP: As if explained to a 10 year old

Let’s say you are training a model with a huge dataset (Image, text or numerical/categorical features). Once the training is complete, you will be have the following output in hand: The training & Validation accuracy, the prediction values from your test set & even feature importance (a beautiful graph that gives how important is each feature to the model based on different attributes like Gain, permutation etc.,)

Now, what you don’t have is, how did that model make these decisions. How does the variable Age have more weightage than the variable Salary?

Photo by Chris Malinao Burgett on Unsplash

For many years, these questions have remained a mystery in the Black Box called ML models.

With the booming development of a new field called the Explainable AI, there has been several explorations to explain how AI models make decisions. One such useful yet simple solution is the SHAP Values.

Photo by Rohan Makhecha on Unsplash

SHAP which is short for SHapley Additive exPlanations values is a way to explain the output of a machine learning model. Think of a machine learning model as a magic box that takes in numbers and spits out an answer. The SHAP values tell us which numbers (or features) are most important in making the magic box give us the response it did.

Photo by Elena Mozhvilo on Unsplash

Let’s say we have a magic box that can tell us what kind of animal is in a picture. We give it a picture of a dog and it tells us the animal in the picture is a dog. The SHAP values would tell us which parts of the picture (like the ears or the tail) helped the magic box decide that the animal in the picture is a dog.

Photo by charlesdeluvio on Unsplash

It’s also worth mentioning that SHAP values are based on the concept of Shapley values from cooperative game theory. Shapley values assign a value to each player in a cooperative game based on their contribution to the total value of the game. The SHAP values use a similar method to assign a value to each feature in a machine-learning model.

For instance, imagine that you are playing a game with your friends and everyone is contributing to a common pot. Each feature is like one of your friends, and the game is to predict the outcome. SHAP values are like the money each friend contributes to the pot. More the money, the more important the friend is in the game.

Photo by Erik Mclean on Unsplash

SHAP values are unique in that they take into account both the feature’s impact on the prediction and its relationship with other features. This means that if two features are highly correlated, the SHAP values will show how much each feature contributes independently to the prediction.

Another great feature of SHAP values is that they can be used for both classification and regression models, and can be easily integrated into existing workflows. This makes it easy to use SHAP values to explain the predictions of any machine learning model you’re working with.

Also, SHAP values have a built-in interpretation as feature importances. The sum of the absolute values of the SHAP values for each feature is equivalent to the feature importances obtained through a feature permutation test. This means that you can use SHAP values to compare the relative importance of different features in the same way you would use feature importances.

Another important aspect of SHAP values is that they are consistent with the no-free-lunch theorem. This means that the sum of the absolute SHAP values for all features is equal to the absolute difference between the base value (expected value) of the model and the actual value of the prediction. This means that SHAP values provide a global explanation for the model’s prediction, and can be used to understand the overall impact of the features on the prediction.

In practice, there might be situations where the model is not performing well, and in such cases, SHAP values can be used to identify the features that are causing the poor performance. By identifying the features that are having the largest impact on the model’s predictions, it is possible to understand the source of the poor performance and make adjustments to the model accordingly.

How to implement SHAP using Python?

Let’s first begin by installing the SHAP library.

# Installing the SHAP Package
pip install shap

Using the dataset: http://lib.stat.cmu.edu/datasets/boston

import xgboost

# train an XGBoost model
X, y = shap.datasets.boston()
model = xgboost.XGBRegressor().fit(X, y)

Upon fitting the XGBoost model, let’s then explain the decisions using SHAP.

import shap
# explain the model's predictions using SHAP
# (same syntax works for LightGBM, CatBoost, scikit-learn, transformers, Spark, etc.)
explainer = shap.Explainer(model)
shap_values = explainer(X)

There are different visualization methods that you can use to understand your model.

# visualize the first prediction's explanation
shap.plots.waterfall(shap_values[0])

Image by Author

# visualize the first prediction's explanation with a force plot
shap.plots.force(shap_values[0])

Image by Author

# visualize all the training set predictions
shap.plots.force(shap_values)

GIF by Author

# create a dependence scatter plot to show the effect of a single feature across the whole dataset
shap.plots.scatter(shap_values[:,"RM"], color=shap_values)

Image by Author

# summarize the effects of all the features
shap.plots.beeswarm(shap_values)

Image by Author

#A Simple bar plot
shap.plots.bar(shap_values)

Image by Author

shap_interaction_values = shap.TreeExplainer(model).shap_interaction_values(X.iloc[:2000,:])
shap.summary_plot(shap_interaction_values, X.iloc[:2000,:])

Image by Author

In conclusion, SHAP values are a powerful tool that can help us understand how a machine learning model is making its predictions. They take into account the relationship between features, can be used for both classification and regression models and can be visualized in several ways. With the help of SHAP, we can understand the contribution of each feature to the outcome and help us in interpreting the model.

The full implementation is available on my GitHub: https://github.com/arthi-rajendran-DS/Medium-Implementations

Thanks for sticking around until the end. Visit me on my Linkedin to have a more in-depth conversation or any questions.

If you liked this content, please follow me & subscribe to receive my updates directly to your email.

Also check out these other articles you might be interested in:

I built an Application using ChatGPT with less than 15 lines of code!

Your 5-step Guide to implementing MLOps

A Deep Dive into ChatGPT: How This AI Chatbot Works?