What is Explainable AI? Concepts & Examples

Explainable AI - SHAP Explainability

Explainable AI is an important part of artificial intelligence. It provides humans with the ability to explain how decisions are made by machines. This helps people trust and understand what’s happening, instead of feeling like their information is being taken advantage of or used without their permission. In this blog post, we’ll explain explainable AI and explain it with examples so you know how it works!

What is Explainable AI and how does it work?

Explainable AI is defined as AI systems that explain their reasoning behind the prediction. Explainable AI is part of the larger umbrella term for artificial intelligence known as “interpretability.” Explainable AI is a type of AI system that is modeled after how humans think and make decisions. Explainable AI Systems can be helpful when trying to understand the reasoning behind a particular prediction or decision made by machine learning models. Explainable AI Systems could also be helpful for situations involving accountability, such as with autonomous vehicles; if something goes wrong with explainable AI, human is still accountable for their actions. The explainable AI models are trained using concepts from explainability techniques, which use human-readable textual descriptions to explain the reasoning behind a model’s prediction. Today, explainability techniques are used in many different areas of artificial intelligence such as natural language processing (NLP), computer vision, medical imaging, health informatics and others. Some of the universities where explainable AI research is being carried out are MIT, Carnegie Mellon University and Harvard.

The key difference between AI and explainable AI is that explainable AI is a type of artificial intelligence that has explanations for its decisions. The explainability techniques used in explainable AI are heavily influenced by how humans make inferences and form conclusions, which allows them to be replicated within an explainable artificial intelligence system.

Explainable AI can help explain the reasoning behind a machine learning model’s decision trained using any of the algorithms such as linear regression, random forest, decision trees, etc. This is one of the most common explainability techniques used in practice and there are many tools out there that provide this functionality. For example, SHapley Additive exPlanations (SHAP) and LIME are explainability tools that allow you to explain the decisions made by a machine learning model using local interpretations.

Explainable Artificial Intelligence Systems can be built using different techniques that focus on the reasoning through the explanation of data or prediction by explaining decisions made by an artificial intelligence algorithm. Some common explainability techniques include:

  • Decomposition (breaking down a model into individual components)
  • Visualization (building explainability features to explain model predictions in a visual way)
  • Explanation Mining (using machine learning techniques for finding relevant data that explains the prediction of an artificial intelligence algorithm).

Benefits & Challenges of Explainable AI

Explainable AI forms a key aspect of ethical AI. Here are the key benefits of using explainable AI:

  • Explainable AI is desired in use cases involving accountability. For example, explainable AI could help create autonomous vehicles that are able to explain their decisions in the case of an accident.
  • Explainable AI is critical for situations involving fairness and transparency where there are scenarios with sensitive information or data associated with it (i.e., healthcare)
  • Enhanced trust between humans and machines
  • Higher visibility into model decision-making process (which helps with transparency)

Here are some challenges related to explainable AI:

  • Explainable AI is a relatively new area of research and there are still many active challenges that explainable models present today. One challenge is explainability can be at the expense of model performance accuracy, since explainable artificial intelligence systems tend to have lower performance compared to non-explainable models or black box models.
  • Another key challenge with explainable artificial intelligence is explainable AI models can be more difficult to train and tune compared to non-explainable machine learning models.
  • Another challenge is explainable AI systems can be more difficult to deploy since explainability features typically need some level of human-in-the-loop intervention.

Why is Explainable AI important for the future?

AI has its downsides such as bias and unfairness. These are going to create trust issues with AI in time to come. Explainability and explainable AI is a way of mitigating these challenges. This is where explainability techniques are gaining momentum fast since they will likely lead to better human-machine interaction, more responsible technologies (such as autonomous vehicles) and increased trust between machines and humans. Explaining decisions made by artificial intelligence systems can help provide transparency on how the model arrives at its decision. For example, explainable AI could be used to explain an autonomous vehicles reasoning on why it decided not to stop or slow down before hitting a pedestrian crossing the street.

Explainable AI is an important part of future of AI because explainable artificial intelligence models explain the reasoning behind their decisions. This provides an increased level of understanding between humans and machines, which can help build trust in AI systems.

Examples of explainable AI in action

Here is an example of explainability of model prediction where Boston housing price prediction model got trained using XGBoost algorithm. The explainability is achieved using SHAP. The explainability shown below represents features each contributing to push the model output from the base value (the average model output over the training dataset we passed) to the model output. Features pushing the prediction higher are shown in red, those pushing the prediction lower are in blue. Another way to visualize the same explanation is to use a force plot (these are introduced in our Nature BME paper):

Explainable AI - SHAP Explainability

Here are some use cases where explainable AI can be used:

  • Healthcare: When diagnosing patients with disease, explainable AI can explain their diagnosis. It can help doctors explain their diagnosis to patients and explain how a treatment plan is going to help. This will help create greater trust between patients and their doctors while mitigating any potential ethical issues. One of the example where AI predictions can explain their decisions might involve diagnosing patients with pneumonia. Another example where explainable AI can be extremely useful is in healthcare with medical imaging data for diagnosing cancer.
  • Manufacturing: Explainable AI could be used to explain why an assembly line is not working properly and how it needs adjustment over time. This is important for improved machine-to-machine communication and understanding, which will help create greater situational awareness between humans and machines.
  • Defense: Explainable AI can be useful for military training applications to explain the reasoning behind a decision made by an artificial intelligence system (i.e., autonomous vehicles). This is important because it helps mitigate potential ethical challenges such as why it misidentified an object or did not fire on a target.
  • Autonomous vehicles: Explainable AI is becoming increasingly important in the automotive industry due to highly publicized events involving accidents caused by autonomous vehicles (such as Uber’s fatal crash with a pedestrian). This has placed an emphasis on explainability techniques for AI algorithms, especially when it comes to use cases that involve safety-critical decisions. Explainable AI can be used for autonomous vehicles where explainability provides increased situational awareness in accidents or unexpected situations, which could lead to more responsible technology operation (i.e., preventing crashes).
  • Loan approvals: explainable artificial intelligence can be used to explain why a loan was approved or denied. This is important because it helps mitigate any potential ethical challenges by providing an increased level of understanding between humans and machines, which will help create greater trust in AI systems.
  • Resume screening: explainable artificial intelligence could be used to explain why a resume was selected or not. This provides an increased level of understanding between humans and machines, which helps create greater trust in AI systems while mitigating issues related to bias and unfairness.
  • Fraud detection: Explainable AI is important for fraud detection in financial services. This can be used to explain why a transaction was flagged as suspicious or legitimate, which helps mitigate potential ethical challenges associated with unfair bias and discrimination issues when it comes to identifying fraudulent transactions.

Some interview questions on Explainable AI

Here are some interview questions you would like to be aware of, when going for AI / machine learning interviews:

  • What is explainable AI? What is explainability in artificial intelligence?
  • What are some explainable AI use cases?
  • How can explainable AI be used for fraud detection?
  • Why would someone want to explain the reasoning behind an autonomous vehicle’s decision making process?
  • Why would a company need explainable Artificial Intelligence solutions at all times, not just during reviews and audits?
  • Why is explainable artificial intelligence important for trust? What ethical issues does explainable artificial intelligence address?
  • What are some frameworks for explainable artificial intelligence?
  • What are some explainable AI examples which involve medical imaging data for diagnosing cancer?
  • Name at least three explainable artificial intelligence use cases.

If explainability is very important for your business decisions, explainable AI should be a key consideration in your analytics strategy. With explainable AI, you can provide transparency on how decisions are made by AI systems and help build trust between humans and machines. The use cases where explainable AI has been used include healthcare (diagnoses), manufacturing (assembly lines) and defense (military training). If these sound like areas of interest to you or if this content piqued your curiosity about explainable AI in general, we’d love to hear from you! We have experience consulting with organizations interested in incorporating explainability into their products/services – let us know what’s keeping you up at night so that we can offer our expertise to create an explainable artificial intelligence solution that’s right for your organization.

Ajitesh Kumar
Follow me
Latest posts by Ajitesh Kumar (see all)

Ajitesh Kumar

I have been recently working in the area of Data Science and Machine Learning / Deep Learning. In addition, I am also passionate about various different technologies including programming languages such as Java/JEE, Javascript, Python, R, Julia etc and technologies such as Blockchain, mobile computing, cloud-native technologies, application security, cloud computing platforms, big data etc. I would love to connect with you on Linkedin and Twitter.
Posted in AI. Tagged with , .

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.