Explainable AI

Explainable AI develops methods to make machine learning model predictions interpretable by humans. It covers attribution techniques, model-agnostic explanations, inherently interpretable models, and the trust, debugging, and compliance needs that drive the requirement to understand why a model produces a given output.

itArtificial intelligence and machine learning

Don't Panic — Explainable AI

Explainable AI, usually called XAI, is what you add when a machine learning model produces a score or label and somebody has to ask why. The model may be accurate. It may also have the conversational skills of a locked filing cabinet. A number alone does not let a developer debug a failure, an auditor check for unfair treatment, or a person understand a decision that affects them.

The important split is between intrinsic interpretability and post-hoc explanation. A linear regression or shallow decision tree shows much of its reasoning in its own coefficients or branches. A complex ensemble or neural network generally does not, so a post-hoc method studies its inputs and outputs after training. That method is not a tiny detective living inside the model; it is a separate calculation describing the model's behavior.

Two more labels keep the furniture from being rearranged in your head. A local explanation answers why one particular prediction happened. A global explanation describes behavior across many predictions. A method can also be model-specific, using internals such as gradients, or model-agnostic, working from inputs and outputs alone. These pairs are not rival teams. They answer different questions, which is fortunate because a model can look rather different close up.

LIME perturbs an input and fits a small local model around it. SHAP assigns feature credit using Shapley values from game theory. A counterfactual asks what smallest change would have flipped the result. Each can be useful, but none proves the model is correct or that a feature caused something in the outside world. An explanation can faithfully reveal biased reasoning. It does not remove the bias, because charts lack that sort of administrative authority.

Start with the Intro when you need the full map: black boxes, local and global scope, and the limits of explanations. The Slides make the relationships quicker to scan. The Cheatsheet compares LIME, SHAP, counterfactuals, and the older global tools. Then use the Practice tab to keep the prediction fixed while testing an explanation's baseline, because the answer to compared with what is doing more work than it first appears.

Don't Panic — Explainable AI

Relevant careers