DeepLIFT
In plain wordsA method that explains a neural network's output by comparing each internal signal with its value on a neutral reference input, then passing the differences back to the inputs as credit.
Deep Learning Important FeaTures: a feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down. method that accounts for how far a network's output moves away from its value on a reference (baseline) input, by sending each neuron's departure from its own reference activation, rather than a gradient, back toward the input features. Multipliers obeying a summation-to-delta property split each departure among a neuron's inputs, so the input contributions add up to the output's total change, and a chain rule for these multipliers lets everything be computed in one backward pass. Because it uses finite differences, DeepLIFT avoids the saturation problem of gradients — a neuron in a flat region of its activation function still passes on credit — and handles thresholds. Its Rescale rule approximates integrated gradientsintegrated gradients, n.: A way to score how much each input feature contributed to a neural network's output by gradually morphing a blank 'baseline' input into the real one and adding up the network's sensitivity along the way. cheaply, while its RevealCancel rule treats positive and negative contributions separately. DeepSHAP, part of SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players., builds on it by averaging over many references.
DeepLIFT scores picked out the binding motif in the DNA sequence that drove the prediction.
Origin
Introduced by Avanti Shrikumar, Peyton Greenside and Anshul Kundaje of Stanford University, first in a 2016 preprint and then in 'Learning Important Features Through Propagating Activation Differences' (ICML 2017); it was developed with genomics models in mind.
Related
- feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down.
- integrated gradientsintegrated gradients, n.: A way to score how much each input feature contributed to a neural network's output by gradually morphing a blank 'baseline' input into the real one and adding up the network's sensitivity along the way.
- layer-wise relevance propagationlayer-wise relevance propagation, n.: A method that traces a neural network's decision backward, layer by layer, handing each part of the input its share of the credit until every pixel or word has a relevance score.
- SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players.
- saliency mapsaliency map, n.: A heat map laid over an image or text that highlights which parts most influenced an AI model's output — or, in vision science, which parts of a scene would most catch a viewer's eye.
Sources
- Shrikumar, A., Greenside, P. & Kundaje, A. (2017). Learning Important Features Through Propagating Activation Differences. Proceedings of ICML 2017.
Mentioned in
feature attributionfeature attribution, n.: Giving each input to an AI model — each pixel, word or data column — a score for how much it pushed a particular output up or down., integrated gradientsintegrated gradients, n.: A way to score how much each input feature contributed to a neural network's output by gradually morphing a blank 'baseline' input into the real one and adding up the network's sensitivity along the way., SHAPSHAP, abbr.: A popular method that explains a single prediction by fairly dividing it among the inputs, using a rule from game theory for splitting a team's winnings among its players., layer-wise relevance propagationlayer-wise relevance propagation, n.: A method that traces a neural network's decision backward, layer by layer, handing each part of the input its share of the credit until every pixel or word has a relevance score.