guided backpropagation

A visualization method for convolutional networksconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. that computes a modified gradient of a unit's or class score's activation with respect to the input image, zeroing negative values at every ReLUReLU, abbr.: The most common rule a neuron uses to produce its output: keep the number if it is positive, replace it with zero if it is negative. Simple, fast and surprisingly effective. during the backward pass — both where the forward activation was negative, as in ordinary backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction., and where the backward signal itself is negative, as in the earlier deconvnet method. Keeping only positive contributions yields crisp, high-resolution images of the edges and textures that excite a unit. The maps are not class-discriminative — different classes give nearly identical images — which is why Grad-CAMGrad-CAM, abbr.: A technique that shows, as a coarse heat map, which regions of an image a vision network relied on when choosing a particular label. multiplies them with coarse class maps to form Guided Grad-CAM. Later analyses showed that guided backpropagation largely reconstructs the input image rather than revealing the model's decision: its maps barely change when the network's upper-layer weights are randomized.

Guided backpropagation produced a beautiful outline of the cat, which should have made us suspicious.