prototypical part network
In plain wordsAn image classifier that explains each decision by pointing to parts of the picture that resemble typical examples it learned — this part looks like that part of a known bird.
An interpretableinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed. deep image classifier that reasons by comparison with learned prototypes, each a patch-level representation tied to part of a real training image, such as the head of a particular bird. A convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. backbone maps the input to a grid of patch embeddings; a prototype layer scores how similar each prototype is to the closest patch of the input; and a final linear layer turns these similarity scores into class logitslogit, n.: A raw score that a model gives each possible answer before the scores are turned into probabilities. In statistics, the 'log-odds': a probability rewritten on a scale running from minus to plus infinity.. After training, each prototype is projected onto its nearest training patch so it can be shown, and every prediction can be explained as 'this part of the image looks like that prototypical part', with each match's contribution. Tested on fine-grained recognition of bird species and car models, a single network roughly matched an otherwise identical uninterpretable one, and combining several came close to the strongest deep models then available. Critics note that similarity in the network's latent space need not match similarity as people perceive it.
ProtoPNet justified 'red-winged blackbird' by matching the wing patch to three prototypes from training photos.
Origin
Introduced by Chaofan Chen, Oscar Li, Chaofan Tao, Alina Jade Barnett, Jonathan Su and Cynthia Rudin, mostly of Duke University, in 'This Looks Like That: Deep Learning for Interpretable Image Recognition' (arXiv, June 2018; NeurIPS 2019), building on prototype-based case reasoning by Oscar Li and colleagues (2018).
Usage note
Part of the case-based-reasoning tradition in interpretable machine learning; successors such as ProtoTree and transformer-based variants refine the prototype layer.
Related
- interpretable modelinterpretable model, n.: A model simple or structured enough that a person can follow exactly how it turns inputs into a decision — like a short checklist or a points-based score — with no separate explanation tool needed.
- concept bottleneck modelconcept bottleneck model, n.: A model made to first predict human-understandable concepts — such as 'bone spur present' on an X-ray — and then decide only from those concepts, so people can inspect and correct them.
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- explainable AIexplainable AI, n.: The field and set of methods that let people understand why an AI system produced a particular output — for example, which factors led it to reject a loan application.
- post hoc explanationpost hoc explanation, n.: An explanation produced after a model has been built, by a separate tool that examines the finished model, rather than one that comes from a model designed to be understandable.
- post hoc explanationpost hoc explanation, n.: An explanation produced after a model has been built, by a separate tool that examines the finished model, rather than one that comes from a model designed to be understandable.
Sources
- Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J. & Rudin, C. (2019). This Looks Like That: Deep Learning for Interpretable Image Recognition. NeurIPS 2019.