Inception score
In plain wordsAn early score for image generators that rewards pictures an image-recognition network can label confidently and a collection that covers many different labels; higher is better.
A metric for generated images computed with the Inception image classifier trained on ImageNet. For each generated image x the classifier gives a distribution over 1,000 classes, p(y | x); the score rewards confident, low-entropy predictions for individual images and a high-entropy marginal distribution p(y) across all images, and is defined as the exponential of the average Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right. between the two. It ranges from 1 to the number of classes. The Inception score needs no real images, which is also its weakness: it ignores whether generated images resemble the training data, cannot detect a model that produces one perfect image per class, and is meaningful mainly for ImageNet-like natural images. Widely reported for GANs from 2016, it has largely been superseded by Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better. and by human or preference-based evaluation.
Origin
Proposed by Tim Salimans, Ian Goodfellow and colleagues at OpenAI in 'Improved Techniques for Training GANs' (posted June 2016; NeurIPS 2016) as an automatic proxy that correlated with human judgments of sample quality.
Related
- Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better.
- CLIP scoreCLIP score, n.: A score for how well a picture matches a text description, computed by an AI model that was trained to pair images with their captions.
- generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people.
- Kullback–Leibler divergenceKullback–Leibler divergence, n.: A score for the cost of believing the wrong odds: on average, how much more surprised you will be by real outcomes than someone who knew the true probabilities. It is zero only when your odds are exactly right.
- human evaluationhuman evaluation, n.: Judging an AI's outputs by asking people to rate or compare them, because for many tasks — writing, conversation, translation — no automatic score captures quality well.
- Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better.
Sources
- Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A. & Chen, X. (2016). Improved Techniques for Training GANs. NeurIPS 2016. arXiv:1606.03498. arxiv.org/abs/1606.03498
- Barratt, S. & Sharma, R. (2018). A Note on the Inception Score. arXiv.
Mentioned in
Fréchet inception distanceFréchet inception distance, n.: A score for image generators that compares the overall statistics of their pictures with those of real photos, as seen by an image-recognition network; lower is better.