DenseNet
In plain wordsAn image-recognition network in which every layer receives the outputs of all the layers before it, so features are reused rather than relearned.
A convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. architecture whose dense blocks wire every layer straight to every later layer in the same block: each layer's input is the channel-wise concatenation of the feature mapsfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears. of all layers before it, which for L layers makes L(L + 1)/2 direct connections. Because each layer contributes only a few new channels to this shared pool (the 'growth rate', such as 32), earlier features are reused rather than recomputed; between blocks, transition layers with 1 × 1 convolutions and pooling compress and downsample. Where ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI. adds a block's input to its output, DenseNet concatenates; the design reached strong accuracy with comparatively few parameters and passes gradients easily, but storing all the concatenated activations takes much memory. DenseNet-121 became a common backbone in medical imaging.
We used an ImageNet-pretrained DenseNet-121, the usual choice for chest X-ray classifiers.
Origin
Introduced by Gao Huang, Zhuang Liu, Laurens van der Maaten and Kilian Weinberger in 'Densely Connected Convolutional Networks' (arXiv, August 2016), which won a best-paper award at CVPR 2017.
Related
- ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI.
- residual connectionresidual connection, n.: A shortcut in a neural network that adds a block's input straight to its output, so each block only has to learn a correction. It is what makes networks hundreds of layers deep trainable.
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.
- U-NetU-Net, prop. n.: An image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost.
Sources
- Huang, G., Liu, Z., van der Maaten, L. & Weinberger, K. Q. (2016). Densely Connected Convolutional Networks. arXiv:1608.06993; CVPR 2017. arxiv.org/abs/1608.06993