transposed convolution
In plain wordsAn operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image.
A learnable upsampling layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage. whose forward pass is the backward (gradient) pass of an ordinary convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location. with the same filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture. shape; equivalently, it inserts s − 1 zeros between input values and then convolves. It increases spatial resolution by roughly the stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result. factor and is used in the decoders of autoencodersautoencoder, n.: A neural network trained to squeeze its input into a compact code and then rebuild the original from that code. Learning to compress this way forces it to capture what matters in the data., in GANgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people. generators such as DCGAN, and in segmentation networks such as fully convolutional networks and U-NetU-Net, prop. n.: An image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost.. When the kernel size is not divisible by the stride, overlapping contributions produce 'checkerboard artifacts', so many designs instead upsample by nearest-neighbor or bilinear interpolation followed by an ordinary convolution, or use sub-pixel (pixel-shuffle) convolution.
The decoder doubles the resolution four times with stride-2 transposed convolutions.
Origin
Matthew Zeiler and colleagues' 'deconvolutional networks' (2010) and Zeiler and Rob Fergus's visualization work (2013) spread the misleading name 'deconvolution'. Jonathan Long, Evan Shelhamer and Trevor Darrell's fully convolutional networks (2014) used it for learnable upsampling in segmentation; Vincent Dumoulin and Francesco Visin's convolution-arithmetic guide (2016) promoted 'transposed convolution', and Augustus Odena and colleagues described checkerboard artifacts the same year.
Usage note
Not a true deconvolution: it does not invert a convolution, which is why the older name is discouraged.
Related
- convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.
- U-NetU-Net, prop. n.: An image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost.
- autoencoderautoencoder, n.: A neural network trained to squeeze its input into a compact code and then rebuild the original from that code. Learning to compress this way forces it to capture what matters in the data.
- generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people.
- stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result.
- super-resolutionsuper-resolution, n.: Making a small or blurry image larger and sharper by having AI fill in plausible fine detail, rather than just stretching the pixels.
Mentioned in
convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location., stridestride, n.: How far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result., U-NetU-Net, prop. n.: An image network shaped like the letter U: one half shrinks the picture to understand it, the other enlarges it back, with shortcuts between the halves so fine detail is not lost., fully convolutional networkfully convolutional network, n.: An image network made only of scanning-window layers, with no fixed-size final layers, so it can take pictures of any size and label every pixel.