stride
In plain wordsHow far a scanning window jumps each time it moves across an image. Jumping two pixels instead of one halves the size of the result.
The step size, in input positions, between successive applications of a convolutionalconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location. filterfilter, n.: In an image-recognition network, a small grid of learned numbers that acts as a pattern detector, for vertical edges, a patch of fur or a wheel, scanned across the whole picture. or poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions. window. With stride s the spatial output is about 1/s of the input size in each dimension: for input length n, kernel size k and padding p, the output length is ⌊(n + 2p − k)/s⌋ + 1. Strides greater than 1 downsample feature mapsfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears., enlarging later receptive fieldsreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to. and reducing computation; strided convolutions are a learnable alternative to pooling and are standard in ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI.-style networks. Padding, the border values (usually zeros) added around the input, controls whether output size is preserved ('same') or shrinks ('valid'). In a vision transformervision transformer, n.: A way of applying the transformer, the design behind chatbots, to pictures: the image is cut into small square tiles that the model reads like words in a sentence., splitting an image into patches is a convolution whose stride equals its kernel size.
Use stride 2 in the first convolution of each stage instead of a pooling layer.
Origin
Subsampling by regular steps appears in Kunihiko Fukushima's neocognitron (1980) and LeNet-style networks; the word comes from array indexing and signal processing. Jost Tobias Springenberg and colleagues' 'Striving for Simplicity: The All Convolutional Net' (2014) showed that strided convolutions can replace pooling entirely.
Related
- convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.
- poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions.
- receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to.
- feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears.
- dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost.
- transposed convolutiontransposed convolution, n.: An operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image.
Mentioned in
convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location., poolingpooling, n.: A step in an image network that shrinks its internal picture by summarizing each small patch with one number, usually the largest or the average, keeping what matters and discarding exact positions., receptive fieldreceptive field, n.: The patch of the original input, such as a region of an image, that a particular neuron inside a network can see and respond to., ResNetResNet, prop. n.: A family of image-recognition networks from 2015 whose shortcut connections allowed far deeper networks than before, over 100 layers, and became a standard building block across AI., ConvNeXtConvNeXt, prop. n.: A 2022 image-recognition network showing that an older-style convolutional design, updated with lessons from transformers, can match vision transformers., dilated convolutiondilated convolution, n.: A version of the scanning-window operation that spreads its weights apart with gaps, so it can take in a wider area of an image or sound at no extra cost., transposed convolutiontransposed convolution, n.: An operation that roughly reverses an image network's shrinking steps, enlarging a small grid of features into a bigger one, as when a model paints a full-size image., VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.