Convolutional Neural Networks Explained: Filters, Pooling, Stride, and Padding

A convolutional neural network (CNN) learns to see by sliding small learnable windows across an image, reusing the same weights at every position. That single idea, weight sharing, cuts parameters by orders of magnitude compared to a fully connected network and lets the model detect an edge or texture wherever it appears. Four hyperparameters control the whole mechanism: filter size, stride, padding, and pooling.

Quick Takeaways

• A filter (kernel) is a small weight grid, typically 3×3, that slides over the input and produces a feature map.
• Stride sets how far the filter moves per step; padding controls whether borders shrink the output.
• Pooling downsamples feature maps (commonly 2×2, stride 2), adding small translation invariance and cutting compute.
• Output size follows one formula: ⌊(W − F + 2P) / S⌋ + 1.

Why Fully Connected Layers Fail on Images

A 224×224×3 image has 150,528 input values. One dense layer with 1,000 neurons needs about 150 million weights. It also ignores spatial structure: shifting a cat by two pixels looks like a completely new input.

Convolution fixes both problems. It exploits two properties of images:

• Local connectivity: Nearby pixels correlate strongly, so each neuron looks at a small patch.
• Weight sharing: The same filter applies at every location, so an edge detector learned in one corner works everywhere.

How Convolution Works

A filter is a tensor of shape F × F × C_in. At each position, the network multiplies the filter element-wise with the underlying patch, sums the results, and adds a bias. Sliding the filter across the whole input yields one feature map. Using K filters yields K feature maps, which form the output channels.

Filters (Kernels)

Early-layer filters learn edges, color gradients, and simple textures. Mid-layer filters combine these into corners and patterns. Deep-layer filters respond to object parts such as eyes, wheels, or fur.

A classic hand-designed example is the Sobel filter for vertical edges:

[-1  0  1]
[-2  0  2]
[-1  0  1]

In a CNN, nobody hand-designs these values. Backpropagation learns them.

Why 3×3 dominates: VGG showed that two stacked 3×3 layers cover the same 5×5 receptive field as one 5×5 layer, with 18C² parameters instead of 25C², plus an extra non-linearity. Modern architectures follow this logic.

Parameter Count

For one conv layer:

Params = (F × F × C_in + 1) × K

Example: 64 filters of 3×3 on an RGB input gives (3 × 3 × 3 + 1) × 64 = 1,792 parameters. Compare that to 150 million for the dense layer above.

Stride: Controlling Step Size

Stride (S) is the number of pixels the filter moves each step.

• S = 1: Dense sampling. Output stays high resolution.
• S = 2: Output height and width roughly halve. Many modern networks (ResNet, ConvNeXt) use stride-2 convolutions for downsampling instead of pooling.
• S > 2: Rare. It skips too much information unless the filter is large.

Stride directly reduces FLOPs. Doubling stride cuts the number of output positions by about 4× in 2D.

Padding: Protecting the Borders

Without padding, every convolution shrinks the feature map, and border pixels participate in fewer computations than center pixels.

For stride 1 and an odd filter size, P = (F − 1) / 2 gives “same” padding. A 3×3 filter needs P = 1; a 5×5 needs P = 2. Other padding schemes (reflect, replicate) exist and often suit image generation and segmentation better than zeros.

The Output Size Formula

For input width W, filter size F, padding P, and stride S:

Output = ⌊(W − F + 2P) / S⌋ + 1

Worked examples:

Memorize this formula. It is the most common source of shape-mismatch errors when you flatten features into a classifier head.

Pooling: Downsampling Feature Maps

Pooling slides a window over each channel independently and summarizes it. It has zero learnable parameters.

• Max pooling: Keeps the strongest activation. It preserves the presence of a feature and is the most common choice.
• Average pooling: Keeps the mean. It gives smoother outputs.
• Global average pooling (GAP): Averages each entire feature map to a single value. It replaces large dense layers at the end of networks like ResNet and cuts overfitting.

A 2×2 max pool with stride 2 halves height and width, reducing activations by 75%. It also gives mild invariance to small shifts.

Pooling vs. strided convolution: Pooling is fixed and cheap. A stride-2 convolution learns its own downsampling. Recent architectures lean toward strided convolutions, but pooling remains standard in many classic and lightweight models.

Putting It Together: A Minimal CNN in PyTorch

import torch
import torch.nn as nn

class SimpleCNN(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.features = nn.Sequential(
            # Input: 3x32x32
            nn.Conv2d(3, 32, kernel_size=3, stride=1, padding=1),  # -> 32x32x32
            nn.BatchNorm2d(32),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2),                 # -> 32x16x16

            nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1), # -> 64x16x16
            nn.BatchNorm2d(64),
            nn.ReLU(),
            nn.MaxPool2d(2, 2),                                    # -> 64x8x8

            nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1),# -> 128x8x8
            nn.BatchNorm2d(128),
            nn.ReLU(),
            nn.AdaptiveAvgPool2d(1),                               # -> 128x1x1 (GAP)
        )
        self.classifier = nn.Linear(128, num_classes)

    def forward(self, x):
        x = self.features(x)
        x = torch.flatten(x, 1)
        return self.classifier(x)

model = SimpleCNN()
x = torch.randn(8, 3, 32, 32)   # batch of 8 CIFAR-10-sized images
print(model(x).shape)           # torch.Size([8, 10])
print(sum(p.numel() for p in model.parameters()))  # total params

Annotations:

• padding=1 with kernel_size=3 preserves spatial size, so only the pooling layers shrink it.
• BatchNorm2d stabilizes training and permits higher learning rates (try 1e-3 with Adam).
• AdaptiveAvgPool2d(1) makes the network input-size agnostic and removes the need to compute a flatten dimension by hand.

Receptive Field: What Each Neuron Sees

The receptive field is the region of the input that influences one neuron. It grows with depth, and stride and pooling accelerate that growth. Stacking three 3×3, stride-1 layers yields a 7×7 receptive field. Adding a 2×2 pool between layers expands it faster. Deep layers therefore “see” large objects even though every filter is small.

Real-World Use Cases

Practical Tuning Workflow

1. Start with 3×3 filters, stride 1, “same” padding. This is the safest baseline.
2. Downsample deliberately. Halve resolution every few layers while doubling channels (32 → 64 → 128).
3. Add batch normalization after each convolution.
4. Use data augmentation (random crops, flips, color jitter) before adding capacity.
5. Check shapes with the output formula before training, not after the crash.
6. Move to depthwise-separable convolutions (MobileNet-style) when latency on edge devices matters. They cut compute by roughly 8–9× for 3×3 kernels.

CNNs vs. Vision Transformers

CNNs remain the practical default for small datasets, real-time inference, and edge deployment. Hybrid designs often combine both.

FAQ

What does a filter do in a CNN?

A filter is a small grid of learnable weights that slides across the input and detects a specific pattern, such as an edge or texture. Each filter produces one feature map.

What is the difference between stride and padding?

Stride controls how many pixels the filter moves per step and therefore how much the output shrinks. Padding adds border pixels (usually zeros) around the input so the filter can cover edges and the output size can be preserved.

How do I calculate the output size of a convolutional layer?

Use ⌊(W − F + 2P) / S⌋ + 1, where W is input size, F is filter size, P is padding, and S is stride. A 32×32 input with a 3×3 filter, padding 1, and stride 1 gives a 32×32 output.

Is max pooling still necessary?

Not always. Many modern networks replace it with stride-2 convolutions, which learn their downsampling. Max pooling remains useful in lightweight models because it is simple, parameter-free, and fast.