Cognitive

Part 2 · Neural Networks

Convolutional Networks and Vision

Vision networks scan an image for small patterns, then combine them into larger and larger ideas.

3 min readBuilds on The Artificial Neuron, Generalization and Overfitting

A convolution slides across the image and records where a pattern appears.

Key concepts

  • A convolution slides a small filter across an image and records where its pattern appears.
  • Pooling and strides summarize larger areas.
  • Sharing one filter across the image greatly reduces the number of parameters.
  • Vision Transformers split images into patches and use attention (Chapter 15).
  • Adversarial examples are small, deliberate changes that can fool a model.

A convolutional network builds larger visual features from smaller ones.

Figure 10.1. A convolutional network builds larger visual features from smaller ones.
A Vision Transformer reads an image as a sequence of patches.

Formal view and evidence

  • Established. Convolution layers are translation-equivariant, up to boundary and sampling effects: shifting the input shifts the feature map. Pooling adds limited invariance.
  • Debated. The relative roles of architecture, data scale, and augmentation in comparing CNNs and Vision Transformers. ImageNet CNNs have shown texture bias, which depends on training data and augmentation.
  • Open problem. Robust perception that matches human reliability under natural and adversarial shifts.

Common misconception

Common belief

"Vision AI sees objects like we do."

In practice

A model may rely on textures or background cues that people ignore.

Key takeaways

  • Convolutions reuse the same filter across an image.
  • Layers build complex visual features from simple ones.
  • Vision models can rely on cues that people would not use.

Explore next