Part 1 · Foundations
Data Representation
Computers only work with numbers, so every picture, sound, and word must first be turned into numbers.
Key concepts
- Images are grids of pixels. Audio is air pressure measured thousands of times per second. Text is split into pieces and given ID numbers.
- A feature is a measurable property. A dataset is many examples, often with labels.
- Models are only as good as their data. Missing, skewed, or wrongly labeled data leads to poor results.
- Data is split into training, validation, and test sets so that performance is measured honestly.
Every input type must become numbers before a model can process it.
Formal view and evidence
- Established. The sampling theorem shows a signal band-limited to Hz is exactly recoverable when sampled above Hz. Standard learning guarantees assume samples are drawn i.i.d. from a fixed distribution.
- Debated. The manifold hypothesis holds that high-dimensional natural data concentrates near lower-dimensional structure. It is useful and has empirical support, but its precise form varies by domain.
- Open problem. Reliable performance under distribution shift, when deployment data differs from training data.
Common misconception
Common belief
"Computers see pictures the way we do."
In practice
A model receives only a grid of numbers. Everything else must be learned.
Key takeaways
- Every input must be represented numerically before learning can begin.
- Separate training, validation, and test data keep evaluation honest.
- Models can fail when deployment data differs from training data.