Part 8 · Society and Open Problems
Open Problems in AI
AI is still a young science, and its biggest questions are waiting for curious people to answer them.
Key concepts
The major open questions:
- Why do huge neural networks generalize so well?
- Can AI learn from just a few examples?
- Do these models truly understand?
- How do we verify what a model will do?
- What should people always decide for themselves?
Progress in the field moves between theory, experiments, and deployment.
Research map
A research map, with one entry point per question:
| Question | Status | Start with |
|---|---|---|
| Theory of deep learning generalization | Open | Chapter 9 |
| Understanding and meaning in language models | Debated | Bender & Koller (2020) |
| World models and planning | Debated | LeCun (2022) |
| Efficient skill acquisition | Open | Chollet (2019) |
| Mechanistic explanation at frontier scale | Open | Chapter 19 |
| Scalable oversight | Open | Chapter 52 |
| Robust instruction-data separation | Open | Chapter 30 |
| Machine consciousness and moral status | Debated | Butlin et al. (2023) |
| Measuring long-horizon autonomy | Open | Kwa et al. (2025) |
Common misconception
Common belief
"The experts have AI all figured out."
In practice
Experts disagree about many of the field's most basic questions.
Key takeaways
- Major questions about learning and understanding remain open.
- Experts disagree about several fundamental issues.
- Progress depends on theory, experiments, and real-world evidence.