Notation
Symbols used across the chapters.
| Symbol | Meaning | Introduced |
|---|---|---|
| Model parameters | Chapter 7 | |
| Loss at parameters θ | Chapter 7 | |
| Gradient of the loss | Chapter 7 | |
| Learning rate | Chapter 7 | |
| Momentum coefficient | Chapter 7 | |
| Mini-batch of training examples | Chapter 7 | |
| Query, key and value matrices | Chapter 15 | |
| Sampling temperature | Chapter 18 | |
| Logit of token | Chapter 18 |