Overview
François Fleuret (University of Geneva) — Concise, phone-formatted textbook covering deep learning foundations. Designed to fit on a phone screen. Cover illustration is the Neocognitron (Fukushima, 1980), a key ancestor of modern deep neural networks.
Structure
Part I: Foundations
- Machine Learning — Learning from data, basis function regression, under/overfitting, categories of models
- Efficient Computation — GPUs, TPUs, batches, tensor operations
- Training — Losses, autoregressive models, gradient descent, backpropagation, value of depth, training protocols, benefits of scale
Part II: Deep Models
- Model Components — Layers, linear layers, activation functions, pooling, dropout, normalization, skip connections, attention layers, token embedding, positional encoding
- Architectures — (Multi-layer perceptrons, convolutional networks, transformers, etc.)
Key Characteristics
- Extremely concise: Distills deep learning into its essential mathematical and computational concepts
- Practical focus: Organized around what you need to implement, not historical development
- Reference design: Structured for quick lookup of specific components and their mathematical definitions
Significance
Fills the gap between tutorial blog posts and heavy textbooks. Particularly valuable for practitioners who need a rigorous but compact reference for the building blocks of modern architectures.
Related
- reinforcement-learning-introduction — Sutton & Barto’s complementary foundational text for RL
- ai-engineering-foundation-models — Huyen’s practical engineering perspective building on these foundations
- pretraining-custom-llms — Applied pretraining that uses the concepts Fleuret defines