Overview
Martin Kleppmann (2017, O’Reilly) — The definitive guide to building reliable, scalable, and maintainable data systems. Widely considered the most important single book for backend/infrastructure engineers.
Structure
Part I: Foundations of Data Systems
- Reliable, Scalable, and Maintainable Applications — Three pillars: reliability (hardware/software/human faults), scalability (describing load and performance), maintainability (operability, simplicity, evolvability)
Part II: Distributed Data
- Replication (single-leader, multi-leader, leaderless)
- Partitioning (key-range, hash, secondary indexes)
- Transactions (ACID, isolation levels, serializability)
- Consistency and consensus (linearizability, total order broadcast, distributed transactions)
Part III: Derived Data
- Batch processing (MapReduce, dataflow engines)
- Stream processing (event streams, change data capture)
- The future of data systems (data integration, unbundling)
Core Design Principles
| Principle | Definition |
|---|---|
| Reliability | System works correctly even when faults occur |
| Scalability | System can cope with increased load |
| Maintainability | People can work on the system productively over time |
Significance
DDIA bridges theory and practice — it explains why distributed systems behave the way they do, with enough mathematical rigor to be trustworthy and enough practical examples to be actionable. Required reading for anyone building systems that store or process data.
Related
- database-internals — Petrov’s deeper dive into storage engine implementation
- system-design-interview — Applied design patterns building on DDIA concepts
- the-phoenix-project — Organizational perspective on the systems DDIA engineers build