Knowledge Vault

Load Balancing

2 min read

Overview

Load balancing distributes incoming network traffic across multiple servers to ensure no single server bears too much demand. It improves application availability, responsiveness, and fault tolerance by routing requests only to healthy instances. Load balancers operate at different layers — Layer 4 (TCP/UDP, fast but limited routing options) and Layer 7 (HTTP, content-aware routing with path/header inspection). Modern cloud platforms provide managed load balancers, but understanding the underlying algorithms and trade-offs remains essential for architects designing scalable systems.

Key Concepts

  • Round robin — Distributes requests sequentially across servers. Simple but doesn’t account for server capacity or current load.
  • Least connections — Routes to the server with fewest active connections. Better for uneven request durations.
  • Weighted distribution — Assigns capacity weights to servers, sending proportionally more traffic to more powerful instances.
  • Health checks — Periodic probes that verify server readiness. Unhealthy servers are removed from the pool automatically.
  • Session affinity (sticky sessions) — Routes requests from the same client to the same server. Simplifies stateful applications but reduces distribution effectiveness.
  • Layer 4 vs Layer 7 — L4 load balancers route based on IP/port (faster). L7 inspects HTTP content for path-based routing, header matching, and SSL termination.
  • Auto-scaling integration — Load balancers work with auto-scaling groups to add/remove instances based on demand.

Trade-offs

  • Sticky sessions reduce load distribution effectiveness
  • L7 load balancers add latency for content inspection
  • Single load balancer becomes a single point of failure (solved by redundant pairs)
  • DNS-based load balancing is simple but has TTL-related lag
  • Microservices architectures require load balancing across many service instances
  • Caching Strategies interact with load balancing — cache locality vs distribution
  • Kubernetes provides built-in service load balancing via kube-proxy

Graph View

Backlinks