Load Balancing
Load balancing places one traffic-handling layer in front of several backends. It selects an eligible backend for each request or connection so a service can share work and keep serving when individual backends change or fail.
itNetworking | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Don't Panic
Don't Panic - Load Balancing
Load Balancing is the subject of this course. A client wants one dependable service endpoint. Behind that endpoint, the service may run on many backends.
The useful unit of work is a closed loop: clarify the goal and boundaries, gather the inputs the practice requires, make the decision or change, record evidence, and return with owners for the next cycle. Skipping any link leaves teams busy without durable results.
Tooling supports the loop; it does not replace it. Choose tools after the boundary and evidence model are clear. Comparing products without that model produces feature matrices that do not change how the work runs.
Common failure modes include undefined ownership, metrics that count activity instead of outcomes, and irreversible steps taken without a review path. Treat those as design defects in the practice, not as individual heroics to compensate later.
Operators should be able to explain which signals would change a decision this week. If no signal can change the plan, the practice has become ritual. Keep the feedback path short enough that evidence still influences the next cycle.
Name the owners for each stage of the loop before the work scales. Unowned stages become permanent exceptions. Record decisions with enough context that a future operator can tell why a tradeoff was accepted. Prefer fewer, sharper metrics that change behavior over broad dashboards that only describe activity after the fact.
Read the Intro for the core model. Use the Cheatsheet when you need the operating map. Updates tracks official guidance when this course configures an update source; otherwise the practice is settled without a live feed.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://datatracker.ietf.org/doc/rfc9110/
Supports
- HTTP gateway and reverse-proxy terminology
- Gateways forwarding requests toward one or more servers
- Load balancing as a use of HTTP gateways
- Protocol responsibilities of HTTP intermediaries
- https://docs.cloud.google.com/load-balancing/docs/load-balancing-overview
Supports
- Load balancers distributing client traffic across application instances
- Layer 4 decisions from network and transport data
- Layer 7 decisions from HTTP attributes
- Proxy and passthrough distinctions
- Internal, external, regional, and global scopes
- https://docs.cloud.google.com/load-balancing/docs/health-check-concepts
Supports
- Probes and sequential success or failure thresholds
- Backend health determining eligibility for new work
- Existing connections not necessarily closing when health changes
- Product-specific behavior when all backends are unhealthy
- https://nginx.org/en/docs/http/load_balancing.html
Supports
- Round robin, least connections, weights, and IP hash
- Least connections responding to different request durations
- Passive failure handling
- Protocols supported by NGINX reverse-proxy load balancing
- https://www.haproxy.com/documentation/haproxy-configuration-tutorials/proxying-essentials/configuration-basics/backends/
Supports
- Backend pools
- Round robin, least connections, random, and hash policies
- Policy fit for different connection and membership patterns
- Weights and health checks
- https://www.haproxy.com/documentation/haproxy-configuration-tutorials/reliability/health-checks/
Supports
- Active and passive health-check definitions
- TCP and HTTP probe behavior
- Failure and success thresholds
- Live-traffic errors as passive evidence
- https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers.html
Supports
- Envoy host selection from upstream clusters
- Weighted round robin, least request, random, ring hash, Maglev, and locality-aware policies
- Hash-policy behavior and tradeoffs
- https://www.envoyproxy.io/docs/envoy/latest/faq/load_balancing/transient_failures
Supports
- Interaction among retries, circuit breaking, and outlier detection
- Idempotency as a retry consideration
- Retry budgets and overload risk
- https://www.envoyproxy.io/docs/envoy/latest/faq/load_balancing/lb_panic_threshold.html
Supports
- Special handling required when health policy leaves too few healthy backends
- https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-register-targets.html
Supports
- Initial health checks before a registered target receives traffic
- Deregistration stopping new requests
- Connection draining for in-flight work
- Safe shutdown after deregistration completes
- https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-configure-load-balancer.html
Supports
- Sticky-session support
- Connection-draining behavior
- Cross-zone traffic distribution
- https://kubernetes.io/docs/concepts/services-networking/service/
Supports
- Stable service endpoints over changing backend pods
- Services, EndpointSlices, and service proxying
- Internal and external exposure options
- External load-balancer integration boundary
- https://github.com/sindresorhus/awesome
Supports
- Starting index used for awesome-list discovery
- https://github.com/drsoft-oss/awesome-proxy
Supports
- Discovery of HAProxy, Envoy, Traefik, and Caddy in its Reverse Proxies and Load Balancers section
- https://doc.traefik.io/traefik/master/routing/services/
Supports
- Traefik service load balancing
- Backend distribution and health checks
- Provider-backed endpoint integration
- https://caddyserver.com/docs/caddyfile/directives/reverse_proxy
Supports
- Caddy reverse-proxy upstreams and dynamic discovery
- Selection policies
- Active and passive health checks
- Retry, header, and transport options
