Load balancer

Spread requests across servers; layer 4 vs layer 7; horizontal scaling.

On this page
  1. Layer 4 load balancing
  2. Layer 7 load balancing
  3. Horizontal scaling
  4. SSL termination at the balancer
A client sends a request to a dispatcher in front of a pool of workers, each with its own storage. The dispatcher picks a worker (randomly, round robin, least busy, by sticky session or by request parameters), waits for its response and forwards it to the client.
Source: Scalable system design patterns

Load balancers distribute incoming client requests to computing resources such as application servers and databases. In each case, the load balancer returns the response from the computing resource to the appropriate client. Load balancers are effective at:

  • Preventing requests from going to unhealthy servers
  • Preventing overloading resources
  • Helping to eliminate a single point of failure

Load balancers can be implemented with hardware (expensive) or with software such as HAProxy.

Additional benefits include:

  • SSL termination - Decrypt incoming requests and encrypt server responses so backend servers do not have to perform these potentially expensive operations
  • Session persistence - Issue cookies and route a specific client's requests to same instance if the web apps do not keep track of sessions

To protect against failures, it's common to set up multiple load balancers, either in active-passive or active-active mode.

Load balancers can route traffic based on various metrics, including:

Layer 4 load balancing#

Layer 4 load balancers look at info at the transport layer to decide how to distribute requests. Generally, this involves the source, destination IP addresses, and ports in the header, but not the contents of the packet. Layer 4 load balancers forward network packets to and from the upstream server, performing Network Address Translation (NAT).

Layer 7 load balancing#

Layer 7 load balancers look at the application layer to decide how to distribute requests. This can involve contents of the header, message, and cookies. Layer 7 load balancers terminate network traffic, reads the message, makes a load-balancing decision, then opens a connection to the selected server. For example, a layer 7 load balancer can direct video traffic to servers that host videos while directing more sensitive user billing traffic to security-hardened servers.

At the cost of flexibility, layer 4 load balancing requires less time and computing resources than Layer 7, although the performance impact can be minimal on modern commodity hardware.

Horizontal scaling#

Load balancers can also help with horizontal scaling, improving performance and availability. Scaling out using commodity machines is more cost efficient and results in higher availability than scaling up a single server on more expensive hardware, called Vertical Scaling. It is also easier to hire for talent working on commodity hardware than it is for specialized enterprise systems.

Sources and further reading7 links

Added on this site

SSL termination at the balancer#

Encrypting and decrypting traffic costs processor time, and every server behind a load balancer would otherwise repeat that work. With SSL termination the balancer decrypts each incoming request and encrypts the response, so the backend servers receive and send plain requests. Only the balancer needs the certificate, so you do not have to install an X.509 certificate on every server either.

The trade-off is inside your network. Traffic between the balancer and a backend server is no longer encrypted unless you encrypt it again on that hop, which matters when the servers sit on networks you do not fully control.

Esc
↑ ↓ to move↵ to open