Javatpoint Logo
Javatpoint Logo

HTTP Load Balancing

Proxying HTTP Traffic to a Group of Servers

Before start using Nginx open-source or Nginx Plus to load balance HTTP traffic to a group of servers, first, we need to define the group with the upstream directive. The directive is placed in the http context.

Servers in the group are configured using the server directive. Let's see an example, the following configuration defines a group named backend and consists of three server configurations that may resolve in more than three actual servers.

To pass the requests to a server group, the group name is specified in the proxy_pass directive. In the below example, a virtual server running on Nginx passes all requests to the upstream backend group.

The following example combines the two snippets above and shows how to proxy HTTP request to the backend server group. The group consists of three servers, two of the instances of the same application while the third is a backup server. Because there is no load-balancing algorithm is specified in the upstream block, Nginx uses the default algorithm, Round Robin.

Choosing a Load-Balancing Method

Nginx open-source supports four methods for load-balancing, and Nginx Plus adds two more methods:

1. Round Robin: In this method, requests are distributed equally across the servers, with server weights taken into consideration. There is no directive for enabling it; this method is used by default.

2. Least Connections: A request is sent to the server with the least number of active connections, with server weights taken into considerations.

3. IP Hash: This method is used to determine what server should be selected for the next request. In this type of case, either the first three octets of the IPv4 address or the whole IPv6 address are used to calculate the hash value.

If one of the servers needs to be temporarily removed from the load-balancing rotation, it can be added with the down parameter to preserve the current hashing of client IP addresses.

4. Generic Hash: The server to which a request is sent is determined from a user-defined key which can be a string, text, variable or a combination. For example, the key may be a paired source IP address and port, or key may be a URI like the following example:

Nginx Plus supports two more methods:

5. Least Time: For every request, Nginx selects the server with the lowest average latency and the lowest number of active connections, where the lowest average latency is calculated based on which of the following parameters to the least_time directive is included.

  • header: Time to receive the 1st byte from the server.
  • last_byte: Time to accept the full response from the server.
  • last_byte inflight: Time to receive the full response from the server, taking into account incomplete requests.

6. Random: In this method, each request will be passed to a randomly selected server. If the 2 parameters are specified, 1st nginx randomly selects two servers taking into account server weights, and then choose one of these servers using the specified method:

  • Least_conn - least number of active connections.
  • least_time=header - The least average time to receive the response header from the server.
  • Least_time=last_byte - The least average time to receive the full response from the server.

Server Weights

By default, Nginx uses weights to distribute requests among the servers in the group through the Round Robin method. The weight parameter to the server directive is used to set the weight of a server. The by default weight is 1.

In the above example, the weight of is 5, and the weights of the other two servers have the default weight 1, but the one with the IP address is marked as a backup server and does not receive requests unless both of the other servers are unavailable. In this type of configuration of weights, out of every 6 requests, 5 are sent to and 1 to

Server Slow-Start

The server slow start is used to prevent a recently recovered server from being overwhelmed by connections, which may time out and cause the server to be marked as failed again.

And in Nginx Plus, slow-start allows an upstream server to gradually recover its weight from 0 to its nominal value after it has been recovered or become available. To do this, use the slow_start parameter to the server directive.

Enabling Session Persistence

Nginx Plus identifies user sessions and routes all requests in a given session to the same upstream server. This is called session persistence.

Nginx Plus uses three-session persistence methods. The methods are set with the sticky directive.

Sticky cookie

Nginx Plus includes a session cookie to the first response from the upstream group and identifies the server that sent the response. Next request of the client contains the cookie value and Nginx Plus route the request to the upstream server that responded to the first request.

In the above example, the srv_id parameter is used to set the name of the cookie.

expires parameter is optional which sets the time for the browser to keep the cookie.

a domain which is also an optional parameter defines the domain for which the cookie is set.

the path is also an optional parameter. It defines the path for which the cookie is set.

A sticky cookie is the simplest session persistence method.

Sticky Route

When Nginx Plus receives the first request, it assigns a route to the client. All the subsequent requests are compared to the route parameter of the server directive to identify the server to which the request is proxied. The information of the route is taken from either a cookie or the request URI.

Cookie Learn

First of all, Nginx Plus finds session identifiers by inspecting requests and responses. Then Nginx Plus "learns" which upstream server corresponds to which session identifier. Generally, these identifiers are passed in an HTTP cookie. If a request has a session identifier already "learned", Nginx Plus forwards the request to the corresponding server.

In the above, one of the upstream servers creates a session by setting the cookie EXAMPLECOOKIE in the response.

Next TopicNGINX Directives

Youtube For Videos Join Our Youtube Channel: Join Now


Help Others, Please Share

facebook twitter pinterest

Learn Latest Tutorials


Trending Technologies

B.Tech / MCA