This issue occurs due to session persistence overrides the load balancing algorithm. When vTM receives a connection from a client that is part of a previous session, then the vTM will try to send it to the same node regardless of the load balancing algorithm.
Here are a few scenarios that may cause an uneven traffic/load distribution:
- Clients with various request rates: For example, there are two clients that are actively sending traffic. If both clients are sending traffic at the same rate/frequency (i.e. both clients send 50 requests per hour), so each node will receive 50 requests. However, in the real world, this is not always true as some clients, or a subset of clients that are part of the same session, may send relatively more requests. This will result in some backend nodes will end up getting more requests.
- Backend node failures: Another reason is if a backend node fails, then all the requests for that node are load balanced over the next available nodes. This creates new sessions to the new node(s). Depending on how long the node is inaccessible, and how many sessions are sent from the failed node, this will result a node receiving very few requests when it recovers.
- NAT: If most clients are behind a NAT device and are using the NAT device source address, then all sessions will end up on the same backend node. This only affects IP based session persistence.
If load distribution is the critical factor for the backend applications/servers to operate normally then additional configuration consideration is required. For example, investigate/use a different session persistence class, use a custom logic in Traffic Script rule using universal session persistence or avoid session persistence (if the backend application does not need it).