Yuz: Improving Performance of Cluster-Based Services by Near-L4 Session-Persistent Load Balancing

Large-scale services are deployed in data centers using clusters of servers, and load balancers (LB) are responsible for distributing requests for a service among its servers.Layer-4 (L4) LBs process requests faster than Layer-7 (L7) ones, but they cannot provide session-persistent load balancing, and therefore, they direct connections of an application-level session to different servers. On the other side, L7 LBs can direct all requests of an application-level session to the same server, but they have a very limited capacity because they act as a reverse-proxy and process requests at the application layer. We present "Yuz", a stateless session-persistent load balancer that does not act as a reverse-proxy. Yuz works near layer 4, and it makes use of TLS session data instead of processing incoming requests at the application level. Our evaluations show that the request rate that can be handled by cluster-based services equipped with Yuzis twice as high as when the clusters use the best existing load balancers. Yuz also significantly reduces the average and tail of the clusters' response time. Moreover, while each of the existing session-persistent LBs works only for a specific application, Yuz provides an application-independent session-persistent load balancing.