Noah: Reinforcement-Learning-Based Rate Limiter for Microservices in Large-Scale E-Commerce Services

Modern large-scale online service providers typically deploy microservices into containers to achieve flexible service management. One critical problem in such container-based microservice architectures is to control the arrival rate of requests in the containers to avoid containers from being overloaded. In this article, we present our experience of rate limit for the containers in, one of the largest e-commerce services in the world. Given the highly diverse characteristics of containers in, we point out that the existing rate limit mechanisms cannot meet our demand. Thus, we design, a dynamic rate limiter that can automatically adapt to the specific characteristic of each container without human efforts. The key idea of is to use deep reinforcement learning (DRL) that automatically infers the most suitable configuration for each container. To fully embrace the advantages of DRL in our context, addresses two technical challenges. First, uses a lightweight system monitoring mechanism to collect container status. In this way, it minimizes the monitoring overhead while ensuring a timely reaction to system load changes. Second, injects synthetic extreme data when training its models. Thus, its model gains knowledge on unseen special events and hence remains highly available in extreme scenarios. To guarantee model convergence with the injected training data, adopts task-specific curriculum learning to train the model from normal data to extreme data gradually. has been deployed in the production of for two years, serving more than 50 000 containers and around 300 types of microservice applications. Experimental results show that can well adapt to three common scenarios in the production environment. It effectively achieves better system availability and shorter request response time compared with four state-of-the-art rate limiters.