Real-Time Optimization of Dynamic Speed Scaling for Distributed Data Centers

This paper proposes a new distributed real-time optimization for MapReduce-style framework in emerging cloud platforms supporting dynamic speed scaling functions. Distinctively different from the existing MapReduce parallelism strategy with fixed specific data chuck sizes, the new approach is able to dynamically dispatch input data of adequate sizes and synthesize interim processing results according to applications and the state of data centers (DCs). The key idea is to decouple the optimizations of data dispatching, processing, and result aggregation without loss of optimality, by employing stochastic optimization techniques. Another important aspect is that we optimize the subproblem of data processing to leverage the energy- and speed-configurability of DCs, by optimally deciding the number of servers to be activated at every DC and the CPU speeds of the activated servers. Evident from extensive simulations, the proposed approach is able to increase the throughput-cost ratio by up to 94.3%, as compared to existing initiatives, and substantially improve the throughput in the case of high-rate data streams.