Energy-Aware Heuristic Scheduling Using Bin Packing MapReduce Scheduler for Heterogeneous Workloads Performance in Big Data

Big data refers to diverse large data types from heterogeneous sources such as mobile devices, the web, social media, and the internet of things. The cloud offers a wide variety of tools to handle the big data on-demand for pay-per-service basis through a cluster of virtual machines that are hosted across cloud datacenter in heterogeneous physical machines. The primary goal tends to analyze the big data at the point of creation and scaling by data-intensive computing. Hadoop MapReduce helps to solve scalability and complexity by adding more jobs in a virtual cluster across the racks in a distributed cloud datacenter. By default, MapReduce schedulers do not perform the computational jobs in heterogeneity and the virtual machines will execute the blocks in equal numbers despite their capacity decreasing the performance dynamically. The virtual machines on a virtual cluster are not aware of energy efficiency, which is highly important in a heterogeneous environment. Hence, we propose the dynamic performance heuristic-based bin packing (DP-HBP) MapReduce scheduler, which increases the utilization of resources in heterogeneous virtual machines. The proposed DP-HBP scheduler improves the makespan and latency by 49% and 39% over the roulette wheel scheme and heuristic-based MapReduce job schedulers from our experimentation. DP-HBP derived the average number of data nonlocal execution as 27%, which is lesser, compared to the existing schedulers. The resource utilization for the average number of unused vCPU and memory is improved by 34% and 41%, which enhances the performance workloads in handling big data in a heterogeneous environment.