Smart Scheduler for CUDA Programming in Heterogeneous CPU/GPU Environment

The demand for high performance has driven the technology to grow exponentially requiring the computer systems to work as effectively as possible for a valuable output. Substantial innovation in technology with time has made the use of GPUs working together with CPUs in order to make the system more efficient in performing computations optimally. This paper presents design of a scheduler for heterogeneous CUDA environment which ensures that while the task is fetched into the system, all the nodes participate fully in scheduling, thereby completing the task in less span of time as compared to normal schedulers resulting in efficient results. Tasks have been divided amongst the computing nodes according to the availability of the nodes giving maximum possible throughput of the system according to the workload. The scheduler has the potential of running the GPU code in parallel on different computing nodes within the High Performance Computing environment improving the overall performance of the applications. As a result, it turned out that the Smart scheduler gives better throughput in comparison to SLURM's existing schedulers which indicates that there was a room in SLURM's existing schedulers to increase the number of jobs within less span of time. The overall improvement in the throughput was observed to be up to 70 percent which is also shown in Figure 4.