A multi-class workflow ensemble management system using on-demand and spot instances in cloud

Nowadays, cloud computing is an attractive and competitive market, and many computational jobs have migrated to cloud resources. Scheduling a workflow is a common issue in cloud computing. In some applications, a group of interrelated workflows with the same structure but different sizes and inputs is considered as a single job called an ensemble. Executing such applications in the cloud is expensive, and designing a cost-efficient approach is still challenging. The current paper proposes a new algorithm for scheduling workflow ensembles that are subject aware to of budget and deadline constraints. To achieve this aim, the present study takes advantage of Amazon spot instances. These instances are offered by bid at a lower price but with lower reliability. In the proposed algorithm, workflow tasks are classified according to deadlines and priorities, and the requested spot instances are classified based on bid prices. More reliable instances with higher bid prices are assigned to more important classes of tasks. Furthermore, an efficient workflow acceptance and instance provisioning procedure are proposed to launch and terminate instances based on the number of ready tasks and their deadlines. The present research's simulation results show that the proposed system increases the number of completed workflows and reduces the number of partially executed ones.(c) 2022 Elsevier B.V. All rights reserved.