A pipeline virtual service pre-scheduling pattern and its application in astronomy data processing

Based on Open Grid Services Architecture (OGSA), the concept and the formal model of Pipeline Virtual Service (PVS) are proposed and presented in this paper. PVS is used to model a special type of grid workflow composed of a group of services that can be executed as a pipeline. A PVS Pre-Scheduling Pattern is described in detail, which can overlap the execution time of a PVS with the time of generating and optimizing the scheduling solution pool. The Pattern includes two independent components, the solution generating component (GenFunc) and the solution execution component (ExeFunc). GenFunc can take advantage of the powerful computing potential of grid environment to find the optimal or a near optimal scheduling solution, while ExeFunc can select a suitable scheduling solution and put a PVS into running as soon as possible. Two corresponding algorithms, SODGen and SODExe, are also developed. The analytic and simulation results show that the PVS Pre-Scheduling Pattern can not only balance the scheduling cost and scheduling precision, save the scheduling time greatly, but also reduce the execution time through selecting the optimized solution. A prototype is designed and implemented for a large scale astronomy data processing center. The early engineering evaluation experiments show that the PVS Pre-Scheduling Pattern is feasible, efficient, and flexible in real astronomy data processing.