MapReduce on Opportunistic Resources Leveraging Resource Availability

MapReduce is a popular large-scale parallel data processing framework. In the context of MapReduce processing on volunteer computing environments, it is important to devise scheduling and data placement policies that account for characteristics of opportunistic resources. This paper investigates availability characteristics of opportunistic resources with analyses based on log traces from the SETI@Home project. Based on the analysis, the paper devises heuristics to leverage the uptime of each available session to detect possibly long-lasting resources. Our proposed session uptime-based resource availability prediction approach shows a two-fold reduction in the number of service disturbance compared to an availability-rate based model. The paper paper investigates a heuristic that differentiates stable nodes from unstable nodes while increasing the chance of leveraging existing data blocks.