Grid workflow-integrated resource management system with a policy negotiator

Abstract

Many scientific achievements today are derived from analyzing large amounts of data. However, the computations themselves might be very complex and consume significant resources. These resources are often drawn from a heterogeneous pool of geographically distributed computing and data storage resources. In such environment, running large-scale and collaborative applications arises many challenges: systematic management of resource pool, guarantee of Quality of Service (QoS) according to Service Level Agreement (SLA) with users, as well as successful and efficient execution of applications. In fact, Grid Computing [1][2][3] can bring heterogeneous resources together and allocate them efficiently to applications. However, some of the previous challenges such as the Qos guarantee still remain with Grid Computing technology. One of existing resource management schemes is Policy Quorum based Resource Management (PQRM) system [4][5]. It is a QoS-constrained Grid resource management middleware that can guarantee a specific QoS according to SLA and provide available resource sets determined by decision function based on Markov Decision Process (MDP). However, it overlooks some information which is important to Grid workflow management, because it focuses on mechanism that introduces abstraction of resource status and adaptive resource allocation policy respectively. Meanwhile, the execution of large scale and collaborative application requires analyzing large amounts of data, and the dependency among different applications is very complex. Workflow management can make the process concise and easy to handle for unskilled users. However, a large-scale workflow can be a burdensome job because it may consume significant resources in distributed environment or manage the data between resources. This kind of workflows can be managed by Grid workflow management system. Existing Grid workflow management systems can be classified into Workflow sequence management system and...