A study on cost adaptive cloud resource broker system for bio-workflow computing

Abstract

Cloud computing is a logical resource pool paradigm for multi-tenants and their application services connected by Internet. By virtue of the capability to providing an easily and immediately available resources, the cloud computing paradigm extends the legacy HPC and HTC environments by providing the flexible provision of the infrastructure based on scaling-out and scaling-up. In many scientific fields, such as bioinformatics and astronomy, applications are composed of complex workflow tasks and generate huge amounts of data continuously, which require large space of storage as well as high-speed computing resource. DNA sequence analysis, where very large data sets are now generated at reduced cost using the Next-Generation Sequencing (NGS) methods, is an area which can obtain the cost-effective cloud-based infrastructures. It has become major challenge for these datasets to transfer, store, and analysis. Even though many approaches have been proposed in distributed solutions The unpredictability and the fluctuation of the resource demands vary at time, thus dynamical provision of the resource is core part of the application level middleware systems. Furthermore, the quality-of-service (QoS) requirements of applications, as opposed to the traditional best-effort approach, is more important to maximize the utilization of the resource sharing and outsourcing. However, provisioning cloud resource accompany with the leasing cost. So, the efficient workflow management issues become more important on the cloud resource management characteristics.

This dissertation makes several contributions toward integrating the scientific workflow with cloud environment. The major contributions are: (i) propose a cloud resource broker system which extends the legacy workflow management system to the cloud environment. (ii) propose a cloud workflow scheduling mechanism incorporating with cost adaptive resource provisioning as well as the fine-grained VM resource management heur...