Automated Optimization Methods for Scientific Workflows in e-Science Infrastructures

Automated Optimization Methods for Scientific Workflows in e-Science Infrastructures
Author: Sonja Holl
Publisher: Forschungszentrum Jülich
Total Pages: 207
Release: 2014
Genre:
ISBN: 389336949X

Scientific workflows have emerged as a key technology that assists scientists with the design, management, execution, sharing and reuse of in silico experiments. Workflow management systems simplify the management of scientific workflows by providing graphical interfaces for their development, monitoring and analysis. Nowadays, e-Science combines such workflow management systems with large-scale data and computing resources into complex research infrastructures. For instance, e-Science allows the conveyance of best practice research in collaborations by providing workflow repositories, which facilitate the sharing and reuse of scientific workflows. However, scientists are still faced with different limitations while reusing workflows. One of the most common challenges they meet is the need to select appropriate applications and their individual execution parameters. If scientists do not want to rely on default or experience-based parameters, the best-effort option is to test different workflow set-ups using either trial and error approaches or parameter sweeps. Both methods may be inefficient or time consuming respectively, especially when tuning a large number of parameters. Therefore, scientists require an effective and efficient mechanism that automatically tests different workflow set-ups in an intelligent way and will help them to improve their scientific results. This thesis addresses the limitation described above by defining and implementing an approach for the optimization of scientific workflows. In the course of this work, scientists’ needs are investigated and requirements are formulated resulting in an appropriate optimization concept. In a following step, this concept is prototypically implemented by extending a workflow management system with an optimization framework, including general mechanisms required to conduct workflow optimization. As optimization is an ongoing research topic, different algorithms are provided by pluggable extensions (plugins) that can be loosely coupled with the framework, resulting in a generic and quickly extendable system. In this thesis, an exemplary plugin is introduced which applies a Genetic Algorithm for parameter optimization. In order to accelerate and therefore make workflow optimization feasible at all, e-Science infrastructures are utilized for the parallel execution of scientific workflows. This is empowered by additional extensions enabling the execution of applications and workflows on distributed computing resources. The actual implementation and therewith the general approach of workflow optimization is experimentally verified by four use cases in the life science domain. All workflows were significantly improved, which demonstrates the advantage of the proposed workflow optimization. Finally, a new collaboration-based approach is introduced that harnesses optimization provenance to make optimization faster and more robust in the future.

Future Application and Middleware Technology on e-Science

Future Application and Middleware Technology on e-Science
Author: Ok-Hwan Byeon
Publisher: Springer Science & Business Media
Total Pages: 169
Release: 2009-12-01
Genre: Computers
ISBN: 1441917195

Future Application and Middleware Technology on e-Science presents selected papers from the 2008 Korea e-Science All-Hands-Meeting (AHM 2008). Hosted by the Korea Institute of Science and Technology Information, this meeting was designed to bring together developers and users of e-Science applications and enabling information technologies from international and interdisciplinary research communities. The AHM 2008 conference served as a forum for engineers and scientists to present state-of-the-art research and product/tool developments, and to highlight related activities in all fields of e-Science. The works presented in this edited volume bring together cross-disciplinary information on e-Science in one cohesive source. This book is suitable for the professional audience composed of industry researchers and practitioners of e-Science. This volume should also be suitable for advanced-level students in the field.

Guide to e-Science

Guide to e-Science
Author: Xiaoyu Yang
Publisher: Springer Science & Business Media
Total Pages: 554
Release: 2011-05-26
Genre: Computers
ISBN: 0857294393

This guidebook on e-science presents real-world examples of practices and applications, demonstrating how a range of computational technologies and tools can be employed to build essential infrastructures supporting next-generation scientific research. Each chapter provides introductory material on core concepts and principles, as well as descriptions and discussions of relevant e-science methodologies, architectures, tools, systems, services and frameworks. Features: includes contributions from an international selection of preeminent e-science experts and practitioners; discusses use of mainstream grid computing and peer-to-peer grid technology for “open” research and resource sharing in scientific research; presents varied methods for data management in data-intensive research; investigates issues of e-infrastructure interoperability, security, trust and privacy for collaborative research; examines workflow technology for the automation of scientific processes; describes applications of e-science.

UNICORE Summit 2013

UNICORE Summit 2013
Author: Valentina Huber
Publisher: Forschungszentrum Jülich
Total Pages: 107
Release: 2013
Genre:
ISBN: 3893369104

Towards Interoperable Research Infrastructures for Environmental and Earth Sciences

Towards Interoperable Research Infrastructures for Environmental and Earth Sciences
Author: Zhiming Zhao
Publisher: Springer Nature
Total Pages: 375
Release: 2020-07-24
Genre: Computers
ISBN: 3030528294

This open access book summarises the latest developments on data management in the EU H2020 ENVRIplus project, which brought together more than 20 environmental and Earth science research infrastructures into a single community. It provides readers with a systematic overview of the common challenges faced by research infrastructures and how a ‘reference model guided’ engineering approach can be used to achieve greater interoperability among such infrastructures in the environmental and earth sciences. The 20 contributions in this book are structured in 5 parts on the design, development, deployment, operation and use of research infrastructures. Part one provides an overview of the state of the art of research infrastructure and relevant e-Infrastructure technologies, part two discusses the reference model guided engineering approach, the third part presents the software and tools developed for common data management challenges, the fourth part demonstrates the software via several use cases, and the last part discusses the sustainability and future directions.