Data-Intensive Distributed Systems LaboratoryCFP (TXT) | News | Topics | Dates | Submission | Organization | Program | Sponsors
4th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) 2011
Co-located with Supercomputing/SC 2011Seattle Washington -- November 14th, 2011
Grand Hyatt Theater -- 10AM - 5:30PM
Keynote: Mixing Cloud and Grid Resources for Many Task Computing
Dr. David Abramson
Professor of Computer Science
ARC Professorial Fellow
Faculty of Information Technology
Monash University, Australia
Grand Hyatt Theater
Monday, November 14th, 2011
10:10AM - 11:00AM
Abstract: Over the past 15 years we have developed a sophisticated tool set for solving large parameter sweep and search applications using a Many Task approach. This tool family, called Nimrod, makes it easy to create computational experiments in which multiple independent scenarios are explored. Nimrod supports complete state-space exploration (so called full parameter sweeps), guided search (using non-linear optimisation techniques) and design-of-experiment schemes (for reducing the search space). All of these work by creating many tasks, and farming them out to computational units distributed (potentially) over a Grid. We have run many real world case studies over local clusters and the Grid.
In the Grid we have consistently experienced two obstacles that complicated experiments. First, the heterogeneity that defines the Grid makes application deployment difficult and error prone. The vast variety of operating systems, hardware platforms and runtime environments means that an application needs to be ported to a large number of different platforms, requiring time consuming setup procedures. Second, the lack of quality of service guarantees means that it is difficult to know how much resource can be delivered, thus computing completion times for experiments is difficult. Both of these problems were addressed in the Nimrod tools. First, we extended the OGSA architecture to incorporate a deployment 'service', and leveraged virtualisation to reduce the heterogeneity. Our experimental system used Microsoft's .NET platform. Second, we designed a computational economy that used a payment model to allow users to trade deadlines against costs.
Interestingly, both of these innovations underpin commercial cloud offerings, and make cloud computing a natural platform for running Nimrod experiments. Accordingly, we have added support for Amazon's EC2, and Microsoft's Azure - boht commercial cloud services with very different architectures. Nimrod is able to schedule jobs seamlessly across all three systems, leveraging its computational economy to choose the cheapest allocation that meets a user specified deadline. Moreover, the cloud virtualisation schemes have dramatically simplified application deployment, making it possible to run a variety of applications that use specific software stacks.
Bio: Professor David Abramson has been involved in computer architecture and high performance computing research since 1979. Previous to joining Monash University in 1997, he has held appointments at Griffith University, CSIRO, and RMIT. At CSIRO he was the program leader of the Division of Information Technology High Performance Computing Program, and was also an adjunct Associate Professor at RMIT in Melbourne. He served as a program manager and chief investigator in the Co-operative Research Centre for Intelligent Decisions Systems and the Co-operative Research Centre for Enterprise Distributed Systems.
Abramson is currently an ARC Professorial Fellow; Professor of Computer Science in the Faculty of Information Technology at Monash University, Australia, and science director of the Monash e-Research Centre. He is a fellow of the Association for Computing Machinery (ACM) and the Academy of Science and Technological Engineering (ATSE), and a member of the IEEE. Abramson has served on committees for many conferences and workshops, and has published over 200 papers and technical documents. He has given seminars and received awards around Australia and internationally and has received over $8 million in research funding.
He also has a keen interest in R&D commercialization and consults for Axceleon Inc, who produce an industry strength version of Nimrod, and Guardsoft, a company focused on commercialising the Guard relative debugger.
Abramson’s current interests are in high performance computer systems design and software engineering tools for programming parallel, distributed supercomputers and stained glass windows.