Overview

The 3^rd workshop on Many-Task Computing on Grids and Supercomputers (MTAGS10) will provide the scientific community a dedicated forum for presenting new research, development, and deployment efforts of large-scale many-task computing (MTC) applications on large scale clusters, Grids, Supercomputers, and Cloud Computing infrastructure. MTC, the theme of the workshop encompasses loosely coupled applications, which are generally composed of many tasks (both independent and dependent tasks) to achieve some larger application goal. This workshop will cover challenges that can hamper efficiency and utilization in running applications on large-scale systems, such as local resource manager scalability and granularity, efficient utilization of raw hardware, parallel file system contention and scalability, data management, I/O management, reliability at scale, and application scalability.

News

Scope

The advent of computation can be compared, in terms of the breadth and depth of its impact on research and scholarship, to the invention of writing and the development of modern mathematics. Scientific Computing has already begun to change how science is done, enabling scientific breakthroughs through new kinds of experiments that would have been impossible only a decade ago. Today's science is generating datasets that are increasing exponentially in both complexity and volume, making their analysis, archival, and sharing one of the grand challenges of the 21st century. The support for data intensive computing is critical to advancing modern science as storage systems have experienced an increasing gap between its capacity and its bandwidth by more than 10-fold over the last decade. There is an emerging need for advanced techniques to manipulate, visualize and interpret large datasets. Scientific Computing is the key to many domains' "holy grail" of new knowledge, and comes in many shapes and forms, from high-performance computing (HPC) which is heavily focused on compute-intensive applications, high-throughput computing (HTC) which focuses on using many computing resources over long periods of time to accomplish its computational tasks, many-task computing (MTC) which aims to bridge the gap between HPC and HTC by focusing on using many resources over short periods of time, to data-intensive computing which is heavily focused on data distribution and harnessing data locality by scheduling of computations close to the data.

This workshop will focus on the ability to manage and execute large scale applications on today's largest clusters, Grids, and Supercomputers. Clusters with 50K+ processor cores are now online (e.g. TACC Sun Constellation System - Ranger), Grids (e.g. TeraGrid) with a dozen sites and 100K+ processors, and supercomputers with 150K~200K processors (e.g. IBM BlueGene/P, Cray XT5); furthermore, new supercomputers are scheduled to come online with 300K processor-cores and more than 1M threads (e.g. IBM Blue Waters). These High-End Computing (HEC) systems have traditionally been HPC systems, as they are efficient at executing tightly coupled parallel jobs within a particular machine with low-latency interconnects; applications running on them typically use message passing interface (MPI) to achieve the needed inter-process communication. On the other hand, Grids have been the preferred platform for more loosely coupled applications that tend to be managed and executed through workflow systems, commonly known to fit in the HTC paradigm.

MTC aims to bridge the gap between the two computing paradigms, HTC and HPC. MTC is reminiscent to HTC, but it differs in the emphasis of using many computing resources over short periods of time to accomplish many computational tasks (i.e. including both dependent and independent tasks), where the primary metrics are measured in seconds (e.g. FLOPS, tasks/s, MB/s I/O rates), as opposed to operations (e.g. jobs) per month. MTC denotes high-performance computations comprising multiple distinct activities, coupled typically via file system operations. Tasks may be small or large, uniprocessor or multiprocessor, compute-intensive or data-intensive. The set of tasks may be static or dynamic, homogeneous or heterogeneous, loosely coupled or tightly coupled. The aggregate number of tasks, quantity of computing, and volumes of data may be extremely large. MTC includes loosely coupled applications that are generally communication-intensive but not naturally expressed using standard message passing interface commonly found in HPC, drawing attention to the many computations that are heterogeneous but not "happily" parallel.

There is more to HPC than tightly coupled MPI, and more to HTC than embarrassingly parallel long running jobs. Like HPC applications, and science itself, applications are becoming increasingly complex opening new doors for many opportunities to apply HPC in new ways if we broaden our perspective. Some applications have just so many simple tasks that managing them is hard. Applications that operate on or produce large amounts of data need sophisticated data management in order to scale. There exist applications that involve many tasks, each composed of tightly coupled MPI tasks. Loosely coupled applications often have dependencies among tasks, and typically use files for inter-process communication. Efficient support for these sorts of applications on existing large scale systems will involve substantial technical challenges and will have big impact on science.

Today's existing HPC systems are a viable platform to host MTC applications. However, some challenges arise in large scale applications when run on large scale systems, which can hamper the efficiency and utilization of these large scale systems.  These challenges vary from local resource manager scalability and granularity, efficient utilization of raw hardware, parallel file system contention and scalability, data management, I/O management, reliability at scale, application scalability, and understanding the limitations of the HPC systems in order to identify good candidate MTC applications. Furthermore, the MTC paradigm can be naturally applied to the emerging Cloud Computing paradigm due to its loosely coupled nature, which is being adopted by industry as the next wave of technological advancement to reduce operational costs while improving efficiencies in large scale infrastructures.

This workshop encourages interaction and cross-pollination between those developing applications, algorithms, software, hardware and networking, emphasizing many-task computing for large-scale distributed systems. We believe the workshop will be an excellent place to help the community define the current state-of-the-art, determine future goals, and define architectures and services for future high-end computing infrastructure. 

Topics

We invite the submission of original work that is related to the topics below. The papers can be either short (5 pages) position papers, or long (10 pages) research papers. Topics of interest include (in the context of Many-Task Computing):

• Compute Resource Management

  • Scheduling

  • Job execution frameworks

  • Local resource manager extensions

  • Performance evaluation of resource managers in use on large scale systems

  • Dynamic resource provisioning

  • Techniques to manage many-core resources and/or GPUs

  • Challenges and opportunities in running many-task workloads on HPC systems

  • Challenges and opportunities in running many-task workloads on Cloud Computing infrastructure

• Storage architectures and implementations

  • Distributed file systems

  • Parallel file systems

  • Distributed meta-data management

  • Content distribution systems for large data

  • Data caching frameworks and techniques

  • Data management within and across data centers

  • Data-aware scheduling

  • Data-intensive computing applications

  • Eventual-consistency storage usage and management

• Programming models and tools

  • Map-reduce and its generalizations

  • Many-task computing middleware and applications

  • Parallel programming frameworks

  • Ensemble MPI

  • Service-oriented science applications

• Large-Scale Workflow Systems

  • Workflow system performance and scalability analysis

  • Scalability of workflow systems

  • Workflow infrastructure and e-Science middleware

  • Programming Paradigms and Models

• Large-Scale Many-Task Applications

  • High-throughput computing (HTC) applications

  • Data-intensive applications

  • Quasi-supercomputing applications, deployments, and experiences

  • Volunteer Computing

  • Performance Evaluation

• Performance evaluation

  • Real systems

  • Simulations

  • Reliability of large systems

Paper Submission and Publication

Authors are invited to submit papers with unpublished, original work of not more than 10 pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages, as per IEEE 8.5 x 11 manuscript guidelines; document templates can be found at ftp://pubftp.computer.org/Press/Outgoing/proceedings/instruct8.5x11.pdf and ftp://pubftp.computer.org/Press/Outgoing/proceedings/instruct8.5x11.doc. We are also seeking position papers of no more than 5 pages in length. A 250 word abstract as well as the full paper submission (PDF format) must be submitted online at https://cmt.research.microsoft.com/MTAGS2010/ before the extended deadline of September 15th, 2010 at 11:59PM PST. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the IEEE digital library (pending approval). Notifications of the paper decisions will be sent out by October 15th, 2010. Selected excellent work may be eligible for additional post-conference publication as journal articles or book chapters; see this year's ongoing special issue in the IEEE Transactions on Parallel and Distributed Systems (TPDS) at http://dsl.cs.uchicago.edu/TPDS_MTC/.  Submission implies the willingness of at least one of the authors to register and present the paper. For more information, please visit http://www.cs.iit.edu/~iraicu/MTAGS10/. 

Important Dates

Abstract Due:                                             August 25th, 2010 September 15th, 2010

Papers Due:                                               September 1st, 2010 September 15th, 2010

Notification of Acceptance:                   October 1st, 2010 October 15th, 2010

Camera Ready Papers Due:                  November 11th, 2010

Workshop Date:                                        November 15th, 2010

Committee Members

Workshop Chairs

• Ioan Raicu, Illinois Institute of Technology

• Ian Foster, University of Chicago & Argonne National Laboratory

• Yong Zhao, University of Electronic Science and Technology of China

Steering Committee

• David Abramson, Monash University, Australia

• Pete  Beckman, University of Chicago & Argonne National Laboratory, USA

• Alok Choudhary, Northwestern University, USA

• Jack Dongara, University of Tennessee, USA

• Geoffrey Fox, Indiana University, USA

• Robert Grossman, University of Illinois at Chicago, USA

• Arthur Maccabe, Oak Ridge National Labs, USA

• Dan Reed, Microsoft Research, USA

• Marc Snir, University of Illinois at Urbana Champaign, USA

• Xian-He Sun, Illinois Institute of Technology, USA

• Manish Parashar, Rutgers University, USA

Technical Committee

• Roger Barga, Microsoft Research, USA

• Mihai Budiu, Microsoft Research, USA

• Rajkumar Buyya, University of Melbourne, Australia

• Henri Casanova, University of Hawaii at Manoa, USA

• Jeff Chase, Duke University, USA

• Peter Dinda, Northwestern University, USA

• Catalin Dumitrescu, Fermi National Labs, USA

• Evangelinos Constantinos, Massachusetts Institute of Technology, USA

• Indranil Gupta, University of Illinois at Urbana Champaign, USA

• Alexandru Iosup, Delft University of Technology, Netherlands

• Florin Isaila, Universidad Carlos III de Madrid, Spain

• Michael Isard, Microsoft Research, USA

• Kamil Iskra, Argonne National Laboratory, USA

• Daniel Katz, University of Chicago, USA

• Tevfik Kosar, Louisiana State University, USA

• Zhiling Lan, Illinois Institute of Technology, USA

• Ignacio Llorente, Universidad Complutense de Madrid, Spain

• Gaurang Mehta, University of Southern California, USA

• Reagan Moore, University of North Carolina, Chappel Hill, USA

• Jose Moreira, IBM Research, USA

• Marlon Pierce, Indiana University, USA

• Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory, USA

• Matei Ripeanu, University of British Columbia, Canada

• Alain Roy, University of Wisconsin Madison, USA

• Edward Walker, Texas Advanced Computing Center, USA

• Mike Wilde, University of Chicago & Argonne National Laboratory, USA

• Matthew Woitaszek, The University Coorporation for Atmospheric Research, USA

• Justin Wozniak, Argonne National Laboratory, USA

• Ken Yocum, University of California San Diego, USA

To return to MTAGS10 main page, click here.