MTAGS 2009 Workshop

The 2nd workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) will provide the scientific community a dedicated forum for presenting new research, development, and deployment efforts of loosely coupled large scale applications on large scale clusters, Grids, Supercomputers, and Cloud Computing infrastructure. Many-task computing (MTC), the theme of the workshop encompasses loosely coupled applications, which are generally composed of many tasks (both independent and dependent tasks) to achieve some larger application goal. This workshop will cover challenges that can hamper efficiency and utilization in running applications on large-scale systems, such as local resource manager scalability and granularity, efficient utilization of the raw hardware, parallel file system contention and scalability, data management, I/O management, reliability at scale, and application scalability. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the ACM digital library.  The workshop will be co-located with the IEEE/ACM Supercomputing 2009 Conference in Portland Oregon on November 16th, 2009. We welcome paper submissions on all topics related to MTC on large scale systems. To see last year’s workshop program agenda, and accepted papers and presentations, please see http://dsl.cs.uchicago.edu/MTAGS08/. For more information on this year's workshop, including location and times, please see http://scyourway.nacse.org/conference/view/wksp103. 

News

Scope

This workshop will focus on the ability to manage and execute large scale applications on today's largest clusters, Grids, and Supercomputers. Clusters with 50K+ processor cores are now online (i.e. TACC Sun Constellation System - Ranger), Grids (i.e. TeraGrid) with a dozen sites and 100K+ processors, and supercomputers with 150K~160K processors (i.e. IBM BlueGene/P, Cray XT5). Large clusters and supercomputers have traditionally been high performance computing (HPC) systems, as they are efficient at executing tightly coupled parallel jobs within a particular machine with low-latency interconnects; the applications typically use message passing interface (MPI) to achieve the needed inter-process communication. On the other hand, Grids have been the preferred platform for more loosely coupled applications that tend to be managed and executed through workflow systems, commonly known to fit in the high-throughput computing (HTC) paradigm.

Many-task computing (MTC) aims to bridge the gap between two computing paradigms, HTC and HPC. MTC is reminiscent to HTC, but it differs in the emphasis of using many computing resources over short periods of time to accomplish many computational tasks (i.e. including both dependent and independent tasks), where the primary metrics are measured in seconds (e.g. FLOPS, tasks/s, MB/s I/O rates), as opposed to operations (e.g. jobs) per month. MTC denotes high-performance computations comprising multiple distinct activities, coupled via file system operations. Tasks may be small or large, uniprocessor or multiprocessor, compute-intensive or data-intensive. The set of tasks may be static or dynamic, homogeneous or heterogeneous, loosely coupled or tightly coupled. The aggregate number of tasks, quantity of computing, and volumes of data may be extremely large. MTC includes loosely coupled applications that are generally communication-intensive but not naturally expressed using standard message passing interface commonly found in HPC, drawing attention to the many computations that are heterogeneous but not "happily" parallel.

There is more to HPC than tightly coupled MPI, and more to HTC than embarrassingly parallel long running jobs. Like HPC applications, and science itself, applications are becoming increasingly complex opening new doors for many opportunities to apply HPC in new ways if we broaden our perspective. Some applications have just so many simple tasks that managing them is hard. Applications that operate on or produce large amounts of data need sophisticated data management in order to scale. There exist applications that involve many tasks, each composed of tightly coupled MPI tasks. Loosely coupled applications often have dependencies among tasks, and typically use files for inter-process communication. Efficient support for these sorts of applications on existing large scale systems will involve substantial technical challenges and will have big impact on science.

Today's existing HPC systems are a viable platform to host MTC applications. However, some challenges arise in large scale applications when run on large scale systems, which can hamper the efficiency and utilization of these large scale systems. These challenges vary from local resource manager scalability and granularity, efficient utilization of the raw hardware, parallel file system contention and scalability, data management, I/O management, reliability at scale, application scalability, and understanding the limitations of the HPC systems in order to identify good candidate MTC applications. Furthermore, the MTC paradigm can be naturally applied to the emerging Cloud Computing paradigm due to its loosely coupled nature, which is being adopted by industry as the next wave of technological advancement to reduce operational costs while improving efficiencies in large scale infrastructures.

For an interesting discussion in a blog by Ian Foster on the difference between MTC and HTC, please see his blog at http://ianfoster.typepad.com/blog/2008/07/many-tasks-comp.html;a brief definition of MTC can also be found at Wikipedia. We also published two papers that are highly relevant to this workshop. One paper is titled "Toward Loosely Coupled Programming on Petascale Systems", and was published in SC08; the second paper is titled “Many-Task Computing for Grids and Supercomputers”, which was published in MTAGS08. Furthermore, to see last year’s workshop program agenda, and accepted papers and presentations, please see http://dsl.cs.uchicago.edu/MTAGS08/.

Topics

MTAGS 2009 topics of interest include, but are not limited to:

Compute Resource Management  in large scale clusters, large Grids, Supercomputers, and Cloud Computing infrastructures

• - Scheduling

• - Job execution frameworks

• - Local resource manager extensions

• - Performance evaluation of resource managers in use on large scale systems

• - Challenges in running many-task workloads on HPC systems

• - Challenges in running many-task workloads on Cloud Computing infrastructures

Data Management in large scale Grid and Supercomputer environments:

• - Data-Aware Scheduling

• - Shared File System performance and scalability in large deployments

• - Distributed file systems

• - Data caching frameworks and techniques

Large-Scale Workflow Systems

• - Workflow system performance and scalability analysis

• - Scalability of workflow systems

• - Workflow infrastructure and e-Science middleware

• - Programming Paradigms and Models

Large-Scale Many-Task Applications

• - Large-scale many-task applications

• - Large-scale many-task data-intensive applications

• - Large-scale high throughput computing (HTC) applications

• - Quasi-supercomputing applications, deployments, and experiences

Paper Submission and Publication

Authors are invited to submit papers with unpublished, original work of not more than 10 pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages, as per ACM 8.5 x 11 manuscript guidelines (http://www.acm.org/publications/instructions_for_proceedings_volumes); document templates can be found at http://www.acm.org/sigs/publications/proceedings-templates. A 250 word abstract (PDF format) must be submitted online at https://cmt.research.microsoft.com/MTAGS2009/ before the deadline of August 25th, 2009 at 11:59PM PST; the final 10 page papers in PDF format will be due on September 1st, 2009 at 11:59PM PST. Papers will be peer-reviewed, and accepted papers will be published in the workshop proceedings as part of the ACM digital library. Notifications of the paper decisions will be sent out by October 1st, 2009. Selected excellent work will be invited to submit extended versions of the workshop paper to the IEEE Transactions on Parallel and Distributed Systems (TPDS) Journal, Special Issue on Many-Task Computing (due December 21st, 2009); for more information about this journal special issue, please visit http://dsl.cs.uchicago.edu/TPDS_MTC/.  Submission implies the willingness of at least one of the authors to register and present the paper. For more information, please visit http://dsl.cs.uchicago.edu/MTAGS09/.

Important Dates

Abstract Due:                                        August 1st, 2009 August 25th, 2009

Papers Due:                                          September 1st, 2009

Notification of Acceptance:                   October 1st, 2009

Camera Ready Papers Due:                  October 21st, 2009

Workshop Date:                                     November 16th, 2009

Committee Members

Workshop Chairs

Ioan Raicu, Northwestern University

Ian Foster, University of Chicago & Argonne National Laboratory

Yong Zhao, Microsoft

Technical Committee

* David Abramson, Monash University, Australia
* Pete Beckman, Argonne National Laboratory, USA
* Peter Dinda, Northwestern University, USA
* Ian Foster, University of Chicago & Argonne National Laboratory, USA
* Bob Grossman, University of Illinois at Chicago, USA
* Indranil Gupta, University of Illinois at Urbana Champaign, USA
* Alexandru Iosup, Delft University of Technology, Netherlands
* Kamil Iskra, Argonne National Laboratory, USA 
* Chuang Liu, Ask.com, USA
* Zhou Lei, Shanghai University, China
* Shiyong Lu, Wayne State University, USA
* Reagan Moore, University of North Carolina at Chapel Hill, USA
* Marlon Pierce, Indiana University, USA
* Ioan Raicu, Northwestern University, USA
* Matei Ripeanu, University of British Columbia, Canada
* David Swanson, University of Nebraska, USA
* Greg Thain, Univeristy of Wisconsin, USA
* Mike Wilde, University of Chicago & Argonne National Laboratory, USA
* Matthew Woitaszek, The University Corporation for Atmospheric Research, USA
* Sherali Zeadally, University of the District of Columbia, USA
* Yong Zhao, Microsoft, USA

To return to MTAGS09 main page, click here.