DiPerF / ServMark (Distributed Performance testing Framework)
DiPerF is a distributed performance‑testing framework, aimed at
simplifying and automating service performance evaluation. DiPerF
coordinates a pool of machines that test a target service, collects and
aggregates performance metrics, and generates performance statistics.
The aggregate data collected provide information on service throughput,
on service ‘fairness’ when serving multiple clients concurrently, and
on the impact of network latency on service performance. Furthermore,
using this data, it is possible to build predictive models that
estimate a service performance given the service load.
We have tested
DiPerF on 200+ machines and with 3000+ clients on four testbeds, TeraGrid, Grid3, CS@Uchicago cluster,
and PlanetLab, and
explored the performance of instance creation, gridftp server, job submission services (pre‑WS GRAM and WS GRAM),
MDS (Index Service), DI-GRUBER and other services
built or included with Globus Toolkit® 3.2, GT3.9.5 and GT4.0.
In a schematic way the framework is presented in the next figure.
, the second phase of DiPerF and GrenchMark, blends the two projects
for providing not only better service perfromance evaluation, but also automated Grid environment testing.
ServMark is a novel testing architecture and the integration
of DiPerF and GrenchMark brings together into one single package two orthogonal issues in
performance evaluation, namely the ability to test environments and services in a distributed and scalable
way, and the capacity to generate and run dynamic test workloads with complex structure.
The ServMark project, jointly developed by people from U. Chicago, TUDelft, UBC and UPB,
has been accepted as part of the Globus Incubator project.
April 2005: The first release of the framework is available (sources).
April 2006: The second release (enhanced analyzer and submitters + bug fixes) of the framework is available (sources, RPM-Suse.9.x, and DEB-Ubuntu.6.x).
Additional "tester" examples are posted here.
DiPerF Project Participants:
- to the Globus Team for using the framework for various test scenarios.
- to Jennifer Schopf for her suggestions to include new testing schemas.
- to Peter Gardfjall and Johan Tordsson for their feedback and code patches.
- to Lingyn Yang for requiring additional scenarios
July 2006: A first beta release of the ServMark project is available (sources).
July 20, 2006: Project moved to Globus Incubator/ServMark
ServMark Project Participants:
DiPerF / ServMark Documentation:
- DiPerF: An automated DIstributed PERformance testing Framework, C. Dumitrescu, I. Raicu, M. Ripeanu, I. Foster - GW2004, Pittsburgh, PA ([ps|pdf])
The Design, Performance, and Use of
DiPerF: An automated DIstributed PERformance testing Framework, I. Raicu, C. Dum
itrescu, M. Ripeanu, I. Foster - Special Issue of Journal on Grid Computing ([ps
A Performance Study of the Globus Toolkit and Grid Se
rvices via DiPerF, An Automated Distributed Performance Testing Framework, I
. Raicu, MSc Thesis, The University of Chicago, May 2005 ([ps|pdf])
- A Scalability and Performance Measurements of a Distributed Grid Resource Broker, C. Dumitrescu, I. Foster, I. Raicu - iVDGL/GriPhyN TechReport
- Zebra: The Globus Stripped GridFTP Framework and Server, Allcock, B., Bresnahan, J., Kettimuthu, K., Link, M., Dumitrescu, C., Raicu, I., Foster, I. - SC05, Seattle, WA ([ps|pdf])
a Distributed Approach to Grid Resource Brokering, C. Dumitrescu, I.
Raicu, I. Foster - SC05, Seattle, WA ([ps|pdf])
- A nice MDS testing picture (MDS community testing call-meeting 2005):
- Extending a Distributed Usage SLA Resource Broker to Support Dynamic Grid
Environments, C. Dumitrescu (UChicago), I. Raicu (UChicago), I. Foster (UChica
go/Argonne) - submitted to EuroSys06 ([ps|pdf
- A Layered Framework for Connecting Client Objectives and Resource Capabil
ities - A. Dan (IBM), C. Dumitrescu (TUDelft/CoreGrid), K. Ranganathan (IBM), M. Ripeanu
(UBC), - Journal of Digital Libraries and the Cooperative Information Systems 06
- Problems for Resource Brokering in Large and Dynamic Grid Environments, C. Dumitrescu (TUDelft/CoreGrid) - Euro-Par 2k6, Dresden, 2006 ([ps|pdf])
- Distributed Monitoring and Information Services for the Grid, J. M. Schopf, NeSC, September 2005 ([ps|pdf|ppt])
- Monitoring and Discovery in a Web Services Framework: Functionality and P
of Globus Toolkit MDS4, Jennifer M. Schopf (Argonne), Ioan Raicu (UChicago), La
ura Pearlman (ISI),
Neill Miller (Argonne), Carl
Kesselman (ISI), Ian Foster (Argonne/UChicago), Mike D'Arcy (Argonne) submitted
to HPDC 2006,
available as # MCS Preprint #ANL/MCS-P1315-0106, January 2006.
- Scalable Grid-wide capacity allocation with the SweGrid Accounting System (SGAS), P. Gardfjall, E. Elmroth, L. Johnsson, O. Mulmo, T. Sandholm, Submitted to CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE, 2006 ([ps|pdf])
- Capacity Allocation Mechanisms for Grid Environments, P. Gardfjall, licentiate thesis, Umea University, october 2006 ([ps|pdf])
- A standards-based Grid resource brokering service supporting advance reservations, coallocation and cross-Grid interoperability, E. Elmroth, J. Tordsson, to be submitted to CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE, November 2006 ([ps|pdf])
- ServMark (GrenchMark + DiPerF): A Proposal for Globus Project Incubation, C. Dumitrescu (UMUE), A. Iosup (TUD), D. Epema (TUD), I. Raicu (UChicago), I. Foster (ANL/UChicago), M. Ripeanu (UBC) ([ps|pdf])
- Towards ServMark, an Architecture for Testing Grid Services, M. Andreica (UPB), N. Tapus (UPB), C. Dumitrescu (UMUE), A. Iosup (TUD), D. Epema (TUD), I. Raicu (UChicago), I. Foster (ANL/UChicago), M. Ripeanu (UBC) ([ps|pdf])
- ServMark: A Distributed Grid and Services Testing Framework - GGW slides (ps|pdf) and with GlobusAlliance at SC06 (link)
- Towards ServMark, an Architecture for Testing Grids - DSL Seminar Presentation ([ps|pdf])