SkyServer Web Service

1Ioan Raicu, 1,2Ian Foster

1Computer Science Dept., The University of Chicago, iraicu@cs.uchicago.edu

2Math and Computer Science Div., Argonne National Laboratory, foster@mcs.anl.gov

Abstract

The AstroPortal project [7] operates on the SDSS DR4 [6] dataset to perform a stacking operation on an arbitrary set of images from the SDSS DR4 dataset.  Each image is composed of multiple objects, while each object is described by meta-data information such as the {RA DEC} coordinates, the band the image was taken in, calibration information, etc…  When a user specifies a stacking to be performed, the input is a list of coordinates and bands in the form {RA DEC BAND}.  This tuple needs to be converted to the actual location of the data where the image can be found.  We have had a solution to this problem for quite some time, namely the SkyServer [1], but we wanted to explore some different alternatives and compare the results.  We used a KDTree implementation to allow efficient search of a 2 dimensional space for both the nearest neighbor and for all objects within a rectangular range.    

Originally, we used the SkyServer [1] to pose queries based on this tuple and in return, we would get back the position of the data.  We had several motivations to seek some alternate means to do this translation:

1)       the SkyServer is a shared system so the performance will vary

2)       the interface into the SkyServer via an HTTP request with only 1 translation at a time seemed very inefficient

3)       the production SkyServer has a limit of 60 HTTP queries per minute, much too small for utilizing the AstroPortal resources efficiently. 

We are also pursuing obtaining a copy of the SkyServer and bringing it up online locally on a dedicated resource, at which time we hope that at least limitation #1 and #3 from above will be solved.  There is also an online form at [2] that allows the translation of multiple tuples at the same time which would solve limitation #2, but this form has a limit of 80K input size, which limits the number of translations to about 2500 at a time.  Furthermore, we need to find out how to use this online form in a systematic programming friendly way.

 

Documents / Source Code

  1. Report describing our motivation, implementation, interface, and performance of the SkyServer Web Service: (readme.SkyServerWS.pdf ~ 228KB)

  2. Download the the SkyServer Web Service v0.5 (more difficult to configure, start and use): (SkyServerWS.0.5.tgz ~ 3.7MB)   

  3. Brief report describing the SkyClientWS code and scripts needed to compile and run the client code: (readme.SkyClientWS.txt ~ 5KB)

  4. Download the the SkyClientWS v0.5 (trivial to compile and use): (SkyClientWS.0.5.tgz ~ 3MB)   

 

References

[1]       Sloan Digital Sky Survey / SkyServer, http://cas.sdss.org/astro/en/

[2]       SDSS DR4 Tools, http://cas.sdss.org/dr4/en/tools/crossid/upload.asp

[3]       KDTree Implementation in C++, http://www.cs.umd.edu/~mount/ANN/

[4]       FunTools, http://hea-www.harvard.edu/RD/funtools/

[5]       KDTree Implementation in JAVA, http://www.cs.wlu.edu/~levy/kd/

[6]       SDSS Data Release 4 (DR4), http://www.sdss.org/dr4/

[7]       Ioan Raicu, Ian Foster, Alex Szalay, Gabriela Turcu.  “AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis”, to appear at TeraGrid Conference 2006, June 2006.

 

Webmaster Ioan Raicu: iraicu@cs.uchicago.edu 
Last modified: January 11, 2008