DataSys: Data-Intensive Distributed Systems LaboratoryData-Intensive Distributed Systems Laboratory

Illinois Institute of Technology
Department of Computer Science

XStore: Efficient Time Series Storage System

XStore In recent years, we have seen an un-precedented growth of data in our daily lives ranging from health data from an Apple Watch, financial stock price data, volatile crypto-currency data, to diagnostic data of nuclear/rocket simulations. The increase in high-precision, high-sample-rate timeseries data is a challenge to existing database technologies. We have developed a novel technique that utilizes sparse-file support to achieve O(1) time complexity in create, read, update, and delete (CRUD) operations while supporting time granularity down to 1 millisecond. We designed and implemented XStore to be lightweight and offer high performance without the need to maintain an index of the timeseries data. We are conducting a detailed evaluation between XStore and existing best-of-breed systems such as MongoDB and InfluxDB using real-world cryptocurrency data across dozens of coins, years of data, with millisecond granularity, totaling over a trillion datapoints.  

 

Publications

 

Presentation