Home

![alt text][logo] [logo]: https://raw.githubusercontent.com/epfldata/squall/master/resources/graphics/logo.jpg "Logo Title Text 2"

#Squall Squall is an online query processing engine built on top of Storm. Similar to how Hive provides SQL syntax on top of Hadoop for doing batch processing, Squall executes SQL queries on top of Storm for doing online processing. Squall supports a wide class of SQL analytics ranging from simple aggregations to more advanced UDF join predicates and adaptive rebalancing of load. It is being actively developed by several contributors from the EPFL DATA lab. Squall is undergoing a continuous process of development, currently it supports the following:

• SQL (Select-Project-Join) query processing over continuous streams of data.
• Full fledged & full-history stateful computation essential for approximate query processing, e.g. Online Aggregation.
• Time based Window Semantics for infinite data streams (currently in-progress).
• Theta Joins: complex join predicates, including inequality, band, and arbitrary UDF join predicates. This gives a more comprehensive support and flexibility to data analytics. For example, Hive plans to support theta joins in response to user requests.
• Continuous load balance and adaptation to data skew.
• Usage: An API for arbitrary SQL query processing or a frontend query processor that parses SQL to a storm topology.
• Throughput rates of up to Millions of tuples/second and average latencies of tens of milliseconds on a 220 machine cluster. Scalable to large cluster settings. Further details can be found in this whitepaper.
• Out-of-Core Processing: Can operate efficiently under limited memory resources through efficient disk based datastructures and indexes.
• Elasticity: Scaling out according to the load.
• DashBoard: Integrating support for real time visualizations.

Documentation

• Rationale
• [A High-Level Overview](https://github.com/epfldata/squall/wiki/A high-level overview)
• Stream Query Applications
• [Supported Features](https://github.com/epfldata/squall/wiki/Supported features)
• [Quick Start: Local Mode](https://github.com/epfldata/squall/wiki/Quick Start: Local Mode)
• [Quick Start: Cluster Mode](https://github.com/epfldata/squall/wiki/Quick Start: Cluster Mode)
• [Squall Query Plans vs Storm Topologies](https://github.com/epfldata/squall/wiki/Squall query plans vs Storm topologies)
• [Manually Writing Squall Query Plans](https://github.com/epfldata/squall/wiki/Manually writing Squall query plans)
• Theta-Join Usage & Implementation
• Squall Configurations: Local Mode Configurations and Cluster Mode Configurations
• Code Recompilation
• Troubleshooting
• Extension Ideas

Contributing to Squall

We'd love to have your help in making Squall better. If you're interested, please communicate with us your suggestions and get your name to the Contributors list. Here is a list of some current interests we have. All questions and suggestions are welcomed.

License

Squall is licensed under Apache License v2.0.