Open Source Summit North America (OSS NA)

Virtual Experience: Open for All – Anytime. Anywhere.
June 29 – July 2, 2020

Join us for Altinity presentation:
Breaking out of the Proprietary Cage: Real-time Data Warehouses Come to Open Source
by Robert Hodges, Altinity LTD
Wednesday, July 1 • 12:15pm – 1:05pm

SQL data warehouses are the foundation of high performance analytic solutions on structured information like market tick data or network flow logs. Until recently this type of application has been dominated by proprietary solutions. The situation changed radically with ClickHouse, an Apache 2.0 SQL data warehouse that meets or exceeds performance of Vertica, Redshift, and other closed-source implementations. This talk starts with a quick history of data warehouses including recent open source offerings. We then dive into ClickHouse and study optimizations it uses to achieve record-setting speed. These include compression, vectorized query processing, SIMD instructions, and efficient pre-aggregation through materialized views. Finally we’ll show how open source tools like Kafka and Grafana combine with ClickHouse to create complete solutions that offer sub-second response on datasets running into trillions of rows and ingested at millions of events per second. The resulting stack enables users to add real-time analytics to any application without the constraints of proprietary licenses. It’s a key open source innovation for the age of big data.