ClickHouse Data Integration — September 2020 Meetup Report

ClickHouse Data Integration — September 2020 Meetup Report

Our last ClickHouse SF Bay Area meetup was on September 10th and focused on data integration. This turned out to be a popular subject. We had well over 100 signups with 59 people in attendance. This article summarizes the talks and provides links to slides and videos.

ClickHouse and ProxySQL queries rewrite

ClickHouse and ProxySQL queries rewrite

ProxySQL is a popular open source, high performance and protocol-aware proxy server for MySQL and its forks. Since September 2017 ProxySQL supports ClickHouse as a backend, so clients can connect to ClickHouse via MySQL protocol. In practice, this helps MySQL-aware applications to start using ClickHouse as without changes in the client library.

To avoid some limitations to this approach, ProxySQL creator René Cannaò added additional functionality for query rewrite. With his permission, we cross-post his article describing new functionality in our blog.

Realtime replication from MySQL to ClickHouse in practice

July 2, 2018
Vladislav Klimenko from Altinity and Valery Panov from Ivinco presented a talk at HighLoad Siberia 2018 conference recently. They described the real problem that Ivinco faced and how it has been solved with migration of analytics from MySQL into ClickHouse using MySQL to ClickHouse replication. A few months ago we introduced clickhouse-mysql tool in our blog, and Ivinco was the first company we know that tried it and used it in production.

Archiving MySQL Tables in ClickHouse

Archiving MySQL Tables in ClickHouse

Why Archive? Hard drives are cheap nowadays, but storing lots of data in MySQL is not practical and can cause all sorts of performance bottlenecks. 

In this article Percona’s blogger Alexander Rubin is talking about archiving MySQL tables in ClickHouse for storage and analytics.

Aggregate MySQL data at high speed with ClickHouse

Aggregate MySQL data at high speed with ClickHouse

Feb 12, 2018
There are multiple ways how ClickHouse and MySQL can work together. External Dictionaries, ProxySQL support or [realtime streaming] of MySQL binary logs into ClickHouse. A few weeks ago ClickHouse team has released mysql() table function that allows to access MySQL data directly from ClickHouse. This opens up a number of interesting capabilities. Accidentally we have found [a blog article in Japanese] by Mikage Sawatari? , that tests a new way of integration and translated it for our blog with some minor edits.

MySQL to ClickHouse data migration and replication

Altinity has developed an open source clickhouse-mysql tool that helps to migrate existing MySQL table to ClickHouse and setup realtime replication of changes (INSERTS only). Here we review necessary steps using a simple example.