Our colleague Mikhail Filimonov just published an excellent ClickHouse Kafka Engine FAQ. It provides users with answers to common questions about using stable versions, configuration parameters, standard SQL definitions, and many other topics. Even experienced users are likely to learn something new.
But what if you are getting started and need help setting up Kafka and ClickHouse for the first time? Good news! This article is for you.
Kafka is a popular way to stream data into ClickHouse. ClickHouse has a built-in connector for this purpose -- the Kafka engine. Our friends from Cloudfare originally contributed this engine to ClickHouse. The Kafka engine has been reworked quite a lot since then and is now maintained by Altinity developers. It is not always evident how to use it in the most efficient way, though. We tried to fill the gap with a Kafka webinar, which was a success. In this article we collected typical questions that we get in our support cases regarding the Kafka engine usage. We hope that our recommendations will help to avoid common problems.Read More
April 1, 2020
with Robert Hodges and Ned McClain
You are about to deploy ClickHouse into production. Congratulations! But what about monitoring? In this webinar we will introduce how to track the health of individual ClickHouse nodes as well as clusters. We'll describe available monitoring data, how to collect and store measurements, and graphical display using Grafana. We'll demo techniques and share sample Grafana dashboards that you can use for your own clusters.
Important notice for our beloved Apache Kafka users. We continue to improve Kafka engine reliability, performance and usability, and as a part of this entertaining process we have released 220.127.116.11 Altinity Stable ClickHouse release. This release supersedes the previous stable release 18.104.22.168, and addresses following problems:Read More
January 21, 2020
with Robert Hodges and Mikhail Filimonov
Apache Kafka is a popular way to load large data volumes quickly to ClickHouse. In this webinar we will cover best practices for integrating Kafka and ClickHouse including setup of Kafka clusters, defining materialized views to pull data into ClickHouse, and organization of target tables. We will also cover alternatives like writing your own connector applications. Altinity is now the community maintainer for ClickHouse Kafka. Our lead Kafka engineer Mikhail Filimonov will be in the webinar to share the latest tips for successful Kafka use. Please join us and bring your questions!