Multi-volume storage is crucial in many use cases. It helps to reduce storage costs as well as improves query performance by allowing placement of the most critical application data on the fastest storage devices. Monitoring data is a classic use case. The value of data degrades rapidly over time. The last day, last week, last month, and previous year data have very different access patterns, which in turn correspond to various storage needs.
In data analytics, analysts often use moving averages. Moving averages help to smooth data series as well as identify long term trends. New Live View tables start to bring real-time capabilities to ClickHouse.
One of the applications of Live View tables is a calculation of real-time metrics on the event data. Readings from IoT sensors, price ticks from the stock exchange, or some metrics from your production servers are some of the examples of event data streams. ClickHouse can store all this data with a good compression ratio and excellent analytical query performance.
In the couple of previous blog posts, I have introduced Live Views tables and covered basic usage. Now, in this post, we will take a closer look at Live View tables. Specifically, we will look at the options available for the WATCH query, then introduce temporary Live Views, as well as look at the new JSONEachRowWithProgress format.
My previous blog article introduced Live View tables, a new feature in ClickHouse that enables users to subscribe to query results and get new results as the underlying input changes.
In this article, we will continue to explore using Live View tables but now using a real dataset.
Have you ever wanted to get a notification from your database when the query result has changed due to new data? Have you ever wanted to build a real-time dashboard where your graphs would change in real-time based on fresh aggregates from your ClickHouse cluster? Well, starting in the 22.214.171.124 ClickHouse release, an experimental feature was added to ClickHouse that you most likely did not notice.