When it comes to ClickHouse, how many columns are too many? 100 columns? 5,000 columns? In our latest experiment, we put 10,000 columns to test using some configuration tweaks and enough memory. Click to find out what happened!
A few weeks ago the SingleStore team published interesting research in their blog. They demonstrated how to load 100 billion rows in a database in 10 minutes. While it did not seem outstanding for ClickHouse, we were intrigued to learn how much faster we can accomplish the same task, so we conducted some experiments in Altinity.Cloud.
ClickHouse indices are different from traditional relational database management systems. Instead, ClickHouse uses secondary ‘skipping’ indices. Those are often confusing and hard to tune even for experienced ClickHouse users.
In this article we will dive into skipping indices and learn some of ClickHouse black magic.