The final article of our series on managing large datasets in ClickHouse MergeTree covers how to clean up orphan S3 files. We conclude with ideas for future improvements to S3 storage management.
Our series on managing large datasets in ClickHouse MergeTree tables continues. In Part 2 we discuss best practices for S3 bucket configuration, storage policies, and MergeTree table administration.
ClickHouse datasets are getting big. Store MergeTree table data on S3 object storage to handle them economically and flexibly. Part 1 covers architecture and storage organization.
A few weeks ago we discussed how to read external Parquet data from ClickHouse. The article raised some performance concerns when querying big collections of Parquet files out of AWS S3 buckets. Time flies fast. Altinity engineers have made some…
ClickHouse support for Apache Parquet is outstanding. We blogged about it last year in “What’s Up With Parquet Performance in ClickHouse”. In this article, we will look at Parquet from a different perspective. Let’s say there is public Parquet data…
Open source licensing helped ClickHouse grow. Now the project is moving to open core. There’s reason for users to be concerned.
S3-compatible object storage support is critical for ClickHouse applications. There is a new community proposal to make it much better.
Supported in ClickHouse, Apache Parquet has use cases other than just a storage format in the Hadoop ecosystem. See what results we got when we tested it in Altinity.Cloud to query Parquet files at S3 with the same efficiency as…
A few weeks ago the SingleStore team published interesting research in their blog. They demonstrated how to load 100 billion rows in a database in 10 minutes. While it did not seem outstanding for ClickHouse, we were intrigued to learn…
Learn how you can load and test ClickHouse datasets with lightning speed and leverage the power of cloud operation with this Altinity.Cloud tutorial. PS…You’ll find it’s really easy too!
ClickHouse often runs in a cluster, and cluster operation poses some interesting questions regarding S3 usage. They include parallelizing data load across nodes, benefits of horizontal vs. vertical scaling, and avoiding unnecessary replication. In this article, we will discuss how…
ClickHouse now fully supports both AWS S3 and MinIO as S3-compatible object storage services. In this comparison, we will test the performance of AWS S3 and MinIO when used to store table data from two of our standard datasets: the…
