Scale ClickHouse® Queries Infinitely with 10x Cheaper Storage: Introducing Project Antalya

Recorded: April 16 @ 08:00 am PDT
Presenters: Robert Hodges, Alexander Zaitsev, and Tim Glaser

This webinar introduces Project Antalya, Altinity’s initiative to scale ClickHouse® queries on Apache Iceberg-based data lakes with far cheaper storage and independently scalable compute. Robert Hodges, CEO of Altinity, opens by framing the problem: data sets that were measured in tens of terabytes five years ago now reach multiple petabytes per day for some customers, pushing the shared-nothing ClickHouse architecture to its limits. Block storage replication costs up to 10x more than S3, and a single ClickHouse process handling ingest, merges, and queries simultaneously makes it difficult to scale compute independently, leading to overprovisioning and wasted spend.

Robert introduces Project Antalya as the answer: a 100% open source ClickHouse build that moves shared data onto Apache Iceberg, introduces stateless swarm clusters for scalable parallel query execution, and adds distributed caching and tiered storage to Iceberg. Alexander Zaitsev, CTO of Altinity, then demos the Antalya build live, showing a blockchain dataset query that takes 25 seconds on a single 2-vCPU node drop to 4 seconds with a five-node swarm, then to 2.5 seconds with caching enabled, and ultimately to under one second using HTTP caching and full-query swarm delegation. A live Iceberg catalog demo runs the same query on an Iceberg-backed table via the swarm at around 2 seconds, with Alexander noting that Iceberg performance improvements have only just begun.

Tim Glaser, co-CEO of PostHog, then describes how PostHog, a product analytics platform managing low double-digit petabytes with billions of events per day, is designing its next-generation cluster architecture around Antalya swarm clusters. PostHog is actively testing three storage backends: AWS S3 Table Buckets, AWS Glue, and a self-hosted catalog with native S3. The session closes with a roadmap overview, a feature comparison between Antalya and standard ClickHouse builds, instructions for getting started, and a Q&A covering reliability, ReplacingMergeTree, writes to Iceberg, upstream contribution strategy, and swarm autoscaling.

Here are the slides:

Key Moments (Timestamps)

Key moments generated with AI assistance.

• 0:07 – Welcome and housekeeping
• 1:25 – Introductions: Robert Hodges, Tim Glaser (PostHog), Alexander Zaitsev
• 2:18 – About Altinity: products and open source contributions
• 3:13 – Datasets are getting bigger: the scale problem facing ClickHouse users
• 4:32 – Storage and compute cost limitations in open-source ClickHouse
• 6:24 – Data lakes to the rescue: Parquet, Iceberg, and open table formats
• 8:55 – How Iceberg works: reads, writes, and the challenge for real-time analytics
• 11:03 – Introducing Project Antalya: three levels of innovation
• 15:22 – Stateless swarm clusters: compute-storage separation in action
• 17:31 – Caches make distance disappear: four key cache types
• 19:48 – Coming soon: tiered storage to Iceberg
• 21:08 – Long-term vision: data lakes as primary storage with materialized views
• 23:14 – Demo: swarm query speedups from 25 seconds to sub-1 second
• 34:14 – PostHog: current architecture and plans for Antalya
• 41:00 – Project Antalya roadmap
• 42:32 – Antalya vs. standard ClickHouse builds
• 44:08 – How to get started with Project Antalya
• 49:44 – Q&A

Webinar Transcript

[0:07] – Welcome and Housekeeping

Robert: Welcome to our webinar today, introducing Project Antalya. We’re going to show you how to scale ClickHouse® queries infinitely with 10x cheaper storage. This is a topic of wide interest, and you can look forward to an interesting presentation.

Before we dive in, a little housekeeping. This presentation is being recorded and we will send you a link afterward, along with the slides in PDF form. You can post questions into the Q&A box and we will try to answer them as we go along in the background. We also have office hours coming up next week, and we’ll post the link for that toward the end of the webinar.

[1:25] – Introductions

Robert: My name is Robert Hodges. I’m CEO of Altinity. It’s my pleasure to have with us Tim Glaser, co-CEO of PostHog.

Tim: Hey, everyone. I’m Tim. I founded PostHog with my co-founder about five or six years ago. We are big, big users of ClickHouse, so I’m really excited about Antalya.

Robert: Tim and his team have been a big collaborator on this effort. I’d also like to introduce Alexander Zaitsev, CTO of Altinity and the architect behind what you’re seeing today.

Alexander: Hey, guys. Good to see everybody here.

[2:18] – About Altinity

Robert: Let me introduce Altinity. We help people build better on open source ClickHouse. We’ve been helping users build better ClickHouse applications since 2017. We operate Altinity.Cloud®, which was the first cloud for ClickHouse in the American public clouds: Amazon, GCP, now Azure, and Hetzner. We do Altinity Stable® Builds for ClickHouse®, the enterprise open source build for ClickHouse. Many of you are already using another piece of software we write: the Altinity Kubernetes Operator for ClickHouse®, possibly the most popular thing we’ve ever written. We also build Altinity Backup for ClickHouse® and many other things. Our next step, which we are here to talk about today, is the ability to use Iceberg as shared storage.

[3:13] – Datasets Are Getting Bigger

Robert: This begins with something we’ve experienced, and that you’ve likely experienced too. Five years ago, when we were beginning to manage ClickHouse in the cloud, most of our customers were dealing with data set sizes in the range of a few tens of terabytes at most. You could basically handle those using attached storage on a single host, replicated for performance and availability. Since then, data has grown enormously. We’re dealing with observability data sets that run up to 100 petabytes. To give you a sense of the scale, one customer we’re working with is putting in two petabytes of data per day. That number alone is larger than any of the data sets we were helping manage five years ago. This is beginning to outstrip, and in fact already has outstripped, the capacity of open-source ClickHouse, leading to very practical problems around storage and compute cost.

[4:32] – Storage and Compute Limitations in Open-Source ClickHouse

Robert: The thing that really strikes people, and that we hear from customers regularly, is: I’m looking at my cloud bill and I’m seeing a big chunk devoted to storage and compute for these real-time analytic systems.

There are two root causes. First, in open-source ClickHouse today the normal architecture for storage is block storage, for example Amazon EBS, then replicated. That storage is expensive. It is high performance, but if you compare it to storing data on S3, it is about 10 times more expensive for typical systems.

Second, when you have large systems you have several different types of use cases running simultaneously: ingesting data, querying data, batch reporting, real-time dashboards. The problem in open-source ClickHouse today is that a single server process is handling all of those requirements at once. To ensure adequate performance, people provision for peak load, and it is hard to scale compute up and down quickly. That leads to wasted resources that just keep showing up on your bill.

[6:24] – Data Lakes to the Rescue

Robert: The problem of dealing with large data sets in a scalable, cost-effective way is not new. Data lakes arose over the last decade to address it, driven primarily by batch reporting, data science, and AI training. This started with people just putting files on object storage, but over the past couple of years we’ve seen an explosion in the use of Parquet, a columnar format very similar to what MergeTree does, and it is now pervasive inside enterprises. Not only are people using Parquet, but they’re increasingly using open table formats, which organize those files into something that looks like SQL tables. The most popular is Iceberg, followed by Delta Lake. When we talk to people, it’s roughly 75% Iceberg and 25% Delta Lake. Delta Lake is very popular among Databricks and Spark users.

Why has this been so successful? It’s on object storage, so you get cheap storage. The data is in a single copy that everybody can access independently without blocking each other or multiplying storage costs. The formats are fully open: both Parquet and Iceberg are open specifications backed by a wide range of libraries, including PyIceberg and PyArrow in Python. And a final important point: data lakes run most easily inside user accounts, where everybody can see and connect to them.

[8:55] – How Iceberg Actually Works

Robert: As we think about applying data lakes to real-time analytics, it’s worth digging into how Iceberg actually works. The key insight is that with open table formats, there is no longer a database gatekeeping the storage. You can think of this as database storage that has been freed up so that everybody can access it regardless of their client.

For reads, a Python data science application connects to an Iceberg REST catalog, which serves as a single point of contact for metadata. The catalog tells you where the tables are and what files are in them. The library then finds and reads the Parquet files it needs for the query. For writes, the process is essentially reversed: you write the Parquet files and then update the Iceberg metadata to register the new version of the table.

This works well for batch applications. But applying it to real-time databases creates two problems. First, reading Parquet on Iceberg can be very slow unless you do a lot of work to speed it up. Second, writing to Iceberg is hard. There are not many libraries that support it well, and enabling that write path for Iceberg as proper database storage requires significant extra work. We’ve been working on both of those problems, and they are the core of what we’re doing with Project Antalya.

[11:03] – Introducing Project Antalya

Robert: There are multiple levels of innovation necessary to make this work. The first is to make I/O on Parquet data very fast. That involves taking advantage of features inside Parquet like bloom filters, implementing the PREWHERE optimization that makes ClickHouse fast on MergeTree but applied to Parquet columns, caching metadata so we can decide whether to even look at a particular Parquet file, and carefully managing caches for blocks we’re reading from S3. The bandwidth to S3 is very good, but you still don’t want to go back to it unnecessarily.

The second thing is cheap compute and specifically enabling separation of storage and compute. For that we’re introducing stateless compute swarms. A lot of this presentation is about that. We’re also introducing distributed caching and, in the near future, tiered storage to Iceberg so cold data can move there automatically. These are all changes that appear inside ClickHouse itself as new builds.

The third thing is ensuring you can actually run this in real operational environments. There is more Kubernetes wiring and management than you’d expect to build systems that take advantage of this. We’re working on Kubernetes blueprints for autoscaling, cache deployment, Iceberg tooling, and monitoring.

The result is the full Project Antalya architecture. When you connect to a native ClickHouse cluster, you see all the data through a single SQL connection regardless of whether it’s in Iceberg or native MergeTree storage. Native ClickHouse tables can extend seamlessly to a single copy on Iceberg Parquet. You can scale compute flexibly using cheap spot instances through swarm clusters. And you can deploy anywhere: in a vendor account like ours, in your own account through a BYOC model, or in an on-premises HPC environment.

[15:22] – Stateless Swarm Clusters

Robert: Stateless swarm clusters are simply a collection of ClickHouse servers that have no schema, do no replication, and hold no storage beyond what is needed for caches. This is your basic mechanism for compute-storage separation. There is a new setting available in open-source Antalya builds called object_storage_cluster. You give it the name of a swarm, and when the query is processed, reads of Parquet files are delegated out to the swarm, which goes directly to Iceberg in parallel. The native ClickHouse cluster, the initiator, assembles the results and hands them back. All of the heavy scanning work is pushed to the swarm without affecting your existing ClickHouse system.

Swarms are flexible. You can run as many as you like, each configured for a different workload. Imagine a swarm sized for operational dashboards that runs at constant load, alongside a second swarm you spin up on demand to handle batch reporting that needs to scan the entire table on Iceberg. This is an important form of flexibility, and Tim will show you real examples shortly.

[17:31] – Caches Make Distance Disappear

Robert: Caches are fundamental in distributed systems and are particularly important when reading from Iceberg. Let me walk through the four key cache types available in Antalya.

First is the Iceberg metadata cache. When doing query planning, you don’t want to constantly pull complex JSON files from the Iceberg catalog and parse them. Keeping them in memory is critical.

Second is the filesystem cache in the compute swarm. When you read a block from S3, you drop it into a local filesystem cache. This also loads it into the buffer pool, so the next time you access that block, the operating system may find it in memory directly.

Third is the Parquet metadata cache. This lets ClickHouse avoid reopening Parquet files repeatedly to re-read min-max indexes. The ability to decide quickly whether to skip a file or specific row groups is critical to query speed.

Fourth is an external HTTP cache for S3 list operations. Calls like list-objects on S3 are extremely expensive and return results in pages, typically a thousand objects at a time. Caching these has a huge effect on query performance.

[19:48] – Coming Soon: Tiered Storage to Iceberg

Robert: Tiered storage is a very important existing feature of ClickHouse today. It lets you keep hot data on NVMe SSD while cold data automatically moves to cheaper storage for long-term preservation. What we’re implementing is tiered storage for Iceberg. From a user’s perspective, just like tiered storage today, you select from a single table. Under the hood, the query is split: the hot part goes straight to block storage, and the cold part is dispatched to the compute swarm to read from Iceberg. TTL moves let you configure simple time-based rules for when data migrates off hot storage and out to Iceberg. When this is fully enabled, it should drop costs by around 80% in many cases.

[21:08] – Long-Term Vision: Data Lakes as Primary Storage

Robert: The long-term goal is to open up data lakes as primary storage and build creative new types of applications on top of them. One example I want to highlight is materialized views. This is already a great ClickHouse feature. In future versions of Antalya, materialized views will fire automatically as hot data arrives directly in Iceberg. The materialized view gets built, and when users run a SELECT, they can use it to look for outliers or interesting events, then zero in on source records by dispatching a query out to the swarm. This kind of integration of native ClickHouse storage with Iceberg storage will enable very performant applications that handle large amounts of data while keeping costs down.

Swarm queries work today. We’ve been working hard on these for months. Now let me hand it over to Alexander for the demo.

[23:14] – Demo: Swarm Query Speedups from 25 Seconds to Sub-1 Second

Alexander: Thank you, Robert. We have a lot of interesting questions in the chat, which we’ll get to at the end. Let me share my screen.

What I’m going to show you today is how swarm clusters work. I’m using Altinity.Cloud as a test bed and UI, but this could be run in any Kubernetes environment. In this environment I have three clusters: my-house, which is the entry point and a single-shard, single-replica node, and two swarm clusters, swarm-one and swarm-two. These swarms launch on demand. You don’t need to run them all the time. They have five shards each, are completely stateless, hold no schema, and can be scaled independently to any amount of compute.

We’re running the Antalya build of ClickHouse, based on upstream version 25.2 with a number of improvements. We’ll be rebasing it on future ClickHouse versions as well.

Let’s start with a very basic query, no swarm at all, against a public blockchain data set: all data for the current year with simple grouping and aggregation. On this small 2-vCPU machine, this typically takes 20 to 30 seconds. It finished in 25 seconds. Extremely slow for ClickHouse, of course, but this machine is constrained.

Now I’ll add the object_storage_cluster setting pointing to swarm-one. As you can see, this is a completely separate ClickHouse instance, but configuration allows us to share the swarm. You don’t need to change your query: just reference the cluster name, which can be part of a user profile. The query that took 25 seconds is now down to 4 seconds, already a 6x improvement. If I repeat it, it drops to 2.5 seconds.

I can also control how many nodes are used for processing. Specifying a single node takes about 7 seconds, while leaving it unspecified uses all available swarm nodes. Now let’s enable caching. With the filesystem cache enabled by default and no need for heavy network transfers, the query drops to 1.6 seconds, balancing around 1.2 to 1.6 seconds on repeat runs.

That’s still not fast enough. Even with caching, why is it 1.6 seconds? Let’s profile the query. I can see a substantial number of S3 list-object calls, 13 in this case, taking almost 1 second. The initiator node, which was not supposed to do heavy work, is still calling S3 heavily. About 50% of query time is being spent on those list requests. ClickHouse needs to check what files are available, build a list, and redistribute it to the swarm, and that operation is slow.

The solution is the HTTP cache, which caches S3 requests with different retention policies. List requests are cached for 5 minutes, sufficient if your bucket isn’t updating every few seconds. Switching to swarm-two, which is preconfigured with the distributed HTTP cache, I’m down to 1.3 to 1.2 seconds on repeat runs.

Now the last optimization: delegating the full query to run on the swarm rather than having the initiator do any list work. Using a remote() function call to the swarm, we get under 1 second.

So: from 25 seconds on a single node at the start, down to 700 to 800 milliseconds. That is a 40x performance improvement, and we didn’t change the original ClickHouse cluster at all. It’s the same small single-node machine with the same schema. We just added a swarm, a completely separate cluster, and offloaded the heavy network and compute work to it.

Now for the Iceberg piece. We have an Iceberg catalog as part of Altinity.Cloud, available as a service soon. I’ve preloaded the same public blockchain data set as an Iceberg table. Without the swarm setting it would be very slow, likely over 30 seconds. With the swarm it comes in at around 2 seconds. We’re at the very beginning of optimizing the Iceberg query pipeline, and we expect Iceberg performance to ultimately exceed plain S3 performance through better partitioning and pruning.

All of this is available right now. You just need an Antalya build and a Kubernetes manifest.

[34:14] – PostHog: Current Architecture and Plans for Antalya

Robert: Thank you, Alexander. Let me hand over to Tim to talk about how PostHog is designing for these new features.

Tim: Thanks, everyone. A brief introduction to PostHog: we provide all the tools in one platform that you need to build successful products. That includes product analytics, web analytics, session replay, a data warehouse, error tracking, feature flags, and experimentation. We are also an open source project, which limits what tools we can and can’t use.

Here is how PostHog currently uses ClickHouse. We store low double-digit petabytes of data and ingest billions of events per day. Our cluster has query coordinators, essentially stateless nodes with a lot of memory that aggregate results from replicas. We have two online replicas and one offline replica. The offline replica handles cache warming, cohort recalculation, and our version of materialized views. The two online replicas serve queries from our customer-facing interface. A portion of customer data also lives in Parquet files on S3, mostly data customers send separately such as their Stripe or PostgreSQL data, and we query those directly from the query coordinators.

The issues we face: tight coupling of compute and storage means we can’t easily scale by workload. In AWS, scaling up means at least a 2x cost increase, through resharding or adding a replica. All data is also replicated three times on expensive NVMe and EBS disk. Comparing that to S3, as Robert said, we’d probably see a 10x reduction in cost. And we don’t have reliable caching of block storage query results.

Here’s what we’re looking at with Antalya. We’ve spun up a test cluster that looks really promising and are about to deploy it against real-world use cases. The idea is that we’ll still have coordinators, but now we’ll have swarm clusters that can dynamically scale up and down with our load. Because so much of our traffic is user-driven, we have a classic day/night load pattern, and the ability to scale elastically during the day is really interesting to us. Even more importantly, as we continue to grow, being able to scale up quickly is a huge help.

We’re also testing three different ways of storing data in S3: AWS S3 Table Buckets, AWS Glue, and native S3 with our own self-hosted catalog. We’re testing all three against Antalya.

[41:00] – Project Antalya Roadmap

Robert: Let’s deal with some nuts-and-bolts questions about timing. The swarm is available now. We’re releasing this because it has reached the point where it needs to get into people’s hands. It has already been with a number of our customers and users over the past couple of months, and we want it to be generally used.

In May, we’ll introduce full support in Altinity.Cloud, including a UI for swarm setup and management, Iceberg catalog tools, and good automation around swarm clusters. There is also work in progress on swarm fault tolerance, specifically handling node disappearance gracefully. Later in the year comes tiered storage. Toward the end of the year we’ll be continuously optimizing swarm autoscaling, caching, and materialized views, which are very high on the request list.

[42:32] – Antalya vs. Standard ClickHouse Builds

Robert: It’s worth comparing Antalya to other ClickHouse builds. It is a ClickHouse build with additional features, in the same way that Altinity Stable® Builds for ClickHouse® are ClickHouse builds from upstream with additional features. The difference is that Antalya has a lot more, and we’re following the head of ClickHouse development more closely rather than sticking to LTS releases.

Fast Parquet and S3 reads should be comparable between builds, because we push Parquet performance work back upstream to share with the entire community. Catalog support flows the other way: the ClickHouse core team and community have done a lot of work there and we help with it. Where Antalya diverges: swarm clusters are specific to Antalya builds. HTTP caching and additional cache management settings are Antalya only. Tiered storage to Iceberg is coming in Antalya builds. And the cloud-native Kubernetes blueprints needed to wire all of this together and actually operate it in production are part of Antalya and not in standard ClickHouse builds.

[44:08] – How to Get Started with Project Antalya

Robert: As of yesterday, we have an examples project that is now open. You can clone it and it shows how to bring up Antalya in Docker as well as Kubernetes. If you have Docker Compose installed, you can cd into the Docker directory, type the script name, and have a demo running in about 20 seconds. It runs great on a laptop.

The simplest way to talk to us is to join our Slack workspace, accessible from altinity.com. That’s the best place for interactive conversations. Raise issues on GitHub for documentation or template-related questions, or on our ClickHouse branch under the Altinity organization for code issues. And we love pull requests. If you are C++ enabled, we would love to hear from you. This is how the Altinity Kubernetes Operator for ClickHouse® became great, and it’s how we’ll make Project Antalya great too.

In summary: exponential data growth is making the 10x storage cost of replicated block storage an increasingly acute problem. Scaling compute up and down independently is a critical need. Iceberg solves the storage problem and enables all workloads to share the same data. Project Antalya is the work we have done to make queries faster, enable scaling, and build systems that operate at scale using cloud-native methods. The code is 100% open source. It runs anywhere. We have an aggressive roadmap covering tiered storage and deeper integration with event streams, AI batch reporting, and data science through shared Iceberg data. Please try it out and contribute. It’s a community effort.

I’d like to thank Tim for coming today and sharing the PostHog team’s work. I’d also like to thank Jump Trading, who are very prominent in the ClickHouse community. A lot of the concepts behind this were developed in discussions with them. This is an issue we’ve been exploring for years, and there have been hundreds of conversations with customers and community friends that have shaped the design you’re seeing today. And finally, I’d like to thank the Altinity engineering team. There are 54 of us, spread worldwide. This is a picture of the engineering team hard at work in Antalya, Turkey, which is actually where the name of this project comes from.

[49:44] – Q&A

Alexander: We have 19 questions, so let’s see how many we can get through.

Marty asks about reliability and error handling. This is not yet fully addressed. If there is a failure, a query may fail. The setup and monitoring story is better: if a node fails, Kubernetes will recover it automatically. We also use spot instances aggressively because they are fast to start. We will definitely address the failure-handling question in the next releases.

On ReplacingMergeTree: it will work, but in a different way than it does in standard ClickHouse. It only makes sense in a tiered scenario where you use ReplacingMergeTree in the hot tier, and once you’re confident data won’t be modified further, you TTL it out to Iceberg for longer retention and storage.

Robert: This is a great illustration of why these data lake applications will be built with a combination of cheap Iceberg storage for the long tail and hot native ClickHouse storage. Change is much easier to handle in native structures. ReplacingMergeTree is the obvious example: you need to be able to merge data to make changes appear. You do that in the hot tier, and once data is settled, it can be safely exported to Iceberg. We’re going to see many other use cases with this same pattern.

Alexander: Regarding Parquet bloom filters: we have implemented bloom filter support for Parquet. If the application writing the Parquet files creates bloom filters, ClickHouse can now read and utilize them. There are other index types in Parquet similar to those in MergeTree, and we’re trying to bring the full MergeTree index experience to Parquet.

Timothy asks about comparing this to SharedMergeTree in ClickHouse Cloud. That is actually 50% of the motivation for Project Antalya. ClickHouse, Inc. decided to keep SharedMergeTree as a proprietary feature of ClickHouse Cloud, leaving the community without a good shared object storage implementation. We initially started community work on building a SharedMergeTree replacement, but eventually redirected that into Project Antalya. Instead of re-implementing SharedMergeTree, we moved to Iceberg, which gives you shared object storage plus a lot of additional good properties: open format, shareable with other workloads, and broad community support. This is the open source alternative to SharedMergeTree, and it has considerably broader capabilities.

Ali asks whether Project Antalya can be used for self-hosted ClickHouse. Absolutely. This is exactly what the Antalya examples project shows. Everything is open source. You can install it and use every feature on a self-hosted cluster.

Gunhar asks whether Antalya is available in the GCP Marketplace. Altinity.Cloud is on the GCP Marketplace, and Antalya clusters will be available there in May. Right now it’s possible to set things up, but in a more manual way. We’re adding automation.

Nathan asks about using serverless Kubernetes providers like AWS Fargate or Google Cloud Run as swarm compute. Not at the moment. We currently assume everything runs in one Kubernetes cluster, which keeps security and connectivity simpler. External compute is possible but requires configuring connectivity properly. This is something we may consider for future releases.

Timafi asks whether we plan to merge all of this into the main ClickHouse repository. We try to merge as much as possible, especially Parquet performance work. Some features we cannot merge because the ClickHouse core team does not accept them, particularly where contributions might conflict with their cloud product. This is why we maintain our own branch. It is not a fork: it is a branch. We keep it in sync with upstream as closely as possible and contribute back as much as we can.

On cache strategy: we are still experimenting with distributed caches. This work is only a couple of weeks old and there is a lot of room for tuning, including different strategies for different use cases.

Joseph asks about dynamically changing query execution as swarm nodes come up during a query. Partially. If you add compute while a query is running, it won’t affect that particular query because planning is done at the start. But new queries will use the additional compute. We plan to build autoscaling tools that monitor swarm workload and add or remove nodes as needed. Because we use spot instances, this will be fast.

Aditya asks whether Antalya writes to Iceberg and whether this is comparable to SharedMergeTree. Writing to Iceberg is in development from both the ClickHouse core team and from us. We need write support for TTL moves and other use cases.

Robert: We’re focused on writing parts initially, which is most consistent with the right patterns for Iceberg at this time. The spec is moving toward better real-time write capabilities. The second thing is that we want to offload the work of building Parquet parts to the swarm, because compressing and serializing Parquet is memory- and CPU-intensive. Moving that into the swarm is something you’ll see coming.

We’re out of time. We’ll answer remaining questions offline. All great questions. Thank you everyone for asking.

Please join our Slack channel, try out the project, give us feedback, and log issues. Tell your friends. Two interesting things to try are Cloudflare Iceberg Buckets and AWS S3 Table Buckets. There was also a suggestion from Sentry to try running swarms on Cloudflare Durable Objects, which would be a fascinating experiment. Come discuss ideas like these with us. This is exactly the kind of innovation that will make Project Antalya truly great. It’s better if we build it together.

FAQ Section

What is Project Antalya, and how does it differ from standard ClickHouse? Project Antalya is a 100% open source ClickHouse build by Altinity that adds stateless swarm clusters for scalable parallel query on Iceberg and Parquet data, distributed caching, and upcoming tiered storage to Iceberg. Standard ClickHouse builds share most Parquet and catalog functionality, but swarm clusters, HTTP caching, cache management settings, and Kubernetes cloud-native blueprints are specific to Antalya builds. Altinity pushes as much Parquet performance work as possible back into upstream ClickHouse.

What is a stateless swarm cluster, and how does it speed up queries? A swarm cluster is a pool of stateless ClickHouse servers that hold no schema and no replicated data, only caches. When you add the object_storage_cluster setting to a query, the initiating ClickHouse node delegates all Parquet file reads to the swarm, which goes to Iceberg in parallel. The initiator then assembles and returns the results. The demo in this webinar shows a query dropping from 25 seconds on a single 2-vCPU node to under 1 second with a five-node swarm and distributed HTTP caching, a 40x improvement.

How does PostHog plan to use Project Antalya? PostHog, which manages low double-digit petabytes of data and ingests billions of events per day on ClickHouse, is building a test cluster around Antalya swarm clusters. Their goal is to replace statically provisioned compute replicas with dynamically scaling swarms that grow and shrink with user-driven load. They are also testing three storage backends: AWS S3 Table Buckets, AWS Glue, and a self-hosted Iceberg catalog with native S3.

What caches are available in Project Antalya, and why do they matter? Antalya provides four categories of caching. The Iceberg metadata cache keeps catalog metadata in memory to avoid constant round trips during query planning. The filesystem cache in swarm nodes caches S3 blocks locally and in the OS buffer pool. The Parquet metadata cache avoids re-reading min-max indexes from Parquet files repeatedly. The external HTTP cache caches expensive S3 list-object responses, which can account for up to 50% of query latency. Together, these caches are what enable sub-second response times on Iceberg data.

What is the Project Antalya roadmap for tiered storage and materialized views? Tiered storage to Iceberg is coming later in the year. It will use TTL rules to automatically move cold data from MergeTree block storage to Iceberg, reducing storage costs by roughly 80% while presenting users with a single table for queries. Materialized views on Iceberg are also planned, allowing views to fire automatically as data arrives in Iceberg and enabling users to combine fast materialized view lookups with swarm-powered source data scans.

How can I get started with Project Antalya today? Clone the Antalya examples repository on GitHub. It includes scripts for Docker Compose and Kubernetes. With Docker Compose installed, you can have a working demo in about 20 seconds. Join the Altinity Slack for support, raise GitHub issues for bugs or feature requests, and submit pull requests if you want to contribute. All code is Apache 2.0 licensed and 100% open source.

© Altinity, Inc. Altinity®, Altinity.Cloud®, and Altinity Stable® are registered trademarks of Altinity, Inc. ClickHouse® is a registered trademark of ClickHouse, Inc.; Altinity is not affiliated with or associated with ClickHouse, Inc. Kubernetes, MySQL, and PostgreSQL are trademarks and property of their respective owners.