Getting Started with Altinity’s Project Antalya

Project Antalya is extending ClickHouse^Ⓡ to use shared Iceberg tables for data storage with scalable separation of storage and compute. Our goal is to lower storage costs and enable dynamic compute scaling using cheap spot instances. 

This article will show you how to get started with Project Antalya using our antalya-examples repo. The project is 100% open source and structured to get you up and running in a few minutes. 

We’ll also show you how to make Project Antalya better. Our code additions are stored in the antalya branch of the Altinity ClickHouse repo. It uses the same CI/CD pipelines we developed for Altinity Stable Builds. You can log issues on the code and also submit PRs. We welcome community contributions of all kinds. 

p.s., Don’t know what Project Antalya is about? Check out the launch webinar, where all is revealed. There’s also an accompanying blog if you prefer a nice read. 

Project Antalya Architecture 

Here’s an intro to the demo setups in one picture. It shows how we implement reads on Parquet files in Iceberg using separation of compute and storage. 

Users send queries to the “vector” server which is a normally configured ClickHouse server. It plans queries and sends the results back to clients. The “swarm” cluster handles queries on Iceberg. The swarm also works great on plain Parquet files as well as Hive tables so long as they are on S3 storage.

Quickstart with Docker

Here’s the fastest way to see a working Antalya setup including use of Python and Spark. On your favorite Linux host: 

1. Install Docker Desktop and Docker Compose. 
2. Clone the antalya-examples repo, cd to the docker directory, and run the up.sh script. 

git clone https://github.com/Altinity/antalya-examples
cd antalya-examples/docker
./up.sh

3. Follow the instructions in docker/README.md to step through examples. These include using Python and Spark to generate Iceberg data.

The demo runs great on a laptop – I use it on Windows WSL2 with Ubuntu 22.04. 

Quickstart with Kubernetes on AWS

Antalya is all about scaling and performance. For that we need Kubernetes, which will allow us to allocate additional hosts easily. You’ll need an account on AWS to run our demo out of the box. The examples are located in the kubernetes directory. The README.md file will guide you through installation, which looks like the following. 

1. Install aws-cli, kubectl and terraform. I additionally recommend installing k9s, an outstanding terminal-based UI for Kubernetes. 
2. Set up an EKS Kubernetes cluster using Terraform. 

cd ../kubernetes/terraform
terraform init
terraform apply
aws eks update-kubeconfig --name my-eks-cluster

3. Install the Altinity Kubernetes Operator for ClickHouse. 

kubectl apply -f 
https://raw.githubusercontent.com/Altinity/clickhouse-operator/master/deploy/operator/clickhouse-operator-install-bundle.yaml

4. Install the vector server, the swarm cluster, and a ClickHouse Keeper ensemble from manifests. 

cd ../kubernetes/manifests
kubectl apply -f gp3-encrypted-fast-storage-class.yaml
kubectl apply -f keeper.yaml
kubectl apply -f swarm.yaml
kubectl apply -f vector.yaml

5. Follow the examples in kubernetes/README.md to see effects of scaling using real datasets. Our favorite is the AWS Public Blockchain Data. 

We also have a helm chart for Project Antalya installation and will be brushing up the templates to match the latest release. Watch for more improvements in this area. Extra credit: make it work on another Kubernetes distribution like GKE or Minikube and post a PR with your suggestions.

Documentation and Help

Project Antalya binaries in general behave like normal ClickHouse. Swarm clusters and and other niceties are extensions. You can refer to upstream docs or Altinity documentation and knowledge base as you would normally.

Behavior specific to Project Antalya binaries is documented in the altinity-examples docs directory. The documentation is somewhat parsimonious, but we plan to improve it. We’ll also be building up official Altinity documentation pages over time.

To get quick help on Project Antalya, please join the Altinity Slack workspace and post questions in the #antalya channel. We all watch for questions. And don’t hesitate to post answers to other users when you know them. 

Release Policies

Project Antalya is effectively a branch of the master branch in upstream ClickHouse. Our code is in the antalya branch in the Altinity/ClickHouse GitHub repo. All additions are Apache 2.0 licensed, just like ClickHouse in general. 

Here are the main policies we follow for Project Antalya releases. 

• Major release every two months. We “jump” to the next release of upstream ClickHouse.  We’re currently working on 25.3, with 25.5 to follow when it’s available. 
• Backport fixes and features between major releases. We apply fixes of our own as well as from upstream and generate minor releases every couple of weeks or so. Our 25.3 build will include a number of fixes backported from 25.4.  
• Push Altinity contributions to upstream. Wherever possible we submit our fixes as PRs to upstream ClickHouse. It benefits all users to keep codebases as consistent as possible. 
• Run the entire ClickHouse suite of stateless and stateful tests plus Altinity’s own tests on each new build. 
• Post packages for amd64 and arm64 on the builds.altinity.cloud page, which offers YUM and APT repos as well as tarball releases. Containers are available on Docker Hub. Note: we don’t use “latest” tags, so you’ll need to update your image names to specific releases to upgrade containers. 

Moving fast while extending ClickHouse has been a learning experience for all of us. I hope we can carve out some time to expand on the lessons in a future blog article. 

Improving Project Antalya

Project Antalya is a community effort. The more of us who work on it, the faster it will improve. Here are two easy ways you can help move the project forward. 

1. Log issues on the Altinity/ClickHouse repo when you run into problems or see opportunities for improvement in Project Antalya binaries.  If you see doc or example problems, log them on the antalya examples project. 
2. Submit pull requests. We love contributions to the code. If you need to discuss design it’s helpful if you log an issue so we can comment on your ideas (and vice versa). If you are looking for something to do, scan the current open issues. If you see a problem you know how to fix, just dive in and submit a PR. 

We are working on documentation to make contributions easier. Please contact us on Slack if you have questions or want to step up. 

More to come!

This is the first of many articles offering practical advice on applying Project Antalya to real-time query on Iceberg. In addition to blogs we’ll be releasing new software–lots of it. Here’s what we’re working on right now. 

• Releasing Altinity Ice, a new open source project to load Parquet data easily to Iceberg. 
• Building Project Antalya features into Altinity.Cloud. Soon you’ll be able to launch swarm clusters in minutes without worrying about configuration details. 
• Improving performance of swarms for reading data, especially use of caches and more efficient query execution against Iceberg/Parquet. We’re aiming for real-time response, of course. 
• Writes to Iceberg–this is the big one that will get us to 10x cheaper storage for large tables. We’ll have a lot more to say on this topic over the next few months.

Big changes in storage technology like the shift to open table formats on S3 come along every couple of decades. We’re having fun adapting ClickHouse to deliver cheap, scalable, real-time analytics on Iceberg. Try out Project Antalya, let us know how you like it, and help us make it better!