Altinity.Cloud Extends Managed ClickHouse to ARM

ARM processors are hot. Amazon AWS and Apple have been building their own ARM processors for a few years already. They use them in servers and notebooks and get outstanding results. Ampere builds server-class ARM processors for everybody else. At Altinity we could not stand aside from this A-march. Today we are happy to announce the first ever managed ClickHouse on ARM!

Why ARM?

What is common between an Android phone, Apple M1 Mac or most advanced AWS hardware? They all run ARM processors inside. ARM stands for Advanced RISC Machines, and today we see a RISC revenge. 

Like most modern computer technologies, RISC architecture was invented back in the 70s. When I was young, SPARC and PowerPC RISC processors dominated the market of high-performance servers and workstations. However, in the late 90s it looked like Intel won the CPU battle. All major server and PC vendors with very few exceptions adopted Intel x86 architecture. RISC survived in mobile devices. It took another 20 years before RISC processors struck back to break the Intel monopoly.

Before going into more detail, let’s ask a basic question: why should a user care about ARM at all? Why do Apple and Amazon care? The answer is simple: it is cost/performance efficiency that drives users to ARM! RISC architecture allows the building of more efficient chips. The downside is more complex machine code, but with current compilers this is negligible. 

Apple M1 devices are vastly superior to Intel counterparts for a lower price. Amazon claims ARM-based Graviton instances deliver better performance at 30% smaller cost compared to Intel ones. This is definitely better for users, including ClickHouse users. We will see proof in a minute.

What does ARM support mean?

New CPU architectures do not appear too often for many reasons. From the end user perspective the most significant is that software needs to be re-compiled. MacBook M1 users were probably the first who realized it en masse last year when many applications could not run on new Macs. ClickHouse is not an exception, and it took us a while to produce reliable native ARM builds. The story about this adventure is yet to be told in future articles. But once we saw the first results we were confident that work was worth the effort.

Altinity.Cloud runs inside Kubernetes clusters. So in order to support ARM, the Kubernetes provider needs to support ARM worker nodes first — which Amazon EKS does. Then all relevant software needs to be re-compiled for ARM, and ARM versions of docker images need to be published – Altinity did that for ClickHouse. The docker images for ARM-built ClickHouse are available at Altinity dockerhub. Note, we consider current images as still beta. Altinity Stable ARM builds will be released by the end of year.

Altinity.Cloud managed ClickHouse at AWS Graviton instances

AWS provides several instance families that use Graviton ARM processors. In order to start Altinity.Cloud managed ClickHouse on ARM-based Graviton instance, users need to do two or three simple things:

1. If you are not an Altinity.Cloud customer yet, subscribe to a free Altinity.Cloud trial account.
2. Ask Altinity via one of the support channels to enable Graviton instances for the account. If you are curious, these are just two lines in a configuration file.
3. Start a new one or upgrade an existing ClickHouse cluster using Graviton instance type.

In order to demonstrate the performance of ARM-based ClickHouse I conducted the following experiment. I started 3 ClickHouse clusters powered by different instance types:

• m5.xlarge — the ‘classic’ AWS general purpose Intel-based instance type
• m6i.xlarge — latest generation Intel-based instance type introduced recently
• m6g.xlarge — Graviton ARM-based instance type introduced 2 years ago

All instances are equally sized having 4 vCPUs and 16GB of RAM each (14GB available for ClickHouse).

To make it easy to distinguish between different ClickHouse clusters, I named clusters as ‘m5’, ‘m6i’ and ‘m6g’ respectively:

Then, I took the 600M rows Star Schema Benchmark (SSB) dataset, and applied a technique from “Ultra Fast Data Loading and Testing in Altinity.Cloud” to load the data and get benchmark results. The query times for different clusters are shown in the table below:

As you can see, Graviton-based m6g cluster delivered better performance compared to a baseline m5 instance for all queries except Q2.1. In a number of queries, the 2 year old ARM instance outperformed the latest generation Intel m6i instance as well, though in other cases it was slightly behind. This is an excellent result, given that m6g instances are priced 30% less than m5 and m6i.

Known issues and limitations

ClickHouse ARM support is still in beta. We’ve got some tests failing, and there are two ClickHouse modules that are disabled in ARM builds:

• HDFS
• Hyperscan library

While HDFS is not used very often in Altinity.Cloud, hyperscan may hit some of ClickHouse users. Hyperscan is a heavily optimized regular expression library developed and maintained by Intel. It is not a surprise that it is only available for x86 processors. If you are using ClickHouse multi-match functions, you can get this error on ARM:

SELECT multiFuzzyMatchAny('abcd', 0, ['a', 'b'])

Code: 48, e.displayText() = DB::Exception: Edit distance multi-search is not implemented when hyperscan is off (is it x86 processor?): While processing multiFuzzyMatchAny('abcd', 0, ['a', 'b']) (version 21.8.12.29.altinitydev.dev (altinity build))

Another possible issue is missing SSE support in ARM processors. ClickHouse uses some SSE optimizations that are specific to x86 processors, in particular when processing strings, so certain functions may work slower. We have not validated this with real cases yet.

Other than noted above we have not found any major issues. Testing is continuing.

Conclusion

The revenge of ARM requires software vendors to adapt in order to deliver efficient and competitive products. Altinity is first again to deploy managed ClickHouse on ARM instances with Altinity.Cloud. No matter where users need to run ClickHouse — we are here to help them. ARM support is yet another brick to this long term strategy. Stay tuned for more news about Altinity.Cloud and ClickHouse ARM builds!