Altinity was the first to provide ARM builds for ClickHouse back in 2021, but we paused support for them due to a lack of test coverage. Today, we are excited to announce full support for AARCH64 (ARM64) architecture in Altinity Stable Builds for ClickHouse, starting with the latest release, 18.104.22.168.altinitystable.
You can try ARM binaries using Docker or install them directly using our APT repository containing DEB packages or YUM repository containing RPM packages by following the installation instructions on the Altinity Stable Builds page.
The ARM Docker images are also immediately available for use by our customers on our Altinity.Cloud and now allow all our customers to take advantage of the benefits of Altinity Stable Builds for ClickHouse, irrespective of server architecture.
For more information, refer to the Altinity Stable Builds documentation. If you are not sure if Altinity Stable Builds are right for you, check out the Why Every ClickHouse User Should Appreciate Altinity Stable Builds article, where we talk about why many users choose them over other distributions.
Multi-arch Docker Images
One of the simplest ways to use the new ARM64 builds is by using Docker. Starting with 22.214.171.124.altinitystable, Docker images for our Altinity Stable Builds for ClickHouse are now multi-arch. You can find them on Docker Hub.
Click the `altinitystable` link to find currently available images.
If you are on an AMD64 machine or using an ARM64 server, you can run the same command to pull the Docker image:
docker pull altinity/clickhouse-server:126.96.36.199.altinitystable
When you enter the running container, the following queries can be used to confirm that everything is working as expected:
SELECT version() ┌─version()────────────────┐ │ 188.8.131.52.altinitystable │ └──────────────────────────┘ SELECT name, value FROM system.build_options WHERE name = 'SYSTEM_PROCESSOR' ┌─name─────────────┬─value───┐ │ SYSTEM_PROCESSOR │ aarch64 │ └──────────────────┴─────────┘
Notable Differences in Behavior Between ARM64 and AMD64 Builds
Given that ARM64 and AMD64 are completely different architectures, some aspects, such as floating point arithmetic, operate differently. You can expect that results for some queries might differ between the AMD64 and ARM64 architectures. Here are a few common examples.
Floating Point Results
Differences in floating point results can be easily seen using aggregate functions over floating-point numbers.
SELECT stddevPopStable(number * 3.23456711e-11) FROM numbers(1000000)
On AMD64 returns:
But on ARM64, the same query returns:
As can be seen, the results differ by the rightmost decimal digit. This also means that the hashes of the results might not be the same between ARM64 and AMD64.
For example, the query
SELECT simpleLinearRegression(salary, empno) OVER (ORDER BY salary, empno ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) AS func FROM empsalary
had the following differences between results on each architecture:
Also, differences were detected between the results of the tests that were using cityHash64 over floating-point results.
For example, the query
SELECT anyLast(id), anyLast(time), cityHash64(groupArray(w)) FROM (SELECT id, f_timestamp AS time, exponentialTimeDecayedAvg(1)(id, f_timestamp) OVER (ROWS BETWEEN CURRENT ROW AND 1 FOLLOWING) AS w FROM datetimes2 ORDER BY id)
produced different hash values due to the exponentialTimeDecayedAvg function returning a floating-point result.
9999 2020-01-01 02:46:39 6998628102243875284
9999 2020-01-01 02:46:39 9677279760188586566
Aggregate Function States
Aggregate function states can also be different for the two architectures. Here is an example of differences in states detected by our tests for the corrState function used with Nullable(Int64) and UInt64 data types:
SELECT hex(corrState(nullable_int64_,uint64)) FROM table_eef28883_6380_11ee_b6ee_25d441995fb1
Due to this and other possible incompatibilities in binary representation of states and intermediate results, running heterogeneous clusters that mix both AMD64 and ARM64 ClickHouse instances is not recommended. Though tests do not cover this, you can share data via textual or intermediate formats like Parquet.
No HDFS and GRPC
Two ClickHouse capabilities are disabled for ARM completely, since corresponding libraries are not available for this architecture. Those are the HDFS Engine, the hdfs() table function, and the GRPC connection protocol.
This can be checked by querying the system.build_options table:
SELECT * FROM system.build_options WHERE name in ( 'USE_GRPC', 'USE_HDFS') ┌─name─────┬─value─┐ │ USE_HDFS │ │ │ USE_GRPC │ │ └──────────┴───────┘
VectorScan Instead of HyperScan
Originally, ClickHouse used the Intel HyperScan library for some advanced search scenarios. HyperScan was not available for the ARM architecture, so ClickHouse ARM builds were failing on certain search functions. Starting from 22.9, ClickHouse has migrated to the VectorScan library, which is a portable fork of HyperScan, so it is not an issue anymore.
Added Test Coverage for AARCH64 (ARM64)
The road to adding support for ARM64 architecture was not easy, and one of the main pain points was that our QA team had to port all the Altinity integration tests for ClickHouse to ARM64 architecture to make sure that our ARM64 and AMD64 test coverage was the same when running our Altinity test suites.
If you are interested, you can find our open-source test suites for ClickHouse on GitHub at https://github.com/Altinity/clickhouse-regression.
Test coverage is not exactly the same between the two architectures. One major difference is that the community integration tests have not yet been ported to ARM64 and therefore are not executed as part of the standard CI/CD pipeline.
You can check the exact test coverage for our Altinity Stable Builds by looking at the publicly available build reports that accompany each of our releases. For example, you can check our v184.108.40.206.altinitystable build report to find a link to the GitHub Actions pipeline that runs all our tests.
Overall, due to differences in test coverage between AMD64 and ARM64 platforms and the fact that the ARM64 platform is not yet as popular and well used as AMD64, using ARM binaries should be done with caution.
Conclusion and Where to Find Out More
ARM64 support is an important step in our journey to deliver 100% open-source ClickHouse builds for enterprise users who value stability and long-tail maintenance for business-critical analytics. We will continue to improve the build process and extend our testing over the coming releases. We look forward to your feedback on this latest improvement or any aspect of our work on Altinity Stable Builds.
For information about Altinity Stable Builds and the philosophy behind them, check out our blog article Why Every ClickHouse User Should Appreciate Altinity Stable Builds. For details on builds including how to install, check out the Altinity Stable Builds for ClickHouse documentation. You can also run Altinity Stable Builds in Altinity.Cloud by selecting a stable build version when you start or upgrade ClickHouse clusters.
Please log build issues in our ClickHouse fork on GitHub or join the conversion on our public Altinity Slack Workspace. You can contact us directly for information about support. In the meantime, take our new 220.127.116.11.altinitystable out for a spin!