ClickHouse Altinity Stable 21.8

A few months ago we certified ClickHouse 21.3 as an Altinity Stable release. Since then, we have worked on newer releases and run them in-house. We completed several new features, and even more have been added by community contributors. We were running ClickHouse 21.8 to power our public datasets at Altinity.Cloud instance and testing it in our environments. We have also performed a comprehensive QA of ClickHouse 21.8 to make sure upgrades go smoothly. As of 21.8.8 we are confident in certifying 21.8 as an Altinity Stable release.

This release is a significant upgrade since the previous Altinity Stable release. It includes 1472 pull requests from 224 contributors. Please look below for the detailed release notes.

Major new features since the previous stable release 21.3

A new release introduces a lot of changes and new functions. The full list is available in the Appendix, so refer to this section for more detail. The following new features are worth mentioning on the front page:

• SQL features:
  • DISTINCT ON a subset of columns
  • Partial support of SQL/JSON standard
  • Arrays in dictionaries are now supported
  • Arrays and nested data types are now supported for Parquet and Arrow formats
  • DateTime64 extended range, Now dates between 1925 to 2283 years are supported. ^a)
• Security features:
  • Disk level encryption
  • Kerberos authentication for HTTP protocol ^a)
  • Active Directory groups mapping for LDAP user directory  ^a)
• Replication and Cluster improvements:
  • ClickHouse Keeper (experimental) — in-process ZooKeeper replacement
  • SYSTEM RESTORE REPLICA — a handy tool that makes life easier in the unfortunate occasion if ZooKeeper metadata is lost ^a)
  • Support for MySQL/PostgreSQL clusters when using mysql/postgresql table functions
  • Zero-copy replication for S3 tables (see our blog post for details)
  • Parts movement between shards (experimental)
• MergeTree features:
  • Projections (experimental)
  • _partition_id and _partition_value virtual columns were added
  • Deduplication for non-replicated tables. See non_replicated_deduplication_window merge tree setting
• Integrations:
  • MaterializedPostgreSQL database engine for replication from PostgreSQL
  • HDFS disk support (experimental)
  • Allow to catch Kafka errors into a separate stream (see the KB article on this)
• Other:
  • YAML configuration format as an alternative to XML

As usual with ClickHouse, there are many performance and operational improvements in different server components.

^a) – contributed by Altinity developers.

Backward Incompatible Changes

The following changes are backward incompatible and require user attention during an upgrade:

• Users of LowCardinality(Nullable(...)) can not safely downgrade to versions older than 21.4. Data in tables with columns of those types inserted / merged by 21.8 will be lost after the downgrade.
• Values of UUID type cannot be compared with integers. For example, instead of writing uuid != 0 type uuid != '00000000-0000-0000-0000-000000000000'
• The toStartOfIntervalFunction will align hour intervals to midnight (in previous versions they were aligned to the start of unix epoch). For example, toStartOfInterval(x, INTERVAL 11 HOUR) will split every day into three intervals: 00:00:00..10:59:59, 11:00:00..21:59:59 and 22:00:00..23:59:59.
• It’s not possible to rollback to the older ClickHouse version after executing ALTER ... ATTACH query as the old servers would fail to process the new command entry ATTACH_PART in the replicated log.
• The behavior of remote_url_allow_hosts has changed. In previous versions the empty section did nothing, in 21.8 it will block access to all external hosts. Remove this section from the configuration files after an upgrade if you experience issues with url() or s3() functions.
• If you will downgrade to version before 21.1 clickhouse will not be able to start automatically — you will need to remove the system.*_log tables manually to downgrade

Upgrade Notes

There were several changes between versions that may affect the rolling upgrade of big clusters. Upgrading only part of the cluster is not recommended. 

• Distributed queries with explicitly defined large sets are now executed differently. Compatibility setting legacy_column_name_of_tuple_literal may be enabled during the rolling upgrade of the cluster. Otherwise distributed queries with explicitly defined sets at IN clause may fail during upgrade.
• ATTACH PART[ITION] queries may not work during cluster upgrade

Other Important Changes

ClickHouse embedded monitoring has become a bit more aggressive.  It now collects several system stats, and stores them in the table system.asynchronious_metric_log.  This can be visible as an increase of background writes, storage usage, etc. To return to the old rate of metrics refresh / flush, adjust those settings in config.xml:

<asynchronous_metrics_update_period_s>
	60
</asynchronous_metrics_update_period_s>
<asynchronous_metric_log>
	<flush_interval_milliseconds>
		60000
	</flush_interval_milliseconds>
</asynchronous_metric_log>

Alternatively, metric_log and asynchronous_metric_log tables can be completely disabled:

<yandex>
    <asynchronous_metric_log remove="1"/>
    <metric_log remove="1"/>
</yandex>

Some new ClickHouse features are now enabled by default. It may lead to a change in behaviour, so review those carefully and disable features that may affect your system:

• async_socket_for_remote
• compile_aggregate_expressions
• compile_expressions
• cross_to_inner_join_rewrite
• insert_null_as_default
• optimize_skip_unused_shards_rewrite_in
• query_plan_enable_optimizations
• query_plan_filter_push_down

In the previous releases we recommended disabling optimize_on_insert. This recommendation stays for 21.8 as well, as inserts into Summing and AggregatingMergeTree can slow down.

Known issues in 21.8.8

The development team continues to improve the quality of the 21.8 release. The following issues still exist in the 21.8.8 version and may affect ClickHouse operation. Please inspect them carefully to decide if those are applicable to your applications:

• system.events for event = 'Merge' are overstated. ClickHouse incorrectly increments this counter. 

You may also look up GitHub issues using a special v21.8-affected label.​​

For more detail on new features and settings see an Appendix section on Altinity documentation site.

ClickHouse Altinity Stable Releases are based on the community versions. For more information on installing ClickHouse from either the Altinity Stable build or the Community builds, see the Altinity Stable Builds Install Guide.

Please contact us at info@altinity.com if you experience any issues with the upgrade.