Blog

Essential Tips for Safely Upgrading to Altinity Stable® 26.3

The Altinity.Stable 26.3 Build is a major addition for users of Altinity builds of CloudHouse®. This is a big release with a lot of new features and improvements. However, the new version is still available only by request in Altinity.Cloud. Why is that? The answer is simple – with so many changes between 25.8 and 26.3 it was really hard for us to recommend a safe upgrade for production systems. There were also many breaking changes that required careful attention. In this article, we fill the gap and provide the minimum guidelines for a safe upgrade and a possible downgrade path.

Bottom Line

PathSafe by default?Notes
Upgrade 25.8 → 26.3YesOld parts are readable. Do pre-checks first.
Downgrade 26.3 → 25.8No (unless prepared)New serialization defaults make new parts unreadable on 25.8.

Upgrade Instructions (25.8 → 26.3)

Because ClickHouse development is moving super fast, there are typically three major issues with every ClickHouse upgrade:

  • Some rarely used or experimental features might be deprecated in a new version. If those were used in DDL (like codecs, data types or indexes) ClickHouse may not start after an upgrade.
  • Server behavior may change, producing different query results or performance characteristics.
  • New server version may start writing data using a new data format that is incompatible with the older version preventing a downgrade.

When planning an upgrade, databases and applications needs to be checked very carefully to avoid unexpected downtime and data corruption. It is also important to plan and test the downgrade path in case something goes wrong.

One last note – if you are planning to upgrade from the earlier version, like 24.8 or 25.3, it is better to upgrade in increments. Otherwise it is much harder to guarantee safety.

Pre-upgrade Checks on 25.8

Some features were deprecated. You may run following queries to check if those were used or not in your ClickHouse installations:

  1. Deprecated Object type (removed in 25.11)
SELECT database, table, name, type
FROM system.columns
WHERE type LIKE '%Object%';
  1. Experimental hypothesis indexes (removed in 26.3)
SELECT database, table, name, type_full
FROM system.data_skipping_indices
WHERE type_full LIKE '%hypothesis%';
  1. LIVE VIEW (removed in 25.11). That was one of the very first Altinity contributions in 2019, and a great feature. It is sad to see it removed from the open source ClickHouse, but there is a new alternative coming soon in 26.6 – streaming queries (#99868) .
SELECT database, name, engine
FROM system.tables
WHERE engine LIKE '%LiveView%';
  1. Some codecs (removed in 26.1)
SELECT database, table, name, type, default_expression, compression_codec
FROM system.columns
WHERE compression_codec ILIKE '%DEFLATE_QPL%'
  OR compression_codec ILIKE '%ZSTD_QAT%';
  1. Full-text and inverted indexes (reworked in 25.9 and 25.12). Needs to be rebuilt after upgrade
SELECT database, table, name, type_full
FROM system.data_skipping_indices
WHERE type_full ILIKE '%text%' OR type_full ILIKE '%inverted%';

If any of the first four return rows, fix them on 25.8 first by changing data type or dropping unsupported objects. 

Also make sure pending mutations are finished and avoid mixed-version DDL races. The queries below should return no data:

SELECT * FROM system.mutations WHERE is_done = 0;
SELECT * FROM system.distributed_ddl_queue WHERE status != 'Finished';

Downgrade Ability

It is recommended for the first production upgrade to keep the ability to downgrade. Set this in the default profile before the upgrade:

<!-- users.d/compatibility.xml -->
<clickhouse>
  <profiles>
    <default>
      <compatibility>25.8</compatibility>
    </default>
  </profiles>
</clickhouse>

That restores 25.8 defaults for both session settings and MergeTree on-disk formats, including the critical values:

Setting25.8 default26.3 defaultNotes
propagate_types_serialization_versions_to_nested_types01Blocks downgrade to 25.8 if enabled and new parts are written
serialization_info_versionbasicwith_typesChanged in 25.11
string_serialization_versionsingle_streamwith_size_streamChanged in 25.11; effective with with_types
object_serialization_versionv2v3Changed in 25.12 (JSON advanced shared data)
dynamic_serialization_versionv2v3Changed in 25.12
object_shared_data_serialization_versionmapadvancedChanged in 25.12
object_shared_data_serialization_version_for_zero_level_partsmapmap_with_bucketsChanged in 25.12
async_insert01Behavioral change

Perform the Upgrade

Once you are ready, it is simple:

  1. Make sure you are using the latest LTS or Altinity Stable 26.3 build.
  2. Test the upgrade in staging first with a realistic schema/workload.
  3. Take a backup on the 25.8 prior to upgrade.
  4. Roll out one node at a time and pause if it does not start on a new version. If you are using the Altinity Kubernetes Operator or Altinity.Cloud, it will stop an upgrade automatically in case of a problem.

On an upgraded node confirm:

SELECT version();  -- expect 26.3.16.10001.altinitystable

SELECT name, value FROM system.merge_tree_settings
WHERE name IN (
 'propagate_types_serialization_versions_to_nested_types',
 'serialization_info_version',
 'string_serialization_version',
 'object_serialization_version',
 'dynamic_serialization_version',
 'object_shared_data_serialization_version'
);

If 25.8 compatibility mode was on, you should see 25.8-compatible values.

Post-upgrade Tasks

  • Rebuild full-text / inverted indexes if you use them:
ALTER TABLE db.table DROP INDEX idx_name;
ALTER TABLE db.table ADD INDEX idx_name (col) TYPE full_text(...);
  • If you use column statistics and hit format issues:
ALTER TABLE db.table MATERIALIZE STATISTICS ALL;
  • Review behavioral defaults (see “Require attention” / backward-incompatible sections on the 26.3 page).

When you are confident and want 26.3 on-disk features, remove <compatibility>25.8</compatibility> (or set it empty) and restart — after that, a downgrade to 25.8 is no longer safe.

Downgrade Instructions (26.3 → 25.8)

Downgrade is safe only if all of these are true:

  1. You ran 26.3 with compatibility = '25.8' in the default profile from the first start (or equivalent explicit MergeTree settings keeping 25.8 formats).
  2. No parts were written with 26.3-native serialization (no period with new defaults enabled).
  3. You did not rely on 26.3-only features (new types/indexes/codecs) that 25.8 cannot read.

If 26.3 ran with default settings and wrote data, do not downgrade in place. Restore from backup / rebuild instead. Typical failure: parts end up in detached/ as broken-on-start_* because older binaries cannot parse new serialization.json fields (notably propagate_types_serialization_versions_to_nested_types).

Steps (when safe)

  1. Stop writes if possible; wait for merges/mutations to finish.
  2. Keep compatibility = '25.8' until after the rollback (or remove 26.3-only config).
  3. Install previous packages, e.g. latest 25.8 Stable 25.8.28.10001.altinitystable.
    • RPM: while 26.3 is still installed, install the older version with --setopt=obsoletes=0 (see RPM install guide).
    • Docker: point the image back to altinity/clickhouse-server:25.8.28.10001.altinitystable (or your prior 25.8 tag).
  4. Start 25.8 and verify tables load; check for detached/broken parts:
  5. SELECT database, table, name, reason FROM system.detached_parts;
  6. If anything is broken, stop and restore from backup — do not try to “repair” by deleting parts casually.

Final Words

ClickHouse is one of the most rapidly evolving open source databases; it will soon reach 50K GitHub stars. The speed of changes is even more accelerated with the use of AI. (For example, about 30% of ClickHouse commits during last month show AI signatures). That speed has a flip side. Some new features may change their implementation after user feedback. Some older ones are discontinued in favor of new developments. Improvements in the storage layer may be incompatible with the previous versions. It becomes challenging to maintain the clean upgrade and downgrade paths between ClickHouse versions. For the ordinary ClickHouse user, that means a risk of outages after upgrades. We minimize this risk with Altinity Stable Builds, detailed release notes and upgrade instructions. Stay tuned, a new release is coming soon!

Join our Slack

ClickHouse® is a registered trademark of ClickHouse, Inc.; Altinity is not affiliated with or associated with ClickHouse, Inc.

Table of Contents:

Related:

Leave a Reply

Your email address will not be published. Required fields are marked *