Kubernetes Cluster Logging with the OpenTelemetry Collector and ClickHouse®

Recorded: September 26 @ 08:00 am PDT
Presenter: Joshua Lee – Open Source Developer Advocate

In this webinar, Altinity Open Source Developer Advocate, Josh Lee, walks through a complete, working Kubernetes cluster logging solution built on three open-source components: the OpenTelemetry Collector, ClickHouse®, and Grafana. The talk is structured in three parts: a conceptual foundation, a live demo, and a discussion of how to scale the solution to production.

Josh opens by making the case that, despite the buzz around distributed tracing, logs remain the foundation of any useful observability system. A well-structured request log contains everything you need to derive RED metrics (requests, errors, duration), root-cause errors, and even AI-ready training data. Distributed tracing is structurally just a well-annotated log with trace propagation context, and it requires a large upfront investment before becoming useful, whereas logs are valuable from day one.

The talk then covers the specific challenges of logging in Kubernetes: no built-in cluster-wide log aggregation, multiple log formats from different container runtimes, the ephemeral nature of containers and pods, the difficulty of correlating logs across services, and the cost of storing verbose text data at scale. Josh argues that centralized logging solves all of these: it provides a single place for normalization, filtering, and correlation with metrics and infrastructure logs.

The OpenTelemetry Collector is presented as the right tool for the job because it handles all telemetry signals (logs, metrics, traces, profiles), has 90-plus output destinations, and can translate between dozens of receiver formats. Josh walks through the specific components used in the demo: the filelog receiver, the Kubernetes attributes processor, the batch processor, and the ClickHouse exporter. The Kubernetes attributes processor is highlighted for its role in adding consistent resource attribute metadata to all telemetry, making it possible to correlate signals across tools and vendors.

ClickHouse is chosen as the log storage backend because of its massive ingest capability, flexible storage options (including S3 tiering), powerful real-time transformations via materialized views, and columnar compression, which is especially beneficial for log data.

The live demo uses Argo CD to deploy a Helm chart that installs the OpenTelemetry Collector as a DaemonSet, a ClickHouseInstallation managed by the Altinity Kubernetes Operator for ClickHouse®, and Grafana with pre-provisioned dashboards. Josh walks through the Helm values, the collector configuration (including the new container log parser preset that replaced 90 lines of YAML), the ClickHouseInstallation resource, and the Grafana query interface. Both the Altinity and Grafana-maintained ClickHouse plugins for Grafana are demonstrated.

The session closes with a production scaling discussion (DaemonSet collectors feeding a gateway Deployment, replicated ClickHouse with Keeper, and schema tuning) and a Q&A on Argo CD app-of-apps patterns, self-healing and pruning policies, and how to deploy the same system with plain Helm charts.

Here are the slides:

Key Moments (Timestamps)

Key moments generated with AI assistance.

• 00:00 – Welcome and waiting for attendees
• 02:04 – Recording notice and kick-off
• 03:04 – Why logs matter: from raw log lines to RED dashboards
• 05:18 – Why not just use distributed tracing? Logs vs. traces comparison
• 06:57 – The airplane cockpit metaphor for observability
• 09:29 – Metrics, traces, and logs: why you need all three
• 10:10 – Why do you need application logs?
• 11:07 – Logging in Kubernetes: challenges and the current state
• 14:43 – Why centralized logging is essential
• 17:50 – The OpenTelemetry Collector: why it beats alternatives for this use case
• 21:07 – Key OpenTelemetry Collector components: filelog receiver, k8s attributes processor, batch processor, ClickHouse exporter
• 21:07 – Kubernetes resource attributes and semantic conventions
• 23:23 – The Altinity Kubernetes Operator for ClickHouse®
• 26:19 – Why ClickHouse for log storage?
• 28:11 – Demo: full architecture overview
• 29:46 – Demo: Argo CD app-of-apps and Helm values walkthrough
• 30:37 – Demo: OpenTelemetry Collector configuration in detail
• 34:04 – Demo: ClickHouseInstallation resource and Kubernetes resources created
• 37:07 – Demo: Grafana dashboards and log visualization
• 39:31 – Demo: writing SQL queries with the Altinity plugin macros
• 42:46 – Scaling to production: gateway architecture and replicated ClickHouse
• 45:24 – Q&A: Argo CD setup, self-healing, pruning, and plain Helm deployment

Webinar Transcript

[02:04] – Welcome and Introduction

Josh: This webinar is being recorded, and you will receive a link to the recording and the slides within a few hours. Hi everyone, welcome. Thank you for joining us. I am Josh Lee, and I am the Open Source Developer Advocate at Altinity. Today we are going to talk about Kubernetes cluster logging.

This is a fairly intimate group, so if you have any questions, feel free to pop them in the chat as we go, and I will try to stop and answer them along the way. We are going to talk a little bit about theory: why we are doing this, what tools are available, and how we might use them. Then at the end I am going to show a demo of working code and how you can put all of this together yourself using tools like Helm and Argo. Then we will have a final round of Q&A.

[03:04] – Why Logs Matter

Josh: Why do logs matter? Logs get the short end of the stick from marketing and thought leadership these days. People make fun of them a lot. The challenge of reading through all the log messages to find needles in a haystack is real. But behind those jokes is an important fact: log messages are essential. When you have the right log message, it is an invaluable piece of your debugging journey. The challenge is simply finding that right one, and that is where things get a lot more complicated than tailing a single log file in a development environment.

If we look at a humble request log, this has so much information about the health and operation of a microservices application. If we have this for every single request and can aggregate and count these things, we have the time of the request, the duration, the HTTP verb, the path, and the status code. These are really valuable things to count and aggregate to understand how our system is operating and to graph on a dashboard. Those metrics are commonly referred to as your RED or golden signals: requests, errors, duration. We can get all of that from request logs. Then we have all the extra verbosity that comes with logs to contextualize that with more details that the developers wanted to provide. This is really powerful and it is actually the foundation of any useful observability.

[05:18] – Why Not Just Use Distributed Tracing?

Josh: You might be asking: it is 2024, I have seen a lot of presentations about distributed tracing, everybody is all in on this OpenTelemetry thing, why are we talking about logs? My response is: distributed tracing is awesome. If you can get that waterfall view and piece together every request within your system into a linked chronological tree with all the RED data in there, that is fantastic. But if you look at the structure of what a trace actually is, it is really just a request log that has the context of the request that came before it. That trace ID is the big technological difference between a trace and a well-structured log message that follows good naming conventions for metadata.

So if we have the log, we have a lot of the value we would be getting out of a trace, without the hard work. Distributed tracing is very hard to implement because it is only useful once it is end-to-end. In our modern microservices architectures we might be using 17 different technologies in a single request path, and ensuring good instrumentation and trace propagation through all of those technologies is a very hard challenge. That challenge is worth solving in many cases, but on your journey to solving it, logs are useful the entire way.

[06:57] – The Airplane Cockpit Metaphor for Observability

Josh: Taking a step back: why do we have observability? Why do we even care? I think an airplane is a really good metaphor for a modern microservices architecture. You have a lot of different interconnected subsystems, probably more interconnected than we would want to believe, and keeping them running requires a huge amount of feedback and information available to the pilots via all those gauges and dials. Pilots have levers and buttons for providing inputs back into the system and controlling it. This is exactly what we are doing with our modern microservices: monitoring and observing a bunch of systems and providing feedback into them to get them to behave the way we want.

Old cockpits had about 80 dials and gauges. Boeing did try at one point to replace everything with touchscreens, and pilots hated it because if you have ever tried to use a touchscreen in your car while it is bumpy, imagine trying to fly a plane through turbulence and hit the right button. So pilots like physical buttons. But for the information displays, we replaced those 80 dials with four multi-purpose monitors that can contextually display the information the pilot needs in that moment, rather than requiring them to scan for a specific piece of information. That is really the crux of observability: taking all the wealth of information we have and putting it into a format that is immediately usable, for example by putting it all into a single database where we can analyze and query it.

[09:29] – Metrics, Traces, and Logs: Why You Need All Three

Josh: Metrics, traces, and logs. No single one of them is enough. We need all of them. Metrics are early warning signs, the foundation of an alerting framework. Traces give us a really good way to zero in on where a problem is occurring in a massive distributed system and how an error is impacting users. But at the end of the day, all of those things are going to lead us back to a log message that tells us exactly what went wrong.

[10:10] – Why Do You Need Application Logs?

Josh: Application logs give us extremely granular insights into system operations if we want them to. They are invaluable in root cause analysis and troubleshooting. We simply cannot do it without them. They enable incremental adoption of advanced observability: logs are useful from day one and you can incrementally improve them by adding more metadata, more annotations, standardizing your structures. They are useful along that entire journey, whereas traces require a huge upfront investment before they can become useful. And because logs are verbose text, they are excellent input for large language models if you want to use them to generate models or create automated remediation mechanisms.

[11:07] – Logging in Kubernetes: Challenges

Josh: That brings us to our specific challenge: how do we deal with all of this in Kubernetes? We are running our applications in Kubernetes, we might be running platforms where various developers are running applications that we do not have direct control over. We cannot go in and say “you need to use this specific logging library.”

We have some tools available. We have the ability through the Kubernetes API to directly pull logs from any pod. Nodes put their own logs into journald, which is great. But there is really no aggregation or centralization. Once we start trying to deal with logs in Kubernetes at scale, it starts to feel overwhelming.

Specific challenges:

There is no built-in cluster-wide log aggregation, so we need to bring our own. There are many different choices for which one to use. There are no standard log formats: Kubernetes out of the box supports three different container runtimes, and between those three there are two different log output formats, regardless of what format the containers running in them might use. Within the Kubernetes ecosystem alone we see divergence in log formats. And then we have journald logs in yet another format.

There is a contextual disconnection when looking at logs. Pulling logs for a single service at a single point in time is useful if you have already narrowed down the problem and know where it is. But if you are still looking for it, those logs are not very valuable unless you can pull them into a system where you can see logs for your entire operation in one place, correlated with the metadata, metrics, and traces that go with them.

The ephemeral nature of containers is a real challenge too. Containers spin up, do their thing, error, and go away. They get stuck in CrashLoopBackOff and we have no idea at what point in the waterfall of errors the original issue actually occurred. And if we cannot find the logs because the container restarted, we are stuck.

Finally, it is expensive. These challenges can all be solved if you go to a vendor and spend a lot of money. But if you are doing it yourself, the challenge becomes: how do I store log data efficiently? Unlike numeric metrics, it is hard to compress, cardinality is basically infinite, and you are dealing with potentially petabytes of data.

[14:43] – Why Centralized Logging Is Essential

Josh: Cluster logging is essential for all the reasons we discussed. We need the ability to retain all logs even across container restarts and query them in a single unified place. We need continuity: a single service might be made up of many different concrete entities inside our Kubernetes environment, and those entities change. When a pod restarts it gets a new ID. We need to be able to look at a pattern of metadata and say this pattern represents the thing I am examining right now, across restarts and node migrations.

Centralized logging gives us a lever: one central place to put all our annotations, normalization, and filtering. We can have one giant log configuration that handles all our expected log formats, filters out that one chatty service that is always in debug mode, and so on. It also lets us pull logs from our applications and combine them with logs from our infrastructure. We can correlate resource contention at the infrastructure level with an application error, which is a very common occurrence.

Centralization gives us one place to deal with the scalability and performance issues of our logging solution, rather than having individual teams deal with it on individual clusters. And by putting all of our logs into one place into something like a data lake, we can do really advanced analytics on them beyond just those RED dashboards, including long-term data mining, feeding logs into generative AI models, or tiered data storage where we keep only the interesting logs in hot storage and archive or discard the rest.

[17:50] – The OpenTelemetry Collector

Josh: There are a lot of different choices for how you get your logs out of your cluster. Fluentd and Promtail are probably the two most popular right now. Those are great. If you are already using one of them, keep using it. Do not go out and replace it with the OpenTelemetry Collector today. Those tools are really awesome and do what they need to do.

What makes the OpenTelemetry Collector different and more powerful for this use case is that it can pull in any telemetry signal, whether logs, metrics, traces, soon profiles, events, and exceptions. It can filter and process all of those signals using a unified pipeline, and it has over 90 output destinations. You can output to multiple data stores simultaneously. If you have multiple tools you want to use for analysis, you can send your data anywhere you need it. This versatility, the ability to translate from many different receiver formats across all telemetry signals and export into many different formats, is the real value of the OpenTelemetry Collector. It does come with a learning curve because the wealth of options means the configuration is complex.

Key components used in the demo:

The filelog receiver reads logs from a file, such as the log file that kubelet writes on each node. The Kubernetes attributes processor pulls the Kubernetes API for things like node name and pod ID and adds those attributes to any logs flowing through, creating consistent resource attribute metadata. It can do this for all signals, not just logs. The batch processor splits incoming telemetry into large batches, which is important because ClickHouse prefers fewer and larger inserts. And the ClickHouse exporter writes all this information to ClickHouse using SQL.

[21:07] – Kubernetes Resource Attributes and Semantic Conventions

Josh: The metadata describing our entities in OpenTelemetry is called resource attributes, and there are semantic conventions for them. This is really important because it means that whether we are looking at a log, a metric, a trace, or some other signal, if we are following these resource attributes and the semantic conventions according to the spec, we get these bundles of metadata that we can use to identify the relevant entity. We can tie together telemetry whether it came from our own agent, a vendor agent, or instrumentation built into the tool. These resource attributes are a consistent way to identify and correlate all interesting telemetry.

Naming things is hard, so it is helpful to have a convention that says this is how you should name things. And if you are dealing with an area where conventions do not exist yet, for example database servers, it is a welcoming community. You can go and say “I do not see conventions for this thing I care about, here is my proposal,” and potentially get a PR merged.

[23:23] – The Altinity Kubernetes Operator for ClickHouse®

Josh: We are putting this all together and we need somewhere to put the data. In the demo, we are using ClickHouse as our database, running in Kubernetes. The best way to run ClickHouse in Kubernetes is with the Altinity Kubernetes Operator for ClickHouse®. It is totally open source and runs on top of open-source ClickHouse. It handles challenging things like migrating nodes, creating persistent volume claims, upgrading ClickHouse versions, and scaling persistent volume claims. These are all really challenging aspects of stateful workloads in Kubernetes, and the Altinity Operator makes them a lot easier. If you are using ClickHouse in Kubernetes, definitely take a look at the operator.

[26:19] – Why ClickHouse for Log Storage?

Josh: Why ClickHouse specifically for log storage? The first reason is its massive ingest capability. Logging volumes can be in some cases larger than the volume of data that the application being logged is actually dealing with. ClickHouse can handle that and scale horizontally to handle any volume of data you want to throw at it, efficiently using the resources available.

There is also the wealth of storage options. ClickHouse can store logs in many different places, including S3, giving you the ability to store them wherever it is most cost-efficient or effective for the type of data you are dealing with. Then you have very powerful tools for moving and analyzing data within the database itself: real-time transformations, materialized views, and things like that can build new, more refined data sets from existing data sets. This starts to get us toward that 787 cockpit we were looking at earlier.

[28:11] – Demo: Architecture and Argo CD Overview

Josh: Here is our demo architecture: Kubernetes, ClickHouse, OpenTelemetry, and Grafana. In Kubernetes, the default is that containers send output to standard out and standard error. The container runtime reads that, parses it, and saves it to a log file accessible to kubelet. The OpenTelemetry Collector filelog receiver then reads those files (assuming you have the correct permissions) and parses them using the parsers you define. It then writes to the ClickHouse database. Inside Grafana, we have the Altinity Grafana Plugin for ClickHouse®, a data source plugin that teaches Grafana how to query your ClickHouse cluster. It lets you write SQL queries instead of PromQL for your visualizations and comes with useful macros.

This is just an Argo CD app. It is a Helm chart deployed via Argo. The Altinity Operator is a prerequisite: it needs to be installed in your cluster before deploying this chart. Then we install the OpenTelemetry Collector with specific configuration and deploy Grafana with pre-provisioned dashboards.

[29:46] – Demo: OpenTelemetry Collector Configuration

Josh: Looking at the Helm values, this is where the interesting configuration lives. We have these presets. These are provided by the community and they set a whole bunch of commonly needed options for you without requiring you to build everything yourself. For example, you can enable the Kubernetes attributes processor and it just works. You can enable log collection and kubelet metrics collection. You can include logs for built-in services like the collector itself.

All these presets do is add configuration to the collector. We can look at what gets generated as the final configuration directly in the cluster. Because this is YAML, we can also add our own keys and override whatever we want. For example, I am running in a cluster without TLS certificates so I set insecure to true.

Then there is the filelog receiver configuration. This is where things get really interesting. We have operators, which are how we configure parsing for the various different log formats.

A container parser was added to the project and it basically replaces about 90 lines of parsers and routers that you used to need in order to handle the three different log formats from the container runtimes. Now you can just say “I am running in a containerized environment, if you see one of these three formats, parse it as if it is that.” Really useful. More of these types of parsers are being developed. Logging in OpenTelemetry is still maturing very quickly. I have also started working on a parser specifically for ClickHouse logs, which include things like thread ID and query ID that can get put into resource attributes.

The exporter configuration is really straightforward. Basically just give it an endpoint and any authentication you need. For a replicated cluster, you will need to add a few more options including cluster_name and a table_engine specification for ReplicatedMergeTree. We have a blog post on this covering what options you need for replication.

[34:04] – Demo: ClickHouseInstallation Resource and Kubernetes Objects

Josh: Looking at the Argo app in the cluster, we can see the monitoring app and all its config maps: one for Grafana dashboards and one for the OpenTelemetry Collector. In the operator, the most important custom resource is called a ClickHouseInstallation. From this single resource we basically tell the Altinity Operator: I need a ClickHouse cluster that looks like this. The operator then creates all the Kubernetes resources necessary to bring that cluster up. We can see a StatefulSet has been created, because we are running a stateful application. There is a single pod, a single replica and shard in this demo. A persistent volume claim was created for the data volume that stores all the data for our ClickHouse databases.

If we wanted to scale this out, if we had a replicated cluster, the Altinity Operator would handle scaling up these volumes one at a time so that each replica’s data would not get lost as the replicas scale up and discard their PVCs. This is really powerful for stateful workloads.

Then of course we have Grafana deployed as a ReplicaSet via the standard Helm chart, with some pre-provisioned dashboards.

[37:07] – Demo: Grafana Log Visualization

Josh: In the dashboard, we can see all the different log files that have been collected, shown as a parameter on this simple panel. We have logs from the OpenTelemetry Collector, logs for the built-in Kubernetes components, logs for Argo which I used to deploy all of this, and of course ClickHouse logs.

This is a simple panel with data formatted for the Grafana log panel. We can see the metadata we can extract: process ID or thread ID, query ID, log level. We can extract all of this into metadata using those parsers and then start creating dashboards that look like those RED dashboards we showed earlier, but now with all of the logs in one place combined with aggregated metrics, all of it queryable via SQL.

I am using both the Altinity plugin and the Grafana-maintained plugin for ClickHouse in this demo. The Grafana-maintained plugin does have a very polished UI and built-in support for OpenTelemetry, so you can check a box to indicate you are using the ClickHouse OpenTelemetry exporter and it already knows the column names. The Altinity plugin in particular comes with really useful macros. One of the most convenient is $timeFilter, which replaces the entire WHERE clause for time filtering and automatically maps to whatever time range is selected in Grafana.

[42:46] – Scaling to Production

Josh: What I showed was a lightweight demo, not fit for production as-is. If we wanted to scale this to production readiness, there are a few things to change.

In a production cluster we might have a lot of nodes, so those DaemonSets might have a lot of OpenTelemetry Collector instances. Having each one with a direct connection back to ClickHouse would get expensive in terms of compute and network. ClickHouse can handle massive ingest volumes but does not like many incoming connections with many small batches from many different sources.

What is better is to centralize it by leveraging the OpenTelemetry Collector deployed in two ways. One set is the DaemonSet, with one instance per node acting as the node agent, scraping kubelet stats and logs from that node, and acting as a local place for applications to send their telemetry without going over the network first. The second set is a Collector Deployment acting as a Gateway: it gathers all the telemetry from the DaemonSet collectors, possibly also gathering cluster API metrics, and then creates a few concentrated streams of data to your ClickHouse cluster.

Within that ClickHouse cluster, for production you absolutely need replication: at least three replicas and Keeper with replication so the cluster is fault-tolerant. Keep Keeper instances on separate nodes from the ClickHouse server instances.

The Altinity Operator manages the ClickHouse cluster instances. For Keeper, you can currently manage those with Helm.

Additional optimizations include fine-tuning the schema that the ClickHouse exporter creates for you, tweaking cardinality values for your data, potentially introducing sharding to partition data by tenancy or time, and things like that.

[45:24] – Q&A: Argo CD, Helm, and Deployment

Robert: Can you talk a little bit about Argo CD setup and how to get familiar with using it to deploy applications? Your setup is pretty dependent on everything being managed by Argo CD.

Josh: Sure. If we look at the repo, Argo CD in this case uses the app-of-apps pattern. Each YAML file defines an Argo CD application. The Argo CD operator creates a CRD for us and we can define applications just like those ClickHouseInstallation resources. When we define an application, it tells Argo CD: “I have this thing that I want you to deploy to my cluster.”

We have some options like self-healing and pruning. In honesty, these are something I would probably change in a production instance. I would turn pruning off so it does not prune resources that the Altinity Operator was trying to keep around in case you brought a cluster back after a failure. Self-healing is a little safer, but I might also turn it off in production if you want a person in the loop and you do not want Argo CD doing things on its own. With both set to false, Argo CD is basically just watching a Git repository. When the repository changes, Argo CD sees the change and applies it to the cluster by inflating the Helm templates and applying them directly to the cluster via the Kubernetes API, without using Helm releases. This means Argo kind of owns the resources: anything you change directly that does not match the manifest, Argo will put back.

Robert: Any comments on deploying this kind of system with just Helm charts and nothing else?

Josh: This chart could just be deployed using Helm. If you wanted to use this without Argo CD, you would need to either publish the chart somewhere or clone the repository and run helm install <release-name> . from the directory. It will deploy the chart the same way that Argo does. You lose the GitOps continuous reconciliation, but for a one-time or manually-triggered deployment it works just fine.

FAQ Section

Q: Why should I use the OpenTelemetry Collector for Kubernetes logging instead of Fluentd or Promtail?

A: If you are already using Fluentd or Promtail and they are working well for you, there is no urgent reason to switch. However, the OpenTelemetry Collector has one significant advantage: it handles all telemetry signals, not just logs. It can receive and process logs, metrics, traces, and profiles using a unified pipeline and can export to over 90 different backends simultaneously. This means you can use a single agent across your entire cluster for all observability signals, rather than running separate agents for logs, metrics, and traces. The batch processor also makes it easy to produce the large insert batches that ClickHouse prefers.

Q: What is the Kubernetes attributes processor and why is it important?

A: The Kubernetes attributes processor enriches all telemetry flowing through the OpenTelemetry Collector with Kubernetes-specific metadata by querying the Kubernetes API. It adds resource attributes such as pod name, pod ID, namespace, node name, and container name. These follow OpenTelemetry semantic conventions, which means the same metadata fields will be present on logs, metrics, and traces from the same pod. This makes it possible to correlate all three signals together using SQL joins in ClickHouse, for example correlating a spike in error logs with a corresponding metric anomaly and a specific trace showing where the failure originated.

Q: Why is ClickHouse a good choice for storing Kubernetes cluster logs?

A: ClickHouse has several properties that make it well-suited for log storage at scale. Its columnar storage and compression are very effective on log data, often achieving very high compression ratios. Its massive ingest capability, which can scale horizontally across many nodes, handles the volume of logs that large clusters generate. Materialized views and real-time transformations let you build refined data sets from raw logs without separate ETL jobs. And because it is a full SQL database, you can write complex queries that join log data with other tables, filter by any combination of resource attributes, and aggregate over time windows in ways that would be difficult or impossible with purpose-built log management tools.

Q: Do I need Argo CD to use this setup, or can I just use Helm?

A: Argo CD is not required. The demo uses Argo CD for its GitOps continuous reconciliation (watching a Git repository and keeping the cluster state in sync with it), but you can deploy the same Helm chart directly with helm install. The difference is that with Argo CD you get automatic re-application whenever the repository changes, while with plain Helm you trigger deployments manually. For a production setup you would want some form of continuous deployment, but Argo CD is just one option: Flux, Jenkins, or any other CI/CD tool that can run helm upgrade would also work.

Q: What changes are needed to scale the OpenTelemetry Collector to production in a large cluster?

A: In a large cluster the DaemonSet architecture creates too many direct connections to ClickHouse with many small batches. The production pattern is to use two tiers of collectors: DaemonSet collectors on each node acting as local agents, and a separate Collector Deployment acting as a gateway. The DaemonSet collectors forward all their telemetry to the gateway via OTLP. The gateway accumulates telemetry from all the agents, applies any cluster-wide processing, and then sends concentrated, properly-batched streams to ClickHouse. This reduces the number of connections to ClickHouse and improves the batch sizes, which is important for performance. You should also run a replicated ClickHouse cluster with at least three replicas and a Keeper ensemble managed by the Altinity Kubernetes Operator for ClickHouse®.

© 2026 Altinity, Inc. All rights reserved. Altinity®, Altinity.Cloud®, and Altinity Stable® are registered trademarks of Altinity, Inc. ClickHouse® is a registered trademark of ClickHouse, Inc.; Altinity is not affiliated with or associated with ClickHouse, Inc. Kubernetes, MySQL, and PostgreSQL are trademarks and property of their respective owners.