Why BYOC is the New Management Paradigm for Analytic Databases

An exciting paradigm shift is shaking up SaaS data management. Surprisingly, it’s not AI! It’s called Bring Your Own Cloud or “BYOC” and runs databases directly in user accounts. Users who want control of data and costs are driving rapid growth. This article explains BYOC and shows why it’s a better choice than centralized SaaS models for managing databases, especially those used for analytics.

What Problem Does BYOC Solve?

BYOC balances convenient cloud operation with user control over cost, location of data, and security. To understand why this combination is so enticing, it’s helpful to understand how BYOC evolved. 

The first generation of databases were self-managed servers–think Oracle or Sybase–that users operated entirely by themselves, often at considerable inconvenience. Users were responsible for all aspects of hardware procurement, installation, and database management. Acquiring hardware alone could take up to six months. It was equally inconvenient for vendors to distribute and support enterprise releases. 

SaaS offerings like Snowflake decisively solved the convenience problem by putting databases in centrally managed, multi-tenant cloud services. For users, database procurement time dropped from months to minutes and enabled an era of unprecedented experimentation in data analysis. On the vendor side it enabled new efficiencies: fixing a bug for one customer in a multi-tenant system fixed it for all tenants. It also made support vastly easier by allowing vendors direct access to problematic systems and their logs. 

However…The tremendous convenience of centralized SaaS databases obscured a significant drawback for users: loss of control. Snowflake hides compute, storage, and networking resources, along with the database code. Data lives in the vendor cloud, creating a lock-in hazard and potentially exposing confidential information. Users can optimize cost, data location, and security, but only to the extent that the vendor provides levers to do so.

BYOC is a middle ground that balances cloud automation and user control. Like traditional SaaS, BYOC services offer convenient cloud database operation: everything from initial setup to backup to proactive monitoring and support. Unlike traditional SaaS, BYOC data planes run in user accounts. Users specify where data lives, can see cloud resources like VMs, can pay for them directly, and can set security policies. Key levers to control cost, performance, data location, and access are entirely in user hands. 

Altinity’s cloud service is Altinity.Cloud. It manages ClickHouse® using BYOC as well as SaaS models. From the user perspective, there is no difference in the management features.

How Does BYOC Work?

There are a number of architectural patterns for BYOC database services. For this discussion we’ll focus on two common approaches. They are both based on Kubernetes, which is now a default platform for databases in both cloud and on-prem environments. We call these patterns Pure BYOC and BYOK. The difference is how they split the management responsibility boundary between the BYOC vendor and users.

Pure BYOC sets up a VPC in the user cloud account, installs Kubernetes to orchestrate cloud resources, and adds services to control database clusters. The vendor takes care of everything, including complex administration tasks like wiring cloud IAM policies and upgrading Kubernetes. This is a good pattern for users who want to pull databases into their own environments but keep management as hands-off as possible. 

The BYOK pattern manages databases in an existing Kubernetes cluster. The user sets up Kubernetes to vendor specifications, for example with Terraform, then registers the cluster with the vendor management plane. The vendor then installs the management services as before. This is a good pattern for users familiar with Kubernetes administration who run curated Kubernetes, who do not permit external vendors to manage their cloud infrastructure security, or want to run other applications alongside managed databases. It’s also a standard path to run in on-prem environments, since Kubernetes runs there as well. 

Altinity.Cloud supports both Pure BYOC and BYOK deployment patterns. It runs in any environment where Kubernetes can run and open an outbound network connection to the Altinity.Cloud management plane.

Why Is BYOC Especially Good for Analytic Databases?

Analytic databases have a number of properties that make them especially well suited for BYOC management. They are often large and use correspondingly large amounts of compute and storage. The size means they are costly to operate and hard to move. In addition, analytic databases often contain sensitive data that users do not want exposed to the Internet, vendor AI, or prying eyes in general. BYOC addresses each of these challenges.

	In the BYOC models we have described, users are free to choose specific machine and storage types. For example, users can apply spot instances, ARM processors, or even opt for low-cost providers like Hetzner. Users can also apply their own PPAs, savings plans, and startup credits to reduce bills.
	Users can directly control the location of databases and backups. This enables them to support regulations like GDPR as well as meet contractual obligations to specific customers related to data location and access.
	Controlling location of databases as well as infrastructure opens up a cornucopia of ways to boost performance. Databases that read from object storage can be located in the same region for fast scans. Users can also select high-performance providers like Nirvana Labs, which offer optimized networking, fast storage, and rack-collocation with data sources to lower latency on time-sensitive data.
	BYOC databases run within the user security perimeter, which means users are free to control access policies. They can also undertake additional measures like scanning containers used in BYOC services to meet in-house security requirements.

Altinity.Cloud supports all of the features described above and more. We’re actively expanding support for new cloud providers as well as on-prem environments.

What Should You Look for in a BYOC Vendor?

Beyond a solid architecture and product features, there are three important questions users should answer when evaluating prospective BYOC services for analytic databases. 

Is the pricing user-friendly? 
Cost optimization is a key benefit of BYOC, which is why you should ensure that the vendor’s pricing does not penalize you when selecting preferred VMs, storage, and networking configurations. Per vCPU pricing in some form is an attractive model. The costs are easy to estimate and scale linearly with compute usage, leaving you free to optimize CPU types, storage, and network transfer costs. Beware of BYOC pricing that charges for storage. It’s the part of your bill that increases fastest in analytic applications.

What level of database support does the vendor provide? 
Databases are complex beasts and it’s extremely helpful to have help in application design as well as operation. The best BYOC offerings forward monitoring data to the vendor for fast, proactive management of database systems. Look for vendors that combine solid DBA and cloud support. Make sure your preferred vendor has protections on data to prevent support staff from seeing confidential data. At the same time, ensure you can grant temporary access to solve problems quickly.

Are database builds free from lock-in? 
BYOC offerings that run on proprietary database builds move SaaS lock-in problems to another location. If you turn off the service you may forfeit your legal right to operate the database itself. BYOC vendors that run open source builds avoid this problem. If you need to migrate, you should be able to unplug and run it yourself. Test before you sign up!

Altinity.Cloud uses per vCPU pricing, includes Enterprise DBA support by default, and only runs open source builds of ClickHouse. You can unplug from Altinity.Cloud any time you want. It’s your data, in your account. 

Conclusion

BYOC is a new paradigm for managing analytic databases that balances convenient cloud management and user control. Cloud management allows developers to deploy new analytic applications quickly and reduce labor to operate them. User control makes the infrastructure visible, allowing a wealth of creative optimizations around cost, performance, data location, and security. 

Altinity.Cloud introduced the first BYOC managed service for ClickHouse in 2022. We have years of experience operating it across a wide range of environments. We’re excited about the many possibilities that BYOC offers to build high-performance analytics in almost any environment. If you would like to find out more, contact us, or start a BYOC trial on Altinity.Cloud today.