Learning from Rockset: Avoiding the Dangers of Lock-in to Cloud Database-as-a-Services
A few days ago the database world was surprised by the news: Open AI has acquired the database company Rockset. We know Rockset very well and respect the team–they tried to compete with ClickHouse® for speed. We were happy to hear their technology was recognized and positioned to reach the next level in a new home.
But the happiness quickly switched to discouragement when we learned that the Rockset cloud service is going to be terminated, with Rockset customers required to migrate to something else by the end of September. This is painfully short, especially for proprietary software that has no direct equivalent. It’s not an exaggeration to say it could be a disaster for some users! It’s also an opportunity to learn.
The risk of trusting data to cloud vendors
Cloud database-as-a-services automate complex administration and free developers to focus on applications. The cost is a new risk for the business that is hard to measure. What if they raise prices? What if the vendor is acquired? What if they disappear? The dilemmas range from extra expense to mortal threats to your business.
The conventional approach is to get lawyers involved and hammer out contracts that prevent rug pulls. But not every vendor will even agree to reasonable terms. Some risks only become apparent later when it’s too late. And no contract protects you if the vendor just stops paying bills to their own cloud vendors. So the question arises – is there a way to keep benefits of DBaaS service providers but remove the downside of vendor lock-in, including risks you can’t even see?
The answer is yes. But you need to pick technology and a vendor considering three important requirements.
1. Use open source DBMS with a well established community
The most important step is to use an open source database. If the vendor goes away, the technology remains available. But there are two nuances to keep in mind.
Open source community
Sometimes open source technology is developed by a single company. They have all expertise about the design, code, build process etc. If the company disappears, the open source project may be abandoned. This almost happened to RethinkDB. Company closed the project, and it took three years (!) before the community was able to pick it up. For three years users were unsure about the future.
Having a strong open source community–including users and committers–is the best safeguard that the project will continue. .
Governance model
Vendor interests are not always aligned with users. The technology may therefore evolve in ways that are not beneficial to users at large. The vendor may eventually decide to change the license, which can put applications at risk. We have seen this happen with ElasticSearch, MongoDB, and many others.
The safest path is when the open source project is covered by an established governance model, like Apache Foundation or Linux Foundation. The risk one vendor may take the control of user’s destiny is far lower. It also eliminates the problem of relicensing. This is why the Valkey community chose Linux Foundation as a home when they created a community fork of Redis.
2. Use an open source operations stack
No database is an island. Deployment, management, and monitoring tools are required to operate it efficiently. This is the operations stack. Ideally, it should also be composed from open source components. One of the popular deployment models today is Kubernetes. It is open source and encourages the use of open source components. It works well with public clouds like AWS, GCP, or Azure – all major cloud providers have a managed Kubernetes offering that is totally compatible with open source.
Databases are usually managed in Kubernetes using operators. Having an open source operator allows one to manage it directly, or build an extra management layer on top.
The last part is observability. There are plenty of open source options available. They often include a hosted solution that is totally compatible with open source. Grafana is a good example: You can use open source Grafana or Grafana Cloud.
3. Keep data in your account
Even with a fully open source database and operating stack, there’s still a practical problem. The database infrastructure and data are in the vendor account, not yours. Migrating away is time-consuming and risky, as Rockset is demonstrating. Fortunately, there’s an alternative way for vendors to manage databases: keep the data, compute resources, and other resources in user accounts!
With this approach, the vendor runs a management plane that manages resources fully owned by the user. Even if a vendor disappears overnight, all critical assets are under the user’s control! It may take a while to learn how to manage it, but since all components are open source this is possible to do.
Altinity.Cloud Anywhere – a no lock-in DBaaS for analytics on ClickHouse®
Guided by our users, we started to work years ago on a solution to avoid vendor lock-in in our popular cloud for ClickHouse databases. It’s called Altinity.Cloud Anywhere. It incorporates the requirements of open source and user ownership.
Altinity.Cloud Anywhere runs open source ClickHouse. The operations stack is built on top of Kubernetes using open source clickhouse-operator, clickhouse-backup, Prometheus, Grafana and other components. And it allows everything to run in the user’s cloud account or even on-prem. Users may disconnect at any time from the management plane and continue running ClickHouse stack on their own. That gives 100% confidence there is no lock-in, and it builds a high trust between users and the vendor.
If you are shocked with the Rockset news and concerned by DBaaS vendor lock-in – reach out to Altinity. We have a solution for you.