Share this post
Much of the last decade or two of the tech industry has been dominated by the idea of the cloud: the simple, powerful idea that all of your applications and data can be accessed from any device with an internet connection. Enterprise businesses around the world have decommissioned server rooms in favor of subscribing to services maintained by other people, reducing overheads of all kinds. Even companies that once built and operated vast datacenters now rely on cloud providers. At the same time, cloud services have helped individuals save money upfront (even if subscriptions often cost more in the long run) and have taken the need for installation and troubleshooting out of the picture. Overall, cloud services have saved lots of people huge amounts of time and money.
But this convenience comes with trade-offs, some of which have become more apparent over time.
- You only ever rent: There’s no real ownership, and vendors can modify, discontinue, or increase prices at will.
- Privacy concerns: Because your data and activity pass through the provider’s infrastructure, they can be easily monitored, tracked, or even resold.
- Jurisdictional constraints: Your data often resides in the provider’s chosen region, subjecting it to local laws, which may not align with your needs.
- Downtime and dependence: If your cloud provider goes down, so does your access — sometimes across multiple services.
- Vendor lock-in: Moving away from a cloud provider can be complex and expensive, discouraging competition and user control.
These trade-offs become even riskier when your work involves sensitive information. When software and infrastructure are controlled by the same entity, it not only enables easy monitoring and resale of your data but also makes it a prime target for subpoenas. Courts can compel a cloud service provider to produce your data, often without your involvement.
If your work involves sensitive personal information, for example of patients or sources, that could put their privacy and safety at risk. That might be particularly problematic in a situation where, for example, you work on reproductive health issues, and your software is hosted in a jurisdiction that has abortion bans. This risk extends across professions: journalists protecting sources, lawyers safeguarding client data, and healthcare providers managing patient records all face heightened exposure when their software, data, and infrastructure are all controlled by the same party.
At the same time, moving away from the convenience of the cloud is not really an option for most organizations. To date, most have opted to pay for more expensive enterprise contracts, which promise greater data protections alongside features like stronger audit logs and SSO. These provide some legal protections, but still amount to little more than an enforced promise: the vendor physically can inspect your data in most cases, you’re just paying them extra so that they’ll promise not to. These contracts also don’t address jurisdictional issues: if the vendor is based in Texas, your use of their platform is still subject to Texan law. The power dynamics at play remain unaddressed.
This is also problematic when you consider the increasing popularity of LLMs. If you’re dealing with sensitive or proprietary information, you probably don’t want an AI model to be trained on your data. You can pay vendors like OpenAI to promise that they won’t look at your data or train their models on it — but, again, you need to take their word for it.
If we want to retain the benefits of cloud software without its fundamental risks, we need a different model: one that restores control to users and organizations rather than vendors.
- Retain the ease of deployment, access, and collaboration that makes cloud software so appealing.
- De-couple software and infrastructure so that the company making the software is not the company that hosts the software.
- Allow customers to pick an infrastructure host in the jurisdiction of their choice.
- Ensure that data is encrypted at rest and in transit, so that even the hosting provider cannot access it.
Self-hosted cloud software is, of course, absolutely a thing that already exists. Some of it is even end-to-end encrypted. But it’s also largely free and open source, and requires a fair amount of configuration and maintenance from an organization’s IT department. There’s nothing wrong with open source software (I ran two open source startups!), but the complexity of configuration and lack of clear business model can introduce problems for both the customer and the vendor. Vendors like Cloudron are making this easier for open source software — and they should serve as a model for what could come next.
Some cloud infrastructure providers, like AWS, already host marketplaces of software you can install. The trick is, you usually have to decide which kinds of virtual servers to use — are you going to go for an m3.medium or a t2.xlarge? — and then consider how your private cloud will be configured. AWS also offers self-hosting for LLM models through Amazon Bedrock, but the same problems present themselves. There’s a lot of technical overhead which many organizations can’t easily address — and in stark contrast to a cloud offering like Google Workspace, which is completely turn-key.
But this doesn’t have to be the case. What if we could combine the ease of cloud-based software with the control and flexibility of locally managed applications?
Consider an iPhone: here, your software runs on your device, wherever it might be, but is seamlessly downloaded from an App Store on demand. Some of that software is free; some of it is paid-for, either as a one-off or on a subscription basis. The underlying operating system is a variant of the FreeBSD UNIX system with significant proprietary additions, including some sophisticated sandboxing, but you wouldn’t know it, and you certainly don’t need to configure anything: you request an app, and zip!, there it is on your phone.
Consider this user journey:
- The customer signs up to a certified provider in the jurisdiction of their choice. There are providers tailored for different levels of customer and different industries.
- They add their payment information.
- They choose the software they want to provide to their organization from an App Store accessed through the provider. As soon as they install it, it is near-instantly available to them.
- They can make it available to every user in their organization or a subset of users.
- For every user for whom it is available, the app shows up on a web-based dashboard. It can also be configured to automatically show up in providers like Okta.
- They never have to care about the speed or capacity of the underlying hardware: they just pay for a recurring license to the software.
- They never have to care about configuring or upgrading the software: as soon as they select it, it’s available. Customers can opt for updates to be pushed out automatically, or they can hold back non-security updates for more testing.
The App Store distributes revenue to the vendor and the hosting provider, and takes a cut for itself. Apps are charged for on a predictable, monthly, per-seat basis, with each app able to set its own prices. As is the case with a phone App Store, the store itself does some vetting of each application, certifying it for security and a set of core rules that each app must abide by. Unlike a phone App Store, it also does vetting and certification of the hosting provider itself, reducing the customer’s need to undertake security auditing.
Because every hosting provider associated with an App Store would necessarily need to adhere to the same open standards, the customer could move providers easily. They’d just sign up to another hosting provider associated with the App Store and migrate their apps. The App Store itself would handle the rest, dealing with migrating block storage, databases, and so on behind the scenes.
This model isn’t just about redistributing power from giant cloud vendors to customers. It’s about enabling organizations that deal with sensitive data to more easily use the cloud to begin with. It makes it easier to know that there is an enforced separation between an LLM and its training infrastructure. And it creates new opportunities for vendors that might not be in a position to offer their own cloud infrastructure, too. It lowers the barrier to both privacy and innovation for everyone involved.
Existing cloud providers aren’t incentivized to build this. It’ll take a new entrant or someone willing to make a big bet. The technology to do this already exists. The only question is: who will build it first?
If it’s you, I’d love to hear from you.
