Why does Kubernetes exist?

And what you probably want instead.

Photo by Growtika on Unsplash

Kubernetes, k8s, Uberkitties—whatever you call it. You don’t need it. You probably just want a thing to run containers for you.

I know Kubernetes sounds really cool. But after using it for about 5 years, I can give some broad advice here and save you some pain: 99% of the companies using Kubernetes probably shouldn’t be doing so.

This has been my own opinion for a while, but I am not alone in it:

src tweet

What problem did k8s solve?

People think Kubernetes solves scaling. It doesn’t really, not directly.

So where did this opinion come from? Let’s step back in time and remember the good ole days.

It’s 2005. Setting up new VMs takes time. So when a surge of traffic comes in and your site gets saturated with requests, you lose out on dropped traffic or potentially go down.

Your company advertises at the "Big Game"¹, where an enormous amount of traffic comes in a window of 5-10 minutes. This short traffic burst is more than the previous 364 days of prior operation.

Given that’s an exceptional scenario, most companies vaguely know to plan for it and buy extra capacity and then turn it off after the event.

This is the way. It’s best practice.

However, this approach doesn’t help you as much for day-to-day or week-to-week operations of setting up and tearing down.

Depending on the kind of business, this trough to peak delta in usage could be 95% to 5%, and the 95% case happened only a fraction of the day. That is, you have 20 VMs around, but you’re only using 1 (on average) most of the time.

So the scaling problem was not just up but also down.

The answer to this problem was not Kubernetes. It was another concept that is tightly associated with Kubernetes now known as containers, Docker being the most popular brand associated with this concept.

How did containers fix the problem, you ask? They didn’t directly. What they allowed for you to do is quickly a buy a bunch of VMs (or up scale your existing ones), download a file (the container image) and turn it on with your app in seconds if not less than a second.

Great! Problem solved! Let’s go have beers!

But now you had a new problem, you needed a way to manage this new container abstraction and its whole lifecycle. Kubernetes solved this problem: container lifecycle management.

“Let’s use Kubernetes!” Now you have 8 problems. - Itamar Turner-Trauring

What were we solving again?

If you followed the plot, you’ll notice that the solution didn’t quite match the problem, but was an answer to yet another problem that was formed.

Kubernetes didn’t solving scaling. It solved the problems related to needing to manage containers.

Ah, but you might think it was the necessary complement, the rest of the puzzle, right?

Not really. Kubernetes didn’t help you manage the problem of scaling up and down the underlying VMs that you still totally needed. By that I mean, it didn’t go out and buy and then sell off VMs for you.

That was still on you manage.

You just had to write the bit to scale up and down your infrastructure based on demand and provision Kubernetes to the new VMs. This of course is still quite a lot to make. Also you also had to now learn Kubernetes too, which is famously hard.²

Eventually, though, various cloud providers gave you fully-managed Kubernetes engines so you didn’t need to manage even that any more.

With that, you might think that this history lesson is interesting, but no longer relevant, since it’s “fully managed”, right?

That “fully managed” bit is a bit misleading, though. All the raw and gnarly bits of understanding everything about how Kubernetes services work for your application is 100% still your problem.³

How do we simplify the problem managing all these servers with containers on them?

Serverless?

Somewhere in all of this, people decided that the problem must be servers themselves: clearly you don’t actually want to buy and run servers. You just want your application run somewhere.

A large quantity of “serverless” offerings began to pop up, AWS Lambdas, Azure Functions, and Google Cloud Functions being the most popular.

The problem with “serverless” is multi-faceted:

• each cloud vendor’s serverless option works differently, so high-lock in

• couldn’t deploy generic containers, which was now the defacto abstraction

• sent up a package of ZIP files with no OS-level dependencies

• had a max process lifetime of minutes to at most an hour

• had no direct handling of background jobs

• significantly more expensive if you had regularly sized load

• had potentially long cold starts ⁴

• difficult to replicate the serverless topology anywhere except on the specific cloud vendor

To be honest, the area has lots of potential still. But with all the ergonomic problems above, serverless adoption train has not moved very fast.

The State of the Art

In the last 5 years or so, there’s been a middle ground that’s evolved: serverless-like scaling of containers. The cloud vendor actually manages everything for you. You don’t need to know about things like NetworkIngresses or ClusterIPs. And most if not all of the idiosyncrasies of each vendor are fewer.

Here’s a quick list of products for you:

AWS - Fargate
Azure - Container Instances
GCP - Cloud Run

For those who don’t like Big Tech or just want an even simpler experience, you might like to try:

Fly.io
Railway
Render
Easypanel⁵

But who should run Kubernetes then?

I can see three different kinds of use cases which might have an advantage on k8s.

1. A large quantity of cyclical load—traffic or background jobs

2. Have unusual networking or file system needs

3. Want to use multiple cloud providers simultaneously

This list isn’t meant to be exhaustive by the way.

I’d almost say you could be like DHH and run your own physical hardware again, because it makes financial sense for you. But even he ditched Kubernetes.

Kubernetes is far more flexible than any single cloud service. That’s probably its greatest strength. If you need it, you can come up with a way to configure it.

But you’re most likely not doing something that special.

Addendum: Buildpacks

A few astute readers will notice that I completely left out Heroku from my narrative, and their contribution to this space called Buildpacks, all the way back in 2011. Part of my reason for this is that historically this had been a Heroku only configuration option. That was until 2018, where they donated this to the Linux foundation.

In the meantime, we have had several competing alternatives⁶ in the “buildpack” space that all differ greatly from each other. Further, they’re not exactly a competitor for containers; they’re more like competitors for Dockerfiles. Each vendor’s buildpack spec allows devs to declaratively define what kind of app they want, and behind the scenes each vendor builds a container.

1. For what it’s worth this problem still happens. It happened to Coinbase in 2022.

2. This is not a secret. One does not simply learn Kubernetes. People have written about it before.

3. And to repeat, it is a big problem to get your head around.

4. This article is a bit older, but gives a good flavor of the range and nuance involved.

5. Note that Easypanel is not like the others. It sort of stands in-between being managed Kubernetes and a fully-managed container service. Worth paying attention to since you can use it on vendors who don’t have a container service.

6. Paketo, Nixpacks