# Kubernetes ## (Docs) [Overview](https://kubernetes.io/docs/concepts/overview/) ### Going back in time Kubernetes is a portable, extensible, open source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. ![[Pasted image 20231019182955.png]] **Traditional deployment era:** Early on, organizations ran applications on physical servers. There was no way to define resource boundaries for applications in a physical server, and this caused resource allocation issues. For example, if multiple applications run on a physical server, there can be instances where one application would take up most of the resources, and as a result, the other applications would underperform. A solution for this would be to run each application on a different physical server. But this did not scale as resources were underutilized, and it was expensive for organizations to maintain many physical servers. **Virtualized deployment era:** As a solution, virtualization was introduced. It allows you to run multiple Virtual Machines (VMs) on a single physical server's CPU. Virtualization allows applications to be isolated between VMs and provides a level of security as the information of one application cannot be freely accessed by another application. Virtualization allows better utilization of resources in a physical server and allows better scalability because an application can be added or updated easily, reduces hardware costs, and much more. With virtualization you can present a set of physical resources as a cluster of disposable virtual machines. Each VM is a full machine running all the components, including its own operating system, on top of the virtualized hardware. **Container deployment era:** Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. Similar to a VM, a container has its own filesystem, share of CPU, memory, process space, and more. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions. Containers have become popular because they provide extra benefits, such as: - Agile application creation and deployment: increased ease and efficiency of container image creation compared to VM image use. - Continuous development, integration, and deployment: provides for reliable and frequent container image build and deployment with quick and efficient rollbacks (due to image immutability). - Dev and Ops separation of concerns: create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure. - Observability: not only surfaces OS-level information and metrics, but also application health and other signals. - Environmental consistency across development, testing, and production: runs the same on a laptop as it does in the cloud. - Cloud and OS distribution portability: runs on Ubuntu, RHEL, CoreOS, on-premises, on major public clouds, and anywhere else. - Application-centric management: raises the level of abstraction from running an OS on virtual hardware to running an application on an OS using logical resources. - Loosely coupled, distributed, elastic, liberated micro-services: applications are broken into smaller, independent pieces and can be deployed and managed dynamically – not a monolithic stack running on one big single-purpose machine. - Resource isolation: predictable application performance. - Resource utilization: high efficiency and density. ### Why you need Kubernetes and what it can do Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn't it be easier if this behavior was handled by a system? That's how Kubernetes comes to the rescue! Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of scaling and failover for your application, provides deployment patterns, and more. For example: Kubernetes can easily manage a canary deployment for your system. Kubernetes provides you with: - **Service discovery and load balancing** Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable. - **Storage orchestration** Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more. - **Automated rollouts and rollbacks** You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container. - **Automatic bin packing** You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources. - **Self-healing** Kubernetes restarts containers that fail, replaces containers, kills containers that don't respond to your user-defined health check, and doesn't advertise them to clients until they are ready to serve. - **Secret and configuration management** Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration. - **Batch execution** In addition to services, Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired. - **Horizontal scaling** Scale your application up and down with a simple command, with a UI, or automatically based on CPU usage. - **IPv4/IPv6 dual-stack** Allocation of IPv4 and IPv6 addresses to Pods and Services - **Designed for extensibility** Add features to your Kubernetes cluster without changing upstream source code. ## [Two reasons Kubernetes is so complex](https://buttondown.email/nelhage/archive/two-reasons-kubernetes-is-so-complex/) [[2022-01-27]] (excerpts) ### Kubernetes is a cluster operating system It’s easy to think of Kubernetes as a system for deploying containerized applications, or some similar functional description. While this can be a useful perspective, I think it’s much more productive to think of Kubernetes as a **general-purpose cluster operating system kernel**. What do I mean by that, and what’s the difference? The job of a traditional operating system is to take a single computer, and all of its attendant hardware, and to expose an interface that programs can use to access this hardware. While the exact details vary, in general this interface has some number of the following goals: - **Resource sharing** — We want to take the one physical computer and subdivide its resources among multiple programs, in such a way that they are isolated from each other to some extent. - **Portability** — We want to abstract the precise details of the underlying hardware to some extent, such that the same program can run on different pieces of hardware without modifications, or with only minor modifications. - **Generality** — As we come up with new kinds of hardware, or plug new hardware into our computer, we want to be able to fit those into our abstractions and interfaces in an incremental way, ideally without (a) drastically changing any interfaces or (b) breaking any existing pieces of software which don’t use that hardware. - **Totality** — Related to generality, we want the OS to mediate **all** access to hardware: it should be rare or impossible for software to completely bypass the OS kernel. Software can use the OS kernel to **set up** a direct connection to hardware such that future interaction happens directly (e.g. setting up a memory-mapped command pipe), but the initial allocation and configuration is still under the OS’ aegis. - **Performance** — We want to pay an acceptably small performance cost for having this abstraction, as compared to “directly writing a special-purpose piece of software that ran directly on the hardware and had exclusive direct access to the hardware” (ala a unikernel). In some cases we want to achieve higher performance **in practice** than such a system, by offering optimizations like I/O schedulers or caching layers. While “ease of programming” is often an additional goal, in practice it often loses out to the above concerns. Operating system kernels often get designed around the above goals, and then userspace libraries are written to wrap the low-level, general-purpose, high-performance interfaces into easier-to-use abstractions. OS developers tend to be far more concerned with “How **fast** is it possible to make nginx run on my OS” than they are with “How many lines of code shorter is the nginx port to my OS?” I‘ve come to think of Kubernetes as operating in a very similar design space; instead of abstracting over a single computer, however, it aims to abstract **an entire datacenter or cloud**, or a large slice thereof. The reason I find this view helpful is that that problem is much harder and more general than, say, “making it possible to deploy HTTP applications in containers,” and it points at specific reasons Kubernetes is so flexible. Kubernetes aspires to be general and powerful enough to deploy any kind of application on any kind of hardware (or VM instances), without necessitating that you “go around” or “go outside” the Kubernetes interface. I won’t try to opine here on whether or not it achieves that goal (or, perhaps, **when** it does or doesn’t achieve that goal in practice); the mere perspective of viewing that as its problem statement causes a lot of design decisions I encounter to “make sense” to me, and seems like a useful lens. I think that perhaps the biggest design choice this perspective explains is how **pluggable** and **configurable** Kubernetes is. It is, in general, impossible to make choices which can be everything to everyone, especially if you aspire to do so without extravagant performance cost. This is true especially in the modern cloud environment, where the types of applications and type of hardware deployed vary vastly and are very fast moving targets. Thus, if you want to be everything to everyone, you end up needing to be enormously configurable, which ends up creating a powerful system, but one which can be hard to understand, or which makes even “simple” tasks complex. ### Everything in Kubernetes is a control loop One could imagine a very imperative “cluster operating system,” like the above, which exposed primitives like “allocate 5 CPUs worth of compute” or “create a new virtual network,” which in turn backed onto configuration changes either in the system’s internal abstractions or into calls into the EC2 API (or other underlying cloud provider). Kubernetes, as a core design decision, does not work like that. Instead, Kubernetes makes the core design decision that all configuration is **declarative**, and that all configuration is implemented by way of “[operators](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/)” which function as **control loops**: They continually compare the desired configuration with the state of reality, and then attempt to take actions to bring reality in line with the desired state. This is a very deliberate design choice, and one made with good reasons. In general, any system which is **not** designed as a control loop will inevitably drift out of the desired configuration, and so, at scale, **someone** needs to be writing control loops. By internalizing them, Kubernetes hopes to allow most of the core control loops to be written only once, and by domain experts, and thus make it much easier to build reliable systems on top of them. It’s also a natural choice for a system that is, by its nature, distributed and designed for building distributed systems. The defining nature of distributed systems is the possibility of **partial failure**, which necessitates that systems past some scale be self-healing and converge on the correct state regardless of local failures. However, this design choice also brings with it an enormous amount of complexity and opportunity for confusion[1](https://buttondown.email/nelhage/archive/two-reasons-kubernetes-is-so-complex/?utm_source=hackernewsletter&utm_medium=email&utm_term=code#fn:frontload). To pick two concrete ones: **Errors are delayed** Creating an object (e.g. a pod) in Kubernetes, in general, just creates an object in the configuration store asserting the desired existence of that object. If it turns out to be impossible to actually fulfill that request, either because of resource limitations (the cluster is at capacity), or because the object is internally-inconsistent in some way (the container image you reference does not exist), you will not, in general, see that error at creation time. The configuration creation will go through, and then, when the relevant operator wakes up and attempts to implement the change, only then will an error be created. This indirectness makes everything harder to debug and reason about, since you can’t use “the creation succeeded” as a good shorthand for “the resulting object exists.” It also means that log messages or debug output related to a failure do not appear in the context of the process that created an object. A well-written controller will emit Kubernetes events explaining what’s happening, or otherwise annotate the troublesome object; but for a less well-tested controller or a rarer failure, you might just get logspam in the controller’s own logs. And some changes may involve multiple controllers, acting independently or even in conjunction, making it that much harder to track down just which damn piece of code is actually failing. **Operators may be buggy** The declarative control-loop pattern provides the implicit promise that you, the user, don’t need to worry about **how** to get from state A to state B; you need merely write state B into the configuration database, and wait. And when it works well, this is in fact a tremendous simplification. However, sometimes it’s not possible to get from state A to state B, even if state B would be achievable on its own. Or perhaps it is possible, but would require downtime. Or perhaps it’s possible, but it’s a rare use case, and so the author of the controller forgot to implement it. For the core built-in primitives in Kubernetes, you have a decent guarantee that they are well-tested and well-used, and hopefully work pretty well. But when you start adding third-party resources, to manage [TLS certificates](https://cert-manager.io/) or [cloud load balancers](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/) or [hosted databases](https://aws.amazon.com/blogs/opensource/aws-service-operator-kubernetes-available/) or [external DNS names](https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/gke.md) (and the design of Kubernetes tends to push you in this direction, because it’s happier when it can be the source-of-truth for your entire stack), you wander off the beaten path, and it becomes much less clear how well-tested all the paths are. And, in line with the previous point about delayed errors, the failure modes are subtle and happen at a distance; and it can be difficult to tell the difference between “the change hasn’t gotten picked up yet” and “the change will never be picked up.” ### Conclusion I’ve tried to avoid making value judgments on whether I think these design decisions were good choices or not in this post. I think there is plenty of scope for debate about when and for what kinds of systems Kubernetes makes sense and adds value, versus when something simpler might suffice. However, in order to make those kinds of decisions, I find it tremendously valuable to come to them with a decent understanding of Kubernetes on its own terms, and a good understanding of where its complexity comes from, and what goals it is serving.