Three rings of abstraction in Kubernetes

17 May 2021 Tags: Distributed systems

A common analogy for Kubernetes presents it as “the operating system for microservices”, identifying Kubernetes as a kind of operating system. I don’t think this analogy is wrong but I do think it’s underinformative, leaving the key question unanswered: What abstractions does this “operating system” provide? I suggest that unlike the more focused set of abstractions provided by the classic Unix model, the abstractions of Kubernetes can be grouped into three concentric rings. This highlights the different kinds of benefits Kubernetes affords and clarifies the decisions an organization must make when adopting Kubernetes.

A mercifully brief history of operating systems and their textbooks

The description of an operating system in terms of its supported abstractions took time to develop. Early textbooks emphasized that operating systems provided machine independence and resource sharing. “Machine independence” in these early days might best be thought of as device independence. Operating systems insulated the developer from specific devices within the hardware families offered by a given vendor but didn’t provide vendor independence. System services exposed a large number of implementation details, requiring the application programmer to manage them. For example, IBM’s MVS/370 operating system featured a welter of file access methods, with acronyms such as BPAM, QSAM, BDAM, ISAM, and VSAM, each with its own strengths, weaknesses, and tuning parameters.

Unix was the first widely-available system to take a different approach, offering a select set of higher-level abstractions substantially above the level of the underlying hardware. The CPU was abstracted into a process, a coherent set of services such as file descriptors, memory space control, and signals. These services operated at a distinctly more abstract level than anything provided by the underlying hardware. The input and output devices were abstracted into a file system. Unlike the many distinct access methods provided by systems such as MVS, Unix emphasized the commonalities supported by different hardware and made them as similar as possible. An application sequentially writing text made the same system calls regardless of whether the text was going to a terminal, a tape, or a disk.

From this perspective, the operating system implements an abstract machine that is more systematic than the actual machine: regular, consistent, simple, operating at a level meaningful to application programmers. This abstract machine was implemented by the kernel.

The development of this understanding was reflected in early textbooks. I believe that the first text I saw that presented operating systems in terms of their support for an abstract machine was the first edition of Andrew Tannenbaum’s Modern Operating Systems.

If Kubernetes is an operating system, what abstractions does it implement? What abstract machine model does it provide? I suggest that the scope of the term “Kubernetes abstract machine” is broader than that of older operating systems, comprising three concentric rings.

Innermost ring: abstractions implemented by the control plane

The innermost ring of Kubernetes’s abstractions is directly implemented in the control plane, supported by the highly-available consistent store etcd. Their specifications are located in the core API group. A sampling of these abstractions includes:

• Those for the underlying hardware, such as Node and Volume.
• Those for applications and their interconnections, such as Service and Deployment.
• Those for processes and groups of processes, such as ReplicationController, Pod, and Container.
• Those for external and internal signals, such as Event.
• Those for security, isolation, and configuration such as Secret, Namespace, and ConfigMap.

Inner ring design patterns

Additionally, there are two important meta-abstractions, design patterns fundamental to the control plane:

• The controller loop
• The Kubernetes API, specified in OpenAPI syntax and exemplified in the client library for Go

Purpose of the inner ring

Together, these core abstractions define the basic objects managed by Kubernetes and the operations that might be performed on them. They establish the fundamental model of the system, such as reconciling the observed state to the desired state. Their implementations run on nodes which are often reserved for their exclusive use and typically in a high-availability configuration of multiple replicas coordinating via distributed consensus. The choices embodied in these designs ripple through the entire system, touching every facet of a Kubernetes installation.

Correspondence to operating systems

The inner ring of Kubernetes abstractions corresponds closely to the kernel of an operating system: critical for performance, availability, and security, while establishing the basic resources that will be interconnected by the middle ring.

Middle ring: interfaces with multiple implementations

The middle ring comprises systems that are essential to a cluster, similar to the inner ring, but with the difference that Kubernetes merely specifies interfaces, which in turn have multiple implementations. In many cases, Google considered these systems distinct from orchestration and so did not package them with Kubernetes. Over time, the interfaces for these systems became codified into standards.

DNS

DNS lookup is the first step of Kubernetes service discovery, using specified DNS queries. This is most commonly implemented by installing CoreDNS with the kubernetes plugin in the cluster. On a cloud, clients might use the cloud provider’s DNS instead.

The “**I” specifications

Many of the middle ring specifications are named according to an “**I” convention, a three-letter acronym ending with “I” for “Interface”. These specify the interfaces for basic operations within the cluster:

• Container Networking interface (CNI)
• Container Storage Interface (CSI)
• Container Runtime Interface (CRI). Clusters often install the open-source (thus the “-O suffix) CRI-O implementation, which in turn relies upon the Open Container Interface (OCI) runtime specification. Alternatives that implement similar functionality but do not entirely match the CRI specification are containerd, Docker, or other runtimes.

Note that although the SMI specification for service meshes matches the above “**I” naming, as an optional feature it belongs in the next section as part of the outer ring.

The “**I” specifications are not necessarily the optimal way to implement their respective features. Many components offer a **I-compatible interface providing a subset of their features and an extended interface providing more control. The “**I” specifications codify a standardized common core of features, ensuring some degree of plug-and-play between offerings but not total compatibility.

Cluster ingress

Any real cluster will require an agent that provides external clients with controlled access. As Kubernetes has matured, it has offered increasingly flexible approaches to specifying such an agent and the operations it might perform on incoming traffic:

• The type LoadBalancer Service
• The Ingress resource and associated Ingress controller
• Service mesh ingress (service meshes are described in the outer ring below)

I distinguish these approaches in a later post but in terms of the three rings all forms of ingress are equivalent. They are all interfaces from the cluster to an agent, often provided by a cloud vendor, connecting external clients to services inside the cluster.

Note that this section describes the external interface, connecting the cluster to the external agent managing client requests. This is distinct from the YAML files describing the Service or Ingress, which comprise the internal interface, describing the external agent to the cluster and the services inside it.

Middle ring design patterns

The middle ring also adds an important design pattern, the operator pattern. This pattern extends the controller pattern to manage more demanding resources, such as services with extensive state or those requiring more elaborate administration, such as backups or log rotation. The distinction between a Controller and an Operator is qualitative, not hard and fast. I have placed the operator pattern in the middle ring because it is most typically used for the more complicated components in this ring, whereas the basic controller pattern suffices for the simpler components of the inner ring.

Implementations for components in the middle ring

The implementations for the above interfaces can be provided in any of several ways.

Vendors of managed Kubernetes services in the cloud typically provide default implementations of these interfaces. Their control loops are collected into a separate cloud-controller-manager, isolating the vendor-specific controllers from the core vendor-independent Kubernetes controllers in the kube-controller-manager. Some of these cloud-vendor-provided components can be overridden by the client but others may be required.

Packaged distributions of Kubernetes designed to be entirely administered by the user, such as OpenShift, microk8s, kind, and k3s, typically include implementations of all these components. The user can override any of these choices with a preferred alternative.

Finally, if the user wishes to define their own configuration, they must choose implementations for every interface in this middle ring. This is a lot of work but obviously offers the most control.

Purpose of the middle ring of abstractions

The middle ring of abstractions comprises all the components that are essential to a cluster but for which there are many possible valid implementations. For example, every cloud vendor has their own implementation of DNS and networking, optimized for high throughput in large datacentres. On the other hand, distributions of Kubernetes aimed at edge and IoT hardware, such as k3s, may instead opt to implement networking with an emphasis on low power consumption.

The Kubernetes approach to this middle layer provides stable, vendor-independent interfaces for the applications to access networking and other essential components while allowing the platform architect to pick the implementation most-suited to their use case. No single implementation could possibly suffice so instead Kubernetes standardizes the interface all implementations must satisfy.

Correspondence to operating systems

The middle ring of Kubernetes abstractions corresponds very roughly to the device drivers of an operating system. Both isolate the details of the environment from the algorithms controlling and responding to that environment. Beyond this, the two are very different. Where drivers are low level and primitive, the components of the Kubernetes middle ring are large, sophisticated systems with many subcomponents of their own. Many run in whole or part as pods managed by Kubernetes itself. They are more properly seen as wrapped around the Kubernetes core, rather than a primitive level beneath it.

Outer ring: Prevalent tools

The outermost ring contains entirely optional items. Optional in theory, at least. In practice, when an organization commits to “moving to Kubernetes”, to provide a genuinely functional application platform they must adopt versions of these services. There are no standards set by Kubernetes itself, but there are both standardized interfaces defined by industry consortia and de facto standards, products that have become so prevalent that their interfaces have been adopted by their competition.

In most organizations, these items are the responsibility of the platform team, as with the the other two rings.

Performance monitoring and observability

Both the operations staff and the development staff (who may overlap in some organizations) will need tools for monitoring the system state, monitoring security, and delving into the detailed state to resolve problems and defects. Detailed runtime records will also be useful for marketing and for planning service extensions.

Application performance monitoring (APM) and observability is a large, diverse field with ongoing controversies about best practice but the general structure is well-agreed. A cluster will need:

• Tools for gathering metrics, logs, traces, and other data in real time
• Tools for storing and querying that detailed data
• Tools for displaying aggregate data to the operations staff (“dashboards”)

Within this structure, there are widely-varying approaches. Many of the following only solve part of the general problem and so actual clusters will include some combination.

Vendor-based services

Cloud vendors offer solutions specific to their systems. These may not match the conventions supported by other vendors or industry standards, so multicloud systems will require integration of different data formats.

Independent cloud-based services

Many companies offer cloud-based services performing these functions. The cluster being monitored sends its data to the monitoring service, which processes the data and provides user interfaces for monitoring and querying it. Such services include Datadog, Splunk, Honeycomb, New Relic, Lightstep, and Glasnostics.

In-cluster tools

Platform architects may implement their own monitoring and observability services using open-source tools. Widely-used tools include:

• Prometheus for gathering metrics
• Grafana for displaying Prometheus metrics and other data
• ELK stack for collecting and querying logs
• Jaeger for tracing

This list is just a sample. Competing tools exist for most of these applications.

Standards

The OpenTelemetry consortium is developing a common standard for metrics, logs, and traces. This ambitious standard is designed to provide a common basis for all three streams, allowing a platform team to mix underlying libraries and provide a consistent API for their applications teams. If the standard becomes widely-adopted, it will greatly simplify the task of combining these streams.

Service meshes

As of early 2021, Service meshes appear to be widely discussed and have diverse competing implementations (there are ten meshes in the CNCF Landscape alone) but are not widely adopted in production. The adoption rate will surely increase in the next year or two.

Some cloud vendors provide branded versions (AWS App Mesh, Google Anthos Service Mesh, Alibaba Cloud Service Mesh), while others support multiple third-party meshes (supported meshes on Azure AKS). Regardless of the cloud vendor’s offerings, an organization can choose to install any mesh that they consider well-matched to their requirements. The mesh designs are compatible with any Kubernetes implementation.

Mesh vendors are collaborating on an interface standard, the Service Mesh Interface (SMI). As with the “**I” interfaces in the middle ring, the SMI supports a common subset of mesh features. If you wish to use all the features of your mesh, you will have to supplement SMI with vendor-specific requests. The SMI is under active development though, supporting an increasing range of common mesh features.

Messaging / queueing

The final widely-used component of Kubernetes clusters is a queuing or messaging system. The architectures for such a component are even more diverse than those for performance monitoring or meshes. An organization might well adopt several different architectures, sometimes even in the same cluster.

Open-source systems for messaging and queueing include Kafka, RedPanda, NATS, and RabbitMQ. Vendor-specific services include Amazon SQS, Google PubSub, Azure Service Bus, and Alibaba MNS. There are many more of each type; this list is just a sample.

The messaging/queuing component lies on the boundary between applications and platform. Application designers may choose an approach specific to their needs. Nonetheless, I place them in this ring because most often one or more queuing services are supported by the platform team for use by application designers, as a default if not a prescribed approach.

Purpose of the outer ring

The outer ring of abstractions is the most diverse, sharing the common thread of deep connectedness to applications. Metrics and traces record operations at the application level, requiring annotations by the application. Service meshes route traffic between application services, often rewriting application-specified routes. Messaging and queuing systems interconnect services and must be selected and tuned to meet application service level objectives. These abstractions of the outer ring form the most direct interface between the platform and applications running upon it.

Correspondence to operating systems

The outer ring of Kubernetes abstractions corresponds to the system-level services of an operating system such as systemd and syslog that run as processes atop the kernel. At this level, the notion of an “abstract machine” seems less useful and it makes more sense to consider these as services common to all applications and that are often best addressed with a single implementation supported by the systems staff rather than having every application implement its own version. The parallels between these operating system services and the services in the Kubernetes outer ring are straightforward.

How might we use this taxonomy?

For me, the first benefit of formulating this taxonomy is the way it orders the apparent chaos of the Kubernetes environment, which otherwise can appear as a slurry of words and acronyms with vague or indiscernible relationships. Assigning all these pieces to larger categories, with the categories themselves forming a broader whole, gives an immediate sense of each piece’s role. Knowing where something lies gives you clues about what it does.

Distinguishing the components consistently implemented by the Kubernetes control plane, such as Nodes and Pods, from the components that may vary with cloud provider, such as networking or DNS, locates the cluster within a cloud vendor’s documentation. Need to understand how your cluster’s applications are made externally visible? Read your cloud vendor’s documentation, because DNS and ingress are ultimately provided by the vendor.

This taxonomy also highlights why setting up a Kubernetes cluster is so much work. There are a lot of pieces, many of which are complex systems in their own right. The three rings clarify why so many organizations find value in the managed systems offered by cloud vendors and packaged “Kubernetes-in-a-bottle” distributions. Let someone else make the hard decisions of systems integration!

Describing Kubernetes as a “cluster operating system” doesn’t tell us enough. Viewing it in terms of its many abstractions is far more informative.