In our previous article, we highlighted how Kubernetes could be used as a declarative control system. In this part, we’re focusing on Crossplane, a tool that I find very useful in this context.

When we run applications and services supporting our business goals, we must provision infrastructure such as databases, networks, Cloud accounts, clusters, serverless functions and load balancers. There are a few options for achieving what we need:

1. Manually create infrastructure using Cloud vendor web portal

2. Manually create infrastructure using Cloud vendor CLI tool

3. Automate the creation of infrastructure from code using Cloud vendor API’s

For other than the tiniest projects or testing, the first two options are only feasible in the short term. Manually created infrastructure cannot be versioned, controlled sufficiently, tested, and maintained properly.

There are quite a few tools for automating infrastructure provisioning from code (commonly called IaC or "Infrastructure as Code"). For this article, I will look into Crossplane and draw a comparison against the widely used IaC tool, Hashicorp's Terraform.

Crossplane is, according to their own material, a framework for building cloud-native control planes without needing to write code. It has an extensible backend to manage any infrastructure in any environment, and a configurable frontend to expose declarative APIs for developer self-service.

The concept of a control plane sets Crossplane apart from other traditional IaC tools. The control plane is always running and detecting drift in our infrastructure, applies all changes in real time, and allows us to interact with it using an API.

Assuming we already have a cluster running, e.g. in Docker Desktop, we can install Crossplane:

# install crossplane kubectl create namespace crossplane-system helm repo add crossplane-stable https://charts.crossplane.io/stable helm repo update helm install crossplane --namespace crossplane-system crossplane-stable/crossplane # install kubectl plugin curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | sh

For this demo, we're going to set an example configuration package with GCP:

kubectl crossplane install configuration registry.upbound.io/xp/getting-started-with-gcp:v1.10.1 # wait to be ready watch kubectl get pkg

We can see that the configuration package pulled provider-gcp, and we now have a new provider pod running in crossplane-system namespace. Providers are an important concept in Crossplane: they bundle a set of managed resources and their respective controllers to allow Crossplane to provision a particular class of managed resources, in this case resources on top of Google Cloud Platform. Managed resources could be any services or resources that can be provisioned and configured using some form of APIs. You can create your own providers using Golang, but common use cases are already covered by contributed providers.

For the next step, we need to get the GCP account key, and install that on the cluster as a secret and configure GCP provider for Crossplane. I’m not going to detail how to do that here, but you can follow official instructions.

Now with everything set up, we can try to do something. Let’s try to create a storage bucket. We can submit the following Kubernetes resource on our cluster:

apiVersion: storage.gcp.crossplane.io/v1alpha3 kind: Bucket metadata: name: bucket-for-demo namespace: default labels: example: "true" annotations: # Note that this will be the actual bucket name so it has to be globally unique/available. crossplane.io/external-name: bucket-for-demo-gdfg34g3 spec: location: EU storageClass: MULTI_REGIONAL providerConfigRef: # this needs to match with provider name created earlier name: default deletionPolicy: Delete

After some time, we can observe in the console our bucket got created:

kubectl get bucket NAME READY SYNCED STORAGE_CLASS LOCATION AGE bucket-for-demo True True MULTI_REGIONAL EU 102s

While this was an simplified example, there are a few fundamental points we can take from this:

• We created the bucket from within the Kubernetes namespace using a simple and understandable YAML definition. Imagine we are deploying, let's say, WordPress using a Helm chart: this would allow us to create associated storage buckets directly from the chart template using just Kubernetes definitions and connect it to the application.

• On the client side (our laptop), we did not need to run any custom tools. We could get things done by submitting Kubernetes manifests to the Kubernetes API. Even the Crossplane kubectl plugin is optional. Everything resides persistently on the server side.

• As Crossplane definitions are Kubernetes manifests, we can plug this into any Kubernetes deployment tooling we prefer, such as Argo-CD, Flux, or plain old kubectl. We can also use any standard templating tooling such as Helm, Kustomize, or Jsonnet.

• Definitions themselves are quite verbose but straightforward. There is no custom domain-specific language in Crossplane, unlike in Terraform. When advanced generation from code is desired we're free to choose tooling ourselves, one good option being

  https://cdk8s.io/

• We can apply our choice of policy controller to Crossplane manifests, such as Kyverno, OPA Gatekeeper, or Datree. With these and Kubernetes RBAC it is possible to tell the system which actors are allowed to deploy a certain class of managed resources (new clusters for example), and if their configuration matches what is permitted. We can easily apply business rules, such as “all storage buckets must be located; due to GDPR, within EU”

• Because we are dealing with Kubernetes entities and API, it is relatively straightforward to answer questions like "how many databases do we have and who owns them, and what are their versions". All entities can have metadata associated with them. Crossplane entities have standard status field that gives out the state of the entity.

  • We could potentially integrate other tooling like Panopticon to expose this data as Prometheus metrics, and collect data about resources that exists, and generate to reports for business needs.

With providers and managed resources we learned how Crossplane could deploy Cloud resources. There’s another compelling concept in Crossplane: compositions. With them, Platform Builders can create abstractions that users can consume. We use compositions to hide specific detail and complexity our business and users don’t care, and create higher-level abstractions, like an entity representing entire “WordPress” instance, that, in reality, is composed of managed MySQL database, a storage bucket, and a Helm chart, but only exposes what users should be able to modify in it.

We’re going into detail of compositions in the next series of this article, but it is important to note they are major feature and benefit of Crossplane.

These tools are quite different, while they can do same things. It is worth noting Crossplane has Terraform provider, so everything Terraform can do, Crossplane can also do.

One major drawback of Crossplane is that it requires a running Kubernetes cluster, to which its controllers are installed. Of course, Terraform also needs to be executed from somewhere, but that is often a simpler setup in CI pipeline, or even developer’s own machine. With Crossplane how does the “management cluster”, or the first cluster get bootstrapped? Should it be hand-crafted or created using another IaC tool? There is also the option for managed control plane from Upbound, a commercial part of the Crossplane ecosystem.

Use Crossplane when you:

• Already use Kubernetes, and prefer API’s to control and view state of infrastructure

• Need to build self-serviceable abstractions for developers to consume

• Don’t like enforced custom DSLs

Use Terraform when:

• Don’t have or are not familiar with Kubernetes at all

• You, or central Ops team, creates all infrastructure, and you don’t have a need for self-serviceable abstractions

• Prefer once-off workflows to manage infrastructure instead of a continuously running control plane