Kustomize is a command line tool that lets you customize application configuration in a template free way.
In Kubernetes world the declarative way is the recommended approach to create the resources. However, it is difficult to use only
to follow the declarative way, another tools are required like, like Helm, Kapitan, ktmpl.
The full list of these tools you can find here
The drawback of these tools is that you have to learn new complicated DSLs, they use templating which can only override parameterized config, they provide multiple features like package or dependency management, come with dashboards which live in your cluster, they allow to manage the lifecycle of specific version when to rollback specific version, and come with customization features too, when you deploy to different environments
Kustomize instead is focusing only on the
customization domain and allows you to tailor your YAML files to a specific environment.
It is using
overlay approach and exposes and teaches the native k8s APIs, not trying to hide them. This makes sure, that the user gets a deeper understanding about Kubernetes.
Kustomize is available as standalone executable (
kustomize) and since 1.14 is part of
kubectl using the
kubectl apply with
In this blog post we will use the standalone executable, which can be easily installed following the guide specific to your platform.
For Mac users is straightforward:
$ brew install kustomize
In this article we assume you have already a
Docker environment and
kubectl is already installed. To create a Kubernetes cluster we use kind:
$ kind create cluster --config=kind-cluster-config.yaml
kind-cluster-config.yaml specifies a 3 node cluster.
# three node (two workers) cluster config kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 name: my-cluster nodes: - role: control-plane - role: worker - role: worker
The command also creates a
context and switches to it.
Kind is a great tool to run multiple-node Kubernetes clusters locally. With
kind get clusters we can query the created clusters.
docker ps you can view the Kubernetes nodes
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 067290147e33 kindest/node:v1.19.1 "/usr/local/bin/entr…" 4 days ago Up 31 minutes my-cluster-worker c1ab152ccf6e kindest/node:v1.19.1 "/usr/local/bin/entr…" 4 days ago Up 31 minutes 127.0.0.1:61807->6443/tcp my-cluster-control-plane 9d64cefcd1e7 kindest/node:v1.19.1 "/usr/local/bin/entr…" 4 days ago Up 31 minutes my-cluster-worker2
Next, we are going to use the kustomize-demo application which is very simple Spring Boot application in order to showcase
After cloning the repository and building the example we have a docker image
$ git clone https://github.com/altfatterz/kustomize-demo $ cd kustomize-demo $ mvn clean package
Next we need to load the created Docker image into our Kubernetes cluster.
$ kind load docker-image kustomize-demo:0.0.1-SNAPSHOT --name my-cluster [email protected]:~/projects/demos/kustomize-demo|master⚡ ⇒ kind load docker-image kustomize-demo:0.0.1-SNAPSHOT --name my-cluster Image: "kustomize-demo:0.0.1-SNAPSHOT" with ID "sha256:adb9bc431439dea21dc31155587da79183bda9f8636d80127d08d0dbdce6d4c7" not yet present on node "my-cluster-worker", loading... Image: "kustomize-demo:0.0.1-SNAPSHOT" with ID "sha256:adb9bc431439dea21dc31155587da79183bda9f8636d80127d08d0dbdce6d4c7" not yet present on node "my-cluster-control-plane", loading... Image: "kustomize-demo:0.0.1-SNAPSHOT" with ID "sha256:adb9bc431439dea21dc31155587da79183bda9f8636d80127d08d0dbdce6d4c7" not yet present on node "my-cluster-worker2", loading...
We can verify that the image is present on Kubernetes nodes using
$ docker exec -it 067290147e33 crictl images | grep kustomize-demo docker.io/library/kustomize-demo 0.0.1-SNAPSHOT adb9bc431439d 259MB
Finally, we create two namespaces where we will deploy the application with different configurations:
$ kubectl create namespace dev $ kubectl create namespace prod
kustomize-demo/ops folder we can find the traditional way of declarative Kubernetes configuration duplicating the resource definitions for
Kustomize allows to specify the resource definitions without duplicating common elements. It does this Kubernetes way, using to use Custom Resource Definitions (CRDs) to configure the differences, rather than variable-replacement.
We move the common yaml configuration into a
base directory and create two
prod environments using the following directory structure:
├── base │ ├── deployment.yaml │ └── service.yaml └── overlays ├── dev │ └── kustomization.yaml └── prod └── kustomization.yaml
dev/kustomization.yaml content is
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ../../base/deployment.yaml - ../../base/service.yaml namespace: dev
In order to see the yaml generated for the
dev environment we can use:
$ kustomize build --load_restrictor none k8s/overlays/dev
--load_restrictor flag set to
none, allows that customizations may load files from outside their root.
To create the resource in the dev namespace then we can pipe the output of the
kustomize command to the
$ kustomize build --load_restrictor none k8s/overlays/dev | kubectl apply -f - service/kustomize-demo-service created deployment.apps/kustomize-demo-deployment created
To delete the resources we can use:
$ kustomize build --load_restrictor none k8s/overlays/dev | kubectl delete -f -
In production environment is good approach to add a prefix like
prod- to all product resource names in order to avoid
modifying or deleting these resources by mistake. With
Kustomize we can easily achieve this adding the following snippet
prod/kustomization.yaml configuration file
We want also that resources in production environment to have certain labels so that we can query them by label selector:
commonLabels: env: prod
We also want to increase the replicas count in production environment (
apiVersion: apps/v1 kind: Deployment metadata: name: kustomize-demo spec: replicas: 3
Another important configuration in production is to set the resource constraints (
apiVersion: apps/v1 kind: Deployment metadata: name: kustomize-demo spec: template: spec: containers: - name: kustomize-demo resources: requests: memory: 512Mi cpu: 256m limits: memory: 1Gi cpu: 512m
and health check configuration (
apiVersion: apps/v1 kind: Deployment metadata: name: kustomize-demo spec: template: spec: containers: - name: kustomize-demo readinessProbe: httpGet: path: /actuator/health/readiness port: 8080 initialDelaySeconds: 5 periodSeconds: 5 livenessProbe: httpGet: path: /actuator/health/liveness port: 8080 initialDelaySeconds: 5 periodSeconds: 5
We need to reference all these patches in
prod/kustomization.yaml under the
patchesStrategicMerge: - resource_constraints_patch.yaml - environment_patch.yaml - health_check_patch.yaml - increase_replicas_patch.yaml
To create the resources in
prod environment we use
$ kustomize build --load_restrictor none k8s/overlays/prod | kubectl apply -f -
Indeed, the pods where created
$ kubectl get pods -n prod NAME READY STATUS RESTARTS AGE prod-kustomize-demo-548ccc8db9-dtc52 1/1 Running 0 25s prod-kustomize-demo-548ccc8db9-m9xzb 1/1 Running 0 25s prod-kustomize-demo-548ccc8db9-qjhxw 1/1 Running 0 25s
The source code for this blog post can be found here https://github.com/altfatterz/kustomize-demo