Promote changes across environments

monedula-gitops has no built-in environment concept: -f and --cluster-config-file are paths, so promoting a change from development through staging to production is a matter of how you lay out the files in Git.

Before you start

• The CLI, and one cluster config file for each environment’s cluster.
• For the overlay layout, kubectl, whose kubectl kustomize renders the overlays, or the standalone kustomize build.

Steps

1. Keep one cluster config per environment

Keep a cluster config per environment next to one manifest tree:

env/
  dev/cluster.yaml
  staging/cluster.yaml
  prod/cluster.yaml
manifests/
  topics/orders.yaml
  access/checkout.yaml

Each cluster.yaml holds that environment’s bootstrap servers and credentials. Because every environment loads the same manifests, give the KafkaCluster in each file the same metadata.name, the name your manifests use in clusterRef.name. A clusterRef that names no cluster in the loaded config fails validation.

2. Promote shared manifests

To promote, apply the same manifests with the next environment’s cluster config. The manifests sit in subdirectories, so add -R to read them recursively; without it, -f ./manifests loads no manifests at all:

monedula-gitops apply -f ./manifests -R --cluster-config-file env/staging/cluster.yaml
monedula-gitops apply -f ./manifests -R --cluster-config-file env/prod/cluster.yaml

This works as long as topic settings, such as partitions and retention, are meant to be identical in every environment. As soon as production needs more partitions or a longer retention than development, render the manifests per environment, as in step 3.

3. Override values per environment with Kustomize

Keep a base manifest and a patch per environment:

base/
  kustomization.yaml
  orders.yaml               # partitions: 3, retention.ms: "86400000"
overlays/
  prod/
    kustomization.yaml
    orders-patch.yaml        # partitions: 12, retention.ms: "604800000"
env/
  prod/cluster.yaml

base/orders.yaml is the KafkaTopic orders with the development values, and base/kustomization.yaml lists it as a resource. The production overlay, overlays/prod/kustomization.yaml, builds on the base and applies a patch to that topic:

resources:
  - ../../base
patches:
  - path: orders-patch.yaml
    target:
      kind: KafkaTopic
      name: orders

The patch, overlays/prod/orders-patch.yaml, is a strategic-merge patch that overrides only the fields that differ in production:

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaTopic
metadata:
  name: orders
spec:
  partitions: 12
  config:
    retention.ms: "604800000"

Render the overlay and pipe it into apply:

kubectl kustomize overlays/prod | monedula-gitops apply -f - --cluster-config-file env/prod/cluster.yaml

-f - reads the manifests from stdin. Any renderer that prints manifests, such as helm template or a script, works the same way, and so do diff and verify.

4. Promote through review

Run each environment’s preview before its apply, for example in the pull request that promotes the change. Validate and preview changes in CI shows the commands; with an overlay, pipe the rendered manifests into diff the same way as into apply.

Check it worked

Run verify per environment, with the same rendering and the environment’s own cluster config, here with an overlays/staging built the same way as overlays/prod:

kubectl kustomize overlays/staging | monedula-gitops verify -f - --cluster-config-file env/staging/cluster.yaml
kubectl kustomize overlays/prod    | monedula-gitops verify -f - --cluster-config-file env/prod/cluster.yaml

Each command exits 0 when its environment matches the rendered manifests. With shared manifests, run verify -f ./manifests -R with each environment’s cluster config instead. The overlays differ on purpose, so gate every environment on its own verify.

Related

• The CLI overview explains manifest input from files, directories and stdin.
• Detect and fix drift runs verify on a schedule.