PostgreSQL on Kubernetes: CloudNativePG, Crunchy PGO and Zalando

conceptThe concept

Kubernetes restarts a crashed pod and reattaches its volume, which covers process failures. It knows nothing about replication, promotion, WAL archiving or major upgrades. An operator watches a custom resource that describes the cluster you want and keeps reconciling the real pods, volumes, services and PostgreSQL settings toward it.

The Percona Operator for PostgreSQL is built on the same design as Crunchy PGO, with Patroni and pgBackRest. Whichever you pick, the application reaches the primary through a Service that the operator repoints on failover: CloudNativePG creates <cluster>-rw, -ro (replicas only) and -r (any instance).

whyWhy it matters

Running PostgreSQL next to the applications that use it, with the same GitOps workflow, is attractive. But Kubernetes adds failure modes a VM does not have: a memory limit that triggers the OOM killer, node drains during cluster upgrades, volumes pinned to one zone, and noisy neighbours on shared nodes. The operator handles the database runbook; you still own the platform side.

Use caseReal-world scenario

A shop runs a three-instance CloudNativePG cluster with one pod per availability zone and quorum synchronous replication (any one standby must confirm). WAL is archived continuously to object storage with a nightly base backup, so point-in-time recovery is always possible. The application connects to a PgBouncer Pooler in front of shop-rw. Kubernetes version upgrades drain one node at a time; the operator switches the primary away before its node drains, so each upgrade costs a few seconds of reconnects.

Try itExamples

Run and edit these in the interactive lesson.

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: shop
spec:
  instances: 3
  imageName: ghcr.io/cloudnative-pg/postgresql:18
  storage:
    size: 100Gi
    storageClass: fast-ssd
  walStorage:
    size: 20Gi
  resources:
    requests: { memory: "8Gi", cpu: "2" }
    limits:   { memory: "8Gi", cpu: "2" }
  affinity:
    enablePodAntiAffinity: true
    topologyKey: topology.kubernetes.io/zone
  postgresql:
    parameters:
      shared_buffers: "2GB"
      max_connections: "200"
    synchronous:
      method: any
      number: 1
  plugins:
    - name: barman-cloud.cloudnative-pg.io
      isWALArchiver: true
      parameters:
        barmanObjectName: shop-backups

kubectl cnpg status shop                    # primary, replicas, lag, last archived WAL
kubectl get pods -l cnpg.io/cluster=shop -L cnpg.io/instanceRole
kubectl cnpg promote shop shop-2            # planned switchover to a chosen replica
kubectl cnpg backup shop                    # on-demand base backup
kubectl cnpg psql shop -- -c 'SELECT now() - pg_last_xact_replay_timestamp()'   # runs on the primary: NULL
kubectl cnpg fencing on shop shop-1         # stop PostgreSQL on one instance but keep the pod for inspection

# Crunchy PGO
apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
metadata: { name: shop }
spec:
  postgresVersion: 17
  instances:
    - name: pgha
      replicas: 3
      dataVolumeClaimSpec:
        accessModes: [ReadWriteOnce]
        resources: { requests: { storage: 100Gi } }
  backups:
    pgbackrest:
      repos:
        - name: repo1
          s3: { bucket: shop-backups, endpoint: s3.eu-west-1.amazonaws.com, region: eu-west-1 }
  proxy:
    pgBouncer: { replicas: 2 }
---
# Zalando (the cluster name must start with the teamId)
apiVersion: acid.zalan.do/v1
kind: postgresql
metadata: { name: acid-shop }
spec:
  teamId: acid
  numberOfInstances: 3
  postgresql: { version: "17" }
  volume: { size: 100Gi }
  enableConnectionPooler: true

SELECT name, setting, unit
FROM pg_settings
WHERE name IN ('shared_buffers', 'effective_cache_size', 'work_mem', 'maintenance_work_mem', 'max_connections', 'huge_pages')
ORDER BY name;

internalsUnder the hood

Memory. Set memory requests equal to limits (Guaranteed QoS) and size from the limit, not from the node: shared_buffers around 25% of the limit, and leave room for work_mem per sort, maintenance work and the page cache that counts against the limit. If the kernel OOM killer kills one backend, the postmaster restarts every connection and runs crash recovery.

Storage. Use block storage with predictable latency; put WAL on its own volume if the operator supports it. A ReadWriteOnce volume is usually bound to one zone, so a pod can only be rescheduled inside that zone; failover to a replica in another zone is what keeps you available when a zone fails.

Disruptions. Operators create PodDisruptionBudgets and handle node drains by switching over first. A planned switchover is clean; a forced delete of the primary pod is a failover with possible loss of unreplicated WAL unless replication is synchronous.

Avoid / preferDo this, not that

resources:
  limits: { memory: "8Gi" }
postgresql:
  parameters:
    shared_buffers: "6GB"

resources:
  requests: { memory: "8Gi" }
  limits:   { memory: "8Gi" }
postgresql:
  parameters:
    shared_buffers: "2GB"

kind: StatefulSet
spec:
  replicas: 3   # "high availability"
  template: { spec: { containers: [{ name: pg, image: postgres:18 }] } }

kind: Cluster          # CloudNativePG, or PostgresCluster / postgresql
spec:
  instances: 3

PitfallsCommon mistakes

• Scaling a plain StatefulSet to three replicas and calling it HA. Each pod gets its own empty database; nothing replicates and nothing fails over.
• Leaving memory limits far above requests. Under node pressure the pod is among the first to be evicted or OOM-killed.
• Putting all instances in one zone because the default scheduler happened to, then losing the whole cluster with that zone.
• Deleting the primary pod to "restart" it instead of asking the operator for a switchover or restart.

LimitsWhen not to use it

If the team does not already operate Kubernetes well, a production database is a poor first workload: learn the platform with stateless services first. A single very large database with heavy IO may also be simpler and cheaper on dedicated VMs with Patroni, where storage latency and memory are not shared with other workloads.

SecuritySecurity notes

Operators run with wide RBAC rights in the cluster; install them from the project's signed releases and restrict who can edit the custom resources, because editing one can promote, restore or delete a database. Keep superuser access disabled for applications (CloudNativePG disables it by default), use TLS between pods (also on by default in CloudNativePG), add NetworkPolicies so only the application namespace can reach port 5432, and give backup jobs bucket access through workload identity rather than long-lived keys in Secrets.

ChallengeTry it yourself

WITH pod AS (SELECT 8 * 1024 AS limit_mb, 200 AS max_conn)
SELECT limit_mb / 4 AS shared_buffers_mb,
       limit_mb * 3 / 4 AS effective_cache_size_mb,
       (limit_mb - limit_mb / 4) / (max_conn * 3) AS work_mem_mb
FROM pod;

InterviewQuestions you should be able to answer

QuizCheck your understanding

5 questions with instant feedback in the interactive lesson; a wrong answer goes into your review deck.

Go deeperOfficial documentation

• Failover
• Patroni documentation
• CloudNativePG