Kubernetes Autoscaling in Production: HPA, KEDA, and Karpenter

Autoscaling looks simple in a demo and gets subtle fast in production. Three controllers operate on different axes, and the failure modes only show up under real traffic. This is the mental model we use to keep clusters both cheap and reliable.

The three axes

• HPA, scales pod replicas on CPU, memory, or custom metrics.

• VPA, right-sizes CPU/memory requests for a workload over time.

• Cluster Autoscaler / Karpenter, adds and removes nodes to fit pending pods.

The classic mistake is running HPA and VPA on the same resource metric, they fight. Use VPA in recommendation mode alongside HPA, or split them by metric.

HPA on a custom metric

CPU is a lagging signal for most services. Scale on the thing that actually predicts saturation, here, requests-per-second from Prometheus Adapter:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
 name: checkout
spec:
 scaleTargetRef:
 apiVersion: apps/v1
 kind: Deployment
 name: checkout
 minReplicas: 3
 maxReplicas: 40
 metrics:
 - type: Pods
 pods:
 metric: { name: http_requests_per_second }
 target: { type: AverageValue, averageValue: "50" }
 behavior:
 scaleDown:
 stabilizationWindowSeconds: 300 # avoid flapping
 policies: [{ type: Percent, value: 10, periodSeconds: 60 }]

The behavior block is the part teams skip and then page themselves over. A 5-minute scale-down stabilization window turns a thundering herd into a smooth curve.

Event-driven scaling with KEDA

For queue workers, replica-per-RPS is the wrong unit. Scale on queue depth instead, and let KEDA scale to zero when idle:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
 name: image-worker
spec:
 scaleTargetRef:
 name: image-worker
 minReplicaCount: 0
 maxReplicaCount: 100
 cooldownPeriod: 120
 triggers:
 - type: aws-sqs-queue
 metadata:
 queueURL: https://sqs.eu-west-1.amazonaws.com/1234/jobs
 queueLength: "20" # target messages per replica

Nodes: Karpenter over static node groups

Karpenter provisions right-sized nodes directly from pending-pod shape, consolidates underused nodes, and mixes spot with on-demand. Bin-packing plus consolidation is where most of the real savings live.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata: { name: default }
spec:
 disruption:
 consolidationPolicy: WhenEmptyOrUnderutilized
 consolidateAfter: 30s
 template:
 spec:
 requirements:
 - key: karpenter.sh/capacity-type
 operator: In
 values: ["spot", "on-demand"]

Pitfalls we have hit

1. No PodDisruptionBudget, so consolidation evicts your last healthy replica

2. Readiness probes too optimistic, so HPA scales into pods that are not ready

3. Requests set to zero, so the scheduler and autoscaler are both flying blind

4. Scaling on averages during bursty traffic, use max or a percentile

Autoscaling does not fix an unschedulable workload. It just adds nodes faster than your budget can complain.

Rule of thumb: scale replicas on a leading demand signal, scale nodes with Karpenter, right-size with VPA recommendations, and always pair it with a PDB. Cheap and reliable is a choice you configure.