Workflow

Step 1: Define the Deployment

Declare the desired state: image, replicas, resource requests/limits, environment variables, and rolling update strategy.

• Pin images to immutable references (use image@sha256:... digests) rather than mutable tags.
• Do not use the tag latest in production manifests.

Step 2: Configure health probes

Add livenessProbe (restart on failure), readinessProbe (remove from service until ready), and optionally startupProbe (slow-starting apps).

Step 3: Create the Service

Expose the Deployment inside the cluster with a ClusterIP Service. Use NodePort or LoadBalancer only when external access is needed without an Ingress controller.

Step 4: Add Ingress for HTTP routing

Map hostnames and paths to Services. Configure TLS termination with cert-manager or a cloud load balancer.

Step 5: Tune resources and autoscaling

Set CPU/memory requests and limits. Add an HPA to scale on utilization thresholds. Add a PodDisruptionBudget for safe rollouts.

• Use autoscaling/v2 (stable) for the HorizontalPodAutoscaler — older v2beta versions were removed in recent Kubernetes releases. Ensure manifests use autoscaling/v2 and the metrics fields expected by that API.

Step 6: Isolate with namespaces and network policies

Create a namespace per environment or team. Apply NetworkPolicies to restrict traffic to only what is explicitly allowed.

Security & supply-chain (new)

Recent supply-chain incidents (e.g., compromised scanner images) make image provenance and runtime admission controls mandatory in production clusters.

• Pin by digest and verify image provenance: always reference images by digest (image@sha256:...) in manifests and CI pipelines so the runtime pulls a known-good image. This prevents silent changes when a tag is repointed.
• Rotate credentials if a compromised image was pulled into CI or build infrastructure. If third-party images are compromised, rotate affected secrets and credentials immediately.
  • Example incident: the Mar 19–23, 2026 compromise of specific aquasec/trivy images — Docker recommends removing affected images and rotating credentials when compromised digests were pulled.
• Require signed images and enforce at admission time: use sigstore/cosign and the Sigstore Policy Controller (policy-controller) as a Kubernetes admission controller to require valid signatures/attestations before admission.
  • Policy Controller can validate cosign signatures and attestations, resolve tags to digests at admission, and enforce per-namespace policies (opt-in by namespace label or cluster-wide).
• Produce and store SBOMs (software bill of materials) and attestation metadata for your builds. Use them as part of admission policies and incident response.
• Enforce runtime image allowlists/deny-lists with admission controllers (Sigstore policy-controller, OPA Gatekeeper, Kyverno) and use image scanning in CI, but do not rely on scanner images pulled at runtime without verification.

Sources: Docker advisory on Trivy compromise (Mar 23, 2026) and Sigstore policy-controller docs.

Examples

Example 1: Production Deployment with probes and security

apiVersion: apps/v1 kind: Deployment metadata: name: web-app namespace: production labels: app: web-app spec: replicas: 3 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 selector: matchLabels: app: web-app template: metadata: labels: app: web-app spec: terminationGracePeriodSeconds: 30 securityContext: runAsNonRoot: true runAsUser: 1001 seccompProfile: type: RuntimeDefault containers: - name: web-app # Use immutable digest references for production images image: ghcr.io/org/web-app@sha256:abcde12345f... ports: - name: http containerPort: 3000 env: - name: DATABASE_URL valueFrom: secretKeyRef: name: web-app-secrets key: database-url resources: requests: cpu: 250m memory: 256Mi limits: cpu: "1" memory: 512Mi livenessProbe: httpGet: path: /api/health port: http initialDelaySeconds: 10 periodSeconds: 15 failureThreshold: 3 readinessProbe: httpGet: path: /api/health/ready port: http initialDelaySeconds: 5 periodSeconds: 10 startupProbe: httpGet: path: /api/health port: http periodSeconds: 5 failureThreshold: 12 lifecycle: preStop: exec: command: ["/bin/sh", "-c", "sleep 5"]

Example 2: Service + Ingress with TLS

apiVersion: v1 kind: Service metadata: name: web-app namespace: production spec: type: ClusterIP selector: app: web-app ports: - name: http port: 80 targetPort: http protocol: TCP

apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: web-app namespace: production annotations: cert-manager.io/cluster-issuer: letsencrypt-prod nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/proxy-body-size: "10m" spec: ingressClassName: nginx tls: - hosts: - app.example.com secretName: web-app-tls rules: - host: app.example.com http: paths: - path: / pathType: Prefix backend: service: name: web-app port: name: http

Example 3: HPA + PodDisruptionBudget

apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: web-app namespace: production spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: web-app minReplicas: 3 maxReplicas: 20 behavior: scaleDown: stabilizationWindowSeconds: 300 policies: - type: Percent value: 25 periodSeconds: 60 scaleUp: stabilizationWindowSeconds: 60 policies: - type: Percent value: 50 periodSeconds: 60 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80

apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: web-app namespace: production spec: minAvailable: 2 selector: matchLabels: app: web-app

Decision tree

• Single container, no scaling → Deployment with 1 replica + ClusterIP Service
• HTTP traffic from the internet → Add Ingress with TLS; use cert-manager for automatic certs
• Need auto-scaling → Add HPA targeting 70% CPU; set minReplicas ≥ 2 for availability
• Slow-starting app → Add startupProbe with generous failureThreshold × periodSeconds window
• Multi-tenant cluster → Separate namespaces + NetworkPolicies + ResourceQuotas
• Stateful workload → Use StatefulSet instead of Deployment; attach PersistentVolumeClaims
• Background workers → Use Deployment without a Service; consider Jobs or CronJobs for batch work
• Rolling updates must not drop traffic → Set maxUnavailable: 0 + preStop sleep + readinessProbe

Edge cases and gotchas

1. Requests vs limits confusion — Requests guarantee scheduling; limits cap burst. Setting limits too low causes OOMKill. Setting requests too high wastes cluster resources. Start with requests = p95 observed usage, limits = 2× requests.
2. Missing readinessProbe — Without it, Kubernetes sends traffic to Pods before they are ready, causing 502 errors during deployments. Always define a readinessProbe.
3. preStop hook for zero-downtime deploys — When a Pod terminates, the endpoint removal and SIGTERM race. A sleep 5 preStop hook gives the load balancer time to drain connections.
4. Image tag latest in production — Kubernetes caches images by tag. latest can silently serve stale code. Always use immutable tags (SHA or semver). Prefer digests in production manifests.
5. HPA + manual replica count conflict — If you set replicas: 3 in the Deployment and also use HPA, the HPA controls replicas. Remove replicas from the Deployment spec or use Kustomize to strip it.
6. ConfigMap/Secret updates don't restart Pods — Changes to a ConfigMap or Secret are not automatically detected. Use a hash annotation on the Pod template or a reloader controller.
7. Namespace-scoped vs cluster-scoped resources — Ingress, Services, Deployments are namespaced. ClusterRole, ClusterRoleBinding, IngressClass are cluster-scoped. Getting this wrong causes not found errors.
8. ResourceQuota blocking deployments — If a namespace has a ResourceQuota and your Pod spec lacks resource requests, admission is denied. Always set requests and limits when quotas are enabled.
9. APIs are removed over time — check the deprecation guide — Kubernetes has been removing older API versions across releases (examples: cronjob batch/v1beta1 removed in v1.25; autoscaling/v2beta2 removed in v1.26; newer removals are tracked in the upstream deprecation guide). Always validate manifests against the cluster's served APIs before upgrades and use the Kubernetes deprecation guide to plan migrations.

Evaluation criteria

• Deployment has resource requests and limits on every container
• livenessProbe, readinessProbe, and (if needed) startupProbe are configured
• Pods run as non-root with a securityContext
• Images are pinned with immutable references (SHA digest or semver); never latest in production
• Ingress has TLS configured with cert-manager or equivalent
• HPA is present for user-facing workloads with minReplicas ≥ 2
• PodDisruptionBudget ensures availability during node drains
• Secrets are mounted from Secret resources, not hardcoded in YAML
• Image provenance is enforced (signed images or admission policies such as Sigstore policy-controller, OPA/Gatekeeper, or Kyverno)

Notes on runtimes and compatibility

• containerd maintains a release cadence aligned with Kubernetes releases; check the containerd release docs for supported versions and timelines. Since containerd v2.3 (April 2026) the project publishes minor releases on a 4-month cadence to stay synchronized with Kubernetes. Keep runtime versions and CRI integration in mind during cluster upgrades.

Additional resources

• Kubernetes Deprecated API Migration Guide — authoritative list of removed APIs and migration notes
• Docker advisory: Trivy supply chain compromise (Mar 23, 2026) — follow remediation steps if affected
• Sigstore Policy Controller docs — admission controller to enforce image signature/attestation policies