Skip to content

Deployment rollback: new version fails health checks

Quick summary

Your new deployment failed health checks — the orchestrator rolled back to the previous working version.

Why this happens

  • The new version has a bug that causes the health check endpoint to fail

  • The new version requires a database migration that was not run before deployment

  • Configuration or environment variables for the new version are missing or incorrect

Minimal example

✗ Broken code
# kubernetes deployment — new version fails health check
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: app
          image: myapp:v2 # has a bug — health check fails
          livenessProbe:
            httpGet:
              path: /health
              port: 3000
✓ Fixed code
# Fix the bug in v2, rebuild, and redeploy
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: app
          image: myapp:v2.1 # bug fixed — health check passes
          livenessProbe:
            httpGet:
              path: /health
              port: 3000

Fixing the bug in the new version and rebuilding the image makes the health check pass, preventing rollback.

How to diagnose

  • Check the deployment logs for health check failures and error messages

  • Verify that all required migrations and configuration are in place before deployment

  • Test the new version in a staging environment with the same configuration as production

How to fix

  • Fix the bug in the new version that causes health check failures

  • Run required database migrations before deploying the new version

  • Add missing configuration or environment variables for the new version

How to prevent

  • Deploy to staging first and run the same health checks before promoting to production

  • Use progressive deployment strategies like canary or blue-green to catch issues early

Related Resources

Related Glossary

Related Lessons

← Back to language