Deployment rollback: new version fails health checks
Quick summary
Your new deployment failed health checks — the orchestrator rolled back to the previous working version.
Why this happens
The new version has a bug that causes the health check endpoint to fail
The new version requires a database migration that was not run before deployment
Configuration or environment variables for the new version are missing or incorrect
Minimal example
# kubernetes deployment — new version fails health check
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: app
image: myapp:v2 # has a bug — health check fails
livenessProbe:
httpGet:
path: /health
port: 3000# Fix the bug in v2, rebuild, and redeploy
apiVersion: apps/v1
kind: Deployment
spec:
template:
spec:
containers:
- name: app
image: myapp:v2.1 # bug fixed — health check passes
livenessProbe:
httpGet:
path: /health
port: 3000Fixing the bug in the new version and rebuilding the image makes the health check pass, preventing rollback.
How to diagnose
Check the deployment logs for health check failures and error messages
Verify that all required migrations and configuration are in place before deployment
Test the new version in a staging environment with the same configuration as production
How to fix
Fix the bug in the new version that causes health check failures
Run required database migrations before deploying the new version
Add missing configuration or environment variables for the new version
How to prevent
Deploy to staging first and run the same health checks before promoting to production
Use progressive deployment strategies like canary or blue-green to catch issues early
Related Resources
Related Glossary
Related Lessons
- Deployment
Learn about rollbacks.