Log Collection Failed
Error message
Error: Fluentd failed to send logs to Elasticsearch
Quick summary
The log collector (Fluentd, Filebeat) cannot forward logs to the storage backend, resulting in missing or delayed logs.
Why this happens
The log storage backend (Elasticsearch, Loki) is unavailable
The log collector is not running or is misconfigured
Network issues prevent the collector from reaching the log store
The log store is full or has insufficient disk space
Minimal example
# Fluentd cannot send to Elasticsearch
# Error: connection refused to elasticsearch:9200# Check and restart Elasticsearch
kubectl get pods -l app=elasticsearch
kubectl rollout restart deployment/elasticsearch
# Then restart Fluentd
kubectl rollout restart daemonset/fluentdElasticsearch is down, so Fluentd cannot send logs. The fix restarts Elasticsearch and then Fluentd.
How to diagnose
Check if the log collector is running:
kubectl get pods -l app=fluentdView collector logs for errors:
kubectl logs <fluentd-pod>Verify the log store is running and reachable
Check disk space on the log storage backend
How to fix
Restart the log collector:
kubectl rollout restart daemonset/fluentdFix the log store connection (restart Elasticsearch, fix network)
Correct the collector configuration (URL, credentials, index)
Free up disk space on the log store or increase retention rotation
How to prevent
Monitor log collection health and alert on failures
Use buffer/queue in the collector to handle log store outages
Set up log retention policies to prevent disk full issues