Centralized Logging
Definition
Collecting and storing log data from multiple applications and systems in one central location for easier searching, monitoring, and analysis.
Use Cases
- Netflix: Troubleshooting and monitoring a large microservices environment by correlating application and infrastructure logs during incidents. — Netflix aggregates logs from services and cloud infrastructure into centralized logging pipelines and searchable stores, using standardized log formats and correlation identifiers to trace requests across services. (Faster incident investigation and improved operational visibility across many distributed services, reducing time to identify root causes.)
- Spotify: Central visibility into service health and errors across many teams and microservices to support on-call operations. — Spotify centralizes logs from services into shared logging backends and dashboards, enabling engineers to search across services and correlate logs with metrics during debugging. (More efficient on-call workflows and quicker debugging through unified search and shared operational context.)
Provider Equivalents
- AWS: Amazon CloudWatch Logs
- Azure: Azure Monitor Logs (Log Analytics workspace)
- GCP: Cloud Logging
- OCI: OCI Logging
Frequently Asked Questions
- What's the difference between centralized logging and distributed tracing?
- Centralized logging collects text/event records (like errors, warnings, and audit events) from many systems into one place so you can search and analyze them. Distributed tracing follows a single request as it moves through multiple services and records timing and dependencies (spans/trace IDs). They work best together: logs explain what happened, traces show where and how long it took.
- When should I use centralized logging?
- Use it when you have more than one application, service, or environment and need a single place to troubleshoot issues, monitor errors, meet audit/compliance needs, or support on-call teams. It’s especially useful for microservices, Kubernetes, autoscaling systems, and multi-account/subscription setups where logs are otherwise scattered across hosts.
- How much does centralized logging cost?
- Cost usually depends on (1) log ingestion volume (GB/day), (2) retention period (days/months), (3) indexing/analytics features, (4) query volume, and (5) data export/egress to other tools. To control cost, filter noisy logs, sample high-volume debug logs, set shorter retention for low-value data, and archive older logs to cheaper storage when possible.
Category: monitoring
Difficulty: intermediate
Related Terms
See Also