Distributed Tracing
Definition
Distributed Tracing is a technique for monitoring and tracking requests as they navigate through various services in a cloud application ecosystem.
Use Cases
- Netflix: Troubleshooting latency and failures across a large microservices architecture during streaming and API requests — Netflix built and open-sourced distributed tracing tooling (notably Zipkin, originally from Twitter, and Netflix’s own ecosystem integrations) to propagate trace IDs across services and visualize call paths and timing to isolate slow dependencies. (Faster identification of which downstream service or dependency caused elevated latency or errors, reducing time to diagnose issues in complex service graphs.)
- Uber: Understanding end-to-end request behavior across microservices for ride dispatch and trip lifecycle operations — Uber open-sourced Jaeger (a distributed tracing system) and used it to collect traces across services, visualize critical paths, and investigate performance regressions and dependency bottlenecks. (Improved observability across microservices and quicker root-cause analysis for latency spikes by pinpointing slow spans and problematic dependencies.)
- Shopify: Diagnosing performance bottlenecks in a high-traffic commerce platform with many internal services and dependencies — Shopify has shared engineering work around observability and tracing practices to follow requests across components, correlate traces with logs/metrics, and identify slow database or downstream calls affecting request latency. (Reduced time to isolate performance issues by identifying the specific component and operation responsible for slow requests, improving reliability during peak traffic.)
Provider Equivalents
- AWS: AWS X-Ray
- Azure: Azure Monitor Application Insights (Distributed Tracing)
- GCP: Cloud Trace
- OCI: OCI Application Performance Monitoring (APM) - Distributed Tracing
Frequently Asked Questions
- What's the difference between distributed tracing and logging?
- Logging records discrete events (messages) from a service, like "payment started" or "DB query failed." Distributed tracing follows one request end-to-end across many services using a shared trace ID, showing the full path (service-to-service calls) and how long each step took. In practice, traces answer "where is the time going?" while logs answer "what happened here?"
- When should I use distributed tracing?
- Use it when a single user action (login, search, checkout, API call) touches multiple services and you need to find where latency or errors originate. It’s especially useful for microservices, serverless workflows, and systems with many dependencies (databases, caches, third-party APIs). If you frequently see "it’s slow but we don’t know which service is responsible," tracing is a strong fit.
- How much does distributed tracing cost?
- Cost depends on how many traces you collect, how much data each trace contains (number of spans, attributes), retention period, and whether you sample (store only a percentage of requests). Vendor pricing is typically based on trace ingestion/analysis and storage. To control cost, teams commonly use sampling (e.g., 1–10% of requests), keep high-cardinality attributes in check, and increase sampling temporarily during incidents.
Category: monitoring
Difficulty: advanced
Related Terms
See Also