Event Sourcing
Definition
Pattern of storing all changes to application state as a sequence of events, enabling better tracking, auditing, and rebuilding of application states.
Use Cases
- Netflix: Capturing and processing large volumes of domain and operational events to support near-real-time data pipelines and system observability. — Netflix built event-driven systems and streaming data pipelines using Apache Kafka as a durable event log, with downstream consumers materializing views and feeding analytics/monitoring systems. (Enabled scalable, decoupled services and near-real-time processing of events across many systems, improving agility and operational insight.)
- Uber: Streaming event data for operational workflows and analytics across a large microservices ecosystem. — Uber has used Apache Kafka as a central event streaming backbone, with producers emitting events and multiple consumer applications processing them for different purposes (e.g., pipelines, monitoring, and derived data sets). (Improved scalability and integration between services by allowing multiple independent consumers to react to the same event streams.)
- Walmart: Integrating systems and processing business events at scale for retail operations and data pipelines. — Walmart has described using Apache Kafka for event streaming, enabling systems to publish events that downstream services consume to build derived data products and operational integrations. (Supported high-throughput event processing and reduced coupling between systems by using streams as an integration mechanism.)
Frequently Asked Questions
- What's the difference between Event Sourcing and CQRS?
- Event Sourcing is about how you store state: you persist every change as an immutable event (e.g., MoneyDeposited, MoneyWithdrawn). CQRS (Command Query Responsibility Segregation) is about how you model access: you separate writes (commands) from reads (queries), often using different models. They’re commonly used together: Event Sourcing provides the write-side history, and CQRS read models are built by replaying events into query-optimized views.
- When should I use Event Sourcing?
- Use it when you need a complete audit trail, the ability to reconstruct past state (“what did the account balance look like last Tuesday?”), or when multiple downstream systems need to react to the same business events. It’s a good fit for domains like finance, orders, inventory, and workflows. Avoid it if your domain is simple, you don’t need historical reconstruction, or your team can’t invest in the extra operational and design complexity (event versioning, replay, idempotency, and read-model maintenance).
- How much does Event Sourcing cost?
- Costs come from the infrastructure you choose for the event store and event distribution, plus operational overhead. Key cost drivers include: event volume (writes per second), retention period (how long you keep events), storage type (SSD vs. object storage), replication and backups, and consumer processing (compute for projections/read models). If you use managed streaming (e.g., Kafka-as-a-service) and managed databases, expect costs to scale with throughput, storage, and availability requirements; if you self-manage, you may reduce direct service spend but increase engineering and operations cost.
Category: software
Difficulty: advanced
See Also