Step Functions
Definition
AWS service for coordinating multiple AWS services into serverless workflows using visual state machines, simplifying application development.
Use Cases
- Netflix: Coordinating large-scale microservice workflows for content delivery and cloud operations (e.g., deployments and operational runbooks). — Netflix built and uses Conductor, an open-source workflow orchestration engine, to define workflows (DAG-style) that coordinate calls to many microservices with retries, timeouts, and compensation logic. (Improved reliability and visibility of complex, multi-service processes by centralizing workflow state, enabling consistent retries/timeout handling, and providing operational monitoring for long-running workflows.)
- Uber: Orchestrating long-running, multi-step business processes across microservices (e.g., trip- and marketplace-adjacent workflows that require coordination and retries). — Uber developed and open-sourced Cadence (later continued as Temporal by the community/company), using workflow code to coordinate activities with durable state, retries, and versioning for evolving workflows. (Reduced operational complexity for long-running processes by making workflow state durable and recoverable, improving resilience to service failures and simplifying retry/timeout patterns.)
- Airbnb: Scheduling and orchestrating data pipelines and batch workflows (ETL, reporting, machine learning feature generation). — Airbnb has used Apache Airflow to define DAGs that coordinate tasks across data systems, with centralized scheduling, dependency management, and monitoring. (More dependable and observable data workflows through explicit task dependencies, automated retries, and a unified view of pipeline health.)
Provider Equivalents
- AWS: AWS Step Functions
- Azure: Azure Logic Apps
- GCP: Google Cloud Workflows
- OCI: Oracle Cloud Infrastructure (OCI) Workflows
Frequently Asked Questions
- What's the difference between workflow orchestration and choreography in microservices?
- Orchestration uses a central controller (the orchestrator) that tells each service what to do and when, and it tracks the workflow state. Choreography has no central controller; services publish and react to events, and the overall process emerges from those event interactions. Orchestration is often easier to monitor and debug for complex, multi-step flows, while choreography can reduce coupling but may be harder to trace end-to-end.
- When should I use workflow orchestration?
- Use it when you have a multi-step process that spans multiple services and needs reliable sequencing, retries, timeouts, and visibility. Common triggers include: long-running processes (minutes to days), the need for human approvals, complex branching logic, compensation/rollback steps (saga patterns), and a requirement to audit or replay what happened in an order or request.
- How much does workflow orchestration cost?
- Costs usually depend on how many workflow executions you run, how many state transitions/steps occur, and how long workflows run. Managed services may also charge for connectors, API calls, or compute used by tasks (for example, serverless functions or containers invoked by the workflow). To estimate cost, count: (1) executions per month, (2) average steps/state transitions per execution, (3) any external compute and network usage, and (4) logging/monitoring retention.
Category: software
Difficulty: advanced
Related Terms
See Also