Kinesis
Definition
AWS platform for real-time data streaming and analytics. Like a high-speed conveyor belt for processing millions of data records per second.
Use Cases
- Netflix: Real-time monitoring and analytics of application and playback events to improve reliability and user experience — Netflix publishes high-volume operational and event telemetry into Amazon Kinesis Data Streams, then processes and routes the data to downstream analytics and monitoring systems for near-real-time insights. (Faster detection of issues, improved operational visibility, and better ability to respond to incidents affecting streaming quality.)
- Zynga: Real-time game analytics and player behavior tracking for live operations and product decisions — Zynga streams gameplay events into Amazon Kinesis and processes them to generate near-real-time metrics and dashboards used by game teams. (Quicker insight into player behavior and game performance, enabling faster iteration and more responsive live operations.)
Provider Equivalents
- AWS: Amazon Kinesis (Data Streams, Data Firehose, Data Analytics, Video Streams)
- Azure: Azure Event Hubs (and Event Hubs Capture), Azure Stream Analytics
- GCP: Pub/Sub, Dataflow
- OCI: OCI Streaming, OCI Service Connector Hub
Frequently Asked Questions
- What's the difference between Kinesis and Amazon SQS?
- Kinesis is built for high-throughput streaming where many consumers may read the same ordered stream of events (for example, analytics, monitoring, and real-time dashboards). Amazon SQS is a message queue designed to decouple systems, where messages are typically processed by one consumer (or one consumer group) and then removed from the queue. Use Kinesis when you need stream processing and multiple readers; use SQS when you need reliable task/work distribution between services.
- When should I use Kinesis?
- Use Kinesis when you need to ingest and process continuous data in near real time—such as clickstreams, IoT telemetry, application logs, fraud detection signals, or operational metrics. It’s a good fit when event volume is high, you need low-latency processing, you want multiple applications to consume the same stream, or you need to fan out data to analytics, storage, and monitoring tools.
- How much does Kinesis cost?
- Pricing depends on which Kinesis capability you use and how much data you ingest, process, and store. For Kinesis Data Streams, major cost drivers include stream capacity (shards or on-demand throughput), data retention duration, and enhanced fan-out consumers. For Kinesis Data Firehose, costs typically depend on the volume of data ingested and any optional data transformation (for example, AWS Lambda). For Kinesis Data Analytics, costs are based on the resources used to run streaming SQL or Apache Flink applications. Always estimate using your expected records/second, average record size, number of consumers, and retention needs.
Category: data
Difficulty: advanced
Related Terms
See Also