Amazon Kinesis Data Streams
Real-time data streams
Durable data streams organized into shards; multiple consumers can re-read data within the retention period (24 hours by default, extendable up to 365 days). Available in provisioned and on-demand modes.
- Cost$$ Medium
- Difficulty●●●●●
- Costs money when idle?Yes
- Serverless
- In examsSAA · DVA · SOA
Think of it asLike a continuously recording tape: many people can rewind and watch the same segment.
When to use it
✓ Use it when
Low-latency real-time processing, multiple apps reading the same stream, or needing ordering by partition key.
✗ Avoid it when
Just need to dump data into S3/Redshift → Data Firehose; simple work queues → SQS.
Common uses
- Clickstreams for real-time dashboards
- Centralized log collection for multiple consumers
- IoT data processed with Lambda
What does it cost?
- Pricing model: Provisioned: per shard-hour and PUT payload unit; On-demand: per stream-hour and GB written/read.
Provisioned shards are billed hourly even with no data; ProvisionedThroughputExceeded errors come from "hot" partition keys.
When the question says multiple applications read the same data, replay, order by key, real-time, choose Kinesis Data Streams; for growing consumers getting slow, enhanced fan-out; for unpredictable load without shard management, on-demand mode.
Frequently asked questions
What is a shard and why does it affect performance?
A shard is the capacity unit of a stream: each shard has fixed write and read throughput limits, and records with the same partition key always go to the same shard to preserve order. If one partition key is too "hot", that shard becomes a bottleneck while others sit idle, so choose keys that spread evenly. To add throughput, increase the number of shards (resharding).
How do provisioned and on-demand modes differ?
Provisioned: you choose the shard count and pay per shard-hour, suited to predictable load. On-demand: AWS scales automatically with traffic and charges by data volume, with no shard planning, suited to variable or unknown load. Both retain data for the retention period you choose.
When do I use enhanced fan-out?
By default consumers share each shard's read throughput, so adding consumers slows each other down. Enhanced fan-out gives each consumer its own dedicated, pushed read path with low latency, suited to many consumers reading one stream, at extra cost. Consumers using the KCL also need DynamoDB to store checkpoints.
Related services
SAA-C03 study guide
Kinesis Data Streams is covered in these study modules: