Add and read stream entries

You've built a blazing-fast cache, managed session stores with TTL precision, and even pushed messages through Pub/Sub channels. But what happens when your application needs to process a continuous, ordered log of events — user clicks, sensor readings, orders — without losing a single entry? A simple key-value structure can't handle that. Enter Redis Streams, the log-based data structure that feels like a Kafka-lite built into your Redis instance. This lesson walks you through the two fundamental gateways: adding entries to a stream and reading them back in sequence.

The problem this lesson solves

Applications that handle event logs, activity feeds, or message queues need an append-only log where entries persist in insertion order. Standard Redis lists (LPUSH / RPOP) are great for simple queues, but they lack automatic IDs, consumer groups, and time-based range queries. Pub/Sub fires-and-forgets messages — if no client is listening, the message is lost forever. A Redis Stream fills this gap: it keeps every entry, assigns it a unique timestamped ID, and lets you read from any point in the log.

Without streams, you'd have to build custom logic for FIFO ordering, duplicate detection, and cursor management. Redis Streams provide all of that out of the box, making them ideal for event sourcing, activity tracking, and job queues.

Core concept / mental model

Imagine a stream as a never-ending, ordered scroll or a cassette tape. Each entry you append is written to the end of the tape, and you can start reading from the beginning (or any position) and move forward.

• Stream: A append-only, ordered data structure identified by a key (e.g., mystream).
• Entry: A timestamp-based ID (e.g., 1734192000000-0) plus a set of field-value pairs (like a mini hash).
• ID: Auto-generated by Redis as millisecondsTime-sequenceNumber or provided manually for precise control.
• Reader: A client that consumes entries from the stream, either from the start, from a specific ID, or only new entries (like $).

Pro tip: The ID is both the ordering mechanism and a physical timestamp. Redis guarantees that each ID under a given stream is unique and monotonically increasing.

How it works step by step

1. Adding entries with XADD

The XADD command appends a new entry to the stream. Redis automatically generates a unique ID unless you explicitly provide one.

XADD mystream * sensor-id 1234 temperature 19.8

Breaking it down: - XADD — the command. - mystream — the key of the stream (created automatically if it doesn't exist). - * — tells Redis to generate an ID based on current server time and a sequence counter. - sensor-id 1234 temperature 19.8 — alternating field-value pairs (like a flat hash).

Redis responds with the generated ID, e.g., 1734192000000-0.

2. Reading entries with XREAD

The simplest way to read entries is XREAD, which returns entries in insertion order.

XREAD COUNT 2 STREAMS mystream 0

This reads up to 2 entries from stream mystream starting at ID 0 (the very beginning). The response is an array of stream keys, each containing a list of entries.

3. Reading only new entries

To block and wait for new entries (like a real-time listener), use:

XREAD BLOCK 5000 COUNT 1 STREAMS mystream $

• BLOCK 5000 — wait up to 5000 milliseconds for new data.
• $ — special ID meaning "only entries arriving after this command was issued".

If no new entry arrives within the timeout, Redis returns (nil).

Hands-on walkthrough

Let's simulate a log aggregator that collects application logs and reads them back.

Step 1: Add several log entries

Open redis-cli and run:

redis-cli

> XADD app-logs * level ERROR message "disk full"
"1734192000000-0"

> XADD app-logs * level INFO message "user logged in"
"1734192000000-1"

> XADD app-logs * level WARN message "high memory usage"
"1734192000000-2"

Step 2: Read all entries from the beginning

> XREAD COUNT 10 STREAMS app-logs 0

Expected output:

1) 1) "app-logs"
   2) 1) 1) "1734192000000-0"
         2) 1) "level"
            2) "ERROR"
            3) "message"
            4) "disk full"
      2) 1) "1734192000000-1"
         2) 1) "level"
            2) "INFO"
            3) "message"
            4) "user logged in"
      3) 1) "1734192000000-2"
         2) 1) "level"
            2) "WARN"
            3) "message"
            4) "high memory usage"

Step 3: Read from a specific ID onward (cursor-like)

If you want to resume reading after the first entry, use its ID as the starting point:

> XREAD COUNT 10 STREAMS app-logs 1734192000000-0

This returns all entries after ID 1734192000000-0 — that is, the second and third entries.

Step 4: Use Python to write and read streams

import redis

r = redis.Redis(host='localhost', port=6379, decode_responses=True)

# Add entries
for i in range(3):
    entry_id = r.xadd('events', {'user': f'user_{i}', 'action': 'click'})
    print(f"Added entry {entry_id}")

# Read all entries
entries = r.xread({'events': '0'}, count=10)
for stream, messages in entries:
    for msg_id, fields in messages:
        print(f"ID: {msg_id} -> {fields}")

Expected output:

Added entry 1734192000000-0
Added entry 1734192000000-1
Added entry 1734192000000-2
ID: 1734192000000-0 -> {'user': 'user_0', 'action': 'click'}
ID: 1734192000000-1 -> {'user': 'user_1', 'action': 'click'}
ID: 1734192000000-2 -> {'user': 'user_2', 'action': 'click'}

Compare options / when to choose what

When to choose what: - Use XREAD with 0 for initial full read. - Use XREAD with a saved ID to resume from where you left off. - Use XREAD BLOCK $ for a live subscriber pattern. - For production workloads with multiple consumers, move to XREADGROUP.

Troubleshooting & edge cases

1. Stream key does not exist

If you call XREAD on a non-existent key, Redis returns an empty array — it does not create the stream automatically. Run XADD first to create it.

2. Using * as ID conflicts with manual IDs

If you manually pass an ID that is less than or equal to the first ID Redis would generate, you'll get an error:

> XADD mystream 0-0 field value
(error) ERR The ID specified in XADD is equal or smaller than the target stream top item

Always use * unless you need a specific timestamp for replay or backfilling.

3. Reading a stream that has never had any new data after $

If you use $ immediately after creating the stream, you'll never see existing entries. This is by design — $ means "only messages arriving after this command". To get both existing and new, you need to track the last read ID manually or use XREADGROUP.

4. Memory usage

Streams are append-only by default, so they can consume a lot of memory over time. Use XTRIM to cap the stream length (e.g., XTRIM mystream MAXLEN 1000) or set a maximum length at creation time with XADD MAXLEN ~ 1000 * ....

What you learned & what's next

You now understand the core idea of add and read stream entries: how to append structured data with XADD, how to read entries sequentially with XREAD, and how to use blocking reads for real-time consumption. You can: - Add entries with auto-generated IDs. - Read all entries, from a specific ID, or only new ones. - Use Python's redis-py library to interact with streams.

This foundational skill powers everything else in Redis Streams: consumer groups, acknowledgments, and stream-to-stream processing. In the next lesson, you'll learn how to scale your stream reader by splitting work across multiple consumers with XREADGROUP — turning your single consumer into a distributed processing pipeline.

Keep streaming!