Set up a Sentinel cluster

You have a single Redis server humming along in development. In production, that single node is a single point of failure. If it crashes, your cache goes cold, your session data vanishes, and your application starts serving errors or stale data. The pain is real: an hour of unplanned downtime can cost thousands, and manual failover is slow and error-prone. Redis Sentinel automates this problem — it monitors your master and replica instances, performs automatic failover when the master fails, and notifies your application with updated connection details so you never notice the switch. In this lesson, you'll learn set up a Sentinel cluster step by step, from core concepts to a fully working deployment.

The problem this lesson solves

A single Redis instance is like a tightrope walker without a safety net. If it goes down, your entire application loses its fast data store. You could set up a replica manually, but even then:

• Manual failover requires an engineer to SSH in, promote the replica, and update application configs. This takes minutes (or hours) and risks human error.
• No monitoring — you might not even know the master is down until users complain.
• Split-brain scenarios can corrupt data if two servers think they're the master.

Redis Sentinel solves all three problems by providing:

• Continuous monitoring — every Sentinel instance checks the master and replicas every second.
• Automatic failover — when the master is declared ODOWN (Objectively Down) by a quorum of Sentinels, they promote a replica to master and reconfigure all other replicas to follow the new master.
• Notification — Sentinels act as a source of truth; your application queries them via GET-MASTER-ADDR-BY-NAME to always get the current master address.

The lesson assumes you have already set up a Redis master and at least one replica (see previous lessons in this track). If you haven't, do that first — a Sentinel cluster without replicas is useless.

Core concept / mental model

Think of Sentinel as a distributed watchdog with voting power. It's not a single process — you deploy an odd number of Sentinel instances (typically 3 or 5) across different machines or containers. Each Sentinel:

• Connects to the master and all replicas.
• Sends PING every second and expects PONG within a configurable timeout (down-after-milliseconds).
• Communicates with other Sentinels via pub/sub channels to share their view of the world.

The quorum and majority

Two thresholds control failover decisions:

• quorum (Q): The number of Sentinels that must agree the master is subjectively down (SDOWN) — typically 2 out of 3. Once quorum is reached, the master is marked ODOWN.
• majority (M): The number of Sentinels needed to actually perform failover — always floor(N/2) + 1 where N is the total number of Sentinels. If you have 3 Sentinels, majority is 2; if 5, majority is 3.

Pro tip: Always deploy an odd number of Sentinels (3, 5, 7) to avoid tie votes. Two Sentinels cannot form a quorum and majority simultaneously — you need at least 3.

The Sentinel life cycle

1. Detection: A Sentinel notices the master hasn't responded to PING for down-after-milliseconds → marks it SDOWN.
2. Voting: It asks other Sentinels, "Is the master down?" If at least quorum Sentinels agree, the master becomes ODOWN.
3. Election: A Sentinel leader is elected among those that want to perform failover (using Raft-like consensus).
4. Failover: The leader picks the best replica (based on priority, replication offset, etc.), promotes it to master, and reconfigures all other replicas to follow the new master.
5. Reconfiguration: The old master, when it comes back, is demoted to a replica of the new master.

How it works step by step

Step 1: Configure sentinel.conf

Each Sentinel instance has its own configuration file. At minimum, you need:

# sentinel.conf
port 26379
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1

• port 26379 — Sentinel uses a different port than Redis (6379) to avoid confusion.
• sentinel monitor mymaster <host> <port> <quorum> — declares the master to monitor; the name mymaster is arbitrary but must be consistent across all Sentinels. The quorum is 2 here (for 3 Sentinels).
• down-after-milliseconds — how long without PONG before the master is considered down.
• failover-timeout — total time allowed for a failover attempt.
• parallel-syncs — how many replicas can synchronize with the new master simultaneously. Setting 1 avoids overwhelming the new master.

Step 2: Start the Sentinel

redis-sentinel /path/to/sentinel.conf
# Or:
redis-server /path/to/sentinel.conf --sentinel

Sentinel will rewrite its own config file when it detects changes (e.g., new master address after failover). Make sure the config file is writable by the Sentinel process.

Step 3: Verify the setup

Connect to any Sentinel using redis-cli -p 26379 and run:

SENTINEL masters
# Output shows mymaster status
SENTINEL get-master-addr-by-name mymaster
# Output: "127.0.0.1", "6379"
SENTINEL replicas mymaster
# Lists connected replicas

Hands-on walkthrough

Let's set up a 3-node Sentinel cluster monitoring a master and one replica. You'll use Docker for isolation, but the steps are identical for bare-metal or VMs.

Prerequisites

• Docker and Docker Compose installed.
• Basic understanding of Redis replication (previous lessons).

docker-compose.yml

version: '3.8'
services:
  redis-master:
    image: redis:7-alpine
    ports:
      - "6379:6379"

  redis-replica:
    image: redis:7-alpine
    ports:
      - "6380:6379"
    command: redis-server --slaveof redis-master 6379
    depends_on:
      - redis-master

  sentinel-1:
    image: redis:7-alpine
    ports:
      - "26379:26379"
    volumes:
      - ./sentinel-1.conf:/sentinel.conf
    command: redis-sentinel /sentinel.conf
    depends_on:
      - redis-master
      - redis-replica

  sentinel-2:
    image: redis:7-alpine
    ports:
      - "26380:26379"
    volumes:
      - ./sentinel-2.conf:/sentinel.conf
    command: redis-sentinel /sentinel.conf
    depends_on:
      - redis-master
      - redis-replica

  sentinel-3:
    image: redis:7-alpine
    ports:
      - "26381:26379"
    volumes:
      - ./sentinel-3.conf:/sentinel.conf
    command: redis-sentinel /sentinel.conf
    depends_on:
      - redis-master
      - redis-replica

sentinel-1.conf

port 26379
dir /tmp
sentinel monitor mymaster redis-master 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1

Create files sentinel-2.conf and sentinel-3.conf with the same content, but change port to 26380 and 26381 respectively (though Docker container port mapping makes this optional).

Start and test

docker-compose up -d
sleep 5
docker exec -it <sentinel-1-container> redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
# Expected output: 1) "172.17.0.2" (or container IP)
#                2) "6379"

Now kill the master:

docker stop redis-master
# Wait ~10 seconds for detection
sleep 15
docker exec -it <sentinel-1-container> redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
# Output should now show the replica's IP and port 6379

You've just witnessed automatic failover. The replica was promoted to master, and your application can now connect via Sentinel to get the new master address.

Compare options / when to choose what

When to choose Sentinel cluster:

• You need high availability but only one write endpoint.
• Your data fits in a single Redis instance (memory limits).
• You want transparent failover without application changes (use Sentinel client libraries).
• You're already running a master-replica setup.

When not to choose Sentinel:

• Your dataset exceeds a single node's memory — use Redis Cluster.
• You can tolerate manual failover in dev/staging — keep it simple.
• You need strong consistency guarantees (Sentinel is eventually consistent).

Troubleshooting & edge cases

Sentinel cannot connect to master

Symptom: SENTINEL masters shows flags containing s_down or odown immediately.

Fix: - Check network: telnet master-host 6379 from Sentinel container. - Verify sentinel monitor points to the correct hostname/IP (Docker service names work only within the same network). - Ensure master's bind directive allows connections (avoid bind 127.0.0.1).

Split-brain scenarios

Problem: If network partitions break the quorum briefly, two Sentinels might promote different replicas, creating two masters. This is rare but possible.

Mitigation: - Always use at least 3 Sentinels with quorum 2. With a partition of 1 vs 2, the side with 2 holds quorum and becomes authoritative. - Use min-replicas-to-write on the master to reject writes if too few replicas are connected (reduces divergence). - Monitor redis-sentinel.log for +switch-master events.

Sentinel not rewriting config

Symptom: After a failover, Sentinel loses the new master address on restart.

Fix: - Ensure the Sentinel config file is writable by the Redis user: chown redis:redis /etc/redis/sentinel.conf. - The file must have a filename ending in .conf — Sentinel refuses to rewrite if it can't. - Run redis-sentinel as the same user that owns the config file.

Clients connect to dead master

Problem: Application code has the master IP hardcoded and never queries Sentinel.

Fix: - Use a client library that supports Sentinel (e.g., redis-py with RedisSentinel). - Example connection via redis-py:

from redis.sentinel import Sentinel
sentinel = Sentinel([('sentinel-1', 26379)], socket_timeout=0.1)
master = sentinel.master_for('mymaster', socket_timeout=0.1)
slave = sentinel.slave_for('mymaster', socket_timeout=0.1)
print(master.get('key'))  # Always goes to current master

What you learned & what's next

You now understand how set up a Sentinel cluster provides automatic failover for Redis. You've configured three Sentinels, verified monitoring, and observed a failover in action. You can troubleshoot common pitfalls like split-brain and config file permissions.

Key takeaways: 1. A Sentinel cluster solves the single-point-of-failure problem with automatic failover. 2. Deploy an odd number of Sentinels (≥3) with a quorum of floor(N/2) + 1. 3. Use sentinel monitor to declare the master; replicas are auto-discovered. 4. Test failover by stopping the master and observing SENTINEL get-master-addr-by-name. 5. Clients must query Sentinel for the current master — hardcoded IPs break during failover.

Next lesson: You'll explore Redis Cluster for horizontal scaling — distributing data across multiple master nodes while maintaining high availability.