redis-sentinel

A high-availability supervisor for primary/replica deployments. Sentinels gossip with each other, monitor the health of a set of declared masters, elect a leader when a master fails, and orchestrate the failover.

Active contributors: antirez, Oran Agra, Yossi Gottlieb, zhaozhao.zz.

Purpose

redis-sentinel is the same binary as redis-server, started in a different runtime mode. It does not hold any user dataset; it only stores a topology view in memory and persists a small config file. The flags that activate Sentinel mode:

• Argument 0 ends with redis-sentinel.
• The first argument is --sentinel.
• The configuration sets sentinel monitoring directives.

When Sentinel mode is on, the server swaps the command table for sentinelcmds (declared in src/sentinel.c) and the cron callback delegates to sentinelTimer. There is no SET, no GET, no DBSIZE — only SENTINEL ..., INFO, PING, SUBSCRIBE, etc.

How a Sentinel works

sequenceDiagram
    participant S as Sentinel
    participant M as Master
    participant R as Replica
    participant Other as Other Sentinels

    S->>M: PING (every 1s)
    S->>R: PING + INFO replication (every 1s)
    S->>Other: HELLO over Pub/Sub (every 2s)
    M->>S: PONG
    Note over S,M: After down-after-milliseconds<br/>without PONG → +sdown
    S->>Other: is-master-down-by-addr
    Note over S,Other: Quorum reached → +odown
    S->>Other: failover request<br/>raft-style election
    S->>R: SLAVEOF NO ONE (winner promotes a replica)
    S->>Other: New master broadcast

Sentinel's protocol is a hand-rolled Raft-style consensus where the term is called the current epoch. The leader for a failover is the Sentinel that gets a majority vote. Vote is only granted to a Sentinel with a higher configured epoch.

Source layout

A single file: src/sentinel.c (~5,500 lines). It contains:

• The Sentinel-specific command table.
• Subscriber/Publisher logic for the __sentinel__:hello channel.
• The TILT mode (Sentinel temporarily refusing to act when its clock jumps).
• The sentinelEvent framework — every state change emits a Pub/Sub event that operators or scripts can subscribe to (e.g. +sdown, +odown, +failover-state-..., +switch-master).
• The runtime configuration loader/writer (Sentinel rewrites sentinel.conf whenever it learns a new replica or completes a failover).
• The SENTINEL command implementations (SENTINEL MASTERS, SENTINEL REPLICAS, SENTINEL CKQUORUM, SENTINEL FAILOVER, SENTINEL RESET, SENTINEL MONITOR, …).

Sentinel command surface

Command	Purpose
SENTINEL MASTERS	List monitored masters with their state.
SENTINEL MASTER <name>	Detailed info about one master.
SENTINEL REPLICAS <name> (formerly SLAVES)	Replicas of a monitored master.
SENTINEL SENTINELS <name>	Other Sentinels watching the same master.
SENTINEL GET-MASTER-ADDR-BY-NAME <name>	The classic client-discovery call.
SENTINEL RESET <pattern>	Wipe runtime state for matching masters.
SENTINEL FAILOVER <name>	Manually trigger a failover.
SENTINEL CKQUORUM <name>	Check if there are enough Sentinels for a quorum.
SENTINEL FLUSHCONFIG	Force-write the runtime config file.
SENTINEL MONITOR / REMOVE / SET	Add, remove, reconfigure monitored masters at runtime.
SENTINEL SIMULATE-FAILURE <flag>	Inject a fault for testing.
SENTINEL DEBUG, SENTINEL CONFIG SET/GET	Operator levers.

The full schema is in src/commands/sentinel-*.json.

Pub/Sub events

Sentinel publishes every state transition on internal Pub/Sub channels. Subscribers (operator dashboards, alerting systems) listen with PSUBSCRIBE *. Common events:

• +sdown / -sdown — subjective down.
• +odown / -odown — objective down (quorum).
• +failover-state-... — leader-election and failover state machine transitions.
• +switch-master — a failover completed; the master address has changed.
• +sentinel, -sentinel — a Sentinel joined or left.
• +slave / -slave — a replica was discovered or removed.
• +tilt / -tilt — Sentinel entered or exited TILT mode.

Why TILT mode?

If the Sentinel's monotonic clock advances by more than sentinel-tilt-period ms relative to wall-clock or the cron is delayed past a threshold, Sentinel enters TILT. While in TILT it stops voting in elections, stops promoting replicas, and only continues to gossip and answer INFO. This prevents a Sentinel that just woke up from a long pause (e.g. a swap storm) from triggering bad failovers.

Configuration

Sentinel's persisted state lives in sentinel.conf (or whatever path you pass on the command line). Sentinel rewrites this file each time it learns about a new replica, an epoch bump, or completes a failover. Hand-editing while Sentinel runs is a recipe for losing edits.

Key directives:

sentinel monitor <name> <ip> <port> <quorum>
sentinel down-after-milliseconds <name> <ms>
sentinel parallel-syncs <name> <count>
sentinel failover-timeout <name> <ms>
sentinel auth-pass <name> <password>
sentinel auth-user <name> <user>
sentinel notification-script <name> <path>
sentinel client-reconfig-script <name> <path>
sentinel deny-scripts-reconfig yes

The example sentinel.conf in the repo root is heavily commented and is the de facto reference.

Testing

Tests live in tests/sentinel/. They are driven by runtest-sentinel, which uses tests/instances.tcl to spin a topology of three Sentinels + one master + two replicas, kill nodes in scripted patterns, and assert that the failover protocol works.

Where to start modifying

• Tweak failover state machine — search for sentinelStartFailover and the SENTINEL_FAILOVER_STATE_* enum in src/sentinel.c.
• Add a SENTINEL subcommand — add the JSON, regenerate commands.def, add the C handler in the giant sentinelCommand switch.
• Change Pub/Sub event emission — sentinelEvent() in src/sentinel.c.
• Add a config directive — sentinelHandleConfiguration() parses; sentinelFlushConfig() writes back.

Related pages

• redis-server
• Cluster — for shard-and-fail-over via cluster instead of Sentinel.