matcher

Active contributors: Andrew Gallant

Purpose

crates/matcher (published as grep-matcher) defines the Matcher trait — the abstract interface that every regex implementation in ripgrep satisfies. The crate has no concrete matchers of its own; the implementations live in grep-regex (Rust regex) and grep-pcre2 (PCRE2). The trait is what lets grep-searcher work with any regex engine without knowing which one.

A key design decision is internal iteration (the "push" model): the Matcher drives the search and invokes a callback when a match is found. The trait's docs justify this choice over external iteration on the grounds of expressiveness and performance.

Directory layout

crates/matcher/src/
├── lib.rs              # The Matcher trait, Match struct, NoError, default methods
└── interpolate.rs      # Replacement-string interpolation ($1, ${name}, $$)

lib.rs is large (~46K) because it contains the trait, the helper types, and the default-method machinery for plumbing captures, line endings, and replacements through.

Key abstractions

Type	File	Description
Matcher	crates/matcher/src/lib.rs	The core trait. Methods like find_at, captures_at, replace_with_captures_at, find_iter_at.
Match	crates/matcher/src/lib.rs	A (start, end) byte range, structurally identical to std::ops::Range<usize> but Copy and with a start <= end invariant.
Captures	crates/matcher/src/lib.rs	Trait for accessing capture groups.
LineTerminator	crates/matcher/src/lib.rs	Configurable line terminator (default \n, may be \r\n or \0 for null-data).
LineMatchKind	crates/matcher/src/lib.rs	Discriminates a confirmed match from a candidate that still needs verification — useful for fast-path optimisations.
NoError	crates/matcher/src/lib.rs	An uninhabited error type for matchers that can never fail.
interpolate	crates/matcher/src/interpolate.rs	Helper for $1, ${name}, $$ replacement-string substitution, used by replace_with_captures.

How it works

A search proceeds in three layers:

graph LR
    Searcher["grep_searcher::Searcher"] -->|chunks of bytes| MatcherImpl["impl Matcher"]
    MatcherImpl -->|Match ranges| Sink["impl Sink"]
    Sink -->|"sink_match / sink_context callbacks"| Output[stdout/buffer]

The Searcher doesn't know what kind of pattern is being matched; it just calls Matcher::find_at (or related methods) with byte slices. The matcher implementation decides whether to use Aho-Corasick, finite automata, or PCRE2 internally.

Default method machinery

Most of lib.rs is default-method implementations layered on top of two required methods. A regex implementation only has to implement find_at (and optionally captures_at for capture support); Matcher provides everything else (find_iter, replace, captures_iter, etc.) by composing them.

This is why crates/regex/src/lib.rs is just six lines of re-exports — most of the weight is shared by grep-matcher.

Replacement interpolation

interpolate.rs parses a replacement string like prefix-$1-${name}-suffix and produces output by looking up $1 and ${name} in a Captures impl. The escape $$ produces a literal $. Implementations:

• interpolate(replacement, dst, name_to_index, captures) — bytes-only.
• interpolate_bytes and interpolate_string — UTF-8 wrappers.

This is the engine behind rg -r '$1' 'capture(group)'.

Integration points

• Implemented by grep_regex::RegexMatcher (crates/regex/src/matcher.rs) and grep_pcre2::RegexMatcher (crates/pcre2/src/matcher.rs).
• Used by grep_searcher::Searcher (crates/searcher/src/searcher/mod.rs) for every match.
• Used by grep_printer (crates/printer/src/standard.rs, summary.rs, json.rs) when printing replaced output and capture groups.

Why a custom trait instead of the regex crate's own?

Three reasons:

1. Engine-agnosticism. Switching between Rust's regex and PCRE2 must be transparent to the search loop.
2. Line-aware operations. ripgrep needs primitives like "find the next match in a single line" that the regex crate doesn't offer directly. The Matcher trait has methods that take &[u8] slices already pre-divided into lines.
3. Substring extensions. Matcher::is_match_at and similar take a starting offset, allowing a Searcher to scan from a specific cursor without reslicing.

Entry points for modification

• Implementing a new matcher: implement the Matcher trait. See crates/regex/src/matcher.rs for a full reference implementation.
• Adding a new interpolation feature: change crates/matcher/src/interpolate.rs. The format is intentionally minimal; expanding it is rarely the right call.
• Adding a method to the trait: think hard. Every implementation in the workspace and downstream needs to be updated.