core (the rg binary)

Active contributors: Andrew Gallant

Purpose

crates/core is the source of the rg binary itself. Despite the name, it isn't a library crate; it is the [[bin]] defined in the root Cargo.toml. It owns CLI parsing, the high-level search/files/types/generate dispatch, and the threading model.

Everything else in the workspace is a library; this crate is what users actually run.

Directory layout

crates/core/
├── main.rs           # Entry point + search() / search_parallel() / files() / files_parallel() / types() / generate() / special()
├── search.rs         # SearchWorker: per-file search loop wrapping a Searcher + Matcher + Printer
├── haystack.rs       # Haystack: the file (or stdin) currently being searched
├── messages.rs       # err_message! and stderr locking
├── logger.rs         # Wires the `log` crate's level to --debug / --trace
├── README.md
└── flags/
    ├── mod.rs        # The Flag trait and Category enum
    ├── parse.rs      # CLI argument parser (lexopt-based)
    ├── lowargs.rs    # LowArgs: low-level arg representation
    ├── hiargs.rs     # HiArgs: resolved, ready-to-use arg representation
    ├── defs.rs       # The big static array of every flag definition (~235K)
    ├── config.rs     # Reading $RIPGREP_CONFIG_PATH
    ├── doc/          # Man page + --help text generation
    └── complete/     # Bash, zsh, fish, PowerShell completion generators

Key abstractions

Type	File	Description
Mode	crates/core/flags/lowargs.rs	Top-level operating mode: Search(SearchMode), Files, Types, Generate(GenerateMode). Decides which top-level entry point in main.rs runs.
SearchMode	crates/core/flags/lowargs.rs	Which output format: Standard, Summary, JSON.
SpecialMode	crates/core/flags/lowargs.rs	Short-circuit modes: --help, --version, --pcre2-version. Skip everything else and exit.
LowArgs	crates/core/flags/lowargs.rs	Flat struct populated by every flag. Near 1:1 with the parsed CLI.
HiArgs	crates/core/flags/hiargs.rs	Higher-level struct built from LowArgs plus config and environment. Has methods that build WalkBuilder, Matcher, Searcher, Printer, and the worker pool.
ParseResult<T>	crates/core/flags/parse.rs	Ok(T) | Err(anyhow::Error) | Special(SpecialMode). Encodes the three outcomes of CLI parsing.
Flag	crates/core/flags/mod.rs	Trait every flag implements. Provides short/long names, doc strings, category, and update(value, &mut LowArgs).
SearchWorker	crates/core/search.rs	Wraps a Searcher + a Matcher + a Printer. One per worker thread in parallel mode.
Haystack	crates/core/haystack.rs	The file or stdin being searched. Holds the path and metadata.

How it works

graph TD
    Argv["std::env::args_os()"] --> Parse["flags::parse() -> ParseResult<HiArgs>"]
    Parse --> Run["run(result)"]
    Run -->|Mode::Search singlethread| Search1["search()"]
    Run -->|Mode::Search multithread| SearchN["search_parallel()"]
    Run -->|Mode::Files singlethread| Files1["files()"]
    Run -->|Mode::Files multithread| FilesN["files_parallel()"]
    Run -->|Mode::Types| Types[types()]
    Run -->|Mode::Generate| Gen[generate()]
    Parse -->|ParseResult::Special| Special[special()]
    Search1 --> Walker["HiArgs::walk_builder()"]
    SearchN --> Walker
    Walker --> Worker["SearchWorker (search.rs)"]
    Worker --> Out[stdout/stderr]

The dispatch happens in run() in crates/core/main.rs:

let matched = match args.mode() {
    Mode::Search(_) if !args.matches_possible() => false,
    Mode::Search(mode) if args.threads() == 1 => search(&args, mode)?,
    Mode::Search(mode) => search_parallel(&args, mode)?,
    Mode::Files if args.threads() == 1 => files(&args)?,
    Mode::Files => files_parallel(&args)?,
    Mode::Types => return types(&args),
    Mode::Generate(mode) => return generate(mode),
};

matches_possible() is a fast-path: if the user provided no patterns and no pattern files, ripgrep returns "no matches" without spinning up the walker.

Parallel search

search_parallel() (in crates/core/main.rs) is the hottest path. It:

1. Builds a per-thread cloneable SearchWorker.
2. Calls WalkBuilder::build_parallel().run(...) from the ignore crate. This spawns N worker threads (default = num CPUs) and feeds each a closure.
3. Each worker clones the SearchWorker, searches one file, writes its output to a per-thread termcolor::Buffer, then hands the buffer to a shared BufferWriter for atomic stdout writes.
4. Tracks matched: AtomicBool and searched: AtomicBool for the exit-code and "no files searched" diagnostic.

The BufferWriter ensures that the output of two threads can never interleave mid-line, even though ordering between files is non-deterministic.

When --sort is requested, HiArgs::threads() returns 1 to disable parallelism, since sorted output requires sequential processing.

Flag system

The CLI is defined as a static array of &dyn Flag values in crates/core/flags/defs.rs. Each flag is a unit struct (struct AfterContext;) implementing the Flag trait. Flag::update mutates a LowArgs. The parser in parse.rs uses lexopt to tokenise argv, then dispatches each flag to the right struct via a hash table keyed on long/short/aliased names.

This design has three notable properties:

• The man page, --help short text, --help long text, and shell completions are all generated from the same array. A new flag is documented in exactly one place.
• Flag categories (Input, Search, Filter, Output, OutputModes, Logging, OtherBehaviors) drive sectioning in --help output.
• Negation flags (--no-foo) are not separate types; they are declared via Flag::name_negated.

See features: cli flags for the user-visible side and patterns and conventions for the conventions when adding a new flag.

Integration points

• grep crate — reexported submodules used to construct matchers, searchers, printers (use grep::matcher::...; use grep::searcher::...;).
• ignore crate — HiArgs::walk_builder() returns an ignore::WalkBuilder.
• anyhow — the binary's Result type.
• lexopt — argument parsing.
• textwrap — wraps doc strings to fit terminal width in --help.

Entry points for modification

• Adding a new flag: edit crates/core/flags/defs.rs (struct + Flag impl + register in the master slice), add a field to LowArgs, plumb it through HiArgs if needed.
• Changing search behaviour: usually means modifying crates/core/search.rs (SearchWorker) or one of the grep-* libraries.
• Adding a new operating mode: edit lowargs::Mode, add a top-level fn in main.rs, register it in run().

For wider context see matcher, searcher, and printer.