ignore

Active contributors: Andrew Gallant

Purpose

crates/ignore is the recursive directory walker that knows about .gitignore, .ignore, .rgignore, hidden-file rules, and the core.excludesFile global gitignore. It also exposes a parallel walker that distributes work across a thread pool.

It is the most-cited library outside ripgrep itself: cargo uses it, and many build tools depend on it.

Directory layout

crates/ignore/src/
├── lib.rs               # Public API (Walk, WalkBuilder, ParallelVisitor, Error)
├── walk.rs              # The walker implementations (~85K, the single largest file in the workspace)
├── dir.rs               # Per-directory ignore stack: combines .gitignore, .ignore, exclude files, types, overrides (~46K)
├── gitignore.rs         # Gitignore matcher: parsing and matching .gitignore files (~30K)
├── overrides.rs         # Override matcher: --glob / --iglob CLI patterns
├── types.rs             # Type matcher: the -t/-T file-type filter (~19K)
├── default_types.rs     # Default file-type definitions (Rust, Python, JS, ...)
└── pathutil.rs          # Path manipulation helpers

Key abstractions

Type	File	Description
Walk	crates/ignore/src/walk.rs	Single-threaded directory iterator. for entry in Walk::new(".") {}.
WalkBuilder	crates/ignore/src/walk.rs	Builder for both Walk and WalkParallel. Configures gitignore behaviour, hidden files, max depth, max filesize, types, overrides, sorting, custom ignore filenames, and the parallel thread count.
WalkParallel	crates/ignore/src/walk.rs	Parallel walker. WalkParallel::run(visitor) farms work across a thread pool.
WalkState	crates/ignore/src/walk.rs	Returned from per-entry visitor: Continue, Skip (don't descend into a directory), Quit (stop entire walk).
DirEntry	crates/ignore/src/walk.rs	Yielded entry: path, depth, file type, whether it's a symlink, error. Wraps walkdir::DirEntry.
Gitignore	crates/ignore/src/gitignore.rs	A parsed and compiled set of gitignore patterns. Methods: matched, matched_path_or_any_parents.
GitignoreBuilder	crates/ignore/src/gitignore.rs	Builder. Knows about add() for files, add_line() for inline rules, case_insensitive, BOM stripping.
Match<T>	crates/ignore/src/lib.rs	Three-state result: None, Ignore(T), Whitelist(T).
Override / OverrideBuilder	crates/ignore/src/overrides.rs	The --glob / --iglob matcher.
Types / TypesBuilder	crates/ignore/src/types.rs	The -t/-T matcher. Maps file-type names ("rust") to glob sets.
default_types::DEFAULT_TYPES	crates/ignore/src/default_types.rs	The static table of pre-defined file types: ~270 of them at the moment.

Ignore precedence

When the walker hits a path, it asks each matcher in priority order. The first matcher that returns Ignore(...) or Whitelist(...) wins:

graph TD
    Path[path candidate] --> Override["1. Overrides (--glob)"]
    Override -->|no match| Types["2. Types (-t/-T)"]
    Types -->|no match| GlobalGitignore["3. Global gitignore (core.excludesFile)"]
    GlobalGitignore -->|no match| LocalIgnore["4. .gitignore / .ignore / .rgignore in this dir + ancestors"]
    LocalIgnore -->|no match| Hidden["5. hidden file rule"]
    Hidden -->|no match| Allow[walked]
    Override -->|whitelist| Allow
    Types -->|whitelist| Allow
    GlobalGitignore -->|whitelist or ignore| Decision[final]
    LocalIgnore -->|whitelist or ignore| Decision
    Hidden -->|ignore| Excluded[skipped]

The ordering is implemented in crates/ignore/src/dir.rs. A whitelist match in an earlier matcher prevents a later matcher from ignoring the path, which is what makes --glob '!build/keepme' work as expected.

Parallel walking

WalkParallel::run(visitor) spawns a thread pool (default = num CPUs) and uses crossbeam channels to distribute paths. Each worker thread pulls a directory path, lists its contents, applies the per-directory ignore stack, and:

• Pushes child directories back into the work queue.
• Calls the visitor closure on each file.

The visitor returns a WalkState that controls the descent and termination.

ripgrep uses WalkParallel for both --files mode (just listing paths) and the search mode (searching each file as it's discovered). The closures live in crates/core/main.rs::files_parallel and search_parallel.

Watching for explosions

The walker has a few important defensive features:

• Symlink loops: detected by tracking inodes/file IDs (Unix) or full canonical paths (Windows). Configured via WalkBuilder::follow_links.
• Max depth: WalkBuilder::max_depth(N) caps recursion, useful for rg --max-depth.
• Max filesize: WalkBuilder::max_filesize(bytes) skips files above the threshold during ignore-stage filtering, before opening.
• Per-thread directory cache: on Linux, walkdir reuses the directory file descriptor for fstatat, avoiding repeated path canonicalisation. This is one of the reasons ripgrep is fast on huge trees.

Notable bug history

The CHANGELOG cites a long string of fixes around gitignore precedence, especially across parent directories:

• 14.1.0 (Jan 2024): Fixed unbounded memory growth in the ignore crate (BUG #2664).
• 15.0.0 (Oct 2025): Fixed BUG #829 / #2731 / #2747 / #2770 / #2778 / #2836 / #2933 / #3067, all related to gitignores from parent directories.
• 15.0.0: Fixed BUG #2750 — memory regression for very large gitignore files.

The precedence rules are subtle. When making changes to gitignore handling, read the existing tests in gitignore.rs and walk.rs carefully.

Default file types

crates/ignore/src/default_types.rs is a single static table that defines the default -t types. Each entry pairs a name ("rust") with a list of globs (["*.rs"]). Adding a new file type usually means a one-line addition there. The CHANGELOG includes "added or improved file type filtering for X" entries on most releases.

Integration points

• Used by: crates/core/flags/hiargs.rs (the binary's WalkBuilder constructor), and any external project that wants gitignore-aware traversal.
• Depends on: globset (for glob matching), walkdir, crossbeam-deque, crossbeam-channel, same-file (for symlink loop detection on POSIX), bstr, regex, log.

Entry points for modification

• New ignore semantics: changes likely belong in dir.rs (the precedence stack) or walk.rs (when files are filtered).
• New default file type: add an entry to default_types::DEFAULT_TYPES.
• New WalkBuilder option: add a config field, plumb through dir.rs if it affects per-directory matching.

For the glob matcher this crate uses internally, see globset. For the user-facing description of how filtering works, see features: gitignore handling and features: file typing.