Filters

Active contributors: tmccombs, sharkdp, Alexandru Macovei, Shun Sakai

Purpose

Filters are the per-entry predicates the walker applies after ignore has decided to consider an entry. They cover file types, extensions, size, modification time, ownership, depth, and the --exclude/ignore_contain mechanics.

Directory layout

src/
├── filetypes.rs          # FileTypes (the -t bag of booleans)
└── filter/
    ├── mod.rs            # re-exports SizeFilter, TimeFilter, OwnerFilter
    ├── size.rs           # SizeFilter parsing and predicate
    ├── time.rs           # TimeFilter parsing and predicate
    └── owner.rs          # OwnerFilter (Unix only)

The walker calls them inline inside WorkerState::spawn_senders (src/walk.rs).

Key abstractions

Type	File	Purpose
FileTypes	src/filetypes.rs	A Default struct of booleans plus executables_only and empty_only gates. should_ignore(&entry) -> bool decides per-entry inclusion.
SizeFilter	src/filter/size.rs	enum { Max(u64), Min(u64), Equals(u64) } parsed by a single regex, with both SI (k, m, g, t) and binary (ki, mi, gi, ti) prefixes.
TimeFilter	src/filter/time.rs	enum { Before(SystemTime), After(SystemTime) } parseable from durations, datetimes, RFC3339 timestamps, or @<unix-seconds>. Backed by the jiff crate.
OwnerFilter	src/filter/owner.rs	A (uid, gid) constraint, each independently ignored, equal, or not-equal. Unix only.
extensions	src/config.rs (built in construct_config)	A case-insensitive regex::bytes::RegexSet of \.<ext>$ patterns.
exclude_patterns	src/config.rs	Forwarded to ignore::overrides::OverrideBuilder as !<glob> rules during walker setup.
ignore_contain	src/config.rs	List of marker filenames; directories that contain one are skipped entirely.

How they compose

The order in which filters run matters because some require metadata() (a syscall) and others don't:

graph TD
    entry[Entry from walker] --> ignore_contain[ignore_contain<br/>directory marker check]
    ignore_contain --> root[Skip if depth == 0]
    root --> min_depth[min_depth check]
    min_depth --> pattern[Pattern regex AND-conjunction]
    pattern --> ext[Extensions RegexSet]
    ext --> ftype[FileTypes::should_ignore]
    ftype --> owner[OwnerFilter.matches<br/>Unix only]
    owner --> size[Size constraints<br/>files only]
    size --> time[Time constraints]
    time --> emit[Emit WorkerResult::Entry]

The cheapest checks (depth, basename regex, extension RegexSet) run before any metadata access. FileTypes::should_ignore only inspects entry.file_type(), which is also cheap. The expensive metadata-reading checks (owner, size, time) come last.

File types (-t)

FileTypes (src/filetypes.rs) holds eleven booleans (file/dir/symlink/block-device/char-device/socket/pipe + executables_only + empty_only). When the user passes -t file -t dir, the relevant booleans are flipped on. should_ignore returns true if the entry is not one of the requested basic types, or if executables_only is set and the entry is not executable (via the faccess crate), or if empty_only is set and the entry is not empty.

A subtle behaviour: -t empty alone implies both files = true and directories = true, because "empty" without saying what would otherwise return nothing. This is handled in construct_config:

if file_types.empty_only && !(file_types.files || file_types.directories) {
    file_types.files = true;
    file_types.directories = true;
}

Size (-S/--size)

Parsed by SizeFilter::from_string against the regex ^([+-]?)(\d+)(b|[kmgt]i?b?)$:

Suffix	Multiplier
b	1
k/kb	1000
ki/kib	1024
m/mb	1,000,000
mi/mib	1,048,576
g/gb, gi/gib	10⁹ / 2³⁰
t/tb, ti/tib	10¹² / 2⁴⁰

The leading sign picks the variant: + → Min, - → Max, none → Equals. is_within(size) is the predicate. Multiple -S flags compose by AND — every constraint must match.

The walker only applies size constraints to files; directories and other types are rejected outright when any size constraint is set.

Time (--changed-within / --changed-before)

TimeFilter::from_str (src/filter/time.rs) tries four parsers in order, all from the jiff crate:

1. Span — duration like 30min, 1h30m, 2 days, 3 weeks, 4 months. Subtracted from "now".
2. Timestamp — RFC3339 like 2024-02-12T07:36:52+00:00.
3. DateTime — civil datetime like 2010-10-10 10:10:10 or 2010-10-10. Resolved via TimeZone::system().to_ambiguous_zoned(...).later().
4. @<unix-seconds> — explicit seconds-since-epoch.

applies_to(t) does the obvious less-than/greater-than against the SystemTime. --changed-within DURATION becomes After(now - DURATION); --changed-before DURATION becomes Before(now - DURATION).

The 10.3.0 release notes explicitly call out that M no longer means "month" (it would clash with minutes), and that month/year arithmetic is now calendar-aware rather than a hard-coded number of seconds.

For testability, TimeFilter::time::now() is #[cfg(test)]-replaced with a thread-local that the unit tests override.

Owner (-o/--owner, Unix only)

OwnerFilter::from_string("alice:wheel") splits on the single allowed :. Each side is parsed independently:

• Empty / missing → Check::Ignore.
• !name → Check::NotEq(...) after resolving the user/group via nix::unistd::User::from_name / Group::from_name.
• 123 → Check::Equal(123) directly (numeric uids/gids work too).
• name → Check::Equal(uid) after lookup.

matches(metadata) calls MetadataExt::uid/gid. Errors during parse (more than one :, unknown user) propagate as anyhow::Error. The convenience filter_ignore returns None when both sides are Ignore, which is how --owner ":" becomes a no-op.

Extensions (-e/--extension)

Multiple -e flags accumulate in Opts::extensions, which construct_config turns into a single case-insensitive regex::bytes::RegexSet of patterns like r".\.<escaped-ext>$". The walker matches this set against the basename. The leading . in the regex (.\.) ensures -e txt does not match files literally named .txt.

Exclude (-E/--exclude)

Opts::exclude becomes Config::exclude_patterns with a ! prefix prepended (so *.bak becomes !*.bak). WorkerState::build_overrides plugs them into ignore::overrides::OverrideBuilder, so the exclude logic is enforced by the ignore crate during traversal — not as a per-entry filter. Malformed patterns surface as anyhow!("Malformed exclude pattern: {}", e).

ignore-contain (10.4.0)

--ignore-contain CACHEDIR.TAG causes the walker to check, for each directory it visits, whether it contains the named entry. If so, that subtree is skipped. The check is in spawn_senders, before the rest of the filter chain, so unaffected directories pay no cost. The implementation honours the CACHEDIR.TAG specification by virtue of letting the user pick the marker name.

Entry points for modification

• A new constraint that uses metadata only. Add a FooFilter under src/filter/, expose it in Config, and wire it into the closure in WorkerState::spawn_senders after the existing metadata-using filters. Add unit tests next to the parser.
• A new constraint that does not need metadata. Insert it earlier in the chain to short-circuit before metadata is fetched.
• New file-type alias. Add a variant or #[value(alias = "...")] to FileType in src/cli.rs and wire it into the match arm in construct_config.
• New extension matching semantic (e.g. multi-component extensions like .tar.gz). Edit the regex construction in construct_config and the docstring on --extension.

Related pages

• Walker — the place where these filters are invoked.
• CLI parsing — where flags become Config fields.
• Ignore rules — the related but separate gitignore/fdignore mechanism.