sharkdp/fd
Architecture
fd is a single-binary Rust application. It has no daemon, no server, and no plugin system. The whole program is one process that parses command line options, walks the filesystem in parallel, applies a chain of filters to each entry, and either prints matches or runs commands against them.
This page describes the major components in src/ and how a search request flows through them.
High-level flow
graph TD
user[User shell] -->|argv| main[fn main / fn run<br/>src/main.rs]
main -->|parse| cli[Opts<br/>src/cli.rs]
main -->|build patterns| regex_helper[regex_helper.rs]
main -->|construct| config[Config<br/>src/config.rs]
main -->|scan paths, regexps, config| walk[walk::scan<br/>src/walk.rs]
walk -->|WalkBuilder + WalkParallel| ignore_crate[ignore crate]
walk -->|spawn senders| senders[Worker threads<br/>spawn_senders]
senders -->|Batch of WorkerResult| chan[crossbeam-channel]
chan -->|receiver| receive[receive / ReceiverBuffer]
senders -->|filter| filters[Filters:<br/>FileTypes, extensions,<br/>SizeFilter, TimeFilter,<br/>OwnerFilter, ignore_contain]
receive -->|no command| stdout[Print to stdout<br/>src/output.rs]
receive -->|--exec / --exec-batch| exec[CommandSet::execute<br/>src/exec/mod.rs]
exec --> children[Child processes via argmax::Command]Components
| Module | File | Responsibility |
|---|---|---|
| Entry point | src/main.rs |
fn main and fn run. Parses Opts, builds regular expressions, constructs Config, calls walk::scan. Also picks tikv-jemallocator as the global allocator on supported targets. |
| CLI definition | src/cli.rs |
The Opts struct (clap Parser), all flag definitions, the custom Exec parser for -x/-X, enums (FileType, ColorWhen, HyperlinkWhen, StripCwdWhen), and helpers like default_num_threads. |
| Configuration | src/config.rs |
The Config struct that bundles every effective setting after CLI parsing and smart-case detection. Read-only at search time. |
| Regex helpers | src/regex_helper.rs |
pattern_has_uppercase_char (smart case detection) and pattern_matches_strings_with_leading_dot (the "you probably need --hidden" hint). |
| Walker | src/walk.rs |
The parallel walker. Wraps ignore::WalkBuilder/WalkParallel, spawns sender threads that filter entries, and runs a ReceiverBuffer that buffers/streams output or hands results to the exec subsystem. |
| Directory entry | src/dir_entry.rs |
DirEntry wraps either a normal ignore::DirEntry or a broken symlink path, lazily caches metadata and lscolors::Style. |
| File-type filtering | src/filetypes.rs |
The FileTypes struct used for -t/--type. |
| Filters | src/filter/ |
SizeFilter (-S), TimeFilter (--changed-within/--changed-before), OwnerFilter (-o, Unix only). |
| Format templates | src/fmt/ |
The FormatTemplate and Token types used for both --format output and for --exec placeholder expansion ({}, {/}, {//}, {.}, {/.}). |
| Command execution | src/exec/ |
CommandSet (one-by-one and batch modes), CommandTemplate, CommandBuilder (uses argmax::Command to stay below ARG_MAX), and the job/batch receiver functions. |
| Output | src/output.rs |
Path printing with optional colors, hyperlinks, custom path separators, and trailing slashes for directories. |
| Hyperlinks | src/hyperlink.rs |
OSC 8 file:// URL construction with percent encoding. |
| Filesystem helpers | src/filesystem.rs |
Cross-platform helpers: absolute paths, is_existing_directory, file-type predicates, OS-string-to-bytes conversion, and default_path_separator for MSYS environments. |
| Exit codes | src/exit_codes.rs |
The ExitCode enum (Success, HasResults, GeneralError, KilledBySigint) with merge_exitcodes and a Unix re-raise of SIGINT. |
| Errors | src/error.rs |
A single print_error helper that writes [fd error]: ... to stderr. |
Lifecycle of a search
- Parse arguments.
Opts::parse()(clap derive) builds the option struct. If--gen-completionsis set,print_completionswrites a clap-generated completion script and exits. - Resolve search paths.
set_working_dirhonours--base-directory, thenopts.search_paths()collects positional paths or defaults to the current directory. - Build regular expressions. Each pattern (the main one plus any
--andexpressions) is converted bybuild_pattern_regex. Globs are translated throughglobset::GlobBuilder;--fixed-stringsruns throughregex::escape. The actualRegexobjects are built later, after smart-case detection, bybuild_regex. - Construct
Config.construct_configderives effective settings: smart case sensitivity (viapattern_has_uppercase_char), path separator, time/size/owner constraints, color decision (terminal detection plusNO_COLOR),LsColors(env or the bundledDEFAULT_LS_COLORS), hyperlink mode, file-type set, extensions (compiled into aRegexSet),--formattemplate, and theCommandSetfor--exec/--exec-batch/--list-details. - Sanity checks.
ensure_search_pattern_is_not_a_pathrejects patterns like/etc/passwd(without--full-path) andensure_use_hidden_option_for_leading_dot_patternwarns when a pattern only matches dotfiles but--hiddenwas not passed. - Walk.
walk::scanconstructs aWorkerState, builds anignore::WalkParallelwalker that respects ignore files and--exclude, spawns one sender closure per worker thread (spawn_senders), and runs a receiver in the same thread scope. - Filter and forward. Each sender filters an entry through:
ignore_contain(skip directory if it contains a marker file), depth limits, the conjunction of pattern regexes against either the basename or the full path, the extensionRegexSet,FileTypes, owner constraints, size constraints, and time constraints. Surviving entries are batched into aBatchand pushed onto a boundedcrossbeam-channel. - Receive.
ReceiverBufferstarts inBufferingmode, accumulating entries until either the buffer hits 1000 items ormax_buffer_time(default 100 ms) elapses. It then sorts the buffered batch and switches toStreaming, writing each subsequent entry directly tostdout. - Or execute. When
--exec/--exec-batch/--list-detailsis set, the receiver path goes throughexec::job(one-by-one in a thread pool) orexec::batch(single argv built up viaargmax::Command::args_would_fit). - Exit.
ExitCode::exitreturns0for success,1for general errors (or--quietwith no match),1from the OR-reduced exit codes of child commands, or130after re-raisingSIGINTon Unix.
Concurrency model
- Sender side.
WalkParallel::run(from theignorecrate) createsconfig.threadsworker threads. Each worker runs the closure built inspawn_senders. The number of threads defaults tostd::thread::available_parallelism()clamped at 64 (default_num_threadsinsrc/cli.rs). - Channel. Senders push
Batchvalues onto a bounded channel of capacity2 * threads, defined inWorkerState::scan. EachBatchholds up to 256 results, or 1 result when running in per-result--execmode with multiple threads (so batches are spread across exec workers). - Receiver side. Without
--exec, a single receiver thread runsReceiverBuffer::process. With--execandOneByOnemode, a thread-scoped pool ofconfig.threadsworker threads each consume the channel and run one command per result. With--exec-batch, a single thread collects all results, thenCommandBuilderflushes them in chunks that fit insideARG_MAX. - Cancellation. A shared
quit_flag: Arc<AtomicBool>is set by the receiver when it stops (e.g.,--max-resultsreached or stdout broken pipe). A separateinterrupt_flagis set by actrlchandler; pressing Ctrl-C twice exits immediately, otherwise the walker drains gracefully and the process re-raisesSIGINTon Unix to preserve the right exit status.
Global allocator
On most targets, fd uses tikv-jemallocator as the #[global_allocator] (see src/main.rs). Jemalloc is excluded on Windows, Android, macOS, FreeBSD, OpenBSD, illumos, 32-bit musl, and riscv64. The choice is made for performance (see PR #481 and the comment in Cargo.toml).
External crates that do most of the heavy lifting
| Crate | Used for |
|---|---|
clap (with derive, wrap_help, cargo) |
Command-line parsing and help generation. |
ignore |
Parallel directory walking that respects .gitignore, .ignore, .fdignore, and a global ignore file. |
globset |
Translating --glob patterns into regular expressions. |
regex and regex-syntax |
Pattern matching against file names and AST inspection for smart-case and leading-dot detection. |
crossbeam-channel |
Bounded channels between senders and the receiver. |
lscolors and nu-ansi-term |
LS_COLORS parsing and ANSI styling. |
argmax |
A Command wrapper that knows the platform ARG_MAX so batch exec never overflows. |
jiff |
Date and duration parsing for --changed-within / --changed-before (replaced chrono and humantime in 10.3.0). |
etcetera |
Locating the global ignore file in the platform's config directory. |
normpath |
Path normalisation, especially on Windows. |
nix (Unix), libc, faccess |
Unix-specific syscalls for owner filtering, signal handling, hostnames, and executable detection. |
ctrlc |
Cross-platform SIGINT handler. |
For a full dependency list see Cargo.toml and reference/dependencies.
Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.