Command execution

Active contributors: tmccombs, sharkdp, Michael Aaron Murphy, Jonah Caplan

Purpose

-x/--exec runs a command for each search result; -X/--exec-batch runs a command once with all results as arguments. This subsystem owns both modes, the placeholder template engine they share with --format, the argmax-based argv builder that prevents ARG_MAX overflows, and the per-thread receivers that drive child processes from the walker's output channel.

Directory layout

src/
└── exec/
    ├── mod.rs       # CommandSet, CommandTemplate, CommandBuilder, ExecutionMode
    ├── command.rs   # execute_commands, OutputBuffer (per-result stdio buffering)
    └── job.rs       # job() and batch() — receivers that consume WorkerResults

src/fmt/mod.rs provides the placeholder engine; this page links to format-templates for the substitution details and focuses on the execution side.

Key abstractions

Type	File	Purpose
ExecutionMode	src/exec/mod.rs	OneByOne or Batch.
CommandSet	src/exec/mod.rs	A non-empty set of CommandTemplates with a single mode. The Exec parser in src/cli.rs constructs this from --exec/--exec-batch argv.
CommandTemplate	src/exec/mod.rs	A parsed argv whose elements are each a FormatTemplate. Gains an implicit trailing {} if no token was used.
CommandBuilder	src/exec/mod.rs	Stateful builder for batch mode: holds the argmax::Command, the path-template argument, and any post-args that come after the placeholder.
OutputBuffer	src/exec/command.rs	Per-result stdout/stderr buffer used when threads > 1 so output from concurrent commands does not interleave.
job / batch	src/exec/job.rs	The receiver-side functions called from WorkerState::receive.

Execution modes

graph TD
    walker[Walker channel of WorkerResult] --> receive[WorkerState::receive]
    receive -->|cmd.in_batch_mode| batch[exec::batch]
    receive -->|otherwise| pool[thread::scope spawns N exec workers]
    pool --> job[exec::job per thread]
    batch --> CB[CommandBuilder<br/>per CommandTemplate]
    CB -->|push paths until ARG_MAX| spawn[Command.status]
    job -->|generate argv per result| exec_cmds[execute_commands]
    exec_cmds -->|threads > 1| buffered[Buffered stdout/stderr,<br/>flushed under stdout lock]
    exec_cmds -->|threads == 1| inherit[Inherited stdio<br/>for live output]

OneByOne (-x)

exec::job (in src/exec/job.rs) runs in each of config.threads threads spawned by WorkerState::receive. For each WorkerResult::Entry, it calls CommandSet::execute, which generates one Command per CommandTemplate and runs them sequentially via execute_commands (src/exec/command.rs). The exit code of the worker thread is the OR-merged exit code of all commands it ran.

• When config.threads == 1, output is not buffered — the child inherits stdin/stdout/stderr so users can interact (e.g., fd ... -x vim opens the editor connected to the user's terminal).
• When config.threads > 1, each child is output()-ed (waited on with full output captured), and stdout/stderr are appended into an OutputBuffer. After all commands for a result are done, the buffer is flushed under a single stdout/stderr lock so output blocks are atomic and don't garble. --print0 adds a \0 terminator after the per-result block.

Batch (-X)

exec::batch consumes the entire WorkerResult stream and feeds it through one CommandBuilder per CommandTemplate. The builder partitions the args into:

• pre_args — the literal arguments before the placeholder.
• path_arg — the single FormatTemplate containing the placeholder.
• post_args — anything after the placeholder.

For every incoming path it calls CommandBuilder::push. If the builder's argmax::Command::args_would_fit(...) says the next argv would exceed the platform ARG_MAX, or the --batch-size limit was hit, it finish()es the current command (appends post_args, runs cmd.status(), resets), then continues. After the whole stream is drained, a final finish() flushes any remaining args.

Batch-mode constraints

CommandSet::new_batch enforces:

• Exactly one placeholder per command (multiple {...} tokens are rejected with Only one placeholder allowed for batch commands).
• The first argv element must be a fixed executable: if cmd.args[0].has_tokens() { bail!("First argument of exec-batch is expected to be a fixed executable"); }.

These rules match the documented --exec-batch UX and rule out things like fd … -X {/} that would have no command to run.

CLI integration

Opts::exec is populated by the manual Exec parser in src/cli.rs. It collects each occurrence of --exec/--exec-batch (clap's get_occurrences::<String>) and maps the right one through CommandSet::new or CommandSet::new_batch. Multiple occurrences of --exec accumulate into multiple CommandTemplates, all run for each result.

construct_config in src/main.rs then calls extract_command(&mut opts, colored_output). If --exec/--exec-batch were not used but --list-details/-l was, fd synthesises a one-element batch CommandSet with ls -lhd --color=…, picking GNU gls on macOS/BSD when present. This is implemented in determine_ls_command (also src/main.rs).

The argmax integration

CommandBuilder holds an argmax::Command rather than a std::process::Command. The crate keeps a running tally of the bytes already added to argv and exposes args_would_fit(args) to ask "would adding these arguments overflow ARG_MAX?" without actually adding them. fd uses it like this:

fn push(&mut self, path: &Path, separator: Option<&str>) -> io::Result<()> {
    if self.limit > 0 && self.count >= self.limit {
        self.finish()?;
    }
    let arg = self.path_arg.generate(path, separator);
    if !self.cmd.args_would_fit(iter::once(&arg).chain(&self.post_args)) {
        self.finish()?;
    }
    self.cmd.try_arg(arg)?;
    self.count += 1;
    Ok(())
}

Both --batch-size (the user-facing knob) and the platform ARG_MAX are checked. The result: fd -e log -X grep -F panic works even on a directory with a million log files, transparently splitting into multiple grep invocations.

Output buffering

execute_commands in src/exec/command.rs is the inner loop:

for result in cmds {
    let mut cmd = match result { Ok(cmd) => cmd, Err(e) => return handle_cmd_error(None, e) };
    let output = if enable_output_buffering {
        cmd.output()
    } else {
        cmd.spawn().and_then(|c| c.wait_with_output())
    };
    match output {
        Ok(output) => {
            if enable_output_buffering {
                output_buffer.push(output.stdout, output.stderr);
            }
            if output.status.code() != Some(0) {
                output_buffer.write();
                return ExitCode::GeneralError;
            }
        }
        Err(why) => {
            output_buffer.write();
            return handle_cmd_error(Some(&cmd), why);
        }
    }
}
output_buffer.write();
ExitCode::Success

OutputBuffer::write takes the stdout and stderr locks once and writes all buffered bytes contiguously, keeping per-result output blocks intact.

Error handling

• If the executable is not found, fd prints [fd error]: Command not found: <program> and returns GeneralError.
• Other spawn or wait errors print [fd error]: Problem while executing command: ….
• Any non-zero child exit code propagates as GeneralError from the entire fd invocation, via merge_exitcodes over the per-thread/per-builder results.

Entry points for modification

• Add a new placeholder. Edit Token in src/fmt/mod.rs, the aho-corasick literal list in FormatTemplate::parse, the Display impl, and the match in FormatTemplate::generate. Add tests next to the existing token tests. The execution side does not need changes — it consumes FormatTemplate opaquely.
• Change batch-flush behaviour (e.g., a different size heuristic). Edit CommandBuilder::push in src/exec/mod.rs.
• Change output buffering rules. Edit execute_commands in src/exec/command.rs. The default of "buffer when threads > 1" is set in exec::job from config.threads > 1.
• Add a new built-in command (analogous to --list-details). Add a flag in src/cli.rs and synthesise the CommandSet in extract_command (src/main.rs).

Related pages

• Format templates — the engine that owns FormatTemplate/Token.
• Walker — the producer of the WorkerResult stream consumed here.
• CLI parsing — where Opts::exec is populated.