Secret detection

The fundamental feature: scan content for hardcoded secrets using regex rules, optionally filtered by Shannon entropy and allowlists.

How a finding happens

A "finding" is the output of a successful match between a fragment of source code and one of the configured Rules. The flow:

graph TD
    Frag[Fragment from a Source] --> KW[Aho-Corasick prefilter:<br/>which rule keywords appear?]
    KW --> Rules[for each Rule]
    Rules --> Has{rule has keywords?}
    Has -->|yes| Hit{any keyword<br/>in fragment?}
    Has -->|no| Run[run regex]
    Hit -->|yes| Run
    Hit -->|no| Skip1[skip rule]
    Run --> Path{path filter passes?}
    Path -->|no| Skip2[skip]
    Path -->|yes| Match[regex matches]
    Match -->|secretGroup| S[extract secret]
    S --> Ent{entropy > threshold?}
    Ent -->|no| Skip3[skip finding]
    Ent -->|yes| AL[allowlist filters]
    AL -->|allowed| Skip4[skip finding]
    AL -->|not| F[Finding]

Implementation lives in Detector.detectRule (detect/detect.go). See the detect system page for the full pipeline.

Anatomy of a rule

A config.Rule (config/rule.go) is the unit of detection:

type Rule struct {
    RuleID        string             // unique identifier
    Description   string             // human-readable
    Entropy       float64            // optional Shannon entropy threshold
    SecretGroup   int                // capture group index for the secret
    Regex         *regexp.Regexp     // detection pattern (optional if Path is set)
    Path          *regexp.Regexp     // optional file path filter
    Tags          []string           // metadata for reporting
    Keywords      []string           // pre-regex filter strings
    Allowlists    []*Allowlist       // suppress false positives
    RequiredRules []*Required        // composite rule auxiliaries
    SkipReport    bool               // for auxiliaries: compute but don't emit findings
}

In TOML form (from the README):

[[rules]]
id = "beamer-api-token"
description = "Beamer API token"
regex = '''(?i)(?:beamer)(?:[\w\-\.]{0,30})?[\s]?[\=\:]?[\s]?["']?b_[a-z0-9=_\-]{44}'''
secretGroup = 1
entropy = 3.5
keywords = ["beamer"]
tags = ["api", "token"]

The keyword pre-filter

If you have ~220 rules and ~1MB of source code, naively running every regex against every byte would be wasteful. Gitleaks keeps things tractable with a single Aho-Corasick trie built from every keyword across every rule. The trie is consulted once per fragment in DetectContext (detect/detect.go):

matches := d.prefilter.MatchString(normalizedRaw)
for _, m := range matches {
    keywords[normalizedRaw[m.Pos():int(m.Pos())+len(m.Match())]] = true
}

Then for each rule, the regex only runs if at least one of the rule's keywords is in the resulting keywords map. Rules with no keywords are unconditional — that's the "generic credential" type pattern.

The trie comes from BobuSumisu/aho-corasick and is built once at Detector construction time:

prefilter: *ahocorasick.NewTrieBuilder().AddStrings(maps.Keys(cfg.Keywords)).Build(),

Path-only rules

A rule with Path set but no Regex matches based on filename alone. When detectRule sees a path-only rule, it constructs a Finding with Match: "file detected: <path>" and no secret/line content. Useful for sensitive filenames like *.pem or id_rsa. See detect/detect.go's detectRule.

Path + regex

A rule with both Path and Regex requires the path to match and the regex to match. The path serves as a coarse filter to skip irrelevant files entirely.

Secret extraction (secretGroup)

Often the regex captures more than just the secret — for example, an identifier prefix or a quote character. secretGroup is the (1-indexed) capture group containing the actual secret value. If unset, detectRule walks the capture groups in order and uses the first non-empty one.

The extracted secret is what:

• Goes into Finding.Secret
• Has its Shannon entropy checked
• Is the default target for allowlist regexes (unless regexTarget = "match" or "line")
• Is what --redact masks

Entropy filtering

Per-rule, Entropy: 3.5 (or any positive float) tells detectRule to compute shannonEntropy(secret) (detect/utils.go) and discard the finding if entropy is below the threshold. This kills the most common kind of false positive — the literal word password=password matched against a regex looking for password=….

The entropy is reported on every finding (Finding.Entropy), even when no threshold is set.

Generic vs. specific deduplication

After a fragment is scanned, filter() (detect/utils.go) dedupes findings: if a rule whose ID contains generic produced a finding on the same line as a more specific rule, and the specific finding's secret contains the generic finding's secret, the generic one is dropped. The trace log notes "skipping … finding, … rule takes precedence".

This means the default config's generic-api-key won't double-report a leak that another, more specific rule already caught.

Reporting findings

After a fragment is scanned, every accepted finding is passed to Detector.AddFinding, which:

1. Computes both the global fingerprint (<file>:<rule-id>:<line>) and the commit fingerprint (<commit>:<file>:<rule-id>:<line>).
2. Drops it if either matches the loaded .gitleaksignore.
3. Drops it if it matches a baseline entry (see baselines).
4. Appends to the detector's findings slice (under findingMutex).
5. If --verbose is on, pretty-prints to stdout.

After scanning completes, the configured Reporter writes the entire slice in JSON / CSV / SARIF / etc. (see systems/report).

Related

• Allowlists — how to suppress findings
• Composite rules — multi-part rules with proximity constraints
• Decoding — finding secrets inside base64/hex/percent-encoded text
• systems/detect — the package-level deep dive
• systems/config — how rules are parsed