Decoding

Recursively decode base64, hex, percent, and unicode-escaped substrings inside fragments so rules can match the decoded value. Enabled by --max-decode-depth=N (default 5 in cmd/root.go's persistent flag).

When you'd use it

The most common case is environment files that contain URL-encoded credentials, or configuration files where a token has been base64-encoded to "obfuscate" it. Without decoding, a rule looking for AKIA[A-Z0-9]{16} won't find an AWS key hidden inside QUtJQUlPU0ZPRE5ON0VYQU1QTEU=.

Supported encodings

Encoding	Pattern (from detect/codec/encodings.go)	Notes
percent	%[0-9A-Fa-f]{2}(?:.*%[0-9A-Fa-f]{2})?	URL-style %XX. Highest precedence.
unicode	(?:U\+[a-fA-F0-9]{4}…)+ or \\u[a-fA-F0-9]{4}+	U+xxxx or \uxxxx
hex	[0-9A-Fa-f]{32,}	At least 32 hex chars
base64	[\w\/+-]{16,}={0,2}	At least 16 chars; standard or URL-safe alphabet

The order in the encodings slice determines precedence at overlap. See systems/codec for the full deep dive.

How it interacts with rules

graph TD
    Pass1[Pass 1: scan original Fragment.Raw] --> Find1[findings from undecoded text]
    Find1 --> Decode[decode encoded segments in place]
    Decode --> Pass2[Pass 2: scan spliced text]
    Pass2 --> Find2[findings tagged decoded:base64, decode-depth:1]
    Find2 --> Continue{more passes?<br/>depth ≤ MaxDecodeDepth and<br/>new segments?}
    Continue -->|yes| Decode
    Continue -->|no| End[exit]

The loop is in Detector.DetectContext (detect/detect.go):

ScanLoop:
for {
    // run rules on currentRaw

    currentDecodeDepth++
    if currentDecodeDepth > d.MaxDecodeDepth {
        break ScanLoop
    }
    currentRaw, encodedSegments = decoder.Decode(currentRaw, encodedSegments)
    if len(encodedSegments) == 0 {
        break ScanLoop
    }
}

So the depth counter increments after each rule pass, meaning depth 0 is "scan as-is", depth 1 is "scan after one decode pass", etc.

Finding adjustments

When a rule matches inside a decoded region, the codec helpers translate the offsets back to the original text:

• codec.SegmentsWithDecodedOverlap(segments, start, end) — returns the segments overlapping the match. If the match is entirely outside any segment, no adjustment is needed.
• codec.AdjustMatchIndex(segments, [start, end]) — maps decoded-text offsets back to original-text offsets so Finding.StartLine/StartColumn are correct in the user's actual file.
• codec.Tags(segments) — returns decoded:<encoding> and decode-depth:N tags appended to Finding.Tags.
• codec.CurrentLine(segments, currentRaw) — returns the decoded line for allowlist regex evaluation against RegexTarget = "line".

Output

A finding inside decoded text shows up like a normal finding, but the secret is the decoded value, the location is the bounds of the encoded text in the original (adjusted to include the encoded characters), and the tags include decoded:<kind> and decode-depth:<depth>.

Finding:     QUtJQUlPU0ZPRE5ON0VYQU1QTEU=
Secret:      AKIAIOSFODNN7EXAMPLE
Tags:        [aws decoded:base64 decode-depth:1]
File:        config/.env.prod
Line:        4

Notice that Match is the encoded substring (so users can find it in their file) but Secret is the decoded value (so it can be matched against allowlists, redacted, etc.).

Tuning

--max-decode-depth=0 disables decoding entirely for that scan. Higher values are usually safe — recursion stops automatically when no new encoded segments are found. The default is 5 (set in rootCmd.PersistentFlags().Int("max-decode-depth", 5, ...) in cmd/root.go).

Related

• systems/codec — package-level deep dive
• systems/detect — the scan loop that drives the decoder
• Archive scanning — a separate but conceptually similar recursion