RPC

Active contributors: knqyf263, simar7

Purpose

pkg/rpc/ and the top-level rpc/ directory together implement the wire protocol between trivy client and trivy server. The wire format is Twirp: an HTTP+protobuf RPC framework with a smaller surface area than gRPC and full HTTP/1.1 compatibility. There are two services: Cache (blob upload) and Scanner (scan request).

Directory layout

rpc/                              # generated stubs and proto definitions
├── cache/
│   ├── service.proto             # Cache service
│   ├── service.pb.go             # generated protobuf
│   └── service.twirp.go          # generated Twirp client+server
├── scanner/
│   ├── service.proto             # Scanner service
│   ├── service.pb.go
│   └── service.twirp.go
└── common/
    ├── service.proto             # shared messages
    └── service.pb.go             # generated; ~3,500 lines

pkg/rpc/
├── client/
│   ├── client.go                 # remote.Service implementing scan.Backend
│   └── headers.go                # auth + tracing headers
├── server/
│   ├── server.go                 # registers Twirp handlers
│   └── listen.go                 # HTTP listener + middleware
├── convert.go                    # in-process ↔ protobuf conversions (~36k lines)
└── retry.go                      # retry policy on transient errors

Key abstractions

Symbol	File	Purpose
Scanner Twirp service	rpc/scanner/service.proto	Single Scan RPC.
Cache Twirp service	rpc/cache/service.proto	PutArtifact, PutBlob, MissingBlobs, DeleteBlobs.
client.NewClient	pkg/rpc/client/client.go	Returns a scan.Backend backed by Twirp.
server.NewServer	pkg/rpc/server/server.go	Registers Twirp handlers and returns an http.Handler.
server.Listen	pkg/rpc/server/listen.go	HTTP listener with timeout, gzip, and auth middleware.
rpc.Convert*	pkg/rpc/convert.go	In-process ↔ protobuf conversions for every report shape.
rpc.Retry	pkg/rpc/retry.go	Wraps client calls with retry on transient HTTP errors.

Two services, two client roles

graph TB
    subgraph Client[trivy client]
        Art[artifact.Artifact]
        RemoteCache[cache.RemoteCache]
        RemoteBackend[scan.Backend client]
    end
    subgraph Server[trivy server]
        CacheHandler[Cache Twirp handler]
        ScannerHandler[Scanner Twirp handler]
        FSCache[FSCache / Redis]
        Local[local.Service]
    end
    Art -->|PutBlob/PutArtifact via Twirp| CacheHandler
    Art -->|MissingBlobs| CacheHandler
    RemoteBackend -->|Scan| ScannerHandler
    CacheHandler --> FSCache
    ScannerHandler --> Local
    Local --> FSCache

The two services are deliberately split:

• The cache service is read/write — clients upload blobs, query missing blobs, and (rarely) delete blobs.
• The scanner service is write/respond — clients ask the server to scan an artifact they already uploaded.

This split means clients only need to ship blobs the server hasn't seen, which makes scans of common base images very fast.

convert.go

pkg/rpc/convert.go is, at ~36,000 lines, the largest hand-maintained file in the repo. It contains symmetric conversions between Go types in pkg/types/ and the protobuf-generated types in rpc/common/. Because every field in the report passes through this file, adding a new field is a five-step process:

1. Add it to the relevant .proto file in rpc/.
2. Regenerate via buf generate.
3. Add it to the matching Go struct in pkg/types/ (if it isn't already there).
4. Add a conversion clause in both directions in pkg/rpc/convert.go.
5. Add round-trip tests in pkg/rpc/convert_test.go.

Authentication

The HTTP listener in pkg/rpc/server/listen.go supports a shared-secret token check. Clients pass --token (and optionally --token-header); servers compare against --token of their own. This is suitable for trusted networks; for production exposure use mTLS at the listener (--listen-tls --cert-file --key-file) or front the server with an authenticated proxy.

Retries

pkg/rpc/retry.go retries on transient HTTP errors (5xx, connection reset). Configurable via the standard cenkalti/backoff package.

Code generation

buf.gen.yaml and buf.yaml configure protobuf generation:

buf generate

The generated files are committed to keep go build self-contained. Regeneration is part of mage build:rpc (see magefiles/). Do not edit *.pb.go or *.twirp.go by hand.

Integration points

• Scan service — the server's Scanner.Scan ultimately calls local.Service.Scan.
• Cache — the server's Cache.* ultimately calls a real Cache backend.
• CLI — --server/--token/--listen flags wire up RPC.

Entry points for modification

• Add a new RPC method — extend the .proto file, regenerate, then add server handler and client wrapper.
• Add observability — middleware in listen.go. Twirp clients also have hooks for tracing.
• Change retry behavior — pkg/rpc/retry.go.

Key source files

File	Purpose
rpc/scanner/service.proto	Scanner contract.
rpc/cache/service.proto	Cache contract.
rpc/common/service.proto	Shared messages (Vulnerability, Misconfiguration, etc.).
pkg/rpc/convert.go	Wire-format conversion.
pkg/rpc/client/client.go	Client backend for scan.Backend.
pkg/rpc/server/server.go	Server handler registration.
pkg/rpc/server/listen.go	HTTP listener and middleware.
pkg/rpc/retry.go	Retry policy.
buf.gen.yaml, buf.yaml	Protobuf generation config.

See also

• Server — user-facing surface.
• Cache — what the cache service ultimately drives.