astral-sh/uv

uv-client

crates/uv-client/ is uv's HTTP layer: a cached, retrying, TLS-aware client for talking to PyPI-compatible registries. It also handles the Simple HTML/JSON index format, flat indexes, and remote metadata files.

Purpose

uv has to talk to package indexes a lot — finding versions, fetching metadata, downloading wheels and sdists, hitting the PEP 658 metadata files when available. This crate provides:

A configured reqwest::Client with retries, redirect limits, native TLS or rustls, authentication injection, and a custom User-Agent that matches PyPI's analytics expectations.
A cached client wrapper that stores zstd-compressed rkyv archives of registry responses keyed by URL.
HTML and JSON parsers for the PyPI Simple API (PEP 503 / 691).
Flat-index support (a directory of wheels exposed as a URL).
Remote metadata reading for PEP 658 (sidecar metadata files) and PEP 691 (JSON index).

Directory layout

crates/uv-client/src/
├── lib.rs                 # Re-exports
├── base_client.rs         # The shared reqwest client builder (TLS, retries, auth, redirects)
├── cached_client.rs       # CachedClient: read-through cache with rkyv archives
├── registry_client.rs     # RegistryClient: PEP 503/691 search + wheel/sdist fetch
├── flat_index.rs          # Flat directory/URL index parser
├── html.rs                # PEP 503 HTML index parser (the larger of the two)
├── error.rs               # ClientError, BuildError, ErrorKind, status-code mapping
├── linehaul.rs            # Linehaul User-Agent matching pip's analytics format
├── middleware.rs          # tower-style middleware wiring (retries, auth)
├── retry.rs               # ExponentialBackoff retry policy
├── tls.rs                 # TLS configuration: rustls + webpki-root-certs, native fallback
├── remote_metadata.rs     # Fetching PEP 658 sidecar metadata
├── rkyvutil.rs            # OwnedArchive helpers for memory-mapped rkyv reads
└── httpcache/             # Cache key / freshness primitives shared with uv-cache

Key abstractions

Type	File	Role
`BaseClient` / `BaseClientBuilder`	`base_client.rs`	Wrapped `reqwest::Client` with retries, auth, TLS, and the linehaul User-Agent.
`RegistryClient` / `RegistryClientBuilder`	`registry_client.rs`	High-level API: list versions, fetch metadata, download wheel / sdist, with cache integration.
`CachedClient`	`cached_client.rs`	Read-through HTTP cache that stores responses as zstd-compressed rkyv archives in the cache.
`RetryState`	`retry.rs`	Retry budgeting against `ExponentialBackoff`.
`RetryParsingError`	`cached_client.rs`	Specific error type for unrecoverable parse failures of cached archives.
`MetadataFormat`	`registry_client.rs`	Whether the wire format is HTML, JSON, or a sidecar metadata file.
`OwnedArchive<T>`	`rkyvutil.rs`	Owns the bytes of an rkyv archive so callers can hold zero-copy references to its contents.
`RequestBuilder`	`cached_client.rs`	Wrapper around `reqwest::RequestBuilder` that participates in the cache.
`Linehaul`	`linehaul.rs`	The structured User-Agent header pip uses, replicated so PyPI can categorize uv's traffic correctly.
`Tls` configuration	`tls.rs`	Resolves cert sources: native CA store via `rustls-native-certs`, bundled `webpki-root-certs`, or `--allow-insecure-host` exceptions.

How it works

sequenceDiagram
    participant Resolver as uv-resolver
    participant Distro as uv-distribution
    participant Registry as RegistryClient
    participant Cached as CachedClient
    participant Cache as ~/.cache/uv (uv-cache)
    participant Net as PyPI / mirror

    Resolver->>Distro: get versions for `requests`
    Distro->>Registry: simple_api(`requests`)
    Registry->>Cached: GET https://pypi.org/simple/requests/
    Cached->>Cache: read simple-vN/<hash>
    alt fresh in cache
        Cache-->>Cached: zstd-rkyv archive
        Cached-->>Registry: deserialize -> SimplePackage
    else stale or missing
        Cached->>Net: HTTP GET (with auth + retry middleware)
        Net-->>Cached: HTML or JSON body
        Cached->>Cached: parse + serialize to rkyv
        Cached->>Cache: write zstd-compressed archive
        Cached-->>Registry: SimplePackage
    end
    Registry-->>Distro: versions list
    Distro-->>Resolver: VersionsResponse

TLS

tls.rs builds the rustls configuration. By default uv uses rustls + webpki-root-certs. --native-tls (deprecated as of #18705 in Apr 2026) switched to rustls-native-certs to read the OS trust store. --allow-insecure-host can disable verification for specific hosts (with a loud warning per STYLE.md security expectations).

Retries

retry.rs implements exponential backoff matching astral-reqwest-retry's policy. Most unauthorized responses are retried up to a small bound; 5xx responses trigger backoff; connection-level errors are retried more aggressively.

Caching

CachedClient::get_serde is the workhorse. It:

Computes a cache key from the URL (crates/uv-cache/src/).
Checks for a cached archive. If present and fresh (according to HTTP cache semantics or uv's Refresh policy), it deserializes it and returns.
Otherwise, makes the HTTP request, parses the body via the caller-provided callback, serializes the result with rkyv, compresses with zstd, and writes the archive atomically.

Because the result is rkyv, subsequent reads can memory-map the file and access fields without allocating — particularly important for large indexes like PyPI's numpy page.

The httpcache/ subdirectory implements RFC 7234 cache control (max-age, etags) on top of uv's filesystem cache.

Linehaul

linehaul.rs reproduces pip's structured User-Agent header. PyPI uses Linehaul to categorize download traffic by tool, OS, Python version, etc. Replicating it means uv's downloads show up correctly in the pypistats dashboards rather than as unattributed "other" traffic.

HTML and JSON index parsing

html.rs (~65 KB) parses the PEP 503 Simple HTML index — a deceptively complex format because real-world indexes (Devpi, Artifactory, Nexus, AWS CodeArtifact, GitLab) all interpret it slightly differently. The parser is deliberately lenient about whitespace and quoting but strict about the actual link semantics. JSON parsing for PEP 691 is a much smaller serde-based implementation in the same module.

Integration points

uv-distribution is the primary consumer. It uses RegistryClient to list versions and fetch metadata, then converts the result into Distribution types.
uv-cache owns the on-disk layout. CachedClient writes into the appropriate bucket (simple-v18, wheels-v6, …).
uv-auth plugs into BaseClient via reqwest middleware. Credentials are resolved per-realm, optionally refreshed via the keyring or trusted-publishing flows.
uv-redacted is used to strip credentials from any URL we log or display.

Entry points for modification

A new index format — registry_client.rs (MetadataFormat) + parsing in html.rs / remote_metadata.rs.
A new auth provider — most of the work is in uv-auth, but the middleware wiring is in middleware.rs.
A new cache freshness policy — cached_client.rs and the httpcache/ module.
TLS changes — tls.rs. Note that --native-tls and UV_NATIVE_TLS are deprecated; rustls + bundled roots is the recommended path.

Key source files

File	Purpose
`crates/uv-client/src/registry_client.rs`	`RegistryClient` and the high-level API.
`crates/uv-client/src/base_client.rs`	Shared reqwest client construction.
`crates/uv-client/src/cached_client.rs`	Read-through caching.
`crates/uv-client/src/html.rs`	PEP 503 Simple HTML parser.
`crates/uv-client/src/tls.rs`	TLS configuration.
`crates/uv-client/src/linehaul.rs`	Custom User-Agent.
`crates/uv-client/src/retry.rs`	Backoff policy.