Dynamic configuration

Active contributors: david, alex, yichaoyang

Purpose

common/dynamicconfig is the runtime-tunable configuration system. Anything that an operator might want to flip without restarting the cluster — rate limits, feature flags, queue sizes, sampling rates — comes from here. There are over 600 dynamic-config keys in the codebase today.

Directory layout

common/dynamicconfig/
├── client.go                  # Client interface
├── config.go                  # Per-service config struct (defaults + accessors)
├── constants.go               # Every dynamic-config key ever defined (long file)
├── file_based_client.go       # File-based config (the dev / OSS default)
├── nop_client.go              # No-op client (test default)
├── setting.go                 # NewGlobal{Int,Bool,String,Duration}Setting helpers
├── ...
└── (per-package keys: matching.go, history.go, frontend.go ...)

Key abstractions

Type	What it is
Client (common/dynamicconfig/client.go)	The pluggable backend. Defaults: file-based reading from dynamicconfig.yaml.
Setting[T] (common/dynamicconfig/setting.go)	A typed handle to one config key. Created via NewGlobalIntSetting(...) etc.
Per-namespace, per-task-queue, per-shard variants	The setting type knows how to scope itself.

A typical declaration:

// in common/dynamicconfig/constants.go
var MatcherForwarderTimeout = NewTaskQueueDurationSetting(
    "matching.matcherForwarderTimeout",
    100*time.Millisecond,
    "How long the matcher waits before giving up on forwarding to a parent partition",
)

A consumer:

timeout := s.config.MatcherForwarderTimeout(namespace, taskQueue)

The setting type carries its own scope — TaskQueueDurationSetting in this example — so the override key in the YAML file looks like:

matching.matcherForwarderTimeout:
  - constraints:
      taskQueueName: 'my-tq'
    value: '50ms'
  - value: '100ms'

How config flows

graph LR
    YAML[dynamicconfig.yaml] --> File[file_based_client]
    File --> Cache[in-process cache]
    Cache --> Setting[Setting[T]]
    Setting --> Service[Frontend / History / Matching / Worker]

• The file-based client polls the YAML file periodically and updates the cache.
• Setting[T] reads from the cache on every call — there is no need to "restart" or reload anything in service code.
• Default values are baked into the setting declaration; if the YAML has no entry, the default applies.

Validation

The gendynamicconfig tool validates that every key in YAML files has a known declared Setting and the right type. The server's validate-dynamic-config CLI subcommand (cmd/server/main.go) runs the same validator and is meant to be invoked in CI before deploying a new config file.

Production backends

Operators can supply their own Client (e.g. one backed by an internal config service). The standard contract is read-only from the server's perspective; the client does the work of updating the cache.

Common patterns

• Feature flags — boolean settings with safe-by-default. Search for EnableXxx in common/dynamicconfig/constants.go.
• Per-namespace overrides — many quotas and timeouts can be scoped per namespace, so a single noisy tenant can be tuned in isolation.
• Sampling rates — used by metrics and logging to keep volume manageable.

Entry points for modification

• Adding a key: declare it in common/dynamicconfig/constants.go (or in the package-specific file) using the right scope helper. Use a safe default. The next gendynamicconfig run will pick it up.
• Adding a scope — rare. Existing scopes (global, namespace, task-queue, destination) cover almost every case.

Related pages

• Reference → configuration — every config knob the server understands.
• How to contribute → patterns