istio/istio

Multi-cluster

Active contributors: hzxuzhonghu, costinm, ramaraochavali, keithmattix, howardjohn

What users do

Run a single mesh that spans multiple Kubernetes clusters. Three deployment models are supported:

Multi-primary — istiod in every cluster; clusters share a trust root and discover each other's services.
Primary-remote — istiod in one cluster (the "primary") drives data planes in remote clusters.
External istiod — istiod runs outside Kubernetes (e.g. on a VM) and controls one or more clusters.

The user-facing docs are at https://istio.io/latest/docs/setup/install/multicluster/. This page is the implementation map.

How it works

graph LR
    subgraph C1[cluster1 - primary]
        istiod1[istiod]
        sec1[Secret kubeconfig=c2<br/>label istio/multiCluster=true]
    end
    subgraph C2[cluster2]
        api2[kube-apiserver]
        wls2[Workloads]
    end
    sec1 -.kubeconfig.-> istiod1
    istiod1 -->|watch services/pods| api2
    istiod1 -->|xDS, multi-network| wls2
    wls2 -.-|HBONE / mTLS through<br/>east-west gateway| C1

Cluster discovery via secrets

pkg/kube/multicluster/secretcontroller.go watches Secrets in the istiod namespace with the label istio/multiCluster=true. Each secret holds a kubeconfig for a remote cluster. On secret create:

The controller builds a kube client.
Spawns a per-cluster kube.Controller (pilot/pkg/serviceregistry/kube/controller/).
Plumbs its events into the aggregate registry, tagged with the cluster ID.

istioctl create-remote-secret is the command that generates these secrets — see istioctl/pkg/multicluster/.

Network ID and the cross-network gateway

When clusters are on different networks (e.g. separate VPCs), traffic must hop through a gateway:

Each cluster declares its network ID (topology.istio.io/network label on namespace, or --set values.global.network=<id>).
MeshNetworks (the second key in the istio ConfigMap) maps each network to its east-west gateway address(es).
At xDS time, when an endpoint is on a different network, istiod points the proxy at the remote network's east-west gateway IP rather than the pod IP. The gateway terminates mTLS, switches networks, and re-originates to the destination.

This translation lives in pilot/pkg/networking/core/ for sidecars (filtering endpoints by network) and in the gateway compiler for the east-west gateway listener (port 15443 with SNI routing).

Endpoint slice merging

The aggregate model.ServiceDiscovery returns endpoints from all clusters. Each endpoint carries:

The cluster ID.
The network ID.
The locality (region/zone).
The original IP and port.

Locality-aware load balancing (per DestinationRule.localityLbSetting) prefers same-cluster, same-region, then crosses to other clusters when local endpoints fail. The tag topology.istio.io/cluster identifies the cluster in metrics and tracing.

Trust

For a mesh to span clusters, all istiods must sign with certs that chain to a common root. Two patterns:

Common root from operator — the user pre-creates a cacerts Secret in every cluster with intermediates signed by a single offline root.
Federated trust — each cluster's root is added to the others via MeshConfig.caCertificates.

The first is simpler; the second is for users who can't pre-distribute keys.

Multicluster Services (MCS)

pilot/pkg/serviceregistry/kube/controller/ understands the MCS API:

ServiceImport resources (from kubernetes-sigs/mcs-api) are merged into the local service catalog as if local.
The MCSAPI feature flag (PILOT_ENABLE_K8S_SELECT_WORKLOAD_ENTRIES) gates the older opt-in behavior.

Deployment topology

Multi-primary (recommended for HA):
  cluster1: istiod + workloads
  cluster2: istiod + workloads
  Both istiods watch each other's services via remote secrets.

Primary-remote:
  cluster1: istiod (primary)
  cluster2: workloads only; pilot-agents on workloads in cluster2 connect to istiod in cluster1.
  Requires the primary's istiod to be reachable from cluster2 (load-balancer, mesh.global, etc.).

External istiod:
  Anywhere: istiod
  cluster1, cluster2, …: workloads connect to the external istiod

The deployment topology is enforced at install time by selecting a profile (remote, external, default multicluster) and the appropriate Helm values.

Trade-offs

Latency vs single mesh — cross-cluster calls add a network hop; a multi-network mesh adds two (sidecar → east-west gateway → remote sidecar). Use localityLbSetting to keep traffic local where possible.
Trust complexity — a shared root is the simplest model but requires offline key handling. Per-cluster roots with federated trust are easier to operate but mean every cert presented in a cross-cluster handshake must validate through n roots.
Failure modes — istiod for cluster A talking to apiserver for cluster B: a network partition silently hides cluster B's services. Watchdog metrics (pilot_k8s_cfg_events, kube_remotes_metrics) are the canary.

Key source files

File	Purpose
`pkg/kube/multicluster/secretcontroller.go`	Secret-driven cluster discovery
`pilot/pkg/serviceregistry/kube/controller/multicluster.go`	Per-cluster controller spawn
`pilot/pkg/serviceregistry/aggregate/controller.go`	Cross-cluster aggregate
`istioctl/pkg/multicluster/`	`create-remote-secret`, `cluster list`, etc.
`pilot/pkg/networking/core/cluster_endpoints.go`	Per-network endpoint filtering
`pilot/pkg/networking/core/gateway.go`	East-west gateway listeners
`manifests/profiles/remote.yaml`	Remote cluster install profile
`manifests/profiles/external.yaml`	External-istiod profile