istio/istio

VM workloads

Active contributors: costinm, hzxuzhonghu, ramaraochavali

What users do

Add VMs (or any non-Kubernetes process) into the mesh as first-class workloads. The VM:

Runs the same pilot-agent binary as is injected into pods.
Receives an mTLS identity from istiod (via JWT bootstrap with a per-VM token).
Is registered in the mesh as a WorkloadEntry, optionally auto-registered via a WorkloadGroup.

CRDs

CRD	Purpose
`WorkloadGroup`	Template for VM workloads in a logical group; defines labels, identity, ports
`WorkloadEntry`	A specific VM instance: address, labels, network, identity
`ServiceEntry` (`workloadSelector`)	Bind VM workloads as endpoints of a service

The auto-registration feature lets a VM register itself: it presents a JWT signed by a WorkloadGroup token and istiod creates a WorkloadEntry automatically.

How it works

graph LR
    vm[VM running pilot-agent] -->|mTLS bootstrap with WG token| istiod[istiod]
    istiod -->|create WorkloadEntry| api[kube-apiserver]
    api -->|watch| reg[ServiceEntry registry]
    reg -->|cross-feed| kreg[Kube registry]
    istiod -->|xDS| envoy[Envoy on VM]
    istiod -->|certs via SDS| envoy

Implementation:

Auto-registration — pilot/pkg/autoregistration/. When a pilot-agent connects with a JWT signed by a WorkloadGroup's SA, istiod creates a WorkloadEntry and tracks the connection. On disconnect, the WE is marked dead after a configurable grace period.
Health probes — pilot-agent on the VM periodically sends health updates to istiod. Implementation: pilot/pkg/autoregistration/internal/health/.
Sidecar bootstrap on VM — pilot-agent is configured via cluster.env and mesh.yaml files placed on the VM by an installer script. The script is generated by istioctl x workload entry configure.

ServiceEntry × VM

A ServiceEntry with workloadSelector matching VM labels treats those VMs as endpoints of the service:

apiVersion: networking.istio.io/v1
kind: ServiceEntry
metadata:
  name: my-vm-svc
spec:
  hosts: [my-vm-svc.local]
  ports:
    - number: 8080
      name: http
      protocol: HTTP
  resolution: STATIC
  workloadSelector:
    labels:
      app: my-vm-svc

Endpoints are populated from any WorkloadEntry (or auto-registered VM) with matching labels. VMs can also be selected by Kubernetes Services (the cross-feeding described in systems/service-discovery).

Trade-offs

VM lifecycle complexity — installing pilot-agent, projecting tokens, configuring routing, restarting on cert rotation. Operators usually package this into a config-management role (Ansible, Chef, systemd unit).
Identity bootstrap chicken-and-egg — the VM needs a JWT from somewhere. The WorkloadGroup pattern uses a Kubernetes Service Account token (one-shot), then long-lived mTLS once registered. Token distribution is the operator's problem.
Networking — the VM must reach istiod (port 15012). For private istiods, this means a load-balancer, peering, or VPN.
Less-tested path — sidecar-on-VM is supported but not the focus of recent development. Several VM-specific bugs have been long-running.

Key source files

File	Purpose
`pilot/pkg/autoregistration/controller.go`	Auto-registration controller
`pilot/pkg/autoregistration/internal/health/`	VM health probe handling
`pilot/pkg/serviceregistry/serviceentry/controller.go`	ServiceEntry / WorkloadEntry registry
`istioctl/pkg/workload/workload.go`	`istioctl x workload entry configure`
`tools/packaging/`	Distro packages for pilot-agent (RPM, deb)