temporalio/temporal

History service

Active contributors: wxing1292, alex, yichaoyang

Purpose

History owns workflow execution state. It is the largest, most stateful, and most complex service in the cluster. Every workflow's history events, mutable state, and pending tasks live behind a History shard. The full architectural deep-dive lives in docs/architecture/history-service.md; this page is a structural map of the source.

Directory layout

service/history/
├── service.go               # Service bootstrap
├── fx.go                    # fx wiring (16KB)
├── handler.go               # gRPC handler — every History RPC starts here (~87KB)
├── history_engine.go        # Per-shard engine (43KB)
├── history_engine_factory.go
├── chasm_engine.go          # CHASM execution engine (43KB)
├── chasm_notifier.go
├── workflow_rebuilder.go    # Workflow Reset support
├── statemachine_environment.go
│
├── api/                     # 52 sub-packages: one per public RPC group
├── archival/                # Archival queue task processing
├── circuitbreakerpool/      # Circuit breakers for outbound calls
├── configs/                 # Default config for History
├── consts/                  # Internal constants
├── deletemanager/           # Async workflow deletion
├── events/                  # Event-sourcing primitives
├── hsm/                     # Hierarchical State Machine library
├── historybuilder/          # Build new history events
├── interfaces/              # Internal interface definitions
├── ndc/                     # N-DC replication conflict resolution
├── queues/                  # Queue framework (transfer/timer/visibility/outbound)
├── replication/             # Replication tasks and stream processors
├── shard/                   # Shard ownership and controllers
├── tasks/                   # History task type registry
├── tests/                   # Cross-package integration tests
├── vclock/                  # Vector-clock helpers for replication
├── workflow/                # Mutable State, workflow context, locks
│   ├── mutable_state_impl.go  # The heart of the engine (~357KB)
│   ├── context.go
│   └── ...
└── transfer_queue_active_task_executor.go (and friends)
   timer_queue_active_task_executor.go     (active vs standby variants)
   visibility_queue_task_executor.go
   outbound_queue_active_task_executor.go
   archival_queue_task_executor.go

service/history/ alone holds 592 Go files — the largest single subsystem in the repo.

Key abstractions

Type / file	What it is
`Handler` (`service/history/handler.go`)	gRPC server. Routes every RPC to the right shard's `historyEngine`.
`historyEngineImpl` (`service/history/history_engine.go`)	Per-shard execution engine. One instance per shard owned by the host.
`MutableStateImpl` (`service/history/workflow/mutable_state_impl.go`)	In-memory + persisted summary of one workflow execution.
`workflow.Context` (`service/history/workflow/context.go`)	Per-execution mutex + cache slot; loads/saves MutableState.
`ShardController` (`service/history/shard/`)	Acquires/releases shards based on the membership ring; bumps `RangeID` on takeover.
`queues.Queue` (`service/history/queues/queue_base.go`)	Abstract queue with reader / executor / checkpointer goroutines.
`Executor` (`service/history/transfer_queue_active_task_executor.go` and siblings)	Concrete handler for one type of history task.
`chasm_engine.go` (`service/history/chasm_engine.go`)	Drives CHASM Library executions (workflow, scheduler, nexusoperation) inside a shard.

How it works

graph TD
    RPC[Inbound RPC] --> Handler
    Handler -->|locate shard owner| Engine[historyEngineImpl]
    Engine -->|load MS via Context| Workflow[workflow.Context]
    Workflow --> Persist[(persistence)]
    Engine -->|state transition| Tx[Atomic transaction]
    Tx -->|append events| Persist
    Tx -->|update MS| Persist
    Tx -->|insert tasks| Tx2[transfer / timer / visibility tables]
    Tx2 --> QP[Queue Processors]
    QP -->|transfer task| MatchingClient
    QP -->|timer task| TimerExecutor
    QP -->|visibility task| VisibilityStore
    QP -->|outbound task| Nexus
    QP -->|replication task| ReplicationStream
    QP -->|archival task| ArchiverProvider

The two halves of the engine — synchronous RPC handling, and asynchronous queue processing — share the same per-shard transactional contract. GetAndUpdateWorkflowWithNew in service/history/api/update_workflow_util.go is the shared utility most state transitions go through.

For the long-form treatment of RPC handling, queue processing, sharding, and consistency guarantees, read docs/architecture/history-service.md. It's roughly 24KB of carefully maintained prose with code references.

Active vs standby

Every executor has an active and a standby variant:

Active runs in the cluster that owns the namespace; it advances the workflow.
Standby runs in passive replicas; it verifies replicated state without driving forward execution.

Pairs of files (*_active_* and *_standby_*) implement each side; the dispatch mechanism is in ndc_standby_task_util.go / ndc_task_util.go.

Integration points

Inbound: Frontend (client.HistoryClient), Matching (e.g. RecordActivityTaskStarted), Worker (e.g. internal admin RPCs), other clusters' History (replication).
Outbound: Matching (Transfer tasks → tasks added to task queues), Visibility store (Visibility tasks), Archival providers (common/archiver/, Outbound Nexus targets (components/nexusoperations/), Other clusters' History (replication push).
Persistence: Heavy. The shard owns rows in executions, history_node, history_tree, transfer/timer/visibility task tables, and shard-level metadata.
Membership: Ringpop ring drives shard assignment; the ShardController reacts to ring changes by acquiring or releasing shards.

Consistency guarantees

The transactional outbox pattern is the load-bearing invariant:

Mutable State and History Tasks are written in the same DB transaction.
History Events are versioned by an event ID stored in Mutable State; on partial failure, the dirty state is reloaded.
History Tasks are processed at-least-once; executors must be idempotent.
RangeID on every shard write fences out previous owners.

Together these mean that work scheduled by the engine cannot be lost (Transfer Tasks always eventually reach Matching) and stale shards cannot corrupt new state.

CHASM and HSM inside History

History hosts the runtime for two state-machine frameworks:

HSM — service/history/hsm/. The older, in-shard hierarchical state-machine library, used by callbacks and Nexus operations. See Systems → HSM.
CHASM — chasm/. The generalised framework that wraps workflow, scheduler, nexusoperation. The shard runs a chasm_engine alongside the classic historyEngine. See Systems → CHASM.

Both are gradually consuming behaviour that used to live directly in the workflow engine.

Entry points for modification

Adding a History RPC: edit proto/internal/temporal/server/api/historyservice/v1/, regen with make proto, add the method to Handler, and create a sub-package under service/history/api/<rpcname>/ for its implementation.
Adding a new history-task type: edit service/history/tasks/ to register the type, then add an executor pair (active/standby) under service/history/.
Changing MutableState semantics: service/history/workflow/mutable_state_impl.go is the single largest file in the repo at 357KB; expect long PR review cycles.
Adding a queue type: see service/history/queues/. Each new queue gets a factory file in service/history/.

Architecture overview
Persistence — what the shard writes to.
Replication / XDC — multi-cluster path.
HSM — in-shard state machines.
CHASM — generalised state-machine framework.
Workflow execution primitive

Built by Factory AutoWiki from public repository content. It is a generated preview for codebase exploration, not source-maintained documentation.