Core runtime

The heart of TensorFlow: OpKernels, Tensors, Graphs, the executor that runs them, and the device/allocator abstractions underneath. Lives mainly in tensorflow/core/framework/ (definitions) and tensorflow/core/common_runtime/ (local execution).

Purpose

• Define the type system (Tensor, DataType, Shape, Resource).
• Define what an op is (OpDef, OpKernel).
• Provide a graph (tensorflow::Graph) containing nodes and edges.
• Provide the executor that runs a graph asynchronously across devices.
• Manage devices and allocators (CPU, GPU, virtual devices).
• Manage functions — a function library that turns saved graphs into callable ops.

Directory layout

tensorflow/core/framework/
├── tensor.{h,cc}              # Tensor + TensorBuffer
├── tensor_shape.{h,cc}        # TensorShape
├── op_kernel.{h,cc}           # OpKernel base, OpKernelContext
├── op_def_builder.{h,cc}      # REGISTER_OP DSL
├── op.{h,cc}                  # OpRegistry
├── allocator.{h,cc}           # Allocator base, allocator stats
├── device.{h,cc}              # Device base
├── resource_mgr.{h,cc}        # ResourceMgr (long-lived state)
├── rendezvous.{h,cc}          # Cross-device tensor exchange
├── function.{h,cc,proto}      # FunctionDef, FunctionLibraryDefinition
├── types.{h,cc}               # DataType, DType conversions
├── graph_def_util.{h,cc}      # GraphDef helpers
├── attr_value*.{h,cc,proto}   # Attr serialization
└── ...

tensorflow/core/common_runtime/
├── direct_session.{h,cc}      # In-process Session implementation
├── executor.{h,cc}             # Async graph executor
├── placer.{h,cc}              # Op→device placement
├── device_factory.{h,cc}      # Registry of device kinds
├── threadpool_device.{h,cc}   # CPU device
├── gpu/                        # GPU device + BFC allocator
├── eager/                      # Eager runtime (see systems/eager-execution)
├── function.{h,cc}             # FunctionLibraryRuntime
├── colocation_graph.{h,cc}     # Co-location constraint solver
├── graph_def_optimizer.{h,cc}
├── shape_refiner.{h,cc}
├── partitioning_utils.{h,cc}
└── ...

tensorflow/core/graph/
├── graph.{h,cc}               # Graph, Node, Edge
├── algorithm.{h,cc}            # Topological sort, traversals
├── optimizer_cse.{h,cc}        # Common subexpression elimination
├── partition.{h,cc}            # Cross-device partitioning
└── ...

Key abstractions

Type	File	Description
Tensor	tensorflow/core/framework/tensor.h	n-dim array. Stores type, shape, and a TensorBuffer.
TensorShape	tensorflow/core/framework/tensor_shape.h	Shape of a tensor.
OpKernel	tensorflow/core/framework/op_kernel.h	Base of every op implementation.
OpKernelContext	tensorflow/core/framework/op_kernel.h	What an op gets access to during Compute.
OpDef	tensorflow/core/framework/op_def.proto	Schema of an op (name, inputs, attrs).
OpRegistry	tensorflow/core/framework/op.h	Global registry of all ops.
Graph / Node / Edge	tensorflow/core/graph/graph.h	The computation graph.
Executor	tensorflow/core/common_runtime/executor.h	Runs a partitioned graph asynchronously.
Device / DeviceFactory	tensorflow/core/framework/device.h	Abstraction over CPU, GPU, TPU.
Allocator	tensorflow/core/framework/allocator.h	Memory allocator (BFC for GPU, process state for CPU).
ResourceMgr / ResourceBase	tensorflow/core/framework/resource_mgr.h	Long-lived state attached to a device.
Rendezvous	tensorflow/core/framework/rendezvous.h	Producer-consumer queue for cross-device tensors.
FunctionLibraryRuntime	tensorflow/core/common_runtime/function.h	Calls registered functions like ops.
Placer	tensorflow/core/common_runtime/placer.h	Decides which device every op runs on.
DirectSession	tensorflow/core/common_runtime/direct_session.h	In-process Session for graph mode.

How a graph runs

graph TD
    Client[User: Session::Run]
    Direct[DirectSession]
    Placer[Placer]
    Partition[Partitioning by device]
    Exec[Executor per partition]
    Kernel[OpKernel::Compute]
    Rdv[Rendezvous]

    Client --> Direct
    Direct --> Placer
    Placer --> Partition
    Partition --> Exec
    Exec -->|async ops| Kernel
    Kernel -->|cross-device send/recv| Rdv
    Rdv --> Kernel

1. The user constructs a Graph (or imports a GraphDef) and calls Session::Run({fetch_outputs}, ...).
2. DirectSession looks up or builds an executor for the run: it places nodes (Placer), inserts _Recv/_Send ops at device boundaries, partitions the graph, optionally runs Grappler, and creates an Executor per partition.
3. The Executor walks the partition's nodes, scheduling each ready node onto the device's threadpool. OpKernel::Compute runs synchronously for compute-bound ops; AsyncOpKernel::ComputeAsync is used for ops that wait for I/O or RPCs.
4. Cross-device data flows through Rendezvous keys (e.g. "src:dst:tensor"), which makes it transparent for the kernel to ship tensors between CPU↔GPU or host↔host.

Memory model

• Every device has one or more Allocators.
• The CPU uses pool/process-state allocators (tensorflow/core/framework/allocator_registry.h, tensorflow/core/common_runtime/process_state.h).
• GPU uses BFC (Best-Fit-with-Coalescing) allocator at tensorflow/core/common_runtime/bfc_allocator.cc and tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc. It pre-reserves a large chunk of GPU memory and slices it.
• Tensors use ref-counted TensorBuffers. Aliasing (returning an input as an output) is a common pattern.
• Resources (variables, iterators, hashtables) live until their ResourceMgr is destroyed.

Functions and call ops

A FunctionDef (tensorflow/core/framework/function.proto) is a graph-of-ops that can be called like an op. The FunctionLibraryRuntime (tensorflow/core/common_runtime/function.h) inlines or PartitionedCalls a function on demand. @tf.function traced graphs end up here.

Resources and state

ResourceMgr is the canonical place for stateful long-lived objects: variables, iterators, hash tables, queues. Each is a ResourceBase subclass and lives in a named container per device. Lookups happen via Lookup/LookupOrCreate keyed by (container, name).

Important specific resources:

• Var (tensorflow/core/framework/resource_var.h) — holds a Tensor for a ResourceVariable.
• IteratorResource (tensorflow/core/data/) — holds a tf.data iterator's state.

Integration points

• Eager runtime in tensorflow/core/common_runtime/eager/ reuses OpKernel, Device, ResourceMgr. See systems/eager-execution.
• Distributed runtime in tensorflow/core/distributed_runtime/ extends the same model with RemoteRendezvous and gRPC. See systems/distributed-runtime.
• Grappler at tensorflow/core/grappler/ operates on GraphDefs before execution. See systems/grappler.
• Compiler bridges (tensorflow/compiler/tf2xla/, tensorflow/compiler/jit/) hook in by inserting special ops (XlaCompile, XlaRun) and consuming graphs.

Entry points for modification

• New OpKernel — tensorflow/core/kernels/, register with REGISTER_KERNEL_BUILDER. See kernels-and-ops.
• New device — implement Device and a DeviceFactory, register with REGISTER_LOCAL_DEVICE_FACTORY (tensorflow/core/common_runtime/device_factory.h). The TPU and GPU devices are the canonical examples.
• New allocator — implement Allocator, register or wire into a Device.
• Executor changes — tensorflow/core/common_runtime/executor.cc is dense; coordinate with maintainers.
• Placement / Co-location — tensorflow/core/common_runtime/placer.cc, colocation_graph.cc.

Related

• systems/kernels-and-ops — the ~10 000 op kernels.
• systems/eager-execution — eager-specific runtime.
• systems/distributed-runtime — multi-host execution.
• systems/grappler — graph-level optimization.