Grappler

Graph-level optimizer that runs on a GraphDef between graph construction and execution. Lives in tensorflow/core/grappler/.

Purpose

• Apply standard compiler-style optimizations to a TF graph before the executor runs it: constant folding, common-subexpression elimination, layout transformations, arithmetic simplification, memory optimization.
• Produce a smaller, faster graph without changing semantics.
• Inject device-specific patterns (e.g., NHWC↔NCHW for GPU, MKL fusions for CPU).

Directory layout

tensorflow/core/grappler/
├── grappler_item.{h,cc}            # Wraps a GraphDef + run metadata
├── clusters/                       # Abstraction over the device cluster (for cost models)
├── costs/                          # Op-level cost estimates
├── op_types.{h,cc}                 # Op categorisation helpers
├── optimizers/                     # Each pass is a class here
│   ├── arithmetic_optimizer.{h,cc}
│   ├── auto_mixed_precision.{h,cc}
│   ├── auto_parallel.{h,cc}
│   ├── common_subgraph_elimination.{h,cc}
│   ├── constant_folding.{h,cc}
│   ├── debug_stripper.{h,cc}
│   ├── dependency_optimizer.{h,cc}
│   ├── function_optimizer.{h,cc}
│   ├── generic_layout_optimizer.{h,cc}
│   ├── implementation_selector.{h,cc}
│   ├── loop_optimizer.{h,cc}
│   ├── memory_optimizer.{h,cc}
│   ├── meta_optimizer.{h,cc}      # Top-level driver
│   ├── pin_to_host_optimizer.{h,cc}
│   ├── remapper.{h,cc}             # Op fusion (Conv+BiasAdd+Relu, MatMul+BiasAdd+Activation)
│   ├── scoped_allocator_optimizer.{h,cc}
│   └── shape_optimizer.{h,cc}
├── utils/                          # Helpers used across optimizers
└── verifiers/                      # Post-pass validators

Key passes

Optimizer	What it does
constant_folding	Evaluates ops with all-constant inputs at graph-build time.
arithmetic_optimizer	Algebraic simplifications (e.g., x*1 → x, x+0 → x, Sqrt(Square(x)) → Abs(x)).
dependency_optimizer	Removes redundant control dependencies.
loop_optimizer	Hoists invariants, eliminates dead loops.
function_optimizer	Inlines or specialises PartitionedCall/StatefulPartitionedCall.
remapper	Fuses common patterns: Conv2D+BiasAdd+Relu, MatMul+BiasAdd+Activation, LayerNorm.
auto_mixed_precision	Inserts casts to use bfloat16/float16 where safe.
generic_layout_optimizer	Picks the best tensor layout (NHWC vs NCHW) for the device.
memory_optimizer	Reduces peak memory by adding swap-in/out and recomputation hints.
auto_parallel	Replicates a graph across replicas for parallelism.
implementation_selector	Picks among different kernel implementations of the same op (e.g., MKL vs default).
scoped_allocator_optimizer	Aggregates small allocations into shared scopes.

The driver is MetaOptimizer (tensorflow/core/grappler/optimizers/meta_optimizer.cc), which runs the configured set of optimizers in a fixed order and re-runs them until the graph stabilises.

When Grappler runs

• Inside a Session.run call (graph mode), before the partitioned graph is handed to the executor.
• Inside @tf.function traces — the function's FuncGraph is optimized by Grappler before being registered with the function library.
• For SavedModel loading, Grappler runs after the graph is materialised into the importing process.

It does not run for individual eager ops; eager calls bypass the graph optimizer entirely.

Configuration

tf.config.optimizer.set_experimental_options({...}) toggles individual optimizers from Python. Internally these flag bits are passed via RewriterConfig (tensorflow/core/protobuf/rewriter_config.proto) and into MetaOptimizer. Common knobs:

• disable_meta_optimizer — turn off Grappler entirely.
• constant_folding, arithmetic_optimization, remapping, layout_optimizer, ... — per-pass on/off.
• auto_mixed_precision, auto_mixed_precision_mkl, auto_mixed_precision_onednn_bfloat16.

Cost models

Many passes use cost estimates to decide whether a rewrite is worth it. The costs/ directory contains:

• op_level_cost_estimator — per-op compute and memory estimates.
• analytical_cost_estimator — drives a graph-wide cost model.
• Cluster abstractions in clusters/ (single machine, virtual cluster) so optimizers can reason about device counts and bandwidth.

Integration points

• Inputs: GraphDef plus a GrapplerItem (the graph plus fetches/feed metadata).
• Outputs: a rewritten GraphDef.
• Called from: MasterSession, DirectSession, FunctionLibraryRuntime, SavedModel loader.
• XLA bridge runs after Grappler — the JIT auto-clustering pass operates on the rewritten graph.

Entry points for modification

• New optimizer pass: subclass GraphOptimizer (tensorflow/core/grappler/optimizers/graph_optimizer.h), register with MetaOptimizer. Add a flag to RewriterConfig if it needs to be opt-in.
• New fusion pattern: remapper.cc is the right place for op-fusion patterns; add a matcher and a builder.
• New cost estimate: extend op_level_cost_estimator.cc with a per-op rule.

Related

• core-runtime — the executor that consumes Grappler's output.
• compilers/xla — the JIT runs after Grappler, on the optimized graph.