Treesitter integration

Purpose

Tree-sitter is an incremental parsing library; Neovim ships C bindings to it plus a Lua API for parsing buffers, running queries against the resulting trees, and using the results for syntax highlighting, folding, indentation, and structural editing. The integration replaces (and now happily coexists with) Vim's regex-based syntax engine for languages that have a tree-sitter grammar.

Directory layout

src/nvim/lua/
└── treesitter.c (~50k bytes)            C bindings: parser, tree, node, query, query_cursor

runtime/lua/vim/treesitter/
├── _fold.lua                              Treesitter-driven folding
├── _headings.lua                          Markdown/help heading queries
├── _meta/                                 Type stubs
├── _query_linter.lua                      Lint query files
├── _range.lua                             Range arithmetic helpers
├── _select.lua                            Treesitter text-objects
├── dev.lua                                :InspectTree, :EditQuery, the playground
├── health.lua                             :checkhealth treesitter
├── highlighter.lua                        The syntax highlighter (decoration provider)
├── language.lua                           Parser registration / loading
├── languagetree.lua (~44k bytes)          Per-buffer parse state
└── query.lua (~38k bytes)                 Query API + predicate registry
runtime/lua/vim/treesitter.lua             Top-level entry point
runtime/queries/                           Bundled query files
runtime/parser/                            Bundled parser .so files

Key abstractions

Type / function	File	Description
LanguageTree	treesitter/languagetree.lua	Per-buffer parser state. Tracks the source range it parses, child trees for embedded languages, edit ranges.
vim.treesitter.get_parser(buf, lang)	treesitter.lua	Get-or-create the per-buffer LanguageTree.
Query	treesitter/query.lua	A compiled query (a tree-sitter S-expression). Has captures and predicates.
Highlighter	treesitter/highlighter.lua	The decoration provider that turns parsed trees into highlight extmarks.
vim.treesitter.start(buf, lang)	treesitter.lua	Attach a Highlighter to the buffer.
inspect_tree, edit_query	treesitter/dev.lua	Developer playground commands.

How it works

graph LR
    Buf[Buffer change] --> LT[LanguageTree]
    LT --> Edit[ts_tree_edit<br/>tell parser the edit]
    LT --> Parse[parser:parse<br/>incremental]
    Parse --> Tree[New tree]
    Tree --> HL[Highlighter<br/>decoration provider]
    HL --> Vis[visible lines]
    Vis --> Q[query iter_captures]
    Q --> EM[set_extmark with hl_group]

Per-buffer state

When a plugin (or vim.treesitter.start) calls vim.treesitter.get_parser(buf, lang) for the first time, a LanguageTree is created and stored on the buffer. The LanguageTree:

• Loads the parser shared object (runtime/parser/<lang>.so).
• Parses the whole buffer initially.
• Subscribes to nvim_buf_attach callbacks so it sees every byte change.
• Tracks "edit ranges" — ts_tree_edit is called with each byte-range change, then parser:parse(tree) reuses the unaffected subtrees.
• Recursively manages child trees for injected languages (e.g., a code block inside markdown).

The "languagetree" name reflects the recursion: each language tree may have child trees of other languages, walking down to the leaves.

Queries

A query is an S-expression that captures patterns:

(function_declaration
  name: (identifier) @function.name)

(comment) @comment

The query language and the bundled grammar live under runtime/queries/. They're plain text files at runtime/queries/<lang>/<topic>.scm (highlights.scm, injections.scm, folds.scm, indents.scm, locals.scm, textobjects.scm).

treesitter/query.lua compiles a query against a parser, exposes iter_captures(tree, start, end) to walk matches, and includes a registry of predicates (#match?, #eq?, #has-ancestor?, #lua-match?, ...).

Highlighting

treesitter/highlighter.lua is a decoration provider. On every on_win redraw event for a buffer with treesitter highlighting active, it:

1. Asks LanguageTree for the current tree (parsing if needed).
2. Walks the visible-line region with the highlights.scm query.
3. For each capture, sets an extmark with the corresponding highlight group.

The capture name (@function.name) maps to a highlight group (@function.name itself is a group; falls back to @function, then Function). The highlight.c side is responsible for the cascade.

Injections

A second query (injections.scm) tells the languagetree where to spawn child trees. A typical entry:

((code_block
  (info_string) @injection.language
  (code_fence_content) @injection.content))

When the parent tree is parsed, the injection query runs and the matched ranges become a child tree of the language whose name matched @injection.language. This is how a markdown file gets syntax highlighting for the languages of its embedded code blocks, with no special-casing.

Edit propagation

Edit propagation is the reason treesitter is fast on large files. When the buffer changes:

1. nvim_buf_attach sees the change and posts an on_bytes event.
2. The languagetree converts the byte range to a TSInputEdit and calls ts_tree_edit(old_tree, &edit).
3. On the next parse(), only subtrees overlapping the edit are reparsed; everything outside is reused.

The result: typing one character in a 10,000-line buffer is roughly free.

Other consumers

The same parser/tree/query infrastructure powers:

• Folding (_fold.lua) — generic folder driven by folds.scm queries.
• Indentation — runtime/queries/<lang>/indents.scm plus a Lua engine for indentexpr.
• Text objects (_select.lua) — [m, ]m, vif, etc., backed by textobjects.scm.
• Locals — variable scope inference for gd and rename, via locals.scm.
• Headings (_headings.lua) — markdown / vimdoc heading navigation.

Bundled grammars

Several parsers are bundled in the binary via cmake.deps/:

• tree-sitter-c, tree-sitter-lua, tree-sitter-vimdoc, tree-sitter-vim, tree-sitter-markdown, tree-sitter-bash, tree-sitter-query.

Other languages need a parser installed by the user, typically via nvim-treesitter or vim.pack. The runtime looks for parsers in runtime/parser/, $VIMRUNTIME/parser/, and any directory on the runtimepath.

Integration points

• C-side bindings — src/nvim/lua/treesitter.c exposes the C library. The Lua wrappers above are thin layers over those bindings.
• Decoration provider — the highlighter is registered with nvim_set_decoration_provider, so it cooperates with the renderer's per-line callback contract.
• buf_attach — every languagetree subscribes to changes via nvim_buf_attach.
• Folding — vim.opt_local.foldmethod = 'expr', foldexpr = 'v:lua.vim.treesitter.foldexpr()'.
• Indent — same idea, indentexpr = 'v:lua.require"nvim-treesitter".indentexpr()' (community plugin) or the built-in helpers.
• Query linter — :checkhealth treesitter and :lua require'vim.treesitter._query_linter'.check() validate query files at edit time.

Entry points for modification

• Improve the highlighter. treesitter/highlighter.lua is a single file. Most user-visible behavior changes (priority handling, cterm fallbacks, conceal interaction) live there.
• Add a query predicate. Extend treesitter/query.lua's predicate registry. Predicates are pure functions over capture matches.
• Bind a new tree-sitter API. Add it to src/nvim/lua/treesitter.c and expose the wrapper in runtime/lua/vim/treesitter/_meta/.
• Bundle a new grammar. Edit cmake.deps/deps.txt, add to cmake.deps/cmake/BuildTreeSitterParsers.cmake, ship runtime/queries/<lang>/.

Key source files

File	Purpose
src/nvim/lua/treesitter.c	C bindings (parser, tree, node, query, cursor)
runtime/lua/vim/treesitter.lua	Top-level entry point (vim.treesitter.*)
runtime/lua/vim/treesitter/languagetree.lua	Per-buffer parse state, injections, edit propagation
runtime/lua/vim/treesitter/highlighter.lua	Syntax highlighter (decoration provider)
runtime/lua/vim/treesitter/query.lua	Query API + predicates
runtime/lua/vim/treesitter/_fold.lua	Folding
runtime/lua/vim/treesitter/_select.lua	Text objects
runtime/lua/vim/treesitter/dev.lua	Inspector and query playground
runtime/queries/*	Bundled query files
runtime/parser/*	Bundled compiled parsers
runtime/doc/treesitter.txt	User docs