Buffer and memline

Purpose

The buffer is the in-memory representation of a file. Neovim inherits Vim's storage strategy: line content is paged to disk through a memfile so very large files don't blow up memory, and metadata (options, autocmds-attached, marks, signs, undo) hangs off a per-buffer struct. This page covers the storage layer (memfile.c, memline.c), the buffer lifecycle (buffer.c), and the file I/O paths that read content into and write content out of buffers.

Directory layout

src/nvim/
├── buffer.c (~135k bytes)           Buffer lifecycle: open, close, switch
├── buffer.h, buffer_defs.h          buf_T struct (~62k bytes of struct fields)
├── memfile.c, memfile_defs.h        Disk-backed page allocator
├── memline.c (~144k bytes)          Tree of pages → line lookup
├── memline_defs.h
├── bufwrite.c (~60k bytes)          Save buffer to file
├── fileio.c (~123k bytes)           Read file into buffer
├── buffer_updates.c                 Notify API listeners on text change
└── change.c (~76k bytes)            ml_replace + ml_append + change tracking

Key abstractions

Type	File	Description
buf_T	buffer_defs.h	The buffer. Hundreds of fields: file name, options, marks, syntax state, undo, b_changedtick, ...
win_T	buffer_defs.h	A window onto a buffer. Has its own cursor, options-by-window, fold state.
memline_T	memline_defs.h	The per-buffer storage handle. Contains a memfile_T *.
memfile_T	memfile_defs.h	The page allocator. Backed by an open file or pure RAM.
bhdr_T	memfile_defs.h	A page header. Pages are 4 KB by default.
ml_find_line	memline.c	The hot path: given a line number, return a pointer to its bytes.

How it works

A buffer's content is a tree of fixed-size pages. The pointer block (root) holds offsets and child pointers; data blocks (leaves) hold the actual line bytes. Editing a line means:

1. Find the data block for that line (ml_find_line).
2. Replace the line's bytes in place if it fits, or split the block if it doesn't.
3. Update the parent's offset cache.
4. Mark the block dirty so the memfile flushes it on the next sync.

graph TD
    BUF[buf_T] --> ML[memline_T]
    ML --> MF[memfile_T]
    MF --> P0[Pointer block<br/>root]
    P0 --> P1[Pointer block]
    P0 --> D0[Data block<br/>lines 1..40]
    P1 --> D1[Data block<br/>lines 41..80]
    P1 --> D2[Data block<br/>lines 81..120]
    MF -.->|swap to disk on demand| DISK[(.swp file)]

ml_find_line is called everywhere — by syntax highlighting, by getline(), by every motion, by extmark queries. It maintains a hot-line cache (ml_line_lnum) so repeated lookups for the same line are O(1).

The data block layout is: a packed array of line offsets at the top, and the line content (without trailing newline) growing from the bottom. An entry is (offset, length) to the bytes; the offset is from the page start. This makes getline() one indirection.

Swap files

Every buffer is by default backed by a swap file (<dir>/<name>.swp). The swap file is the same memfile-page format as in memory, so a crashed Neovim can be recovered by reading the pages back. Swap creation, locking, and recovery live in memline.c and memfile.c. The user-facing options are 'swapfile', 'directory', and 'updatecount' (how many changes between flushes).

:recover reads a swap file back into a fresh buffer. The recovery logic is one of the older, scarier parts of memline.c and is largely Vim-as-imported.

Reading files

fileio.c (readfile() is the entry point) does:

1. Open the file (with os_open).
2. Detect encoding ('fenc' heuristics, BOMs, etc.).
3. Read in chunks, applying line-ending conversion (CR / LF / CRLF).
4. Apply 'fileformat' and 'fileencoding'.
5. Push lines into the memline via ml_append.
6. Fire BufReadPre/BufReadPost autocmds.

Encoding detection is the part most contributors get bitten by. 'fileencoding' interacts with 'encoding' (which is now always UTF-8) via iconv (src/nvim/mbyte.c).

Writing files

bufwrite.c (buf_write()) is the symmetric path. It writes to a temporary file, fsyncs, optionally creates a backup ('backupcopy', 'backupext'), and renames over the original. It also handles file-permissions preservation, the &binary mode, and BufWritePre/BufWritePost autocmds.

The whole flow is more careful than it looks because of edge cases: symlinks, hardlinks, network filesystems, write atomicity, locked files on Windows. Roughly two-thirds of bufwrite.c is platform-quirk handling.

Buffer updates

When a line changes, the editor must notify:

• The undo subsystem (undo.c) — push the change onto the undo tree.
• The marktree (marktree.c) — shift extmarks past the change.
• API listeners (buffer_updates.c) — call nvim_buf_attach callbacks.
• Treesitter (runtime/lua/vim/treesitter/) — incrementally reparse.
• LSP (runtime/lua/vim/lsp/_changetracking.lua) — track changes for didChange.
• Decoration providers — invalidate cached highlights.

changed_bytes and changed_lines in change.c are the dispatch points. Anything that changes buffer content must call one of them; if you bypass them, you'll see "the cursor moved but the screen didn't update" bugs.

buf_T fields you care about

Field	Purpose
b_fname, b_ffname, b_sfname	File name (display, full, short).
b_ml	The memline.
b_p_*	Buffer-local options. The b_p_ prefix is for "buffer parameter".
b_changedtick	Monotonic counter — increments on every change. The basis for b:changedtick.
b_marktree	The extmark B+-tree.
b_namedm	Named marks.
b_syn_state	Syntax highlighting state cache.
b_terminal	Non-NULL if this is a :terminal buffer.
b_lsp_*	LSP attachment data (set from Lua).

Total field count is in the hundreds; buffer_defs.h is 62 KB. Most of those fields exist for one or two specific subsystems.

Integration points

• Windows — win_T references a buf_T. Many windows can share a buffer. See Windows and screen rendering.
• Marks — both legacy named marks (mark.c) and extmarks (extmark.c) live on the buffer. See Marks and extmarks.
• Undo — every change is recorded in a per-buffer undo tree. See Undo.
• API — nvim_buf_* functions in src/nvim/api/buffer.c, src/nvim/api/extmark.c, src/nvim/api/buf_set_lines. Conversions go through src/nvim/api/private/.
• Lua — vim.api.nvim_buf_*, vim.api.nvim_buf_set_lines, vim.api.nvim_buf_attach. The Lua side is generated from the C declarations.

Entry points for modification

• Add a new buffer-local property. Add a field to buf_T in buffer_defs.h, initialize in buflist_new, free in free_buffer. If the property has an Ex/Lua binding, also declare it as an option (options.lua).
• Hook into changes. Use nvim_buf_attach() from a plugin, or register a decoration provider, or hook TextChanged* autocmds. From C, the dispatch points are in change.c.
• Custom file I/O. BufReadCmd / BufWriteCmd autocmds preempt the default read/write. Used by remote filesystem plugins. The reference implementation is runtime/plugin/man.lua for man:// URIs.

Key source files

File	Purpose
src/nvim/buffer.c	Buffer create/destroy/switch, :buffer command, autoload.
src/nvim/buffer_defs.h	The buf_T, win_T, tabpage_T struct definitions.
src/nvim/memline.c	The line storage tree, swap-file logic, recovery.
src/nvim/memfile.c	The page allocator that backs memline.
src/nvim/fileio.c	readfile() and friends.
src/nvim/bufwrite.c	buf_write() and friends.
src/nvim/buffer_updates.c	API listener dispatch on change.
src/nvim/change.c	changed_bytes, changed_lines, change tracking.
src/nvim/api/buffer.c	The nvim_buf_* public API.