Multi interface

Active contributors: Daniel Stenberg, Stefan Eissing

Purpose

The multi interface drives many transfers in one thread without blocking. It is the same engine the easy interface uses internally — curl_easy_perform() is implemented as a tiny loop around curl_multi_perform(). Applications that need event-driven I/O, pipelining, HTTP/2 multiplexing, or HTTP/3 use the multi interface directly.

Public surface

Function	Purpose
curl_multi_init / curl_multi_cleanup	Create/destroy a CURLM * handle
curl_multi_add_handle / _remove_handle	Attach or detach an easy handle
curl_multi_perform	Poll-loop convenience entry point
curl_multi_wait / _poll / _wakeup	Block until activity (or timeout) on any of the multi's FDs
curl_multi_socket_action	Event-driven entry point — application reports FD readiness
curl_multi_info_read	Drain CURLMSG_DONE messages reporting completed transfers
curl_multi_setopt	Configure the multi handle (max connections, push function, timer/socket callbacks)
curl_multi_strerror	Map CURLMcode to a string

include/curl/multi.h is the canonical header. The implementation is lib/multi.c plus lib/multi_ev.c (event distribution) and lib/multi_ntfy.c (wakeup).

Two flavours

graph LR
    subgraph "Polling style"
      A1[curl_multi_perform] --> A2[Curl_multi_runsingle × N]
      A1 --> A3[curl_multi_wait]
    end
    subgraph "Event-driven style"
      B1[application loop<br/>using libuv / libev / select] --> B2[curl_multi_socket_action]
      B2 --> B3[Curl_multi_runsingle × N]
      B2 --> B4[CURLMOPT_SOCKETFUNCTION callback]
      B2 --> B5[CURLMOPT_TIMERFUNCTION callback]
    end

Polling style

The simplest pattern: call curl_multi_perform, check CURLMSG_DONE messages, sleep on curl_multi_poll, repeat. Examples in docs/examples/multi-app.c. Good enough for tens of concurrent transfers.

Event-driven style

The application runs its own event loop (libuv, libev, epoll, kqueue, select, …) and tells curl about FD readiness via curl_multi_socket_action. curl tells the application which FDs to watch via CURLMOPT_SOCKETFUNCTION and when to fire timers via CURLMOPT_TIMERFUNCTION. The result is non-blocking I/O scaling to thousands of concurrent transfers. See docs/examples/ephiperfifo.c and docs/examples/multi-uv.c.

Multi options

Option	Effect
CURLMOPT_MAXCONNECTS	Cap on connection cache size
CURLMOPT_MAX_HOST_CONNECTIONS	Cap on connections to one origin
CURLMOPT_MAX_TOTAL_CONNECTIONS	Cap on total simultaneous connections
CURLMOPT_MAX_PIPELINE_LENGTH	(HTTP/2 streams per connection)
CURLMOPT_PUSHFUNCTION + CURLMOPT_PUSHDATA	Accept/reject HTTP/2 push streams
CURLMOPT_SOCKETFUNCTION / CURLMOPT_TIMERFUNCTION	Required for curl_multi_socket_action

Share interface

curl_share_setopt(sh, CURLSHOPT_SHARE, CURL_LOCK_DATA_*) lets multi handles (and easy handles) share state across the boundary:

Resource	Flag
Connection cache	CURL_LOCK_DATA_CONNECT
DNS cache	CURL_LOCK_DATA_DNS
TLS session cache	CURL_LOCK_DATA_SSL_SESSION
Cookies	CURL_LOCK_DATA_COOKIE
HSTS cache	CURL_LOCK_DATA_HSTS
Public-suffix list	CURL_LOCK_DATA_PSL

The application supplies lock callbacks (CURLSHOPT_LOCKFUNC/CURLSHOPT_UNLOCKFUNC) so libcurl is thread-safe with respect to the shared resource.

Threading model

A multi handle, like an easy handle, is not safe to call from multiple threads concurrently. The classic pattern is:

• One thread per multi handle, each driving N easy handles
• Or share caches via curl_share to coordinate state across threads

For applications that do drive a multi from one thread but want to nudge it from another, curl_multi_wakeup() triggers a no-op wake of curl_multi_poll/_wait so the owner thread can pick up new work.

Completion handling

When a transfer reaches MSTATE_COMPLETED, curl posts a CURLMSG_DONE message into the multi's message queue. The application drains the queue with curl_multi_info_read after each _perform / _socket_action:

int still_running;
curl_multi_perform(m, &still_running);

CURLMsg *m_msg;
int msgs_in_queue;
while((m_msg = curl_multi_info_read(m, &msgs_in_queue))) {
  if(m_msg->msg == CURLMSG_DONE) {
    handle_done(m_msg->easy_handle, m_msg->data.result);
  }
}

After completion the easy handle must be removed from the multi (curl_multi_remove_handle) and either reset+re-added or cleaned up.

Related pages

• Transfer engine — what runs inside the multi handle
• Connection management — connection cache and share semantics
• Easy interface