Language models

The interfaces every model integration implements. Source: libs/core/langchain_core/language_models/.

Purpose

langchain-core defines three contracts:

• BaseLanguageModel — the absolute minimum: a Runnable[LanguageModelInput, LanguageModelOutput] that takes prompts/messages and returns text or a message. Hardly anyone subclasses this directly.
• BaseChatModel — the legacy chat-model contract. Every partner subclasses it (or the v1 variant). Defined in libs/core/langchain_core/language_models/chat_models.py (~2,500 lines).
• BaseLLM — the completions-style contract. Used for the older "text in, text out" APIs. Defined in libs/core/langchain_core/language_models/llms.py (~1,400 lines).
• BaseChatModelV1 — the streaming-first v1 contract for new partner integrations. Defined in libs/core/langchain_core/language_models/chat_model_stream.py (~1,500 lines).

A _compat_bridge.py shim lets the v1 contract coexist with the legacy contract during the transition.

Directory layout

libs/core/langchain_core/language_models/
├── __init__.py
├── _compat_bridge.py        # Bridges BaseChatModel (legacy) <-> BaseChatModelV1
├── _utils.py                # Shared helpers
├── base.py                  # BaseLanguageModel, LanguageModelInput, LanguageModelOutput
├── chat_model_stream.py     # BaseChatModelV1 (~1,500 lines)
├── chat_models.py           # BaseChatModel (~2,500 lines)
├── fake.py, fake_chat_models.py
                            # FakeChatModel, FakeListChatModel, FakeMessagesListChatModel,
                            # GenericFakeChatModel — used by tests across the repo
├── llms.py                  # BaseLLM (~1,400 lines)
└── model_profile.py         # ModelProfile typed dict (capability data)

Key abstractions

Symbol	File	Description
BaseLanguageModel	libs/core/langchain_core/language_models/base.py	Common base for chat and completion models
LanguageModelInput, LanguageModelOutput	libs/core/langchain_core/language_models/base.py	Type aliases for "messages or string" / "AIMessage or string"
BaseChatModel	libs/core/langchain_core/language_models/chat_models.py	The legacy contract: _generate(messages, stop, run_manager, **kwargs), _stream, async counterparts
BaseChatModelV1	libs/core/langchain_core/language_models/chat_model_stream.py	Streaming-first contract for new partners
BaseLLM	libs/core/langchain_core/language_models/llms.py	Completions contract: _generate(prompts, stop, run_manager, **kwargs) -> LLMResult
FakeListChatModel, GenericFakeChatModel	libs/core/langchain_core/language_models/fake_chat_models.py	Test doubles with scripted responses
ModelProfile	libs/core/langchain_core/language_models/model_profile.py	Capability descriptor (context window, supports vision, etc.)
_compat_bridge helpers	libs/core/langchain_core/language_models/_compat_bridge.py	Convert between legacy and v1 contract calls

What BaseChatModel exposes

Every model that users instantiate (e.g. ChatOpenAI, ChatAnthropic) provides:

• invoke(input, config=None, **kwargs) -> AIMessage
• ainvoke(...), batch(...), abatch(...), stream(...), astream(...)
• bind_tools(tools, *, tool_choice=None) -> Runnable — return a new runnable that calls the model with tools= set
• with_structured_output(schema, *, method="function_calling" | "json_mode" | "json_schema", strict=None) — return a runnable that emits an instance of schema
• with_retry(...), with_fallbacks(...), with_config(...) — inherited from Runnable
• bind(**kwargs) — pre-set call kwargs
• profile — the ModelProfile for this model

The default implementations of invoke and ainvoke call _generate / _agenerate, which the partner overrides. The default stream calls _stream / _astream. Partners that don't implement streaming get a one-shot fallback.

How a partner subclasses it

A typical partner has:

class ChatX(BaseChatModel):
    model: str
    api_key: SecretStr | None = None
    # ... fields ...

    @property
    def _llm_type(self) -> str: return "chat-x"

    def _generate(self, messages, stop=None, run_manager=None, **kwargs) -> ChatResult:
        api_messages = self._convert_messages(messages)
        response = self._client.chat.completions.create(model=self.model, messages=api_messages, **kwargs)
        ai_message = self._convert_response(response)
        return ChatResult(generations=[ChatGeneration(message=ai_message)])

    async def _agenerate(...) -> ChatResult: ...
    def _stream(...): yield ChatGenerationChunk(...)
    async def _astream(...): yield ChatGenerationChunk(...)
    def bind_tools(self, tools, **kwargs): ...
    def with_structured_output(self, schema, **kwargs): ...

The bulk of the work is in _convert_messages / _convert_response, which translate between LangChain messages/content blocks and the provider's wire format. Partners centralize this in a _compat.py module.

v1 streaming-first contract

BaseChatModelV1 flips the relationship: instead of _generate being primary and _stream a fallback, streaming is primary. The single entry point is an async iterator of AIMessageChunks, and the non-streaming invoke accumulates the chunks. This matches how modern provider SDKs work and avoids the awkward dual-implementation requirement of the legacy contract.

_compat_bridge.py provides:

• A wrapper that exposes BaseChatModelV1 as BaseChatModel for downstream consumers.
• A wrapper that adapts legacy BaseChatModel partners into the v1 contract.

This shim lets the new agent factory in libs/langchain_v1/ work with both old and new partners during the transition period.

Tool binding

bind_tools(tools, tool_choice=None) returns a new runnable that, when invoked, includes the tools in every model call. The default implementation in BaseChatModel raises NotImplementedError; partners override it. The resulting AIMessage has tool_calls: list[ToolCall] populated when the model decides to invoke a tool.

tool_choice accepts:

• None / "auto" — model decides whether to call tools
• "any" — must call some tool
• "none" — must not call tools
• a tool name (e.g. "search") — must call that specific tool

Structured output

with_structured_output(schema, method=..., strict=...) returns a runnable that emits a typed value matching schema. schema may be a Pydantic class, a TypedDict, a dataclass, or a JSON Schema dict. The supported methods:

• "function_calling" — bind a single tool whose schema matches schema, then parse the tool call arguments.
• "json_mode" — request JSON output from the model and parse it.
• "json_schema" — use the provider's native structured-output endpoint (OpenAI's response_format={"type": "json_schema"} or Anthropic's tool-with-JSON-schema).

Partners implement this in their own with_structured_output overrides; the base class provides a default function-calling implementation.

Token counting and streaming usage

AIMessage.usage_metadata carries input_tokens, output_tokens, total_tokens, and optional details (InputTokenDetails, OutputTokenDetails). For streaming, partners set stream_usage=True (where supported) and accumulate usage chunks; AIMessageChunk + AIMessageChunk sums the usage metadata.

Integration points

• Every partner package subclasses BaseChatModel or BaseChatModelV1 (some subclass both for compatibility).
• init_chat_model in libs/langchain_v1/langchain/chat_models/base.py constructs partner classes via the _BUILTIN_PROVIDERS registry.
• langchain.agents.create_agent accepts a model instance or a string; if a string, it calls init_chat_model.
• Standard tests in libs/standard-tests/langchain_tests/integration_tests/ exercise the full surface against real providers.

Entry points for modification

• For a new feature on every partner (e.g. a new tool-binding option), add it to BaseChatModel in chat_models.py with a sensible default. Partners override where needed.
• For a new structured-output method, extend with_structured_output in BaseChatModel and add provider-specific overrides where the provider has a native endpoint.
• For fakes used in tests, edit fake_chat_models.py. Tests across every package use GenericFakeChatModel.

Related

• primitives/messages — what models consume and emit
• primitives/tools — what bind_tools accepts
• primitives/runnables — the protocol every model implements
• packages/standard-tests — the cross-provider conformance suite
• partners/openai, partners/anthropic — concrete implementations