langchain-tests (standard tests)

A package of reusable test base classes that every partner package inherits. Source: libs/standard-tests/langchain_tests/. PyPI: langchain-tests. Current version: 1.1.7.

Purpose

LangChain integrates with 15+ providers in this monorepo and many more in sibling repositories. To keep their behavior consistent — same input shapes, same streaming semantics, same tool-calling protocol, same error types — every partner package's test suite inherits from a common set of base classes defined here. When langchain-core adds a capability (say, a new content-block type), the test for it is added to the standard suite, and every partner immediately runs it on the next CI run.

Directory layout

libs/standard-tests/langchain_tests/
├── __init__.py
├── base.py                # The shared `BaseStandardTests` foundation
├── conftest.py            # Pytest fixtures used across all suites
├── unit_tests/            # No-network test base classes
├── integration_tests/     # Network-allowed test base classes
└── utils/                 # Helpers (e.g. fake response builders)

What's covered

The typical test base classes:

• ChatModelUnitTests, ChatModelIntegrationTests — the largest suite. Verifies the chat-model contract: invoke, stream, tool calling, structured output, token usage, multimodal content, streaming token usage, prompt caching where supported, error handling, retries.
• ChatModelV1UnitTests, ChatModelV1IntegrationTests — counterparts for the new BaseChatModelV1 contract.
• EmbeddingsUnitTests, EmbeddingsIntegrationTests — embedding shape, batching, async.
• ToolsUnitTests, ToolsIntegrationTests — BaseTool round-trips and schema generation.
• VectorStoreUnitTests, VectorStoreIntegrationTests — vector store CRUD and similarity search.
• BaseStoreUnitTests — KV store basic operations.
• RetrieverUnitTests — BaseRetriever shape.
• CacheUnitTests — cache hit/miss behavior.

Each base class declares abstract @property methods that the partner test fills in (e.g. chat_model_class, chat_model_params). The tests themselves are concrete and shared.

How a partner uses it

# libs/partners/openai/tests/integration_tests/chat_models/test_standard.py
from langchain_tests.integration_tests import ChatModelIntegrationTests
from langchain_openai import ChatOpenAI

class TestChatOpenAIStandard(ChatModelIntegrationTests):
    @property
    def chat_model_class(self):
        return ChatOpenAI

    @property
    def chat_model_params(self):
        return {"model": "gpt-5", "stream_usage": True}

    @property
    def supports_image_inputs(self):
        return True

    @property
    def supports_audio_inputs(self):
        return True

Capability flags (supports_image_inputs, supports_audio_inputs, supports_tool_choice, has_structured_output, returns_usage_metadata, …) let a partner declare what their model actually supports; the test base skips suites that don't apply.

Running standard tests

From a partner's package directory:

cd libs/partners/openai
make test                  # unit tests including standard unit tests
make integration_test      # integration tests including standard integration tests

The standard tests are registered as discovered pytest classes — no extra config needed beyond inheriting from the right base.

Integration points

• Imports langchain-core types (AIMessage, BaseTool, Document, …).
• Imports the langchain package for some agent/middleware integration tests.
• Used by every partner package in this repo plus partners in sibling repos (langchain-google, langchain-aws, langchain-cohere, …).

Why standard tests matter

Without this package, every partner would write its own ad-hoc tests, and behavioral drift between providers would be inevitable. The standard suite is how the project enforces:

• Tool calling round-trips work the same way (input message → tool call → tool message → final response).
• Streaming usage metadata accumulates correctly when stream_usage=True.
• Structured output parses to the requested schema regardless of provider.
• Error types raised on rate limit, auth failure, invalid input are consistent.
• Async and sync invocations produce identical content.

A partner failing a standard test is treated as a bug in the partner, not in the standard test, except when the test itself has a known limitation (in which case the partner skips it via a capability flag).

Entry points for modification

• To add a new behavior to enforce across all partners, add a test method to the appropriate base class in libs/standard-tests/langchain_tests/integration_tests/ or unit_tests/. Open coordinated PRs against partners that fail.
• To add a new capability flag, add the property to the base class with a default and document it. Partners override when they support the feature.
• Helpers shared across base classes live in libs/standard-tests/langchain_tests/utils/.

Related

• packages/core — the abstractions partners implement
• partners — the consumers of this package
• how-to-contribute/testing — how testing is organized in this repo