langchain-model-profiles

A small package whose entire purpose is to keep model capability metadata (context window, multimodal support, tool support, pricing) consistent across every partner integration. Source: libs/model-profiles/langchain_model_profiles/. PyPI: langchain-model-profiles.

Purpose

Every partner package ships a data/ directory of JSON files describing the models it supports. The files look roughly like:

{
  "gpt-5": {
    "context_window": 200000,
    "max_output_tokens": 32768,
    "supports_tool_choice": true,
    "supports_vision": true,
    "supports_audio_input": false,
    "input_token_price": 0.0,
    "output_token_price": 0.0,
    ...
  },
  ...
}

These profiles are consumed at runtime by BaseChatModel.profile (defined in libs/core/langchain_core/language_models/model_profile.py) so that downstream code can ask "does this model support tool calling?" without hard-coding provider knowledge.

The langchain-profiles CLI in this package downloads canonical capability data from models.dev and merges it with provider-specific augmentations from profile_augmentations.toml files in each partner's data/ directory. The result is the JSON that gets committed.

Directory layout

libs/model-profiles/langchain_model_profiles/
├── __init__.py            # (empty)
└── cli.py                 # The `langchain-profiles` CLI (~404 lines)

That's it — the package is just a CLI; the actual profile data is committed in each partner package.

Key abstractions

Symbol	File	Description
langchain-profiles refresh	libs/model-profiles/langchain_model_profiles/cli.py	Fetch latest model data from models.dev, merge with profile_augmentations.toml, write JSON
_validate_data_dir	libs/model-profiles/langchain_model_profiles/cli.py	Resolves and confirms the target directory, prompting before writing outside the cwd
_load_augmentations	libs/model-profiles/langchain_model_profiles/cli.py	Reads provider-specific overrides from profile_augmentations.toml

How it works

graph LR
    A[models.dev API]
    B[profile_augmentations.toml]
    C[langchain-profiles refresh]
    D["partner data/*.json"]
    E[BaseChatModel.profile]

    A -->|HTTP GET| C
    B -->|read| C
    C -->|write| D
    D -->|loaded at runtime| E

A typical refresh cycle:

cd libs/model-profiles
uv run langchain-profiles refresh \
  --provider openai \
  --data-dir ../partners/openai/langchain_openai/data

The CLI:

1. Fetches the latest model list and capability data from models.dev for the given provider.
2. Reads profile_augmentations.toml from the target --data-dir for any human-curated overrides (e.g. correcting a value, adding a model models.dev doesn't know about, marking a model as deprecated).
3. Writes the merged JSON files into the partner's data/ directory.
4. Sanity-checks the result against the runtime ModelProfile schema using get_type_hints.

When refreshes happen

Two GitHub workflows automate this:

• .github/workflows/refresh_model_profiles.yml — manual trigger, refreshes profiles for a chosen provider.
• .github/workflows/_refresh_model_profiles.yml — reusable workflow that other workflows call on a schedule.

The PRs the bot opens are labeled chore(profiles): refresh ... and merged after CI passes.

Why a separate package?

langchain-model-profiles is the only piece of the toolchain. The data itself lives in partner packages. Keeping the CLI standalone:

• Avoids forcing every partner to depend on the refresh tool.
• Lets the same CLI refresh profiles in sibling repositories (langchain-google, langchain-aws) by passing --data-dir to a path outside this repo (the CLI prompts before writing outside the current working directory).
• Means the schema lives next to the tool that produces it, while the partners only consume the JSON.

Integration points

• Reads from models.dev over HTTPS via httpx.
• Writes into any partner's data/ directory.
• Used by langchain-core's BaseChatModel.profile at runtime to expose capability flags.

Entry points for modification

• To add a new schema field, edit the ModelProfile definition in libs/core/langchain_core/language_models/model_profile.py, then update the merge logic in libs/model-profiles/langchain_model_profiles/cli.py to populate it from models.dev + augmentations.
• To augment a specific model's data, edit the profile_augmentations.toml file in the partner's data/ directory. The next refresh will preserve the augmentation.
• To support a new provider, add a mapping from models.dev provider name to the partner's data directory in the CLI, plus the partner-side glue that exposes the data via BaseChatModel.profile.

Related

• packages/core — defines ModelProfile and BaseChatModel.profile
• partners — every partner has a data/ directory consumed here
• CLAUDE.md at the repo root — documents the canonical CLI invocations