Skip to content
Docs Pydantic Docs

Cohere

Install

To use CohereModel, you need to either install pydantic-ai, or install pydantic-ai-slim with the cohere optional group:

Terminal
pip install "pydantic-ai-slim[cohere]"

Configuration

To use Cohere through their API, go to dashboard.cohere.com/api-keys and follow your nose until you find the place to generate an API key.

CohereModelName contains a list of the most popular Cohere models.

Environment variable

Once you have the API key, you can set it as an environment variable:

Terminal
export CO_API_KEY='your-api-key'

You can then use CohereModel by name:

from pydantic_ai import Agent

agent = Agent('cohere:command-r7b-12-2024')
...

Or initialise the model directly with just the model name:

from pydantic_ai import Agent
from pydantic_ai.models.cohere import CohereModel

model = CohereModel('command-r7b-12-2024')
agent = Agent(model)
...

provider argument

You can provide a custom Provider via the provider argument:

from pydantic_ai import Agent
from pydantic_ai.models.cohere import CohereModel
from pydantic_ai.providers.cohere import CohereProvider

model = CohereModel('command-r7b-12-2024', provider=CohereProvider(api_key='your-api-key'))
agent = Agent(model)
...

You can also customize the CohereProvider with a custom http_client:

from httpx import AsyncClient

from pydantic_ai import Agent
from pydantic_ai.models.cohere import CohereModel
from pydantic_ai.providers.cohere import CohereProvider

custom_http_client = AsyncClient(timeout=30)
model = CohereModel(
    'command-r7b-12-2024',
    provider=CohereProvider(api_key='your-api-key', http_client=custom_http_client),
)
agent = Agent(model)
...

SDK retries

The Cohere client retries server errors, rate limits and status 408 and 409 twice by default, above whatever transport you pass here. It takes its retry count per request rather than per client, so set ModelSettings['max_retries'] to change it, for example to 0 to keep the retry policy in your transport alone. See Provider SDK retries for where this layer sits.

Model settings

You can customize model behavior using CohereModelSettings:

from pydantic_ai import Agent
from pydantic_ai.models.cohere import CohereModel, CohereModelSettings

model = CohereModel('command-r7b-12-2024')
settings = CohereModelSettings(
    temperature=0.2,
    top_k=40,
)
agent = Agent(model, model_settings=settings)
...