Skip to content
Docs Pydantic Docs

Groq

Install

To use GroqModel, you need to either install pydantic-ai, or install pydantic-ai-slim with the groq optional group:

Terminal
pip install "pydantic-ai-slim[groq]"

Configuration

To use Groq through their API, go to console.groq.com/keys and follow your nose until you find the place to generate an API key.

GroqModelName lists the latest Groq models for autocomplete. Any other model id the API accepts works too.

Environment variable

Once you have the API key, you can set it as an environment variable:

Terminal
export GROQ_API_KEY='your-api-key'

You can then use GroqModel by name:

from pydantic_ai import Agent

agent = Agent('groq:llama-3.3-70b-versatile')
...

Or initialise the model directly with just the model name:

from pydantic_ai import Agent
from pydantic_ai.models.groq import GroqModel

model = GroqModel('llama-3.3-70b-versatile')
agent = Agent(model)
...

provider argument

You can provide a custom Provider via the provider argument:

from pydantic_ai import Agent
from pydantic_ai.models.groq import GroqModel
from pydantic_ai.providers.groq import GroqProvider

model = GroqModel(
    'llama-3.3-70b-versatile', provider=GroqProvider(api_key='your-api-key')
)
agent = Agent(model)
...

You can also customize the GroqProvider with a custom httpx.AsyncClient:

from httpx import AsyncClient

from pydantic_ai import Agent
from pydantic_ai.models.groq import GroqModel
from pydantic_ai.providers.groq import GroqProvider

custom_http_client = AsyncClient(timeout=30)
model = GroqModel(
    'llama-3.3-70b-versatile',
    provider=GroqProvider(api_key='your-api-key', http_client=custom_http_client),
)
agent = Agent(model)
...

SDK retries

The AsyncGroq client that the provider builds also retries failed requests on its own — max_retries=2 by default, like the OpenAI client it is modelled on. Pass groq_client=AsyncGroq(max_retries=0) to keep the retry policy in your transport alone, or set ModelSettings['max_retries'] to change the count without building the client. See Provider SDK retries for where this layer sits.