# pydantic\_ai.models.outlines

## Setup

For details on how to set up this model, see [model configuration for Outlines](/docs/ai/models/outlines).

### OutlinesModel

**Bases:** `Model`

A model that relies on the Outlines library to run non API-based models.

#### Methods

##### \_\_init\_\_

```python
def __init__(
    model: OutlinesBaseModel | OutlinesAsyncBaseModel,
    provider: Literal['outlines'] | Provider[OutlinesBaseModel] = 'outlines',
    profile: ModelProfileSpec | None = None,
    settings: ModelSettings | None = None,
)
```

Initialize an Outlines model.

###### Parameters

**`model`** : `OutlinesBaseModel` | `OutlinesAsyncBaseModel`

The Outlines model used for the model.

**`provider`** : [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['outlines'\] | `Provider`\[`OutlinesBaseModel`\] _Default:_ `'outlines'`

The provider to use for OutlinesModel. Can be either the string 'outlines' or an instance of `Provider[OutlinesBaseModel]`. If not provided, the other parameters will be used.

**`profile`** : `ModelProfileSpec` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The model profile to use. Defaults to a profile picked by the provider.

**`settings`** : [`ModelSettings`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ModelSettings) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Default model settings for this model instance.

##### from\_transformers

`@classmethod`

```python
def from_transformers(
    cls,
    hf_model: transformers.modeling_utils.PreTrainedModel,
    hf_tokenizer_or_processor: transformers.PreTrainedTokenizer | transformers.processing_utils.ProcessorMixin,
    provider: Literal['outlines'] | Provider[OutlinesBaseModel] = 'outlines',
    profile: ModelProfileSpec | None = None,
    settings: ModelSettings | None = None,
)
```

Create an Outlines model from a Hugging Face model and tokenizer.

###### Parameters

**`hf_model`** : `transformers.modeling_utils.PreTrainedModel`

The Hugging Face PreTrainedModel or any model that is compatible with the `transformers` API.

**`hf_tokenizer_or_processor`** : `transformers.PreTrainedTokenizer` | `transformers.processing_utils.ProcessorMixin`

Either a HuggingFace `PreTrainedTokenizer` or any tokenizer that is compatible with the `transformers` API, or a HuggingFace processor inheriting from `ProcessorMixin`. If a tokenizer is provided, a regular model will be used, while if you provide a processor, it will be a multimodal model.

**`provider`** : [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['outlines'\] | `Provider`\[`OutlinesBaseModel`\] _Default:_ `'outlines'`

The provider to use for OutlinesModel. Can be either the string 'outlines' or an instance of `Provider[OutlinesBaseModel]`. If not provided, the other parameters will be used.

**`profile`** : `ModelProfileSpec` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The model profile to use. Defaults to a profile picked by the provider.

**`settings`** : [`ModelSettings`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ModelSettings) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Default model settings for this model instance.

##### from\_llamacpp

`@classmethod`

```python
def from_llamacpp(
    cls,
    llama_model: llama_cpp.Llama,
    provider: Literal['outlines'] | Provider[OutlinesBaseModel] = 'outlines',
    profile: ModelProfileSpec | None = None,
    settings: ModelSettings | None = None,
)
```

Create an Outlines model from a LlamaCpp model.

###### Parameters

**`llama_model`** : `llama_cpp.Llama`

The llama\_cpp.Llama model to use.

**`provider`** : [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['outlines'\] | `Provider`\[`OutlinesBaseModel`\] _Default:_ `'outlines'`

The provider to use for OutlinesModel. Can be either the string 'outlines' or an instance of `Provider[OutlinesBaseModel]`. If not provided, the other parameters will be used.

**`profile`** : `ModelProfileSpec` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The model profile to use. Defaults to a profile picked by the provider.

**`settings`** : [`ModelSettings`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ModelSettings) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Default model settings for this model instance.

##### from\_mlxlm

`@classmethod`

```python
def from_mlxlm(
    cls,
    mlx_model: nn.Module,
    mlx_tokenizer: transformers.PreTrainedTokenizer,
    provider: Literal['outlines'] | Provider[OutlinesBaseModel] = 'outlines',
    profile: ModelProfileSpec | None = None,
    settings: ModelSettings | None = None,
)
```

Create an Outlines model from a MLXLM model.

###### Parameters

**`mlx_model`** : `nn.Module`

The nn.Module model to use.

**`mlx_tokenizer`** : `transformers.PreTrainedTokenizer`

The PreTrainedTokenizer to use.

**`provider`** : [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['outlines'\] | `Provider`\[`OutlinesBaseModel`\] _Default:_ `'outlines'`

The provider to use for OutlinesModel. Can be either the string 'outlines' or an instance of `Provider[OutlinesBaseModel]`. If not provided, the other parameters will be used.

**`profile`** : `ModelProfileSpec` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The model profile to use. Defaults to a profile picked by the provider.

**`settings`** : [`ModelSettings`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ModelSettings) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Default model settings for this model instance.

##### from\_sglang

`@classmethod`

```python
def from_sglang(
    cls,
    base_url: str,
    api_key: str | None = None,
    model_name: str | None = None,
    provider: Literal['outlines'] | Provider[OutlinesBaseModel] = 'outlines',
    profile: ModelProfileSpec | None = None,
    settings: ModelSettings | None = None,
)
```

Create an Outlines model to send requests to an SGLang server.

###### Parameters

**`base_url`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str)

The url of the SGLang server.

**`api_key`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The API key to use for authenticating requests to the SGLang server.

**`model_name`** : [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The name of the model to use.

**`provider`** : [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['outlines'\] | `Provider`\[`OutlinesBaseModel`\] _Default:_ `'outlines'`

The provider to use for OutlinesModel. Can be either the string 'outlines' or an instance of `Provider[OutlinesBaseModel]`. If not provided, the other parameters will be used.

**`profile`** : `ModelProfileSpec` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The model profile to use. Defaults to a profile picked by the provider.

**`settings`** : [`ModelSettings`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ModelSettings) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Default model settings for this model instance.

##### from\_vllm\_offline

`@classmethod`

```python
def from_vllm_offline(
    cls,
    vllm_model: Any,
    provider: Literal['outlines'] | Provider[OutlinesBaseModel] = 'outlines',
    profile: ModelProfileSpec | None = None,
    settings: ModelSettings | None = None,
)
```

Create an Outlines model from a vLLM offline inference model.

###### Parameters

**`vllm_model`** : [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)

The vllm.LLM local model to use.

**`provider`** : [`Literal`](https://docs.python.org/3/library/typing.html#typing.Literal)\['outlines'\] | `Provider`\[`OutlinesBaseModel`\] _Default:_ `'outlines'`

The provider to use for OutlinesModel. Can be either the string 'outlines' or an instance of `Provider[OutlinesBaseModel]`. If not provided, the other parameters will be used.

**`profile`** : `ModelProfileSpec` | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

The model profile to use. Defaults to a profile picked by the provider.

**`settings`** : [`ModelSettings`](/docs/ai/api/pydantic-ai/settings/#pydantic_ai.settings.ModelSettings) | [`None`](https://docs.python.org/3/library/constants.html#None) _Default:_ `None`

Default model settings for this model instance.

##### format\_inference\_kwargs

```python
def format_inference_kwargs(model_settings: ModelSettings | None) -> dict[str, Any]
```

Format the model settings for the inference kwargs.

###### Returns

[`dict`](https://docs.python.org/3/reference/expressions.html#dict)\[[`str`](https://docs.python.org/3/library/stdtypes.html#str), [`Any`](https://docs.python.org/3/library/typing.html#typing.Any)\]

### OutlinesStreamedResponse

**Bases:** `StreamedResponse`

Implementation of `StreamedResponse` for Outlines models.

#### Attributes

##### model\_name

Get the model name of the response.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str)

##### provider\_name

Get the provider name.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str)

##### provider\_url

Get the provider base URL.

**Type:** [`str`](https://docs.python.org/3/library/stdtypes.html#str) | [`None`](https://docs.python.org/3/library/constants.html#None)

##### timestamp

Get the timestamp of the response.

**Type:** [`datetime`](https://docs.python.org/3/library/datetime.html#module-datetime)