Pydantic V2 Plan

#Breaking Changes & Compatibility 🙏

While we'll do our best to avoid breaking changes, some things will break.

As per the greatest pun in modern TV history.

You can't make a Tomelette without breaking some Greggs.

Where possible, if breaking changes are unavoidable, we'll try to provide warnings or errors to make sure those changes are obvious to developers.

#Performance 👍

As a result of the move to Rust for the validation logic (and significant improvements in how validation objects are structured) pydantic V2 will be significantly faster than pydantic V1.

Looking at the pydantic-core benchmarks today, pydantic V2 is between 4x and 50x faster than pydantic V1.9.1.

In general, pydantic V2 is about 17x faster than V1 when validating a model containing a range of common fields.

#Strict Mode 👍

People have long complained about pydantic for coercing data instead of throwing an error. E.g. input to an int field could be 123 or the string "123" which would be converted to 123 While this is very useful in many scenarios (think: URL parameters, environment variables, user input), there are some situations where it's not desirable.

pydantic-core comes with "strict mode" built in. With this, only the exact data type is allowed, e.g. passing "123" to an int field would result in a validation error.

This will allow pydantic V2 to offer a strict switch which can be set on either a model or a field.

#Formalised Conversion Table 👍

As well as complaints about coercion, another legitimate complaint was inconsistency around data conversion.

In pydantic V2, the following principle will govern when data should be converted in "lax mode" (strict=False):

If the input data has a SINGLE and INTUITIVE representation, in the field's type, AND no data is lost during the conversion, then the data will be converted; otherwise a validation error is raised. There is one exception to this rule: string fields - virtually all data has an intuitive representation as a string (e.g. repr() and str()), therefore a custom rule is required: only str, bytes and bytearray are valid as inputs to string fields.

Some examples of what that means in practice:

Field Type	Input	Single & Intuitive R.	All Data Preserved	Result
int	"123"			Convert
int	123.0			Convert
int	123.1			Error
date	"2020-01-01"			Convert
date	"2020-01-01T00:00:00"			Convert
date	"2020-01-01T12:00:00"			Error
int	b"1"			Error

(For the last case converting bytes to an int could reasonably mean int(bytes_data.decode()) or int.from_bytes(b'1', 'big/little'), hence an error)

In addition to the general rule, we'll provide a conversion table which defines exactly what data will be allowed to which field types. See the table below for a start on this.

#Built in JSON support 👍

pydantic-core can parse JSON directly into a model or output type, this both improves performance and avoids issue with strictness - e.g. if you have a strict model with a datetime field, the input must be a datetime object, but clearly that makes no sense when parsing JSON which has no datatime type. Same with bytes and many other types.

Pydantic V2 will therefore allow some conversion when validating JSON directly, even in strict mode (e.g. ISO8601 string -> datetime, str -> bytes) even though this would not be allowed when validating a python object.

In future direct validation of JSON will also allow:

• parsing in a separate thread while starting validation in the main thread
• line numbers from JSON to be included in the validation errors

(These features will not be included in V2, but instead will hopefully be added later.)

While in theory other formats could be specifically supported, the overheads and development time are significant and I don't think there's another format that's used widely enough to be worth specific logic. Other formats can be parsed to python then validated,similarly when serializing, data can be exported to a python object, then serialized, see below.

#Validation without a Model 👍

In pydantic V1 the core of all validation was a pydantic model, this led to a significant performance penalty and extra complexity when the output data type was not a model.

pydantic-core operates on a tree of validators with no "model" type required at the base of that tree. It can therefore validate a single string or datetime value, a TypedDict or a Model equally easily.

This feature will provide significant addition performance improvements in scenarios like:

• Adding validation to dataclasses
• Validating URL arguments, query strings, headers, etc. in FastAPI
• Adding validation to TypedDict
• Function argument validation
• Adding validation to your custom classes, decorators...

In effect - anywhere where you don't care about a traditional model class instance.

We'll need to add standalone methods for generating JSON Schema and dumping these objects to JSON, etc.

#Required vs. Nullable Cleanup 👍

Pydantic previously had a somewhat confused idea about "required" vs. "nullable". This mostly resulted from my misgivings about marking a field as Optional[int] but requiring a value to be provided but allowing it to be None - I didn't like using the word "optional" in relation to a field which was not optional.

In pydantic V2, pydantic will move to match dataclasses, thus:

from pydantic import BaseModel


class Foo(BaseModel):
    f1: str  # required, cannot be None
    f2: str | None  # required, can be None - same as Optional[str] / Union[str, None]
    f3: str | None = None  # not required, can be None
    f4: str = 'Foobar'  # not required, but cannot be None

#Validator Function Improvements 👍 👍 👍

This is one of the changes in pydantic V2 that I'm most excited about, I've been talking about something like this for a long time, see pydantic#1984, but couldn't find a way to do this until now.

Fields which use a function for validation can be any of the following types:

• function before mode - where the function is called before the inner validator is called
• function after mode - where the function is called after the inner validator is called
• plain mode - where there's no inner validator
• wrap mode - where the function takes a reference to a function which calls the inner validator, and can therefore modify the input before inner validation, modify the output after inner validation, conditionally not call the inner validator or catch errors from the inner validator and return a default value, or change the error

An example how a wrap validator might look:

from datetime import datetime
from pydantic import BaseModel, ValidationError, validator


class MyModel(BaseModel):
    timestamp: datetime

    @validator('timestamp', mode='wrap')
    def validate_timestamp(cls, v, handler):
        if v == 'now':
            # we don't want to bother with further validation,
            # just return the new value
            return datetime.now()
        try:
            return handler(v)
        except ValidationError:
            # validation failed, in this case we want to
            # return a default value
            return datetime(2000, 1, 1)

As well as being powerful, this provides a great "escape hatch" when pydantic validation doesn't do what you need.

#More powerful alias(es) 👍

pydantic-core can support alias "paths" as well as simple string aliases to flatten data as it's validated.

Best demonstrated with an example:

from pydantic import BaseModel, Field


class Foo(BaseModel):
    bar: str = Field(aliases=[['baz', 2, 'qux']])


data = {
    'baz': [
        {'qux': 'a'},
        {'qux': 'b'},
        {'qux': 'c'},
        {'qux': 'd'},
    ]
}

foo = Foo(**data)
assert foo.bar == 'c'

aliases is a list of lists because multiple paths can be provided, if so they're tried in turn until a value is found.

Tagged unions will use the same logic as aliases meaning nested attributes can be used to select a schema to validate against.

#Improvements to Dumping/Serialization/Export 👍 😕

(I haven't worked on this yet, so these ideas are only provisional)

There has long been a debate about how to handle converting data when extracting it from a model. One of the features people have long requested is the ability to convert data to JSON compliant types while converting a model to a dict.

My plan is to move data export into pydantic-core, with that, one implementation can support all export modes without compromising (and hopefully significantly improving) performance.

I see four different export/serialization scenarios:

1. Extracting the field values of a model with no conversion, effectively model.__dict__ but with the current filtering logic provided by .dict()
2. Extracting the field values of a model recursively (effectively what .dict() does now) - sub-models are converted to dicts, but other fields remain unchanged.
3. Extracting data and converting at the same time (e.g. to JSON compliant types)
4. Serializing data straight to JSON

I think all 4 modes can be supported in a single implementation, with a kind of "3.5" mode where a python function is used to convert the data as the user wishes.

The current include and exclude logic is extremely complicated, but hopefully it won't be too hard to translate it to Rust.

We should also add support for validate_alias and dump_alias as well as the standard alias to allow for customising field keys.

#Validation Context 👍

Pydantic V2 will add a new optional context argument to model_validate and model_validate_json which will allow you to pass information not available when creating a model to validators. See pydantic#1549 for motivation.

Here's an example of context might be used:

from pydantic import BaseModel, EmailStr, validator


class User(BaseModel):
    email: EmailStr
    home_country: str

    @validator('home_country')
    def check_home_country(cls, v, context):
        if v not in context['countries']:
            raise ValueError('invalid country choice')
        return v


async def add_user(post_data: bytes):
    countries = set(await db_connection.fetch_all('select code from country'))
    user = User.model_validate_json(post_data, context={'countries': countries})
    ...

#Model Namespace Cleanup 👍

For years I've wanted to clean up the model namespace, see pydantic#1001. This would avoid confusing gotchas when field names clash with methods on a model, it would also make it safer to add more methods to a model without risking new clashes.

After much deliberation (and even giving a lightning talk at the python language submit about alternatives, see this discussion). I've decided to go with the simplest and clearest approach, at the expense of a bit more typing:

All methods on models will start with model_, fields' names will not be allowed to start with "model" (aliases can be used if required).

This will mean BaseModel will have roughly the following signature.

class BaseModel:
    model_fields: List[FieldInfo]
    """previously `__fields__`, although the format will change a lot"""
    @classmethod
    def model_validate(cls, data: Any, *, context=None) -> Self:  # (1)
        """
        previously `parse_obj()`, validate data
        """
    @classmethod
    def model_validate_json(
        cls,
        data: str | bytes | bytearray,
        *,
        context=None
    ) -> Self:
        """
        previously `parse_raw(..., content_type='application/json')`
        validate data from JSON
        """
    @classmethod
    def model_is_instance(cls, data: Any, *, context=None) -> bool: # (2)
        """
        new, check if data is value for the model
        """
    @classmethod
    def model_is_instance_json(
        cls,
        data: str | bytes | bytearray,
        *,
        context=None
    ) -> bool:
        """
        Same as `model_is_instance`, but from JSON
        """
    def model_dump(
        self,
        include: ... = None,
        exclude: ... = None,
        by_alias: bool = False,
        exclude_unset: bool = False,
        exclude_defaults: bool = False,
        exclude_none: bool = False,
        mode: Literal['unchanged', 'dicts', 'json-compliant'] = 'unchanged',
        converter: Callable[[Any], Any] | None = None
    ) -> Any:
        """
        previously `dict()`, as before
        with new `mode` argument
        """
    def model_dump_json(self, ...) -> str:
        """
        previously `json()`, arguments as above
        effectively equivalent to `json.dump(self.model_dump(..., mode='json'))`,
        but more performant
        """
    def model_json_schema(self, ...) -> dict[str, Any]:
        """
        previously `schema()`, arguments roughly as before
        JSON schema as a dict
        """
    def model_update_forward_refs(self) -> None:
        """
        previously `update_forward_refs()`, update forward references
        """
    @classmethod
    def model_construct(
        self,
        _fields_set: set[str] | None = None,
        **values: Any
    ) -> Self:
        """
        previously `construct()`, arguments roughly as before
        construct a model with no validation
        """
    @classmethod
    def model_customize_schema(cls, schema: dict[str, Any]) -> dict[str, Any]:
        """
        new, way to customize validation,
        e.g. if you wanted to alter how the model validates certain types,
        or add validation for a specific type without custom types or
        decorated validators
        """
    class ModelConfig:
        """
        previously `Config`, configuration class for models
        """

1. see Validation Context for more information on context
2. see is_instance checks

The following methods will be removed:

• .parse_file() - was a mistake, should never have been in pydantic
• .parse_raw() - partially replaced by .model_validate_json(), the other functionality was a mistake
• .from_orm() - the functionality has been moved to config, see other improvements below
• .schema_json() - mostly since it causes confusion between pydantic validation schema and JSON schema, and can be replaced with just json.dumps(m.model_json_schema())
• .copy() instead we'll implement __copy__ and let people use the copy module (this removes some functionality) from copy() but there are bugs and ambiguities with the functionality anyway

#Strict API & API documentation 👍

When preparing for pydantic V2, we'll make a strict distinction between the public API and private functions & classes. Private objects will be clearly identified as private via a _internal sub package to discourage use.

The public API will have API documentation. I've recently been working with the wonderful mkdocstrings package for both dirty-equals and watchfiles documentation. I intend to use mkdocstrings to generate complete API documentation for V2.

This wouldn't replace the current example-based somewhat informal documentation style but instead will augment it.

#Error descriptions 👍

The way line errors (the individual errors within a ValidationError) are built has become much more sophisticated in pydantic-core.

There's a well-defined set of error codes and messages.

More will be added when other types are validated via pure python validators in pydantic.

I would like to add a dedicated section to the documentation with extra information for each type of error.

This would be another key in a line error: documentation, which would link to the appropriate section in the docs.

Thus, errors might look like:

[
    {
        'kind': 'greater_than_equal',
        'loc': ['age'],
        'message': 'Value must be greater than or equal to 18',
        'input_value': 11,
        'context': {'ge': 18},
        'documentation': 'https://pydantic.dev/errors/#greater_than_equal',
    },
    {
        'kind': 'bool_parsing',
        'loc': ['is_developer'],
        'message': 'Value must be a valid boolean, unable to interpret input',
        'input_value': 'foobar',
        'documentation': 'https://pydantic.dev/errors/#bool_parsing',
    },
]

I own the pydantic.dev domain and will use it for at least these errors so that even if the docs URL changes, the error will still link to the correct documentation. If developers don't want to show these errors to users, they can always process the errors list and filter out items from each error they don't need or want.

#No pure python implementation 😦

Since pydantic-core is written in Rust, and I have absolutely no intention of rewriting it in python, pydantic V2 will only work where a binary package can be installed.

pydantic-core will provide binaries in PyPI for (at least):

• Linux: x86_64, aarch64, i686, armv7l, musl-x86_64 & musl-aarch64
• MacOS: x86_64 & arm64 (except python 3.7)
• Windows: amd64 & win32
• Web Assembly: wasm32 (pydantic-core is already compiled for wasm32 using emscripten and unit tests pass, except where cpython itself has problems)

Binaries for pypy are a work in progress and will be added if possible, see pydantic-core#154.

Other binaries can be added provided they can be (cross-)compiled on github actions. If no binary is available from PyPI, pydantic-core can be compiled from source if Rust stable is available.

The only place where I know this will cause problems is Raspberry Pi, which is a mess when it comes to packages written in Rust for Python. Effectively, until that's fixed you'll likely have to install pydantic with pip install -i https://pypi.org/simple/ pydantic.

#Pydantic becomes a pure python package 👍

Pydantic V1.X is a pure python code base but is compiled with cython to provide some performance improvements. Since the "hot" code is moved to pydantic-core, pydantic itself can go back to being a pure python package.

This should significantly reduce the size of the pydantic package and make unit tests of pydantic much faster. In addition:

• some constraints on pydantic code can be removed once it no-longer has to be compilable with cython
• debugging will be easier as you'll be able to drop straight into the pydantic codebase as you can with other, pure python packages

Some pieces of edge logic could get a little slower as they're no longer compiled.

#is_instance like checks 👍

Strict mode also means it makes sense to provide an is_instance method on models which effectively run validation then throws away the result while avoiding the (admittedly small) overhead of creating and raising an error or returning the validation result.

To be clear, this isn't a real isinstance call, rather it is equivalent to

class BaseModel:
    ...

    @classmethod
    def model_is_instance(cls, data: Any) -> bool:
        try:
            cls(**data)
        except ValidationError:
            return False
        else:
            return True

#I'm dropping the word "parse" and just using "validate" 😐

Partly due to the issues with the lack of strict mode, I've gone back and forth between using the terms "parse" and "validate" for what pydantic does.

While pydantic is not simply a validation library (and I'm sure some would argue validation is not strictly what it does), most people use the word "validation".

It's time to stop fighting that, and use consistent names.

The word "parse" will no longer be used except when talking about JSON parsing, see model methods above.

#Changes to custom field types 😐

Since the core structure of validators has changed from "a list of validators to call one after another" to "a tree of validators which call each other", the __get_validators__ way of defining custom field types no longer makes sense.

Instead, we'll look for the attribute __pydantic_validation_schema__ which must be a pydantic-core compliant schema for validating data to this field type (the function item can be a string, if so a function of that name will be taken from the class, see 'validate' below).

Here's an example of how a custom field type could be defined:

from pydantic import ValidationSchema


class Foobar:
    def __init__(self, value: str):
        self.value = value

    __pydantic_validation_schema__: ValidationSchema = {
        'type': 'function',
        'mode': 'after',
        'function': 'validate',
        'schema': {'type': 'str'},
    }

    @classmethod
    def validate(cls, value):
        if 'foobar' in value:
            return Foobar(value)
        else:
            raise ValueError('expected foobar')

What's going on here: __pydantic_validation_schema__ defines a schema which effectively says:

Validate input data as a string, then call the validate function with that string, use the returned value as the final result of validation.

ValidationSchema is just an alias to pydantic_core.Schema which is a type defining the schema for validation schemas.