Upgrading to Pydantic 2: More quirky than you would expect.

illustrations illustrations illustrations illustrations illustrations illustrations

Upgrading to Pydantic 2: More quirky than you would expect.

Published on Nov 27, 2023 by Sep Dehpour

Table Of Contents

Bye bye class Config 

The Pydantic V1 behavior to create a class called Config in the namespace of the parent BaseModel. Now, you need to add a class attribute call model_config.

class Schema(BaseModel):
    class Config:
        orm_mode = True

becomes

class Schema(BaseModel):
    model_config = ConfigDict(from_attributes=True)

Even the names of the attributes have changed. orm_mode is renamed to from_attributes.

Read more here.

Pydantic 2 does not automatically coerce to the type you want to 

In Pydantic 1, it tried to coerce the type when it could. For example:

from pydantic import BaseModel


class Schema(BaseModel):
    path: str

m = Schema(path=123)
>>> m.path
'123'

In Pydantic 2:

>>> m.path
Traceback (most recent call last):
pydantic_core._pydantic_core.ValidationError: 1 validation error for Schema
path
  Input should be a valid string [type=string_type, input_value=123, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/string_type

You have to explicitly allow specific type conversions:

from pydantic import BaseModel, ConfigDict


class Schema(BaseModel):
    path: str
    model_config = ConfigDict(coerce_numbers_to_str=True)


m = Schema(path=123)
>>> m.path
'123'

Private field names don’t work with Pydantic 2 

If you have field names in pydantic 1 that started with _, you will have to rename them to be public names.

For example I had private field names in our data such as _id, and _history that worked perfectly fine with Pydantic 1. Once upgraded to Pydantic 2, I tried to monkey patch Pydantic 2 so it reads those fields.

import pydantic

def is_valid_field_name(name: str) -> bool:
    """
    Monkey patching Pydantic's check for valid field names because it is a breaking change
    that is causing our code from Pydantic 1 not to work.
    """
    return not name.startswith('__')

pydantic._internal._fields.is_valid_field_name = is_valid_field_name
pydantic._internal._model_construction.is_valid_field_name = is_valid_field_name

However, patching worked for simple cases but there were edge cases that Pydantic was still throwing exceptions. The Pydantic writers recommend you to use the new PrivateAttr feature.

I found it to be useless. You can’t still set those attributes as you would with the rest of your model:

from datetime import datetime
from pydantic import BaseModel, PrivateAttr


class Schema(BaseModel):
    _processed_at: datetime


m = Schema(_processed_at=datetime.now())

>>> m._processed_at
KeyError: '_processed_at'
AttributeError: 'Schema' object has no attribute '_processed_at'

You have the option to use alias

from datetime import datetime
from pydantic import BaseModel, PrivateAttr, Field


class Schema(BaseModel):
    processed_at: datetime = Field(alias='_processed_at')


m = Schema(_processed_at=datetime.now())

In [15]: m.processed_at
Out[15]: datetime.datetime(2023, 11, 27, 12, 49, 4, 557375)

In [16]: m._processed_at
AttributeError: 'Schema' object has no attribute '_processed_at'

That means your original data can have a private field name. Once it is converted to the Pydantic model, the field name will change. That was not what exactly I wanted. What I needed was to keep the field names intact between our data and Pydantic objects.

After wrestling too many hours on this issue, I decided to change all private keys in my data to have the underline as a suffix instead of prefix. So in this case, the _processed_at became processed_at_.

Pydantic 2 is tricky when multiple types are allowed. 

from pydantic import BaseModel
from typing import List, Union


class Schema(BaseModel):
    path: List[Union[int, str]]

In Pydantic 1:

>>> obj = Schema(path=['0', 'item'])
>>> obj
Schema(path=[0, 'item'])

In Pydantic 2, even though int is the first allowed type, it goes for the str type. Even if you set the coerce_numbers_to_str=True in the model_config.

>>> obj = Schema(path=['0', 'item'])
>>> obj
Schema(path=['0', 'item'])

Pydantic 2 Optional fields are defined very differently 

In Pydantic 1, if you provided a default value for a field, it would become an optional field.

from pydantic import BaseModel, Field, ConfigDict


class Schema(BaseModel):
    history_id: int = None


m = Schema()
>>> m.id is None
True

In Pydantic 2: