Not Required in Pydantic's Base Models

Question

Im trying to accept data from an API and then validate the response structure with a Pydantic base model. However, I have the case where sometimes some fields will not come included in the response, while sometimes they do. The problem is, when I try to validate the structure, Pydantic starts complaining about those fields being "missing" even though they can be missing sometimes. I really don't understand how to define a field as "missible". The docs mention that a field that is just defined as a name and a type is considered this way, but I haven't had any luck

This is a simple example of what I'm trying to accomplish

# Response: {a: 1, b: "abc", c: ["a", "b", "c"]}
response: dict = json.loads(request_response)

# Pydantic Base Model
from pydantic import BaseModel
class Model(BaseModel):
   a: int
   b: str
   c: List[str]
   d: float

# Validating 
Model(**response)

# Return: ValidationError - Missing "d" field

How do I make it so that "d" doesnt cause the validation to throw an error? I have tried to switch "d" to d: Optional[float] and d: Optional[float] = 0.0 , but nothing works.

Thanks!

Answer 1

As @python_user said, both your suggestions work.

Admittedly, the behavior of typing.Optional for Pydantic fields is poorly documented. Perhaps because it is assumed to be obvious. I personally don't find it obvious because Optional[T] is just equivalent to Union[T, None] (or T | None in the new notation).

Annotating a field with any other union of types, while omitting a default value will result in the field being required. But if you annotate with a union that includes None , the field automatically receives the None default value. Kind of inconsistent, but that is the way it is.

However, the question is ultimately what you want your model fields to be. What value should be assigned to field d , if it is not set explicitly? Should it be None or should it be some default float value? (like eg 0. ) If None is fine, then you can use Optional[float] or float | None float | None , and you don't need to specify the default value. (Specifying Optional[float] = None is equivalent.) If you want any other default value, you'll need to specify it accordingly, eg d: float = 0.0 ; but in that case None would be an invalid value for that field.

from pydantic import BaseModel, ValidationError


class Model(BaseModel):
    a: int
    b: str
    c: list[str]
    d: float | None  # or typing.Optional[float]
    e: float = 0.


if __name__ == '__main__':
    print(Model.parse_obj({
        "a": 1,
        "b": "abc",
        "c": ["a", "b", "c"],
    }), "\n")
    try:
        Model.parse_obj({
            "a": 1,
            "b": "abc",
            "c": ["a", "b", "c"],
            "e": None,
        })
    except ValidationError as e:
        print(e)

Output:

a=1 b='abc' c=['a', 'b', 'c'] d=None e=0.0 

1 validation error for Model
e
  none is not an allowed value (type=type_error.none.not_allowed)