Nov 4, 2024 · Oct 31, 2024 · Oct 31, 2024 · Oct 31, 2024 · Oct 31, 2024 · Nov 1, 2024
diff --git a/docs/concepts/experimental.md b/docs/concepts/experimental.md
 from pydantic import TypeAdapter


 class Foobar(TypedDict):
 class Foobar(TypedDict): # (1)!
    a: int
    b: NotRequired[float]
    c: NotRequired[Annotated[str, MinLen(5)]]


 ta = TypeAdapter(List[Foobar])

 v = ta.validate_json('[{"a": 1, "b"', experimental_allow_partial=True)  # (1)!
 v = ta.validate_json('[{"a": 1, "b"', experimental_allow_partial=True)  # (2)!
 print(v)
 #> [{'a': 1}]

 v = ta.validate_json(
    '[{"a": 1, "b": 1.0, "c": "abcd', experimental_allow_partial=True  # (2)!
    '[{"a": 1, "b": 1.0, "c": "abcd', experimental_allow_partial=True  # (3)!
 )
 print(v)
 #> [{'a': 1, 'b': 1.0}]

 v = ta.validate_json(
    '[{"b": 1.0, "c": "abcde"', experimental_allow_partial=True  # (3)!
    '[{"b": 1.0, "c": "abcde"', experimental_allow_partial=True  # (4)!
 )
 print(v)
 #> []
 print(v)
 #> [{'a': 1, 'b': 1.0, 'c': 'abcde'}]

 v = ta.validate_python([{'a': 1}], experimental_allow_partial=True)  # (4)!
 v = ta.validate_python([{'a': 1}], experimental_allow_partial=True)  # (5)!
 print(v)
 #> [{'a': 1}]

 v = ta.validate_python(
    [{'a': 1, 'b': 1.0, 'c': 'abcd'}], experimental_allow_partial=True  # (5)!
    [{'a': 1, 'b': 1.0, 'c': 'abcd'}], experimental_allow_partial=True  # (6)!
 )
 print(v)
 #> [{'a': 1, 'b': 1.0}]
 ```

 1. Parsing JSON, the input is valid JSON up to the point where the string is truncated.
 2. In this case truncation of the input means the value of `c` (`abcd`) is invalid as input to `c` field, hence it's omitted.
 3. The `a` field is required, so validation on the only item in the list fails and is dropped.
 4. Partial validation also works with Python objects, it should have the same semantics as with JSON except of course you can't have a genuinely "incomplete" Python object.
 5. The same as above but with a Python object, `c` is dropped as it's not required and failed validation.
 1. The TypedDict `Foobar` has three field, but only `a` is required, that means that a valid instance of `Foobar` can be created even if the `b` and `c` fields are missing.
 2. Parsing JSON, the input is valid JSON up to the point where the string is truncated.
 3. In this case truncation of the input means the value of `c` (`abcd`) is invalid as input to `c` field, hence it's omitted.
 4. The `a` field is required, so validation on the only item in the list fails and is dropped.
 5. Partial validation also works with Python objects, it should have the same semantics as with JSON except of course you can't have a genuinely "incomplete" Python object.
 6. The same as above but with a Python object, `c` is dropped as it's not required and failed validation.

 ### How Partial Validation Works

 Partial validation follows the zen of Pydantic- it makes no guarantees about what the input datamust be, but it does guarantee to return a valid instance of the type youprovide, or raise a validation error.
 Partial validation follows the zen of Pydantic— it makes no guarantees about what the input datamight have been, but it does guarantee to return a valid instance of the type yourequired, or raise a validation error.

 To do this, the `experimental_allow_partial` flag enables two pieces of behavior:


 For example:

 * if a string has a constraint `MinLen(5)`, when you only see part of the input, validation might fail because part of the string is missing (e.g. `Sam` instead of `Samuel`)
 * if a string has a constraint `MinLen(5)`, when you only see part of the input, validation might fail because part of the string is missing (e.g. `{"name": "Sam` instead of `{"name": "Samuel"}`)
 * if an `int` field has a constraint `Ge(10)`, when you only see part of the input, validation might fail because the number is too small (e.g. `1` instead of `10`)
 * if a `BaseModel` fieldexpects 3 fields, but the partial input only has two of the fields, validationmight fail because some field are missing
 * etc. etc.
 * if a `TypedDict` fieldhas 3requiredfields, but the partial input only has two of the fields, validationwould fail because some field are missing
 * etc. etc. — there are lost more cases like this

 The point is that if you only see part of some valid input data, validation errors can often occur in the last element of a sequenceof value of mapping.
 The point is that if you only see part of some valid input data, validation errors can often occur in the last element of a sequenceor last value of mapping.

 To avoid these errors breaking partial validation, Pydantic will ignore ALL errors in the last element of the input data.


 ### Limitations of Partial Validation

 ####TypAdapter only
 ####TypeAdapter only

 You can only pass `experiment_allow_partial` to [`TypeAdapter`][pydantic.TypeAdapter] methods, it's not yet supported via other Pydantic entry points like [`BaseModel`][pydantic.BaseModel].

 #### Types supported

 Right now onlythe following know how to handle partial validation:
 Right now onlya subset of collection validators know how to handle partial validation:

 - `list`
 - `set`
 - `frozenset`
 - `dict` (as in `dict[X, Y]`)
 - `TypedDict`(only`NotRequired` fields may be missing)
 - `TypedDict`—onlynon-required fields may be missing, e.g. via [`NotRequired`][typing.NotRequired] or [`total=False`][typing.TypedDict.__total__])

 While you can use `experimental_allow_partial` while validating against types that include othercomposite types, those types will be validated "all or nothing", and partial validation will not work on more nested types.
 While you can use `experimental_allow_partial` while validating against types that include othercollection validators, those types will be validated "all or nothing", and partial validation will not work on more nested types.

 E.g. in the [above](#2-ignore-errors-in-last) example partial validation works although the second item in the list is dropped completely since `BaseModel` doesn't (yet) support partial validation.

 But partial validation won't work at all inthisexample because `BaseModel` doesn't support partial validation so it doesn't forward the `allow_partial` instructionon to the list validator in `b`:
 But partial validation won't work at all inthe followexample because `BaseModel` doesn't support partial validation so it doesn't forward the `allow_partial` instructiondown to the list validator in `b`:

 ```py
 from typing import List

 class MyModel(BaseModel):
    a: int = 1
    b: List[Annotated[str, MinLen(5)]] = ['foobar']  # (1)!
    b: List[Annotated[str, MinLen(5)]] = []  # (1)!


 ta = TypeAdapter(MyModel)
    """
 ```

 1. The list validator for `b` doesn't get the `allow_partial` instruction so it doesn't know to ignore errors in the last element of the input.
 1. The list validator for `b` doesn't get the `allow_partial` instructionpassed down to it by the model validtorso it doesn't know to ignore errors in the last element of the input.

 #### Some invalid but complete JSON will be accepted


 #### Any error in the last field of the input will be ignored

 As described [above](#2-ignore-errors-in-last),any errorin the lastfield of the inputwill be ignored.
 As described [above](#2-ignore-errors-in-last),many errors can result from truncating the input. Rather than trying to specifically ignore errors that could result from truncation, Pydantic ignores all errorsin the lastelement of the inputin partial validation mode.

 This means clearly invalid data will pass validation if theerrors is in the last field of the input:
 This means clearly invalid data will pass validation if theerror is in the last field of the input:

 ```py
 from typing import List
diff --git a/pdm.lock b/pdm.lock
diff --git a/pyproject.toml b/pyproject.toml
 requires-python = '>=3.8'
 dependencies = [
    "typing-extensions>=4.12.2",
 'annotated-types>=0.6.0',
 "annotated-types>=0.6.0",
    "pydantic-core @ git+https://github.com/pydantic/pydantic-core.git@allow_partial",
 ]
 dynamic = ['version', 'readme']
Original file line number	Diff line number	Diff line change
Expand Up		@@ -176,26 +176,26 @@ from typing_extensions import Annotated, NotRequired, TypedDict
		from pydantic import TypeAdapter


		class Foobar(TypedDict):
		class Foobar(TypedDict): # (1)!
		a: int
		b: NotRequired[float]
		c: NotRequired[Annotated[str, MinLen(5)]]


		ta = TypeAdapter(List[Foobar])

		v = ta.validate_json('[{"a": 1, "b"', experimental_allow_partial=True) # (1)!
		v = ta.validate_json('[{"a": 1, "b"', experimental_allow_partial=True) # (2)!
		print(v)
		#> [{'a': 1}]

		v = ta.validate_json(
		'[{"a": 1, "b": 1.0, "c": "abcd', experimental_allow_partial=True # (2)!
		'[{"a": 1, "b": 1.0, "c": "abcd', experimental_allow_partial=True # (3)!
		)
		print(v)
		#> [{'a': 1, 'b': 1.0}]

		v = ta.validate_json(
		'[{"b": 1.0, "c": "abcde"', experimental_allow_partial=True # (3)!
		'[{"b": 1.0, "c": "abcde"', experimental_allow_partial=True # (4)!
		)
		print(v)
		#> []
Expand All		@@ -206,26 +206,27 @@ v = ta.validate_json(
		print(v)
		#> [{'a': 1, 'b': 1.0, 'c': 'abcde'}]

		v = ta.validate_python([{'a': 1}], experimental_allow_partial=True) # (4)!
		v = ta.validate_python([{'a': 1}], experimental_allow_partial=True) # (5)!
		print(v)
		#> [{'a': 1}]

		v = ta.validate_python(
		[{'a': 1, 'b': 1.0, 'c': 'abcd'}], experimental_allow_partial=True # (5)!
		[{'a': 1, 'b': 1.0, 'c': 'abcd'}], experimental_allow_partial=True # (6)!
		)
		print(v)
		#> [{'a': 1, 'b': 1.0}]
		```

		1. Parsing JSON, the input is valid JSON up to the point where the string is truncated.
		2. In this case truncation of the input means the value of `c` (`abcd`) is invalid as input to `c` field, hence it's omitted.
		3. The `a` field is required, so validation on the only item in the list fails and is dropped.
		4. Partial validation also works with Python objects, it should have the same semantics as with JSON except of course you can't have a genuinely "incomplete" Python object.
		5. The same as above but with a Python object, `c` is dropped as it's not required and failed validation.
		1. The TypedDict `Foobar` has three field, but only `a` is required, that means that a valid instance of `Foobar` can be created even if the `b` and `c` fields are missing.
		2. Parsing JSON, the input is valid JSON up to the point where the string is truncated.
		3. In this case truncation of the input means the value of `c` (`abcd`) is invalid as input to `c` field, hence it's omitted.
		4. The `a` field is required, so validation on the only item in the list fails and is dropped.
		5. Partial validation also works with Python objects, it should have the same semantics as with JSON except of course you can't have a genuinely "incomplete" Python object.
		6. The same as above but with a Python object, `c` is dropped as it's not required and failed validation.

		### How Partial Validation Works

		Partial validation follows the zen of Pydantic- it makes no guarantees about what the input datamust be, but it does guarantee to return a valid instance of the type youprovide, or raise a validation error.
		Partial validation follows the zen of Pydantic— it makes no guarantees about what the input datamight have been, but it does guarantee to return a valid instance of the type yourequired, or raise a validation error.

		To do this, the `experimental_allow_partial` flag enables two pieces of behavior:

Expand All		@@ -243,12 +244,12 @@ Only having access to part of the input data means errors can commonly occur in

		For example:

		* if a string has a constraint `MinLen(5)`, when you only see part of the input, validation might fail because part of the string is missing (e.g. `Sam` instead of `Samuel`)
		* if a string has a constraint `MinLen(5)`, when you only see part of the input, validation might fail because part of the string is missing (e.g. `{"name": "Sam` instead of `{"name": "Samuel"}`)
		* if an `int` field has a constraint `Ge(10)`, when you only see part of the input, validation might fail because the number is too small (e.g. `1` instead of `10`)
		* if a `BaseModel` fieldexpects 3 fields, but the partial input only has two of the fields, validationmight fail because some field are missing
		* etc. etc.
		* if a `TypedDict` fieldhas 3requiredfields, but the partial input only has two of the fields, validationwould fail because some field are missing
		* etc. etc. — there are lost more cases like this

		The point is that if you only see part of some valid input data, validation errors can often occur in the last element of a sequenceof value of mapping.
		The point is that if you only see part of some valid input data, validation errors can often occur in the last element of a sequenceor last value of mapping.

		To avoid these errors breaking partial validation, Pydantic will ignore ALL errors in the last element of the input data.

Expand DownExpand Up		@@ -277,25 +278,25 @@ print(v)

		### Limitations of Partial Validation

		####TypAdapter only
		####TypeAdapter only

		You can only pass `experiment_allow_partial` to [`TypeAdapter`][pydantic.TypeAdapter] methods, it's not yet supported via other Pydantic entry points like [`BaseModel`][pydantic.BaseModel].

		#### Types supported

		Right now onlythe following know how to handle partial validation:
		Right now onlya subset of collection validators know how to handle partial validation:

		- `list`
		- `set`
		- `frozenset`
		- `dict` (as in `dict[X, Y]`)
		- `TypedDict`(only`NotRequired` fields may be missing)
		- `TypedDict`—onlynon-required fields may be missing, e.g. via [`NotRequired`][typing.NotRequired] or [`total=False`][typing.TypedDict.__total__])

		While you can use `experimental_allow_partial` while validating against types that include othercomposite types, those types will be validated "all or nothing", and partial validation will not work on more nested types.
		While you can use `experimental_allow_partial` while validating against types that include othercollection validators, those types will be validated "all or nothing", and partial validation will not work on more nested types.

		E.g. in the [above](#2-ignore-errors-in-last) example partial validation works although the second item in the list is dropped completely since `BaseModel` doesn't (yet) support partial validation.

		But partial validation won't work at all inthisexample because `BaseModel` doesn't support partial validation so it doesn't forward the `allow_partial` instructionon to the list validator in `b`:
		But partial validation won't work at all inthe followexample because `BaseModel` doesn't support partial validation so it doesn't forward the `allow_partial` instructiondown to the list validator in `b`:

		```py
		from typing import List
Expand All		@@ -308,7 +309,7 @@ from pydantic import BaseModel, TypeAdapter, ValidationError

		class MyModel(BaseModel):
		a: int = 1
		b: List[Annotated[str, MinLen(5)]] = ['foobar'] # (1)!
		b: List[Annotated[str, MinLen(5)]] = [] # (1)!


		ta = TypeAdapter(MyModel)
Expand All		@@ -325,7 +326,7 @@ except ValidationError as e:
		"""
		```

		1. The list validator for `b` doesn't get the `allow_partial` instruction so it doesn't know to ignore errors in the last element of the input.
		1. The list validator for `b` doesn't get the `allow_partial` instructionpassed down to it by the model validtorso it doesn't know to ignore errors in the last element of the input.
samuelcolvin marked this conversation as resolved. Show resolvedHide resolved

		#### Some invalid but complete JSON will be accepted

Expand DownExpand Up		@@ -365,9 +366,9 @@ print(v)

		#### Any error in the last field of the input will be ignored

		As described [above](#2-ignore-errors-in-last),any errorin the lastfield of the inputwill be ignored.
		As described [above](#2-ignore-errors-in-last),many errors can result from truncating the input. Rather than trying to specifically ignore errors that could result from truncation, Pydantic ignores all errorsin the lastelement of the inputin partial validation mode.

		This means clearly invalid data will pass validation if theerrors is in the last field of the input:
		This means clearly invalid data will pass validation if theerror is in the last field of the input:

		```py
		from typing import List
Expand Down
Original file line number	Diff line number	Diff line change
Expand Up		@@ -49,7 +49,7 @@ classifiers = [
		requires-python = '>=3.8'
		dependencies = [
		"typing-extensions>=4.12.2",
		'annotated-types>=0.6.0',
		"annotated-types>=0.6.0",
		"pydantic-core @ git+https://github.com/pydantic/pydantic-core.git@allow_partial",
		]
		dynamic = ['version', 'readme']
Expand Down