NotificationsYou must be signed in to change notification settings
Fork2.2k
Star24.6k

Add`experimental_allow_partial` support#10748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

sydney-runkle merged 15 commits intomainfromallow_partial

Nov 4, 2024

Merged

Add`experimental_allow_partial` support#10748

sydney-runkle merged 15 commits intomainfromallow_partial

Nov 4, 2024

Conversation

Copy link

Member

samuelcolvin commentedOct 31, 2024•
edited
Loading

This relies on:

partial support in jiter for JSON parsing, seepartial parsing withJsonValue jiter#157
partial support in pydantic-core, seeAddallow_partial pydantic-core#1512

So farallow_partial support is only implemented on validators for the following types:

list
set
frozenset
dict (e.g.dict[X, Y])
TypedDict

There are 2 fundamental things introduced:

(in jiter) support for parsing partial json, e.g. any valid JSON input which "stops early", for example[1, 2, "thr
(in pydantic-core) support for ignoring errors while validating the last item in a sequence or last value in a mapping (the last item can be invalid because the original data stopped part way through its definition)

Example:

frompydanticimportTypeAdapter,BaseModelfromtypingimportLiteralclassUserRecord(BaseModel):id:intname:strrole:Literal['admin','user']ta=TypeAdapter(list[UserRecord])# allow_partial if the input is a python objectd=ta.validate_python(    [        {'id':'1','name':'Alice','role':'user'},        {'id':'1','name':'Ben','role':'user'},        {'id':'1','name':'Char'},    ],experimental_allow_partial=True,)print(d)#> [UserRecord(id=1, name='Alice', role='user'), UserRecord(id=1, name='Ben', role='user')]# allow_partial if the input is a json stringd=ta.validate_json('[{"id":"1","name":"Alice","role":"user"},{"id":"1","name":"Ben","role":"user"},{"id":"1","name":"Char"}]',experimental_allow_partial=True,)print(d)#> [UserRecord(id=1, name='Alice', role='user'), UserRecord(id=1, name='Ben', role='user')]

github-actionsbot added the relnotes-fixUsed for bugfixes. label

Oct 31, 2024

Copy link

codspeed-hqbot commentedOct 31, 2024•
edited
Loading

CodSpeed Performance Report

Merging#10748 willnot alter performance

_{Comparingallow_partial (3ea36b0) withmain (b4308e0)}

Summary

✅ 44 untouched benchmarks

Copy link

cloudflare-workers-and-pagesbot commentedOct 31, 2024•
edited
Loading

Deploying pydantic-docs with Cloudflare Pages

Latest commit:	`3ea36b0`
Status:	✅ Deploy successful!
Preview URL:	https://ad0fa9a1.pydantic-docs.pages.dev
Branch Preview URL:	https://allow-partial.pydantic-docs.pages.dev

View logs

samuelcolvin added relnotes-feature and removed relnotes-fixUsed for bugfixes. labels

Oct 31, 2024

Copy link

Contributor

github-actionsbot commentedOct 31, 2024•
edited
Loading

Coverage report

This PR does not seem to contain any modification to coverable code.

samuelcolvin changed the title~~Addexperimental_allow_partial support toTypeAdapter~~Addexperimental_allow_partial support

Oct 31, 2024

samuelcolvin mentioned this pull request

Oct 31, 2024

Addallow_partialpydantic/pydantic-core#1512

Merged

sydney-runkle reviewed

Oct 31, 2024

View reviewed changes

Copy link

Contributor

sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is super exciting overall - definitely a fan of theexperimental_ prefix for now. I'm impressed with the turnaround here.

A few follow up questions:

We should add a section to theexperimental docs talking about this flag (happy to have this between the beta and official release)
I'm wondering, does it make sense to only have this as a runtime flag? Should we also have a config setting for this - we have one forcache_strings, which I believe has a similar flow down tojitter? Even if not on config, I think we should make this accessible for fields in models, typed dicts, etc.

As a general note on the above, I feel we don't have a super clear system for when something belongs as a:

Runtime flag
Config setting
Annotation
Field setting

And we should probably document that to guide consistency in future development.

tests/test_allow_partial.py OutdatedShow resolvedHide resolved

devraj mentioned this pull request

Oct 31, 2024

Support retrieving partial responses from endpoints (refactor for pydantic2.10)anomaly/gallagher#20

Open

hyperlint-aibot reviewed

Nov 1, 2024

View reviewed changes

Copy link

Contributor

hyperlint-aibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

1 files reviewed, 2 total issue(s) found.

The style guide flagged several spelling errors that seemed like false positives. We skipped posting inline suggestions for the following words:

[Aa]nnotated_types

docs/concepts/experimental.mdShow resolvedHide resolved

samuelcolvin mentioned this pull request

Nov 1, 2024

Fix documentation code examples not being upgraded#10755

Merged

hyperlint-aibot reviewed

Nov 2, 2024

View reviewed changes

Copy link

Contributor

hyperlint-aibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The style guide flagged several spelling errors that seemed like false positives. We skipped posting inline suggestions for the following words:

annotated_types

samuelcolvin added8 commits

November 2, 2024 16:23

add "experimental_allow_partial" support to TypeAdapter

98f0759

support 3.8

11e75c5

uprev to latest pydantic-core

d07b041

parmeterize tests

fc2ffea

add documentation, uprev to latest commit of core pr

3b4aee3

fix docs build

b47e3f9

fix tests

452e833

fix examples for 3.8

cd1647b

samuelcolvin force-pushed theallow_partial branch fromd01149c tocd1647bCompare

November 2, 2024 16:24

samuelcolvin added2 commits

November 3, 2024 10:36

format examples

d9d739b

uprev core pr, improve docs

48ea40e

hyperlint-aibot reviewed

Nov 3, 2024

View reviewed changes

Copy link

Contributor

hyperlint-aibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

1 files reviewed, 3 total issue(s) found.

docs/concepts/experimental.md OutdatedShow resolvedHide resolved

samuelcolvinand others added4 commits

November 3, 2024 11:56

fix typo

80187a7

Co-authored-by: hyperlint-ai[bot] <154288675+hyperlint-ai[bot]@users.noreply.github.com>

fix example formatting

3d24c16

bump core version

3ceda20

fix dt tests

40c84d7

hyperlint-aibot reviewed

Nov 4, 2024

View reviewed changes

Copy link

Contributor

hyperlint-aibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

1 files reviewed, 3 total issue(s) found.

docs/concepts/experimental.mdShow resolvedHide resolved

sydney-runkle reviewed

Nov 4, 2024

View reviewed changes

docs/concepts/experimental.md OutdatedShow resolvedHide resolved

Update docs/concepts/experimental.md

3ea36b0

hyperlint-aibot reviewed

Nov 4, 2024

View reviewed changes

Copy link

Contributor

hyperlint-aibot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The style guide flagged several spelling errors that seemed like false positives. We skipped posting inline suggestions for the following words:

jiter

sydney-runkle approved these changes

Nov 4, 2024

View reviewed changes

Copy link

Contributor

sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Amazing!

Thanks for including such thorough docs - those will be easy to move to the concepts section, eventually!

sydney-runkleenabled auto-merge (squash)

November 4, 2024 17:08

sydney-runkle merged commit678ec30 intomain

Nov 4, 2024

53 checks passed

sydney-runkle deleted the allow_partial branch

November 4, 2024 17:11

bruno-f-cruz mentioned this pull request

Nov 8, 2024

Assess the feasability of using NiceGUI to automatically generate simple formsAllenNeuralDynamics/clabe#12

Open

samuelcolvin mentioned this pull request

Nov 10, 2024

Allow partial needs'trailing-strings' mode, same aspydantic_core.from_json#10806

Closed

Labels

relnotes-feature

2 participants

Movatterモバイル変換

Uh oh!

Addexperimental_allow_partial support#10748

Addexperimental_allow_partial support#10748

Uh oh!

Conversation

samuelcolvin commentedOct 31, 2024• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Uh oh!

codspeed-hqbot commentedOct 31, 2024• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

CodSpeed Performance Report

Merging#10748 willnot alter performance

Summary

Uh oh!

cloudflare-workers-and-pagesbot commentedOct 31, 2024• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Deploying pydantic-docs with Cloudflare Pages

Uh oh!

github-actionsbot commentedOct 31, 2024• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Coverage report

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hyperlint-aibot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hyperlint-aibot left a comment

Choose a reason for hiding this comment

Uh oh!

hyperlint-aibot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hyperlint-aibot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hyperlint-aibot left a comment

Choose a reason for hiding this comment

Uh oh!

sydney-runkle left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Add`experimental_allow_partial` support#10748

Add`experimental_allow_partial` support#10748

samuelcolvin commentedOct 31, 2024•
edited
Loading

codspeed-hqbot commentedOct 31, 2024•
edited
Loading

cloudflare-workers-and-pagesbot commentedOct 31, 2024•
edited
Loading

github-actionsbot commentedOct 31, 2024•
edited
Loading