Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

merge() method to merge two instances of the same model#3416

alkasm started this conversation inFeature Request
Discussion options

First, my motivating use-case. I have two sources of a model instance: a default that an application provides, and an override which gets parsed from a file (if it exists). As a library writer, to merge these two instances, I will have to implement some recursive merger that works on nested dictionaries so that my users don't have to manually implement merging in their application.

Python'sdict.update() is probably less useful since it's only the top-level fields (and can be implemented rather simply by sending both model instances to dicts, using update, then constructing a new instance). A different example which is maybe more interesting (but still intuitive) is protobuf'sMergeFrom() method:

This method merges the contents of the specified message into the current message. Singular fields that are set in the specified message overwrite the corresponding fields in the current message. Repeated fields are appended. Singular sub-messages and groups are recursively merged.

This would work great for my use-case.

For inspiration, some related Python merging libraries that I found:

  • deepmerge "Deepmerge is a flexible library to handle merging of nested data structures in Python (e.g. lists, dicts)."
  • jsonmerge "Merge a series of JSON documents." - has multiple built-inmerge strategies, and you can even provide a json schema to use different strategies per-field, which is pretty cool.
  • hiyapyco, "A Hierarchical Yaml Python Config"
You must be logged in to vote

Replies: 3 comments 5 replies

Comment options

I have a similar use case here, but TBH, I don't see the need for amerge(...) function: I usemodel_dump(...) to turn my pydantic data model instances into dicts and merge those using python built-in functionality (typically the| operator). Concise and works like a charm ✅.

You must be logged in to vote
5 replies
@rlkelly
Comment options

that would just overwrite the fields with the latest model. merge might mean addition of int fields, and things like that.

@dd-ssc
Comment options

Anaddition of int fields is well supported by python built-in functionality (e.g.|), butdeep merging of the dicts is required for anything non-trivial; see@mccauleyp's code sample.

@alkasm
Comment options

I think they meant literal addition of int values, e.g.{d: 5} + {d: 6} = {d: 11}; the point being that merge may have different semantics for different value types.

@dd-ssc
Comment options

Oh,that kind of addition! OK, didn't think of that - I thought ofadding model fields with integer type.
Thanks for pointing that out.

Your code sample is not a merge, though; in fact, it's not even valid python - missing quotes for starters, but more importantly,+ is not supported for dicts:

>>> {d: 5} + {d: 6}Traceback (most recent call last):  File "<python-input-0>", line 1, in <module>    {d: 5} + {d: 6}     ^NameError: name 'd' is not defined. Did you mean: 'id'?>>> {'d': 5} + {'d': 6}Traceback (most recent call last):  File "<python-input-1>", line 1, in <module>    {'d': 5} + {'d': 6}    ~~~~~~~~~^~~~~~~~~~TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

Also, why would anyone try to add numbers in model instance field value - and expect the result to be the sum ?
I can't seem to think of a use case - that does not involve mixing concerns.

Anyway, recipe for merging model instances: Turn them into dicts usingmodel_dump(...) and deepmerge those ✅

@alkasm
Comment options

Your code sample is not a merge, though; in fact, it's not even valid python - missing quotes for starters, but more importantly, + is not supported for dicts:

I was presenting it to communicate the idea, not as a literal Python example.

Also, why would anyone try to add numbers in model instance field value - and expect the result to be the sum ?
I can't seem to think of a use case - that does not involve mixing concerns.

Here's some valid Python thatdoes behave exactly in this way, to motivate:

>>>fromcollectionsimportCounter>>>Counter({"a":5})+Counter({"a":6})Counter({'a':11})
Comment options

Here's an example that's working well for my use-case:

fromtypingimportTypeVarfrompydanticimportBaseModelfromdeepmergeimportalways_mergerT=TypeVar("T",bound=BaseModel)defmerge_pydantic_models(base:T,nxt:T)->T:"""Merge two Pydantic model instances.    The attributes of 'base' and 'nxt' that weren't explicitly set are dumped into dicts    using '.model_dump(exclude_unset=True)', which are then merged using 'deepmerge',    and the merged result is turned into a model instance using '.model_validate'.    For attributes set on both 'base' and 'nxt', the value from 'nxt' will be used in    the output result.    """base_dict=base.model_dump(exclude_unset=True)nxt_dict=nxt.model_dump(exclude_unset=True)merged_dict=always_merger.merge(base_dict,nxt_dict)returnbase.model_validate(merged_dict)
You must be logged in to vote
0 replies
Comment options

I've come across use cases like this many times, but I think it should stay outside the scope of Pydantic. Generally speaking, there is no "one correct way" to merge fields of a model. Lists are probably fine, sure, but most everything else will conflict.

You must be logged in to vote
0 replies
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Labels
None yet
5 participants
@alkasm@rlkelly@dd-ssc@cdjameson@mccauleyp

[8]ページ先頭

©2009-2025 Movatter.jp