Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

bpo-29729: uuid.UUID now accepts bytes-like object#3801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
vstinner wants to merge8 commits intopython:masterfromvstinner:uuid_bytes
Closed

bpo-29729: uuid.UUID now accepts bytes-like object#3801

vstinner wants to merge8 commits intopython:masterfromvstinner:uuid_bytes

Conversation

@vstinner
Copy link
Member

@vstinnervstinner commentedSep 28, 2017
edited by bedevere-bot
Loading

uuid.UUID doesn't require the 'bytes' argument to be an instance of
bytes: accept bytes-like types, bytearray and memoryview for example.

https://bugs.python.org/issue29729

uuid.UUID doesn't require the 'bytes' argument to be an instance ofbytes: accept bytes-like types, bytearray and memoryview for example.
Copy link
Member

@serhiy-storchakaserhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Add a "versionchanged" directive in the module documentation.

Hmm,memoryview is not accepted asbytes_le. And the length check works correctly only with memoryview with byte format.

* Accept also bytes-like for bytes_le: convert memoryview to bytes to  use slices. Check also the length after the conversion to integer  to raise a TypeError even if the length is not 16.* Fix unit tests: pass bytes to 'bytes' and 'bytes_le' parameters* Add more tests on Unicode passed to bytes and bytes_le* Document UUID changes
@vstinner
Copy link
MemberAuthor

Hmm, memoryview is not accepted as bytes_le

Oh, I didn't look at bytes_le. I also modified bytes_le to accept bytes-like objects.

And the length check works correctly only with memoryview with byte format.

I took care of such issues in my updated PR.

Lib/uuid.py Outdated
raiseValueError('badly formed hexadecimal UUID string')
int=int_(hex,16)
ifbytes_leisnotNone:
ifnotisinstance(bytes_le, (bytes_,bytearray)):

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Addint.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't understand why bytes_le should accept an int? Do you have an example please?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It should not! But accepts. Add a test forbytes_le=16. 😉

@@ -0,0 +1,3 @@
uuid.UUID now accepts bytes-like object. The constructor doesn't require the
'bytes' argument to be an instance of bytes: accept bytes-like types,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

... and 'bytes_le'

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Oops, fixed.

@serhiy-storchaka
Copy link
Member

A ValueError is raised if 'bytes_le' is a str of len != 16. To solve this duplicateint.from_bytes() forbytes_le instead of reusing the code forbytes.

@vstinner
Copy link
MemberAuthor

A ValueError is raised if 'bytes_le' is a str of len != 16.

I added an unit tests to ensure that aTypeError is raised if you pass a str of any length to bytes or bytes_le.

@vstinner
Copy link
MemberAuthor

Please don't merge my PR until PR#3796 is merged. IMHO PR#3796 is more important :-)

@serhiy-storchaka
Copy link
Member

I added an unit tests to ensure that aTypeError is raised if you pass a str of any length to bytes or bytes_le.

Ah, yes, a bytes constructor raises an error.

But still there is a problem with memoryview with non-byte format.bytes_le=memoryview(b'\0'*16).cast('I') works, butbytes=memoryview(b'\0'*16).cast('I') is rejected andbytes=memoryview(b'\0'*64).cast('I') is accepted by mistake.

@vstinner
Copy link
MemberAuthor

PR#3796 has been merged. I merged master into my branch.

But still there is a problem with memoryview with non-byte format. memoryview(b'\0'*64).cast('I') is accepted by mistake.

I don't think that we should overengineer the code here. We are consenting adults here. You can also find a value to skip input validations. It's really hard to write the perfect validation. If you can a random memoryview, yeah, maybe we get a valid UUID object whereas you shouldn't. Well, I don't think that we should detect this corner case. Why would you do that on purpose?

Copy link
Member

@vadmiumvadmium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It may not be worth supporting every kind of invalid input (unless you have to handle untrusted input). But it is important to implement what the documentation promises (or fix the documentation). As of reva169e89,bytes has to also supportlen.

# Don't cast int to bytes to get a TypeError above
ifnotisinstance(bytes_le, (bytes_,bytearray,int_)):
bytes_le=bytes_(bytes_le)
iflen(bytes_le)!=16:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Byabove in the comment, do you mean this check, which is actually below the comment? :)

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I removed the comment.

@vstinner
Copy link
MemberAuthor

@vadmium: "It may not be worth supporting every kind of invalid input (unless you have to handle untrusted input). But it is important to implement what the documentation promises (or fix the documentation). As of reva169e89, bytes has to also support len."

I'm sorry, but I don't understand what you are expecting from me. What do you mean by "bytes has to also support len"? Do you expect a nicer error message if len() fails?

@vstinner
Copy link
MemberAuthor

@serhiy-storchaka,@vadmium: Would you mind to review the latest version of my change, please?

@vadmium
Copy link
Member

For your code to work, the object passed as thebytes argument has to implement__len__ returning 16. But your documentation promises any bytes-like object should work (as long as it is a string of 16 bytes). Since the glossary definition of bytes-like objects has no requirements about implementing__len__, I would expectlen to not be used on the argument. You could change the code forbytes to make a copy like you do forbytes_le:

bytes=bytes_(bytes)# Convert bytes-like to bytes objectiflen(bytes)!=16 . . .

Or you could usememoryview:

withmemoryview(bytes)asview:ifview.nbytes!=16 . . .

Or you could change the documentation:

bytes_le now accepts any bytes-like object, andbytes now accepts any bytes-like object for whichlen returns 16.

@vstinner
Copy link
MemberAuthor

Or you could change the documentation:

UUID constructor documentation is explicit:

   Create a UUID from either a string of 32 hexadecimal digits, a string of 16   bytes as the *bytes* argument, a string of 16 bytes in little-endian order as   the *bytes_le* argument, ...

I guess that your remarks was on my "versionchanged" markup and the NEWS entry. I added "of 16 bytes" there.

int=int_.from_bytes(bytes,byteorder='big')
# test length after the conversion to raise a TypeError exception
# if 'bytes' type is str even if 'bytes' length is not 16
iflen(bytes)!=16:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

>>> a = array.array('i', [-1]*4)>>> x = int.from_bytes(a, byteorder='big')>>> len(a)4>>> x.bit_length()/816.0

And conversely:

>>> a = array.array('i', [-1]*16)>>> x = int.from_bytes(a, byteorder='big')>>> len(a)16>>> x.bit_length()/864.0

@vadmium
Copy link
Member

Yes I was talking about theversionchanged note, which now says "any bytes-like objects of 16 bytes". But I don't think this is clear enough. Antoine and Serhiy already gave examples of bytes-like objects containing exactly 16 bytes wherelen returns a different number.

Since you already copybytes_le into a true bytes object, why not parallel this for thebytes parameter?

ifbytesisnotNone:# Check for int so that bytes=16 triggers an exception belowifnotisinstance(bytes, (bytes_,bytearray,int_)):bytes=bytes_(bytes)# Copy any bytes-like objectiflen(bytes)!=16:raiseValueError('bytes is not a 16-byte string')int=int_.from_bytes(bytes,'big')

@vstinner
Copy link
MemberAuthor

I decided to work onhttps://bugs.python.org/issue29729 because I understand that the proposed change was just to remove an assertion. That's all.

It seems like you expect much more changes, much more complex code. It's already my 6th version of my pull request.

Honestly, I don't care much of the uuid module. I don't think that anyone ever tried to call UUID(bytes=value) with a type different than bytes (ok, apart the bug reporter ;-)). If I would really need to pass non-bytes, having to cast manually with bytes(value) wouldn't burn my hands.

For all these reasons, I abandon my change. Feel free to reuse my code if someone wants to pursue the effort on UUID enhancement ;-)

@vstinnervstinner deleted the uuid_bytes branchOctober 5, 2017 11:56
@serhiy-storchaka
Copy link
Member

The problem with adding support of bytes-like objects is that in different cases this term has different meaning. The definition in the glossary is too wide, in many particular cases we need the support of not just the buffer protocol, but len(), indexing, slicing, iterating, startswith() or even lower().

I think we need different terms for different levels of "bytesness".

@vstinner
Copy link
MemberAuthor

I think that we should replace the assertion with a regular if and raise a proper TypeError, rather than trying to support "bytes-like" objects. It's fine to require bytes on UUID. UUID is not a common type, and passing bytes to UUID is even less common (IMHO).

serhiy-storchaka reacted with thumbs up emoji

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@vadmiumvadmiumvadmium left review comments

@pitroupitroupitrou left review comments

@serhiy-storchakaserhiy-storchakaserhiy-storchaka left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@vstinner@serhiy-storchaka@vadmium@pitrou@Mariatta@the-knights-who-say-ni@bedevere-bot

[8]ページ先頭

©2009-2025 Movatter.jp