NotificationsYou must be signed in to change notification settings
Fork33.7k
Star70.4k

gh-108963: using random to generate unique string in sys.intern test#109491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Merged

vstinner merged 6 commits intopython:mainfromaisk:fix-gh108963

Oct 2, 2023

Merged

gh-108963: using random to generate unique string in sys.intern test#109491

vstinner merged 6 commits intopython:mainfromaisk:fix-gh108963

Oct 2, 2023

Conversation

Copy link

Contributor

aisk commentedSep 16, 2023•
edited by bedevere-appbot
Loading

Issue:test_sys.test_intern() fails if run more than once in the same process #108963

using random to generate unique string in sys.intern test

bdcae08

bedevere-appbot added awaiting review testsTests in the Lib/test dir labels

Sep 16, 2023

aisk changed the title~~using random to generate unique string in sys.intern test~~gh-108963: using random to generate unique string in sys.intern test

Sep 16, 2023

bedevere-appbot mentioned this pull request

Sep 16, 2023

test_sys.test_intern() fails if run more than once in the same process#108963

Closed

serhiy-storchaka requested a review fromvstinner

September 17, 2023 08:54

serhiy-storchaka approved these changes

Sep 17, 2023

View reviewed changes

bedevere-appbot added awaiting core review and removed awaiting review labels

Sep 17, 2023

Copy link

Member

vstinner commentedSep 18, 2023

This change hides the issue that the test actually leaks memory: intern a string which is then only released at Python exit. Can we add a private deintern() function in _testcapi? Or run the test in a subprocess?

Well, i don't know that it's a big deal, this change is ooookay-ish.

vstinner reviewed

Sep 18, 2023

View reviewed changes

Lib/test/test_sys.py Outdated

		INTERN_NUMRUNS+=1
		self.assertRaises(TypeError,sys.intern)
		s="never interned before"+str(INTERN_NUMRUNS)
		s="never interned before"+str(random.random())

Copy link

Member

vstinnerSep 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I would prefer using random.randint(), for example 10**9 to have 9 random digits. I prefer to avoid float when possible 😁 Same below.

Copy link

ContributorAuthor

aiskSep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Done

Copy link

Member

serhiy-storchakaSep 19, 2023•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

But this increases the probability of random failure from 1/253 (about 1/1016) to 1/(9*10**8). If run the test every second, there will be a failure every 30 years in average.

Copy link

ContributorAuthor

aiskSep 19, 2023•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

On my machine, therandom.random() returns something like0.35934878814573135, which have 18 digits (with the first zero). So can we just increase the max range to10 ** 18 to keep the random resolution almost like as therandom.random()?

aisk added2 commits

September 19, 2023 19:34

remove float in intern test

1653d05

fix test cases

a586b39

serhiy-storchaka reviewed

Sep 19, 2023

View reviewed changes

Lib/test/test_sys.py Outdated

		deftest_intern(self):
		self.assertRaises(TypeError,sys.intern)
		s="never interned before"+str(random.int(108,109))
		s="never interned before"+str(random.randint(108,109))

Copy link

Member

serhiy-storchakaSep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If the intention is to generate random 9-digit numbers, userandrange() instead ofrandint().

vstinner reviewed

Sep 19, 2023

View reviewed changes

Lib/test/test_sys.py Outdated

		deftest_intern(self):
		self.assertRaises(TypeError,sys.intern)
		s="never interned before"+str(random.random())
		s="never interned before"+str(random.int(108,109))

Copy link

Member

vstinnerSep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

random.int doesn't exist. What's the point of putting a minimum at 10**8? Why not 0?

Copy link

ContributorAuthor

aiskSep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

There is a more recent commit in the PR fixed the bug. The minimal start is for ensure the random parts have 9 digits, I think it's more consistent.

Copy link

Member

vstinnerSep 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

i don't think that the string length (number of random int digits) matters here, does it?

using random.randrange instead of random.randint and increase range

4978df9

vstinner reviewed

Sep 21, 2023

View reviewed changes

Lib/test/test_sys.py Outdated

		INTERN_NUMRUNS+=1
		self.assertRaises(TypeError,sys.intern)
		s="never interned before"+str(INTERN_NUMRUNS)
		s="never interned before"+str(random.randrange(0,10**18))

Copy link

Member

vstinnerSep 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Oh, for me 9 digits was already way enough. 18 digits is just too much 😁 You can just pass the max value:

Suggested change

	s="never interned before"+str(random.randrange(0,10**18))
	s="never interned before"+str(random.randrange(10**9))

using 9 digits random range in intern test

785393d

vstinner approved these changes

Sep 21, 2023

View reviewed changes

Copy link

Member

vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM.

It makes me sad to leak memory. But I'm not sure about a private API to de-intern a string. I'm not sure of the consequences.

Merge branch 'main' into fix-gh108963

14008d9

vstinner added skip news needs backport to 3.12only security fixes labels

Oct 2, 2023

vstinner merged commit44b1e4e intopython:main

Oct 2, 2023

Copy link

Contributor

miss-islington commentedOct 2, 2023

Thanks@aisk for the PR, and@vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

bedevere-appbot removed the awaiting core review label

Oct 2, 2023

Copy link

Contributor

miss-islington commentedOct 2, 2023

Sorry,@aisk and@vstinner, I could not cleanly backport this to3.12 due to a conflict.
Please backport usingcherry_picker on command line.

cherry_picker 44b1e4ea4842c6cdc1bedba7aaeb93f236b3ec08 3.12

miss-islington assignedvstinner

Oct 2, 2023

vstinner pushed a commit to vstinner/cpython that referenced this pull request

Oct 2, 2023

pythongh-108963: using random to generate unique string in sys.intern…

52c001d

… test (python#109491)(cherry picked from commit44b1e4e)

vstinner mentioned this pull request

Oct 2, 2023

[3.12] gh-108963: using random to generate unique string in sys.intern test …#110216

Merged

Copy link

bedevere-appbot commentedOct 2, 2023

GH-110216 is a backport of this pull request to the3.12 branch.

bedevere-appbot removed the needs backport to 3.12only security fixes label

Oct 2, 2023

Yhg1s pushed a commit that referenced this pull request

Oct 2, 2023

[3.12]gh-108963: using random to generate unique string in sys.inter…

f911d7e

…n test … (#110216)gh-108963: using random to generate unique string in sys.intern test (#109491)(cherry picked from commit44b1e4e)Co-authored-by: AN Long <aisk@users.noreply.github.com>

aisk deleted the fix-gh108963 branch

October 3, 2023 07:16

Glyphack pushed a commit to Glyphack/cpython that referenced this pull request

Sep 2, 2024

pythongh-108963: using random to generate unique string in sys.intern…

34f3941

… test (python#109491)

Labels

skip news tests

Tests in the Lib/test dir

Movatterモバイル変換

Uh oh!

gh-108963: using random to generate unique string in sys.intern test#109491

gh-108963: using random to generate unique string in sys.intern test#109491

Uh oh!

Conversation

aisk commentedSep 16, 2023• edited by bedevere-appbotLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Uh oh!

vstinner commentedSep 18, 2023

Uh oh!

vstinnerSep 18, 2023

Choose a reason for hiding this comment

Uh oh!

aiskSep 19, 2023

Choose a reason for hiding this comment

Uh oh!

serhiy-storchakaSep 19, 2023• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aiskSep 19, 2023• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

serhiy-storchakaSep 19, 2023

Choose a reason for hiding this comment

Uh oh!

vstinnerSep 19, 2023

Choose a reason for hiding this comment

Uh oh!

aiskSep 19, 2023

Choose a reason for hiding this comment

Uh oh!

vstinnerSep 19, 2023

Choose a reason for hiding this comment

Uh oh!

vstinnerSep 21, 2023

Choose a reason for hiding this comment

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington commentedOct 2, 2023

Uh oh!

miss-islington commentedOct 2, 2023

Uh oh!

bedevere-appbot commentedOct 2, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

aisk commentedSep 16, 2023•
edited by bedevere-appbot
Loading

serhiy-storchakaSep 19, 2023•
edited
Loading

aiskSep 19, 2023•
edited
Loading