Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
Fix flaky text tests#8708
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Fix flaky text tests#8708
Uh oh!
There was an error while loading.Please reload this page.
Conversation
2.5.5 is what's in my conda env, and 2.6.1 is what's built with thelocal_freetype option and is used on Travis and AppVeyor without issue.
The result of id() is only guaranteed to be unique for objects thatexist at the same time. This global integer is a quick and light way toensure that new renderers that match an old one don't produce the samecaching key.
The Mathtext tests should no longer be flaky based on the previouschange.Mark some PS tests as flaky because they require a lock that sometimesgets stuck on AppVeyor due to the multiple processes.
Huh, just realized that@tacaswell outlined basically this procedure in#7911 but I didn't recall it; must have been stuck in my subconscious somewhere. |
tacaswell commentedJun 3, 2017 • edited by QuLogic
Loading Uh oh!
There was an error while loading.Please reload this page.
edited by QuLogic
Uh oh!
There was an error while loading.Please reload this page.
I don't think this is thread safe, and that is ok. I do not think we make the claim that Matplotlib is threadsafe in any way. Further most of our time is spent in python and I do not think we release the GIL in the c extensions so it is not clear to me you would get any major gains (unless you really are I/O bound). Although the way that this will fail to be thread safe also means both objects existing at the same time in which case the There are two places to get race conditions here (see https://www.youtube.com/watch?v=7SSYhuk5hmc for lots more details) If you look x=5classTest:deffoo():globalxx+=1self.cache=x and In [91]:t=Test()In [93]:dis.dis(t.foo)80LOAD_GLOBAL0 (x)2LOAD_CONST1 (1)4INPLACE_ADD6STORE_GLOBAL0 (x)98LOAD_GLOBAL0 (x)10LOAD_GLOBAL1 (self)12STORE_ATTR2 (cache)14LOAD_CONST0 (None)16RETURN_VALUE I think you can get races conditions between all of the op-codes in line 8 which missing an increment and between lines 8 and 9 (both increment, but then both use the higher value). |
Should this just go into |
We probably do not want to go down the |
QuLogic commentedJun 3, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
For thread-safety, I was thinking more of subsequent renderers. Across the threads, the renderers would have different Theoretically, thread 1 could create a couple renderers in the time it takes thread 2 to store the wrong value. Then the next time thread 1 created a new |
Actually you can just use itertools.count as a threadsafe counter (create _counter at the module level and call |
That's a good idea; even if not thread-safe on all implementation, it's at least clearer. |
lib/matplotlib/backend_bases.py Outdated
part of a caching key. | ||
""" | ||
if self._id is None: | ||
self._id = next(_unique_renderer_id) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I would just make this a regular attribute created at instantiation (and leave a comment in the ctor). I'm not sure what making this a property buys you...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I meant to remove this now when switching to itertools but forgot about it.
As suggested by@anntzer.
@tomspur This patch should help withhttps://bugzilla.redhat.com/show_bug.cgi?id=1401267 (though leave out the pytest parts if on 1.5.3.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Approved before and it is much better now.
thanks! |
@QuLogic - great - thanks for doing that. |
As noted in#7911, the mathtext tests sometimes fail if the
id
of the renderer is the same between tests. Since it's difficult to add the mathtext fontset to the key, I fixed/hacked it into working by adding a cache-busting "unique" value to each renderer, which is just a globally-incremented number. I thought of using a UUID or random number, but that seemed too expensive since we don't need to worry about threading locks on globals (AFAIK).I ran the test from#7911, but using the repeat plugin to run everything 100 times over (so 50000 tests total.) So I'm not 100% certain, but I believe thisfixes#7911.
Additionally, we've lately been seeing several failures of
test_savefig_to_stringio
on AppVeyor. This doesn't usually fail locally, but seems to be a timeout on the TeX cache. I've added some retries on these tests to hopefully reduce the problem.PR Checklist