Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
Make test_stem less flaky.#16735
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Since parametrizing the test allows it to run in parallel, this makes itflaky, as one process can overwrite the test result image of another.Our standard way for dealing with tests that use the same baseline imageis to pass duplicate filenames to `image_comparison`, because that isserialized.
Would this be fixed by#16693? |
No, that's only for |
timhoffm commentedMar 11, 2020 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Sorry about the confusion with This approach here unrolls the parametrization into a helper loop. Exchanging the declarative parametrization for to code logic makes it less readable. But what bothers me more is that pytest will now not give the information which of the parameters failed. I Would prefer to solve this in the administrative logic if possible. I quickly searched for disabling parallelization for some tests, but didn’t find anything. Could we set a lock based on the filename in Alternatively/additionally, could |
We could use a lock, Cartopy uses |
I don‘t see a problem backporting a lock to 3.2.1. It‘s an internal bug fix. But if you want to bring this explicit fix back I’m also fine with that. |
It's not just a lock, but also a new dependency. And also 3.2.1 is scheduled for Friday, and that will take some time to test, I think. |
…735-on-v3.2.xBackport PR#16735 on branch v3.2.x (Make test_stem less flaky.)
PR Summary
Since parametrizing the test allows it to run in parallel, this makes it flaky, as one process can overwrite the test result image of another.
Our standard way for dealing with tests that use the same baseline image is to pass duplicate filenames to
image_comparison
, because that is serialized.This is basically the same as#16656 for
test_stem
.PR Checklist