Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
Use test cache for test result images too.#15932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
If we have a few tests that are not deterministic, will this cause the cache to increase without bound? I think that we cache these folders on at least travis so we would want to make sure we don't start spending more time moving around invalid images than we save by not computing them. |
anntzer commentedDec 13, 2019 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Fair enough, added a simple (automatic) cache eviction mechanism. |
I don't fully follow this, but itseems useful. But needs a rebase, and probably a review from@tacaswell or@QuLogic |
For image comparison tests in svg/pdf/ps formats, the result images areconverted to png for comparison. Previously the conversion results werecached for the baseline images, but not for the test-generated images(because of non-deterministic svg/pdf/etc. results, due tohash-salting, dict ordering, etc.).Now that the test-generated images are generally deterministic, we canenable the cache for baseline images as well. This speeds up`pytest -k '[svg]'` by ~30% (81s initially -> 55s on a seeded cache) and`pytest -k '[pdf]'` by ~10% (62s -> 55s) (there are too few (e)ps imagecomparison tests to see an effect). Also add logging regarding thecache which may help troubleshooting determinacy problems.A simple cache eviction mechanism prevents the cache from growingwithout bounds, limiting it to 2x the size of the baseline_imagesdirectory.This is a much simpler version of PR7764, which added more sophisticatedreporting of cache hits and misses and cache eviction.
rebased |
Uh oh!
There was an error while loading.Please reload this page.
For image comparison tests in svg/pdf/ps formats, the result images are
converted to png for comparison. Previously the conversion results were
cached for the baseline images, but not for the test-generated images
(because of non-deterministic svg/pdf/etc. results, due to
hash-salting, dict ordering, etc.).
Now that the test-generated images are generally deterministic, we can
enable the cache for baseline images as well. This speeds up
pytest -k '[svg]'
by ~30% (81s initially -> 55s on a seeded cache) andpytest -k '[pdf]'
by ~10% (62s -> 55s) (there are too few (e)ps imagecomparison tests to see an effect). Also add logging regarding the
cache which may help troubleshooting determinacy problems.
This is a much simpler version of PR#7764, which added more sophisticatedreporting of cache hits and misses, as well as cache invalidation (I
think the cache can reasonably be cleaned manually, at least for now).
A simple cache eviction mechanism prevents the cache from growing
without bounds, limiting it to 2x the size of the baseline_images
directory.
This is a much simpler version of PR7764, which added more sophisticated
reporting of cache hits and misses and cache eviction.
Supersedes (mostly)#7764 (@jkseppan).
(Note that the gain in speed is less than reported than in#7764 (comment), because I have also optimized the svg and gs converters in the meantime by running a single process and interacting with it over stdin/stdout, rather that launching a new instance of inkscape/ghostscript for each image.)
PR Summary
PR Checklist