Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
TST: Calculate RMS and diff image in C++#29102
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation
So I no longer have any memory-based skips on the PR adding WASM, but maybe we still want to do this to save memory in general? |
This seems to make sense! Should we also use this in |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
QuLogic commentedJun 18, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Put together a small benchmark: importtimeitimporttracemallocimportnumpyasnpfromPILimportImageforNin [100,555,1000,2000]:N=int(N)fornamein ['expected','actual']:image= (np.random.random((N,N,3))*255).astype(np.uint8)Image.fromarray(image).save(f'{name}{N}.png')delimagetracemalloc.start()timer=timeit.Timer(f'compare_images("expected{N}.png", "actual{N}.png", 0)',setup='from matplotlib.testing.compare import compare_images')runtime=timer.autorange()print(N,runtime[1]/runtime[0],tracemalloc.get_traced_memory()[1])tracemalloc.stop() which prints runtime and peak memory up to 2000x2000. I think our largest test image is probably around 1800x900, and 555x555 is approximately equal to 640x480 pixels (the default figure size.) Time-wise, this probably doesn't work out to a lot, maybe about 80% of before on the default figure size, and maybe 55-60% at the larger end: But memory-wise, we're at 10% at the default figure size and even less for larger figures: |
Uh oh!
There was an error while loading.Please reload this page.
The current implementation is not slow, but uses a lot of memory perimage.In `compare_images`, we have:- one actual and one expected image as uint8 (2×image)- both converted to int16 (though original is thrown away) (4×)which adds up to 4× the image allocated in this function.Then it calls `calculate_rms`, which has:- a difference between them as int16 (2×)- the difference cast to 64-bit float (8×)- the square of the difference as 64-bit float (though possibly the original difference was thrown away) (8×)which at its peak has 16× the image allocated in parallel.If the RMS is over the desired tolerance, then `save_diff_image` iscalled, which:- loads the actual and expected images _again_ as uint8 (2× image)- converts both to 64-bit float (throwing away the original) (16×)- calculates the difference (8×)- calculates the absolute value (8×)- multiples that by 10 (in-place, so no allocation)- clips to 0-255 (8×)- casts to uint8 (1×)which at peak uses 32× the image.So at their peak, `compare_images`→`calculate_rms` will have 20× theimage allocated, and then `compare_images`→`save_diff_image` will have36× the image allocated. This is generally not a problem, but onresource-constrained places like WASM, it can sometimes run out ofmemory just in `calculate_rms`.This implementation in C++ always allocates the diff image, even whennot needed, but doesn't have all the temporaries, so it's a maximum of3× the image size (plus a few scalar temporaries).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Not merging b/c you keep pushing, but you're welcome to merge when you're done tweaking. The memory improvements look awesome!
That was just fixing stubtest; it should be good now. |
e325459 intomatplotlib:mainUh oh!
There was an error while loading.Please reload this page.
PR summary
The current implementation is not slow, but uses a lot of memory per image.
In
compare_images, we have:which adds up to 4× the image allocated in this function.
Then it calls
calculate_rms, which has:which at its peak has 16× the image allocated in parallel.
If the RMS is over the desired tolerance, then
save_diff_imageis called, which:which at peak uses 32× the image.
So at their peak,
compare_images→calculate_rmswill have 20× the image allocated, and thencompare_images→save_diff_imagewill have 36× the image allocated. This is generally not a problem, but on resource-constrained places like WASM, it can sometimes run out of memory just incalculate_rms.This implementation in C++ always allocates the diff image, even when not needed, but doesn't have all the temporaries, so it's a maximum of 3× the image size (plus a few scalar temporaries).
PR checklist