Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
Description
tl;dr: When I repeatedly create a large figure, save it, and close it, memory usage keeps growing.
Over atthis discussion about when MPL should trigger garbage collection,@efiring had some lingering doubts about the chosen solution:
It would certainly be good to have a clearer understanding of when, if ever in practice, it would lead to troublesome increases in memory consumption
I ran into such a case today, when my batch job filled up 60G of RAM over night.
I repeatedly create a large figure, save it, then close it. If I don't manually callgc.collect()
after closing each figure, memory consumption saturates at around 10x of what an individual figure needs. In my case, with several fairly complex figures, this was enough to fill a big machine.
Since this is not obvious from the docs, I think there should be an official way to go back to more aggressive GC for cases like this where the trade-off discussed at#3045 fails. Maybeclose(force_gc=True)
?
Code for reproduction
frommemory_profilerimportprofile# https://pypi.python.org/pypi/memory_profilerfrommemory_profilerimportmemory_usageimportmatplotlib.pyplotaspltimportnumpyasnpimportgcN=80@profiledefdo_plots():fig=plt.figure()plt.plot(np.random.rand(50000))plt.savefig('/tmp/bla.png')plt.close(fig)defdefault():forkinrange(N):print(k)do_plots()defmanual_gc():forkinrange(N):print(k)do_plots()gc.collect()mem_manual_gc=memory_usage((manual_gc, [], {}))mem_default=memory_usage((default, [], {}))plt.plot(mem_manual_gc,label='gc.collect() after close')plt.plot(mem_default,label='default behaviour')plt.ylabel('MB')plt.xlabel('time (in s * 0.1)')# `memory_usage` logs every 100msplt.legend()plt.title('memory usage')plt.show()
Matplotlib version
- Operating System: Ubuntu 16.10
- Matplotlib Version: 2.0.0
- Python Version: 3.5.2