Movatterモバイル変換
[0]ホーム
[Python-Dev] [numpy wishlist] Interpreter support for temporaryelision in third-party classes
Sturla Moldensturla.molden at gmail.com
Fri Jun 6 18:06:18 CEST 2014
Julian Taylor <jtaylor.debian at googlemail.com> wrote: > The problem with this approach is that it is already difficult enough to> handle memory in numpy.I would not do this in a way that complicates memory management in NumPy. Iwould just replace malloc and free with temporarily cached versions. Fromthe perspective of NumPy the API should be the same.> Having a cache that potentially stores gigabytes> of memory out of the users sight will just make things worse.Buffer don't need to stay in cache forver, just long enough to allow resuewithin an expression. We are probably talking about delaying the call tofree with just a few microseconds.We could e.g. have a setup like this:NumPy thread on "malloc":- tries to grab memory off the internal heap- calls system malloc on failureNumPy thread on "free":- returns a buffer to the internal heap- signals a conditionBackground daemonic GC thread:- wakes after sleeping on the condition- sleeps for another N microseconds (N = magic number)- flushes or shrinks the internal heap with system free- goes back to sleeping on the condition It can be implemented with the same API as malloc and free, and pluggeddirectly into the existing NumPy code. We would in total need two mutexes, one condition variable, a pthread, anda heap.Sturla
More information about the Python-Devmailing list
[8]ページ先頭