Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork7.9k
COMPAT: use tkagg backend on PyPy#9356
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
anntzer commentedOct 11, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Looks like the only problematic use of id() is to get the underlying address of the ndarray object (the PyObject, not the memory buffer), right? I would suggest instead just passing a memoryview object, which contains all the relevant information even in Py2 (https://docs.python.org/2/library/stdtypes.html#memoryview), and adapting the extension code to accept such an input (it's all private anyways). Actually now that I look at it it's not clear whether tk.call can pass memoryviews, but you can also just pass in the address of the underlying buffer and the size and strides of the array (and fix the extension code accordingly). The address of the underlying buffer is accessible at If possible, I'd prefer avoid too much pypy specific code, because I fear it's quite likely to bitrot... |
@anntzer thanks for picking this up. I tried to modify the id() calls as you said, but have not been able to get |
mattip commentedNov 5, 2017 • edited by dopplershift
Loading Uh oh!
There was an error while loading.Please reload this page.
edited by dopplershift
Uh oh!
There was an error while loading.Please reload this page.
I actually needed to change very little to get this working, in the end I:
On PyPy unfortunately there seems to be an issue with opening the "configure subplot" subfigure on a basic plot, the mouse events are not reliably reaching the event handlers in |
_tkagg.tkinit(tk.interpaddr(), 1) | ||
else: | ||
# very old python? | ||
_tkagg.tkinit(tk, 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
interpaddr was added in 1998(!python/cpython@9d1b7ae) so I cannot actually imagine a case where this is triggered today (but that can go to another PR of course)
@tacaswell I got a mail reviewing the changes to |
How do I fix the codecov/patch check failure? |
Don't worry about the codecov check there. The Tk GUI code, unfortunately, has no automated tests for CI to run. |
We tend to ignore that check to be honest... |
ping |
src/_tkagg.cpp Outdated
#endif | ||
# define BBOX_FORMAT "%f %f %f %f" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
extra space?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
fixed incc7b73e
src/_tkagg.cpp Outdated
deststride = 4 * destwidth; | ||
destbuffer = newagg::int8u[deststride * destheight]; | ||
destbuffer = newuint8_t[deststride * destheight]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
any reason for the change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
now it no longer uses#include py_converters.h
which supplied theagg:int8u
src/_tkagg.cpp Outdated
return TCL_ERROR; | ||
} | ||
if (sscanf(argv[4], BBOX_FORMAT, &x1, &x2, &y1, &y2) == 4) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
should probably error out cleanly if an invalid string is passed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
yesish, these are basically "private" functions parsing a call from python, but I will add more checking code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
updated incc7b73e
src/_tkagg.cpp Outdated
has_bbox = true; | ||
} | ||
else { | ||
elseif (bbox_parse == 1 && x1 == 0){ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
parens
Can you rebase on master (to try to make appveyor pass... perhaps)? |
src/_tkagg.cpp Outdated
desty = hdata - y2; | ||
destwidth = x2 - x1; | ||
destheight = y2 - y1; | ||
destx = (int)x1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Shouldn't BBOX_FORMAT just be 4 integers and just use integers all along? (not sure if tk would have the bad idea to always cast things to float)
Edit: Actually I guess not, what should really happen is that we should explicitly cast on the caller's side but this can just go as it is for now...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The call sequence isbackend.blit(image, bbox)
->tkagg.py:blit(image, bbox)
->PyAggImagePhoto(as_string(image, bbox))
so we cannot change the rootbackend.blit
call. AFAICT the choice is:
- convert early in
tkagg.py:blit
- convert later in
PyAggImagePhoto
I usually prefer converting as late as possible to postpone rounding and allow future refactoring with floats to be as painless as possible, but will flow with whatever is matplotlib best practices
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think we can leave it as it is for now, we were using floats to start with and we can change that later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
assuming appveyor passes
Tested locally on cpython, not actually tested myself on pypy...
thanks, I tested on pypy HEAD and tkagg now works. However the way matplotlib uses |
just to be clear, the weakref issue is not a matplotlib issue, rather a PyPy issue that matplotlib exposes |
after#9899 was merged, this is all that is needed for the next release of PyPy to work with the TkAgg backend |
embedding_in_tk2_sgskip is essentially a slightly simplified duplicateof embedding_in_tk_sgskip.
src/_tkagg.cpp Outdated
deststride = 4 * destwidth; | ||
destbuffer = newagg::int8u[deststride * destheight]; | ||
destbuffer = newuint8_t[deststride * destheight]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why change away from the agg types here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Previously there was a conversion from aPyArrayObject *
(in other words ndarray) to a data pointer, width, height, etc via py_converters. Now I pass that information directly from python. I no longer need the "py_converters.cpp" and ""py_converters.h" since there is no use of ndarray in the c++ code. Should I leave the include for this one type definition?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I am concerned about the corner case whereunint8_t
andagg:int8u
are different underlying types. Not sure how likely that actually is, hence leaning towards being conservative about changing it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I wonder what different types you think these can be.
I would also bet that there must be parts of the code where we assume that this is interconvertible with np.uint8, or that this fits in a single byte... doesn't leave you a lot of options!
I never mentioned that what I thought was a PyPy weakref or JIT issue was actually solved by PR#9899 so this PR is all that is needed for TkAgg and PyPy |
what's up with docs build? It seems to have timed out with no connection to my pull request @tacaswell converted back to |
mattip commentedDec 19, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
what's up with docs build? It seems to have timed out with no connection to my pull request @tacaswell converted back to edit: docs build seems to have passed |
Sometimes the builds need to be restarted manually because the host has a problem... |
can I get this marked 2.2-critical? |
"Making PyPy work" sounds like a good thing to have... |
By the way would be nice to add a whatsnew entry (including info about what versions of PyPy are supposed to work). |
Not sure270f05d is the proper way to do it, let me know if I should have done it differently |
Perfect@mattip |
Thanks@mattip ! |
Uh oh!
There was an error while loading.Please reload this page.
PR Summary
the
tkagg::blit
function uses theid
built-in to obtain memory adresses. This is not compatible with PyPy. I have written a proof-of-concept alternativeblit
function that uses a pure-python_tkagg
backend via CFFI. Since CPython prefers a c-extension (_tkagg.so) over a python file (_tkagg.py) and PyPy reverses this, the two implementations can live side by side.It is working but not yet complete:
on the matplotlib side I should probably only conditionally build the _tkagg c-extension on CPythonon the PyPy side the serialization/deserialization of a numpy ndarray should not make a copy of the dataupdating only a bbox of data not supported yetdocumentationbut I wanted to see if this approach is acceptable before continuing. Any thoughts?
Edit: now point three (bbox handling) is complete
Edit: (5.Nov.17) now points one and two are done, and documentation does not change