Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

ENH: Adding callbacks to Norms for update signals#19553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
tacaswell merged 1 commit intomatplotlib:masterfromgreglucas:norm-updater
Sep 1, 2021

Conversation

greglucas
Copy link
Contributor

@greglucasgreglucas commentedFeb 21, 2021
edited by QuLogic
Loading

PR Summary

This adds a callback registry to Norm instances that can be connected to by other objects to be notified when the Norm is updated. This is particularly relevant for ScalarMappables to be notified when the vmin/vmax are changed on the Norm. With this update, the axis is now immediately registered as stale in the following example.

importmatplotlib.pyplotaspltfig,ax=plt.subplots()norm=plt.Normalize(-2,2)ax.imshow([[0,1], [2,-1]],norm=norm)norm.vmax=0.1print(ax.stale)

Closes#4387
Fixes#17052

PR Checklist

  • Has pytest style unit tests (andpytest passes).
  • IsFlake 8 compliant (runflake8 on changed files to check).
  • [N/A] New features are documented, with examples if plot related.
  • Documentation is sphinx and numpydoc compliant (the docs shouldbuild without error).
  • Conforms to Matplotlib style conventions (installflake8-docstrings and runflake8 --docstring-convention=all).
  • New features have an entry indoc/users/next_whats_new/ (follow instructions in README.rst there).
  • [N/A] API changes documented indoc/api/next_api_changes/ (follow instructions in README.rst there).

jklymak
jklymak previously requested changesMar 18, 2021
Copy link
Member

@jklymakjklymak left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Sorry, I'll block on this feature, and suggest#4387 be closed.

In general I am against magic callbacks like this. I think an artist should reflect the state of the artist at creation time, or when it is explicitly acted upon, not at draw time, and possibly based on actions applied to other artists. If you need to keep two artists limits in sync, you should explicitly do so, and not rely on a black box callback.

I appreciate there was a push to make the library cache more things until draw time, and maybe if the whole library had been designed with that goal from the start that would make sense. But as-is, implicit state is simply confusing and leads to unexpected behaviour.

Of course happy to unblock if there is a real strong case for why this is needed, and can't be done at the user level.

@greglucas
Copy link
ContributorAuthor

I think I agree with you...? This isn't caching until draw-time, it is just creating an update mechanism that will automatically trigger the re-drawing chain (norm -> scalar-mappable -> artist). Someone still has to explicitly modify the norm.

This was motivated by me wanting to have interactivity with the colorbars and panning/zooming on that axis, see here:#19515

Basically, I was trying to bring things a little more in-sync throughout the pipeline of the scalar-mappable updates to not force a user to callfig.draw() to update the image. See here for other scalar-mappable cleanup work as well:
#19552

@jklymak
Copy link
Member

This was motivated by me wanting to have interactivity with the colorbars and panning/zooming on that axis

I see. But that feature should also be discussed I guess? Are you suggesting if I zoom on a colorbar, that should change the mappable somehow? How? And why is that a useful thing to do? I guess I could imagine the converse, where you set the limits of the colorbar to change vmin and vmax, but that would require a text box or something, rather than zooming.

But here, you are pushing it further, and not only changing the mappable that belongs to the colorbar, but changing the norm and wanting the changes to propagate to other mappables that may have the "same" norm.

I guess I'm arguing we should be doing exactly the opposite, and making the norms a copy when added to the mappable, and hence a change to one mappable doesnot affect the others. But maybe I haven't though that through and many folks are making a single norm, and sharing it, and then expecting to manipulate the norm and have it show up on the next draw. I guess the PR would benefit from a bit more context of what the current state is, and why this is an improvement?

@dopplershift
Copy link
Contributor

@jklymak The current state is that this:

fig,ax=plt.subplots()norm=plt.Normalize(-2,2)cmap=copy.copy(plt.get_cmap('BuGn'))ax.imshow(data,norm=norm,cmap=cmap)norm.vmax=0.1norm.vmin=-0.1cmap.set_over('red')

plots with a norm from 0.1 to -0.1, and a colormap with an "over" color of red.

I'm not a fan of having the API make a copy of an object. I don't know of anywhere else in the API that we do that (e.g.add_artist).

@jklymak
Copy link
Member

OK, sure, but without running it, are you sure what this does?

importmatplotlib.pyplotaspltimportnumpyasnpimportcopyfig,axs=plt.subplots(1,2)norm=plt.Normalize(-2,2)data=np.linspace(-1,1,900).reshape(30,30)cmap=copy.copy(plt.get_cmap('BuGn'))axs[0].imshow(data,norm=norm,cmap=cmap)norm.vmax=0.1norm.vmin=-0.1cmap.set_over('red')im=axs[1].imshow(data,norm=norm,cmap=cmap)fig.colorbar(im,ax=axs)im.set_clim(-0.3,0.3)plt.show()

@greglucas
Copy link
ContributorAuthor

I will say what Ithink should happen without running it. Everything is using the same norm, so everything should be in-sync with the final updates that were made before theplt.show() call. So, I'd expect the norm to be (-0.3, 0.3), and the images to show red at any values over 0.3.

If you run this in an interactive session, currently when you modify the normnorm.vmax = 0.1 nothing will happen. However, with the new callback added in this PR, the norms would signal astale figure and trigger a redraw automatically. This could be attached to a text/slider widget as you mentioned above which would be a great way to change the color limits too.

I'll try to add some more context to the original PR summary

@tacaswelltacaswell added this to thev3.5.0 milestoneMar 19, 2021
@jklymak
Copy link
Member

OK, thanks@greglucas and@dopplershift - I thought about this some and agree with both of you. This sort of behaviourcan be a bit mysterious - I wouldn't expect the user to know that the clim on an object is linked up to a global norm instance, but I agree with you that this is how it is, and that such behaviour can have uses....

@jklymakjklymak dismissed theirstale reviewMarch 19, 2021 03:11

my objections were not well-founded

@tacaswell
Copy link
Member

As the author of the linked issue I am (still) in favor of this :)

The problem this is trying to solve is that we do have a number of caches (for performance reasons) that need to be invalidated when we change the norm. If you do it one way (throughset_clim) (which invalidates the cache) and the other is to set the

fig, axs = plt.subplots(1,2)norm = plt.Normalize(-2, 2)data = np.linspace(-1, 1, 900).reshape(30, 30)cmap = copy.copy(plt.get_cmap('BuGn'))axs[0].imshow(data, norm=norm, cmap=cmap)# norm.vmax = 0.1# norm.vmin = -0.1norm = plt.Normalize(-2, 2)axs[1].imshow(data, norm=norm, cmap=cmap)print(f'{norm.vmin}, {norm.vmax}')plt.pause(5)norm.vmax = 0.1norm.vmin = -0.1print(f'{norm.vmin}, {norm.vmax}, {axs[1].images[0].get_clim()}')# at this point the two Axes will look identical despite reporting different clims!plt.show()# if you force a full redraw by e.g. resizing the window they will snap to be different.

This is adding an explicit coupling (via the callback) to get rid of an implicit coupling (with hysteresis and dependencies on exactly where draws are!).

I have not run the code on this PR, but I think it will fix this.

We will need to check thanobj.changed() is relatively cheap or de-bounce it somehow inim.set_clim() (which will not trigger this 3 times: 2 each from vmin/vmax and once from an explicit call toself.changed() inset_clim.

jklymak reacted with thumbs up emoji

@greglucas
Copy link
ContributorAuthor

@jklymak, just to be clear there isno global Norm here. This is a specific norm object and youcan share that object between images to keep them in sync. But, if you putplt.Normalize() on one image andplt.Normalize() on a second image, those will be two separate norms that can be modified independently and this won't sync those.

@tacaswell, I got rid of the 3rdon_changed call in set_clim. There will still be two calls (vmin and vmax) as you mention, but as far as I can tell these seem like cheap updates that updateself.stale and notify other listeners which also appear to be cheap at first glance.

@jklymak
Copy link
Member

@jklymak, just to be clear there isno global Norm here.

No I understand, but there is a top-levelnorm object that the artist methods can manipulate (i.e. viaartist.set_clim) and it is not necessarily clear that would affect a norm buried on the artist and thus potentially affect other artists. i.e. you can easily imagine a user wanting two images to have log-norm scaling, but not necessarily the same vmin and vmax and doing:

norm=plt.LogNorm()im1=ax1.imshow(X1,norm=norm)im2=ax2.inshow(X2,norm=norm)im2.set_clim(1e-3,1e5)

and be surprised that this affectsim1. i.e they weren't thinking of the norm holistically as scale, vmin, vmax,, but just as a way to get logarithmic scaling.

@greglucas
Copy link
ContributorAuthor

That makes sense, but that is the case regardless of this PR (both images are modified on current master), and I think not too bad to explain to someone not to re-use an object like that but rather just create separateLogNorms.

jklymak and tacaswell reacted with thumbs up emoji

@@ -43,6 +44,9 @@ def test_imagegrid_cbar_mode_edge():
# "second" ones. To achieve this, clear out the axes first.
for ax in grid:
ax.cax.cla()
# We don't want to share the norms between the colorbar axes
# so create a deepcopy for each image
ax.images[0].norm = copy.deepcopy(ax.images[0].norm)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Did we used to rely on a race condition in drawing to make this test work?

I'm also not seeing in the above code how they ended up coupled.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This one took me a long time to figure out that this one-line fix is all that is needed. If I remove the deepcopy, the scales on the colorbar axes seem to stick around or not get updated appropriately? There are some extra ticks floating around and some spines don't get fully removed. I think there is some state that isn't fully getting cleared inaxes_grid with the way the colorbars are handled there.

I don't think this is the most ideal test with this comment about clearing and overwriting portions of the axes.

# In each row/column, the "first" colorbars must be overwritten by the# "second" ones.  To achieve this, clear out the axes first.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This test bit me again... I just added some new code to connect/disconnect the callbacks when changing the norm on a ScalarMappable that already exists. Now, I'm able to remove theax.cax.cla() call, but I need a brand new ScalarMappable for only this test. I think there is something with the colorbars trying to modify old state here, which is why creating a new instance that has no sharing is what we need.

I still don't 100% follow this test code and haven't taken the time to really investigate it in depth and determine whether this test could be rewritten, or if something else in colorbars should be updated.

cb = ax.cax.colorbar(
ax.images[0],
sm,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think@anntzer worked on this last, trying to make these colorbar-like objects the same as main-library colorbars.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This still feels like there is something different aboutaxes_grid that I am missing and we shouldn't need to deepcopy the norm on the mappable... My guess is there is possibly a locator or something similar attached toax.cax that needs to be cleared/reset on theax.cax.cla() call above, but I haven't been able to find it while debugging and have spent quite a lot of time on this.

Is there a way to step through a debugger session while keeping the plot window open to see how things move around while stepping into functions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Not that I know of, but I never use a debugger (probably a bad idea).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I'm still deeply confused by this test. It seems that it intentionally over-rides the axes so I guess the point is to show that 'cax' for both of the axes in a given row / column?

imagegrid_cbar_mode

I am not sure what the expected behavior here actual is to sort out how big of a change this is...

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I am very confused as well. Ithink what is happening is that on the first pass of colorbar addition to the cax, some locators/formatters get updated and set due to the new callback. Then on the second addition of the colorbar, that state is still hanging around. Rather than deepcopying, I switched the test to remove the callback from the colorbar.

The difference with/without the final commit is that the second colorbar axis has ticklabels showing up. I'm not exactly sure where this state creeps in, so this is a hard one to trace down.
imagegrid_cbar_mode-failed-diff

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

OK, I figured out this problem. This is basically due to the interaction between two points.

First, as discussed during the call, cla() doesn't call Colorbar.remove(), which means that the callback of the first image doesn't get deleted, which it should. Making cla() actually go through each and every artist to call cla() on it may be too expensive, but at least perhaps we can check if there's any colorbar on the axes and properly remove() them.

Still, even knowing that, it was unclear why the first image would overwrite the second colorbar. The reason is because the sequence of events was the following:

  • instantiate first image, create associated colorbar.
  • instantiate second image, create associated colorbar.
  • actually walk the draw tree and draw the images... which has the effect of calling _ImageBase._make_image, which contains the following snippet:
# we have re-set the vmin/vmax to account for small errors# that may have moved input values in/out of ranges_vmin,s_vmax=vrangeifisinstance(self.norm,mcolors.LogNorm)ands_vmin<=0:# Don't give 0 or negative values to LogNorms_vmin=np.finfo(scaled_dtype).epswithcbook._setattr_cm(self.norm,vmin=s_vmin,vmax=s_vmax,                                       ):output=self.norm(resampled_masked)

so here s_vmin and s_vmax were recomputed to take interpolation into account. Unfortunately, this means they may be slightly different due to the rescaling and unscaling, which can be checked by printings_vmin - A.min(), s_vmax - A.max(). One then sees that there's indeed a tiny (1e-16) difference for the first image's norm, which means it'll say, oh, I need to redraw the colorbar; but there's no difference(!) for the second image's norm, so it doesn't redraw itself. A quickfix to confirm that this works is that forcings_vmin = A.min(); s_vmax = A.max() here fixes the issue (but would cause other problems with over/under interaction with resampling; I suspect that the "correct" fix is to temporarily disable emission of callbacks byself.norm just for the call to_setattr_cm.

That was nasty :p

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

That looks like it did the trick. I disconnected from the callback before this and reconnected after it. I wonder if more fundamentally there should be something within the context manager to remove the callbacks on enter/exit...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I guess we could add adisabling_callbacks() context manager on CallbackRegistry?

withself.norm.callbacks.disabling_callbacks(),cbook._setattr_cm(self.norm, ...): ...

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yeah, that looks pretty clean to me. I'm not sure how much more use it would have in more than this one situation though, so is it worth adding or not...

Lets keep it as a separate issue and implement it later.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Rebased and updated this with the blocked callback feature now.

@greglucasgreglucasforce-pushed thenorm-updater branch 2 times, most recently fromf3064f5 tob968fc5CompareMay 13, 2021 00:21
jklymak
jklymak previously approved these changesMay 13, 2021
@jklymak
Copy link
Member

@tacaswell I'll try and pop this back on your PR review queue...

@greglucasgreglucasforce-pushed thenorm-updater branch 2 times, most recently from5cf5015 toe97155bCompareMay 25, 2021 15:20
@@ -1017,8 +1017,8 @@ def test_imshow_bool():
def test_full_invalid():
fig, ax = plt.subplots()
ax.imshow(np.full((10, 10), np.nan))
with pytest.warns(UserWarning):
fig.canvas.draw()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Why is this not warning anymore?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The previous warning was:

matplotlib/lib/matplotlib/image.py:442: UserWarning: Warning: converting a masked element to nan.  vmid = np.float64(self.norm.vmin) + dv / 2

If I print out the previousim.norm.vmin, im.norm.vmax, they were bothnp.ma.masked, which stems from passing in a fully masked array (np.max(np.ma.array([1, 2], mask=[True, True])) returnsmasked), so here we are callingnp.float64(np.ma.masked). I don't think we wantmasked as vmin/vmax, so this perhaps even helps here... The vmin/vmax turn into(0, 0) now.

Note that the image doesn't change at all, just no warning about norm vmin/vmax anymore.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is fun second-order behavior change!

Previously if you did

im=plt.imshow(np.full((10,10),np.nan),vmin=np.ma.masked,vmax=np.ma.masked)assertim.norm.vmin==0assertim.norm.vmax==0

but if you did

im=plt.imshow(np.full((10,10),np.nan)assertim.norm.vminisnp.ma.maskedassertim.norm.vmaxisnp.ma.masked

(== is nan-like withnp.ma.masked).

I would argue that this is a bug, in the sense that explicitly passing thenp.max(data) and letting us auto-computer the max gave different results.

We also say that if vmin == vmax the result of__call__ will be 0.

This will lead to a subtle behavior change in the case where:

  • the user passed us all nan
  • let us infer the vmin/vmax
  • later updates the array with non-nan values

Previously the user would get all "bad" color, now they will get the "0" color. Given the other difference in behavior of explicitly passing vmin/vmax, the agreement with "if vmin/vmax match -> go to 0", this feeling like a very cornery corner case, and the results already being useless from a communications point of view, I think we can leave this as-is and do not need a further API change note.

@jklymakjklymak dismissed theirstale reviewMay 28, 2021 14:36

I probably need to spend more time with this to understand what is going on...

@greglucasgreglucas marked this pull request as draftJune 3, 2021 20:14
@greglucasgreglucas marked this pull request as ready for reviewJuly 31, 2021 21:19
@timhoffm
Copy link
Member

timhoffm commentedAug 5, 2021
edited
Loading

If we go with a callback registry, it should be namedcallbacks. That's the name used for the registry inAxes,Axis andFigure. OnlyScalarMappable usescallbacksSM for no obvious reason (callbackSM dates back to 2008 (662581f)).

greglucas reacted with thumbs up emoji

@greglucas
Copy link
ContributorAuthor

Also fixes#17052 which I just found.

This adds a callback registry to Norm instances that can beconnected to by other objects to be notified when the Normis updated. This is particularly relevant for ScalarMappablesto be notified when the vmin/vmax are changed on the Norm.Quadcontourset overrides ScalarMappable's `changed()` function,which meant that autoscaling would get called with the wrong datatoo early. Therefore, we force an autoscale with the proper dataearlier in the `Quadcontourset.changed()` function.The Quadcontourset.changed() method assumes some attributes tobe there. If we called changed from a parent class, the objectmay not have been initialized with those attributes yet, soskip that portion of the update.Remove the ScalarMappable callback from axes_grid as thestate isn't fully cleared when updating the axes.
@tacaswelltacaswell merged commit72f7595 intomatplotlib:masterSep 1, 2021
meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull requestSep 1, 2021
@greglucasgreglucas deleted the norm-updater branchSeptember 1, 2021 20:43
dstansby added a commit that referenced this pull requestSep 1, 2021
…553-on-v3.5.xBackport PR#19553 on branch v3.5.x (ENH: Adding callbacks to Norms for update signals)
tacaswell added a commit that referenced this pull requestOct 20, 2021
ENH: Adding callbacks to Norms for update signals
ericpre pushed a commit to ericpre/matplotlib that referenced this pull requestOct 20, 2021
ENH: Adding callbacks to Norms for update signals
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@anntzeranntzeranntzer left review comments

@jklymakjklymakjklymak left review comments

@tacaswelltacaswelltacaswell approved these changes

@QuLogicQuLogicQuLogic approved these changes

Assignees
No one assigned
Projects
None yet
Milestone
v3.5.0
Development

Successfully merging this pull request may close these issues.

Colorbar update error with clim change in multi_image.py example makeNormalize objects notifiy scalar-mappables on changes
7 participants
@greglucas@jklymak@dopplershift@tacaswell@timhoffm@QuLogic@anntzer

[8]ページ先頭

©2009-2025 Movatter.jp