attn@timhoffm, can you take a quick skim to see if this is a reasonable API and plan/share any concerns?

The initial description says

NOTE: This PR has been superseded by#28454

Is there still something relevant here, or can we close and move the discussion to#28454.

@timhoffm 'superseded' was maybe a bad choice of word. This PR and#28454 will work together, but Ithink there will be some conflicts if both this and#28454 are accepted (since some changes exist in both), I thereforebelieve it is easier if I migrate the changes here to a new branch that builds on top of#28454, and make a new PR (and also includes tests). However, this will take a bit of time.

For now, it I think it is a good idea to keep the discussion of this part of the API here, even if this PR is ultimately replaced by another.
We are trying to separate concerns (and also the discussions), by having#28454 be the new colormap classes, and this be the introduction of vectormappable.

I rebased this code so that it correctly builds on#28454

Can you please try to summarize the mapping logic for multivariate and 2D colormaps. I cannot follow the architecture between the implementation details and mixing with ScalarMappable logic.

Edit: I think the OP is sufficient. I‘ll have a closer look.

Can you please try to summarize the mapping logic for multivariate and 2D colormaps. I cannot follow the architecture between the implementation details and mixing with ScalarMappable logic.

TheVectorMappable stores vector data as a dtype with multiple fields. This is so thatX.shape will return the same shape regardless of the number of variates.

This PR does not contain changes to the plotting methods, but it is intended to go something like this.

classAxes:defpcolor(self,data,cmap=None,norm=None,vmin=None,vmax=None):cmap=colors._ensure_cmap(cmap)data,norm,vmin,vmax=colors._ensure_multivariate_params(cmap.n_variates,data,norm,vmin,vmax)

Which transformsdata from what is reasonable to accept (proposed: scalar array, complex array, sequence of arrays*, dtype with multiple fields) to a dtype with the correct number of fields (as determined bycmap.n_variates).

*I prefer an input signature of (cmap.n_variates, n, m) for multivariate data, but we could use (n, m, cmap.n_variates) instead.

Edit:@timhoffm Thank you for taking a look, I didn't see your edit before I made the additional figure :D

Disclaimer: This contains some fundamental thoughts beyond the concrete MultiVar case, but since MultiVar requires significant architecture changes any, I believe it's worth looking at the whole picture.

I have not yet come to a complete solution, but I put down my working thoughts.

1. Looking at the structures involved, the three usecases behave alike for data channels and norms, but quite different in terms of colormaps and postprocessing:

   	Scalar	Bivar	Multivar
   data channels	1	2	2..N
   norms	1	2	2..N
   colormaps	1x 1D	1x 2D	2..N x 1D
   postprocessing	-	-	merge colors

   It is reasonable to abstact the step "normalized data channels -> color" in different Colormap classes. I'm inclined to have a similar abstraction for the normalization so that we have
   
   The norm may also have multiple channels (in the simple form just a collection of scalar norms). The mappable should not have to care about managing the details (i.e. a list of scalar norms). To be checked: How much of the ScalarMappable interface exposes details on the number of channels?

   Also t.b.d.: ScalarMappable attaches the mapping logic to artists via inheritance. Would it make sense to extract the data -> color logic into some separate "Mapper" entity (name t.b.d.)?
   

2. We have mutual references betweenScalarMappable andColormap
   
   We still have to determine how this interferes with the more general mappings.

3. A current limitation of the mapping mechanism is an assumed 1:1 relation between ScalarMappable and Norm concerning auto-scaling: The norm does not know about the data. If it has no predefined limits, they are automatically inferred from the first call. While it is possible to assign a Norm to multiple Mappables, autoscaling will thus only take into account values from the first mappable. When touching the whole mapping infrastructure anyway, can we improve on this so that a norm can conistently serve multiple mappables?

1. It is reasonable to abstact the step "normalized data channels -> color" in different Colormap classes. I'm inclined to have a similar abstraction for the normalization so that we have ...

It is not quite clear to me what the distinction between between theVectorMappable andMappble class would be in your case. In the weekly GSOC meetings we have considered having aVectorNorm similar to the way we have aVectorMappable (sugegsted by@story645).VectorNorm would then be a generalization ofNormalize to support multiple variates.

Having aVectorNorm class would give us certain advantages, and I believe your inclination to have aMappable class essentially deals with the same issues. From what I can tell, these are:

• a. When using the same norm on all axes (point 3. above) the litmits should be autoset using the full data, aMappable class could manage this.
• b. If there is a coupled norm, i.e. the normalization on axis 1 depends on the value on axis 2.
• c. Separation of concerns - fewer if-statements inVectorMappable

The alternative, as this PR suggests, is by simply attaching a list of norms toVectorMappable. I am a proponent of this approach because I believe it makes it easier to trace the code (fewer levels of inheritance). Also, I have looked for a use case where a coupled norm would be desirable (point b. above), and I have yet to find one. An example would be if the data is represented in an orthogonal coordinate system, but the normalization should map this in a shear coordinate system. While this would make some mathematical sense, I cannot see a use case.[Note that a circular 2D colormap places different limits than a 2D square colormap, but this is handled after the norm (in the cmap), and does not require aVectorNorm. Limiting the colorspace in other ways can be done by subclassingBivarColormap orMultivarColormap (should be a tutorial).]

However, (all of) you are more experienced to evaluate what makes code more easy to maintain than I am, and if you think aMappable class is favorable in this regard, we should implement it.

2. We have mutual references betweenScalarMappable andColormap ...

I have been thinking we need to determine how the equivalent toColorbar works in the multivariate case, but as you say this topic is probably a bit more general. I have some thoughts on this, but it might be good to discuss this in a weekly development meeting.

3. A current limitation of the mapping mechanism is an assumed 1:1 relation between ScalarMappable and Norm concerning auto-scaling: ...

To me, the intuitive behavior has always been that each axis has a separate norm:

ax.imshow((A,B),cmap='BiOrangeBlue')# orax.imshow((A,B),cmap='BiOrangeBlue',norm='LogNorm')

should give two separate norms with independent limits.

I further believe that the intuitive action for a user to take if they want identical limits on both axis is to simply set the limits:

ax.imshow((A,B),cmap='BiOrangeBlue',vmin=-1,vmax=2)

However, advanced users might want the behavior you describe above, a shared norm, and we should therefore support the following:

n=mpl.colors.Normalize()ax.imshow((A,B),cmap='BiOrangeBlue',norm=(n,n))

Where both axis are explicitly declared to have the same norm.
With this call signature, I think it would be comparatively difficult for us to change the default (set limits only with first axis), and comparatively easy for the user to get the desired behavior, i.e.:

n=mpl.colors.Normalize()norm.vmin=np.min((A,B))norm.vmax=np.max((A,B))ax.imshow((A,B),cmap='BiOrangeBlue',norm=(n,n))

NOTE: I made acommit this morning that explicitly forbids the call following call signature:

n=mpl.colors.Normalize()ax.imshow((A,B),cmap='BiOrangeBlue',norm=n)# → ValuError('When using a colormap with more than one variate, norm must be None,# a valid string, a sequence of strings, or a sequence of mpl.Colors.Normalize objects.'

because it I found it unclear if this should be a shared norm (shared limits) or if the norm should be duplicated (independent limits), and I believe we should only accept input where the users intent is clear.

@timhoffm As I understand it, the weekly development meeting today is cancelled, but I will try to be there next week.
If you give me some more details on how you expect theMappable class to integrate (call signature?), I could prototype an alternative branch before then.

Thanks for the detailed answer, I'll have a closer look later.

Re: 3. Multiple mappables with one colorbar:
I'm referring tohttps://matplotlib.org/stable/gallery/images_contours_and_fields/multi_image.html. T.b.d. maybe this can be slightly simplified nowadays, but we still have a fundamental disconnect between the images and the colorbar.

T.b.d. maybe this can be slightly simplified nowadays

I took a look, and the norm now seems to connect automatically, but you still need to manually connect the images before you swap out the colormap.

If we made a 'Mappable' class, we could definitely simplify this, by having a single object that holds the norm+colormap for all images in the example above. I imagine this class would then handle everything that deals with the data→norm→colormap transform, i.e.VectorMappable.to_rgba(...) and parts ofImage.make_image(...).

I think you are right that if we want to implement this, it would be a good idea to bundle it with this PR.

Personally I would suggestNormAndColor as a name for the class (I like descriptive names), and I would suggest you can pass it as an alternative to the norm, i.e.:

nac=mpl.cm.NormAndColor(norm='LogNorm',cmap='Viridis')ax[0].imshow(A,norm=nac)ax[1].imshow(B,norm=nac)

If this rhymes with what you are thinking, I'll take a look at implementing it.

b. If there is a coupled norm, i.e. the normalization on axis 1 depends on the value on axis 2.

Maybe the usecase in#19515 (by@greglucas ) of choosing vmin and vmax, which I've often done that based on histograms so a linked zoom on histogram and update vmin and vmax w/ the corresponding values.

Also just to make sure I'm on the same page, I'm interpreting the figures above as:

$Bivariate: channel_1 \times channel_2 \rightarrow norm_1(channel_1) \times norm_2(channel_2) \rightarrow RGB$

$multivariate: channel_1 \times channel_2 \rightarrow norm_1(channel_1) \times norm_2(channel_2) \rightarrow RGB_1 \times RGB_2 \rightarrow RGB$

And now I'm wondering where one of the use cases I was originally thinking of fits:

$nvariate: channel_1 \times channel_2 \rightarrow norm(channel_1, channel_2) \rightarrow RGB$

ETA: nvariate is also weird cause it could maybe work w/ scalermappables and the hard part is it knowing how to input match. And yes these all have the same top level signature:
$vnorm: channel_0 \times ...\times channel_i \times ... \times channel_n \rightarrow RGB$

@story645 The PR as it stands works like this:
$multivariate:\underbrace{channel_1×channel_2}_{VectorMappabe} \underbrace{→ norm_1×norm_2} _{norm\ layer} \underbrace{→RGB×RGB →RGB} _{cmap\ layer}$

$bivariate: \underbrace{channel_1×channel_2} _{VectorMappable} \underbrace{→ norm_1×norm_2} _{norm\ layer} \underbrace{→ RGB} _{cmap\ layer}$

as you say it does not support:
$\underbrace{channel_1×channel_2} _{VectorMappable} \underbrace{→ norm(channel_1,channel_2)} _{norm\ layer} \underbrace{→ RGB} _{cmap\ layer}$
with a 1D colormap, nor does it support
$\underbrace{channel_1×channel_2} _{VectorMappable} \underbrace{→ norm _1(channel_1,channel_2)×norm _2(channel_1,channel_2)} _{norm\ layer} \underbrace{→ RGB} _{cmap\ layer}$
with a 2D colormap, etc..

b. If there is a coupled norm, i.e. the normalization on axis 1 depends on the value on axis 2.

Maybe the usecase in#19515 (by@greglucas ) of choosing vmin and vmax, which I've often done that based on histograms so a linked zoom on histogram and update vmin and vmax w/ the corresponding values.

When I was talking about a coupled norm, I was thinking of norms that take two arguments, i.e.$norm(channel_1,channel_2)$, which this PR does not support right now.

If we want to introduce norms that take two arguments, we definitely need aNormAndColor kind of object to hold them, and classes for$norm(channel_1,channel_2)$ objects.
However, it is not clear to me what mathematical operation$norm(channel_1,channel_2)$ should perform.
@story645 do you have a specific use case (mathematical transform or example) in mind?

EDIT: I'll take a look at implementing aNormAndColor kind of object anyways, and then we will at least have the infrastructure to support$norm(channel_1,channel_2)$ kinds of things

Re: 3. Multiple mappables with one colorbar: I'm referring tohttps://matplotlib.org/stable/gallery/images_contours_and_fields/multi_image.html. T.b.d. maybe this can be slightly simplified nowadays, but we still have a fundamental disconnect between the images and the colorbar.

I took a crack at implementing aNormAndColor class (name subject to change).
Using this as a container allows us to easily set up multiple plots that share a$data→norm→color$ pipeline.
https://matplotlib.org/stable/gallery/images_contours_and_fields/multi_image.html can now be simplified as follows:

importmatplotlibmatplotlib.use('qtagg')importmatplotlib.pyplotaspltimportnumpyasnpnp.random.seed(19680801)data=np.linspace(0.1,0.6,6)[:,np.newaxis,np.newaxis]*np.random.rand(6,10,20)nac=matplotlib.cm.NormAndColor()nac.vmin=np.min(data)nac.vmax=np.max(data)fig,axs=plt.subplots(2,3)fig.suptitle('Multiple images')fori,axinenumerate(axs.ravel()):im=ax.imshow(data[i],norm=nac)ax.label_outer()cb=fig.colorbar(im,ax=axs,orientation='horizontal',fraction=.1)plt.show()

The video shows how the images couple:

(Additionally, inhttps://matplotlib.org/stable/gallery/images_contours_and_fields/multi_image.html, changing the colormap on the colorbar does not propagate to the images, but these changes propagate in the video above)

This implementation also allows us to couple ND colormaps and 1D colormaps, so that changes in one propagates to the others:

importmatplotlibmatplotlib.use('qtagg')importmatplotlib.pyplotaspltimportnumpyasnpnp.random.seed(19680801)data=np.random.rand(2,10,20)fig,axes=plt.subplots(1,3,figsize=(10,3))im_2D=axes[0].imshow(data,cmap='BiOrangeBlue')im_0=axes[1].imshow(data[0],norm=im_2D.nac[0])im_1=axes[2].imshow(data[1],norm=im_2D.nac[1])cb=fig.colorbar(im_0)cb=fig.colorbar(im_1)foraxinaxes:ax.label_outer()plt.show()

@timhoffm Is this what you had in mind?

I had to build this in a branch further down the pipeline than this PR, because it needs some features of the plotting functions not available in this PR. If you want to take a look, theNormAndColor class can be found here:https://github.com/trygvrad/matplotlib/blob/Multivariate_NormAndColor_class/lib/matplotlib/cm.py

I pushed some updates to this branch, but this is largely just because I want to see the code coverage, and this is the easiest way for me to see how the code coverage changes.

Provisional names areColorizer instead ofMapper (in cm.py)
ColorizingArtist as the base class for image and collection (in artist.py).
andColorizerShim to provide access from to the variables on theColorizer from a class that subclasses bothColorizingArtist andColorizerShim [i.e. to match theScalarMappable interface] (in cm.py)
Note that the methods on theColorizerShim are not used internally, and all internal use acts onColorizer objects directly.

We discussed this in the Matplotlib weekly meeting today, and we will split this into 2 PRs:

1. Minimal implementation of theColorizer,ColorizingArtist andColorizerShim classes
2. Changes needed to support multivariate data.

I'll make a comment here once 1. is ready.

I made a new PR with a minimal implementation of theColorizer class#28658
I would like to move further discussion of the implementation to#28658
and keep any further discussion of feature branch vs main etc. in#28428 (this thread)

and keep any further discussion of feature branch vs main etc. in#28428 (this thread)

Also, just to explain why#28658 was opened on main, consensus from the call is that we're gonna try the "merging to main" approach in the stages/PRs@trygvrad outlined.

@trygvrad I think this is superseeded. Can this be closed?

yes