The filtering weight at one extreme is not being calculated correctly. In the case of linear interpolation, this makes the one weight that should be zero be instead nonzero. This then results in a pixels of constant values manifesting a small blip after linear interpolation if the center of an input pixel exactly coincides with the center of a output pixel (within the agg backend, its pixels may not be exactly aligned with the true pixels).
The filtering weights are not aligned correctly in their array as expected by subsequent code. For a weight array of N values, the "pivot" needs to be at N / 2 - 1, not at N / 2. This results in a bias where the linearly interpolated values are calculated slightly to the right of where they should be calculated.
Specific to nonaffine transforms, the values in the lookup table used to store the calculation are being truncated when they should be rounded. This results in a bias (smaller than the above) to the right.

Below are illustrative plots and the generating script for linear resampling of a 2-pixel array. There are fundamental limitations of the accuracy due to the subpixel approach of the agg backend, but the motivation here is for the average behavior to be accurate (i.e., eliminate bias).

Before this PR

After this PR

Script

importmatplotlib.pyplotaspltimportnumpyasnpfrommatplotlib.transformsimportAffine2D,Transformfrommatplotlib.imageimportresample,BILINEARin_data=np.array([[0.1,0.9]])in_shape=in_data.shapein_edges=np.arange(in_shape[1]+1)out_shape= (1,20)out_edges=np.arange(out_shape[1]+1)ideal_data=np.array([[0.1,0.1,0.1,0.1,0.1,0.14,0.22,0.3,0.38,0.46,0.54,0.62,0.7,0.78,0.86,0.9,0.9,0.9,0.9,0.9]])# Create a simple affine transform for scaling the input arrayaffine=Affine2D().scale(sx=out_shape[1]/in_shape[1],sy=1)# Create a nonaffine version of the same transform by compositing with a nonaffine identity transformclassNonAffineIdentityTransform(Transform):input_dims=2output_dims=2definverted(self):returnselfnonaffine=NonAffineIdentityTransform()+affineaffine_data=np.empty(out_shape)nonaffine_data=np.empty(out_shape)resample(in_data,affine_data,affine,interpolation=BILINEAR)resample(in_data,nonaffine_data,nonaffine,interpolation=BILINEAR)fig,axs=plt.subplots(3,1,figsize=(4.8,6.4),layout="constrained")axs[0].stairs(in_data[0, :],in_edges)axs[0].grid(ls='dotted')axs[0].set_xticks(in_edges)axs[0].set_title('Original data')axs[1].stairs(affine_data[0, :],out_edges,label='affine')axs[1].stairs(nonaffine_data[0, :],out_edges,ls='dashed',label='nonaffine')axs[1].grid(ls='dotted')axs[1].set_xticks(out_edges)axs[1].legend()axs[1].set_title('Interpolated data')axs[2].stairs(affine_data[0, :]-ideal_data[0, :],out_edges,label='affine')axs[2].stairs(nonaffine_data[0, :]-ideal_data[0, :],out_edges,ls='dashed',label='nonaffine')axs[2].grid(ls='dotted')axs[2].set_xticks(out_edges)axs[2].set_ylim(-0.006,0.006)axs[2].legend()axs[2].set_title('Interpolated data minus ideal result')plt.show()

PR checklist

[N/A] "closes #0000" is in the body of the PR description tolink the related issue
new and changed code istested
[N/A]Plotting related features are demonstrated in anexample
[N/A]New Features andAPI Changes are noted with adirective and release note
[N/A] Documentation complies withgeneral anddocstring guidelines

github-actionsbot added the topic: images label

Jun 18, 2025

ayshih commented

Jun 18, 2025

View reviewed changes

extern/agg24-svn/include/agg_image_filters.h

		}
		unsigned end = (diameter() << image_subpixel_shift) - 1;
		m_weight_array[0] = m_weight_array[end];

Copy link

ContributorAuthor

ayshihJun 18, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This line is one of the bugs.0 andend are different distances frompivot, so the corresponding weights typically should not be the same.

Copy link

Contributor

anntzerJun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Please put the new version under a macro guard, as was done in#28122, so that we keep a track of what's the original Agg and what we changed on top of it.

Copy link

ContributorAuthor

ayshihJun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Makes sense, done

Copy link

ContributorAuthor

ayshihJun 18, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It turns out that I can't add a preprocessor definition in_image_resample.h a la#28122. There is buggy code inagg_image_filters.cpp that needs to be skipped, but since it is a CPP file rather than a H file, it gets compiled separately with no awareness of_image_resample.h. I have put the preprocessor definition (MPL_FIX_IMAGE_FILTER_LUT_BUGS) here at the top ofagg_image_filters.h, which is lower level than would be nice, but at least still makes it clear what is custom code.

Copy link

ContributorAuthor

ayshih commentedJun 18, 2025•
edited
Loading

The filtering weight at one extreme is not being calculated correctly. In the case of linear interpolation, this makes the one weight that should be zero be instead nonzero. This then results in a pixels of constant values manifesting a small blip after linear interpolation if the center of an input pixel exactly coincides with the center of a output pixel (within the agg backend, its pixels may not be exactly aligned with the true pixels).

You can see such a blip in the example above, but it's easy to overlook. Here's a starker example: an array of two pixels with the same value (0.1) is resampled to an array of ten pixels. Before this PR, two of the pixels in the output array have values slightly greater than 0.1.

>>>importnumpyasnp>>>frommatplotlib.transformsimportAffine2D>>>frommatplotlib.imageimportresample,BILINEAR>>>>>>in_data=np.array([[0.1,0.1]])>>>in_shape=in_data.shape>>>>>>out_shape= (1,10)>>>>>># Create a simple affine transform for scaling the input array>>>affine=Affine2D().scale(sx=out_shape[1]/in_shape[1],sy=1)>>>>>>affine_data=np.empty(out_shape)>>>resample(in_data,affine_data,affine,interpolation=BILINEAR)>>>>>>print(affine_data)[[0.10.100012210.10.10.10.10.100012210.10.10.1       ]]

After this PR, the output is the expected:

>>>print(affine_data)[[0.10.10.10.10.10.10.10.10.10.1]]

ayshih force-pushed themore_resample_bugs branch fromd2fa1e5 to7eb18b2Compare

June 18, 2025 13:54

github-actionsbot added the topic: mpl_toolkit label

Jun 18, 2025

ayshih force-pushed themore_resample_bugs branch from7eb18b2 tod9f8bb5Compare

June 18, 2025 14:35

github-actionsbot removed the topic: mpl_toolkit label

Jun 18, 2025

Copy link

Member

jklymak commentedJun 18, 2025

This killed the log-scale images?

Also this changed abunch of image tests, but I can't see any difference by eye. Is it worth just relaxing the tolerance on those tests a bit and at the same time adding new tests for these fixes?

ayshih force-pushed themore_resample_bugs branch from7eb1cea to5cbb7bbCompare

June 18, 2025 15:31

Copy link

ContributorAuthor

ayshih commentedJun 18, 2025

This killed the log-scale images?

Indeed, I need to track down why.

Also this changed abunch of image tests, but I can't see any difference by eye. Is it worth just relaxing the tolerance on those tests a bit and at the same time adding new tests for these fixes?

Yes, the other 17 image tests are mostly subtle differences, but merely relaxing the tolerance doesn't make sense to me. The updated baseline images do represent what the output should be.

Copy link

Member

jklymak commentedJun 18, 2025

Fair enough - however, it would be good if there were some tests devised where it showed a visual effect of these inaccuracies.

ayshih force-pushed themore_resample_bugs branch from5cbb7bb toc967025Compare

June 18, 2025 17:38

ayshih added3 commits

June 18, 2025 15:59

Fix bugs in the calculation of the weight array

23656e8

Fix bug with using the distortion table for nonaffine transformations

a0902fe

Update values in a test

717a221

ayshih force-pushed themore_resample_bugs branch 2 times, most recently frome198af6 to717a221Compare

June 18, 2025 20:39

ayshih commented

Jun 18, 2025

View reviewed changes

extern/agg24-svn/src/agg_image_filters.cpp

		@@ -88,6 +88,7 @@ namespace agg
		}
		}

		#ifndef MPL_FIX_IMAGE_FILTER_LUT_BUGS

Copy link

ContributorAuthor

ayshihJun 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The following code block is removed because it not only has the same bugs as the code inagg_image_filters.h, but also it shouldn't even exist because it modifies the weight array after it has been normalized, thus potentially destroying the normalization.

Labels

topic: images

3 participants

Movatterモバイル変換

Uh oh!

Fixed several accuracy bugs with image resampling#30184

Are you sure you want to change the base?

Fixed several accuracy bugs with image resampling#30184

Conversation

ayshih commentedJun 18, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

PR summary

Before this PR

After this PR

Script

PR checklist

Uh oh!

ayshihJun 18, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzerJun 18, 2025

Choose a reason for hiding this comment

Uh oh!

ayshihJun 18, 2025

Choose a reason for hiding this comment

Uh oh!

ayshihJun 18, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ayshih commentedJun 18, 2025• editedLoading Uh oh!There was an error while loading.Please reload this page.

Uh oh!

Uh oh!

jklymak commentedJun 18, 2025

Uh oh!

ayshih commentedJun 18, 2025

Uh oh!

jklymak commentedJun 18, 2025

Uh oh!

ayshihJun 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ayshih commentedJun 18, 2025•
edited
Loading

ayshihJun 18, 2025•
edited
Loading

ayshihJun 18, 2025•
edited
Loading

ayshih commentedJun 18, 2025•
edited
Loading