Repository files navigation

• MIT License - all source code
• Copyright (c) 2020-2024 Sherman Lo

The mode filter is an edge-preserving smoothing filter by taking the local modeof the empirical density. This may have applications in image processing such asimage segmentation. The filter is available:

• As anImageJ plugin which uses either the CPU or anNvidia GPU. Byextension, developers may use the Java API or the provided CLI.
• As a Python package using anNvidia GPU only.

TheCUDA binding toJava andPython was done usingJCuda andCuPyrespectively. The use of a GPU speeds up the filtering by a huge margin.

Where appropriate, please cite the thesis

• Lo, S.E. (2020).Characterisation of Computed Tomography Noise in ProjectionSpace with Applications to Additive Manufacturing. PhD thesis, University ofWarwick, Department of Statistics.

The mode filter was applied to theMandrill test image.Top left to top right, bottom left to bottom right: mandrill test image withthe mode filter with a radius of 2, 4, 8, 16, 32, 64, 128 applied.

Clone this repository and follow the instructions below in order:

• If you require the use of a GPU (mandatory for the Python implementation),follow the instructions for compiling theCUDA code.
• Afterwards, there are instructions for:
  • Compiling theJava code into a.jar file. This is required for the useof a GPU.
  • Downloading the compiled.jar. files from Releases. Only CPU computationis support with this method.
  • How to install the.jar file as anImageJ plugin and using the providedCLI.
  • Building the Python package and installing it. A GPU is required in thePython implementation.

CompilingCUDA code for the use of anNvidia GPU requiresGCC and theNvidia CUDA Development Kit, a version appropriate for your GPU, which shouldinclude annvcc compiler.

Older versions of theNvidia CUDA Development Kit can be found in thearchive.

Identify the architecture of your GPU by looking it up in the manual fortheNVIDIA CUDA Compiler Driver NVCC Section 5.2or other sources such asthis.For example:

• AnNvidia Tesla K80 has a Kepler architecture with codesm_37.
• AnNvidia Tesla V100 has a Volta architecture with codesm_70.
• AnNvidia GeForce GTX 1660 has a Turing architecture with codesm_75.
• AnNvidia A100 has an Ampere architecture with codesm_80.

Compile the code into a.ptx file by callingmake and providing thearchitecture. For example, for anNvidia A100 with codesm_80

make NVCC_ARCH=sm_80

The compiled.ptx file should be located in bothcuda/ andpython/modefilter/.

RequiresJava Runtime Environment,Java Development Kit andMaven. For theuse of the GPU, you must compile theCUDA beforehand.

Atpom.xml, run

mvn package

to compile theJava code. The compiled.jar file should be located intarget/Empirical_Null_Filter-*.*.*.jar and can be used as anImageJ plugin.Copies of required libraries are stored intarget/libs/ and would need to beinstalled inImageJ as well.

Downloadtarget.zip from thereleases (CPU only) andextract it.

The compiled.jar file is namedEmpirical_Null_Filter-*.*.*.jar and can beused as anImageJ plugin. Copies of required libraries are stored inlibs/and would need to be installed inImageJ as well.

Installation ofEmpirical_Null_Filter-*.*.*.jar can be done by copying thefile intoFiji'splugins/ directory or, inFiji, using thePlugins menufollowed byInstall... (or Ctrl + Shift + M).

The required.jar libraries are to be copied intoFiji'sjars/ directory.They are:

• commons-math3-3.6.1.jar (may already be provided)
• jcuda-10.1.0.jar (for GPU usage)
• jcuda-natives-10.1.0-linux-x86_64.jar (or similar for GPU usage)

• Number of (CPU) threads
  • Number of CPU threads to use when doing mean, median and quantile filtering.Currently, they are only implemented on the CPU. These are used as inputsfor mode filtering. Thus there will be some CPU computation even in theGPU version of the mode filter. It will default to use all detectablethreads.
• Number of initial values
  • Number of initial values for the Newton-Raphson method. Increase this formore accurate filtering at a price of more computational time. Compared toother options, this has a big effect on the resulting image. The defaultvalue is 3 but should be in the order of 50-100 if this filter is to beapplied to (non-Gaussian) images.
• Number of steps
  • Number of iterations in the Newton-Raphson method. Increase this for moreaccurate filtering at a price of more computational time.
• Log tolerance (CPU version only)
  • The tolerance allowed for the Newton-Raphson method to accept the solution.Decrease this for more accurate filtering at a price of more computationaltime.
• Block dim x and y (GPU version only)
  • Sets the dimensions of the block of threads on the GPU. This affects theperformance of the filter. Good suggestions are 16 and 32. Solutions areshared between neighbours within blocks.

The mode filter can be used via the terminal by calling theEmpirical_Null_Filter-x.x.x.jar file. To use a GUI for parameter selection

java -jar Empirical_Null_Filter-x.x.x.jar gui ['cpu' or'gpu'] \<loc of image to filter>

This will make a GUI appear to select your parameters. Once selected, click OKto filter the image. A dialogue box will appear to save the resulting image in.png format.

To run the mode filter without a GUI

java -jar Empirical_Null_Filter-x.x.x.jar run ['cpu' or'gpu'] \<loc of image to filter><loc to save resulting .png> [options]

where the options are

• -r radius of the kernel
• -n number of CPU threads
• -i number of initial points for Newton-Raphson
• -s number of steps for Newton-Raphson
• -t stopping condition tolerance for Newton-Raphson (recommend negativenumber), only for CPU
• -x x block dimension, only for GPU
• -y y block dimension, only for GPU

Requiressetuptools. Verify the compiled.ptx file is located inpython/modefilter/.

Atpyproject.toml, run

python -m build

which will build the package. To install it, run

pip install dist/modefilter*.whl

You may require a version ofCuPy which uses a specific version ofCUDA. Inthat case, you may edit the filepyproject.toml with the version ofCuPyyou want. For example, if you requireCuPy withCUDA 11, edit thedependencies to contain

dependencies = ["numpy","scipy","cupy-cuda11x",]

Please refer to theCuPy installation documentation.

Apptainer definition files are provided as a way tocompileCUDA andJava code in a container as well as use it. There is alsoa definition file which builds a wheel forPython.

To build the container

apptainer build modefilter-cpu.sif modefilter-cpu.def

To apply the mode filter on an image using the container via the terminal

apptainer run modefilter-cpu.sif run cpu \<loc of image to filter><loc to save resulting .png> [options]

where the options are the same in the previous section.

The compiled.jar files can be extracted using

apptainerexec \    modefilter-cpu.sif cp -r /usr/local/src/modefilter/target<destination>

Identify the architecture of your GPU (as discussed previously here).For example:

• AnNvidia Tesla V100 has a Volta architecture with codesm_70.
• AnNvidia GeForce GTX 1660 has a Turing architecture with codesm_75.
• AnNvidia A100 has an Ampere architecture with codesm_80.

Editmodefilter-gpu.def so thatnvcc_arch has the correct architecture code,for example:

• For anNvidia GeForce GTX 1660
  • nvcc_arch="-arch=sm_75"

The container can be built

apptainer build modefilter-gpu.sif modefilter-gpu.def

To apply the mode filter on an image using the container via the terminal, usethe--nv flag

apptainer run --nv modefilter-gpu.sif run ['cpu' or'gpu'] \<loc of image to filter><loc to save resulting .png> [options]

Similarly to the Apptainer forJava, editmodefilter-python.def so thatnvcc_arch has the correct architecture code.

The container can be built

apptainer build modefilter-python.sif modefilter-python.def

The compiled.whl file can be extracted using

apptainerexec \    modefilter-python.sif bash -c \"cp /usr/local/src/modefilter/dist/modefilter-*.whl <destination>"

Running the container will runPython with themodefilter package available.

apptainer run --nv modefilter-python.sif run

Depending on your GPU architecture, you may require an older version of theNvidia CUDA Toolkit. For example, aNvidia Tesla K80 is supported by theNvidia CUDA Toolkit version 10.1. Refer toNvidia'sCUDA toolkitarchive.

The mode filter is an image filter much like the mean filter and median filter.They process each pixel in an image. For a given pixel, the value of the pixelis replaced by the mean or median over all pixels within a distancer away.The mean and median filter can be used inImageJ, it results in a smoothing ofthe image.

Top left:Mandrill test image.Top right: Mean filter with radius 32. Bottom left: Median filter withradius 32. Bottom right: Mode filter with radius 32.

The mode filter is a by-product of the empirical null filter. Instead of takingthe mean or median, the mode is taken, more specifically, the argmax of theempirical density. The optimisation problem was solved using the Newton-Raphsonmethod. Various random initial values were tried to home in on the globalmaximum. Because the filtered image is expected to be smooth, the differentinitial values were influenced by neighbouring pixels to aid in the optimisationproblem.

The resulting mode-filtered image gives a smoothed image which has an impastoeffect and preserved edges. This may have applications in noise removal or imagesegmentation.

The mode filter was implemented on the CPU by modifying existingJava codefromImageJ. Each thread filters a row of the image in parallel from left toright. The solution to one pixel is passed to the pixel to the right. The filterwas also implemented on the GPU by writingCUDA code which can be compiled andread by theJCuda package. The image is split into blocks. Within a block,each thread filters a pixel and shares its answer with neighbouring pixelswithin that block.

One difficulty is that with the introduction ofCUDA code, the ability to"compile once, run anywhere" is difficult to keep hold of. A design choice wasthat the user is to compile theCUDA code into a.ptx file. This is thenfollowed by compiling theJava code with the.ptx file into a.jar filewhich can be installed as a Plugin inImageJ orFiji. The compiled.jarfile can be used byMATLAB as well.

• Lo, S.E. (2020).Characterisation of Computed Tomography Noise in ProjectionSpace with Applications to Additive Manufacturing. PhD thesis, University ofWarwick, Department of Statistics.
• Efron, B. (2004). Large-scale simultaneous hypothesis testing: The choice of anull hypothesis.Journal of the American Statistical Association,99(465):96.
• Griffin, L. D. (2000). Mean, median and mode filtering of images.Proceedingsof the Royal Society of London A: Mathematical, Physical and EngineeringSciences, 456(2004):2995–3004.
• Charles, D. and Davies, E. R. (2003). Properties of the mode filter whenapplied to colour images.International Conference on Visual InformationEngineering VIE 2003, pp. 101-104.