Operation
flimview consists of several utility libraries for storage, visualization and manipulation of FLIM data. After reading and processing the raw data (.ptu or.sdt files), the input data is stored in the main class of the package, aFlimCube, which is an object that represents 3D data (2D spatial dimensions and one temporal dimension) allowing multiple methods to access and manipulate the data cube.
Among its attributes is the header which includes all the metadata available for the dataset, sizes of the arrays, resolution and whether this cube is binned and/or masked (see Binning and Masking sections below). When the data cube is masked, a process described in the following sections, the pixel-level mask is also included as an attribute in theFlimCube object. Due to its construction, we consider aFlimCube to be self-explanatory as it contains all needed information (including its corresponding metadata) to analyze, visualize and fit the data.
Binning.flimview includes a binning function with several pre-defined kernels, although a user-defined kernel is also possible and can be easily added. The binning procedure is simply a convolution kernel applied to the image to increase the signal or to enhance its features. This function takes aFlimCube as an input and returns aFlimCube as output by copying all of the metadata and properties from the input data. The following snippet shows an example on how to bin an existingFlimCube using a Gaussian kernel of size 9×9 and a sigma of 3 (the size of the kernel is 2 ∗ b + 1, whereb is the bin size)
import flimview.flim as flimimport flimview.io_utils as io# IO utilsdata, header = io.read_sdt_file (sdtfile)FC = flim.FlimCube (data, header)# FlimCube ClassFCbinned = flim.binCube (FC,bin=4, kernel=’gauss’, sigma=3)# FCbinned is also a FlimCube object#
The kernel functions included inflimview are Gaussian kernel, Airy disk kernel, linear kernel, and a flat kernel. Customized kernels can be easily incorporated.Figure 1 shows examples of different 9×9 kernels where we can observe how the weight is distributed around the central pixel. In the case of a Gaussian kernel or an Airy disk kernel, the width of the kernel can be customized using the sigmaσ parameter. For reference, the Gaussian kernel is given by:
where
x is the distance to the center pixel and
I0 is a normalization factor (usually just a unit). In the case of the Airy disk, the kernel is given by:
where
J1(
x) is the Bessel function of the first kind of order one. In both cases,
I(
x) is discretized and normalized to the unit across the 2d binning window.
Figure 1. Example of different 9x9 kernels used for binning.
Masking.FlimCube also includes methods for looking at the header information and for masking pixels below a given integrated intensity, or below a given peak threshold in the time series, or even for a custom made geometry mask. The pixels masked this way will not be used during the fitting analysis while the mask is also saved within the same inputFlimCube object. The following snippet shows how to apply a mask for a givenFlimCube :
import flimview.flim as flimimport flimview.io_utils as io#IO utils# Read data# sdtfile = ...data, header = io.read_sdt_file (sdtfile)FC = flim.FlimCube (data, header)# To mask by intensityFC.mask_intensity(100)# To mask by peakFC.mask_peak(5)# To mask by a given geometryFC.mask_peak(0, mask=custom_numpy_masked_array)#
Figure 2 shows different masks applied to the example raw (top) and binned (bottom) data. Masks can be combined and can be transferred between binned and raw data.
Figure 2. Different Masking approaches to the raw data (top) and binned data (bottom).
Masks can be combined. Masked pixels will not be used in the analysis.
Fitting. Once the data is read and binned, and have the low signal and error pixels masked,flimview provides a function to fit a decay function for every pixel in the temporal data using a predefined model (from the modulemodels.py which can be customized and also parallelized. By default a double exponential is used as follows:
where
a1 and
a2 are the amplitudes for the given exponential factor. Usually, there is an extra constraint given by
a1 +
a2 = 1.
τ1 and
τ2 are the mean lifetime for each exponential.
l0 is a level constant.
Before fitting the entire cube, a single fit is done to the mean intensity values to obtain initial guess parameters for the fitting procedures as shown inFigure 3, as well as the boundaries to each parameter which can also be provided by hand. The snippet below shows how this is done usingflimview :
import flimview.flim as flim# Assuming a FlimCube FC, let’s compute the average of the pixel valuestimesteps, mean_pixel = flim.meanDecay (FCbin)# Let’s clean these by a threshold, normalize and shift to the maximumtimesteps_clean, mean_pixel_clean, max_value, time_shift = flim.cleanCurve (timesteps, mean_pixel, norm=True, threshold=0.02)# Let’s assume a model (double exponential with a level parameter)mymodel = models.model1# Let’s fit that mean pixel using fitPixel function# xf, yf are the time and value used for the fitting, pfit is the fitted values, pcov is the parameters covariance matrix and chi2# is the Chi - square value of the fitxf, yf, pfit, pcov, chi2 = flim.fitPixel( timesteps, mean_pixel, mymodel, initial_p =[0.8, 0.3, 3, 0.03], bounds =(0, np.inf), norm = True)#
Figure 3. Example of fitting the average and normalized pixel decay to determine initial parameters.
Note that the data is normalized, cleaned, and shifted with respect its peak. For example, to generateFigure 3 we can use the following:
# We can plot the fitted data (only considering from the maximum value)plt.plot(timesteps - time_shift, mean_pixel/max_value,’.’, label=’original data’)plt.plot(xf, yf,’.’, label=’fitted data’)plt.plot(xf, mymodel(xf, *pfit))plt.xlabel(’time [ns]’, fontsize=15)plt.ylabel(’Intensity’, fontsize=15)plt.text(6, 0.5, flim.printModel(mymodel, pfit, pcov, chi2, oneliner=False))plt.legend(loc=0)
After those parameters are defined and a model is selected a functionfitCube takes aFlimCube as input and produces anotherflimview class calledFlimFit which contains the fitting information for every pixel, including all parameters from the model, their errors andχ2 values. It is a collection of 2D arrays for the results of the fitting.
To generate aFlimFit object we can use the following:
# Define the boundaries for parametersbounds_lower=[0.0, 0.0, 0., 0.]bounds_upper=[1, 1., 5., 1.]# Fit the whole cube to create a FlimFit objectFfit = flim.fitCube( FCbin, mymodel, pfit, bounds =(bounds_lower, bounds_upper), norm=True, threshold=0.02)
Visualization. The package also includes means to visualize theFlimCube andFlimFit objects using intensity or peak values. Additionally, one can visualize the results from the fitting procedure using a function calledplotFit, which takes aFlimCube and aFlimFit as input, along with a pixel coordinate, to produce a figure as the one shown inFigure 4, showing the fit results for a given pixel, the parameters, the residuals, and the residuals distribution which is a useful way to explore the results. These can be combined in an interactive widget showing, for example, the same figure for a given point selected interactively. It is also possible to combine multiple visualizations to generate a sequence of plots into an animation of the datacube. Thepackage repository includes examples on how this can be done to generate a short animation. Other visualization functions are also included and documented in the package.
Figure 4. Visualization example showing the fit for an individual pixel, the residuals and the location on the image for both,.sdt (a) and.ptu (b) files.
Storage.FlimCube andFlimFit can be easily stored and easily retrieved by using a hierarchical and compression data store format for array-like data called HDF5 (The HDF Group, 2000–2020). Using this data format, we can store multiple data in different ‘folders’, including parameters, different models, masks, raw, binned and fitted data in one file for easy retrieval, as it only loads what is asked for, saving memory consumption if necessary. In the case of multiple files, we can serve them on-demand using a web server and only load the data as necessary, including over the web, which is an HDF5 feature. The methodssaveCube, saveFit, loadCube, loadFit, viewH5 are implemented in theio_utils modules insideflimview. For example, after runningfitCube and saving the example data in a file, one can easily see its internal structure with:
import flimview.io_utils as ioio.viewH5(h5file)
which produces the following output, showing the file content of the processed data:
File: test_ptu .h5----> example_ptu: /example_ptu--------> v0: /example_ptu /v0------------> binned: /example_ptu /v0/ binned----------------> data: (256, 256, 107)----------------> mask: (256, 256)------------> fit: /example_ptu /v0/ fit----------------> a1: (256, 256)----------------> a1_err: (256, 256)----------------> chi2: (256, 256)----------------> level: (256, 256)----------------> level_err: (256, 256)----------------> mask: (256, 256)----------------> residuals: (256, 256)----------------> tau1: (256, 256)----------------> tau1_err: (256, 256)----------------> tau2: (256, 256)----------------> tau2_err: (256, 256)------------> raw: /example_ptu /v0/ raw----------------> data: (256, 256, 107)--------> v1: /example_ptu /v1------------> raw: /example_ptu /v1/ raw----------------> data: (256, 256, 107)--------> v2: /example_ptu /v2------------> raw: /example_ptu /v2/ raw----------------> data: (256, 256, 107)--------> v3: /example_ptu /v3------------> raw: /example_ptu /v3/ raw----------------> data: (256, 256, 107)
More examples on how to save and retrieve FLIM data using this format are included in the example notebooks at theGithub repository.
Comments on this articleComments (0)