技术领域technical field
本发明涉及光谱分析技术领域，具体涉及一种光谱曲线降噪生成式模型的训练方法、装置、光谱曲线的降噪方法及存储介质。The invention relates to the technical field of spectral analysis, in particular to a training method and device for a spectral curve noise reduction generative model, a spectral curve noise reduction method and a storage medium.
背景技术Background technique
近年来，显微成像技术的飞速发展极大地推动生命科学技术的进步。其中，荧光显微镜通过荧光标记的探针来观测生物样本，其基本发光原理是荧光团吸收特定波长激发光的光子并发出比激发波长更长的荧光的过程。因此，通过设计合适的特异性结合荧光染料，如荧光蛋白、有机荧光小分子和量子点等，可以实现亚细胞器级别的空间分辨率和毫秒级别的时间分辨率形态学观测。而除了荧光分子的强度信息外，其荧光光谱也是观测标记分子的重要参量之一，且一些环境敏感型荧光染料分子能够提供更多形态学之外的特征，例如分子结构、生物分子空间排列和分子周围微环境等。总之，荧光光谱能够在另一个维度揭示细胞的环境参数，是生命活动研究重要工具之一。In recent years, the rapid development of microscopic imaging technology has greatly promoted the progress of life science technology. Among them, fluorescence microscopy observes biological samples through fluorescently labeled probes. The basic principle of luminescence is the process in which fluorophores absorb photons of excitation light of a specific wavelength and emit fluorescence longer than the excitation wavelength. Therefore, by designing suitable specific binding fluorescent dyes, such as fluorescent proteins, small organic fluorescent molecules, and quantum dots, morphological observations with subcellular organelle-level spatial resolution and millisecond-level temporal resolution can be achieved. In addition to the intensity information of fluorescent molecules, their fluorescence spectra are also one of the important parameters for observing labeled molecules, and some environment-sensitive fluorescent dye molecules can provide more features than morphology, such as molecular structure, biomolecular spatial arrangement and The microenvironment around the molecule, etc. In short, fluorescence spectroscopy can reveal the environmental parameters of cells in another dimension, and is one of the important tools for the study of life activities.
目前常见的光谱分布监测方法为荧光分光光度计，但荧光分光光度计通常不能用于活细胞研究中。针对这一问题，目前还有一些研究将荧光显微成像技术和多光谱成像技术联系起来，即荧光多光谱显微成像方法，其成功实现了细胞膜纳米尺度上的光谱超分辨成像。At present, the common method of monitoring spectral distribution is fluorescence spectrophotometer, but fluorescence spectrophotometer cannot usually be used in the study of living cells. In response to this problem, there are still some studies that combine fluorescence microscopy imaging technology with multispectral imaging technology, that is, fluorescence multispectral microscopy imaging method, which successfully realizes spectral super-resolution imaging at the nanoscale of cell membranes.
然而，由于细胞荧光背景干扰以及系统噪声的影响，导致所获取的荧光分子的光谱信号中也存在噪声，从而根据单个荧光分子的光谱信号难以辨别其反映的环境参数以及变化过程。因此上述的荧光多光谱显微成像方法通常是对荧光分子的光谱信号进行了多次采集，然后进行全局平均的方法来分析环境敏感型染料分子的特征。由于需要多次采集后才能进行分析，因此该方法只能够实现静态地分析细胞的整体性质，却难以应用于动态的生命活动分析，对此还需要提出新的技术方案。However, due to the interference of cellular fluorescence background and the influence of system noise, there is also noise in the acquired spectral signals of fluorescent molecules, so it is difficult to distinguish the environmental parameters and change processes reflected by the spectral signals of a single fluorescent molecule. Therefore, the above-mentioned fluorescent multispectral microscopic imaging method usually collects the spectral signals of the fluorescent molecules multiple times, and then performs a global average method to analyze the characteristics of the environment-sensitive dye molecules. Since multiple acquisitions are required for analysis, this method can only statically analyze the overall properties of cells, but it is difficult to apply to dynamic life activity analysis, and new technical solutions need to be proposed.
发明内容Contents of the invention
本发明主要解决的技术问题是单个荧光分子的光谱信号存在噪声，不能应用于动态的生命活动分析。The technical problem mainly solved by the invention is that there is noise in the spectral signal of a single fluorescent molecule, which cannot be applied to dynamic life activity analysis.
根据第一方面，一种实施例中提供一种光谱曲线降噪生成式模型的训练方法，所述光谱曲线降噪生成式模型用于与判别模型构成生成对抗网络，所述方法包括：According to the first aspect, an embodiment provides a method for training a spectral curve denoising generative model, the spectral curve denoising generative model is used to form a generative confrontation network with a discriminant model, and the method includes:
获取光谱曲线样本集，所述光谱曲线样本集包括多条光谱曲线样本；Obtain a spectral curve sample set, the spectral curve sample set includes a plurality of spectral curve samples;
获取所述光谱曲线样本对应的标签，所述标签为光谱曲线样本进行分类结果的标注，所述分类结果至少包括未降噪和已降噪；Acquiring the label corresponding to the spectral curve sample, the label is the annotation of the classification result of the spectral curve sample, and the classification result includes at least non-noise-reduced and noise-reduced;
将所述标签为未降噪且满足预设条件的所述光谱曲线样本输入所述光谱曲线降噪生成式模型，以生成降噪后的光谱曲线；inputting the spectral curve samples labeled as non-noise-reduced and meeting preset conditions into the spectral curve noise reduction generative model to generate a noise-reduced spectral curve;
获取降噪样本集，所述降噪样本集中的降噪样本包括所述降噪后的光谱曲线和标签为已降噪的所述光谱曲线样本；Obtaining a noise reduction sample set, the noise reduction samples in the noise reduction sample set include the noise-reduced spectral curve and the spectral curve samples labeled as noise-reduced;
将所述降噪样本输入所述判别模型，以得到对所述降噪样本的判断结果，所述判断结果至少包括未降噪和已降噪；Inputting the noise-reduced samples into the discrimination model to obtain a judgment result for the noise-reduced samples, the judgment results at least including non-noise-reduced and noise-reduced;
根据所述判断结果得到生成对抗网络的损失函数，至少根据所述损失函数分别调整所述光谱曲线降噪生成式模型和判别模型的参数，直到所述生成对抗网络收敛，并得到训练完成的光谱曲线降噪生成式模型。According to the judgment result, the loss function of the generative confrontation network is obtained, and at least according to the loss function, the parameters of the spectral curve noise reduction generative model and the discriminant model are respectively adjusted until the generative confrontation network converges, and the trained spectrum is obtained. Generative models for curve denoising.
根据第二方面，一种实施例中提供一种光谱曲线降噪生成式模型的训练装置，所述光谱曲线降噪生成式模型用于与判别模型构成生成对抗网络，所述装置包括：According to the second aspect, an embodiment provides a training device for a spectral curve denoising generative model, the spectral curve denoising generative model is used to form a generative confrontation network with a discriminant model, and the device includes:
样本获取模块，用于获取光谱曲线样本集，所述光谱曲线样本集包括多条光谱曲线样本；A sample acquisition module, configured to acquire a spectral curve sample set, the spectral curve sample set including a plurality of spectral curve samples;
标注模块，用于获取所述光谱曲线样本对应的标签，所述标签为光谱曲线样本进行分类结果的标注，所述分类结果至少包括未降噪和已降噪；A labeling module, configured to obtain a label corresponding to the spectral curve sample, the label is the labeling of the classification result of the spectral curve sample, and the classification result includes at least non-noise-reduced and noise-reduced;
样本降噪模块，用于将所述标签为未降噪且满足预设条件的所述光谱曲线样本输入所述光谱曲线降噪生成式模型，以生成降噪后的光谱曲线；A sample denoising module, configured to input the spectral curve samples labeled as non-noise-reduced and meeting preset conditions into the spectral curve denoising generative model to generate a denoised spectral curve;
判别模块，用于获取降噪样本集，所述降噪样本集中的降噪样本包括所述降噪后的光谱曲线和标签为已降噪的所述光谱曲线样本，将所述降噪样本输入所述判别模型，以得到对所述降噪样本的判断结果，所述判断结果包括未降噪和已降噪；A discrimination module, configured to obtain a noise reduction sample set, wherein the noise reduction samples in the noise reduction sample set include the noise-reduced spectral curve and the spectral curve samples labeled as noise-reduced, and input the noise-reduced samples into The discriminant model is used to obtain a judgment result on the noise-reduced sample, and the judgment result includes non-noise-reduced and noise-reduced;
训练模块，用于根据所述判断结果得到生成对抗网络的损失函数，至少根据所述损失函数分别调整所述光谱曲线降噪生成式模型和判别模型的参数，直到所述生成对抗网络收敛，并得到训练完成的光谱曲线降噪生成式模型。The training module is used to obtain the loss function of the generative confrontation network according to the judgment result, and adjust the parameters of the spectral curve noise reduction generative model and the discriminant model respectively according to the loss function at least until the generative confrontation network converges, and Obtain the trained spectral curve denoising generative model.
根据第三方面，一种实施例中提供一种光谱曲线的降噪方法，包括：According to a third aspect, an embodiment provides a noise reduction method for a spectral curve, including:
获取初始的光谱曲线；Obtain the initial spectral curve;
将所述初始的光谱曲线输入光谱曲线降噪生成式模型，得到所述初始的光谱曲线降噪后的光谱曲线，所述光谱曲线降噪生成式模型由第一方面所述的训练方法训练得到。Inputting the initial spectral curve into the spectral curve denoising generative model to obtain the spectral curve after denoising the initial spectral curve, and the spectral curve denoising generative model is trained by the training method described in the first aspect .
根据第四方面，一种实施例中提供一种光谱曲线的降噪方法，包括：According to a fourth aspect, an embodiment provides a noise reduction method for a spectral curve, including:
获取初始的光谱曲线；Obtain the initial spectral curve;
将所述初始的光谱曲线输入预先训练好的光谱曲线降噪生成式模型，其中所述将所述初始的光谱曲线输入预先训练好的光谱曲线降噪生成式模型，包括：Inputting the initial spectral curve into a pre-trained spectral curve denoising generative model, wherein the inputting the initial spectral curve into a pre-trained spectral curve denoising generative model includes:
通过编码器对所述初始的光谱曲线进行特征提取，得到第一特征；performing feature extraction on the initial spectral curve through an encoder to obtain a first feature;
通过解码器用于对所述第一特征进行升维和信息恢复，以得到与所述初始的光谱曲线维度相同的融合数据；The decoder is used to increase the dimension and restore the information of the first feature, so as to obtain the fusion data with the same dimension as the initial spectral curve;
将所述融合数据与所述初始的光谱曲线进行相加，得到所述初始的光谱曲线降噪后的光谱曲线。The fusion data is added to the initial spectral curve to obtain a spectral curve after denoising the initial spectral curve.
根据第五方面，一种实施例中提供一种计算机可读存储介质，所述介质上存储有程序，所述程序能够被处理器执行以实现如第一方面、第三方面或者第四方面所述的方法。According to the fifth aspect, an embodiment provides a computer-readable storage medium, on which a program is stored, and the program can be executed by a processor to realize the described method.
根据上述实施例的光谱曲线降噪生成式模型的训练方法，将光谱曲线降噪生成式模型与判别模型构成生成对抗网络并进行训练，然后可以基于训练好的光谱曲线降噪生成式模型来对光谱曲线进行降噪，从而降低噪声对单分子光谱信息数据采集的影响。由于无需对荧光分子的光谱信号进行多次采集，因此能够进行分子动态成像的同时获取降噪后的分子光谱，从而可以用于动态监测单个分子在细胞内活动过程中的环境参数变化。According to the training method of the spectral curve denoising generative model of the above-mentioned embodiment, the spectral curve denoising generative model and the discriminant model are formed into a generative confrontation network and trained, and then can be based on the trained spectral curve denoising generative model. Spectral curves are denoised to reduce the impact of noise on data acquisition of single-molecule spectral information. Since there is no need to collect the spectral signals of fluorescent molecules multiple times, the noise-reduced molecular spectra can be acquired while performing molecular dynamic imaging, which can be used to dynamically monitor the changes in environmental parameters of a single molecule during its intracellular activities.
附图说明Description of drawings
图1为一种实施例的光谱曲线降噪生成式模型的结构示意图；Fig. 1 is a schematic structural diagram of a spectral curve denoising generative model of an embodiment;
图2为另一种实施例的光谱曲线降噪生成式模型的结构示意图；Fig. 2 is a structural schematic diagram of a spectral curve denoising generative model of another embodiment;
图3为又一种实施例的光谱曲线降噪生成式模型的结构示意图；Fig. 3 is a structural schematic diagram of a spectral curve denoising generative model of another embodiment;
图4为一种实施例的分类模型的结构示意图；Fig. 4 is a schematic structural diagram of a classification model of an embodiment;
图5为一种实施例的判别模型的结构示意图；Fig. 5 is a schematic structural diagram of a discriminant model of an embodiment;
图6为一种实施例的光谱曲线降噪生成式模型的训练方法的流程示意图；Fig. 6 is a schematic flowchart of a training method of a spectral curve denoising generative model of an embodiment;
图7为另一种实施例的光谱曲线降噪生成式模型的训练方法的流程示意图；Fig. 7 is a schematic flowchart of a training method of a spectral curve denoising generative model of another embodiment;
图8为一种实施例的光谱曲线降噪生成式模型的训练装置的结构示意图；8 is a schematic structural diagram of a training device for a spectral curve denoising generative model of an embodiment;
图9为一种实施例的光谱曲线降噪方法的流程示意图；Fig. 9 is a schematic flow chart of a spectral curve noise reduction method of an embodiment;
图10为一种实施例的进行降噪处理后的光谱曲线。Fig. 10 is a spectral curve after noise reduction processing in an embodiment.
具体实施方式Detailed ways
下面通过具体实施方式结合附图对本发明作进一步详细说明。其中不同实施方式中类似元件采用了相关联的类似的元件标号。在以下的实施方式中，很多细节描述是为了使得本申请能被更好的理解。然而，本领域技术人员可以毫不费力的认识到，其中部分特征在不同情况下是可以省略的，或者可以由其他元件、材料、方法所替代。在某些情况下，本申请相关的一些操作并没有在说明书中显示或者描述，这是为了避免本申请的核心部分被过多的描述所淹没，而对于本领域技术人员而言，详细描述这些相关操作并不是必要的，他们根据说明书中的描述以及本领域的一般技术知识即可完整了解相关操作。The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. Wherein, similar elements in different implementations adopt associated similar element numbers. In the following implementation manners, many details are described for better understanding of the present application. However, those skilled in the art can readily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, and methods. In some cases, some operations related to the application are not shown or described in the description, this is to avoid the core part of the application being overwhelmed by too many descriptions, and for those skilled in the art, it is necessary to describe these operations in detail Relevant operations are not necessary, and they can fully understand the relevant operations according to the description in the specification and general technical knowledge in the field.
另外，说明书中所描述的特点、操作或者特征可以以任意适当的方式结合形成各种实施方式。同时，方法描述中的各步骤或者动作也可以按照本领域技术人员所能显而易见的方式进行顺序调换或调整。因此，说明书和附图中的各种顺序只是为了清楚描述某一个实施例，并不意味着是必须的顺序，除非另有说明其中某个顺序是必须遵循的。In addition, the characteristics, operations or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. At the same time, the steps or actions in the method description can also be exchanged or adjusted in a manner obvious to those skilled in the art. Therefore, the various sequences in the specification and drawings are only for clearly describing a certain embodiment, and do not mean a necessary sequence, unless otherwise stated that a certain sequence must be followed.
本文中为部件所编序号本身，例如“第一”、“第二”等，仅用于区分所描述的对象，不具有任何顺序或技术含义。而本申请所说“连接”、“联接”，如无特别说明，均包括直接和间接连接（联接）。The serial numbers assigned to components in this document, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any sequence or technical meaning. The "connection" and "connection" mentioned in this application include direct and indirect connection (connection) unless otherwise specified.
在本发明实施例中，将光谱曲线降噪生成式模型与判别模型构成生成对抗网络，并基于监督学习方法进行训练，然后基于深度学习的光谱曲线降噪生成式模型来对光谱曲线进行降噪，从而降低噪声对单分子光谱信息数据采集的影响，并用于实现基于单个荧光分子光谱信息的实时动态环境参数监测。由于无需对荧光分子的光谱信号进行多次采集，因此能够进行分子动态成像的同时获取降噪后的分子光谱，相比传统的多光谱成像系统，本发明能够用于动态监测单个分子在细胞内活动过程中的环境参数变化，有望为从分子水平理解细胞的生命活动过程提供新的研究工具。In the embodiment of the present invention, the spectral curve denoising generative model and the discriminant model constitute a generative confrontation network, and are trained based on the supervised learning method, and then the spectral curve denoising is denoised based on the deep learning spectral curve denoising generative model , so as to reduce the influence of noise on the data acquisition of single-molecule spectral information, and be used to realize real-time dynamic environmental parameter monitoring based on single-molecule spectral information. Since there is no need to collect the spectral signals of fluorescent molecules multiple times, molecular dynamic imaging can be performed while obtaining noise-reduced molecular spectra. Compared with traditional multispectral imaging systems, the present invention can be used for dynamic monitoring of single molecules in cells. The change of environmental parameters during the activity process is expected to provide a new research tool for understanding the life activity process of cells from the molecular level.
一些实施例提供一种光谱曲线降噪生成式模型，其用于输入初始的光谱曲线，然后对该初始的光谱曲线进行降噪并输出降噪后的光谱曲线，而降噪后的光谱曲线则可以直接用于分析其中染料分子的特性，从而可以应用于动态的生命活动分析。请参考图1，以下对光谱曲线降噪生成式模型进行具体的说明。Some embodiments provide a spectral curve denoising generative model, which is used to input an initial spectral curve, then denoise the initial spectral curve and output a denoised spectral curve, and the denoised spectral curve is It can be directly used to analyze the characteristics of the dye molecules, so it can be applied to the analysis of dynamic life activities. Please refer to FIG. 1 , the following is a specific description of the spectral curve noise reduction generative model.
光谱曲线降噪生成式模型采用U型网络，该U型网络包括编码器和解码器。编码器用于对初始的光谱曲线进行特征提取，解码器用于对编码器所提取的特征进行特征融合，并得到融合数据，最后将融合数据与初始的光谱曲线进行相加，以得到初始的光谱曲线降噪后的光谱曲线。The spectral curve denoising generative model uses a U-shaped network, which includes an encoder and a decoder. The encoder is used to extract features from the initial spectral curve, and the decoder is used to perform feature fusion on the features extracted by the encoder to obtain fusion data, and finally add the fusion data to the initial spectral curve to obtain the initial spectral curve Spectral curve after noise reduction.
请再参考图1，编码器用于对输入的初始的光谱曲线进行三次特征提取，解码器用于对应的进行三次特征融合。其中，编码器依次通过线性层、特征提取块（FEB ，Featureextraction block）和池化层（Pooling）进行第一次特征提取。请参考图2，其中特征提取块包括两个深度学习层（DL1,DL2），每一个深度学习层均包括一个卷积层（conv）、激活层（Leaky ReLU）和批正则化层（BN ，Batch Normalization）。编码器再分别依次通过特征提取块和池化层进行第二次特征提取和第三次特征提取，最终得到第一特征。解码器则分别依次通过上采样连接模块（UCB ，Upsampling&Conv block）和特征提取块对第一特征进行第一次特征融合和第二次特征融合。请参考图3，其中上采样连接模块包括一个插值模块（Interpolate）和卷积层。解码器再通过上采样连接模块、特征提取块和线性层进行第三次特征融合，并得到最后的融合数据。其中，解码器在三次特征融合中，均先通过上采样连接模块进行上采样，以实现升维，然后通过跳跃连接（Skip connection）获取编码器进行特征提取所得到的特征，并在进行上采样过程中融合该特征，以实现信息恢复，并最终输出融合数据，同时使得融合数据和初始的光谱曲线维度相同。因此将融合数据与初始的光谱曲线进行相加，便可以得到初始的光谱曲线降噪后的光谱曲线，并进行输出。本实施例中，将融合数据与初始的光谱曲线进行相加，可以使得光谱曲线降噪生成式模型学习到系统的噪声，该系统包括用于采集光谱信号的光学装置。Please refer to Figure 1 again. The encoder is used to perform three-time feature extraction on the input initial spectral curve, and the decoder is used to perform corresponding three-time feature fusion. Among them, the encoder performs the first feature extraction through the linear layer, feature extraction block (FEB, Featureextraction block) and pooling layer (Pooling) in sequence. Please refer to Figure 2, where the feature extraction block includes two deep learning layers (DL1, DL2), each of which includes a convolutional layer (conv), an activation layer (Leaky ReLU) and a batch regularization layer (BN, Batch Normalization). The encoder then performs the second feature extraction and the third feature extraction through the feature extraction block and the pooling layer respectively, and finally obtains the first feature. The decoder performs the first feature fusion and the second feature fusion on the first feature through the upsampling connection module (UCB, Upsampling&Conv block) and feature extraction block respectively. Please refer to Figure 3, where the upsampling connection module includes an interpolation module (Interpolate) and a convolutional layer. The decoder performs the third feature fusion through the upsampling connection module, feature extraction block and linear layer, and obtains the final fusion data. Among them, in the three feature fusions, the decoder first performs upsampling through the upsampling connection module to achieve dimensionality enhancement, and then obtains the features obtained by the encoder for feature extraction through the skip connection (Skip connection), and performs upsampling The feature is fused during the process to achieve information recovery, and finally output the fusion data, and at the same time make the fusion data and the original spectral curve dimension the same. Therefore, by adding the fusion data to the initial spectral curve, the spectral curve after denoising the initial spectral curve can be obtained and output. In this embodiment, the fusion data is added to the initial spectral curve, so that the spectral curve noise reduction generative model can learn the noise of the system, and the system includes an optical device for collecting spectral signals.
上述的光谱曲线降噪生成式模型与判别模型构成生成对抗网络后，通过以下训练方法进行训练，以下进行具体的说明。After the above spectral curve denoising generative model and discriminant model constitute a generative confrontation network, it is trained by the following training method, which will be described in detail below.
首先获取光谱曲线样本集，光谱曲线样本集中包括多条光谱曲线样本，例如在荧光分子所处的各种脂质环境下，通过光学装置共采集了6000张图像序列，经过数据筛选后每个脂质环境大约有11000条光谱曲线作为光谱曲线样本，其中10000条光谱曲线样本用于训练，1000条光谱曲线样本用于测试。其中，分别采集了五种不同配比的支撑脂质双层膜的尼罗红（Nile Red）单分子光谱数据，而支撑脂质双层的配比即为获取光谱曲线样本时荧光分子所处的环境类别。在对比其它环境敏感型染料分子时，也可以将不同环境进行编码作为其环境类别。本实施例中，还需要获取各个光谱曲线样本对应的标签，该标签为光谱曲线样本进行分类结果的标注，其中分类结果包括未降噪和已降噪，以及环境类别，即对光谱曲线样本标注是否为已降噪，以及具体的环境类别。而标注方式可以是通过人工进行标注，也可以是自动进行标注，例如通过手动进行标注。Firstly, the spectral curve sample set is obtained. The spectral curve sample set includes multiple spectral curve samples. For example, in various lipid environments where fluorescent molecules are located, a total of 6,000 image sequences were collected through optical devices. After data screening, each lipid There are about 11,000 spectral curves in the qualitative environment as spectral curve samples, of which 10,000 spectral curve samples are used for training, and 1,000 spectral curve samples are used for testing. Among them, five different ratios of Nile Red (Nile Red) single-molecule spectral data of supported lipid bilayer membranes were collected. environment category. When comparing other environment-sensitive dye molecules, different environments can also be encoded as their environment categories. In this embodiment, it is also necessary to obtain the label corresponding to each spectral curve sample, which is the labeling of the classification results of the spectral curve samples, where the classification results include non-noise-reduced and noise-reduced, and the environmental category, that is, label the spectral curve samples Whether it is denoised, and the specific environment category. The labeling method may be manual labeling, or automatic labeling, for example, manual labeling.
对于光谱曲线样本，由于不同的荧光染料其量子产率不同，其最终所获取的光谱曲线样本的信噪比也不同。例如针对量子产率较高的荧光染料，其最终所获取的光谱曲线样本的信噪比较高，此时可以由光谱曲线降噪生成式模型从中恢复出相关的光谱特征，而针对量子产率较低的荧光染料分子，由于在相同的激发光功率条件下，其发射荧光强度较低，单分子荧光受噪声干扰更大，因此光学装置探测到的信号受噪声影响较大，此时其最终所获取的光谱曲线样本的信噪比较低，且无法从中恢复出相关的光谱特征。因此，对于信噪比过于低的光谱曲线样本，则可以认为其属于噪声信息，从而需要排除信噪比过低的光谱曲线样本，以避免对网络训练和测试造成影响，对此可以将不同信噪比的光谱曲线样本进行区别，而其中信噪比不满足条件的光谱曲线样本则不参与后续的网络训练和测试。一些实施例中，可以先利用分类模型对光谱曲线样本的信噪比进行区分，该分类模型包括线性层和残差网络（ResNet）。本实施例中，由于标签为已降噪的光谱曲线样本基本满足预设信噪比，因此主要是对标签为未降噪的光谱曲线样本进行区分。具体的请参考图4，先将未降噪的任一光谱曲线样本输入线性层，而线性层用于对其进行升维，而本实施例中的光谱曲线样本为一维数据，例如将120x1的一维数据升维为4096x1的一维数据，然后再通过reshape函数将升维后的一维数据调整为二维数据，例如将4096x1的调整为64x64x1的，然后将二维数据输入残差网络进行特征提取，以输出是否为噪声的判断结果，例如满足预设信噪比则输出的判断结果为信号（signal），并参与后续的网络训练和测试，反之则输出的判断结果为噪声（noise），从而可以对其进行过滤。For the spectral curve samples, due to the different quantum yields of different fluorescent dyes, the signal-to-noise ratios of the finally acquired spectral curve samples are also different. For example, for fluorescent dyes with high quantum yields, the signal-to-noise ratio of the finally obtained spectral curve samples is high. At this time, the relevant spectral features can be recovered from the spectral curve noise reduction generative model, while for quantum yields For lower fluorescent dye molecules, under the same excitation light power, the emission fluorescence intensity is lower, and the single-molecule fluorescence is more disturbed by noise, so the signal detected by the optical device is greatly affected by noise. At this time, its final The acquired spectral curve samples have a low signal-to-noise ratio and the relevant spectral features cannot be recovered from them. Therefore, for the spectral curve samples with too low signal-to-noise ratio, it can be considered as noise information, so it is necessary to exclude the spectral curve samples with too low signal-to-noise ratio to avoid affecting network training and testing. The spectral curve samples of the signal-to-noise ratio are distinguished, and the spectral curve samples whose signal-to-noise ratio does not meet the conditions do not participate in subsequent network training and testing. In some embodiments, the signal-to-noise ratio of the spectral curve samples may be distinguished first by using a classification model, the classification model including a linear layer and a residual network (ResNet). In this embodiment, since the spectral curve samples whose labels are noise-reduced basically satisfy the preset signal-to-noise ratio, it is mainly to distinguish the spectral curve samples whose labels are not noise-reduced. For details, please refer to Figure 4. First, input any spectral curve sample without noise reduction into the linear layer, and the linear layer is used to increase its dimension. The spectral curve sample in this embodiment is one-dimensional data, for example, the 120x1 The one-dimensional data is upgraded to 4096x1 one-dimensional data, and then the reshape function is used to adjust the upgraded one-dimensional data to two-dimensional data, for example, the 4096x1 is adjusted to 64x64x1, and then the two-dimensional data is input into the residual network Perform feature extraction to output the judgment result of noise. For example, if the preset signal-to-noise ratio is met, the output judgment result is signal (signal), and participate in subsequent network training and testing. Otherwise, the output judgment result is noise (noise ) so that it can be filtered.
然后对于光谱曲线降噪生成式模型，需要将标签为未降噪且满足预设信噪比的光谱曲线样本输入其中，而光谱曲线降噪生成式模型通过编码器进行特征提取，以及解码器进行特征融合后，得到融合数据，最后将融合数据与所输入的光谱曲线样本进行相加，得到对应的降噪后的光谱曲线。对于判别模型，需要先获取降噪样本集，降噪样本集中的降噪样本包括光谱曲线降噪生成式模型生成的降噪后的光谱曲线，以及标签为已降噪的光谱曲线样本。然后将降噪样本输入判别模型，以得到对降噪样本的判断结果，该判断结果包括未降噪和已降噪，其中未降噪和已降噪分别对应真和假，并还可以包括降噪样本的环境类别。本实施例中，由于光谱曲线降噪生成式模型与判别模型构成生成对抗网络，因此当判别模型的判断结果的准确度很高时，则需要调整光谱曲线降噪生成式模型的参数，以对其进行加强。例如，当输入降噪后的光谱曲线时，若判断结果为已降噪（即判断为真）则判断错误，反之（即判断为假）则判断正确，当判断结果的准确度很高时，则说明降噪后的光谱曲线与标签为已降噪的光谱曲线样本之间的差别可能较大，因此判别模型容易得到正确的判断结果，此时需要调整光谱曲线降噪生成式模型的参数，使得降噪后的光谱曲线与标签为已降噪的光谱曲线样本之间的差别减少，从而达到增强光谱曲线降噪生成式模型的目的。而当光谱曲线降噪生成式模型增强后，可能导致判别模型的判断结果的准确度较低，此时说明判别模型已经无法较好的区别降噪后的光谱曲线和标签为已降噪的光谱曲线样本之间的差别，因此需要调整判别模型的参数，从而达到增强判别模型的目的。而通过光谱曲线降噪生成式模型与判别模型之间的对抗，可以不断增强的光谱曲线降噪生成式模型和判别模型，并使得降噪后的光谱曲线和标签为已降噪的光谱曲线样本之间的差别越来越小，从而最终得到训练完成的光谱曲线降噪生成式模型。Then, for the spectral curve denoising generative model, it is necessary to input the spectral curve samples labeled as non-noise-reduced and satisfying the preset signal-to-noise ratio, and the spectral curve denoising generative model uses the encoder for feature extraction, and the decoder for After the features are fused, the fused data is obtained, and finally the fused data is added to the input spectral curve sample to obtain the corresponding noise-reduced spectral curve. For the discriminant model, you need to obtain the denoising sample set first. The denoising samples in the denoising sample set include the denoised spectral curve generated by the spectral curve denoising generative model, and the denoised spectral curve samples labeled as denoised. Then input the noise-reduced samples into the discriminant model to obtain the judgment result of the noise-reduced samples. The judgment results include non-noise-reduced and Environment category for noisy samples. In this embodiment, since the spectral curve denoising generative model and the discriminant model constitute a generative confrontation network, when the accuracy of the judgment result of the discriminant model is high, it is necessary to adjust the parameters of the spectral curve denoising generative model to It is reinforced. For example, when inputting the noise-reduced spectral curve, if the judgment result is noise-reduced (that is, the judgment is true), the judgment is wrong, otherwise (that is, the judgment is false), the judgment is correct. When the accuracy of the judgment result is high, It means that the difference between the denoised spectral curve and the denoised spectral curve sample may be large, so the discriminant model is easy to get the correct judgment result. At this time, it is necessary to adjust the parameters of the spectral curve denoising generative model. The difference between the denoised spectral curve and the denoised spectral curve sample is reduced, thereby achieving the purpose of enhancing the denoising generative model of the spectral curve. However, when the spectral curve denoising generative model is enhanced, the accuracy of the judgment result of the discriminant model may be lower. At this time, it means that the discriminant model has been unable to better distinguish the denoised spectral curve from the denoised spectrum. Therefore, it is necessary to adjust the parameters of the discriminant model to achieve the purpose of enhancing the discriminant model. Through the confrontation between the spectral curve denoising generative model and the discriminant model, the spectral curve denoising generative model and discriminant model can be continuously enhanced, and the denoised spectral curve and label are denoised spectral curve samples The difference between is getting smaller and smaller, so that the trained spectral curve denoising generative model is finally obtained.
一些实施例中，判别模型除了判断降噪样本是否为已降噪，还需要判断其环境类别，从而使得使判别模型可以更加关注染料分子所处环境的信息。具体的，由于荧光分子在不同的环境类别下会存在不同的发射光谱，例如已有研究证明尼罗红分子染料在不同的极性环境下会存在不同的发射光谱，其有序相相较于无序相光谱存在明显的蓝移现象。因此，判别模型可以通过光谱曲线所受到的影响判断其环境类别，从而不仅可以获取荧光分子所处的环境类别，同时还可以提高光谱曲线降噪生成式模型生成降噪后的光谱曲线的准确度。请参考图5，一些实施例中，判别模型包括特征提取层、最大池化层（max pooling）、平均池化层（avg pooling）、第一多层感知机和第二多层感知机。特征提取层用于对所输入的降噪样本进行特征提取，并得到第二特征。具体的，特征提取层包括三层卷积层，每层卷积层均包括一个4x4的二维卷积层、激活层和批正则化层。最大池化层和平均池化层用于分别对第二特征进行降维，以分别得到第三特征和第四特征，第一多层感知机用于输入第三特征，以得到对降噪样本的环境类别的判断结果，第二多层感知机用于输入第四特征，以得到对降噪样本是否为已降噪的判断结果。In some embodiments, in addition to judging whether the noise-reduced sample is noise-reduced, the discriminant model also needs to judge its environment category, so that the discriminant model can pay more attention to the information of the environment where the dye molecule is located. Specifically, since fluorescent molecules have different emission spectra in different environmental categories, for example, studies have proved that Nile red molecular dyes will have different emission spectra in different polar environments, and its ordered phase is compared with There is an obvious blue shift in the spectrum of the disordered phase. Therefore, the discriminant model can judge its environmental category through the influence of the spectral curve, so that it can not only obtain the environmental category of the fluorescent molecule, but also improve the accuracy of the spectral curve noise reduction generative model to generate the noise-reduced spectral curve . Please refer to FIG. 5 , in some embodiments, the discriminant model includes a feature extraction layer, a max pooling layer (max pooling), an average pooling layer (avg pooling), a first multi-layer perceptron and a second multi-layer perceptron. The feature extraction layer is used to perform feature extraction on the input noise reduction samples and obtain second features. Specifically, the feature extraction layer includes three convolutional layers, and each convolutional layer includes a 4x4 two-dimensional convolutional layer, an activation layer, and a batch regularization layer. The maximum pooling layer and the average pooling layer are used to reduce the dimensionality of the second feature to obtain the third feature and the fourth feature respectively, and the first multi-layer perceptron is used to input the third feature to obtain a pair of denoised samples The second multi-layer perceptron is used to input the fourth feature to obtain the judgment result of whether the denoising sample is denoised.
最后根据判别模型的判别结果获取生成对抗网络的整体损失函数，以分别对光谱曲线降噪生成式模型用于和判别模型的参数进行调整，整体损失函数具体如下：Finally, according to the discrimination results of the discriminant model, the overall loss function of the generative adversarial network is obtained to adjust the parameters of the spectral curve denoising generative model and the discriminant model respectively. The overall loss function is as follows:
L=argminGmaxD[lwGAN(G,D)+λ1lL1(G)+λ2lTV(G)+λ3lAux(D)]；L =argminGmaxD [lwGAN (G ,D ) +λ1lL 1 (G ) +λ2lTV (G ) +λ3lAux (D )];
其中，L为整体损失函数，G表示光谱曲线降噪生成式模型，D表示判别模型，argminGmaxD表示对于与G相关的函数取最小值，以及与D相关的函数取最大值，lwGAN表示对抗网络的wGAN损失函数，lL1(G)表示光谱曲线降噪生成式模型的L1正则化损失，lTV(G)表示光谱曲线降噪生成式模型的TV全变分损失，lAux(D)表示判别模型的环境类别分类损失，λ1、λ2和λ3分别表示权重值。其中，通过加入TV全变分损失函数和L1正则化损失，可以使光谱曲线降噪生成式模型输出的光谱曲线更加光滑。Among them,L is the overall loss function,G represents the spectral curve noise reduction generative model,D represents the discriminant model,argminGmaxD represents the minimum value of the function related toG , and the maximum value of the function related toD ,lwGAN Denotesthe wGAN loss function of the adversarial network,lL 1 (G ) representsthe L 1 regularization loss of the spectral curve denoising generative model,lTV (G ) representsthe TV total variation loss of the spectral curve denoising generative model,lAux (D ) represents the environmental category classification loss of the discriminative model, andλ1 ,λ2 andλ3 represent the weight values respectively. Among them, by addingTV full variation loss function andL1 regularization loss, the spectral curve output by the spectral curve denoising generative model can be made smoother.
由上述实施例可知，通过深度学习的光谱曲线降噪生成式模型来对光谱曲线进行降噪，可以用于实现基于单个荧光分子光谱信息的实时动态环境参数监测，有望为从分子水平理解细胞的生命活动过程提供新的研究工具。并且在光谱曲线降噪生成式模型的训练过程中，过滤了信噪比较低的光谱曲线样本，从而保证较好的训练效果，同时光谱曲线降噪生成式模型将融合数据与初始的光谱曲线进行相，使其可以学习到系统的噪声。而判别模型不仅判断光谱曲线样本的真假，还判断光谱曲线样本的环境类别，从而进一步提高光谱曲线降噪生成式模型的性能。It can be seen from the above examples that the spectral curve denoising generative model of deep learning can be used to denoise the spectral curve, which can be used to realize real-time dynamic environmental parameter monitoring based on the spectral information of a single fluorescent molecule, which is expected to provide a basis for understanding cells from the molecular level. Life processes provide new research tools. And in the training process of the spectral curve noise reduction generative model, the spectral curve samples with low signal-to-noise ratio are filtered to ensure a better training effect. At the same time, the spectral curve noise reduction generative model will fuse the data with the original spectral curve phase so that it can learn the noise of the system. The discriminant model not only judges the authenticity of the spectral curve samples, but also judges the environmental category of the spectral curve samples, thereby further improving the performance of the spectral curve noise reduction generative model.
请参考图6，一些实施例提供一种光谱曲线降噪生成式模型的训练方法，应用于上述的生成对抗网络，该训练方法具体包括：Please refer to FIG. 6. Some embodiments provide a training method for a spectral curve denoising generative model, which is applied to the above-mentioned generation confrontation network. The training method specifically includes:
步骤100：获取光谱曲线样本集，所述光谱曲线样本集包括多条光谱曲线样本。Step 100: Obtain a spectral curve sample set, where the spectral curve sample set includes a plurality of spectral curve samples.
步骤200：获取所述光谱曲线样本对应的标签，所述标签为光谱曲线样本进行分类结果的标注，所述分类结果至少包括未降噪和已降噪。Step 200: Obtain a label corresponding to the spectral curve sample, where the label is an annotation of a classification result of the spectral curve sample, and the classification result includes at least non-noise-reduced and noise-reduced.
步骤300：将所述标签为未降噪且满足预设条件的所述光谱曲线样本输入所述光谱曲线降噪生成式模型，以生成降噪后的光谱曲线。Step 300: Input the spectral curve samples labeled as non-noise-reduced and satisfying preset conditions into the spectral curve denoising generative model to generate a denoised spectral curve.
步骤400：获取降噪样本集，所述降噪样本集中的降噪样本包括所述降噪后的光谱曲线和标签为已降噪的所述光谱曲线样本，将所述降噪样本输入所述判别模型，以得到对所述降噪样本的判断结果，所述判断结果至少包括未降噪和已降噪。Step 400: Obtain a denoised sample set, the denoised samples in the denoised sample set include the denoised spectral curve and the denoised spectral curve samples, and input the denoised samples into the A discrimination model is used to obtain a judgment result on the noise-reduced sample, and the judgment result includes at least non-noise-reduced and noise-reduced samples.
步骤500：根据所述判断结果得到生成对抗网络的损失函数，至少根据所述损失函数分别调整所述光谱曲线降噪生成式模型和判别模型的参数，直到所述生成对抗网络收敛，并得到训练完成的光谱曲线降噪生成式模型。Step 500: Obtain the loss function of the generative adversarial network according to the judgment result, and at least adjust the parameters of the spectral curve denoising generative model and discriminant model respectively according to the loss function until the generative adversarial network converges and is trained The completed generative model for spectral curve denoising.
请参考图7，一些实施例中，光谱曲线降噪生成式模型的训练方法还包括：Please refer to FIG. 7. In some embodiments, the training method of the spectral curve denoising generative model also includes:
步骤310：将所述标签为未降噪的任一所述光谱曲线样本输入分类模型中，以判断所述标签为未降噪的任一所述光谱曲线样本是否属于噪声，所述噪声的信噪比低于预设信噪比。Step 310: Input any spectral curve sample whose label is not noise-reduced into the classification model to determine whether any spectral curve sample whose label is not noise-reduced belongs to noise, and the signal of the noise The noise ratio is lower than the preset signal-to-noise ratio.
步骤320：若分类模型判断不属于噪声，则所述标签为未降噪的任一所述光谱曲线样本满足所述预设条件。Step 320: If the classification model judges that it does not belong to noise, then any of the spectral curve samples whose label is not noise-reduced satisfies the preset condition.
一些实施例中，分类模型包括线性层和残差网络，在判断所述标签为未降噪的任一所述光谱曲线样本是否属于噪声时，其具体包括：通过所述线性层对所述标签为未降噪的任一所述光谱曲线样本进行升维，并通过reshape函数将升维后的光谱曲线样本调整为二维数据；将所述二维数据输入残差网络进行特征提取，以输出是否为噪声的判断结果。In some embodiments, the classification model includes a linear layer and a residual network. When judging whether any of the spectral curve samples whose label is not noise-reduced is noise, it specifically includes: using the linear layer to classify the label For any spectral curve sample without noise reduction, the dimension is increased, and the spectral curve sample after the dimension increase is adjusted to two-dimensional data through the reshape function; the two-dimensional data is input into the residual network for feature extraction, and the output Whether it is the judgment result of noise.
一些实施例中，光谱曲线降噪生成式模型采用U型网络，U型网络包括编码器和解码器，在生成降噪后的光谱曲线时，其具体包括；通过编码器对所述标签为未降噪且满足预设条件的所述光谱曲线样本进行特征提取，并得到第一特征；通过解码器用于对所述第一特征进行升维和信息恢复，以得到与所述标签为未降噪且满足预设条件的所述光谱曲线样本维度相同的融合数据；将所述融合数据与所述标签为未降噪且满足预设条件的所述光谱曲线样本进行相加，得到所述降噪后的光谱曲线。In some embodiments, the spectral curve noise reduction generative model adopts a U-shaped network, and the U-shaped network includes an encoder and a decoder. When generating a noise-reduced spectral curve, it specifically includes; Feature extraction is performed on the spectral curve samples that are noise-reduced and meet the preset conditions, and the first feature is obtained; the decoder is used to increase the dimension and restore information on the first feature, so as to obtain the same as the label without noise reduction and Fusion data with the same dimensions of the spectral curve samples meeting the preset conditions; adding the fusion data to the spectral curve samples whose labels are not noise-reduced and satisfying the preset conditions, to obtain the noise-reduced the spectral curve.
一些实施例中，分类结果还包括获取光谱曲线样本时荧光分子所处的环境类别。一些实施例中，判别模型包括特征提取层、最大池化层、平均池化层、第一多层感知机和第二多层感知机，在得到对所述降噪样本的判断结果时，其具体包括：通过所述特征提取层对所述降噪样本进行特征提取，并得到第二特征；分别通过所述最大池化层和所述平均池化层对所述第二特征进行降维，以分别得到第三特征和第四特征；将所述第三特征和第四特征输入第一多层感知机，以得到对所述降噪样本的环境类别的判断结果，将所述第三特征和第四特征输入第二多层感知机，以得到对所述降噪样本是否为已降噪的判断结果。In some embodiments, the classification result also includes the environmental category of the fluorescent molecule when the spectral curve sample is obtained. In some embodiments, the discriminant model includes a feature extraction layer, a maximum pooling layer, an average pooling layer, a first multi-layer perceptron and a second multi-layer perceptron, and when the judgment result of the denoising sample is obtained, its Specifically comprising: performing feature extraction on the noise-reduced sample through the feature extraction layer, and obtaining a second feature; performing dimensionality reduction on the second feature through the maximum pooling layer and the average pooling layer, respectively, To obtain the third feature and the fourth feature respectively; the third feature and the fourth feature are input into the first multi-layer perceptron to obtain the judgment result of the environment category of the noise reduction sample, and the third feature and the fourth feature are input to the second multi-layer perceptron to obtain a judgment result on whether the denoised sample is denoised.
一些实施例中，光谱曲线降噪生成式模型的训练方法还包括：根据所述降噪后的光谱曲线，获取光谱曲线降噪生成式模型的全变分损失和/或正则化损失；还根据所述全变分损失和/或正则化损失调整所述光谱曲线降噪生成式模型的参数。In some embodiments, the training method of the spectral curve denoising generative model further includes: obtaining the full variation loss and/or regularization loss of the spectral curve denoising generative model according to the denoised spectral curve; The total variation loss and/or the regularization loss adjusts parameters of the spectral curve denoising generative model.
请参考图8，一些实施例提供一种光谱曲线降噪生成式模型的训练装置，其包括样本获取模块10、标注模块20、样本降噪模块30、判别模块40和训练模块50，其中光谱曲线降噪生成式模型用于与判别模型构成生成对抗网络，以下具体说明。Please refer to FIG. 8 , some embodiments provide a training device for a spectral curve noise reduction generative model, which includes a sample acquisition module 10, a labeling module 20, a sample noise reduction module 30, a discrimination module 40 and a training module 50, wherein the spectral curve The denoising generative model is used to form a generative adversarial network with the discriminative model, which will be described in detail below.
样本获取模块10用于获取光谱曲线样本集，所述光谱曲线样本集包括多条光谱曲线样本。The sample acquiring module 10 is used to acquire a spectral curve sample set, and the spectral curve sample set includes a plurality of spectral curve samples.
标注模块20用于获取所述光谱曲线样本对应的标签，所述标签为光谱曲线样本进行分类结果的标注，所述分类结果至少包括未降噪和已降噪。The labeling module 20 is configured to acquire a label corresponding to the spectral curve sample, and the label is the labeling of the classification result of the spectral curve sample, and the classification result includes at least non-noise-reduced and noise-reduced.
样本降噪模块30用于将所述标签为未降噪且满足预设条件的所述光谱曲线样本输入所述光谱曲线降噪生成式模型，以生成降噪后的光谱曲线。The sample denoising module 30 is configured to input the spectral curve samples labeled as non-noise-reduced and meeting preset conditions into the spectral curve denoising generative model, so as to generate a denoised spectral curve.
判别模块40用于获取降噪样本集，所述降噪样本集中的降噪样本包括所述降噪后的光谱曲线和标签为已降噪的所述光谱曲线样本，将所述降噪样本输入所述判别模型，以得到对所述降噪样本的判断结果，所述判断结果包括未降噪和已降噪。The discrimination module 40 is used to obtain the noise reduction sample set, the noise reduction samples in the noise reduction sample set include the spectral curve after the noise reduction and the spectral curve sample whose label is denoised, and input the noise reduction sample The discrimination model is used to obtain a judgment result on the noise-reduced sample, and the judgment result includes no noise reduction and noise reduction.
训练模块50用于根据所述判断结果得到生成对抗网络的损失函数，至少根据所述损失函数分别调整所述光谱曲线降噪生成式模型和判别模型的参数，直到所述生成对抗网络收敛，并得到训练完成的光谱曲线降噪生成式模型。The training module 50 is used to obtain the loss function of the generative confrontation network according to the judgment result, at least adjust the parameters of the spectral curve noise reduction generative model and the discriminant model respectively according to the loss function until the generative confrontation network converges, and Obtain the trained spectral curve denoising generative model.
一些实施例中，光谱曲线降噪生成式模型的训练装置还包括过滤模块，过滤模块用于将所述标签为未降噪的任一所述光谱曲线样本输入分类模型中，以判断所述标签为未降噪的任一所述光谱曲线样本是否属于噪声，所述噪声的信噪比低于预设信噪比，若分类模型判断不属于噪声，则所述标签为未降噪的任一所述光谱曲线样本满足所述预设条件。In some embodiments, the training device of the spectral curve noise reduction generative model further includes a filtering module, and the filtering module is used to input any spectral curve sample whose label is not denoised into the classification model to judge the label Whether any of the spectral curve samples without noise reduction belongs to noise, and the signal-to-noise ratio of the noise is lower than the preset signal-to-noise ratio. If the classification model judges that it does not belong to noise, then the label is any sample without noise reduction. The spectral curve sample satisfies the preset condition.
一些实施例中， 训练模块还用于根据所述降噪后的光谱曲线，获取光谱曲线降噪生成式模型的全变分损失和/或正则化损失；还根据所述全变分损失和/或正则化损失调整所述光谱曲线降噪生成式模型的参数。In some embodiments, the training module is also used to obtain the total variation loss and/or regularization loss of the spectral curve denoising generative model according to the denoised spectral curve; and also according to the total variation loss and/or Or a regularization loss to adjust the parameters of the spectral curve denoising generative model.
一些实施例提供一种光谱曲线的降噪方法，其包括以下步骤：Some embodiments provide a noise reduction method for a spectral curve, which includes the following steps:
获取初始的光谱曲线。Get the initial spectral curve.
将所述初始的光谱曲线输入光谱曲线降噪生成式模型，得到所述初始的光谱曲线降噪后的光谱曲线，所述光谱曲线降噪生成式模型由上述训练方法训练得到。Inputting the initial spectral curve into the spectral curve denoising generative model to obtain the spectral curve after denoising the initial spectral curve, and the spectral curve denoising generative model is trained by the above training method.
请参考图9，一些实施例提供一种光谱曲线的降噪方法，其包括以下步骤：Please refer to FIG. 9 , some embodiments provide a noise reduction method for a spectral curve, which includes the following steps:
步骤600：获取初始的光谱曲线。Step 600: Obtain an initial spectral curve.
步骤610：将所述初始的光谱曲线输入预先训练好的光谱曲线降噪生成式模型。Step 610: Input the initial spectral curve into a pre-trained spectral curve denoising generative model.
步骤620：所述将所述初始的光谱曲线输入预先训练好的光谱曲线降噪生成式模型，包括：通过编码器对所述初始的光谱曲线进行特征提取，得到第一特征；通过解码器用于对所述第一特征进行升维和信息恢复，以得到与所述初始的光谱曲线维度相同的融合数据；将所述融合数据与所述初始的光谱曲线进行相加，得到所述初始的光谱曲线降噪后的光谱曲线。Step 620: The inputting the initial spectral curve into the pre-trained spectral curve denoising generative model includes: performing feature extraction on the initial spectral curve through an encoder to obtain the first feature; using a decoder for Carrying out dimension raising and information restoration on the first feature to obtain fusion data having the same dimensions as the initial spectral curve; adding the fusion data to the initial spectral curve to obtain the initial spectral curve Spectral curve after noise reduction.
一些实施例中，所述光谱曲线降噪生成式模型用于与判别模型构成生成对抗网络，所述光谱曲线降噪生成式模型通过以下方式训练：获取光谱曲线样本集，所述光谱曲线样本集包括多条光谱曲线样本；获取所述光谱曲线样本对应的标签，所述标签为光谱曲线样本进行分类结果的标注，所述分类结果至少包括未降噪和已降噪；将所述标签为未降噪且满足预设条件的所述光谱曲线样本输入所述光谱曲线降噪生成式模型，以生成降噪后的光谱曲线；获取降噪样本集，所述降噪样本集中的降噪样本包括所述降噪后的光谱曲线和标签为已降噪的所述光谱曲线样本；将所述降噪样本输入所述判别模型，以得到对所述降噪样本的判断结果，所述判断结果至少包括未降噪和已降噪；根据所述判断结果得到生成对抗网络的损失函数，至少根据所述损失函数分别调整所述光谱曲线降噪生成式模型和判别模型的参数，直到所述生成对抗网络收敛，并得到训练完成的光谱曲线降噪生成式模型。In some embodiments, the spectral curve denoising generative model is used to form a generative confrontation network with the discriminant model, and the spectral curve denoising generative model is trained by the following method: obtaining a spectral curve sample set, the spectral curve sample set Including a plurality of spectral curve samples; obtaining the label corresponding to the spectral curve sample, the label is the annotation of the classification result of the spectral curve sample, the classification result includes at least non-noise reduction and noise reduction; the label is not The spectral curve samples that are denoised and meet the preset conditions are input into the spectral curve denoising generative model to generate a denoised spectral curve; a denoising sample set is obtained, and the denoising samples in the denoising sample set include The noise-reduced spectral curve and label are the noise-reduced spectral curve samples; the noise-reduced samples are input into the discriminant model to obtain a judgment result for the noise-reduced samples, and the judgment result is at least Including non-noise reduction and noise reduction; according to the judgment result, the loss function of the generative confrontation network is obtained, and at least according to the loss function, the parameters of the spectral curve noise reduction generative model and the discriminant model are respectively adjusted until the generative confrontation The network converges, and the trained spectral curve denoising generative model is obtained.
一些实施例中，光谱曲线的降噪方法还包括：将所述标签为未降噪的任一所述光谱曲线样本输入分类模型中，以判断所述标签为未降噪的任一所述光谱曲线样本是否属于噪声，所述噪声的信噪比低于预设信噪比；若分类模型判断不属于噪声，则所述标签为未降噪的任一所述光谱曲线样本满足所述预设条件。In some embodiments, the noise reduction method for spectral curves further includes: inputting any of the spectral curve samples whose label is not denoised into the classification model, so as to judge that the label is any of the spectral curves without denoising Whether the curve sample belongs to noise, and the signal-to-noise ratio of the noise is lower than the preset signal-to-noise ratio; if the classification model judges that it does not belong to noise, then any of the spectral curve samples whose label is not noise-reduced satisfies the preset condition.
一些实施例中，所述分类结果还包括获取光谱曲线样本时荧光分子所处的环境类别，所述判别模型包括特征提取层、最大池化层、平均池化层、第一多层感知机和第二多层感知机，所述得到对所述降噪样本的判断结果包括：通过所述特征提取层对所述降噪样本进行特征提取，并得到第二特征；分别通过所述最大池化层和所述平均池化层对所述第二特征进行降维，以分别得到第三特征和第四特征；将所述第三特征和第四特征输入第一多层感知机，以得到对所述降噪样本的环境类别的判断结果，将所述第三特征和第四特征输入第二多层感知机，以得到对所述降噪样本是否为已降噪的判断结果。In some embodiments, the classification result also includes the environmental category of the fluorescent molecule when the spectral curve sample is obtained, and the discriminant model includes a feature extraction layer, a maximum pooling layer, an average pooling layer, a first multi-layer perceptron and In the second multi-layer perceptron, the obtaining of the judgment result of the noise reduction sample includes: performing feature extraction on the noise reduction sample through the feature extraction layer, and obtaining the second feature; respectively through the maximum pooling Layer and the average pooling layer carry out dimensionality reduction on the second feature to obtain the third feature and the fourth feature respectively; input the third feature and the fourth feature into the first multi-layer perceptron to obtain the pair For the judgment result of the environment category of the noise-reduced sample, input the third feature and the fourth feature into the second multi-layer perceptron to obtain a judgment result of whether the noise-reduced sample is denoised.
一些实施例中，在获取初始的光谱曲线时，其具体包括：获取染料分子的位置图像序列，所述位置图像序列由光学装置进行成像得到。对所述位置图像序列进行预处理，得到染料分子的位置信息。对染料分子的所述位置信息进行到其光谱信息的映射，并得到单分子光谱数据。对所述单分子光谱数据进行数据清洗，以得到所述初始的光谱曲线。In some embodiments, when acquiring the initial spectral curve, it specifically includes: acquiring a sequence of position images of dye molecules, and the sequence of position images is obtained by imaging with an optical device. The position image sequence is preprocessed to obtain the position information of the dye molecules. Mapping the position information of the dye molecule to its spectral information, and obtaining single-molecule spectral data. Data cleaning is performed on the single-molecule spectral data to obtain the initial spectral curve.
本实施例中，在光学装置获取染料分子的位置图像序列后，首先利用开源图像处理软件ImageJ对染料分子的位置图像序列进行预处理，例如通过专为单分子定位显微镜（如光激活定位显微镜和随机光学重建显微镜）数据处理设计的ImageJ模块化插件进行染料分子坐标位置检测，得到位置信息，同时得到其统计学参数，例如像素位置坐标、拟合不确定度和平均光子数等。然后为了实现从染料分子位置信息到光谱信息的映射，需要事先通过标定得到位置-光谱映射矩阵及光谱-像素偏移标定曲线。标定步骤如下：（1）使用上述光学装置对荧光小球进行多光谱成像，并利用611.5/10nm的带通窄带滤光片获取光谱通道中对应611.5nm波段的位置。（2）选取至少6个位置-光谱配对点，并计算其位置-光谱坐标变换矩阵。（3）在光谱通道中，选取某个标定荧光小球的光谱，使用6个不同波段的窄带滤光片获取其对应波长的位置，以611.5nm波长为零点计算每个波段对应的像素偏移距离。（4）通过二次或三次多项式函数拟合出光谱-像素偏移标定曲线。得到位置-光谱映射矩阵及光谱-像素偏移标定曲线后即可以实现从染料分子位置信息到光谱信息的映射，并获取其对应波长下的光谱值。进一步，根据光谱曲线的统计学特征，如变异系数、平均光谱光子数、峰度等对采集得到的单分子光谱数据进行清洗，就可以获取图像中每个染料分子对应的荧光发射光谱信息，并得到一维数据的初始的光谱曲线。In this embodiment, after the image sequence of the position of the dye molecule is acquired by the optical device, the image sequence of the position of the dye molecule is firstly preprocessed by using the open source image processing software ImageJ, for example, through a single-molecule localization microscope (such as a light-activated localization microscope and a Stochastic Optical Reconstruction Microscopy) data processing designed ImageJ modular plug-in to detect the coordinate position of dye molecules, obtain the position information, and obtain its statistical parameters, such as pixel position coordinates, fitting uncertainty and average photon number, etc. Then, in order to realize the mapping from the position information of the dye molecule to the spectral information, it is necessary to obtain the position-spectrum mapping matrix and the spectrum-pixel offset calibration curve through calibration in advance. The calibration steps are as follows: (1) Use the above-mentioned optical device to perform multispectral imaging of the fluorescent beads, and use the 611.5/10nm bandpass narrowband filter to obtain the position corresponding to the 611.5nm band in the spectral channel. (2) Select at least 6 position-spectrum pairing points, and calculate their position-spectral coordinate transformation matrix. (3) In the spectral channel, select the spectrum of a calibrated fluorescent bead, use narrow-band filters of 6 different bands to obtain the position of its corresponding wavelength, and calculate the pixel offset corresponding to each band with the wavelength of 611.5nm as the zero point distance. (4) Fitting a spectrum-pixel offset calibration curve through a quadratic or cubic polynomial function. After obtaining the position-spectrum mapping matrix and the spectrum-pixel offset calibration curve, the mapping from the position information of the dye molecule to the spectral information can be realized, and the spectral value at the corresponding wavelength can be obtained. Further, according to the statistical characteristics of the spectral curve, such as coefficient of variation, average spectral photon number, kurtosis, etc., the collected single-molecule spectral data can be cleaned to obtain the fluorescence emission spectral information corresponding to each dye molecule in the image, and Obtain the initial spectral curve of the 1D data.
以下对上述光谱曲线的降噪方法的效果进行举例说明。The effect of the noise reduction method for the above-mentioned spectral curve will be described with an example below.
初始的光谱曲线是基于尼罗红的光谱映射超分辨成像结果，请参考图10，其中实线avg.GAN表示的是本申请得到的光谱曲线，而虚线avg.raw表示的是现有技术得到的光谱曲线。可以看出，其经过本发明实施例的模型进行数据处理后，整个细胞膜上的脂质序态更加均匀，没有出现突变的现象，更符合细胞的实际情况。同时可以发现传统方法提取的光谱曲线依旧受到噪声影响，难以从中分辨最大发射峰的位置，并且本发明实施例的方法可以大致分辨出当前位置的发射峰大约在620nm左右。进一步验证了本发明实施例的方法相较于传统方法的优越性，提高了传统光学装置的时间分辨率，为单分子水平上的光谱探测提供了强有力的研究工具。且经过本发明实施例的模型处理之后，计算得到的尼罗红最大发射峰位置的方差更小，并在指定在ground truth（真值）中心光谱±8.5nm波长范围内的计算结果为正确值的情况下，本发明的方法能达到99%的最大发射峰识别准确率，相较于传统方法提高了45%，同时在其它不同环境类别中均有不同程度的改善。The initial spectral curve is based on the Nile Red spectral mapping super-resolution imaging results, please refer to Figure 10, where the solid line avg.GAN represents the spectral curve obtained in this application, while the dotted line avg.raw represents the prior art the spectral curve. It can be seen that after data processing by the model of the embodiment of the present invention, the lipid sequence on the entire cell membrane is more uniform, and no mutation occurs, which is more in line with the actual situation of the cell. At the same time, it can be found that the spectral curve extracted by the traditional method is still affected by noise, and it is difficult to distinguish the position of the maximum emission peak, and the method of the embodiment of the present invention can roughly distinguish the emission peak at the current position at about 620nm. It is further verified that the method of the embodiment of the present invention is superior to the traditional method, improves the time resolution of the traditional optical device, and provides a powerful research tool for spectral detection at the single-molecule level. And after the model processing of the embodiment of the present invention, the calculated variance of the maximum emission peak position of Nile Red is smaller, and the calculation result specified in the ground truth (true value) center spectrum ±8.5nm wavelength range is the correct value In the case of , the method of the present invention can reach 99% of the maximum emission peak identification accuracy rate, which is 45% higher than the traditional method, and at the same time, it has different degrees of improvement in other different environmental categories.
一些实施例提供一种计算机可读存储介质所述介质上存储有程序，所述程序能够被处理器执行以实现上述的光谱曲线的降噪方法。Some embodiments provide a computer-readable storage medium, where a program is stored on the medium, and the program can be executed by a processor to implement the above-mentioned spectral curve noise reduction method.
本领域技术人员可以理解，上述实施方式中各种方法的全部或部分功能可以通过硬件的方式实现，也可以通过计算机程序的方式实现。当上述实施方式中全部或部分功能通过计算机程序的方式实现时，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器、随机存储器、磁盘、光盘、硬盘等，通过计算机执行该程序以实现上述功能。例如，将程序存储在设备的存储器中，当通过处理器执行存储器中程序，即可实现上述全部或部分功能。另外，当上述实施方式中全部或部分功能通过计算机程序的方式实现时，该程序也可以存储在服务器、另一计算机、磁盘、光盘、闪存盘或移动硬盘等存储介质中，通过下载或复制保存到本地设备的存储器中，或对本地设备的系统进行版本更新，当通过处理器执行存储器中的程序时，即可实现上述实施方式中全部或部分功能。Those skilled in the art can understand that all or part of the functions of the various methods in the foregoing implementation manners can be realized by means of hardware, or by means of computer programs. When all or part of the functions in the above embodiments are implemented by means of a computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc., through The computer executes the program to realize the above-mentioned functions. For example, the program is stored in the memory of the device, and when the processor executes the program in the memory, all or part of the above-mentioned functions can be realized. In addition, when all or part of the functions in the above embodiments are realized by means of a computer program, the program can also be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a mobile hard disk, and saved by downloading or copying. To the memory of the local device, or to update the version of the system of the local device, when the processor executes the program in the memory, all or part of the functions in the above embodiments can be realized.
以上应用了具体个例对本发明进行阐述，只是用于帮助理解本发明，并不用以限制本发明。对于本发明所属技术领域的技术人员，依据本发明的思想，还可以做出若干简单推演、变形或替换。The above uses specific examples to illustrate the present invention, which is only used to help understand the present invention, and is not intended to limit the present invention. For those skilled in the technical field to which the present invention belongs, some simple deduction, deformation or replacement can also be made according to the idea of the present invention.
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310449238.1ACN116152578B (en) | 2023-04-25 | 2023-04-25 | Training method and device for noise reduction generation model, noise reduction method and medium |
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310449238.1ACN116152578B (en) | 2023-04-25 | 2023-04-25 | Training method and device for noise reduction generation model, noise reduction method and medium |
| Publication Number | Publication Date |
|---|---|
| CN116152578A CN116152578A (en) | 2023-05-23 |
| CN116152578Btrue CN116152578B (en) | 2023-07-18 |
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310449238.1AActiveCN116152578B (en) | 2023-04-25 | 2023-04-25 | Training method and device for noise reduction generation model, noise reduction method and medium |
| Country | Link |
|---|---|
| CN (1) | CN116152578B (en) |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163230A (en)* | 2018-06-15 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of image labeling method and device |
| CN110610156A (en)* | 2019-09-12 | 2019-12-24 | 河海大学 | A Generative Adversarial Network Based Hyperspectral Image Classification Method |
| CN111626317A (en)* | 2019-08-14 | 2020-09-04 | 广东省智能制造研究所 | Semi-supervised hyperspectral data analysis method based on double-flow conditional countermeasure generation network |
| CN113011321A (en)* | 2021-03-17 | 2021-06-22 | 中南大学 | Spectral signal denoising method, system, terminal and readable storage medium based on joint dictionary |
| CN115326783A (en)* | 2022-10-13 | 2022-11-11 | 南方科技大学 | Raman spectrum preprocessing model generation method, system, terminal and storage medium |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021253316A1 (en)* | 2020-06-18 | 2021-12-23 | 深圳先进技术研究院 | Method and apparatus for training image noise reduction model, electronic device, and storage medium |
| KR102238300B1 (en)* | 2020-09-24 | 2021-04-12 | 국방과학연구소 | Method and apparatus for producing infrared spectrum |
| CN113239782B (en)* | 2021-05-11 | 2023-04-28 | 广西科学院 | Pedestrian re-recognition system and method integrating multi-scale GAN and tag learning |
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163230A (en)* | 2018-06-15 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of image labeling method and device |
| CN111626317A (en)* | 2019-08-14 | 2020-09-04 | 广东省智能制造研究所 | Semi-supervised hyperspectral data analysis method based on double-flow conditional countermeasure generation network |
| CN110610156A (en)* | 2019-09-12 | 2019-12-24 | 河海大学 | A Generative Adversarial Network Based Hyperspectral Image Classification Method |
| CN113011321A (en)* | 2021-03-17 | 2021-06-22 | 中南大学 | Spectral signal denoising method, system, terminal and readable storage medium based on joint dictionary |
| CN115326783A (en)* | 2022-10-13 | 2022-11-11 | 南方科技大学 | Raman spectrum preprocessing model generation method, system, terminal and storage medium |
| Publication number | Publication date |
|---|---|
| CN116152578A (en) | 2023-05-23 |
| Publication | Publication Date | Title |
|---|---|---|
| CN110458249B (en) | Focus classification system based on deep learning and probabilistic imaging omics | |
| US10671833B2 (en) | Analyzing digital holographic microscopy data for hematology applications | |
| CN111915596A (en) | A method and device for predicting benign and malignant pulmonary nodules | |
| Rahmouni et al. | Distinguishing computer graphics from natural images using convolution neural networks | |
| Panigrahi et al. | Capsule network based analysis of histopathological images of oral squamous cell carcinoma | |
| Smal et al. | Quantitative comparison of spot detection methods in fluorescence microscopy | |
| US8600143B1 (en) | Method and system for hierarchical tissue analysis and classification | |
| JP2022500744A (en) | Computer implementation methods, computer program products and systems for analysis of cell images | |
| CN109300121A (en) | Method and system for constructing a diagnostic model of cardiovascular disease and the diagnostic model | |
| CN112634159B (en) | Hyperspectral image denoising method based on blind noise estimation | |
| CN112274110B (en) | Pore detection system, device and method based on skin fluorescence image | |
| Sathya et al. | RDA-CNN: Enhanced Super Resolution Method for Rice Plant Disease Classification. | |
| Pandey et al. | Skin cancer classification using non-local means denoising and sparse dictionary learning based CNN | |
| Venkateswara et al. | Deep learning based agricultural pest monitoring and classification | |
| Biradar et al. | Segmentation and detection of crop pests using novel U‐Net with hybrid deep learning mechanism | |
| Puviarasi | Accuracy improvement in disease identification of mango leaf using CNN algorithm compared with fuzzy algorithm | |
| CN116612472A (en) | Image-based single-molecule immune array analyzer and its method | |
| CN116091492A (en) | Image change pixel level detection method and system | |
| CN116386090A (en) | Plankton identification method, system and medium based on scanning atlas | |
| Mao et al. | Deep learning in fluorescence imaging and analysis | |
| CN116152578B (en) | Training method and device for noise reduction generation model, noise reduction method and medium | |
| CN117953970B (en) | A lung cancer multi-gene detection method and system based on hyperspectral imaging | |
| CN119229189A (en) | A method, system and medium for detecting noise labels in hyperspectral images | |
| CN113723453A (en) | Pollen image classification method and device | |
| CN113807438A (en) | Adversarial Hybrid Semi-Supervised Hyperspectral Image Classification Based on Semantic Preservation |
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |